Skip to content

Conversation

fugidev
Copy link
Member

@fugidev fugidev commented Jun 15, 2025

Motivation: Upstream doesn't provide binaries for aarch64-linux.

Part of #296939

I took inspiration from the respective packages on AUR and Flathub, which are also source builds.

Unfortunately, building this package is currently broken on Darwin with sandbox = true due to #415328 and #416077 (merged). Otherwise it does build and work. But because of this, I'd keep the bin package for now and automatically fall back to it on Darwin, to not cause any issues on existing user setups. (Any opinions on this are very welcome.)

On Linux, the included "Backup" plugin doesn't load for some reason. The AUR package has the same issue and it has been reported, see JackGruber/joplin-plugin-backup#92. It works on the flatpak tho, and on this package's Darwin build as well.
Edit: fixed

More usage testing is greatly appreciated, as well as any other reviews and comments. :)

@yajo you seem to be the only active maintainer of the current package, do you want to be added as maintainer here?

Things done

  • Built on platform(s)
    • x86_64-linux
    • aarch64-linux
    • x86_64-darwin
    • aarch64-darwin
  • For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)
    • sandbox = relaxed
    • sandbox = true
  • Tested, as applicable:
  • Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
  • Tested basic functionality of all binary files (usually in ./result/bin/)
  • Nixpkgs 25.11 Release Notes (or backporting 24.11 and 25.05 Nixpkgs Release notes)
    • (Package updates) Added a release notes entry if the change is major or breaking
  • NixOS 25.11 Release Notes (or backporting 24.11 and 25.05 NixOS Release notes)
    • (Module updates) Added a release notes entry if the change is significant
    • (Module addition) Added a release notes entry if adding a new NixOS module
  • Fits CONTRIBUTING.md, pkgs/README.md, maintainers/README.md and other contributing documentation in corresponding paths.

Add a 👍 reaction to pull requests you find important.

@fugidev fugidev force-pushed the joplin-desktop branch 2 times, most recently from d4b3a30 to a160497 Compare June 16, 2025 14:11
@fugidev
Copy link
Member Author

fugidev commented Jun 17, 2025

I reworked the plugin building to remove IFDs

@nix-owners nix-owners bot requested review from HugoReeves and qjoly June 17, 2025 12:56
@github-actions github-actions bot added 10.rebuild-darwin: 1-10 This PR causes between 1 and 10 packages to rebuild on Darwin. 10.rebuild-linux: 1-10 This PR causes between 1 and 10 packages to rebuild on Linux. 8.has: documentation This PR adds or changes documentation labels Jun 17, 2025
@wegank wegank added the 2.status: merge conflict This PR has merge conflicts with the target branch label Jun 18, 2025
src = fetchFromGitHub {
owner = "laurent22";
repo = "joplin";
rev = "refs/tags/v${finalAttrs.version}";
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is simpler:

Suggested change
rev = "refs/tags/v${finalAttrs.version}";
tag = "v${finalAttrs.version}";

Copy link
Member Author

@fugidev fugidev Jun 20, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is, but it causes an error in nix-prefetch in the update script for some reason:

fetchFromGitHub requires one of either `rev` or `tag` to be provided (not both)

As a workaround rev = null can be added, or extending the nix-prefetch flags with --rev --expr null.

I thought the way I did it would be the simplest, but if using tag is much preferred then I can of course add one of these workarounds.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did you make sure to add tag and remove rev? If so, nix-prefetch shouldn't fail.

Otherwise you can leave it as it is. It was just a suggestion based on code review, but if it is problematic then don't worry :)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, that's what I did. Must be a bug in nix-prefetch

@ofborg ofborg bot removed the 2.status: merge conflict This PR has merge conflicts with the target branch label Jun 20, 2025
@github-actions github-actions bot added 10.rebuild-darwin: 1 This PR causes 1 package to rebuild on Darwin. 10.rebuild-linux: 1 This PR causes 1 package to rebuild on Linux. labels Jun 20, 2025
@nixpkgs-ci nixpkgs-ci bot added the 2.status: merge conflict This PR has merge conflicts with the target branch label Jun 25, 2025
@ofborg ofborg bot removed the 2.status: merge conflict This PR has merge conflicts with the target branch label Jun 26, 2025
@fugidev
Copy link
Member Author

fugidev commented Jun 26, 2025

rebased and updated to 3.3.13

@fugidev
Copy link
Member Author

fugidev commented Jun 30, 2025

I refactored buildPlugin.nix as suggested by @yajo.

@yajo
Copy link
Contributor

yajo commented Jul 1, 2025

I tested the app but got this:

image

Clicking restart in safe mode an launching again didn't help.

I'm not sure if this is actually a failure from the new build or from some configuration with local plugins...

@fugidev
Copy link
Member Author

fugidev commented Jul 1, 2025

I'm not sure if this is actually a failure from the new build or from some configuration with local plugins...

Seems like you are running joplin-desktop 2.14.17 there for some reason, not this one

Copy link
Contributor

@yajo yajo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah! I tested the wrong PR. Now it worked!

About dialog provided this info FWIW

Joplin 3.3.13 (prod, linux)

Device: linux, 13th Gen Intel(R) Core(TM) i7-1360P
ID del Cliente: e7cf469d82bd47648320bf15c5040be8
Versión de la Sincronización: 3
Versión del Perfil: 47
Llavero Soportado: No
Alternative instance ID: -

Automatic Backlinks to note: 3.0.3
Freehand Drawing: 3.0.1
Link Graph UI: 1.5.0
Note Tabs: 1.4.0
Outline: 1.5.13
Quick Links: 1.3.2
Rich Markdown: 0.15.0
Toggle Editor Menu & Keyboard Shortcut: 1.0.1

@nixpkgs-ci nixpkgs-ci bot added 12.approvals: 1 This PR was reviewed and approved by one person. 12.approved-by: package-maintainer This PR was reviewed and approved by a maintainer listed in any of the changed packages. labels Jul 2, 2025
@fugidev
Copy link
Member Author

fugidev commented Aug 12, 2025

rebased to resolve conflict and formatted again. successfully built and run on x86_64-linux and aarch64-darwin.

@r-vdp
Copy link
Contributor

r-vdp commented Aug 15, 2025

I'm not very familiar with yarn packaging, but is there no way to avoid vendoring that huge lock file?
Is there a reason that we can't use the upstream one with the offline cache stuff that I see other packages using?

@r-vdp
Copy link
Contributor

r-vdp commented Aug 15, 2025

Hmm, I see now that there is a comment about the vendored lockfile. How would we keep this lockfile up to date? And there is no way to do the needed changes at build time during the patch phase without vendoring the lock file?

A quick search in the repo suggests that we only have a handful of vendored yarn.lock files, so most other packages seem to be able to avoid vendoring lock files.

@fugidev
Copy link
Member Author

fugidev commented Aug 16, 2025

@r-vdp thank you for taking a look at this.

The yarn.lock is generated and kept up to date by the update script here.

My reasoning for vendoring it was:

  • Using the upstream lockfile without deleting unneeded major subpackages causes yarn to install and build additional dependencies that aren't needed for the desktop app. Some of these cause errors (e.g. wasm-pack, required only by the onenote-converter subpackage) that would require additional workarounds to fix. (This causes an increase in size of the offlineCache by ~200MB or 36% btw.)
  • Using the upstream lockfile and pruning it in postFetch or in another FOD might be an option (and I would prefer that too), but I unfortunately don't know if yarn install --mode=update-lockfile is deterministic. I'm assuming it's probably not.
  • Edit: deleting the subpackages we don't need and using the upstream lockfile without pruning it causes yarn to error during the build because it wants to update it

@r-vdp
Copy link
Contributor

r-vdp commented Aug 16, 2025

@yuyuyureka do you have an opinion on the lock file situation here? Is there a way to handle this without vendoring?
(It was suggested to me on matrix to check with you)

@nixpkgs-ci nixpkgs-ci bot added the 2.status: merge conflict This PR has merge conflicts with the target branch label Aug 28, 2025
@nixpkgs-ci nixpkgs-ci bot removed the 2.status: merge conflict This PR has merge conflicts with the target branch label Aug 29, 2025
@fugidev
Copy link
Member Author

fugidev commented Aug 29, 2025

Rebased and updated to v3.4.6 (Note: that's actually a pre-release, but was updated to in #436033).

This required switching to yarn-berry_4 and some other minor changes.

Build tested on aarch64-linux and aarch64-darwin.

@r-vdp if the size of the vendored yarn.lock is an issue, we could instead store and apply the diff? That would be ~500KB, down from 1.6MB. Or we could try pruning it in the FOD and see if it breaks down the road.

@r-vdp
Copy link
Contributor

r-vdp commented Aug 29, 2025

Yeah, the only thing that's keeping me from merging this, is indeed the size of the vendored lock file.

Both of those options sound like they would improve the situation, but I don't know yarn enough to make the call on which approach would be better here.

@fugidev
Copy link
Member Author

fugidev commented Aug 29, 2025

I changed it to store only the diff 👍
This way, I'm confident that it won't break ^^

@yuyuyureka
Copy link
Contributor

fetchYarnBerryDeps should only download the tgz files and never try to compile anything or run build scripts of the dependencies. Your code mentions something about "would require unneccessary complexity to fix". Does that mean that with the original lockfile is broken with fetchYarnBerryDeps?
Otherwise, I wouldn't see the problem with fetching the dependencies for the entire workspace, and then only building a subset of the workspace.

@fugidev
Copy link
Member Author

fugidev commented Aug 30, 2025

@yuyuyureka the issue is not with fetchYarnBerryDeps, that works just fine (I see where the confusion may come from; I only meant to specify the offlineCache's size difference for a rough idea on what portion of the deps is unnecessary).

yarn install is the step where some deps are apparently built from source, like sqlite (required for the desktop app) but also e.g. wasm-pack (which is not). And that would, afaict, require adding additional buildInputs (or otherwise workarounds) in order to not fail. Unless I've missed something ^^'

Edit: I'm taking another look at yarn workspaces focus. I couldn't get that to work before but maybe I've missed something (or it could be different with yarn 4).

@fugidev
Copy link
Member Author

fugidev commented Aug 30, 2025

I got it working! No vendoring a modified lockfile needed anymore. I'm much happier with this.

For some reason I'm getting a hash mismatch for src on darwin now, that is very weird...

Edit: hash mismatch is resolved

@fugidev
Copy link
Member Author

fugidev commented Aug 31, 2025

nixpkgs-review result

Generated using nixpkgs-review-gha

Command: nixpkgs-review pr 417048
Commit: 84cecfdf3cff39a731636cb201f2a167a5c223c4 (subsequent changes)
Merge: 39a7c5b823bdf782fc6edd7dae084eaadc324a9e

Logs: https://github.com/fugidev/nixpkgs-review-gha/actions/runs/17356630392


x86_64-linux

✅ 1 package built:
  • joplin-desktop

aarch64-linux

✅ 1 package built:
  • joplin-desktop

x86_64-darwin (sandbox = relaxed)

✅ 1 package built:
  • joplin-desktop

aarch64-darwin (sandbox = relaxed)

✅ 1 package built:
  • joplin-desktop

@fugidev fugidev requested review from r-vdp and yuyuyureka August 31, 2025 12:12
src = fetchFromGitHub {
owner = "laurent22";
repo = "joplin";
rev = "refs/tags/v${finalAttrs.version}";
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
rev = "refs/tags/v${finalAttrs.version}";
tag = "v${finalAttrs.version}";

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See #417048 (comment)
What would be preferred in your opinion?

Copy link
Contributor

@r-vdp r-vdp Aug 31, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, I didn't see that comment, sorry. The tag attribute is preferred because it avoids git getting confused when the repo contains a commit whose commit hash has the tag as a prefix, or a branch with the same name. And the refs/tags/ is also rather verbose.
So I would opt to add rev = null with a comment to explain why it is needed for now.

Copy link
Member Author

@fugidev fugidev Aug 31, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

rev = null in the derivation didn't work for some reason (it did back then). So I've added the workaround in the update script. Seems more fitting anyway when I think about it, since it's only relevant there.

Copy link
Contributor

@r-vdp r-vdp left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for all your effort on this!

@r-vdp r-vdp merged commit bc6b16b into NixOS:master Aug 31, 2025
29 of 31 checks passed
@fugidev
Copy link
Member Author

fugidev commented Aug 31, 2025

Thank you and everyone else for reviewing and helping get this over the finish line! :)

@fugidev fugidev deleted the joplin-desktop branch September 1, 2025 08:13
@HHR2020
Copy link
Contributor

HHR2020 commented Sep 5, 2025

This pr makes my joplin scale 4x when NIXOS_OZONE_WL is set.

picture

@fugidev
Copy link
Member Author

fugidev commented Sep 5, 2025

@HHR2020 it scales correctly and just the same with or without NIXOS_OZONE_WL set for me (1.5x scale on sway). You should create an issue, maybe others have the same problem?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
8.has: documentation This PR adds or changes documentation 10.rebuild-darwin: 1-10 This PR causes between 1 and 10 packages to rebuild on Darwin. 10.rebuild-darwin: 1 This PR causes 1 package to rebuild on Darwin. 10.rebuild-linux: 1-10 This PR causes between 1 and 10 packages to rebuild on Linux. 10.rebuild-linux: 1 This PR causes 1 package to rebuild on Linux. 12.approvals: 1 This PR was reviewed and approved by one person. 12.approved-by: package-maintainer This PR was reviewed and approved by a maintainer listed in any of the changed packages.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

10 participants