Skip to content

fix(arborist): omit failed optional dependencies from installed deps #8184

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Apr 3, 2025

Conversation

owlstronaut
Copy link
Contributor

@owlstronaut owlstronaut commented Mar 26, 2025

… but keep them 'in the tree'

This PR was authored by @zkat

Fixes: #4828
Fixes: #7961
Replaces: #8127
Replaces: #8077

When optional dependencies fail, we don't want to install them, for whatever reason. The way this "uninstallation" is done right now is by simply removing the dependencies from the ideal tree during reification.

Unfortunately, this means that what gets saved to the "hidden lockfile" is the edited ideal tree as well, and when npm runs next, it'll use that when regenerating the "real" package-lock.

This PR tags failed optional deps such that they're still added to the "trash list" (and thus have their directories cleaned up by the reifier), but prevents Arborist from removing them altogether from the ideal tree (which is usually done by setting their parent to null, making them unreachable, and letting them get GC'd).

PS: It's Friday, this is what I managed to get done together. I'm making this a draft PR for now so folks can look at it, but I want to make sure both reifiers work well, fix some messaging issues, and clean stuff up (I'm pretty sure I'm guarding ideallyInert in more places than I need to, because I was trying to find the right spot). Still, feel free to talk about the approach. I'll get back to this on Monday.

PPS: also hi

@owlstronaut owlstronaut changed the title fix(arborist): omit failed optional dependencies from installed deps, but keep them 'in the tree' fix(arborist): omit failed optional dependencies from installed deps Mar 27, 2025
@owlstronaut
Copy link
Contributor Author

@zkat I did what I could here. Had a couple of comments/lack of my own clarity in things I changed - see above.

@owlstronaut owlstronaut marked this pull request as ready for review March 31, 2025 14:45
@owlstronaut owlstronaut requested a review from a team as a code owner March 31, 2025 14:45
@wraithgar
Copy link
Member

Other than the comment we really really should re-use the exceptional description from #8177, either by copying it into this issue now or when we merge. Probably easier to do now. I'll do it now.

@@ -1527,7 +1528,7 @@ This is a one-time fix-up, please be patient...

const set = optionalSet(node)
for (const node of set) {
node.parent = null
node.ideallyInert = true
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've been thinking about this, and maybe it's better to just call it inert instead of ideallyInert. I think putting the ideally in there is probably a bit too much clever inside baseball and it's more verbose to boot. :)

Some other name would be good, too, as long as it communicates that this is in the tree, but we don't physically install it.

@paulrutter
Copy link

Thanks for pulling this work forward, much appreciated @zkat @owlstronaut @wraithgar.
Happy to see this is almost ready to get merged.

… but keep them 'in the tree'

Fixes: #4828
Fixes: #7961
Replaces: #8127
Replaces: #8077

When optional dependencies fail, we don't want to install them, for whatever reason. The way
this "uninstallation" is done right now is by simply removing the dependencies from the ideal tree
during reification.

Unfortunately, this means that what gets saved to the "hidden lockfile" is the edited ideal tree
as well, and when NPM runs next, it'll use that when regenerating the "real" package-lock.

This PR tags failed optional deps such that they're still added to the "trash list" (and thus have
their directories cleaned up by the reifier), but prevents Arborist from removing them altogether
from the ideal tree (which is usually done by setting their parent to `null`, making them unreachable,
and letting them get GC'd).
@owlstronaut owlstronaut force-pushed the owlstronaut/optionals-in-tree branch from f635c97 to 68a8d05 Compare April 1, 2025 21:17
@owlstronaut owlstronaut merged commit a96d8f6 into latest Apr 3, 2025
16 checks passed
@owlstronaut owlstronaut deleted the owlstronaut/optionals-in-tree branch April 3, 2025 15:45
@github-actions github-actions bot mentioned this pull request Apr 3, 2025
@jakebailey
Copy link

Amazing, I'm so excited for this to be fixed. Thank you @zkat @owlstronaut @wraithgar 🎉

@JounQin
Copy link

JounQin commented Apr 20, 2025

@jakebailey Can this be back ported into old versions?

@wraithgar
Copy link
Member

This is not going to be backported.

Andrew-Chen-Wang added a commit to Andrew-Chen-Wang/template-nextjs that referenced this pull request Apr 22, 2025
npm version 11.3.0 includes a fix (see: npm/cli#8184) that was pointed out in tailwindlabs/tailwindcss#15806 (comment) where it should resolve properly. See this thread for my own investigation parcel-bundler/lightningcss#956 (comment)
Dawnmarie-2578 added a commit to Dawnmarie-2578/cli that referenced this pull request May 17, 2025
Dawnmarie-2578 added a commit to Dawnmarie-2578/cli that referenced this pull request May 17, 2025
Dawnmarie-2578 added a commit to Dawnmarie-2578/cli that referenced this pull request May 17, 2025
Dawnmarie-2578 added a commit to Dawnmarie-2578/cli that referenced this pull request May 17, 2025
@domino14
Copy link

domino14 commented Jul 28, 2025

I still get this bug btw, despite updating node, npm, AND deleting my package-lock.json and node_modules and reinstalling.

Error: Cannot find module @rollup/rollup-linux-arm64-musl. npm has a bug related to optional dependencies (https://github.com/npm/cli/issues/4828). Please try `npm i` again after removing both package-lock.json and node_modules directory.

I even added this to package.json:

    "optionalDependencies": {
        "@rollup/rollup-linux-arm64-musl": "^4.46.0",
        "@rollup/rollup-linux-x64-gnu": "^4.46.0"
    },

The only way I got it to work is by doing this:

docker compose run --rm my_service_name
 sh -c "rm -rf package-lock.json node_modules && npm install"

which is dumb because i have to get all our devs to do it and all the other issues with deleting lockfiles and reinstalling everything. help!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
7 participants