Skip to content

Conversation

serhalp
Copy link
Member

@serhalp serhalp commented Aug 8, 2025

Summary

We were previously only testing against Node.js 18 in CI, even though we've set engines.node to >=18.0.0.

Gatsby appears to just work with Node.js 20 and 22, with a couple caveats noted below. Most of this PR is adjusting our own tests and test infra to work with these Node.js versions.

Test Infrastructure Changes

  • Run unit tests on Node 18, latest 20, and latest 22
  • Run integration tests on Node 18 (earliest we support) and latest 22
  • Separate CircleCI npm cache by Node.js version to avoid native module compilation issues
  • Misc. updates to to handle Ubuntu Noble Numbat-based CircleCI node 20 and 22 images (e.g. no included python); the node 18 image used Focal
  • Note: I decided not to bother testing Node.js 20 and 22 on Windows, at least in this PR
  • Note: I decided not to bother running the e2e tests against Node.js 20 and 22, since "e2e" in this repo means running in a real browser (as opposed to jsdom in integration tests), which is unlikely to be relevant to Node.js versions

Test Changes

  • Fix snapshot tests that varied between Node versions due to irrelevant HTTP header differences (connection defaults changed or something)
  • Conditionally skip test that relied on unstable Node.js 18 garbage collection implementation details that no longer work as expected in Node.js 20+
  • Fix snapshot tests that included full stack trace down to the Node.js layer (these vary a bit across Node.js versions)
  • Adjust a test assertion to work consistently across Node versions in both local dev and in CI
  • Fix module import inconsistencies in test files to match import in tested source (stopped working in node.js 20)
  • Fix sharp vs. lmdb compilation conflict in the lmdb-regeneration integration test fixture (😰 this was a doozy to figure out! see commit for more)

Node.js 22 Compatibility Fixes

  • Bump webpack-virtual-modules to fix Cannot read properties of null (reading 'fileWatchers') error
    (fix). I don't know how this is related to Node.js 22, but it appears to be.

⚠️ Known Issues ⚠️

gatsby develop fails with Node.js 22.7.0, 22.8.0, and 22.9.0

There is a critical bug in Node.js (nodejs/node#55145?) affecting versions 22.7.0, 22.8.0, and 22.9.0 that causes gatsby develop to fail with the error reported in #39068.

Versions before 22.7.0 and version 22.10.0 and later work fine, even before this PR.

Page loads may hang in dev with Node.js ≥20.19.0 or ≥22.14.0 and experimental DEV_SSR enabled

A change landed in Node.js 20.19.0 and 22.14.0 (and presumably 24) results in requests to the gatsby develop dev server to occasionally hang for 15 seconds. This can only occur if you have opted in to the experimental DEV_SSR flag.

To avoid this, downgrade to Node.js 20.18.3 or 22.13.1 or disable DEV_SSR.

To Do

  • Update GitHub required checks before merging

@gatsbot gatsbot bot added the status: triage needed Issue or pull request that need to be triaged and assigned to a reviewer label Aug 8, 2025
@serhalp serhalp changed the title chore(ci): test against Node 18, 20, 22 feat: add support for Node.js 22 Aug 13, 2025
@@ -83,15 +83,19 @@ describe(`report.error`, () => {
const generatedError = getErrorMessages(
reporterActions.createLog as jest.Mock
)[0]
expect(generatedError).toMatchSnapshot()
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

these snapshots contained full stack traces down to the node.js level which weren't stable across node.js versions. it didn't seem like that was material to the tests, so we just simplified this.

@@ -8,7 +8,7 @@ import _ from "lodash"
import { resolve } from "path"
import { setFieldsOnGraphQLNodeType } from "../extend-node-type"
import { generateImageSource } from "../gatsby-plugin-image"
import * as gatsbyCoreUtils from "gatsby-core-utils"
import * as fetchRemoteFileModule from "gatsby-core-utils/fetch-remote-file"
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

matching how it's imported in the source module we're testing. this mismatch stopped working in node 20.

@serhalp serhalp force-pushed the codex/extend-node.js-engines-to-include-22 branch from df870f9 to 9624f12 Compare August 13, 2025 23:28
@serhalp serhalp changed the title feat: add support for Node.js 22 chore: test against node.js 18, 20, and 22 Aug 13, 2025
@serhalp serhalp force-pushed the codex/extend-node.js-engines-to-include-22 branch from 9624f12 to ad01dde Compare August 13, 2025 23:43
@serhalp serhalp changed the title chore: test against node.js 18, 20, and 22 fix: support node 22 Aug 14, 2025
@serhalp serhalp linked an issue Aug 14, 2025 that may be closed by this pull request
2 tasks
@serhalp serhalp force-pushed the codex/extend-node.js-engines-to-include-22 branch 3 times, most recently from bab8e48 to 5db7975 Compare August 14, 2025 19:46
serhalp and others added 8 commits August 15, 2025 18:55
This mismatch stopped working after node 18.
Some of the connection header defaults depend on the node.js major version

wip this is a fix for the headers one
The `lmdb-regeneration` integration test fixture had an `.npmrc` file forcing all packages to be
built from source, which caused sharp to fail compilation in Node 20+ CircleCI images (Ubuntu Noble)
due to... who knows what, honestly.

Since this fixture specifically needs lmdb built from source but not sharp (it's testing something
specific to this), this replaces the global `.npmrc` `build-from-source` flag with a targeted
`postinstall` script that only rebuilds lmdb from source, allowing sharp to use prebuilt binaries.

This resolves the "cannot find -l:libvips-cpp.so.42" linker errors that were blocking integration
tests on Node 20+ while preserving the fixture's intended lmdb compilation testing.

AFAICT this was... basically the only solution. Whew
to get this bug fix: sysgears/webpack-virtual-modules#172

causing this, unclear when:
```
TypeError: Cannot read properties of null (reading 'fileWatchers')
```
- run unit tests against all three
- run integration tests against earliest and latest
- make sure to separate npm cache by node version
@serhalp serhalp force-pushed the codex/extend-node.js-engines-to-include-22 branch from 0915f8a to 5c86fe5 Compare August 15, 2025 23:21
@@ -28,12 +32,6 @@ aliases:
environment:
<<: *e2e-executor-env

restore_cache: &restore_cache
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

these two were CircleCI "aliases" which can't accept parameters (node version), so I changed them to CircleCI "commands"

- bootstrap
- bootstrap-18.2.0

integration-test-workflow: &integration-test-workflow
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we were using e2e-test-workflow for both e2e and integration, which was confusing and inflexible. I split them so I could set up the node version parameter and dependency on parameterized bootstrap version.

- run: sudo apt-get update && sudo apt-get install python -y
- <<: *restore_cache
# python is not built in and node-gyp needs it to build lmdb
- run: sudo apt-get update && sudo apt-get install -y python3 python-is-python3
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is how you get a python bin in noble apparently

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If just for node-gyp, you might be able to get away with just installing setuptools via pip

unit_tests_node20:
executor:
name: node
image: "20.19"
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

latest node 20. hopeful renovate will pick this up.

unit_tests_node22:
executor:
name: node
image: "22.18"
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same here

name: node
image: "22.18"
<<: *test_template

integration_tests_gatsby_source_wordpress:
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

don't know why this one integration test has a whole custom setup. I didn't look into it. seems to work.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the main thing I remember different about this test suite from others is that it spawn wordpress app in docker (to source content from), so possibly when the test was introduced it had needs not fullfilled by setup for other test suites

@@ -1228,47 +1236,48 @@ describe(`Query schema`, () => {
}
`)
})
;(shouldTestGC ? it : it.skip)(
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

view this diff with "hide whitespace", it's mostly indentation

@serhalp serhalp marked this pull request as ready for review August 16, 2025 00:03
@serhalp serhalp added topic: core Relates to Gatsby's core (e.g. page loading, reporter, state machine) dependencies Pull requests that update a dependency file and removed status: triage needed Issue or pull request that need to be triaged and assigned to a reviewer labels Aug 16, 2025
@serhalp serhalp force-pushed the codex/extend-node.js-engines-to-include-22 branch from 7854825 to 8b25061 Compare August 18, 2025 22:30
@serhalp serhalp merged commit 7451d2f into master Aug 19, 2025
55 checks passed
@serhalp serhalp deleted the codex/extend-node.js-engines-to-include-22 branch August 19, 2025 14:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
codex dependencies Pull requests that update a dependency file topic: core Relates to Gatsby's core (e.g. page loading, reporter, state machine)
Projects
None yet
Development

Successfully merging this pull request may close these issues.

gatsby develop fails with Node.js 22.7.0–22.9.0
3 participants