-
-
Notifications
You must be signed in to change notification settings - Fork 4.8k
test: resolve flakiness in --mcp flag test #19993
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
✅ Deploy Preview for docs-eslint ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
// should not have anything on std out | ||
child.stdout.on("data", data => { | ||
assert.fail(`Unexpected stdout data: ${data}`); | ||
}); | ||
|
||
child.stderr.on("data", data => { | ||
assert.match(data.toString(), /@eslint\/mcp/u); | ||
done(); | ||
if (!doneCalled) { | ||
assert.match(data.toString(), /@eslint\/mcp/u); | ||
doneCalled = true; | ||
done(); | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can we just use getOutput
here to test both the error message and the (lack of) stdout output, as it's done in other tests?
Lines 1219 to 1224 in 8662ed1
const outputAssertion = getOutput(child).then(output => { | |
const expectedSubstring = "Syntax error in selector"; | |
assert.strictEqual(output.stdout, ""); | |
assert.include(output.stderr, expectedSubstring); | |
}); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I did use getOutput
first, but the test fails with the error:
Error: Timeout of 10000ms exceeded. For async tests and hooks, ensure "done()" is called; if returning a Promise, ensure it resolves.
I think it's because the MCP server process isn't exiting on its own, so getOutput
never resolves.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ah, of course. Then let's go ahead and merge this change. It looks like this resolves situations like https://github.com/eslint/eslint/actions/runs/16801498351/job/47583769079, where the server manages to print its initial debug message to stderr before the process is terminated.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM, thanks.
Prerequisites checklist
What is the purpose of this pull request? (put an "X" next to an item)
[ ] Documentation update
[ ] Bug fix (template)
[ ] New rule (template)
[ ] Changes an existing rule (template)
[ ] Add autofix to a rule
[ ] Add a CLI option
[ ] Add something to the core
[x] Other, please explain:
This PR resolves a flakiness issue in the test for starting the MCP server with the --mcp flag.
The test failed intermittently with the error
done() called multiple times
. This was because the stderr stream's data event could fire more than once, causing thedone()
callback to be executed for each chunk of data.What changes did you make? (Give an overview)
I introduced a boolean flag,
doneCalled
, to ensure that thedone()
callback is only executed a single time.Is there anything you'd like reviewers to focus on?
See #19951