Skip to content

test: resolve flakiness in --mcp flag test #19993

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Aug 8, 2025
Merged

Conversation

Pixel998
Copy link
Contributor

@Pixel998 Pixel998 commented Aug 7, 2025

Prerequisites checklist

What is the purpose of this pull request? (put an "X" next to an item)

[ ] Documentation update
[ ] Bug fix (template)
[ ] New rule (template)
[ ] Changes an existing rule (template)
[ ] Add autofix to a rule
[ ] Add a CLI option
[ ] Add something to the core
[x] Other, please explain:

This PR resolves a flakiness issue in the test for starting the MCP server with the --mcp flag.

The test failed intermittently with the error done() called multiple times. This was because the stderr stream's data event could fire more than once, causing the done() callback to be executed for each chunk of data.

What changes did you make? (Give an overview)

I introduced a boolean flag, doneCalled, to ensure that the done() callback is only executed a single time.

Is there anything you'd like reviewers to focus on?

See #19951

@Pixel998 Pixel998 requested a review from a team as a code owner August 7, 2025 11:37
@github-project-automation github-project-automation bot moved this to Needs Triage in Triage Aug 7, 2025
@eslint-github-bot eslint-github-bot bot added the chore This change is not user-facing label Aug 7, 2025
Copy link

netlify bot commented Aug 7, 2025

Deploy Preview for docs-eslint ready!

Name Link
🔨 Latest commit c48dd46
🔍 Latest deploy log https://app.netlify.com/projects/docs-eslint/deploys/68948fe52fe9b000089513fb
😎 Deploy Preview https://deploy-preview-19993--docs-eslint.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

@fasttime fasttime moved this from Needs Triage to Triaging in Triage Aug 8, 2025
@fasttime fasttime added the accepted There is consensus among the team that this change meets the criteria for inclusion label Aug 8, 2025
@fasttime fasttime moved this from Triaging to Implementing in Triage Aug 8, 2025
Comment on lines 1348 to +1358
// should not have anything on std out
child.stdout.on("data", data => {
assert.fail(`Unexpected stdout data: ${data}`);
});

child.stderr.on("data", data => {
assert.match(data.toString(), /@eslint\/mcp/u);
done();
if (!doneCalled) {
assert.match(data.toString(), /@eslint\/mcp/u);
doneCalled = true;
done();
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we just use getOutput here to test both the error message and the (lack of) stdout output, as it's done in other tests?

eslint/tests/bin/eslint.js

Lines 1219 to 1224 in 8662ed1

const outputAssertion = getOutput(child).then(output => {
const expectedSubstring = "Syntax error in selector";
assert.strictEqual(output.stdout, "");
assert.include(output.stderr, expectedSubstring);
});

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did use getOutput first, but the test fails with the error:

Error: Timeout of 10000ms exceeded. For async tests and hooks, ensure "done()" is called; if returning a Promise, ensure it resolves.

I think it's because the MCP server process isn't exiting on its own, so getOutput never resolves.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, of course. Then let's go ahead and merge this change. It looks like this resolves situations like https://github.com/eslint/eslint/actions/runs/16801498351/job/47583769079, where the server manages to print its initial debug message to stderr before the process is terminated.

Copy link
Member

@fasttime fasttime left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks.

@fasttime fasttime merged commit 06a22f1 into eslint:main Aug 8, 2025
30 checks passed
@github-project-automation github-project-automation bot moved this from Implementing to Complete in Triage Aug 8, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
accepted There is consensus among the team that this change meets the criteria for inclusion chore This change is not user-facing
Projects
Status: Complete
Development

Successfully merging this pull request may close these issues.

2 participants