Replies: 9 comments 38 replies
-
|
Beta Was this translation helpful? Give feedback.
-
|
Thanks for sharing. It looks like you saw a 1.35x speedup (about 10 seconds off) vs. no concurrency, which is what we expect in most projects few a couple hundred files. We tend to see bigger time savings in bigger projects (> 1000 files). The plugins and rules you're using also have an affect, as well. |
Beta Was this translation helpful? Give feedback.
-
|
Beta Was this translation helpful? Give feedback.
-
Edit: More accurate reply in thread
With Without any cache
With cache: Slower with concurrency
Let us know what could be useful 👍. The project is pretty large, and we have a lot of ESLint rules in place |
Beta Was this translation helpful? Give feedback.
-
|
Cheers, just sent it ✅ |
Beta Was this translation helpful? Give feedback.
-
|
Thanks for the logs @fabiendem. The only line that strikes me is this one:
That means ESLint is taking about 13 seconds to enumerate the 9000 files that need linting in your project, which is a lot. File enumeration is independent from caching or concurrency, so it should be similar across runs with different options. When the config array is loaded for the first time in each thread, it takes about 3.60 seconds. For instance:
That is also a lot, but probably not uncommon at all for TypeScript projects. |
Beta Was this translation helpful? Give feedback.
-
|
Nice!
Looks like it aligns with this output (filtering rules are a tad more advanced in the eslint config I think)
OK makes sense. So does that mean that we're hitting a wall here? Or do you think there is still room for improvement? |
Beta Was this translation helpful? Give feedback.
-
There's some room for improvement on ESLint's side to ensure that threads aren't created unnecessarily when the cache is used. I'm actually working on this in #20067. Arguably, that won't help much when the cache is missing or invalidated.
In the ESLint repo we use lint-staged for local development to make sure that only new and modified files are linted. This may or may not work in your case depending on how your project is organized. |
Beta Was this translation helpful? Give feedback.
-
|
Thanks
Great ok
Yeah we're already using this 👍 |
Beta Was this translation helpful? Give feedback.
-
|
I tried enabling concurrency in GitHub Actions (hosted runner for a private repo). Unfortunately, I could not see any performance gains. Yaml: jobs:
lint:
name: Lint
runs-on: ubuntu-24.04
## timeout-minutes: 30
steps:
## ... setup Node, install deps etc ...
- name: Test performance of ESLint multi-threading
if: ${{ success() || failure() }}
run: |
sudo apt install hyperfine
echo "================================================"
echo "CPU: $(lscpu | awk -F: '/Model name/ {print $2}' | sed 's/^ *//')"
echo "Physical cores: $(lscpu | awk -F: '/Core\(s\) per socket/ {cores=$2} /Socket\(s\)/ {sockets=$2} END {print cores * sockets}' | sed 's/^ *//')"
echo "Logical cores: $(nproc)"
echo "Files: $(find . -type f -not -path "./node_modules/*" | wc -l)"
echo "================================================"
hyperfine --runs 3 --parameter-list threads off,auto,1,2,4 'npx eslint --quiet --concurrency={threads}'Output: ================================================
CPU: AMD EPYC 7763 64-Core Processor
Physical cores: 1
Logical cores: 2
Files: 1607
================================================
Benchmark 1: npx eslint --quiet --concurrency=off
Time (mean ± σ): 132.940 s ± 3.254 s [User: 189.024 s, System: 11.975 s]
Range (min … max): 130.937 s … 136.694 s 3 runs
Warning: Statistical outliers were detected. Consider re-running this benchmark on a quiet system without any interferences from other programs. It might help to use the '--warmup' or '--prepare' options.
Benchmark 2: npx eslint --quiet --concurrency=auto
Time (mean ± σ): 130.597 s ± 1.055 s [User: 186.092 s, System: 12.726 s]
Range (min … max): 129.466 s … 131.555 s 3 runs
Benchmark 3: npx eslint --quiet --concurrency=1
Time (mean ± σ): 131.636 s ± 3.073 s [User: 188.378 s, System: 11.720 s]
Range (min … max): 129.376 s … 135.134 s 3 runs
Benchmark 4: npx eslint --quiet --concurrency=2
Time (mean ± σ): 131.654 s ± 2.461 s [User: 246.427 s, System: 12.567 s]
Range (min … max): 128.870 s … 133.540 s 3 runs
Benchmark 5: npx eslint --quiet --concurrency=4
Time (mean ± σ): 198.077 s ± 2.426 s [User: 353.848 s, System: 23.758 s]
Range (min … max): 195.745 s … 200.587 s 3 runs
Summary
npx eslint --quiet --concurrency=auto ran
1.01 ± 0.02 times faster than npx eslint --quiet --concurrency=1
1.01 ± 0.02 times faster than npx eslint --quiet --concurrency=2
1.02 ± 0.03 times faster than npx eslint --quiet --concurrency=off
36m 31sIt’d be great to be able to shard the check via something like PS: If you want to replicate this test in your project, check if your yaml contains something like |
Beta Was this translation helpful? Give feedback.
-
|
I wouldn’t expect any gains with one physical core. |
Beta Was this translation helpful? Give feedback.
-
|
Fair. I did not know how many cores were included in GitHub Actions for private repos and was hoping there’d be at least two. I guess we'd need to switch to self-hosted runners (or an alternative service) to speed things up. It'd be great to see data from other folks to supplement by findings about GitHub Actions. |
Beta Was this translation helpful? Give feedback.
-
|
The details on GitHub runner environments are here: It looks like private repo runners only have two logical cores on Linux and Windows. If you can run on a Mac, you might see a difference. |
Beta Was this translation helpful? Give feedback.
-
|
I tried switching to macos runner (with 3 cores), here are the results: Looks like the difference is still marginal. Yaml: jobs:
lint:
name: Lint
runs-on: macos-15
## timeout-minutes: 30
steps:
## ... setup Node, install deps etc ...
- name: Test performance of ESLint multi-threading
if: ${{ success() || failure() }}
run: |
brew install hyperfine
## warm up to avoid outlier results
npx eslint --quiet
echo "================================================"
echo "CPU: $(sysctl -n machdep.cpu.brand_string)"
echo "Physical cores: $(sysctl -n hw.physicalcpu)"
echo "Logical cores: $(sysctl -n hw.logicalcpu)"
echo "Files: $(find . -type f -not -path "./node_modules/*" | wc -l)"
echo "================================================"
hyperfine --runs 3 --parameter-list threads off,auto,1,2,4 'npx eslint --quiet --concurrency={threads}' |
Beta Was this translation helpful? Give feedback.
-
|
Internal Frontend Monorepo at Storyblok which is a Vue application with 3622 files.
{
files: ['**/*.vue', '**/*.ts'],
plugins: {
'@typescript-eslint': tseslint.plugin,
},
languageOptions: {
parser: vueParser,
parserOptions: {
ecmaVersion: 'latest',
sourceType: 'module',
parser: {
js: tseslint.parser,
ts: tseslint.parser,
'<template>': 'espree',
},
projectService: true,
extraFileExtensions: ['.vue'],
tsconfigRootDir: import.meta.dirname,
},
},
rules: {
'@typescript-eslint/no-explicit-any': 'error',
// Requires type information: super slow, but extremely useful
'@typescript-eslint/no-unsafe-assignment': 'warn',
'@typescript-eslint/no-unsafe-member-access': 'warn',
'@typescript-eslint/no-unsafe-argument': 'warn',
'@typescript-eslint/no-unsafe-call': 'warn',
'@typescript-eslint/no-unsafe-return': 'warn',
'@typescript-eslint/no-unsafe-type-assertion': 'warn',
'@typescript-eslint/no-deprecated': 'warn',
'@typescript-eslint/consistent-type-assertions': [
'error',
{ assertionStyle: 'as' },
],
'no-unused-vars': 'off',
'@typescript-eslint/no-unused-vars': [
'error',
{
argsIgnorePattern: '^_',
varsIgnorePattern: '^_',
caughtErrorsIgnorePattern: '^_',
},
],
'@typescript-eslint/consistent-type-definitions': ['warn', 'type'],
'@typescript-eslint/ban-ts-comment': [
'error',
{
'ts-check': false,
'ts-expect-error': 'allow-with-description',
'ts-ignore': true,
'ts-nocheck': true,
},
],
},
}, |
Beta Was this translation helpful? Give feedback.
-
|
Thanks for sharing, that’s some great results! Indeed, there is a known performance overhead for typescript-eslint that you’re actually paying for each thread. You can try disabling any rules that perform type checking to see the difference (I realize you probably don’t want to do that for real, so just to see the difference it can make). |
Beta Was this translation helpful? Give feedback.
-
|
We dropped a considerable time off CI for the app at https://mercury.com! Thank you for this. Local runs showed a time loss from threading, so we're staying single-thread on local machines. These benchmarks are from a week or so ago when we were using the final commit in the PR. Number of files: LocalM3Max, 16 core x 16 core. Control run 1: Here's some hyperfine output from today, after updating to the release version. Same results as above. Probably because of the overhead from our massive number of TS defs. On CIControl: The rest on concurrency auto: On GH 2 core runners: |
Beta Was this translation helpful? Give feedback.
-
|
Nice! Thanks for sharing. |
Beta Was this translation helpful? Give feedback.
-
|
Some different kind of feedback: If I run |
Beta Was this translation helpful? Give feedback.
-
|
Please open an issue to request this. |
Beta Was this translation helpful? Give feedback.
-
|
Oddly, my best result comes from no concurrence at all:
1.13 ± 0.23 times faster than npx eslint --quiet --concurrency=off "**/*.{cj,mj,t,j}s?(x)" --cache --cache-location node_modules/.cache/eslint3490 filesfind . -type d \( -name "node_modules" -o -name "dist" \) -prune -o -type f \( -name "*.js" -o -name "*.jsx" -o -name "*.ts" -o -name "*.tsx" -o -name "*.cjs" -o -name "*.mjs" -o -name "*.cjsx" -o -name "*.mjsx" \) -print | wc -l
Did I do something wrong? |
Beta Was this translation helpful? Give feedback.
-
|
Can you try running without the cache? (See previous comments in this thread regarding cache performance when using concurrency.) |
Beta Was this translation helpful? Give feedback.
-
|
Not sure I'm following. Which comment is that? |
Beta Was this translation helpful? Give feedback.
-
|
This thread I think: #20040 (comment) |
Beta Was this translation helpful? Give feedback.
-
|
@mbrevda thanks. So it looks like you're seeing a 1.76x speedup vs. no concurrency if you use @fabiendem thanks, that's the one. |
Beta Was this translation helpful? Give feedback.
-
Oh, I see. With caching I was seeing < ~2 seconds, and was surprised that it was so much faster than concurrency. Didn't realize this was still being worked on. Thanks - looking forward to the next iteration! |
Beta Was this translation helpful? Give feedback.
-
|
We're seeing OOMs when turning concurrency on at any value on a fairly large codebase (4929 typescript files). Do you have any suggestions for things I could look into or more information I could provide? I'm running it on a Github Codespace with 16 cores, 64 GB RAM. It's running through turbojs. I've tried concurrency values of 2, 4, 8, 16 and it OOMs every time, but never OOMs when running without concurrency. Also please let me know if this is the wrong place to ask about this. Here's an example stacktrace: |
Beta Was this translation helpful? Give feedback.
-
|
Have you tried to increase the memory allocated by Node.js with |
Beta Was this translation helpful? Give feedback.
-
|
|
Beta Was this translation helpful? Give feedback.
-
|
That looks correct. I can't say what's causing the error, we'd need a repro to look into. |
Beta Was this translation helpful? Give feedback.
-
|
I agree with @fasttime's assessment -- this is most likely caused by a plugin you're using. |
Beta Was this translation helpful? Give feedback.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
-
ESLint v9.34.0 introduced parallel linting via the
--concurrencyCLI option. We'd like to hear about your performance improvements using it!How to measure
Please use Hyperfine to collect your data and set it up for a number of different concurrency values, including:
off- the baseline without concurrencyauto- ESLint's best guess at the concurrency to use2- use just two threadsAnd of course, you can add more settings too! We encourage you to try a bunch of different concurrency settings to find the one that works best for your project. Here's an example:
hyperfine --runs 3 --parameter-list threads off,auto,1,2,4,5,6,8,10 'npx eslint --quiet --concurrency={threads}'What to post
When you post your results, please include the following:
find . -type f -name "*.js" -not -path "./node_modules/*" | wc -l(please update so you're only including the files that are actually linted by ESLint)Beta Was this translation helpful? Give feedback.
All reactions