-
Notifications
You must be signed in to change notification settings - Fork 18
Description
Motivation
Found this upon investigating facebook/docusaurus#1782 and profiling memory usage in Chrome dev tools
In a large project (~> 400 pages), this plugin will consume huge memory and crash it due to the fact that webpack 4 outputs a large amount of data with its stats.toJson() by default
this.stats = compilation.getStats().toJson(); |
The error will be
FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory
Even increasing to 8gb heap memory doesn't work.
Upon removing this plugin, the website can build flawlessly (even without increasing memory limit)
Recommendation
Possible improvement
- I believe we dont need full output of stats.toJson.because it can be very resource-heavy. We should minimize the amount of data retrieved from the stats object by passing needed stats https://webpack.js.org/configuration/stats/ instead
Refer to stats.toJson source code.
https://github.com/webpack/webpack/blob/1a773336187ec0218e060d453f801bc73ab98a6e/lib/Stats.js#L104
Maybe something like
this.stats = compilation.getStats().toJson({
all: false,
chunks: true,
entrypoints: true,
chunkModules: true,
reasons: true,
}, true);
- Rework the logic of getting chunk origin
react-loadable-ssr-addon/source/ReactLoadableSSRAddon.js
Lines 124 to 151 in 1b0c0a0
/** * Get application chunk origin * @method getChunkOrigin * @param {object} id - Webpack application chunk id * @param {object} names - Webpack application chunk names * @param {object} modules - Webpack application chunk modules * @returns {array} Chunk Keys */ /* eslint-disable class-methods-use-this */ getChunkOrigin({ id, names, modules }) { const origins = new Set(); for (let i = 0; i < modules.length; i += 1) { const { reasons } = modules[i]; for (let j = 0; j < reasons.length; j += 1) { const { type, userRequest } = reasons[j]; if (type === 'import()') { origins.add(userRequest); } } } if (origins.size === 0) { return [names[0] || id]; } if (this.entrypoints.has(names[0])) { origins.add(names[0]); } return Array.from(origins); }
reasons
stats is very heavy and consume a lot of memory (still crash the site)
Can we use a more memory-friendly stats option and achieve same result ?