Improve memory usage for large files #30180
Description
e.g.
- open the file and wait for ~ 30s
1. file A
Download Loan Data for 2016 Q3 from https://www.lendingclub.com/info/download-data.action
| empty file | 73 MB 99127 lines |
actual usage | overhead | overhead per line | |
|---|---|---|---|---|---|
| OSX RESIDENT | 190 MB | 353 MB | 163 MB | 90 MB | 952.0 B |
| OSX Heap Snapshot | 50 MB | 138 MB | 88 MB | 15 MB | 158.6 B |
2. file B
Download checker.ts from https://github.com/Microsoft/TypeScript/blob/296660a2a077c32a2ed41cb762ef530031e56417/src/compiler/checker.ts
Generate a 128x times checker:
var fs = require('fs');
var str = fs.readFileSync('checker.ts').toString();
for (var i = 0; i < 7; i++) {
str = str + '\n' + str;
}
fs.writeFileSync('checker-out.ts', str);| empty file | 177 MB 3171200 lines |
actual usage | overhead | overhead per line | |
|---|---|---|---|---|---|
| OSX RESIDENT | 190 MB | 1474 MB | 1284 MB | 1107 MB | 366.0 B |
| OSX Heap Snapshot | 50 MB | 716 MB | 666 MB | 489 MB | 161.7 B |
- OSX shallow size taken by a ModelLine (via Heap Snapshot) - 72 bytes