I'm honestly not sure what needs to change in the package.json because I thought "type": "module", "main" and "types" would be enough, but apparently not. The ESM situation is a mess. When i turn on "moduleResolution": "nodenext", I get errors like

test.ts:1:22 - error TS2305: Module '"antlr4"' has no exported member 'CommonTokenStream'.

1 import { CharStream, CommonTokenStream }  from 'antlr4';

It works with "moduleResolution": "node", but according to TS docs that's really designed for CJS and doesn't support packages with "export" maps, so I wouldn't be able to use antlr4 and packages with export maps in the same project.