[red-knot] Auto generate statement nodes #16645
Conversation
e8064ed to
713cf28
Compare
8580b75 to
9b2b18b
Compare
9b2b18b to
03ce47e
Compare
|
| { name = "decorator_list", type = "Decorator*" }, | ||
| { name = "name", type = "Identifier" }, | ||
| # check remove | ||
| { name = "type_params", type = "Box<crate::TypeParams>?" }, |
There was a problem hiding this comment.
Interesting, this is boxed here to reduce the size of statement. But it's not the same in StmtTypeAlias was is tested there and the benefit was small?
There was a problem hiding this comment.
You can try, but I suspect that StmtFunctionDef is simply the largest statement node and reducing the size of any other node doesn't help to reduce the size of Stmt.
There was a problem hiding this comment.
Oh right, I forgot about the enum size thing. Thanks.
| { name = "decorator_list", type = "Decorator*" }, | ||
| { name = "name", type = "Identifier" }, | ||
| # check remove | ||
| { name = "type_params", type = "Box<crate::TypeParams>?" }, |
There was a problem hiding this comment.
You can try, but I suspect that StmtFunctionDef is simply the largest statement node and reducing the size of any other node doesn't help to reduce the size of Stmt.
crates/ruff_python_ast/generate.py
Outdated
| # Node structs | ||
|
|
||
|
|
||
| def write_size_test(out: list[str]) -> None: |
There was a problem hiding this comment.
I think I prefer to keep this code outside generate.rs. It doesn't benefit from the codegen and being able to change code directly in rs file has better ergonomics (I don't have to go searching where the code is coming from and I've an IDE that can help me figure out the code)
There was a problem hiding this comment.
+1, it's also easier to just hover over the type and find the size there.
While you're at it, can you also remove the size assertions for Pattern and ExprFString specifically for Rustc < 1.76 as our MSRV is 1.83 now.
There was a problem hiding this comment.
You've already addressed this, but just calling out that I agree with @MichaReiser on this — the generate script should only produce code that actually depends on the AST definitions in ast.toml. Anything that is not parameterized like that should go in a hand-edited .rs file like you have now.
crates/ruff_python_ast/generate.py
Outdated
| # Node structs | ||
|
|
||
|
|
||
| def write_size_test(out: list[str]) -> None: |
There was a problem hiding this comment.
You've already addressed this, but just calling out that I agree with @MichaReiser on this — the generate script should only produce code that actually depends on the AST definitions in ast.toml. Anything that is not parameterized like that should go in a hand-edited .rs file like you have now.
* main: (53 commits) [syntax-errors] Tuple unpacking in `for` statement iterator clause before Python 3.9 (#16558) Ruff v0.10 Release (#16708) Add new `noqa` specification to the docs (#16703) describe requires-python fallback in docs (#16704) [red-knot] handle cycles in MRO/bases resolution (#16693) [red-knot] Auto generate statement nodes (#16645) [`pylint`] Better inference for `str.strip` (`PLE310`) (#16671) [`pylint`] Improve `repeated-equality-comparison` fix to use a `set` when all elements are hashable (`PLR1714`) (#16685) [`pylint`/`pep8-naming`] Check `__new__` argument name in `bad-staticmethod-argument` and not `invalid-first-argument-name-for-class-method` (`PLW0211`/`N804`) (#16676) [`flake8-pyi`] Stabilize fix for `unused-private-type-var` (`PYI018`) (#16682) [`flake8-bandit`] Deprecate `suspicious-xmle-tree-usage` (`S320`) (#16680) [`flake8-simplify`] Avoid double negation in fixes (`SIM103`) (#16684) [`pyupgrade`]: Improve diagnostic range for `redundant-open-mode` (`UP015`) (#16672) Consider all `TYPE_CHECKING` symbols for type-checking blocks (#16669) [`pep8-naming`]: Ignore methods decorated with `@typing.override` (`invalid-argument-name`) (#16667) Stabilize FURB169 preview behavior (#16666) [`pylint`] Detect invalid default value type for `os.environ.get` (`PLW1508`) (#16674) [`flake8-pytest-style`] Allow for loops with empty bodies (`PT012`, `PT031`) (#16678) [`pyupgrade`]: Deprecate `non-pep604-isinstance` (`UP038`) (#16681) [`flake8-type-checking`] Stabilize `runtime-cast-value` (`TC006`) (#16637) ...
Summary
Part of #15655
Replaced statement nodes with autogenerated ones. Reused the stuff we introduced in #16285. Nothing except for copying the nodes to new format.
Test Plan
Tests run without any changes. Also moved the test that checks size of AST nodes to
generated.rssince all of the structs that it tests are now there.