Skip to content

Conversation

xrmx
Copy link
Contributor

@xrmx xrmx commented Apr 15, 2022

References to other Issues or PRs

Fixes #23355
Fixes #19751
Fixes #22625

Brief description of what is fixed or changed

antlr4 4.10 changed the ATN serialization format (https://github.com/antlr/antlr4/releases/tag/4.10) so that the current latex parser built with 4.7.2 does not work anymore with the latest antlr4-python3-runtime. This PR regenerate a python3 version of the latex grammar using latest antlr 4.10.

Other comments

Wondering about adding an optional dependency on a specific antlr4-python3-runtime version (https://setuptools.pypa.io/en/latest/userguide/dependency_management.html?highlight=install_requires#optional-dependencies)

Release Notes

  • parsing
    • Regenerate autolev and latex parsers with antlr 4.10.1
    • Bump required antlr4 python runtime to 4.10.*

@sympy-bot
Copy link

sympy-bot commented Apr 15, 2022

Hi, I am the SymPy bot (v167). I'm here to help you write a release notes entry. Please read the guide on how to write release notes.

Your release notes are in good order.

Here is what the release notes will look like:

This will be added to https://github.com/sympy/sympy/wiki/Release-Notes-for-1.11.

Click here to see the pull request description that was parsed.
<!-- Your title above should be a short description of what
was changed. Do not include the issue number in the title. -->

#### References to other Issues or PRs
<!-- If this pull request fixes an issue, write "Fixes #NNNN" in that exact
format, e.g. "Fixes #1234" (see
https://tinyurl.com/auto-closing for more information). Also, please
write a comment on that issue linking back to this pull request once it is
open. -->
Fixes #23355
Fixes #19751
Fixes #22625
#### Brief description of what is fixed or changed

antlr4 4.10 changed the ATN serialization format (https://github.com/antlr/antlr4/releases/tag/4.10) so that the current latex parser built with 4.7.2 does not work anymore with the latest antlr4-python3-runtime. This PR regenerate a python3 version of the latex grammar using latest antlr 4.10.

#### Other comments

Wondering about adding an optional dependency on a specific antlr4-python3-runtime version (https://setuptools.pypa.io/en/latest/userguide/dependency_management.html?highlight=install_requires#optional-dependencies)

#### Release Notes

<!-- BEGIN RELEASE NOTES -->

* parsing
  * Regenerate autolev and latex parsers with antlr 4.10.1
  * Bump required antlr4 python runtime to 4.10.*

<!-- END RELEASE NOTES -->

Update

The release notes on the wiki have been updated.

@asmeurer
Copy link
Member

Looks like the tests are failing here.

@asmeurer
Copy link
Member

If we merge this, we need to make sure the antlr runtime pin is updated in the conda-forge recipe the next time SymPy is released.

@xrmx
Copy link
Contributor Author

xrmx commented Apr 15, 2022

Will take a look at the latex tests, the authors mailmap one looks unrelated.

@oscarbenjamin
Copy link
Collaborator

the authors mailmap one looks unrelated.

Yes, it is: #23363 (comment)

@xrmx
Copy link
Contributor Author

xrmx commented Apr 22, 2022

Puzzled why antlr 4.10 runtime could not parse the version from the serialized parser from the 4.10.1 jar. Took a bit to recognize it was autolev and not latex. I guess antlr target in setup.py should be extended to build the autolev stuff too.

@asmeurer
Copy link
Member

It doesn't surprise me if there are some inconsistencies there, but they two parsers should definitely both be using the same functionality and tooling where possible.

@xrmx
Copy link
Contributor Author

xrmx commented Apr 24, 2022

PR for building the autolev parser here #23416

@xrmx xrmx force-pushed the update-parser-410 branch 3 times, most recently from e197007 to 8435a2d Compare April 26, 2022 07:35
@xrmx
Copy link
Contributor Author

xrmx commented Apr 29, 2022

A quick update: with the last push latex tests are fine, autolev is still broken because IIRC for some reason the antlr generator visitor is None and so nothing works. It well may be something silly but with all these dynamic imports the root causes tend to be well hidden 😅

@eagleoflqj eagleoflqj force-pushed the update-parser-410 branch from 8435a2d to 59720b1 Compare July 3, 2022 04:25
@eagleoflqj eagleoflqj self-assigned this Jul 3, 2022
@eagleoflqj eagleoflqj marked this pull request as ready for review July 3, 2022 04:38
@github-actions
Copy link

github-actions bot commented Jul 3, 2022

Benchmark results from GitHub Actions

Lower numbers are good, higher numbers are bad. A ratio less than 1
means a speed up and greater than 1 means a slowdown. Green lines
beginning with + are slowdowns (the PR is slower then master or
master is slower than the previous release). Red lines beginning
with - are speedups.

Significantly changed benchmark results (PR vs master)

Significantly changed benchmark results (master vs previous release)

       before           after         ratio
     [77f1d79c]       [b64cfcdb]
     <sympy-1.10.1^0>                 
+      99.4±0.2ms        180±0.4ms     1.81  sum.TimeSum.time_doit

Full benchmark results can be found as artifacts in GitHub Actions
(click on checks at the top of the PR).

@oscarbenjamin
Copy link
Collaborator

I don't know anything about the antlr parsing so can't review this myself. Is this something that should go into the 1.11 release? If so please add the 1.11 milestone.

@eagleoflqj
Copy link
Member

I think it would be good to upgrade this dependency in 1.11.

@eagleoflqj eagleoflqj added this to the SymPy 1.11 milestone Jul 3, 2022
@asmeurer asmeurer merged commit 9276123 into sympy:master Jul 6, 2022
@asmeurer
Copy link
Member

asmeurer commented Jul 6, 2022

Agree. We also need to make sure to update the pin in the conda-forge recipe when the release happens.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
5 participants