Skip to content

Merging long standing refactor branches #1029

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 158 commits into from
Jul 8, 2025
Merged

Merging long standing refactor branches #1029

merged 158 commits into from
Jul 8, 2025

Conversation

michaeljones
Copy link
Collaborator

@michaeljones michaeljones commented May 29, 2025

This PR is designed to take over from:

It merges them into main and goes step by step to resolve the conflicts. It might be reasonable to squash all the conflict resolution down together but I think we're going to end up keeping some of the rest of the history as it is too large a change to reasonably be held in one commit in some ways.

Checklist

Sponsorship

Thank you to the GDAL Project and Melexis for sponsoring this work.

We can't render the full docs for the same class twice.
We're happy enough if the pull request trigger is there and we don't
need both triggering for each push to a pull request.

We might considered having 'push to main' enabled but nothing else.
This renders the title of the groups & pages instead of their less
friendly assigned name.
So that we get 'file' instead of 'DoxCompoundKind.file'
@michaeljones
Copy link
Collaborator Author

I think the comparison is looking ok. Definitely a couple of references not being handled in this branch vs main but possibly just less than ideal rather than blocking. It would be good to get them fixed though.

The tests are failing and it would be good to better understand them. I'm afraid I'm not sure what the intention of the tests are or what needs to be updated when they fail. I guess sometimes it is the code and sometimes the fixtures but I'm not sure which in this case (though likely the code, I guess.)

@Rouslan
Copy link
Contributor

Rouslan commented Jun 22, 2025

@michaeljones Unfortunately, I'm going to be quite busy myself for the next week or so. I'll check it out when I find some time. In the meantime, there are a few things to note:

First of all, for Doxygen 1.9.7 (I don't remember about 1.9.5 and 1.9.6, but 1.9.4 should be fine) and up, test_example[cpp_anon] will always fail. The way Doxygen handles anonymous entities changed and this has never been addressed.

Make sure you merge my latest changes. Updates to Sphinx and Doxygen broke some of the tests, so I had to make some changes.

Unfortunately, my tests are brittle. They rely on the specific output of Doxygen. Several XML attributes are ignored when doing the comparisons but otherwise the tags and non-whitespace text have to match. To accommodate multiple versions, I have a system that allows multiple versions of tests results. For example, in tests/data/examples/test_index, there is "compare.xml" and "compare-1.11.0.xml". For Doxygen versions 1.11.0 and up, "compare-1.11.0.xml" is used; otherwise, "compare.xml" is used. So far, I only made it work, up to version 1.12.0. 1.13.0 introduced a bug that affected the output (doxygen/doxygen#11541), and I had decided to wait instead of incorporating the undesired output.

@michaeljones
Copy link
Collaborator Author

Thank you for taking the time @Rouslan. I appreciate the information. I'll take another look at the tests and at the current state of the output comparison with main. I might end up adding skip/ignore to the tests but we'll see.

I think it is sensible to do a release candidate of a Breathe v5 with these changes given how much of a re-write it all is so we can give people some time to test it out and report any specific issues.

@JasperCraeghs
Copy link
Contributor

JasperCraeghs commented Jun 25, 2025

I tested this in my project with Sphinx' latest master branch and encountered an error that Rouslan already addressed in this commit

      File "/home/developer/venv/lib/python3.11/site-packages/docutils/parsers/rst/states.py", line 2156, in run_directive
        result = directive_instance.run()
                 ^^^^^^^^^^^^^^^^^^^^^^^^
      File "/home/developer/venv/lib/python3.11/site-packages/breathe/directives/function.py", line 81, in run
        args = self._parse_args(argsStr)
               ^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/home/developer/venv/lib/python3.11/site-packages/breathe/directives/function.py", line 169, in _parse_args
        paramQual = parser._parse_parameters_and_qualifiers(paramMode="function")
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    TypeError: DefinitionParser._parse_parameters_and_qualifiers() got an unexpected keyword argument 'paramMode'

Side note: the version that pip reports after pip install git+https://github.com/breathe-doc/breathe.git@b2dc1541ffa7d15206a1ea4ae815aec3c70e52ba shows 4.36.0 instead of something like 4.35.0.dev0+b2dc154, so it can lead to confusing as to which snapshot I'm working with in my Docker container. I like using setuptools-scm to extract the version number from the Git tag and add commit data to a snapshot version.

developer@72939b2b7331:~/repo$ pip show breathe
Name: breathe
Version: 4.36.0

@michaeljones
Copy link
Collaborator Author

Thank you @JasperCraeghs - I'll investigate this and incorporate the change.

@michaeljones
Copy link
Collaborator Author

michaeljones commented Jul 1, 2025

@JasperCraeghs from what I can see that fix is in main and in Rouslan's branch and still in tact in this branch. It isn't showing in the diff because it is already in main and untouched by these changes.

I think there is a chance that you have the wrong version of Breathe installed? Though I understand that I am more out of the loop than most so there is a chance I'm misunderstanding the situation.

I believe this is a link to the current state of the function.py file on this branch:

paramQual = parser._parse_parameters_and_qualifiers("function")

We now have the title properly instead of just the id.
We have a few places where we can update the compare.xml files to
include full titles instead of section ids but otherwise we delete
tests that are failing. This is just to get to green and see what else
needs to be done on the branch.

The plan is to restore these tests when we have a better understanding
of what is going wrong.
Too much going on here and I know too little about the python type
system and how things are resolved. Maybe someone else can help if it
becomes an issue but it doesn't get us much at the moment even if it
was properly typed.
Sphinx 8.x doesn't run on 3.9 and 8.2 doesn't run on 3.10 either.
@michaeljones
Copy link
Collaborator Author

The tests are passing because I've done some minor fixes and then deleted the ones that are failing. They are failing for me locally on main and on Rouslan's branch too. I'm not sure what steps are required to get them working right now. It seems to impact the handling of index entries and references for anonymous structs and enums.

I think I would like to merge this into main and make a Breathe v5 alpha release to signal that those interested could potentially give it a shot and feedback any issues. I would welcome input from @JasperCraeghs and @Rouslan on this when they have time.

@michaeljones
Copy link
Collaborator Author

I've created #1034 with the failing tests that were deleted from here. It makes sense to try to figure out why they are failing but I'm not familiar enough with the full stack at the moment to debug them easily.

@JasperCraeghs
Copy link
Contributor

JasperCraeghs commented Jul 3, 2025

I think there is a chance that you have the wrong version of Breathe installed?

You're right. I detected no issues after installing the tool properly. I'm looking forward to a Breathe v5 alpha release. 🚀

pip install git+https://github.com/breathe-doc/breathe.git@41e05a133a1f3e9d0499a3df21ab56a6f54e9c2a --force-reinstall did not install the parser. I had to run pip install -e ../packages/breathe[build] && make -C ../packages/breathe parser, or build the wheel locally and install it that way. 🙂

@michaeljones michaeljones merged commit 817fe2a into main Jul 8, 2025
90 checks passed
@michaeljones michaeljones deleted the merge-target branch July 8, 2025 19:02
@michaeljones
Copy link
Collaborator Author

I merged it but it did a squash merge (I wasn't paying attention) so I backed that out of main and redid the merge as a normal merge commit which means we preserve the history. The history is a mess but it is better to have some record of it than to have it all in one commit.

@michaeljones
Copy link
Collaborator Author

@JasperCraeghs @Rouslan - v5.0.0a2 is out with your changes when you have the time and energy to try it out. Thanks for all the work. Hopefully we'll fix up any last issues and do a full release.

@michaeljones
Copy link
Collaborator Author

Actually, I don't think the _parser.py file has been generated properly in the release. I'm going to have to look into that.

@michaeljones
Copy link
Collaborator Author

breathe v5.0.0a5 has been released and seems to have the _parser.py file. I haven't done a full build in a test project with it and it is getting late. I'll look into verifying that it works tomorrow.

@JasperCraeghs
Copy link
Contributor

breathe v5.0.0a5 from PyPI works for my project. Thank you.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants