Skip to content

Prepare for conan package / Add ci jobs for ubuntu, mac and windows #234

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 54 commits into from
Jul 10, 2025
Merged

Prepare for conan package / Add ci jobs for ubuntu, mac and windows #234

merged 54 commits into from
Jul 10, 2025

Conversation

phbasler
Copy link
Contributor

@phbasler phbasler commented Apr 8, 2022 •
edited
Loading

In preparation for the conan package I added the options to get the necessary dependencies through conan.
You can build cmaes with conan, as shown in the workflow.

While configuring this I saw that the PythonInterp find_package call was deprecated and so I updated it to find_package(Python3) with the modules. This also allowed me to make the python and t_* tests run with ctest.
You can now just call ctest in your build directory and the 6 tests run.

Furthermore, we can get rid of the custom FindNumpy and FindEigen module. cmake has it's own find modules for eigen and python.

@phbasler
Copy link
Contributor Author

I just saw that it still does not compile on windows with mvsc. I convert this PR to a draft until I fixed it as well.

@phbasler phbasler marked this pull request as draft April 10, 2022 14:24
@phbasler
Copy link
Contributor Author

phbasler commented Apr 10, 2022 •
edited
Loading

@beniz It would not compile on windows until I defined an explicit copy ctr for scalar_normal_dist_op. Can you check if this one is ok?

ATM the python tests are failing because the module is not found for the ctest call, but everything compiles. (Only with MVSC, with gcc ctest runs the python test sucessfully)

@phbasler
Copy link
Contributor Author

phbasler commented Apr 13, 2022 •
edited
Loading

@beniz @nikohansen

I found the problem and maybe one of you has an idea.
On my fork https://github.com/phbasler/libcmaes/tree/conan-windows I set up some CI to run the unit tests, including the python tests, on windows, mac and ubuntu.
Mac and ubuntu work without a problem, but on windows the lcames python module is seperated in two files. One build/python/Release/lcames.pyd and the dll build/src/Release/cmaes.dll. It works there if I copy the cmaes.dll to the directory which contains the .pyd. If I don't copy it, then I get an Import Error because the dll is not found.

What should we do here? Do you have an idea on how to fix it or should we copy it during the cmake call?

Edit: A possible cmake solution to fix this with MVSC using cmake is given in phbasler@bfe0eb4
But it is just copying/installing the cmaes.dll to the python location, so the dll would be in two places.

@nikohansen
Copy link
Collaborator

Sorry, I have no idea regarding the windows test setup.

phbasler and others added 2 commits April 28, 2022 06:30
@phbasler
Conan windows (#5)
f065ae5 
* added CI

* Added gflags

* Includ numpy and boost-python

* Changed pip to pip3

* Extended PYTHONPATH

* Extending pythonpath

* Adjusted  Pythonpath to use Config if available

* Update windows_unit_tests.yml

* Update windows_unit_tests.yml

* Update windows_unit_tests.yml

* Update windows_unit_tests.yml

* Modified python path

* Update CMakeLists.txt

* Moved pythonpath to target generator expression

* Add post_build/install of cmaes.dll to python location on MVSC

* Adjusted workflows to only trigger on master

Co-authored-by: Phil <philipbasler@gmail.com>
Add conan to readme
55f2d39
@phbasler phbasler marked this pull request as ready for review April 28, 2022 04:35
@phbasler phbasler mentioned this pull request May 1, 2022
Open
@beniz
Copy link
Collaborator

beniz commented May 4, 2022

Hi @phbasler I would go for the copy of the DLL is that helps. Thanks for all the work on this!

@phbasler
Copy link
Contributor Author

phbasler commented May 4, 2022

@beniz That's how I implemented now. I also fixed a small mistake I introduced with another PR:
If you include libcmaes as a subdirectory then the doc target was also created, even if you have one of your one (like I have in https://github.com/phbasler/BSMPT) and then I got a cmake error there. I fixed that with the last commit that the doc target is only created if libcmaes is build as the top project

Philipp Basler and others added 3 commits May 4, 2022 21:14
Use the LIBCMAES_TOP_LEVEL variable instead
7827c96
@phbasler
Update README.md
0ecfc3b 
Fixed wrong doxygen link to match the move of the project
@phbasler
Adding conanfile (#6)
b43565e 
Add conanfile to generate conan package
@phbasler
Copy link
Contributor Author

@nikohansen @beniz I finished the conan package. You can build it with `conan create conanfile.py --profile=SomeProfile" where SomeProfile is a conan profile with your compiler and settings how you want to build. You need to fill in the author and topics in the conanfile.

@beniz
Copy link
Collaborator

beniz commented Mar 22, 2023

@phbasler hello, apologies for missing this, you should put your name in the conan file!
I am not familiar with conan, if it works for you, then maybe squash and get ready for a merge!

@phbasler
Copy link
Contributor Author

@beniz In the meantime conan released v2 with some breaking changes, so we would have to make sure everything still works.

@phbasler
Copy link
Contributor Author

Seems that there are some changes necessary. I will have a look at it if I have some time to spare, but this may take a while.

@phbasler phbasler changed the title Enable to get dependencies through conan and cmake Updates Prepare for conan package Jun 2, 2024
@AbrilRBS AbrilRBS mentioned this pull request Nov 22, 2024
Merged
3 tasks
@phbasler
Copy link
Contributor Author

phbasler commented Dec 5, 2024

@beniz just a reminder that this is still open

@phbasler
Copy link
Contributor Author

phbasler commented Dec 5, 2024

@beniz I removed the windows unit tests. There is some extra work required with the boost python setup so the tests fails. But the normal conan create build runs including the test_package, so we see that it still works on windows

@phbasler
Copy link
Contributor Author

@beniz I think this is blocking the conan PR.
Any chance we can finish this before the PR reaches the 3 year mark?

@phbasler phbasler mentioned this pull request May 12, 2025
Open
@phbasler phbasler mentioned this pull request May 12, 2025
Open
@phbasler phbasler changed the title Prepare for conan package Prepare for conan package / Add ci jobs for ubuntu, mac and windows May 12, 2025
@phbasler
Copy link
Contributor Author

@beniz @nikohansen
I saw that other people basically did the same work with #246 and #247 again, maybe we can finish this at some point? I opened this PR now almost 3 years ago

@beniz
Copy link
Collaborator

beniz commented May 12, 2025 •
edited
Loading

Hi @phbasler sorry I have no windows experience or knowhow. If you recommend something that works and can be merged, I'll follow your lead.

Note: if there's a way to test from Linux, let me know

@phbasler
Copy link
Contributor Author

@beniz Please have a look at the CI jobs.
Everything is being build already in the CI on windows.

The only thing not working in windows is the unit tests. This is because the boost python setup still needs adjustments on Windows.
The normal library can be build and used without problems on Windows.

@phbasler
Copy link
Contributor Author

phbasler commented May 21, 2025 •
edited
Loading

@beniz Any news here?
I believe this is activly blocking the conan release as they would prefer it without needing the patch

@phbasler
Copy link
Contributor Author

phbasler commented Jul 1, 2025

@nikohansen @beniz

Another month passed.

@beniz
Copy link
Collaborator

beniz commented Jul 1, 2025

@phbasler thanks your work, just let me what to do, or what rights you'd need to do it yourself as needed.

@phbasler
Copy link
Contributor Author

phbasler commented Jul 1, 2025

@beniz Everything is ready for review. If you can review it, that would be great

@beniz
Copy link
Collaborator

beniz commented Jul 1, 2025

OK the build seems fine for me on linux. I can't test windows.
Maybe if test_package is specific to conan, rename it to reflect so, otherwise this seems OK. I can merge when you tell me so.

@phbasler
Copy link
Contributor Author

phbasler commented Jul 1, 2025

@beniz
test_package is the conan default name for a dummy test_package. If you call conan create then the test_package will be automatically executed.
But if you want we can rename it. The conanfile can be adjusted to look for a different folder.
Do you have a wish for the new name?

@beniz beniz merged commit 35d9a75 into CMA-ES:master Jul 10, 2025
5 checks passed
@phbasler
Copy link
Contributor Author

@beniz I forgot to update the version in the CMakelists.txt

Could you update it and make a new release?
Then I can make the conan package based on that release

@phbasler phbasler mentioned this pull request Jul 14, 2025
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@phbasler @nikohansen @beniz