Skip to content

Conversation

LecrisUT
Copy link
Contributor

@LecrisUT LecrisUT commented Jan 10, 2023

This is a companion PR to spglib/spglib#210
Closes: #83

Checklist

  • Used a personal fork of the feedstock to propose changes
  • Bumped the build number (if the version is unchanged)
  • Reset the build number to 0 (if the version changed)
  • Re-rendered with the latest conda-smithy (Use the phrase @conda-forge-admin, please rerender in a comment in this PR for automated rerendering)
  • Ensured the license file is being packaged.

@conda-forge-webservices
Copy link
Contributor

Hi! This is the friendly automated conda-forge-linting service.

I just wanted to let you know that I linted all conda-recipes in your PR (recipe) and found it was in an excellent condition.

@LecrisUT LecrisUT marked this pull request as draft January 10, 2023 15:55
@LecrisUT LecrisUT mentioned this pull request Jan 10, 2023
5 tasks
@LecrisUT
Copy link
Contributor Author

Why are the builds passing despite the dependent PR not being merged?

@atztogo atztogo mentioned this pull request Feb 8, 2023
@jochym
Copy link
Contributor

jochym commented Feb 24, 2023

I like your minimal changes @LecrisUT . How large the fortran bindings are? Are they built together with main package? I would prefer not to expand the package too much. Especially the dependencies. Maybe we need to split it into separate package.

How about a new dependency (scikit-build). Is this a new dependency of spglib or just a workaround? If this is just build it is fine. If it expands the run dependencies it is less perfect.

@jochym
Copy link
Contributor

jochym commented Feb 24, 2023

@conda-forge-admin, please rerender

@LecrisUT
Copy link
Contributor Author

The build system has evolved a bit since this PR. Basically we use scikit-build-core to build (alternative to setup_tools). I don't know how conda and pip differ, but if it can build the python library from pyproject.toml, it should all be compatible.

I've got a few questions:

  • where does conda put non-python libraries? Does it automatically specify environment variables like CMAKE_INSTALL_PREFIX for the installation?
  • currently I've changed the Python bindings to search for spglib libraries on LD_LIBRARY_PATH. Does conda prepend this to link to conda libraries first?
  • is there an equivalent to pip install . to develop locally?

@jochym jochym marked this pull request as ready for review February 24, 2023 09:09
@jochym
Copy link
Contributor

jochym commented Feb 24, 2023

Conda and pip are quite different, but there is no limitation to only setuptools for building. Essentially: whatever you need. So we can shift the build system to the different one. The only requirement is to have a functioning install at the end.
But if we want to follow the upstream in build tooling (I am all for it) - we need to make sure the build works and the test coverage is good. Could you (@atztogo @LecrisUT ), please, check the testing part of meta.yml to verify if the test is as complete as possible. Multiple packages and people depend on spglib (over 1M downloads...) so we should make sure it stays healthy!

BTW @LecrisUT : shall I add you to maintainers?

@jochym
Copy link
Contributor

jochym commented Feb 24, 2023

@LecrisUT LecrisUT marked this pull request as draft February 24, 2023 10:38
@LecrisUT
Copy link
Contributor Author

Indeed, most changes done in this minor release were on the build system, so for the users it should be a simple plug-and-play replacement. We just have to make sure we reflect the changes from upstream. I am trying to build the conda package manually.

@LecrisUT
Copy link
Contributor Author

@jochym I have pushed the necessary fixes to the build. The conda build system is not as intuitive as the others I have contributed to. I have pushed a temporary commit with the head to develop in order to test the CI build. We'll push a rc tag later.

Question, do we push RC's here as well, or is it automated?

@LecrisUT
Copy link
Contributor Author

LecrisUT commented Feb 24, 2023

The other issue appears to be with macos arm doing cross-compilation. That means we cannot do the compilation, nor can we test the ctest properly unless we ship the testing executables. Any environment variable that tells us if we are cross-compiling?

-DSPGLIB_WITH_Fortran=ON
cmake --build ./build
# TODO: This should be moved to run_test.sh. Cannot get the appropriate work directory to run these tests
ctest --test-dir ./build
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How can we move this to run_test.sh? I can't find a way to detect the previous work directory?

We would like to use ctest so that it detects all the tests that we support, but we should also be testing outside the build directory to make sure that the installation is functioning.

@LecrisUT
Copy link
Contributor Author

  • currently I've changed the Python bindings to search for spglib libraries on LD_LIBRARY_PATH. Does conda prepend this to link to conda libraries first?

Please advise on this.

@atztogo
Copy link
Contributor

atztogo commented Feb 25, 2023

The other issue appears to be with macos arm doing cross-compilation. That means we cannot do the compilation, nor can we test the ctest properly unless we ship the testing executables. Any environment variable that tells us if we are cross-compiling?

It seems failing at linking gtest. Can we use https://anaconda.org/conda-forge/gtest?

@atztogo
Copy link
Contributor

atztogo commented Feb 25, 2023

The other issue appears to be with macos arm doing cross-compilation. That means we cannot do the compilation, nor can we test the ctest properly unless we ship the testing executables. Any environment variable that tells us if we are cross-compiling?

It seems failing at linking gtest. Can we use https://anaconda.org/conda-forge/gtest?

I see it is set in meta.yaml. But only x86-64 version is installed. There exists osx-arm64/gtest-1.12.1-hffc8910_0.conda. Why is it not installed? In another way, gtest is downloaded from https://github.com/google/googletest if we don't install gtest from conda. Am I correct?

@LecrisUT
Copy link
Contributor Author

The issue is that when downloading, it also installs gtest. But I think we can disable that.

Still windows will be failing though.

@LecrisUT LecrisUT marked this pull request as ready for review February 27, 2023 14:20
@LecrisUT LecrisUT force-pushed the cmake-build branch 2 times, most recently from 00dc564 to b508bff Compare March 1, 2023 08:35
@LecrisUT LecrisUT changed the base branch from main to rc March 1, 2023 12:24
@LecrisUT LecrisUT mentioned this pull request Mar 7, 2023
5 tasks
@LecrisUT LecrisUT marked this pull request as draft March 7, 2023 07:20
@LecrisUT LecrisUT marked this pull request as ready for review March 7, 2023 13:11
@LecrisUT
Copy link
Contributor Author

LecrisUT commented Mar 7, 2023

Ok, finally we've got everything setup

@atztogo or @jochym care to review this? a1f9805 and be62f4e are the main commits.

Note, this is on an RC channel, so it would be helpful to push it early so we can confirm everything works

LecrisUT added 3 commits March 7, 2023 14:32
Note: ctest should be run in `test` not `build`

Signed-off-by: Cristian Le <cristian.le@mpsd.mpg.de>
@LecrisUT LecrisUT changed the title Switch to newer cmake build flow (spglib/spglib#210) v2.1.0-rc1 Mar 7, 2023
@jochym
Copy link
Contributor

jochym commented Mar 7, 2023

Guys. We should not release rc packages. It is fine to use the PR as a testing ground before actual releases. But I would advise against merging them. Conda-forge packages should be release-level quality.

@LecrisUT
Copy link
Contributor Author

LecrisUT commented Mar 7, 2023

Note that this is deployed on a different branch and label, so users will not be affected by this. We should be releasing rcs like this so we can catch any errors that we should be fixing upstream.

@atztogo
Copy link
Contributor

atztogo commented Mar 8, 2023

@LecrisUT, can we simply release v2.1.0?

@LecrisUT
Copy link
Contributor Author

LecrisUT commented Mar 8, 2023

There is still 1-2 things to test with a v2.1.0-rc2.

For the long run it is better to go with this design. We should allow the downstream developers some time to check if there aren't any unexpected accuracy or building issues.

@jochym insist that we should only publish release-level, and I agree with that, and this rc branch is helping us achieve that. I don't want downstream users to be surprised that stuff doesn't work as expected and they have to wait for a fix on our side. This gives us a bit of time to confirm that everything is working.

@atztogo
Copy link
Contributor

atztogo commented Mar 8, 2023

I think that to keep it simple may be a choice although we may surprise users in sometimes, because we may have to consider distribution of pains between users and developers or maintainer for sustainability of open source scientific software project where number of developers and maintainers is few.

@LecrisUT
Copy link
Contributor Author

LecrisUT commented Mar 8, 2023

Note that the rc channel is already setup (spglib_rc label) and it is very easy to maintain (just change the target branch of the automated PR). It is a simpler workflow for us as well because when we release rcs we can test these on all platforms. E.g. for this one I've tested how the libraries are detected in #96 and we should simplify one point upstream.

@atztogo
Copy link
Contributor

atztogo commented Mar 10, 2023

@LecrisUT, thanks for your explanation.
I don't know well about convention in the conda-forge community. In general, following convention is sometimes important not to surprise people in community or to avoid hidden pitfalls. So I would like to decide @jochym about this PR.

@LecrisUT
Copy link
Contributor Author

Indeed, this approach is coming from standard convention. matplotlib, pandas, for example. Here is the relevant documentation.

@zklaus
Copy link

zklaus commented Mar 20, 2023

Hi, after a brief discussion in the Conda-forge gitter channel, I just wanted to confirm that, from a Conda-forge perspective, releasing rc builds on the rc channel is fine, even good. That means that downstream library and application developers as well as interested users have the option to test the new release and report any last-minute issues.

Keep in mind, that they have to ask for it explicitly by adding the spglib_rc channel as a source, so there is no risk of ordinary users inadvertently installing pre-releases.

Does that assuage your concerns, @jochym?

@LecrisUT LecrisUT merged commit b2a0b11 into conda-forge:rc Apr 13, 2023
@LecrisUT LecrisUT deleted the cmake-build branch September 15, 2023 07:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants