Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 83.80%. Comparing base (5c7046b) to head (5d80599).

@@           Coverage Diff            @@
##           develop     #477   +/-   ##
========================================
  Coverage    83.80%   83.80%           
========================================
  Files           25       25           
  Lines         8164     8164           
  Branches      1705     1705           
========================================
  Hits          6842     6842           
  Misses        1322     1322           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

I was trying to make a Julia wrapper around SpglibSpacegroupType for use in my package PeriodicGraphEmbeddings.jl, for which I needed the precise layout of the C struct (see here), so the order was actually quite important!
But you are right, apparently the order is indeed not crucial for most people, otherwise it would have been discovered before. Anyway, even if it's useful to only one other person in the next ten years, I'm happy to contribute!

Oh, yeah you're right. I forgot about memory alignment and such, been doing too much coding in Python lately

@Liozou Did you get in touch with @singularitti? They already have a Julia wrapper: https://spglib.readthedocs.io/en/latest/interface.html#julia-interface

Ah yes you are absolutely right! I had started wrapping only the necessary parts in my code in 2020, before Spglib.jl was released, but I should probably switch to it now.

@Liozou, it is better to assume that the order can be changed and that new member can be added though I am not sure if it is possible in C-julia interface.

If you are only consuming it via C or C++ (not sure about Fortran), than the re-ordering doesn't matter unless you are using extern struct. For Python it doesn't matter because we copy all of the struct data to native Python. A similar approach should be possible in Julia if the C/C++ code is doing the copying

However if you expose the bare struct (which I would not recommend because of potential memory leak issues) you do need the exact structure since it does an implicit extern struct.

Well, Julia requires the exact layout of the C struct in order to wrap it, much the same as you need it in Python here. You can have a look at the relevant definitions in Spglib.jl here and there.

But to be clear, the re-ordering does not really matter for my application either! I am accessing the fields by their name anyway. What does matter is that the C struct and the Julia struct must be kept in sync.
To do so, I see three options:

• increment the major version number of spglib whenever you modify a C struct exposed in the C-API. That's the cleanest solution, but probably overkill.
• the simplest solution is probably for me to switch to Spglib.jl, and for you to make sure that its author is made aware when there is a change in a struct exposed by the C API. That way, everything continues to run smoothly, but it requires a little communication.
• alternatively you can restrict yourselves to only adding new fields at the end of the struct, without touching the previous ones. I believe that should not be breaking, but it constrains the design space.

Well, Julia requires the exact layout of the C struct in order to wrap it, much the same as you need it in Python here.

Yes, but as you can see it is done by a C code so when it reads spglib.h it will automatically reorder the pointer when it compiles that code. On the Julia side it is tricky because the wrapper is in the Julia code (which has the effective extern struct).

Of course there is an issue that the compiled libraries are not compatible and the dependencies would need to be re-compiled. That is usually caught by the packager's automation, e.g. Fedora caught a ABI incompatiblity and I am working on rebuilding.

• increment the major version number of spglib whenever you modify a C struct exposed in the C-API. That's the cleanest solution, but probably overkill.

We are trying to do that as much as possible. We have ABI checks within testing-farm which checks the current version of spglib in the Fedora branch. (Using Fedora here because they are graceful enough to provide such tests, would love it if other distros had similar automation). The reported ABI changes these time are about new functions and deleted private functions which would only be used with extern which we do not support.

But about Spglib.jl. My understanding is that it builds Spglib and it doesn't use any dynamic linkage to a pre-build Spglib (don't know if that is supported), so this issue will not matter until the "bundled" Spglib library has to be updated. @singularitti can you comment or ping someone who can verify this part?

Hi @LecrisUT, I am not a skilled C expert but according to Julia's documentation:

Note that no C header files are used anywhere in the process of calling C functions: you are responsible for making sure that your Julia types and call signatures accurately reflect those in the C header file.

Composite types such as struct in C or TYPE in Fortran90 (or STRUCTURE / RECORD in some variants of F77), can be mirrored in Julia by creating a struct definition with the same field layout.

It seems that we have to write the Julia structs' fields in the same order as the C code.

May I ask which version is the Spglib C code affect by this change? I have not updated Spglib.jl to v2.3.0 since you changed some conventions. I plan to release a breaking change with major version update (say, Spglib.jl v1.0.0, etc.). However, I am very busy recently. @Liozou Would you like to help? Thank you!

In fotran, with bind(c), it seems the order does matter, too.

May I ask which version is the Spglib C code affect by this change?

Very long ago (8 years maybe). Don't worry aboit it. There were other breaking changes in between.

It seems that we have to write the Julia structs' fields in the same order as the C code.

That part I've sort of figured out. What I was curious this time is if Julia builds a static library of spglib, or does it always use a dynamic linkage, or both? How does Julia search for the library at build and runtime?

@LecrisUT The way it works in Julia is the following: Spglib.jl depends on a package called spglib_jll.jl, which contains bindings for a specific version of Spglib (here 2.4.0) for all supported platforms.
spglib_jll is built by Yggdrasil using BinaryBuilder.jl upon explicit request like the last one made by @singularitti where the version 2.4.0 is specified: see JuliaPackaging/Yggdrasil#8478. spglib_jll then provides a handle to libsymspg which is used to call Spglib functions. I (weakly) believe that amounts to dynamic linkage, but I'm not completely sure the distinction should be made in those terms in Julia because it's JIT-compiled anyway.

As a consequence, when a user decides to load the Julia package Spglib.jl, the version of Spglib is that matched by the dependency spglib_jll. In order to reflect an update of Spglib in Julia, there are thus two steps:

• first, make a pull request on Yggdrasil: this will build and tag a new release of spglib_jll.

• then check that the dependency constraint of spglib_jll in Spglib.jl allows the latest version of spglib_jll to be loaded. For example, the current constraint is: spglib_jll = "2 - 2.1" which goes from v2.0.0 included to v2.2.0 excluded, so current code that uses Spglib.jl does not actually use the latest release of spglib_jll, i.e. the latest versions of Spglib. This is why @singularitti said:

  I have not updated Spglib.jl to v2.3.0

  although v2.4.0 of spglib_jll is already released.

@singularitti I'm always happy to help, although I am also limited in the time I can spend. The best way forward would probably be for you to open an issue in your repo with all the things you would like to see done for the next release, tag me, and I'll see what I can do!
This particular documentation PR does not affect the Julia bindings though, you got the right order of fields in here so there is nothing to change.

Ok, interesting, I do see some reference that spglib is loaded dynamically here and I do not see any reference of build scripts in spglib_jll.jl. I guess these are build/downloaded on-demand from Yggdrasil. That is indeed rather tricky because the library version dependence is actually in Spglib.jl instead of spglib_jll. Although browsing through that it seems that they don't recommend doing that.

I have not updated Spglib.jl to v2.3.0 since you changed some conventions

There shouldn't have been any breaking changes there, they should still be compatible. Most of the stuff is on the build system, fixes to some segmentation faults, control over the output messages and such. Let me know if you find any issues.