Skip to content

Conversation

Flogex
Copy link
Contributor

@Flogex Flogex commented Apr 12, 2024

Follow-up to 0ecc682:
CMAKE_SOURCE_DIR is used to specify the location of the scripts directory. However, this variable does not point to the duckdb top-level directory if DuckDB is built from within another CMake project, e.g. for extensions. I replaced the variable with CMAKE_CURRENT_FUNCTION_LIST_DIRwhich gives the directory of the CMakeLists.txt where the build_loadable_extension_directory function is defined—in other words, the top-level directory.

Similarly, when looking for the duckdb_platform_out file, CMAKE_BINARY_DIR is the wrong path when DuckDB is not the top-level CMake project. I replaced it with DuckDB_BINARY_DIR where "DuckDB" is the PROJECT-NAME.

@duckdb-draftbot duckdb-draftbot marked this pull request as draft April 12, 2024 16:32
@Flogex Flogex marked this pull request as ready for review April 12, 2024 16:34
@carlopi carlopi self-requested a review April 12, 2024 17:23
Copy link
Contributor

@carlopi carlopi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks a lot for filing this correcting the limitations introduced.

Could you check whether the proposed change works the same? If so, I would prefer the more generic PROJECT_BINARY_DIR given it's used already elsewhere in the CMake logic.

One additional request, independent from the PR: would it be possible to submit a demo CMake that showcases the setup where this was a problem? Then that can be used as part of CI to check modifying the CMake logic is not impacting your setup.
I can help move that into a CI check, but even just a CMake with a add_subproject or so would be cool to have, possibly in an examples folder.

Thanks!

@@ -820,7 +820,7 @@ function(build_loadable_extension_directory NAME OUTPUT_DIRECTORY EXTENSION_VERS
TARGET ${TARGET_NAME}
POST_BUILD
COMMAND
${CMAKE_COMMAND} -DEXTENSION=$<TARGET_FILE:${TARGET_NAME}> -DPLATFORM_FILE=${CMAKE_BINARY_DIR}/duckdb_platform_out -DDUCKDB_VERSION="${DUCKDB_NORMALIZED_VERSION}" -DEXTENSION_VERSION="${EXTENSION_VERSION}" -DNULL_FILE=${CMAKE_SOURCE_DIR}/scripts/null.txt -P ${CMAKE_SOURCE_DIR}/scripts/append_metadata.cmake
${CMAKE_COMMAND} -DEXTENSION=$<TARGET_FILE:${TARGET_NAME}> -DPLATFORM_FILE=${DuckDB_BINARY_DIR}/duckdb_platform_out -DDUCKDB_VERSION="${DUCKDB_NORMALIZED_VERSION}" -DEXTENSION_VERSION="${EXTENSION_VERSION}" -DNULL_FILE=${CMAKE_CURRENT_FUNCTION_LIST_DIR}/scripts/null.txt -P ${CMAKE_CURRENT_FUNCTION_LIST_DIR}/scripts/append_metadata.cmake
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
${CMAKE_COMMAND} -DEXTENSION=$<TARGET_FILE:${TARGET_NAME}> -DPLATFORM_FILE=${DuckDB_BINARY_DIR}/duckdb_platform_out -DDUCKDB_VERSION="${DUCKDB_NORMALIZED_VERSION}" -DEXTENSION_VERSION="${EXTENSION_VERSION}" -DNULL_FILE=${CMAKE_CURRENT_FUNCTION_LIST_DIR}/scripts/null.txt -P ${CMAKE_CURRENT_FUNCTION_LIST_DIR}/scripts/append_metadata.cmake
${CMAKE_COMMAND} -DEXTENSION=$<TARGET_FILE:${TARGET_NAME}> -DPLATFORM_FILE=${PROJECT_BINARY_DIR}/duckdb_platform_out -DDUCKDB_VERSION="${DUCKDB_NORMALIZED_VERSION}" -DEXTENSION_VERSION="${EXTENSION_VERSION}" -DNULL_FILE=${CMAKE_CURRENT_FUNCTION_LIST_DIR}/scripts/null.txt -P ${CMAKE_CURRENT_FUNCTION_LIST_DIR}/scripts/append_metadata.cmake

Would PROJECT_BINARY_DIR work? That is currently used for determining the location of duckdb_platform_binary in:

set_target_properties(duckdb_platform_binary PROPERTIES RUNTIME_OUTPUT_DIRECTORY
                                         ${PROJECT_BINARY_DIR})

and implicitly for the location of duckdb_platform_out.

I think it should work the same, and avoid the naming that looks more confusing to me.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PROJECT_BINARY_DIR will not work when the build_loadable_extension function is called from another CMake project. For example, for the Parquet extension:

build_loadable_extension(parquet ${PARAMETERS} ${PARQUET_EXTENSION_FILES})

The PROJECT_BINARY_DIR inside the function would be build/release/extension/parquet, not build/release.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for double checking.

@Flogex
Copy link
Contributor Author

Flogex commented Apr 12, 2024

One additional request, independent from the PR: would it be possible to submit a demo CMake that showcases the setup where this was a problem? Then that can be used as part of CI to check modifying the CMake logic is not impacting your setup.
I can help move that into a CI check, but even just a CMake with a add_subproject or so would be cool to have, possibly in an examples folder.

Hey @carlopi. I've built a repro using the extension template. But instead of using the build_loadable_extension/build_static_extension functions, duckdb is added as subdirectory and the duckdb targets linked against the extension.
Flogex/duckdb-extension-template@7aa87de

Now when running make release, the CMAKE_BINARY_DIR is extension-template/build/Release/ and the CMAKE_SOURCE_DIR is extension-template/ (as set in the release make target).
CMake will fail with:

CMake Error: Not a file: extension-template/scripts/append_metadata.cmake

Let me know if this is helpful.

@carlopi
Copy link
Contributor

carlopi commented Apr 12, 2024

This is good to go for me.
Thanks @Flogex for sharing also the demo-setup, I will add something inspired by that in the CI, so to flag those problems earlier.
I will also a setup similar to original extension-template. That was working in this case (since I tested it), but it's not a given that any change has to be compatible.

@Mytherin Mytherin merged commit e11b856 into duckdb:main Apr 12, 2024
@Mytherin
Copy link
Collaborator

Thanks!

github-actions bot pushed a commit to duckdb/duckdb-r that referenced this pull request Apr 13, 2024
Merge pull request duckdb/duckdb#11630 from pdet/gz_buffers
Merge pull request duckdb/duckdb#11635 from motherduckdb/flo/fix-scripts-path-cmake
@Flogex Flogex deleted the flo/fix-scripts-path-cmake branch July 3, 2024 12:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants