Extensions: Rework CMake using POSTBUILD steps #9800
Closed
+99
−25
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR ties in to a rework to the extension pipeline that touches both duckdb-wasm extensions and regular extensions.
Idea is that for a bit both the 'old' extension format and the 'new' extension format can coexist, both to allow smooth and piece-by-piece transition of CI and to allows extensions to be packaged and build both for newer and older duckdb version, given there will be no breaking changes.
Currently extensions once built live at
build/release/extension/${extension_name}/${extension_name}.duckdb_extension
, and require signature to be appended in post-processing.Proposed standard is have the old version live at the same place, with new one being preset ad
build/release/extension/${extension_name}.duckdb-extension
, and those will have a standard to package some additional metadata (extension name, platform, DuckDB version, and to be soon standardised distribution), plus a default signature that will need to be overridden during signing step.Note that metadata-enabled extensions are compatible and can be rolled in and tested alongside with regular extensions, given they will still have signature in the same spot.
Binary format is to be specified yet, feedback is welcome on what fields could be of use, and what checks we expect to be able to make from those fields.
Current proposal is:
This align with currently imposed restrictions on length of these fields, and keeping this simple to read and write would allow for tooling to detect what format are those files.
End goal, tentatively to be rolled on for v0.10.0, is allowing checks both on INSTALL and on LOAD, that would prevent problems where INSTALL of a local file is successful (and currently unchecked) and ends up in a corrupt
~/.duckdb/extension
folder. Or on LOAD pointing to an extension for a wrong platform might end up in segfaults.Yet to be added: have the default location for these metadata-enabled extensions be compatible with setting
default_extension_folder
, bringing in more simplification in tests and usability of dynamically loaded extensions for development.