Skip to content

How to build a static library with extensions #2176

@IgKh

Description

@IgKh

What does happen?
The released official source amalgamation (libduckdb-src.zip) doesn't include the source to any of the extensions. For that matter, also the pre-built shared library packages do not include any of them.

To the best of my observation, only the Python / R packages and the CLI tool are built with extensions, but those aren't as usable from a C++ program.

What should happen?
My goal is to create a static build of DuckDB C++ library which also includes extensions (in particular, Parquet and httpfs). Ideally, as a single .a file plus minimal header files. A shared object would be acceptable as well. However there doesn't seem to be a straightforward way to do so.

To Reproduce
I tried building from the full source tree, by configuring CMake with -DBUILD_PARQUET_EXTENSION=1 -DBUILD_HTTPFS_EXTENSION=1. This does build those extensions, but the result of trying to create a re-distributable with a make install is that:

  • The static libraries created for the extensions are not installed, neither are the headers that are needed to be able to call duckdb::DuckDB::LoadExtension<>() for them.

  • There is a separate static library for each extension, which has a somewhat problematic name like libparquet_extension.a (it would have been better if there was a "duckdb" somewhere in the name)

  • Archive files for the DuckDB mangled versions of vendored third party dependencies are also installed, with names that may collide with existing files (e.g. libfmt.a, which is not compatible)

I also noticed that there is a script for generating a Parquet source amalgamation. Did that in conjunction with generating an extended core amalgamation. Together with the extension/include/parquet-extension.hpp header file, it does make it possible to easily build a combined static library. However it looks like that wasn't the purpose of it, and it does not cover other extensions.

Is this use case intended to be supported, and how to best achieve it? Thank you!

Environment (please complete the following information):

  • OS: [e.g. iOS] Linux
  • DuckDB Version [e.g. 22] 0.2.8 + master

Before submitting

  • Have you tried the steps to reproduce? Do they include all relevant data and configuration? Does the issue you report still appear there?
  • Have you tried this on the latest master branch? In case you cannot compile, you may find some binaries here: https://github.com/duckdb/duckdb/releases/tag/master-builds

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions