Skip to content

Conversation

carlopi
Copy link
Contributor

@carlopi carlopi commented Feb 28, 2025

Currently there are 2 settings that need to be overridden to change the default (remote) extension repository:

SET custom_extension_repository = '...'
SET autoinstall_extension_repository = '...'

Having those point to different endpoint I think it's an historical artifact, and mostly used to test autoinstall and autoloading in our unittester.

Documentation do not mention autoinstall_extension_repository at all, and I think users would assume setting custom_extension_repository should download from that one both these cases:

INSTALL delta;
--- this uses custom_extension_repository
FROM 'https://some.org/file.csv';
--- this autoinstall and autoload httpfs from autoinstall_extension_repository

The proposed solution, that I think should keep flexibility when needed (e.g. testing framework) while keeping most current workflow working is using custom_extension_repository as default when autoinstall_extension_repository is not set.

Added a test, that in current DuckDB fails while INSTALLing the extension from default repo (even though custom_extension_repository had been set):

FROM read_csv('https://some.org/file.csv');
Actual result:
================================================================================
Extension Autoloading Error: An error occurred while trying to automatically install the required extension 'httpfs':
Failed to download extension "httpfs" at URL "http://extensions.duckdb.org/c837f1082a/osx_arm64/httpfs.duckdb_extension.gz" (HTTP 403)
Extension "httpfs" is an existing extension.

For more info, visit https://duckdb.org/docs/extensions/troubleshooting/?version=c837f1082a&platform=osx_arm64&extension=httpfs

Fixes also a minor mistake within an #ifdef.

@carlopi carlopi force-pushed the unify_custom_autoinstall_extension_repo branch from 923d307 to f1cafe7 Compare February 28, 2025 09:24
@duckdb-draftbot duckdb-draftbot marked this pull request as draft February 28, 2025 09:24
@carlopi carlopi marked this pull request as ready for review February 28, 2025 09:26
@carlopi carlopi requested a review from samansmink February 28, 2025 09:26
@carlopi
Copy link
Contributor Author

carlopi commented Feb 28, 2025

Unittester could also be updated using this, but that might (and will) break tests and require some more thoughts.

@Mytherin
Copy link
Collaborator

Mytherin commented Mar 6, 2025

Thanks for the PR! Makes sense to me.

@Mytherin Mytherin merged commit 34b9403 into duckdb:v1.2-histrionicus Mar 6, 2025
50 checks passed
@carlopi carlopi deleted the unify_custom_autoinstall_extension_repo branch March 6, 2025 12:43
krlmlr added a commit to duckdb/duckdb-r that referenced this pull request Apr 8, 2025
custom_extension_repository to also be the default autoinstall_extension_repository (duckdb/duckdb#16459)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants