Nice functionality!

This might be more of a "discussion" but I wonder about supported or intended usage when changing source during say a CLI session, when using multiple S3 sources (see #4337) in a session or query?

I guess one could say that I wonder about usage when "reading and setting environment variables after the extension has been loaded". Will it be possible to change environment variables and point to another httpfs S3 source during a session or is the config read and set only once at "load time" when the extension is loaded at "session startup"?

Zooming out a little, some of the extensions (for example httpfs, postgres with the libpq connection string, maybe more?) require credentials to be provided, and this is the first one to support credentials to be picked up from environment variables.

Am I right in assuming that credentials are generally expected to "remain the same" throughout a session, even while #4608 allows fetching data from different object storage servers or buckets with different credentials in a single query (when explicitly given in the S3 uri)?

Related to #4460 (comment) - would "reloading" the environment variables to point to another source cause earlier defined views using "previous credentials" to stop working?

Does it support the AWS_PROFILE environment variable?

@alexey-milovidov

Does it support the AWS_PROFILE environment variable?

Not yet. This is a pretty small improvement, intentionally.

I guess one could say that I wonder about usage when "reading and setting environment variables after the extension has been loaded". Will it be possible to change environment variables and point to another httpfs S3 source during a session or is the config read and set only once at "load time" when the extension is loaded at "session startup"?

This patch just sets s3_access_key_id and etc from env vars on cli starts. So you can (re)set during a session with

SET s3_access_key_id=<your_key>
...

as docs says.

But it is not re-reading env vars, so it might be good for us to have a function like reload_aws_envvars();

would "reloading" the environment variables to point to another source cause earlier defined views using "previous credentials" to stop working?

Yes. In my opinion, if it is view, it should stop.
If we would like to configure multi-account bucket access, a profile must be paired with a bucket.
In those cases, the combination qualified url and getenv might be nice.
Anyway, this PR is far simpler.

Thanks for the fixes! Looks good - could you just merge with the feature branch so the CI can re-run? We had some issues with the feature branch in the last few days.

I wrongly merged master branch (not feature). Resetted

Thanks! LGTM