endpoint_url global option from AWS config is not working #57
Description
Hello! Thank you for the nice S3 utility.
I found a bug with AWS config and endpoint_url option. If you set endpoint_url with DNS records and port, for example:
~/.aws/config
[default]
region = us-east-1
output = json
endpoint_url = http://localhost:9000
s3 =
signature_version = s3v4Then set credentials in correspond file:
~/.aws/credentials
[default]
aws_access_key_id = minioadmin
aws_secret_access_key = minioadminYou'll see buckets, but cannot open them and get an error:
In error.log I see:
2025-03-13 12:44:34.360836 +03:00 Failed to load objects: ServiceError(ServiceError { source: Unhandled(Unhandled { source: XmlDecodeError { kind: Custom("encountered invalid XML root: expected ListBucketResult but got StartEl { name: Name { prefix: \"\", local: \"ListAllMyBucketsResult\" }, attributes: [Attr { name: Name { prefix: \"\", local: \"xmlns\" }, value: \"http://s3.amazonaws.com/doc/2006-03-01/\" }], closed: false, depth: 0 }. This is likely a bug in the SDK.") }, meta: ErrorMetadata { code: None, message: None, extras: None } }), raw: Response { status: StatusCode(200), headers: Headers { headers: {"accept-ranges": HeaderValue { _private: H0("bytes") }, "content-length": HeaderValue { _private: H0("465") }, "content-type": HeaderValue { _private: H0("application/xml") }, "server": HeaderValue { _private: H0("MinIO") }, "strict-transport-security": HeaderValue { _private: H0("max-age=31536000; includeSubDomains") }, "vary": HeaderValue { _private: H0("Origin") }, "vary": HeaderValue { _private: H0("Accept-Encoding") }, "x-amz-id-2": HeaderValue { _private: H0("dd9025bab4ad464b049177c95eb6ebf374d3b3fd1af9251148b658df7ac2e3e8") }, "x-amz-request-id": HeaderValue { _private: H0("182C53996FBE08C5") }, "x-content-type-options": HeaderValue { _private: H0("nosniff") }, "x-ratelimit-limit": HeaderValue { _private: H0("1045") }, "x-ratelimit-remaining": HeaderValue { _private: H0("1045") }, "x-xss-protection": HeaderValue { _private: H0("1; mode=block") }, "date": HeaderValue { _private: H0("Thu, 13 Mar 2025 09:44:34 GMT") }} }, body: SdkBody { inner: Once(Some(b"<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<ListAllMyBucketsResult xmlns=\"http://s3.amazonaws.com/doc/2006-03-01/\"><Owner><ID>02d6176db174dc93cb1b899f7c6078f08654445fe8cf1b6ce98d8855f66bdbf4</ID><DisplayName>minio</DisplayName></Owner><Buckets><Bucket><Name>test-bucket</Name><CreationDate>2025-03-12T11:33:23.176Z</CreationDate></Bucket><Bucket><Name>test-bucket-2</Name><CreationDate>2025-03-12T11:35:32.468Z</CreationDate></Bucket></Buckets></ListAllMyBucketsResult>")), retryable: true }, extensions: Extensions { extensions_02x: Extensions, extensions_1x: Extensions } } })Looks like endpoint_url is ignored and stu is using public AWS instance: s3.amazonaws.com.
If I run stu with --endpoint-url argument, everything works as expected (I see buckets and bucket's data):
stu --endpoint-url http://localhost:9000
How to reproduce:
I've created docker-compose.yaml file to setup MinIO (we are using private MinIO in our project), create buckets and upload test file:
---
version: "3.8"
services:
minio:
image: minio/minio
container_name: minio
restart: unless-stopped
ports:
- "9000:9000"
- "9001:9001"
environment:
MINIO_ROOT_USER: minioadmin
MINIO_ROOT_PASSWORD: minioadmin
MINIO_DEFAULT_BUCKETS: test-bucket
command: server /data --console-address ":9001"
volumes:
- minio_data:/data
create-bucket:
image: minio/mc
depends_on:
- minio
entrypoint: >
/bin/sh -c "
sleep 2;
/usr/bin/mc alias set local http://minio:9000 minioadmin minioadmin;
/usr/bin/mc mb local/test-bucket 2>/dev/null;
/usr/bin/mc mb local/test-bucket-2 2>/dev/null;
echo 'Test file' > /tmp/test.txt;
/usr/bin/mc cp /tmp/test.txt local/test-bucket/test.txt;
/usr/bin/mc cp /tmp/test.txt local/test-bucket-2/test.txt;
exit 0;
"
volumes:
minio_data:Run:
docker-compose up
Create aws directory with necessary config files:
mkdir -p ~/.aws
cat << EOF > ~/.aws/config
[default]
region = us-east-1
output = json
endpoint_url = http://localhost:9000
s3 =
signature_version = s3v4
EOF
cat << EOF > ~/.aws/credentials
[default]
aws_access_key_id = minioadmin
aws_secret_access_key = minioadmin
EOFRun stu without arguments and with --endpoint-url at the next run.
Workaround
I've added alias in my profile:
alias stu='stu --endpoint-url http://localhost:9000'