Thanks for this! Do you know if MinIO working for AWS and GCP? In theory AWS SDK should work for both GCP - https://cloud.google.com/storage/docs/samples/storage-s3-sdk-list-object and DO spaces (https://docs.digitalocean.com/products/spaces/how-to/use-aws-sdks/). To me it feels like we should consolidate on either AWS SDK or MinIO (whatever is more compatible) and remove the two other libraries at some point.

No reason not to to merge this one right now on my side. @barrydeen, ok to merge on your side? (I want to release Haven 1.0.5 today if possible :D)

Just looking at the code it looks good but it could potentially break older implementations, @aaccioly if you test it and nothing breaks feel free to merge it

Will do. Need to setup AWS and GCP

Thanks for this! Do you know if MinIO working for AWS and GCP? In theory AWS SDK should work for both GCP - https://cloud.google.com/storage/docs/samples/storage-s3-sdk-list-object and DO spaces (https://docs.digitalocean.com/products/spaces/how-to/use-aws-sdks/). To me it feels like we should consolidate on either AWS SDK or MinIO (whatever is more compatible) and remove the two other libraries at some point.

No reason not to to merge this one right now on my side. @barrydeen, ok to merge on your side? (I want to release Haven 1.0.5 today if possible :D)

Looking at the minio.New() function:

// New - instantiate minio client with options
func New(endpoint string, opts *Options) (*Client, error) {
	if opts == nil {
		return nil, errors.New("no options provided")
	}
	clnt, err := privateNew(endpoint, opts)
	if err != nil {
		return nil, err
	}
	if s3utils.IsAmazonEndpoint(*clnt.endpointURL) {
		// If Amazon S3 set to signature v4.
		clnt.overrideSignerType = credentials.SignatureV4
		// Amazon S3 endpoints are resolved into dual-stack endpoints by default
		// for backwards compatibility.
		clnt.s3DualstackEnabled = true
	}

	return clnt, nil
}

It does consider the endpoint can be a AWS S3 endpoints so it seems like minio should be able to replace the aws package that is used in this project. I should have done the work of testing on AWS/GCP myself before submitting this PR rather than putting this work on a maintainer

@aaccioly @barrydeen I can adjust this PR to consider the fact that someone has already configured their relay to use Digital Ocean Spaces specific environment variables. So this would mean supporting both the SPACES_... environment variables and the S3_... environment variables and later this can be deprecated in a reasonable way. What do you think?

Highly doubt anyone implemented digital ocean in the past few days since it was merged, I think we can just yolo merge it

@charlieroth, I second Barry on this, everything before we cut a new release to me is basically considered unstable. As long as AWS and GCP are working and the users won't be surprised by configuration changes I'm happy with it.

GCP backup working:

2025/02/08 16:22:32 📦 zipping up the database
2025/02/08 16:22:58 📦 database zipped up!
2025/02/08 16:22:58 ✅ Successfully uploaded "db.zip" to "haven_backup"

Now testing GCP with Min.io

AWS and Generic S3 backups through MinIO are in order. I made some additional changes in 88b1d6d to unify the shared logic for the upload

AWS

Generic

GCP Through Min.io also working:

2025/02/08 16:40:34 📢 subscribing to inbox
2025/02/08 16:42:34 📦 zipping up the database
2025/02/08 16:42:59 📦 database zipped up!
2025/02/08 16:43:00 ✅ Min.io Successfully uploaded "db.zip" to "haven_backup"
2025/02/08 16:44:34 📦 zipping up the database
2025/02/08 16:44:57 📦 database zipped up!
2025/02/08 16:44:58 ✅ Min.io Successfully uploaded "db.zip" to "haven_backup"

However, Min.io authentication to GCP is limited to HMAC Key / Secrets, so I feel more comfortable marking the old GCP method as deprecated for now and advising users to create HMAC Key / Secrets, but keep the old API around for now.

I'll adjust the readme accordingly soon and then I'll merge it.

Many thanks for everything @charlieroth.

Hi @charlieroth , can you please give me access to edit this PR? https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/allowing-changes-to-a-pull-request-branch-created-from-a-fork

Hi @charlieroth , can you please give me access to edit this PR? https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/allowing-changes-to-a-pull-request-branch-created-from-a-fork

Invite sent

Done with everything but README. No more AWS SDK. GCP backup is working with both the deprecated gcp option as well as s3. @charlieroth, do you mind testing one last time with both BACKUP_PROVIDER="s3" and BACKUP_PROVIDER="aws"? (I swear that this is the change code change. Sorry for the journey today :D).

Done with everything but README. No more AWS SDK. GCP backup is working with both the deprecated gcp option as well as s3. @charlieroth, do you mind testing one last time with both BACKUP_PROVIDER="s3" and BACKUP_PROVIDER="aws"? (I swear that this is the change code change. Sorry for the journey today :D).

Yes on it

Done with everything but README. No more AWS SDK. GCP backup is working with both the deprecated gcp option as well as s3. @charlieroth, do you mind testing one last time with both BACKUP_PROVIDER="s3" and BACKUP_PROVIDER="aws"? (I swear that this is the change code change. Sorry for the journey today :D).

Works great! 🚀

Digital Ocean Spaces

AWS