BREAKING CHANGE

• all: Migrated to OpenTelemetry by @pitabwire

NOTE: All modules have migrated to AWSv2; v1 support is still included for docstore/awsdynamodb, but will be removed in the next release.

What's Changed

• blob: use bigger buffer to make writes faster by @vangent in #3599
• blob/memblob: Add an option to disable MD5 hashes by @vangent in #3598
• docstore/memdocstore: nested query by @eqinox76 in #3508
• blob/azblob: for default URL, get default accountName/protocol from connection string env by @vangent in #3593
• docstore/awsdynamodb: support the AWS v2 SDK by @Maldris and @vangent in #3602

New Contributors

• @eqinox76 made their first contribution in #3508
• @Maldris made their first contribution in #3602

Full Changelog: v0.42.0...v0.43.0

Breaking Changes

AWS SDKv1 support dropped except for docstore.

What's Changed

• aws: Drop support for AWS SDK v1 in everything except docstore by @vangent in #3534
• pubsub: Ensure batch flushes on shutdown even if MinBatchSize isn't met by @vangent in #3543
• blob: introduce conditional write option for storage by @ocampeau in #3533

New Contributors

• @ocampeau made their first contribution in #3533
• @pitabwire made their first contribution in #3546

Full Changelog: v0.41.0...v0.42.0

What's Changed

Blob

• blob/s3blob: Allow using s3ForcePathStyle as a synonym of use_path_style, for backwards compatibility with V1 by @vangent in #3505
• blob/s3blob, gcsblob: Add support for anonymous query param for gcsblob and all AWSv2 providers by @vangent in #3513

Docstore

• docstore/aws: Support atomic writes in the docstore by @sandeepvinayak in #3500
• docstore/gcp: Support for atomic writes for gcp by @sandeepvinayak in #3523
• openurl: fix URL scheme handling case-insensitive by @Gofastasf in #3521
• blob/gcsblob: fix error checking for NotFound errors by @BrennaEpp in #3529

New Contributors

• @robertsilen made their first contribution in #3509
• @knbr13 made their first contribution in #3516
• @sandeepvinayak made their first contribution in #3500
• @Gofastasf made their first contribution in #3521
• @BrennaEpp made their first contribution in #3529

Full Changelog: v0.40.0...v0.41.0

What's Changed

• blob/all: disable Upload optimization when WriterOptions.ContentMD5 is set by @vangent in #3478
• blob/s3blob: custom endpoints with s3 and aws sdk v2 by @caarlos0 in #3473
• blob/all: Don't require SetIOFSCallback be called to use io/fs.FS functions by @vangent in #3479
• blob/s3blob: fix data race by @arjunnair1997 in #3480
• blob/azblob: Support AZURE_STORAGEBLOB_CONNECTIONSTRING as an alternative for AZURE_STORAGE_CONNECTION_STRING by @vangent in #3483
• s3blob/blob: support additional endpoint query parameters by @stanhu in #3486
• blob/gcsblob: Allow providing options for storage.NewClient by @vangent in #3493
• aws: Add support for non-camelcased version of two URL parameters by @vangent in #3494
• Add disableHTTPS and usePathStyle s3v2.Options as query param by @khrm in #3491
• aws: add rate_limiter_capacity to configure client-side rate limits by @stanhu in #3497

New Contributors

• @caarlos0 made their first contribution in #3473
• @arjunnair1997 made their first contribution in #3480
• @bdon made their first contribution in #3481
• @khrm made their first contribution in #3491

Full Changelog: v0.39.0...v0.40.0

BREAKING CHANGE (AWS only, V1 vs V2 SDK)

Context: AWS has announced maintenance mode for the Go V1 SDK.

Go CDK has changed the default SDK for URLs across all modules except docstore/awsdynamodb to be V2 (previously you needed to add awssdk=v2 to the URL to get V2). Most URLs should continue to work, but in some cases you may need to add awssdk=v1 to force V1 explicitly.

Also, concrete type constructors (e.g., OpenBucket) for V1 (again, except docstore/awsdynamodb) have been marked deprecated; please migrate to using the V2 versions (e.g., OpenBucketV2).

Our tentative plan is to remove support for V1 in early 2025; please file a bug if you have concerns about that.

What's Changed

• pubsub: Make batch request results independent by @mitsos1os in #3457
• docstore/all: Add support for boolean filter by @ybourgery in #3464
• aws/all: Mark V1 constructors deprecated. by @vangent in #3466
• aws/all: Change the default for AWS URLs from V1 to V2. by @vangent in #3465
• all: update to go version 1.23 by @vangent in #3467

New Contributors

• @mitsos1os made their first contribution in #3457
• @dependabot made their first contribution in #3448

Full Changelog: v0.38.0...v0.39.0

blob

• all: Fix panics if reader recreation fails after Seek by @vangent in #3425
• all: Convert errors in Open() into appropriate fs errors by @milescrabill in #3443
• s3blob: Fix Copy to work with keys that need escaping by @vangent in #3403
• azureblob: Do not panic if Content-Length and Content-Range are missing by @chancez in #3445
• fileblob: Allow customization of the FileMode by @vangent in #3426

pubsub

• awssnssqs: Add support for setting FIFO message metadata by @bartventer in #3435
• kafkapubsub: Configuring key_name when OpenTopicURL by @ssetin in #3404
• rabbitpubsub: Add query string set the qos prefetch count by @peczenyj in #3431
• rabbitpubsub: Add query string to set the routing key from metadata by @peczenyj in #3433
• rabbitpubsub: Wrap pubsub rabbitmq errors by @peczenyj in #3437

docstore

• all: Fix offset handling and extend test coverage by @bartventer in #3409
• awsdynamodb: Ensure Next returns EOF when no more items by @bartventer in #3406
• mongodocstore: Update Mongo dialer when MONGO_SERVER_URL rotates by @concaf in #3429

New Contributors

• @ssetin made their first contribution in #3404
• @concaf made their first contribution in #3429
• @peczenyj made their first contribution in #3431
• @chancez made their first contribution in #3445
• @milescrabill made their first contribution in #3443
• @samlaf made their first contribution in #3450

Full Changelog: v0.37.0...v0.38.0

• blob

  • s3blob: Support S3 server side encryption headers for Write and Copy.
  • gcsblob: Ensure driver sets Content-Type auto-detection properly.

• pubsub

  • azpubsub: Ensure 401s are not retryable.
  • azpubsub: Allow configuring max_recv_batch_size in via URL.
  • mongodocstore: Fix error when calling the update function with no 'set' operations.

• docstore

  • all: Add Offset method, useful for pagination

blob

• all: Allow disabling of ContentType auto-detection during writes.

pubsub

• azuresb: Added a new auth method to support Service principal/kubelet identity/Workload identity auth methods.

docstore

• all: Add in/not-in operators for Query.
• gcpfirestore: Added a missing resource header when running query.

mysql

• Pass TLS config directly to MySQL's config

BREAKING CHANGES

This release updates the dependency on AWS/S3, which included some breaking changes; see #3342 for details.

blob

• gcsblob: Fix unauthenticated requests.

runtimevar

• constantvar: Add support for reading the constant variable value from the environment.

docstore

• gcpfilestore: Add support for non-default databases.

• blob

  • fileblob: Added an option to avoid using os.TempDir for temp files.
  • gcsblob: Don't hard-fail if no DefaultCredentials are found.

• aws/rds: Switch AWS RDS CA bundle to the officially documented URL