Changelog

Refer to the 2.11 Upgrade Guide for backwards compatibility notes with 2.10.x.

Go Version

• 1.24.7 (#7273)

Dependencies

• golang.org/x/time v0.13.0 (#7273)
• golang.org/x/sys v0.36.0 (#7273)

Improved

JetStream

• Offline assets support (#7158)
  • Server version 2.12 will introduce new features that would otherwise break a 2.11 server after a downgrade. The server now reports the streams/consumers as offline and unsupported, keeping the data safe, but allowing to either delete the asset or upgrade back to the supported version without changes to the data itself.
• The raftz endpoint now reports the cluster traffic account (#7186)
• The stream info and consumer info endpoints now return leader_since (#7189)
• The stream info and consumer info endpoints now return system_account and traffic_account (#7193)
• The jsz monitoring endpoint now returns system_account and traffic_account (#7193)

Fixed

General

• Fix a panic that could happen at startup if building from source using non-Git version control (#7178)
• Fix an issue where issuing an account JWT update with a connection limit could cause older clients to be disconnected instead of newer ones (#7181, #7185)
• Route connections with invalid credentials will no longer rapidly reconnect (#7200)
• Allow a default_sentinel JWT from a scoped signing key instead of requiring it to solely be a bearer token for auth callout (#7217)
• Subject interest would not always be propagated for leaf nodes when daisy chaining imports/exports (#7255)
• Subject interest would sometimes be lost if the leaf node is a spoke (#7259)
• Lowering the max connections limit should no longer result in streams losing interest (#7258)

JetStream

• The Nats-TTL header will now be correct if the subject delete marker TTL overwrites it (#7177)
• In operator mode, the cluster_traffic state for an account is now restored correctly when enabling JetStream at startup (#7191)
• A potential data race during a consumer create or update when reading its paused state has been fixed (#7201)
• A race condition that could allow creating a consumer with more replicas than the stream has been fixed (#7202)
• A race condition that could allow creating the same stream with different configurations has been fixed (#7210, #7212)
• Raft will now correctly reject delayed entries from an old leader when catching up in the meantime (#7209, #7239)
• Raft will now also limit the amount of cached in-memory entries as the leader, avoiding excessive memory usage (#7233)
• A potential race condition delaying shutdown if a stream/consumer monitor goroutine was not started (#7211)
• A benign underflow when using an infinite (-1) MaxDeliver for consumers (#7216)
• A potential panic to send a leader elected advisory when shutting down before completing startup (#7246)
• Stopping a stream should no longer wait indefinitely if the consumer monitor goroutine wasn’t stopped (#7249)
• Speed up stream mirroring and sourcing after a leaf node reconnects in complex topologies (#7265)
• Updating a stream with an empty placement will no longer incorrectly trigger a stream move (#7222)

Tests

• Tests have been improved (#7172, #7230, #7213, #7248)

Complete Changes

v2.11.8...v2.11.9

Changelog

This release also contains all changes up to and including v2.11.9-RC.3.

Go Version

• 1.25.1 (#7273)

Added

General

• Exponential backoff on route/gateway reconnection attempts (#7042, #7048)
• Added allow_insecure_cipher_suites configuration option to tls block, as insecure ciphers are now disabled by default (#7144)
• No responders errors from the server now include the original subject in the Nats-Subject header (#5250)
• Added exact_match field to the filtering of various monitoring endpoints for server name, host or cluster matching (#7260)

JetStream

• Atomic batch publishes (#6966, #7057, #7060, #7085, #7090, #7091, #7119, #7152, #7175, #7274)
  • Stage-and-commit style batches for JetStream message publishes with normal consistency checks
  • ADR: https://github.com/nats-io/nats-architecture-and-design/blob/main/adr/ADR-50.md
• Counter streams (#6973, #6988, #7081, #7118)
  • Counters are incremented or decremented by publishing to the stream with Nats-Incr headers, with full aggregate consistency over sources and mirrors
  • ADR: https://github.com/nats-io/nats-architecture-and-design/blob/main/adr/ADR-49.md
• Prioritised mode for consumer priority groups (#7113)
  • Allows for low-latency switching between clients based on the priority set
  • ADR: https://github.com/nats-io/nats-architecture-and-design/blob/main/adr/ADR-42.md
• Added no_hdr flag to message get and direct get API endpoints (#6994)
  • Messages returned with this flag set will not contain headers, allowing fast access to e.g. counter values
  • ADR: https://github.com/nats-io/nats-architecture-and-design/blob/main/adr/ADR-52.md
• Support for trusted proxies (#7153)
  • Allows enforcing that connections arrive via a NATS protocol-aware proxy
  • ADR: https://github.com/nats-io/nats-architecture-and-design/blob/main/adr/ADR-55.md
• Added the ability to promote mirrors to normal streams by removing the mirror configuration (#7171)
  • This allows a mirror to take over operation if the original stream is lost, for example in a disaster recovery scenario
  • This operation cannot be undone and also requires configuring the stream subjects to continue operation
• Added delayed message scheduling (#7170, #7245)
  • Implements single schedule messages using the Nats-Schedule-TTL message header
  • ADR: https://github.com/nats-io/nats-architecture-and-design/blob/main/adr/ADR-51.md
• Added support for offline assets (#7158)
  • Streams and consumers from a higher API version or using unsupported features (i.e. after a downgrade) will be marked as offline to prevent corruption
  • ADR: https://github.com/nats-io/nats-architecture-and-design/blob/main/adr/ADR-44.md
• Added partition(n) for deterministic hash-based partitioning and random(n) for non-deterministic random partitioning to subject transforms (#6950)
• Added the ability to specify the required API level in JetStream API requests with the Nats-Required-Api-Level request header (#7157)
• Add server_metadata block to server config, allowing arbitrary key-pairs to be set on a per-server basis, similar to stream/consumer metadata (#6935)
• Support for empty votes, which improves how the Raft election logic handles nodes that have lost their stable storage and attempt to rejoin the cluster regardless (#7038)
• Added "leader_since" to the stream and consumer info responses (#7189)

Leafnodes

• Added the ability to isolate leafnode subject interest propagation with the hub-side isolate_leafnode_interest configuration option, suppressing east-west traffic and reducing subscription overheads when there are many leafnodes (#7238, #7243)
• Added disabled option for leafnode remotes, with support for reloading from the config at runtime (#7054)

Changed

JetStream

• The JetStream API level has been increased to 2 (#6969)
• The API surface is now in strict mode by default, erroring if unknown fields are provided in the request body (#7049)
• Replicated streams will now default to async flush as long as sync: always is not configured (#7018, #7163)
• The default value for max_buffered_msgs has been increased by 10x to 100,000 messages (#6633)

Improved

General

• When logging closed connections, the remote is now logged if known, making it easier to identify where route/gateway/leafnode connections have dropped (#7077)
• Client-specific log lines now include the account and user if known (#7079)

JetStream

• Raft nodes will no longer respond success to catchup messages and will ignore responses if not leader, fixing some potential stream desync scenarios (#6944)
• Replicated streams can now be created even if some of the replica nodes are offline (#7075)
• The storage type is now logged when logging resource limits exceeded (#7076)
• JetStream API requests are always handled from the worker pool, improving the semantics of the API request queue and logging when requests take too long (#7125)
• Filestore in-memory caches now use weak pointers and can respond to garbage collector (GC) pressure, which should help to avoid a number of possible out-of-memory scenarios (#7180)

Leafnodes

• Leafnode connections without auth no longer unexpectedly connect to the global account (#7116)

Fixed

JetStream

• Improved validation for stream/consumer configs (#7134)
• An error will now be correctly surfaced when the Nats-Expected-Last-Subject-Sequence-Subject header is supplied but the accompanying Nats-Expected-Last-Subject-Sequence header is not (#7196)

Complete Changes

v2.12.0-preview.2...v2.12.0-RC.1

Changelog

Refer to the 2.11 Upgrade Guide for backwards compatibility notes with 2.10.x.

Go Version

• 1.24.7 (#7273)

Dependencies

• golang.org/x/time v0.13.0 (#7273)
• golang.org/x/sys v0.36.0 (#7273)

Improved

JetStream

• Offline assets support (#7158)
  • Server version 2.12 introduces new features that would otherwise break a 2.11 server after a downgrade. The server now reports the streams/consumers as offline and unsupported, keeping the data safe, but allowing to either delete the asset or upgrade back to the supported version without changes to the data itself.

Fixed

General

• Route connections with invalid credentials will no longer rapidly reconnect (#7200)
• Allow a default_sentinel JWT from a scoped signing key instead of requiring it to solely be a bearer token for auth callout (#7217)
• Subject interest would not always be propagated for leaf nodes when daisy chaining imports/exports (#7255)
• Subject interest would sometimes be lost if the leaf node is a spoke (#7259)
• Lowering the max connections limit should no longer result in streams losing interest (#7258)

JetStream

• A potential data race during a consumer create or update when reading its paused state has been fixed (#7201)
• A race condition that could allow creating a consumer with more replicas than the stream has been fixed (#7202)
• A race condition that could allow creating the same stream with different configurations has been fixed (#7210, #7212)
• Raft will now correctly reject delayed entries from an old leader when catching up in the meantime (#7209, #7239)
• Raft will now also limit the amount of cached in-memory entries as the leader, avoiding excessive memory usage (#7233)
• A potential race condition delaying shutdown if a stream/consumer monitor goroutine was not started (#7211)
• A benign underflow when using an infinite (-1) MaxDeliver for consumers (#7216)
• A potential panic to send a leader elected advisory when shutting down before completing startup (#7246)
• Stopping a stream should no longer wait indefinitely if the consumer monitor goroutine wasn’t stopped (#7249)
• Speed up stream mirroring and sourcing after a leaf node reconnects in complex topologies (#7265)
• Updating a stream with an empty placement will no longer incorrectly trigger a stream move (#7222)

Tests

• Tests have been improved (#7230, #7213, #7248)

Complete Changes

v2.11.9-RC.2...v2.11.9-RC.3

Changelog

This release also contains all changes up to and including v2.11.8.

Go Version

• 1.25.0 (#7166)

Added

General

• Exponential backoff on route/gateway reconnection attempts (#7042, #7048)
• Added allow_insecure_cipher_suites configuration option to tls block, as insecure ciphers are now disabled by default (#7144)
• No responders errors from the server now include the original subject in the Nats-Subject header (#5250)

JetStream

• Atomic batch publishes (#6966, #7057, #7060, #7085, #7090, #7091, #7119, #7152, #7175)
  • Stage-and-commit style batches for JetStream message publishes with normal consistency checks
  • ADR: https://github.com/nats-io/nats-architecture-and-design/blob/main/adr/ADR-50.md
• Counter streams (#6973, #6988, #7081, #7118)
  • Counters are incremented or decremented by publishing to the stream with Nats-Incr headers, with full aggregate consistency over sources and mirrors
  • ADR: https://github.com/nats-io/nats-architecture-and-design/blob/main/adr/ADR-49.md
• Prioritised mode for consumer priority groups (#7113)
  • Allows for low-latency switching between clients based on the priority set
  • ADR: https://github.com/nats-io/nats-architecture-and-design/blob/main/adr/ADR-42.md
• Added no_hdr flag to message get and direct get API endpoints (#6994)
  • Messages returned with this flag set will not contain headers, allowing fast access to e.g. counter values
  • ADR: https://github.com/nats-io/nats-architecture-and-design/blob/main/adr/ADR-52.md
• Support for trusted proxies (#7153)
  • Allows enforcing that connections arrive via a NATS protocol-aware proxy
  • ADR: https://github.com/nats-io/nats-architecture-and-design/blob/main/adr/ADR-55.md
• Added the ability to promote mirrors to normal streams by removing the mirror configuration (#7171)
  • This allows a mirror to take over operation if the original stream is lost, for example in a disaster recovery scenario
  • This operation cannot be undone and also requires configuring the stream subjects to continue operation
• Added delayed message scheduling (#7170)
  • Implements single schedule messages using the Nats-Schedule-TTL message header
  • ADR: https://github.com/nats-io/nats-architecture-and-design/blob/main/adr/ADR-51.md
• Added support for offline assets (#7158)
  • Streams and consumers from a higher API version or using unsupported features (i.e. after a downgrade) will be marked as offline to prevent corruption
  • ADR: https://github.com/nats-io/nats-architecture-and-design/blob/main/adr/ADR-44.md
• Added partition(n) for deterministic hash-based partitioning and random(n) for non-deterministic random partitioning to subject transforms (#6950)
• Added the ability to specify the required API level in JetStream API requests with the Nats-Required-Api-Level request header (#7157)
• Add server_metadata block to server config, allowing arbitrary key-pairs to be set on a per-server basis, similar to stream/consumer metadata (#6935)
• Support for empty votes, which improves how the Raft election logic handles nodes that have lost their stable storage and attempt to rejoin the cluster regardless (#7038)
• Added "leader_since" to the stream and consumer info responses (#7189)

Leafnodes

• Added the ability to isolate leafnode subject interest propagation with the hub-side isolate_leafnode_interest configuration option, suppressing east-west traffic and reducing subscription overheads when there are many leafnodes (#7238)
• Added disabled option for leafnode remotes, with support for reloading from the config at runtime (#7054)

Changed

JetStream

• The JetStream API level has been increased to 2 (#6969)
• The API surface is now in strict mode by default, erroring if unknown fields are provided in the request body (#7049)
• Replicated streams will now default to async flush as long as sync: always is not configured (#7018, #7163)
• The default value for max_buffered_msgs has been increased by 10x to 100,000 messages (#6633)

Improved

General

• When logging closed connections, the remote is now logged if known, making it easier to identify where route/gateway/leafnode connections have dropped (#7077)
• Client-specific log lines now include the account and user if known (#7079)

JetStream

• Raft nodes will no longer respond success to catchup messages and will ignore responses if not leader, fixing some potential stream desync scenarios (#6944)
• The stream leader is now responsible for remapping consumer assignments on scaling/moves, fixing some potential placement races (#7071, #7112, #7083)
• Replicated streams can now be created even if some of the replica nodes are offline (#7075)
• The storage type is now logged when logging resource limits exceeded (#7076)
• JetStream API requests are always handled from the worker pool, improving the semantics of the API request queue and logging when requests take too long (#7125)
• Filestore in-memory caches now use weak pointers and can respond to garbage collector (GC) pressure, which should help to avoid a number of possible out-of-memory scenarios (#7180)

Leafnodes

• Leafnode connections without auth no longer unexpectedly connect to the global account (#7116)

Fixed

JetStream

• Improved validation for stream/consumer configs (#7134)
• An error will now be correctly surfaced when the Nats-Expected-Last-Subject-Sequence-Subject header is supplied but the accompanying Nats-Expected-Last-Subject-Sequence header is not (#7196)

Complete Changes

v2.12.0-preview.1...v2.12.0-preview.2

Changelog

Refer to the 2.11 Upgrade Guide for backwards compatibility notes with 2.10.x.

Go Version

• 1.24.6

Improved

JetStream

• The stream info and consumer info endpoints now return leader_since (#7189)
• The stream info and consumer info endpoints now return system_account and traffic_account (#7193)
• The jsz monitoring endpoint now returns system_account and traffic_account (#7193)

Fixed

JetStream

• In operator mode, the cluster_traffic state for an account is now restored correctly when enabling JetStream at startup (#7191)

Complete Changes

v2.11.9-RC.1...v2.11.9-RC.2

Changelog

Refer to the 2.11 Upgrade Guide for backwards compatibility notes with 2.10.x.

Go Version

• 1.24.6

Improved

JetStream

• The raftz endpoint now reports the cluster traffic account (#7186)

Fixed

General

• Fix a panic that could happen at startup if building from source using non-Git version control (#7178)
• Fix an issue where issuing an account JWT update with a connection limit could cause older clients to be disconnected instead of newer ones (#7181, #7185)

JetStream

• The Nats-TTL header will now be correct if the subject delete marker TTL overwrites it (#7177)

Tests

• Tests have been improved (#7172)

Complete Changes

v2.11.8...v2.11.9-RC.1

Changelog

Refer to the 2.11 Upgrade Guide for backwards compatibility notes with 2.10.x.

Go Version

• 1.24.6 (#7156)

Dependencies

• golang.org/x/crypto v0.41.0 (#7166)
• golang.org/x/sys v0.35.0 (#7166)

Added

General

• Community-contributed support for building on Solaris and Illumos (#7122, thanks to @jwntree)

Fixed

General

• String-to-integer parsing has been improved in various places to prevent overflows/underflows (#7145)

JetStream

• Fixed an incorrectly formatted log line when failing to load a block when recovering TTLs (#7150)
• Raft will now step down if a higher term is detected during a catchup (#7151)
• Raft will now more reliably ignore entries from previous/cancelled catchups that arrive late (#7151)
• Fix a potential panic that could happen by a division by zero when applying Raft entries (#7151)
• The healthcheck endpoint should no longer report transient errors for newly created or recently deleted consumers (#7154)
• Fix a potential panic when trying to truncate a filestore block that doesn't exist (#7162)
• Clean up stale index.db file when truncating so that it is not inconsistent if the truncate operation is interrupted (#7162)
• Fix an off-by-one problem when Raft truncates to the correct index at startup (#7162)
• Ephemeral consumers will always select an online server when created on a replicated stream (#7165)

Tests

• Tests have been improved (#7140)

Complete Changes

v2.11.7...v2.11.8

Changelog

Refer to the 2.11 Upgrade Guide for backwards compatibility notes with 2.10.x.

Go Version

• 1.24.6 (#7156)

Dependencies

• golang.org/x/crypto v0.41.0 (#7166)
• golang.org/x/sys v0.35.0 (#7166)

Added

General

• Community-contributed support for building on Solaris and Illumos (#7122, thanks to @jwntree)

Fixed

General

• String-to-integer parsing has been improved in various places to prevent overflows/underflows (#7145)

JetStream

• Fixed an incorrectly formatted log line when failing to load a block when recovering TTLs (#7150)
• Raft will now step down if a higher term is detected during a catchup (#7151)
• Raft will now more reliably ignore entries from previous/cancelled catchups that arrive late (#7151)
• Fix a potential panic that could happen by a division by zero when applying Raft entries (#7151)
• The healthcheck endpoint should no longer report transient errors for newly created or recently deleted consumers (#7154)
• Fix a potential panic when trying to truncate a filestore block that doesn't exist (#7162)
• Clean up stale index.db file when truncating so that it is not inconsistent if the truncate operation is interrupted (#7162)
• Fix an off-by-one problem when Raft truncates to the correct index at startup (#7162)
• Ephemeral consumers will always select an online server when created on a replicated stream (#7165)

Tests

• Tests have been improved (#7140)

Complete Changes

v2.11.7...v2.11.8-RC.1

Changelog

This release also contains all changes up to and including v2.11.7.

Go Version

• 1.24.6

Added

General

• Exponential backoff on route/gateway reconnection attempts (#7042, #7048)
• Added allow_insecure_cipher_suites configuration option to tls block, as insecure ciphers are now disabled by default (#7144)

JetStream

• Atomic batch publishes (#6966, #7057, #7060, #7085, #7090, #7091, #7119, #7152)
  • Stage-and-commit style batches for JetStream message publishes with normal consistency checks
  • ADR: https://github.com/nats-io/nats-architecture-and-design/blob/main/adr/ADR-50.md
• Counter streams (#6973, #6988, #7081, #7118)
  • Counters are incremented or decremented by publishing to the stream with Nats-Incr headers, with full aggregate consistency over sources and mirrors
  • ADR: https://github.com/nats-io/nats-architecture-and-design/blob/main/adr/ADR-49.md
• Prioritised mode for consumer priority groups (#7113)
  • Allows for low-latency switching between clients based on the priority set
  • ADR: https://github.com/nats-io/nats-architecture-and-design/blob/main/adr/ADR-42.md
• Added no_hdr flag to message get and direct get API endpoints (#6994)
  • Messages returned with this flag set will not contain headers, allowing fast access to e.g. counter values
  • ADR: https://github.com/nats-io/nats-architecture-and-design/blob/main/adr/ADR-52.md
• Support for trusted proxies (#7153)
  • Allows enforcing that connections arrive via a NATS protocol-aware proxy
  • ADR: https://github.com/nats-io/nats-architecture-and-design/blob/main/adr/ADR-55.md
• Added partition(n) for deterministic hash-based partitioning and random(n) for non-deterministic random partitioning to subject transforms (#6950)
• Add server_metadata block to server config, allowing arbitrary key-pairs to be set on a per-server basis, similar to stream/consumer metadata (#6935)
• Asynchronous flush support for the filestore with the new allow_async_flush stream option (#7018)
• Support for empty votes, which improves how the Raft election logic handles nodes that have lost their stable storage and attempt to rejoin the cluster regardless (#7038)

Leafnodes

• Added disabled option for leafnode remotes, with support for reloading from the config at runtime (#7054)

Changed

JetStream

• The JetStream API level has been increased to 2 (#6969)
• The API surface is now in strict mode by default, erroring if unknown fields are provided in the request body (#7049)

Improved

General

• When logging closed connections, the remote is now logged if known, making it easier to identify where route/gateway/leafnode connections have dropped (#7077)

JetStream

• Raft nodes will no longer respond success to catchup messages and will ignore responses if not leader, fixing some potential stream desync scenarios (#6944)
• The stream leader is now responsible for remapping consumer assignments on scaling/moves, fixing some potential placement races (#7071, #7112, #7083)
• Replicated streams can now be created even if some of the replica nodes are offline (#7075)
• The storage type is now logged when logging resource limits exceeded (#7076)
• JetStream API requests are always handled from the worker pool, improving the semantics of the API request queue and logging when requests take too long (#7125)

Leafnodes

• Leafnode connections without auth no longer unexpectedly connect to the global account (#7116)

Fixed

JetStream

• Improved validation for stream/consumer configs (#7134)

Complete Changes

v2.11.7...v2.12.0-preview.1

Changelog

Refer to the 2.11 Upgrade Guide for backwards compatibility notes with 2.10.x.

Go Version

• 1.24.5 (#7047)

Dependencies

• golang.org/x/crypto v0.40.0 (#7061)
• golang.org/x/sys v0.34.0 (#7061)

Added

General

• The SubjectMatchesFilter function is now available as an exported function for embedded use (#7051)
• The leafz monitoring endpoint now includes the connection ID (#7063)
• The monitoring endpoint index page now includes the endpoint names on hover (#7066, #7087)

Improved

JetStream

• Consumers with inactivity thresholds should no longer age out before processing acks (#7107)
• The Raft layer will no longer request store state on each apply (#7109)
• Tombstones in Raft log compactions will now be written asynchronously, similar to purges (#7109)
• When enabling per-message TTLs on a stream, existing messages with the Nats-TTL header are now scanned and processed (#7117)

Fixed

General

• Message header lookups with common prefixes will now return correctly in all cases, fixing a problem where the headers could be sensitive to ordering (#7065)
• Validate that the default_sentinel JWT is a bearer token for auth callout (7074)
• The $SYS.REQ.USER.INFO endpoint should now only be answered by the local server, fixing cases where the endpoint may sometimes return without full connection details (#7089)

JetStream

• The Raft layer will require recovery and snapshot handling at startup before campaigning for a leadership election, fixing a situation where a node could continue with an outdated stream (#7040)
• The Raft log will no longer be compacted until after a snapshot is written, improving crash resilience (#7043)
• A race condition when shutting down Raft nodes which could result in no snapshot being written has been fixed (#7045)
• Consumer pull requests that use no_wait or expires behaviour has been fixed with replicated consumers (#7046)
• Pull consumers with an inactive threshold will now consider pending acks when determining inactivity, preventing the consumer from being deleted while messages are being processed (#7052)
• Push consumers will now correctly error when trying to configure priority groups (#7053)
• Committed entry objects will now be correctly returned to the pool on error, reducing allocations (#7064)
• The time hash wheel used for per-message TTLs now correctly detects and expires messages with TTLs over an hour, previously it could take double the expected time (#7070)
• A potential panic when selecting message blocks during TTL recovery has been fixed (#7072)
• A KV purge operation with subject delete markers configured will no longer leave behind a redundant extra delete marker (#7026)
• Raft will now correctly attempt to truncate back to a snapshot if applies were not caught up instead of resetting the entire log (#7095)
• Store cipher conversion will now work correctly when combined with store compression (#7099)
• Truncate and erase operations in the filestore should now be consistent after a hard kill (#7100)
• When a filestore message block is deleted and an unclean shutdown results in a stale index.db, the deleted blocks are now correctly marked as lost data and the index is rebuilt (#7123)
• Fixed a potential underflow that could happen when modifying max_bytes reservations (#7131)

Tests

• Tests have been improved (#7031, #7036, #7050, #6728, #7096)

Complete Changes

v2.11.6...v2.11.7