ref(kafka): Add producer_name tag, and more rdkafka stats #5031
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
ref STREAM-324 * The producer_name tag identifies a particular producer instance in Relay. It's almost identical to broker_name, but broker_name is not generally available because in theory one producer can talk to many brokers (though relay doesn't do that), and some statistics are related to the producer's internal datastructures and don't have that tag. Also, we're going to add this tag to other producers in other services. With a bit of luck we might even have a proper generic dashboard for all of our producers in our entire infrastructure.. someday. * Add a few metrics that we think might be useful to see if Relay is impacted by a broker being unhealthy. Aside, it would be really nice if Relay moved to rust-arroyo for producing. I'm implementing the same metrics there. Or maybe not its entire producer wrapper but at least reusing its stats callback.
Dav1dde
approved these changes
Aug 6, 2025
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
|
@Dav1dde asking for another review because I decided to align some additional metrics with arroyo. if you rely on these metric names in alerts we can also align arroyo with relay instead. |
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
untitaker
added a commit
to getsentry/arroyo
that referenced
this pull request
Aug 6, 2025
ref STREAM-324 See getsentry/relay#5031 for context
I don't think so, but we do have them in the Relay dashboard, would be appreciated if you can take a look and update them if necessary. Otherwise regarding the metrics and their values, I'll just leave that to you, you're the expert on that. |
untitaker
added a commit
to getsentry/arroyo
that referenced
this pull request
Aug 7, 2025
ref STREAM-324 See getsentry/relay#5031 for context
untitaker
added a commit
to getsentry/sentry
that referenced
this pull request
Aug 13, 2025
ref STREAM-324 see getsentry/arroyo#479 and getsentry/relay#5031 --------- Co-authored-by: getsantry[bot] <66042841+getsantry[bot]@users.noreply.github.com>
priscilawebdev
pushed a commit
to getsentry/sentry
that referenced
this pull request
Aug 25, 2025
ref STREAM-324 see getsentry/arroyo#479 and getsentry/relay#5031 --------- Co-authored-by: getsantry[bot] <66042841+getsantry[bot]@users.noreply.github.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
ref STREAM-324
The producer_name tag identifies a particular producer instance in
Relay. It's almost identical to broker_name, but broker_name is not
generally available because in theory one producer can talk to many
brokers (though relay doesn't do that), and some statistics are
related to the producer's internal datastructures and don't have that
tag.
Also, we're going to add this tag to other producers in other
services (and we have many producers in getsentry talking to a
single broker there), so it pays off to be consistent. With a bit
of luck we might even have a proper generic dashboard for all of
our producers in our entire infrastructure..
someday.
Add a few metrics that we think might be useful to see if Relay is
impacted by a broker being unhealthy.
Aside, it would be really nice if Relay moved to rust-arroyo for
producing. I'm implementing the same metrics there. Or maybe not its
entire producer wrapper but at least reusing its stats callback.