feat(common.shim): Add batch to shim #16148
Conversation
|
Thanks so much for the pull request! |
|
!signed-cla |
2cedc7c to
f6ed0af
Compare
|
@revilon1991 Thanks for the PR! I see you commented for the CLA checker, but I don't see your username in the list. Could you double check that you submitted that? Thanks! |
|
@DStrand1 which list? I don't know what I should do else |
|
!signed-cla |
94b8e6a to
524954d
Compare
There was a problem hiding this comment.
@revilon1991 thank you for tackling this long-standing issues!
However, I think your code is far too complicated. Why not simply adding BatchSize as config parameter (as you do) with a default value of 1 (for backward compatibility). Then do something like
metrics := make([]telegraf.Metric, 0, s.BatchSize)
scanner := bufio.NewScanner(s.stdin)
for scanner.Scan() {
// Read metrics from stdin
m, err := parser.ParseLine(scanner.Text())
if err != nil {
fmt.Fprintf(s.stderr, "Failed to parse metric: %s\n", err)
continue
}
metrics = append(metrics, m)
// If we got more than our batch size, flush out the metrics
// until we emptied the buffer enough.
for len(metrics) >= s.BatchSize {
if err = s.Output.Write(metrics[:s.BatchSize]); err != nil {
fmt.Fprintf(s.stderr, "Failed to write metrics: %s\n", err)
}
metrics = metrics[s.BatchSize:]
}
}
// Output all remaining metrics
for len(metrics) >= s.BatchSize {
if err = s.Output.Write(metrics[:s.BatchSize]); err != nil {
fmt.Fprintf(s.stderr, "Failed to write metrics: %s\n", err)
}
metrics = metrics[s.BatchSize:]
}|
Hello! I am closing this issue due to inactivity. I hope you were able to resolve your problem, if not please try posting this question in our Community Slack or Community Forums or provide additional details in this issue and reqeust that it be re-opened. Thank you! |
|
@srebhan hello, can you reopen this pr? |
|
@revilon1991 sorry I missed your message! Reopened... |
Co-authored-by: Thomas Casteleyn <thomas.casteleyn@me.com>
Co-authored-by: Thomas Casteleyn <thomas.casteleyn@me.com>
Co-authored-by: Thomas Casteleyn <thomas.casteleyn@me.com>
Co-authored-by: Thomas Casteleyn <thomas.casteleyn@me.com>
|
@srebhan I’ve reviewed your code — yes, it fits in nicely and isn’t overly complex. It should work correctly as well. Thanks a lot for your contribution! |
|
@mstrandboge and @skartikey can we please get a review from both of you as I was involved in the code... |
|
Download PR build artifacts for linux_amd64.tar.gz, darwin_arm64.tar.gz, and windows_amd64.zip. 📦 Click here to get additional PR build artifactsArtifact URLs |
Summary
I want to use batch with shim within the method
Write.Checklist
Related issues
resolves #11902
resolves #10277