Change in behavior between 1.33 and 1.34 - Additional cloudwatch labels are introduced #17001
Description
Relevant telegraf.conf
[[inputs.cloudwatch]]
region = "us-west-2"
period = "1m"
delay = "1m"
interval = "1m"
cache_ttl = "1h"
namespaces = ["AWS/ApplicationELB"]
ratelimit = 10
[[inputs.cloudwatch.metrics]]
names = ["UnHealthyHostCount", "HealthyHostCount"]
statistic_include = ["average"]
statistic_exclude = [] # required to avoid panic in telegraf 1.34
[[inputs.cloudwatch.metrics.dimensions]]
name = "LoadBalancer"
value = "*"
[[inputs.cloudwatch.metrics.dimensions]]
name = "TargetGroup"
value = "*"Logs from Telegraf
n/a
System info
docker telegraf:1.34
Docker
n/a
Steps to reproduce
Perform a collection on ALB/ELB cloudwatch metrics which now include a metric both with and with out an availabilityZone.
e.g.
cloudwatch_aws_application_elb,availability_zone=us-west-2b,host=XXXXX,load_balancer=app/prometheus-corp-multi/YYYYYY,region=us-west-2,target_group=targetgroup/prom-ZZZZZZ/VVVVVV un_healthy_host_count_average=0,healthy_host_count_average=1 1747147020000000000
cloudwatch_aws_application_elb,host=XXXXX,load_balancer=app/prometheus-corp-multi/YYYYYY,region=us-west-2,target_group=targetgroup/prom1-ZZZZZZ/VVVVVV healthy_host_count_average=1,un_healthy_host_count_average=0 1747147020000000000
while in 1.33 only
cloudwatch_aws_application_elb,host=XXXXX,load_balancer=app/prometheus-corp-multi/YYYYYY,region=us-west-2,target_group=targetgroup/prom1-ZZZZZZ/VVVVVV healthy_host_count_average=1,un_healthy_host_count_average=0 1747147020000000000
would have been returned.
Expected behavior
Either update the Readme plugins/inputs/cloudwatch/README.md to say something about not being able to control which dimensions are used - or add some functionality to drop dimensions.
Actual behavior
N/a - the previous behavior was confusing - the new behavior caused a minor issue in our metrics but was easily resolved. Recommend the docs be updated with a note in the release notes about the impact.
Additional info
related to #16337