Lower log level for Cloud/AWS data fetches #41389
Conversation
|
/test |
|
@gandro just mention you here so you are aware of this PR |
aa2a0b0 to
5505316
Compare
|
/test |
Lowering the log severity from warn to info for cases where stale data is received from AWS. As discussed in cilium#36428 and cilium#41278, stale metadata can cause temporary IP calculation issues, but the operator is expected to eventually reconcile correctly. These warnings have been observed to cause CI failures, despite being ok in Datadog's environment Signed-off-by: Liyi Huang <liyi.huang@isovalent.com>
5505316 to
b8ff65d
Compare
|
/test |
|
I'll merge this as it looks like it will help with CI stability, but I do wonder whether we should iterate on the messages being provided to the user here. These will be "info" level logs but if I look at a message like |
I totally understand where you concern comes from. I'm not sure if you got any time to take a look at the discussion in #41278. Here is the summary. we think this is an error casued by AWS providing eventually consistent so we incorrectly create the new ENI or assign IP address. in datadog environment, they mentioned the following.
From user perspective, I think it is an error but it will get to healthy status within 1 min(IPAM has periodically sync with AWS). Hopefully users will not expereince the issue to look into this message. I could look into it in the future if time allows and the fix doesn't complicate the code too much. |
Lowering the log severity from warn to info for cases where stale data is received from AWS. As discussed in #36428 and #41278, stale metadata can cause temporary IP calculation issues, but the operator is expected to eventually reconcile correctly. These warnings have been observed to cause CI failures, despite being ok in Datadog's environment
Fixes: #36428