Skip to content

Conversation

borkmann
Copy link
Member

@borkmann borkmann commented Jul 10, 2024

(see commit msg)

Fixes: #32689

@borkmann borkmann added kind/cleanup This includes no functional changes. release-note/misc This PR makes changes that have no direct user impact. labels Jul 10, 2024
@borkmann borkmann requested a review from rgo3 July 10, 2024 13:00
@borkmann borkmann requested a review from a team as a code owner July 10, 2024 13:00
@borkmann borkmann requested a review from ldelossa July 10, 2024 13:00
@borkmann borkmann added needs-backport/1.16 This PR / issue needs backporting to the v1.16 branch needs-backport/1.15 labels Jul 10, 2024
@borkmann borkmann requested a review from a team as a code owner July 10, 2024 13:19
@borkmann borkmann requested a review from nathanjsweet July 10, 2024 13:19
@borkmann
Copy link
Member Author

/test

@borkmann borkmann requested a review from gandro July 10, 2024 13:27
Copy link
Member

@gandro gandro left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this looks good. Have you tested if this fixes CI?

@borkmann borkmann force-pushed the pr/health branch 2 times, most recently from 1290aa7 to 02b4ec9 Compare July 11, 2024 11:01
Copy link
Member

@gandro gandro left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code looks good to me. We should also test if this really fixes CI like the previous PR did: dd94c27

@borkmann borkmann requested review from a team as code owners July 11, 2024 11:16
@borkmann borkmann requested review from aanm and christarazi July 11, 2024 11:16
@borkmann
Copy link
Member Author

/test

@borkmann
Copy link
Member Author

borkmann commented Jul 11, 2024

Interestingly, the cleanup in the beginning still did not help:

[...]
2024-07-11T13:06:11.305181694Z time="2024-07-11T13:06:11Z" level=info msg="Restoring endpoints..." subsys=daemon
2024-07-11T13:06:11.305185120Z time="2024-07-11T13:06:11Z" level=debug msg="Killing old health endpoint process" pidfile=/var/run/cilium/state/health-endpoint.pid subsys=cilium-health-launcher
2024-07-11T13:06:11.305196952Z time="2024-07-11T13:06:11Z" level=debug msg="Didn't find existing device" error="Link not found" interface=cilium_health subsys=cilium-health-launcher
2024-07-11T13:06:11.305199717Z time="2024-07-11T13:06:11Z" level=debug msg="Didn't find existing device" error="Link not found" interface=lxc_health subsys=cilium-health-launcher
2024-07-11T13:06:11.305202552Z time="2024-07-11T13:06:11Z" level=debug msg="Removing old health endpoint state directory" endpointID=2527 file-path=/var/run/cilium/state/2527 subsys=daemon
2024-07-11T13:06:11.305205157Z time="2024-07-11T13:06:11Z" level=debug msg="Restoring endpoint" ciliumEndpointName=/ endpointID=2959 subsys=daemon
[...]
2024-07-11T13:06:59.063745646Z time="2024-07-11T13:06:59Z" level=debug msg="Controller func execution time: 7.720565ms" name=ct-map-pressure subsys=controller uuid=98a32dc0-b5da-455b-8f4d-6618ec7af3dd
2024-07-11T13:06:59.064512265Z time="2024-07-11T13:06:59Z" level=info msg="Launching Cilium health endpoint" subsys=daemon
2024-07-11T13:06:59.065306735Z time="2024-07-11T13:06:59Z" level=debug msg="Starting new controller" name=cilium-health-ep subsys=controller uuid=899e6e16-9003-4a06-b216-d3d268d84391
2024-07-11T13:06:59.065948300Z time="2024-07-11T13:06:59Z" level=info msg="Cleaning up Cilium health endpoint" subsys=daemon
2024-07-11T13:06:59.065955123Z time="2024-07-11T13:06:59Z" level=debug msg="Killing old health endpoint process" pidfile=/var/run/cilium/state/health-endpoint.pid subsys=cilium-health-launcher
2024-07-11T13:06:59.066043027Z time="2024-07-11T13:06:59Z" level=debug msg="Didn't find existing cilium-health endpoint to delete" subsys=daemon
2024-07-11T13:06:59.066047125Z time="2024-07-11T13:06:59Z" level=debug msg="Didn't find existing device" error="Link not found" interface=cilium_health subsys=cilium-health-launcher
2024-07-11T13:06:59.066050050Z time="2024-07-11T13:06:59Z" level=debug msg="Didn't find existing device" error="Link not found" interface=lxc_health subsys=cilium-health-launcher
2024-07-11T13:06:59.075560441Z time="2024-07-11T13:06:59Z" level=debug msg="Compiling datapath" clang="clang version 17.0.6 (https://github.com/llvm/llvm-project.git 6009708b4367171ccdbf4b5905cb6a803753fe18)\nTarget: unknown\nThread model: posix\nInstalledDir: /usr/local/bin\n" debug=true subsys=datapath-loader
2024-07-11T13:06:59.075939908Z time="2024-07-11T13:06:59Z" level=debug msg="Launching compiler" args="[-I/var/run/cilium/state/globals -I/var/run/cilium/state/templates/40cd79ad4efe0858c6dddfc5d87fbf57b44f0930942d12240d3bb96ad21aa672 -I/var/lib/cilium/bpf -I/var/lib/cilium/bpf/include -E -O2 --target=bpf -std=gnu89 -nostdinc -Wall -Wextra -Werror -Wshadow -Wno-address-of-packed-member -Wno-unknown-warning-option -Wno-gnu-variable-sized-type-not-at-end -Wdeclaration-after-statement -Wimplicit-int-conversion -Wenum-conversion -D__NR_CPUS__=4 -mcpu=v3 -c /var/lib/cilium/bpf/bpf_lxc.c -o -]" subsys=datapath-loader target=clang
2024-07-11T13:06:59.084486312Z time="2024-07-11T13:06:59Z" level=debug msg="Created veth pair" subsys=endpoint-connector vethPair="[cilium lxc_health]"
2024-07-11T13:06:59.095243265Z time="2024-07-11T13:06:59Z" level=info msg="Started healthz status API server" address="127.0.0.1:9879" subsys=daemon
[...]
2024-07-11T13:06:59.118803233Z time="2024-07-11T13:06:59Z" level=info msg="Configuring Hubble server" eventQueueSize=4096 maxFlows=65535 subsys=hubble
2024-07-11T13:06:59.118809675Z time="2024-07-11T13:06:59Z" level=debug msg="Controller func execution time: 21.568989ms" name=node-neighbor-link-updater subsys=controller uuid=505b153c-19b7-4289-85e3-f237ec485b3a
2024-07-11T13:06:59.120142649Z time="2024-07-11T13:06:59Z" level=debug msg="Controller run failed" consecutiveErrors=1 error="unable to determine next hop IPv4 address for eth1 (172.18.0.4): remote node IP is non-routable\nunable to determine next hop IPv6 address for eth1 (fc00:c111::4): remote node IP is non-routable\nunable to determine next hop IPv4 address for eth1 (172.18.0.5): remote node IP is non-routable\nunable to determine next hop IPv6 address for eth1 (fc00:c111::5): remote node IP is non-routable" name=node-neighbor-link-updater subsys=controller uuid=505b153c-19b7-4289-85e3-f237ec485b3a
2024-07-11T13:06:59.120158228Z time="2024-07-11T13:06:59Z" level=debug msg="Created lxc_health devices!!!" subsys=cilium-health-launcher
2024-07-11T13:06:59.120536372Z time="2024-07-11T13:06:59Z" level=debug msg="Controller func execution time: 499.732µs" name=link-cache subsys=controller uuid=47c1975e-76e8-474d-baec-5f7faf7bb103
2024-07-11T13:06:59.120819660Z time="2024-07-11T13:06:59Z" level=debug msg="Waiting for endpoint restoration before serving resources..." subsys=xds xdsStreamID=1
2024-07-11T13:06:59.121165474Z time="2024-07-11T13:06:59Z" level=debug msg="Group not found" error="group: unknown group cilium" file-path=/var/run/cilium/hubble.sock group=cilium subsys=api
2024-07-11T13:06:59.122092111Z time="2024-07-11T13:06:59Z" level=info msg="Starting local Hubble server" address="unix:///var/run/cilium/hubble.sock" subsys=hubble
2024-07-11T13:06:59.123795906Z time="2024-07-11T13:06:59Z" level=debug msg="Endpoint creation" ciliumEndpointName=/ code=OK containerID= containerInterface= datapathPolicyRevision=0 desiredPolicyRevision=0 endpointID=0 endpointState=waiting-for-identity ipv4= ipv6= k8sPodName=/ policyRevision=0 subsys=endpoint type=0
2024-07-11T13:06:59.125663816Z time="2024-07-11T13:06:59Z" level=info msg="Beginning to read perf buffer" startTime="2024-07-11 13:06:59.120013062 +0000 UTC m=+67.635907056" subsys=monitor-agent
2024-07-11T13:06:59.134908632Z time="2024-07-11T13:06:59Z" level=debug msg="TLS configuration ready" config=tls-server subsys=hubble
2024-07-11T13:06:59.136760091Z time="2024-07-11T13:06:59Z" level=info msg="Starting Hubble server" address=":4244" subsys=hubble tls=true

[...]
ime="2024-07-11T13:11:40Z" level=debug msg="Inserted program handle_policy into cilium_call_policy slot 3813" subsys=datapath-loader
2024-07-11T13:11:40.494169001Z time="2024-07-11T13:11:40Z" level=error msg="Error while reloading endpoint BPF program" ciliumEndpointName=/ containerID= containerInterface= datapathPolicyRevision=0 desiredPolicyRevision=1 endpointID=3813 error="retrieving device lxc_health: Link not found" identity=4 ipv4=10.244.0.89 ipv6="fd00:10:244::8f21" k8sPodName=/ subsys=endpoint
2024-07-11T13:11:40.507246888Z time="2024-07-11T13:11:40Z" level=debug msg="Reverting endpoint changes after BPF regeneration failed" ciliumEndpointName=/ containerID= containerInterface= datapathPolicyRevision=0 desiredPolicyRevision=1 endpointID=3813 identity=4 ipv4=10.244.0.89 ipv6="fd00:10:244::8f21" k8sPodName=/ subsys=endpoint
2024-07-11T13:11:40.508978725Z time="2024-07-11T13:11:40Z" level=debug msg="Reverting proxy redirect removals" ciliumEndpointName=/ containerID= containerInterface= datapathPolicyRevision=0 desiredPolicyRevision=1 endpointID=3813 identity=4 ipv4=10.244.0.89 ipv6="fd00:10:244::8f21" k8sPodName=/ subsys=endpoint
2024-07-11T13:11:40.515374522Z time="2024-07-11T13:11:40Z" level=debug msg="Finished reverting proxy redirect removals" ciliumEndpointName=/ containerID= containerInterface= datapathPolicyRevision=0 desiredPolicyRevision=1 endpointID=3813 identity=4 ipv4=10.244.0.89 ipv6="fd00:10:244::8f21" k8sPodName=/ subsys=endpoint
2024-07-11T13:11:40.517227445Z time="2024-07-11T13:11:40Z" level=debug msg="Reverting proxy redirect additions" ciliumEndpointName=/ containerID= containerInterface= datapathPolicyRevision=0 desiredPolicyRevision=1 endpointID=3813 identity=4 ipv4=10.244.0.89 ipv6="fd00:10:244::8f21" k8sPodName=/ subsys=endpoint
2024-07-11T13:11:40.526316210Z time="2024-07-11T13:11:40Z" level=debug msg="Finished reverting proxy redirect additions" ciliumEndpointName=/ containerID= containerInterface= datapathPolicyRevision=0 desiredPolicyRevision=1 endpointID=3813 identity=4 ipv4=10.244.0.89 ipv6="fd00:10:244::8f21" k8sPodName=/ subsys=endpoint
2024-07-11T13:11:40.535427326Z time="2024-07-11T13:11:40Z" level=debug msg="Finished reverting endpoint changes after BPF regeneration failed" ciliumEndpointName=/ containerID= containerInterface= datapathPolicyRevision=0 desiredPolicyRevision=1 endpointID=3813 identity=4 ipv4=10.244.0.89 ipv6="fd00:10:244::8f21" k8sPodName=/ subsys=endpoint
2024-07-11T13:11:40.537228281Z time="2024-07-11T13:11:40Z" level=info msg="generating BPF for endpoint failed, keeping stale directory" ciliumEndpointName=/ containerID= containerInterface= datapathPolicyRevision=0 desiredPolicyRevision=1 endpointID=3813 error="retrieving device lxc_health: Link not found" file-path=3813_next_fail identity=4 ipv4=10.244.0.89 ipv6="fd00:10:244::8f21" k8sPodName=/ subsys=endpoint
2024-07-11T13:11:40.553245035Z time="2024-07-11T13:11:40Z" level=debug msg="removing directory" ciliumEndpointName=/ containerID= containerInterface= datapathPolicyRevision=0 desiredPolicyRevision=1 directory=3813_next_fail endpointID=3813 identity=4 ipv4=10.244.0.89 ipv6="fd00:10:244::8f21" k8sPodName=/ subsys=endpoint
2024-07-11T13:11:40.554469286Z time="2024-07-11T13:11:40Z" level=debug msg="removing directory" ciliumEndpointName=/ containerID= containerInterface= datapathPolicyRevision=0 desiredPolicyRevision=1 directory=3813_next endpointID=3813 identity=4 ipv4=10.244.0.89 ipv6="fd00:10:244::8f21" k8sPodName=/ subsys=endpoint
2024-07-11T13:11:40.555065818Z time="2024-07-11T13:11:40Z" level=debug msg="Completed endpoint regeneration with no pending regeneration requests" ciliumEndpointName=/ code=OK containerID= containerInterface= datapathPolicyRevision=0 desiredPolicyRevision=1 endpointID=3813 endpointState=ready identity=4 ipv4=10.244.0.89 ipv6="fd00:10:244::8f21" k8sPodName=/ policyRevision=0 subsys=endpoint type=0
2024-07-11T13:11:40.561285586Z time="2024-07-11T13:11:40Z" level=warning msg="Regeneration of endpoint failed" bpfCompilation=0s bpfLoadProg=1.639026581s bpfWaitForELF="209.771µs" ciliumEndpointName=/ containerID= containerInterface= datapathPolicyRevision=0 desiredPolicyRevision=1 endpointID=3813 error="retrieving device lxc_health: Link not found" identity=4 ipv4=10.244.0.89 ipv6="fd00:10:244::8f21" k8sPodName=/ mapSync=0s policyCalculation="95.999µs" prepareBuild=3.13949ms proxyConfiguration="9.368µs" proxyPolicyCalculation=521ns proxyWaitForAck=0s reason="updated security labels" subsys=endpoint total=4m40.369994788s waitingForCTClean=10.346679ms waitingForLock=4m38.649503748s waitingForPolicyRepository=781ns
2024-07-11T13:11:40.562000108Z time="2024-07-11T13:11:40Z" level=debug msg="Error regenerating endpoint: retrieving device lxc_health: Link not found" ciliumEndpointName=/ code=Failure containerID= containerInterface= datapathPolicyRevision=0 desiredPolicyRevision=1 endpointID=3813 endpointState=ready identity=4 ipv4=10.244.0.89 ipv6="fd00:10:244::8f21" k8sPodName=/ policyRevision=0 subsys=endpoint type=200
2024-07-11T13:11:40.569426315Z time="2024-07-11T13:11:40Z" level=debug msg="EventQueue event processing statistics" eventConsumeOffQueueWaitTime="315.989µs" eventEnqueueWaitTime=1.934585ms eventHandlingDuration=4m40.376406694s eventType="*endpoint.EndpointRegenerationEvent" name=endpoint-3813 subsys=eventqueue
2024-07-11T13:11:40.572378476Z time="2024-07-11T13:11:40Z" level=error msg="endpoint regeneration failed" ciliumEndpointName=/ containerID= containerInterface= datapathPolicyRevision=0 desiredPolicyRevision=1 endpointID=3813 error="retrieving device lxc_health: Link not found" identity=4 ipv4=10.244.0.89 ipv6="fd00:10:244::8f21" k8sPodName=/ subsys=endpoint
2024-07-11T13:11:40.576671254Z time=2024-07-11T13:11:40Z level=debug msg="upserting health status" module=health lastLevel=none reporter-id="agent.controlplane.endpoint-manager.cilium-endpoint-3813 (/).datapath-regenerate" status="agent.controlplane.endpoint-manager.cilium-endpoint-3813 (/).datapath-regenerate: [Degraded] Endpoint regeneration failed: retrieving device lxc_health: Link not found"
2024-07-11T13:11:40.577546725Z time="2024-07-11T13:11:40Z" level=debug msg="Deleting endpoint" ciliumEndpointName=/ code=OK containerID= containerInterface= datapathPolicyRevision=0 desiredPolicyRevision=1 endpointID=3813 endpointState=disconnecting identity=4 ipv4=10.244.0.89 ipv6="fd00:10:244::8f21" k8sPodName=/ policyRevision=0 subsys=endpoint type=0
2024-07-11T13:11:40.584349501Z time="2024-07-11T13:11:40Z" level=debug msg="Registered BPF map" path=/sys/fs/bpf/tc/globals/cilium_call_policy subsys=bpf
2024-07-11T13:11:40.589723724Z time="2024-07-11T13:11:40Z" level=debug msg="Unregistered BPF map" path=/sys/fs/bpf/tc/globals/cilium_call_policy subsys=bpf

It looks like sth else must be removing the lxc_health devices in between..

The sysdump itself generally has the lxc_health:

16: lxc_health@if15: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 32:48:21:5f:37:50 brd ff:ff:ff:ff:ff:ff link-netnsid 1
    inet6 fe80::3048:21ff:fe5f:3750/64 scope link 
       valid_lft forever preferred_lft forever

@christarazi christarazi marked this pull request as draft July 11, 2024 18:32
@borkmann
Copy link
Member Author

borkmann commented Jul 12, 2024

logs-cilium-df9gx-cilium-agent-20240711-132748.log

This one looks interesting as well which could have caused an unexpected removal in between:

2024-07-11T13:10:00.339933129Z time="2024-07-11T13:10:00Z" level=debug msg="Cannot find socket" error="stat /var/run/cilium/health.sock: no such file or directory" file-path=/var/run/cilium/health.sock subsys=cilium-health-launcher
2024-07-11T13:10:01.024499589Z time="2024-07-11T13:10:01Z" level=info msg="Cleaning up Cilium health endpoint" subsys=daemon
2024-07-11T13:10:01.024764002Z time="2024-07-11T13:10:01Z" level=debug msg="Killing old health endpoint process" pidfile=/var/run/cilium/state/health-endpoint.pid subsys=cilium-health-launcher
2024-07-11T13:10:01.028980479Z time="2024-07-11T13:10:01Z" level=debug msg="Killed endpoint process" pid=749 pidfile=/var/run/cilium/state/health-endpoint.pid subsys=cilium-health-launcher
2024-07-11T13:10:01.029171454Z time="2024-07-11T13:10:01Z" level=debug msg="Removing existing cilium-health endpoint" subsys=daemon
2024-07-11T13:10:01.030259441Z time="2024-07-11T13:10:01Z" level=debug msg="delete health sub-tree" module=health prefix="agent.controlplane.endpoint-manager.cilium-endpoint-3813 (/)" deleted=0

Also this patch is moot fc8074d
I'll remove it given the health controller calls d.cleanupHealthEndpoint() upon start.

Imho, the problem is rather some unexpected cleanup in between where suddenly the lxc_health device is gone.

I'd think this may be an interesting one.. esp the latter condition ...

                               // On the first initialization (client == nil), or if we have not
                                // successfully pinged it since successfulPingTimeout, restart the health EP.
                                if client == nil || time.Since(lastSuccessfulPing) > successfulPingTimeout {
                                        var launchErr error
                                        d.cleanupHealthEndpoint()

... which could create a race with the daemon when endpoints are reloaded.

One other interesting log find:

2024-07-11T13:11:40.643323253Z time="2024-07-11T13:11:40Z" level=debug msg="Endpoint removed" ciliumEndpointName=/ code=OK containerID= containerInterface= datapathPolicyRevision=0 desiredPolicyRevision=1 endpointID=3813 endpointState=disconnected identity=4 ipv4=10.244.0.89 ipv6="fd00:10:244::8f21" k8sPodName=/ policyRevision=0 subsys=endpoint type=0
2024-07-11T13:11:40.643334814Z time="2024-07-11T13:11:40Z" level=info msg="Removed endpoint" ciliumEndpointName=/ containerID= containerInterface= datapathPolicyRevision=0 desiredPolicyRevision=1 endpointID=3813 identity=4 ipv4=10.244.0.89 ipv6="fd00:10:244::8f21" k8sPodName=/ subsys=endpoint
2024-07-11T13:11:40.643426034Z time="2024-07-11T13:11:40Z" level=debug msg="Waiting for proxy updates to complete..." ciliumEndpointName=/ containerID= containerInterface= datapathPolicyRevision=0 desiredPolicyRevision=1 endpointID=3813 identity=4 ipv4=10.244.0.89 ipv6="fd00:10:244::8f21" k8sPodName=/ subsys=endpoint
2024-07-11T13:11:40.643430493Z time="2024-07-11T13:11:40Z" level=debug msg="Wait time for proxy updates: 31.459µs" ciliumEndpointName=/ containerID= containerInterface= datapathPolicyRevision=0 desiredPolicyRevision=1 endpointID=3813 identity=4 ipv4=10.244.0.89 ipv6="fd00:10:244::8f21" k8sPodName=/ subsys=endpoint
2024-07-11T13:11:40.660352513Z time="2024-07-11T13:11:40Z" level=debug msg="Error occurred while deleting cilium-health endpoint" error="Unable to delete key fd00:10:244::8f21:0 from /sys/fs/bpf/tc/globals/cilium_lxc: unable to delete element fd00:10:244::8f21:0 from map cilium_lxc: delete: key does not exist" subsys=daemon
2024-07-11T13:11:40.669614941Z time="2024-07-11T13:11:40Z" level=debug msg="Error occurred while deleting cilium-health endpoint" error="Unable to delete key 10.244.0.89:0 from /sys/fs/bpf/tc/globals/cilium_lxc: unable to delete element 10.244.0.89:0 from map cilium_lxc: delete: key does not exist" subsys=daemon
2024-07-11T13:11:40.669626673Z time="2024-07-11T13:11:40Z" level=debug msg="Didn't find existing device" error="Link not found" interface=cilium_health subsys=cilium-health-launcher
2024-07-11T13:11:40.669630350Z time="2024-07-11T13:11:40Z" level=debug msg="Didn't find existing device" error="Link not found" interface=lxc_health subsys=cilium-health-launcher
2024-07-11T13:11:40.681232518Z time="2024-07-11T13:11:40Z" level=debug msg="Inserted program handle_policy into cilium_call_policy slot 712" subsys=datapath-loader
2024-07-11T13:11:40.688489680Z time="2024-07-11T13:11:40Z" level=info msg="Updated link /sys/fs/bpf/cilium/endpoints/712/links/cil_from_container for program cil_from_container" subsys=datapath-loader
2024-07-11T13:11:40.688805418Z time="2024-07-11T13:11:40Z" level=debug msg="Replaced map pin /sys/fs/bpf/tc/globals/cilium_calls_00712" subsys=bpf
2024-07-11T13:11:40.688816889Z time="2024-07-11T13:11:40Z" level=info msg="Reloaded endpoint BPF program" ciliumEndpointName=cilium-test/test-conn-disrupt-server-774cf5b968-whgl8 containerID=376dd63f71 containerInterface=eth0 datapathPolicyRevision=1 desiredPolicyRevision=1 endpointID=712 identity=17218 ipv4=10.244.0.115 ipv6="fd00:10:244::fe77" k8sPodName=cilium-test/test-conn-disrupt-server-774cf5b968-whgl8 subsys=endpoint

@borkmann borkmann force-pushed the pr/health branch 2 times, most recently from 2fef02f to 44a8866 Compare July 12, 2024 18:23
@christarazi christarazi added the area/daemon Impacts operation of the Cilium daemon. label Jul 23, 2024
Copy link
Contributor

@ldelossa ldelossa left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@joestringer
Copy link
Member

I see the upgrades test failed with the same original error this is trying to fix, but after downgrading. This means it's actually not the version with this patch/fix that's failing, it's the previous release. Presumably then we can ignore that specific CI result: https://github.com/cilium/cilium/actions/runs/10065195269/job/27826569570 .

@borkmann
Copy link
Member Author

I see the upgrades test failed with the same original error this is trying to fix, but after downgrading. This means it's actually not the version with this patch/fix that's failing, it's the previous release. Presumably then we can ignore that specific CI result: https://github.com/cilium/cilium/actions/runs/10065195269/job/27826569570 .

Yeah, that's correct, @tommyp1ckles and I suspect this will be fixed once backported to 1.16.

@joestringer joestringer added this pull request to the merge queue Jul 23, 2024
@joestringer
Copy link
Member

OK cool. I'll prepare backport for this + one other outstanding 1.16 blocker shortly.

Merged via the queue into main with commit 0f3101c Jul 23, 2024
275 of 278 checks passed
@joestringer joestringer deleted the pr/health branch July 23, 2024 21:21
@joestringer joestringer mentioned this pull request Jul 24, 2024
10 tasks
@joestringer joestringer added backport-pending/1.16 The backport for Cilium 1.16.x for this PR is in progress. and removed needs-backport/1.16 This PR / issue needs backporting to the v1.16 branch labels Jul 24, 2024
@github-actions github-actions bot added backport-done/1.16 The backport for Cilium 1.16.x for this PR is done. and removed backport-pending/1.16 The backport for Cilium 1.16.x for this PR is in progress. labels Jul 24, 2024
borkmann added a commit that referenced this pull request Jul 24, 2024
The PR #33700 got merged into v1.16 by now.

Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
borkmann added a commit that referenced this pull request Jul 24, 2024
The PR #33700 got merged into v1.16 by now.

Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Kaczyniec pushed a commit to Kaczyniec/cilium that referenced this pull request Aug 5, 2024
The PR cilium#33700 got merged into v1.16 by now.

Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
@jrajahalme jrajahalme added backport-pending/1.14 The backport for Cilium 1.14.x for this PR is in progress. backport-pending/1.15 The backport for Cilium 1.15.x for this PR is in progress. labels Oct 30, 2024
@github-actions github-actions bot added backport-done/1.15 The backport for Cilium 1.15.x for this PR is done. backport-done/1.14 The backport for Cilium 1.14.x for this PR is done. and removed backport-pending/1.15 The backport for Cilium 1.15.x for this PR is in progress. backport-pending/1.14 The backport for Cilium 1.14.x for this PR is in progress. labels Oct 31, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/daemon Impacts operation of the Cilium daemon. backport-done/1.14 The backport for Cilium 1.14.x for this PR is done. backport-done/1.15 The backport for Cilium 1.15.x for this PR is done. backport-done/1.16 The backport for Cilium 1.16.x for this PR is done. kind/bug This is a bug in the Cilium logic. kind/cleanup This includes no functional changes. release-note/misc This PR makes changes that have no direct user impact.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

CI: Cilium E2E Upgrade - no-errors-in-logs - retrieving device lxc_health: Link not found
10 participants