Skip to content

CI: new CLI 0.8.6's L7 tests failures on EKS with IPsec enabled #17139

New issue
New issue
Closed
#22617
Closed
CI: new CLI 0.8.6's L7 tests failures on EKS with IPsec enabled#17139
#22617
Assignees
pchaigno
Labels
area/CIContinuous Integration testing issue or flakearea/encryptionImpacts encryption support such as IPSec, WireGuard, or kTLS.ci/flakeThis is a known failure that occurs in the tree. Please investigate me!integration/cloudRelated to integration with cloud environments such as AKS, EKS, GKE, etc.
@nbusseneau

Description

@nbusseneau
nbusseneau
opened on Aug 11, 2021

CLI 0.8.6 added new L7 tests coverage, which consistently fails on EKS with IPsec enabled (see below for logs) when it should be working.

It is probable this new coverage uncovered an undetected bug in Cilium.

Next steps

Behavior

https://github.com/cilium/cilium/actions/runs/1114941992
Sysdump: cilium-sysdump-out.zip.zip
Logs:

[=] Test [client-egress-l7]
....
  ℹ️  📜 Applying CiliumNetworkPolicy 'client-egress-only-dns' to namespace 'cilium-test'..
  ℹ️  📜 Applying CiliumNetworkPolicy 'client-egress-l7-http' to namespace 'cilium-test'..
  [-] Scenario [client-egress-l7/pod-to-pod]
  [.] Action [client-egress-l7/pod-to-pod/curl-0: cilium-test/client-7b7bf54b85-n594s (192.168.103.225) -> cilium-test/echo-other-node-697d5d69b7-4zrxz (192.168.157.144:8080)]
  📄 Following flows...
  [.] Action [client-egress-l7/pod-to-pod/curl-1: cilium-test/client-7b7bf54b85-n594s (192.168.103.225) -> cilium-test/echo-same-node-7967996674-zmn74 (192.168.122.247:8080)]
  📄 Following flows...
  [.] Action [client-egress-l7/pod-to-pod/curl-2: cilium-test/client2-666976c95b-vlkxn (192.168.145.252) -> cilium-test/echo-other-node-697d5d69b7-4zrxz (192.168.157.144:8080)]
  📄 Following flows...
  [.] Action [client-egress-l7/pod-to-pod/curl-3: cilium-test/client2-666976c95b-vlkxn (192.168.145.252) -> cilium-test/echo-same-node-7967996674-zmn74 (192.168.122.247:8080)]
  📄 Following flows...
  ❌ command "curl -w %{local_ip}:%{local_port} -> %{remote_ip}:%{remote_port} = %{response_code} --silent --fail --show-error --connect-timeout 5 --output /dev/null http://192.168.122.247:8080" failed: command terminated with exit code 22
  ℹ️  curl stderr:
  curl: (22) The requested URL returned error: 503
  
  📄 Flow logs for peer cilium-test/client2-666976c95b-vlkxn:
  ❓ [0] Aug 10 01:51:13.623: cilium-test/client2-666976c95b-vlkxn:56676 -> cilium-test/echo-same-node-7967996674-zmn74:8080 from-endpoint FORWARDED (TCP Flags: SYN)
  ❓ [1] Aug 10 01:51:13.623: cilium-test/client2-666976c95b-vlkxn:56676 -> cilium-test/echo-same-node-7967996674-zmn74:8080 L3-L4 FORWARDED (TCP Flags: SYN)
  ❓ [2] Aug 10 01:51:13.623: cilium-test/client2-666976c95b-vlkxn:56676 -> cilium-test/echo-same-node-7967996674-zmn74:8080 to-proxy FORWARDED (TCP Flags: SYN)
  ❓ [3] Aug 10 01:51:13.623: cilium-test/echo-same-node-7967996674-zmn74:8080 -> cilium-test/client2-666976c95b-vlkxn:56676 from-proxy FORWARDED (TCP Flags: SYN, ACK)
  ❓ [4] Aug 10 01:51:13.623: cilium-test/echo-same-node-7967996674-zmn74:8080 -> cilium-test/client2-666976c95b-vlkxn:56676 to-endpoint FORWARDED (TCP Flags: SYN, ACK)
  ❓ [5] Aug 10 01:51:13.623: cilium-test/client2-666976c95b-vlkxn:56676 -> cilium-test/echo-same-node-7967996674-zmn74:8080 from-endpoint FORWARDED (TCP Flags: ACK)
  ❓ [6] Aug 10 01:51:13.623: cilium-test/client2-666976c95b-vlkxn:56676 -> cilium-test/echo-same-node-7967996674-zmn74:8080 to-proxy FORWARDED (TCP Flags: ACK)
  ❓ [7] Aug 10 01:51:13.623: cilium-test/client2-666976c95b-vlkxn:56676 -> cilium-test/echo-same-node-7967996674-zmn74:8080 from-endpoint FORWARDED (TCP Flags: ACK, PSH)
  ❓ [8] Aug 10 01:51:13.623: cilium-test/client2-666976c95b-vlkxn:56676 -> cilium-test/echo-same-node-7967996674-zmn74:8080 to-proxy FORWARDED (TCP Flags: ACK, PSH)
  ❓ [9] Aug 10 01:51:13.623: cilium-test/echo-same-node-7967996674-zmn74:8080 -> cilium-test/client2-666976c95b-vlkxn:56676 from-proxy FORWARDED (TCP Flags: ACK)
  ❓ [10] Aug 10 01:51:13.623: cilium-test/echo-same-node-7967996674-zmn74:8080 -> cilium-test/client2-666976c95b-vlkxn:56676 to-endpoint FORWARDED (TCP Flags: ACK)
  ❓ [11] Aug 10 01:51:13.624: cilium-test/client2-666976c95b-vlkxn:56676 -> cilium-test/echo-same-node-7967996674-zmn74:8080 http-request FORWARDED (HTTP/1.1 GET http://192.168.122.247:8080/)
  ❓ [12] Aug 10 01:51:13.624: cilium-test/client2-666976c95b-vlkxn:56676 -> cilium-test/echo-same-node-7967996674-zmn74:8080 from-network FORWARDED (TCP Flags: SYN)
  ❓ [13] Aug 10 01:51:13.624: cilium-test/client2-666976c95b-vlkxn:56676 -> cilium-test/echo-same-node-7967996674-zmn74:8080 from-stack FORWARDED (TCP Flags: SYN)
  ❓ [14] Aug 10 01:51:13.624: cilium-test/client2-666976c95b-vlkxn:56676 -> cilium-test/echo-same-node-7967996674-zmn74:8080 to-endpoint FORWARDED (TCP Flags: SYN)
  ❓ [15] Aug 10 01:51:13.624: cilium-test/echo-same-node-7967996674-zmn74:8080 -> cilium-test/client2-666976c95b-vlkxn:56676 from-endpoint FORWARDED (TCP Flags: SYN, ACK)
  ❓ [16] Aug 10 01:51:13.624: cilium-test/echo-same-node-7967996674-zmn74:8080 -> cilium-test/client2-666976c95b-vlkxn:56676 to-stack FORWARDED (TCP Flags: SYN, ACK)
  ❓ [17] Aug 10 01:51:13.625: cilium-test/echo-same-node-7967996674-zmn74:8080 -> cilium-test/client2-666976c95b-vlkxn:56676 from-network FORWARDED (TCP Flags: SYN, ACK)
  ❓ [18] Aug 10 01:51:14.636: cilium-test/client2-666976c95b-vlkxn:56676 -> cilium-test/echo-same-node-7967996674-zmn74:8080 from-network FORWARDED (TCP Flags: SYN)
  ❓ [19] Aug 10 01:51:14.636: cilium-test/client2-666976c95b-vlkxn:56676 -> cilium-test/echo-same-node-7967996674-zmn74:8080 from-stack FORWARDED (TCP Flags: SYN)
  ❓ [20] Aug 10 01:51:14.636: cilium-test/client2-666976c95b-vlkxn:56676 -> cilium-test/echo-same-node-7967996674-zmn74:8080 to-endpoint FORWARDED (TCP Flags: SYN)
  ❓ [21] Aug 10 01:51:14.636: cilium-test/echo-same-node-7967996674-zmn74:8080 -> cilium-test/client2-666976c95b-vlkxn:56676 from-endpoint FORWARDED (TCP Flags: ACK)
  ❓ [22] Aug 10 01:51:14.636: cilium-test/echo-same-node-7967996674-zmn74:8080 -> cilium-test/client2-666976c95b-vlkxn:56676 to-stack FORWARDED (TCP Flags: ACK)
  ❓ [23] Aug 10 01:51:14.637: cilium-test/echo-same-node-7967996674-zmn74:8080 -> cilium-test/client2-666976c95b-vlkxn:56676 from-network FORWARDED (TCP Flags: ACK)
  ❓ [24] Aug 10 01:51:14.637: cilium-test/echo-same-node-7967996674-zmn74:8080 -> cilium-test/client2-666976c95b-vlkxn:56676 from-endpoint FORWARDED (TCP Flags: SYN, ACK)
  ❓ [25] Aug 10 01:51:14.637: cilium-test/echo-same-node-7967996674-zmn74:8080 -> cilium-test/client2-666976c95b-vlkxn:56676 to-stack FORWARDED (TCP Flags: SYN, ACK)
  ❓ [26] Aug 10 01:51:14.638: cilium-test/echo-same-node-7967996674-zmn74:8080 -> cilium-test/client2-666976c95b-vlkxn:56676 from-network FORWARDED (TCP Flags: SYN, ACK)
  ❓ [27] Aug 10 01:51:15.665: cilium-test/client2-666976c95b-vlkxn:56676 -> cilium-test/echo-same-node-7967996674-zmn74:8080 from-network FORWARDED (TCP Flags: SYN)
  ❓ [28] Aug 10 01:51:15.665: cilium-test/client2-666976c95b-vlkxn:56676 -> cilium-test/echo-same-node-7967996674-zmn74:8080 from-stack FORWARDED (TCP Flags: SYN)
  ❓ [29] Aug 10 01:51:15.665: cilium-test/client2-666976c95b-vlkxn:56676 -> cilium-test/echo-same-node-7967996674-zmn74:8080 to-endpoint FORWARDED (TCP Flags: SYN)
  ❓ [30] Aug 10 01:51:15.665: cilium-test/echo-same-node-7967996674-zmn74:8080 -> cilium-test/client2-666976c95b-vlkxn:56676 from-endpoint FORWARDED (TCP Flags: ACK)
  ❓ [31] Aug 10 01:51:15.665: cilium-test/echo-same-node-7967996674-zmn74:8080 -> cilium-test/client2-666976c95b-vlkxn:56676 to-stack FORWARDED (TCP Flags: ACK)
  ❓ [32] Aug 10 01:51:15.665: cilium-test/echo-same-node-7967996674-zmn74:8080 -> cilium-test/client2-666976c95b-vlkxn:56676 from-network FORWARDED (TCP Flags: ACK)
  ❓ [33] Aug 10 01:51:16.654: cilium-test/echo-same-node-7967996674-zmn74:8080 -> cilium-test/client2-666976c95b-vlkxn:56676 from-endpoint FORWARDED (TCP Flags: SYN, ACK)
  ❓ [34] Aug 10 01:51:16.654: cilium-test/echo-same-node-7967996674-zmn74:8080 -> cilium-test/client2-666976c95b-vlkxn:56676 to-stack FORWARDED (TCP Flags: SYN, ACK)
  ❓ [35] Aug 10 01:51:16.654: cilium-test/echo-same-node-7967996674-zmn74:8080 -> cilium-test/client2-666976c95b-vlkxn:56676 from-network FORWARDED (TCP Flags: SYN, ACK)
  ❓ [36] Aug 10 01:51:16.694: cilium-test/client2-666976c95b-vlkxn:56676 -> cilium-test/echo-same-node-7967996674-zmn74:8080 from-network FORWARDED (TCP Flags: SYN)
  ❓ [37] Aug 10 01:51:16.694: cilium-test/client2-666976c95b-vlkxn:56676 -> cilium-test/echo-same-node-7967996674-zmn74:8080 from-stack FORWARDED (TCP Flags: SYN)
  ❓ [38] Aug 10 01:51:16.694: cilium-test/client2-666976c95b-vlkxn:56676 -> cilium-test/echo-same-node-7967996674-zmn74:8080 to-endpoint FORWARDED (TCP Flags: SYN)
  ❓ [39] Aug 10 01:51:16.694: cilium-test/echo-same-node-7967996674-zmn74:8080 -> cilium-test/client2-666976c95b-vlkxn:56676 from-endpoint FORWARDED (TCP Flags: ACK)
  ❓ [40] Aug 10 01:51:16.694: cilium-test/echo-same-node-7967996674-zmn74:8080 -> cilium-test/client2-666976c95b-vlkxn:56676 to-stack FORWARDED (TCP Flags: ACK)
  ❓ [41] Aug 10 01:51:16.694: cilium-test/echo-same-node-7967996674-zmn74:8080 -> cilium-test/client2-666976c95b-vlkxn:56676 from-network FORWARDED (TCP Flags: ACK)
  
  📄 Flow logs for peer cilium-test/echo-same-node-7967996674-zmn74:
  ❓ [0] Aug 10 01:51:13.623: cilium-test/client2-666976c95b-vlkxn:56676 -> cilium-test/echo-same-node-7967996674-zmn74:8080 from-endpoint FORWARDED (TCP Flags: SYN)
  ❓ [1] Aug 10 01:51:13.623: cilium-test/client2-666976c95b-vlkxn:56676 -> cilium-test/echo-same-node-7967996674-zmn74:8080 L3-L4 FORWARDED (TCP Flags: SYN)
  ❓ [2] Aug 10 01:51:13.623: cilium-test/client2-666976c95b-vlkxn:56676 -> cilium-test/echo-same-node-7967996674-zmn74:8080 to-proxy FORWARDED (TCP Flags: SYN)
  ❓ [3] Aug 10 01:51:13.623: cilium-test/echo-same-node-7967996674-zmn74:8080 -> cilium-test/client2-666976c95b-vlkxn:56676 from-proxy FORWARDED (TCP Flags: SYN, ACK)
  ❓ [4] Aug 10 01:51:13.623: cilium-test/echo-same-node-7967996674-zmn74:8080 -> cilium-test/client2-666976c95b-vlkxn:56676 to-endpoint FORWARDED (TCP Flags: SYN, ACK)
  ❓ [5] Aug 10 01:51:13.623: cilium-test/client2-666976c95b-vlkxn:56676 -> cilium-test/echo-same-node-7967996674-zmn74:8080 from-endpoint FORWARDED (TCP Flags: ACK)
  ❓ [6] Aug 10 01:51:13.623: cilium-test/client2-666976c95b-vlkxn:56676 -> cilium-test/echo-same-node-7967996674-zmn74:8080 to-proxy FORWARDED (TCP Flags: ACK)
  ❓ [7] Aug 10 01:51:13.623: cilium-test/client2-666976c95b-vlkxn:56676 -> cilium-test/echo-same-node-7967996674-zmn74:8080 from-endpoint FORWARDED (TCP Flags: ACK, PSH)
  ❓ [8] Aug 10 01:51:13.623: cilium-test/client2-666976c95b-vlkxn:56676 -> cilium-test/echo-same-node-7967996674-zmn74:8080 to-proxy FORWARDED (TCP Flags: ACK, PSH)
  ❓ [9] Aug 10 01:51:13.623: cilium-test/echo-same-node-7967996674-zmn74:8080 -> cilium-test/client2-666976c95b-vlkxn:56676 from-proxy FORWARDED (TCP Flags: ACK)
  ❓ [10] Aug 10 01:51:13.623: cilium-test/echo-same-node-7967996674-zmn74:8080 -> cilium-test/client2-666976c95b-vlkxn:56676 to-endpoint FORWARDED (TCP Flags: ACK)
  ❓ [11] Aug 10 01:51:13.624: cilium-test/client2-666976c95b-vlkxn:56676 -> cilium-test/echo-same-node-7967996674-zmn74:8080 http-request FORWARDED (HTTP/1.1 GET http://192.168.122.247:8080/)
  ❓ [12] Aug 10 01:51:13.624: cilium-test/client2-666976c95b-vlkxn:56676 -> cilium-test/echo-same-node-7967996674-zmn74:8080 from-network FORWARDED (TCP Flags: SYN)
  ❓ [13] Aug 10 01:51:13.624: cilium-test/client2-666976c95b-vlkxn:56676 -> cilium-test/echo-same-node-7967996674-zmn74:8080 from-stack FORWARDED (TCP Flags: SYN)
  ❓ [14] Aug 10 01:51:13.624: cilium-test/client2-666976c95b-vlkxn:56676 -> cilium-test/echo-same-node-7967996674-zmn74:8080 to-endpoint FORWARDED (TCP Flags: SYN)
  ❓ [15] Aug 10 01:51:13.624: cilium-test/echo-same-node-7967996674-zmn74:8080 -> cilium-test/client2-666976c95b-vlkxn:56676 from-endpoint FORWARDED (TCP Flags: SYN, ACK)
  ❓ [16] Aug 10 01:51:13.624: cilium-test/echo-same-node-7967996674-zmn74:8080 -> cilium-test/client2-666976c95b-vlkxn:56676 to-stack FORWARDED (TCP Flags: SYN, ACK)
  ❓ [17] Aug 10 01:51:13.625: cilium-test/echo-same-node-7967996674-zmn74:8080 -> cilium-test/client2-666976c95b-vlkxn:56676 from-network FORWARDED (TCP Flags: SYN, ACK)
  ❓ [18] Aug 10 01:51:14.636: cilium-test/client2-666976c95b-vlkxn:56676 -> cilium-test/echo-same-node-7967996674-zmn74:8080 from-network FORWARDED (TCP Flags: SYN)
  ❓ [19] Aug 10 01:51:14.636: cilium-test/client2-666976c95b-vlkxn:56676 -> cilium-test/echo-same-node-7967996674-zmn74:8080 from-stack FORWARDED (TCP Flags: SYN)
  ❓ [20] Aug 10 01:51:14.636: cilium-test/client2-666976c95b-vlkxn:56676 -> cilium-test/echo-same-node-7967996674-zmn74:8080 to-endpoint FORWARDED (TCP Flags: SYN)
  ❓ [21] Aug 10 01:51:14.636: cilium-test/echo-same-node-7967996674-zmn74:8080 -> cilium-test/client2-666976c95b-vlkxn:56676 from-endpoint FORWARDED (TCP Flags: ACK)
  ❓ [22] Aug 10 01:51:14.636: cilium-test/echo-same-node-7967996674-zmn74:8080 -> cilium-test/client2-666976c95b-vlkxn:56676 to-stack FORWARDED (TCP Flags: ACK)
  ❓ [23] Aug 10 01:51:14.637: cilium-test/echo-same-node-7967996674-zmn74:8080 -> cilium-test/client2-666976c95b-vlkxn:56676 from-network FORWARDED (TCP Flags: ACK)
  ❓ [24] Aug 10 01:51:14.637: cilium-test/echo-same-node-7967996674-zmn74:8080 -> cilium-test/client2-666976c95b-vlkxn:56676 from-endpoint FORWARDED (TCP Flags: SYN, ACK)
  ❓ [25] Aug 10 01:51:14.637: cilium-test/echo-same-node-7967996674-zmn74:8080 -> cilium-test/client2-666976c95b-vlkxn:56676 to-stack FORWARDED (TCP Flags: SYN, ACK)
  ❓ [26] Aug 10 01:51:14.638: cilium-test/echo-same-node-7967996674-zmn74:8080 -> cilium-test/client2-666976c95b-vlkxn:56676 from-network FORWARDED (TCP Flags: SYN, ACK)
  ❓ [27] Aug 10 01:51:15.665: cilium-test/client2-666976c95b-vlkxn:56676 -> cilium-test/echo-same-node-7967996674-zmn74:8080 from-network FORWARDED (TCP Flags: SYN)
  ❓ [28] Aug 10 01:51:15.665: cilium-test/client2-666976c95b-vlkxn:56676 -> cilium-test/echo-same-node-7967996674-zmn74:8080 from-stack FORWARDED (TCP Flags: SYN)
  ❓ [29] Aug 10 01:51:15.665: cilium-test/client2-666976c95b-vlkxn:56676 -> cilium-test/echo-same-node-7967996674-zmn74:8080 to-endpoint FORWARDED (TCP Flags: SYN)
  ❓ [30] Aug 10 01:51:15.665: cilium-test/echo-same-node-7967996674-zmn74:8080 -> cilium-test/client2-666976c95b-vlkxn:56676 from-endpoint FORWARDED (TCP Flags: ACK)
  ❓ [31] Aug 10 01:51:15.665: cilium-test/echo-same-node-7967996674-zmn74:8080 -> cilium-test/client2-666976c95b-vlkxn:56676 to-stack FORWARDED (TCP Flags: ACK)
  ❓ [32] Aug 10 01:51:15.665: cilium-test/echo-same-node-7967996674-zmn74:8080 -> cilium-test/client2-666976c95b-vlkxn:56676 from-network FORWARDED (TCP Flags: ACK)
  ❓ [33] Aug 10 01:51:16.654: cilium-test/echo-same-node-7967996674-zmn74:8080 -> cilium-test/client2-666976c95b-vlkxn:56676 from-endpoint FORWARDED (TCP Flags: SYN, ACK)
  ❓ [34] Aug 10 01:51:16.654: cilium-test/echo-same-node-7967996674-zmn74:8080 -> cilium-test/client2-666976c95b-vlkxn:56676 to-stack FORWARDED (TCP Flags: SYN, ACK)
  ❓ [35] Aug 10 01:51:16.654: cilium-test/echo-same-node-7967996674-zmn74:8080 -> cilium-test/client2-666976c95b-vlkxn:56676 from-network FORWARDED (TCP Flags: SYN, ACK)
  ❓ [36] Aug 10 01:51:16.694: cilium-test/client2-666976c95b-vlkxn:56676 -> cilium-test/echo-same-node-7967996674-zmn74:8080 from-network FORWARDED (TCP Flags: SYN)
  ❓ [37] Aug 10 01:51:16.694: cilium-test/client2-666976c95b-vlkxn:56676 -> cilium-test/echo-same-node-7967996674-zmn74:8080 from-stack FORWARDED (TCP Flags: SYN)
  ❓ [38] Aug 10 01:51:16.694: cilium-test/client2-666976c95b-vlkxn:56676 -> cilium-test/echo-same-node-7967996674-zmn74:8080 to-endpoint FORWARDED (TCP Flags: SYN)
  ❓ [39] Aug 10 01:51:16.694: cilium-test/echo-same-node-7967996674-zmn74:8080 -> cilium-test/client2-666976c95b-vlkxn:56676 from-endpoint FORWARDED (TCP Flags: ACK)
  ❓ [40] Aug 10 01:51:16.694: cilium-test/echo-same-node-7967996674-zmn74:8080 -> cilium-test/client2-666976c95b-vlkxn:56676 to-stack FORWARDED (TCP Flags: ACK)
  ❓ [41] Aug 10 01:51:16.694: cilium-test/echo-same-node-7967996674-zmn74:8080 -> cilium-test/client2-666976c95b-vlkxn:56676 from-network FORWARDED (TCP Flags: ACK)
  
  [-] Scenario [client-egress-l7/pod-to-world]
  [.] Action [client-egress-l7/pod-to-world/https-to-cilium-io-0: cilium-test/client-7b7bf54b85-n594s (192.168.103.225) -> cilium-io-https (cilium.io:443)]
  📄 Following flows...
  [.] Action [client-egress-l7/pod-to-world/http-to-cilium-io-0: cilium-test/client-7b7bf54b85-n594s (192.168.103.225) -> cilium-io-http (cilium.io:80)]
  📄 Following flows...
  [.] Action [client-egress-l7/pod-to-world/http-to-jenkins-cilium-0: cilium-test/client-7b7bf54b85-n594s (192.168.103.225) -> jenkins-cilium-io-http (jenkins.cilium.io:80)]
  📄 Following flows...
  [.] Action [client-egress-l7/pod-to-world/https-to-cilium-io-1: cilium-test/client2-666976c95b-vlkxn (192.168.145.252) -> cilium-io-https (cilium.io:443)]
  📄 Following flows...
  [.] Action [client-egress-l7/pod-to-world/http-to-cilium-io-1: cilium-test/client2-666976c95b-vlkxn (192.168.145.252) -> cilium-io-http (cilium.io:80)]
  📄 Following flows...
  [.] Action [client-egress-l7/pod-to-world/http-to-jenkins-cilium-1: cilium-test/client2-666976c95b-vlkxn (192.168.145.252) -> jenkins-cilium-io-http (jenkins.cilium.io:80)]
  📄 Following flows...
  ℹ️  📜 Deleting CiliumNetworkPolicy 'client-egress-only-dns' from namespace 'cilium-test'..
  ℹ️  📜 Deleting CiliumNetworkPolicy 'client-egress-l7-http' from namespace 'cilium-test'..

[=] Test [echo-ingress-l7]
....
  ℹ️  📜 Applying CiliumNetworkPolicy 'echo-ingress-l7-http' to namespace 'cilium-test'..
  [-] Scenario [echo-ingress-l7/pod-to-pod]
  [.] Action [echo-ingress-l7/pod-to-pod/curl-0: cilium-test/client-7b7bf54b85-n594s (192.168.103.225) -> cilium-test/echo-other-node-697d5d69b7-4zrxz (192.168.157.144:8080)]
  📄 Following flows...
  [.] Action [echo-ingress-l7/pod-to-pod/curl-1: cilium-test/client-7b7bf54b85-n594s (192.168.103.225) -> cilium-test/echo-same-node-7967996674-zmn74 (192.168.122.247:8080)]
  📄 Following flows...
  [.] Action [echo-ingress-l7/pod-to-pod/curl-2: cilium-test/client2-666976c95b-vlkxn (192.168.145.252) -> cilium-test/echo-other-node-697d5d69b7-4zrxz (192.168.157.144:8080)]
  📄 Following flows...
  [.] Action [echo-ingress-l7/pod-to-pod/curl-3: cilium-test/client2-666976c95b-vlkxn (192.168.145.252) -> cilium-test/echo-same-node-7967996674-zmn74 (192.168.122.247:8080)]
  📄 Following flows...
  ❌ command "curl -w %{local_ip}:%{local_port} -> %{remote_ip}:%{remote_port} = %{response_code} --silent --fail --show-error --connect-timeout 5 --output /dev/null http://192.168.122.247:8080" failed: command terminated with exit code 52
  ℹ️  curl stderr:
  curl: (52) Empty reply from server
  
  📄 Flow logs for peer cilium-test/client2-666976c95b-vlkxn:
  ❓ [0] Aug 10 01:52:03.426: cilium-test/client2-666976c95b-vlkxn:57048 -> cilium-test/echo-same-node-7967996674-zmn74:8080 from-endpoint FORWARDED (TCP Flags: SYN)
  ❓ [1] Aug 10 01:52:03.426: cilium-test/client2-666976c95b-vlkxn:57048 -> cilium-test/echo-same-node-7967996674-zmn74:8080 to-stack FORWARDED (TCP Flags: SYN)
  ❓ [2] Aug 10 01:52:03.427: cilium-test/client2-666976c95b-vlkxn:57048 -> cilium-test/echo-same-node-7967996674-zmn74:8080 from-network FORWARDED (TCP Flags: SYN)
  ❓ [3] Aug 10 01:52:03.427: cilium-test/client2-666976c95b-vlkxn:57048 -> cilium-test/echo-same-node-7967996674-zmn74:8080 from-stack FORWARDED (TCP Flags: SYN)
  ❓ [4] Aug 10 01:52:03.427: cilium-test/client2-666976c95b-vlkxn:57048 -> cilium-test/echo-same-node-7967996674-zmn74:8080 L3-L4 FORWARDED (TCP Flags: SYN)
  ❓ [5] Aug 10 01:52:03.427: cilium-test/client2-666976c95b-vlkxn:57048 -> cilium-test/echo-same-node-7967996674-zmn74:8080 to-proxy FORWARDED (TCP Flags: SYN)
  ❓ [6] Aug 10 01:52:03.427: cilium-test/echo-same-node-7967996674-zmn74:8080 -> cilium-test/client2-666976c95b-vlkxn:57048 from-network FORWARDED (TCP Flags: SYN, ACK)
  ❓ [7] Aug 10 01:52:03.427: cilium-test/echo-same-node-7967996674-zmn74:8080 -> cilium-test/client2-666976c95b-vlkxn:57048 from-stack FORWARDED (TCP Flags: SYN, ACK)
  ❓ [8] Aug 10 01:52:03.427: cilium-test/echo-same-node-7967996674-zmn74:8080 -> cilium-test/client2-666976c95b-vlkxn:57048 to-endpoint FORWARDED (TCP Flags: SYN, ACK)
  ❓ [9] Aug 10 01:52:03.427: cilium-test/client2-666976c95b-vlkxn:57048 -> cilium-test/echo-same-node-7967996674-zmn74:8080 from-endpoint FORWARDED (TCP Flags: ACK)
  ❓ [10] Aug 10 01:52:03.427: cilium-test/client2-666976c95b-vlkxn:57048 -> cilium-test/echo-same-node-7967996674-zmn74:8080 to-stack FORWARDED (TCP Flags: ACK)
  ❓ [11] Aug 10 01:52:03.428: cilium-test/client2-666976c95b-vlkxn:57048 -> cilium-test/echo-same-node-7967996674-zmn74:8080 from-network FORWARDED (TCP Flags: ACK)
  ❓ [12] Aug 10 01:52:03.428: cilium-test/client2-666976c95b-vlkxn:57048 -> cilium-test/echo-same-node-7967996674-zmn74:8080 from-endpoint FORWARDED (TCP Flags: ACK, PSH)
  ❓ [13] Aug 10 01:52:03.428: cilium-test/client2-666976c95b-vlkxn:57048 -> cilium-test/echo-same-node-7967996674-zmn74:8080 to-stack FORWARDED (TCP Flags: ACK, PSH)
  ❓ [14] Aug 10 01:52:03.428: cilium-test/client2-666976c95b-vlkxn:57048 -> cilium-test/echo-same-node-7967996674-zmn74:8080 from-network FORWARDED (TCP Flags: ACK, PSH)
  ❓ [15] Aug 10 01:52:03.634: cilium-test/client2-666976c95b-vlkxn:57048 -> cilium-test/echo-same-node-7967996674-zmn74:8080 from-endpoint FORWARDED (TCP Flags: ACK, PSH)
  ❓ [16] Aug 10 01:52:03.634: cilium-test/client2-666976c95b-vlkxn:57048 -> cilium-test/echo-same-node-7967996674-zmn74:8080 to-stack FORWARDED (TCP Flags: ACK, PSH)
  ❓ [17] Aug 10 01:52:03.634: cilium-test/client2-666976c95b-vlkxn:57048 -> cilium-test/echo-same-node-7967996674-zmn74:8080 from-network FORWARDED (TCP Flags: ACK, PSH)
  ❓ [18] Aug 10 01:52:03.842: cilium-test/client2-666976c95b-vlkxn:57048 -> cilium-test/echo-same-node-7967996674-zmn74:8080 from-endpoint FORWARDED (TCP Flags: ACK, PSH)
  ❓ [19] Aug 10 01:52:03.842: cilium-test/client2-666976c95b-vlkxn:57048 -> cilium-test/echo-same-node-7967996674-zmn74:8080 to-stack FORWARDED (TCP Flags: ACK, PSH)
  ❓ [20] Aug 10 01:52:03.842: cilium-test/client2-666976c95b-vlkxn:57048 -> cilium-test/echo-same-node-7967996674-zmn74:8080 from-network FORWARDED (TCP Flags: ACK, PSH)
  ❓ [21] Aug 10 01:52:04.269: cilium-test/client2-666976c95b-vlkxn:57048 -> cilium-test/echo-same-node-7967996674-zmn74:8080 from-endpoint FORWARDED (TCP Flags: ACK, PSH)
  ❓ [22] Aug 10 01:52:04.270: cilium-test/client2-666976c95b-vlkxn:57048 -> cilium-test/echo-same-node-7967996674-zmn74:8080 to-stack FORWARDED (TCP Flags: ACK, PSH)
  ❓ [23] Aug 10 01:52:04.270: cilium-test/client2-666976c95b-vlkxn:57048 -> cilium-test/echo-same-node-7967996674-zmn74:8080 from-network FORWARDED (TCP Flags: ACK, PSH)
  ❓ [24] Aug 10 01:52:05.101: cilium-test/client2-666976c95b-vlkxn:57048 -> cilium-test/echo-same-node-7967996674-zmn74:8080 from-endpoint FORWARDED (TCP Flags: ACK, PSH)
  ❓ [25] Aug 10 01:52:05.101: cilium-test/client2-666976c95b-vlkxn:57048 -> cilium-test/echo-same-node-7967996674-zmn74:8080 to-stack FORWARDED (TCP Flags: ACK, PSH)
  ❓ [26] Aug 10 01:52:05.102: cilium-test/client2-666976c95b-vlkxn:57048 -> cilium-test/echo-same-node-7967996674-zmn74:8080 from-network FORWARDED (TCP Flags: ACK, PSH)
  ❓ [27] Aug 10 01:52:06.765: cilium-test/client2-666976c95b-vlkxn:57048 -> cilium-test/echo-same-node-7967996674-zmn74:8080 from-endpoint FORWARDED (TCP Flags: ACK, PSH)
  ❓ [28] Aug 10 01:52:06.765: cilium-test/client2-666976c95b-vlkxn:57048 -> cilium-test/echo-same-node-7967996674-zmn74:8080 to-stack FORWARDED (TCP Flags: ACK, PSH)
  ❓ [29] Aug 10 01:52:06.766: cilium-test/client2-666976c95b-vlkxn:57048 -> cilium-test/echo-same-node-7967996674-zmn74:8080 from-network FORWARDED (TCP Flags: ACK, PSH)
  ❓ [30] Aug 10 01:52:10.189: cilium-test/client2-666976c95b-vlkxn:57048 -> cilium-test/echo-same-node-7967996674-zmn74:8080 from-endpoint FORWARDED (TCP Flags: ACK, PSH)
  ❓ [31] Aug 10 01:52:10.189: cilium-test/client2-666976c95b-vlkxn:57048 -> cilium-test/echo-same-node-7967996674-zmn74:8080 to-stack FORWARDED (TCP Flags: ACK, PSH)
  ❓ [32] Aug 10 01:52:10.190: cilium-test/client2-666976c95b-vlkxn:57048 -> cilium-test/echo-same-node-7967996674-zmn74:8080 from-network FORWARDED (TCP Flags: ACK, PSH)
  ❓ [33] Aug 10 01:52:16.845: cilium-test/client2-666976c95b-vlkxn:57048 -> cilium-test/echo-same-node-7967996674-zmn74:8080 from-endpoint FORWARDED (TCP Flags: ACK, PSH)
  ❓ [34] Aug 10 01:52:16.845: cilium-test/client2-666976c95b-vlkxn:57048 -> cilium-test/echo-same-node-7967996674-zmn74:8080 to-stack FORWARDED (TCP Flags: ACK, PSH)
  ❓ [35] Aug 10 01:52:16.846: cilium-test/client2-666976c95b-vlkxn:57048 -> cilium-test/echo-same-node-7967996674-zmn74:8080 from-network FORWARDED (TCP Flags: ACK, PSH)
  
  📄 Flow logs for peer cilium-test/echo-same-node-7967996674-zmn74:
  ❓ [0] Aug 10 01:52:03.426: cilium-test/client2-666976c95b-vlkxn:57048 -> cilium-test/echo-same-node-7967996674-zmn74:8080 from-endpoint FORWARDED (TCP Flags: SYN)
  ❓ [1] Aug 10 01:52:03.426: cilium-test/client2-666976c95b-vlkxn:57048 -> cilium-test/echo-same-node-7967996674-zmn74:8080 to-stack FORWARDED (TCP Flags: SYN)
  ❓ [2] Aug 10 01:52:03.427: cilium-test/client2-666976c95b-vlkxn:57048 -> cilium-test/echo-same-node-7967996674-zmn74:8080 from-network FORWARDED (TCP Flags: SYN)
  ❓ [3] Aug 10 01:52:03.427: cilium-test/client2-666976c95b-vlkxn:57048 -> cilium-test/echo-same-node-7967996674-zmn74:8080 from-stack FORWARDED (TCP Flags: SYN)
  ❓ [4] Aug 10 01:52:03.427: cilium-test/client2-666976c95b-vlkxn:57048 -> cilium-test/echo-same-node-7967996674-zmn74:8080 L3-L4 FORWARDED (TCP Flags: SYN)
  ❓ [5] Aug 10 01:52:03.427: cilium-test/client2-666976c95b-vlkxn:57048 -> cilium-test/echo-same-node-7967996674-zmn74:8080 to-proxy FORWARDED (TCP Flags: SYN)
  ❓ [6] Aug 10 01:52:03.427: cilium-test/echo-same-node-7967996674-zmn74:8080 -> cilium-test/client2-666976c95b-vlkxn:57048 from-network FORWARDED (TCP Flags: SYN, ACK)
  ❓ [7] Aug 10 01:52:03.427: cilium-test/echo-same-node-7967996674-zmn74:8080 -> cilium-test/client2-666976c95b-vlkxn:57048 from-stack FORWARDED (TCP Flags: SYN, ACK)
  ❓ [8] Aug 10 01:52:03.427: cilium-test/echo-same-node-7967996674-zmn74:8080 -> cilium-test/client2-666976c95b-vlkxn:57048 to-endpoint FORWARDED (TCP Flags: SYN, ACK)
  ❓ [9] Aug 10 01:52:03.427: cilium-test/client2-666976c95b-vlkxn:57048 -> cilium-test/echo-same-node-7967996674-zmn74:8080 from-endpoint FORWARDED (TCP Flags: ACK)
  ❓ [10] Aug 10 01:52:03.427: cilium-test/client2-666976c95b-vlkxn:57048 -> cilium-test/echo-same-node-7967996674-zmn74:8080 to-stack FORWARDED (TCP Flags: ACK)
  ❓ [11] Aug 10 01:52:03.428: cilium-test/client2-666976c95b-vlkxn:57048 -> cilium-test/echo-same-node-7967996674-zmn74:8080 from-network FORWARDED (TCP Flags: ACK)
  ❓ [12] Aug 10 01:52:03.428: cilium-test/client2-666976c95b-vlkxn:57048 -> cilium-test/echo-same-node-7967996674-zmn74:8080 from-endpoint FORWARDED (TCP Flags: ACK, PSH)
  ❓ [13] Aug 10 01:52:03.428: cilium-test/client2-666976c95b-vlkxn:57048 -> cilium-test/echo-same-node-7967996674-zmn74:8080 to-stack FORWARDED (TCP Flags: ACK, PSH)
  ❓ [14] Aug 10 01:52:03.428: cilium-test/client2-666976c95b-vlkxn:57048 -> cilium-test/echo-same-node-7967996674-zmn74:8080 from-network FORWARDED (TCP Flags: ACK, PSH)
  ❓ [15] Aug 10 01:52:03.634: cilium-test/client2-666976c95b-vlkxn:57048 -> cilium-test/echo-same-node-7967996674-zmn74:8080 from-endpoint FORWARDED (TCP Flags: ACK, PSH)
  ❓ [16] Aug 10 01:52:03.634: cilium-test/client2-666976c95b-vlkxn:57048 -> cilium-test/echo-same-node-7967996674-zmn74:8080 to-stack FORWARDED (TCP Flags: ACK, PSH)
  ❓ [17] Aug 10 01:52:03.634: cilium-test/client2-666976c95b-vlkxn:57048 -> cilium-test/echo-same-node-7967996674-zmn74:8080 from-network FORWARDED (TCP Flags: ACK, PSH)
  ❓ [18] Aug 10 01:52:03.842: cilium-test/client2-666976c95b-vlkxn:57048 -> cilium-test/echo-same-node-7967996674-zmn74:8080 from-endpoint FORWARDED (TCP Flags: ACK, PSH)
  ❓ [19] Aug 10 01:52:03.842: cilium-test/client2-666976c95b-vlkxn:57048 -> cilium-test/echo-same-node-7967996674-zmn74:8080 to-stack FORWARDED (TCP Flags: ACK, PSH)
  ❓ [20] Aug 10 01:52:03.842: cilium-test/client2-666976c95b-vlkxn:57048 -> cilium-test/echo-same-node-7967996674-zmn74:8080 from-network FORWARDED (TCP Flags: ACK, PSH)
  ❓ [21] Aug 10 01:52:04.269: cilium-test/client2-666976c95b-vlkxn:57048 -> cilium-test/echo-same-node-7967996674-zmn74:8080 from-endpoint FORWARDED (TCP Flags: ACK, PSH)
  ❓ [22] Aug 10 01:52:04.270: cilium-test/client2-666976c95b-vlkxn:57048 -> cilium-test/echo-same-node-7967996674-zmn74:8080 to-stack FORWARDED (TCP Flags: ACK, PSH)
  ❓ [23] Aug 10 01:52:04.270: cilium-test/client2-666976c95b-vlkxn:57048 -> cilium-test/echo-same-node-7967996674-zmn74:8080 from-network FORWARDED (TCP Flags: ACK, PSH)
  ❓ [24] Aug 10 01:52:05.101: cilium-test/client2-666976c95b-vlkxn:57048 -> cilium-test/echo-same-node-7967996674-zmn74:8080 from-endpoint FORWARDED (TCP Flags: ACK, PSH)
  ❓ [25] Aug 10 01:52:05.101: cilium-test/client2-666976c95b-vlkxn:57048 -> cilium-test/echo-same-node-7967996674-zmn74:8080 to-stack FORWARDED (TCP Flags: ACK, PSH)
  ❓ [26] Aug 10 01:52:05.102: cilium-test/client2-666976c95b-vlkxn:57048 -> cilium-test/echo-same-node-7967996674-zmn74:8080 from-network FORWARDED (TCP Flags: ACK, PSH)
  ❓ [27] Aug 10 01:52:06.765: cilium-test/client2-666976c95b-vlkxn:57048 -> cilium-test/echo-same-node-7967996674-zmn74:8080 from-endpoint FORWARDED (TCP Flags: ACK, PSH)
  ❓ [28] Aug 10 01:52:06.765: cilium-test/client2-666976c95b-vlkxn:57048 -> cilium-test/echo-same-node-7967996674-zmn74:8080 to-stack FORWARDED (TCP Flags: ACK, PSH)
  ❓ [29] Aug 10 01:52:06.766: cilium-test/client2-666976c95b-vlkxn:57048 -> cilium-test/echo-same-node-7967996674-zmn74:8080 from-network FORWARDED (TCP Flags: ACK, PSH)
  ❓ [30] Aug 10 01:52:10.189: cilium-test/client2-666976c95b-vlkxn:57048 -> cilium-test/echo-same-node-7967996674-zmn74:8080 from-endpoint FORWARDED (TCP Flags: ACK, PSH)
  ❓ [31] Aug 10 01:52:10.189: cilium-test/client2-666976c95b-vlkxn:57048 -> cilium-test/echo-same-node-7967996674-zmn74:8080 to-stack FORWARDED (TCP Flags: ACK, PSH)
  ❓ [32] Aug 10 01:52:10.190: cilium-test/client2-666976c95b-vlkxn:57048 -> cilium-test/echo-same-node-7967996674-zmn74:8080 from-network FORWARDED (TCP Flags: ACK, PSH)
  ❓ [33] Aug 10 01:52:16.845: cilium-test/client2-666976c95b-vlkxn:57048 -> cilium-test/echo-same-node-7967996674-zmn74:8080 from-endpoint FORWARDED (TCP Flags: ACK, PSH)
  ❓ [34] Aug 10 01:52:16.845: cilium-test/client2-666976c95b-vlkxn:57048 -> cilium-test/echo-same-node-7967996674-zmn74:8080 to-stack FORWARDED (TCP Flags: ACK, PSH)
  ❓ [35] Aug 10 01:52:16.846: cilium-test/client2-666976c95b-vlkxn:57048 -> cilium-test/echo-same-node-7967996674-zmn74:8080 from-network FORWARDED (TCP Flags: ACK, PSH)
  
  ℹ️  📜 Deleting CiliumNetworkPolicy 'echo-ingress-l7-http' from namespace 'cilium-test'..
[...]
Connectivity test failed: 2 tests failed
📋 Test Report
❌ 2/11 tests failed (2/110 actions), 0 tests skipped, 0 scenarios skipped:
Test [client-egress-l7]:
  ❌ client-egress-l7/pod-to-pod/curl-3: cilium-test/client2-666976c95b-vlkxn (192.168.145.252) -> cilium-test/echo-same-node-7967996674-zmn74 (192.168.122.247:8080)
Test [echo-ingress-l7]:
  ❌ echo-ingress-l7/pod-to-pod/curl-3: cilium-test/client2-666976c95b-vlkxn (192.168.145.252) -> cilium-test/echo-same-node-7967996674-zmn74 (192.168.122.247:8080)

Metadata

Metadata

Assignees

Labels

area/CIContinuous Integration testing issue or flakearea/encryptionImpacts encryption support such as IPSec, WireGuard, or kTLS.ci/flakeThis is a known failure that occurs in the tree. Please investigate me!integration/cloudRelated to integration with cloud environments such as AKS, EKS, GKE, etc.

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions