-
Notifications
You must be signed in to change notification settings - Fork 3.4k
Closed
Labels
area/CIContinuous Integration testing issue or flakeContinuous Integration testing issue or flakeci/flakeThis is a known failure that occurs in the tree. Please investigate me!This is a known failure that occurs in the tree. Please investigate me!
Description
The ConformanceKind workflow seems to be unstable on master while connecting to 1.1.1.1.
Example run:
https://github.com/cilium/cilium/actions/runs/3468759658/jobs/5794941960
cilium-sysdump-out.zip (28).zip
Run cilium connectivity test --flow-validation=disabled --hubble=false
cilium connectivity test --flow-validation=disabled --hubble=false
shell: /usr/bin/bash -e {0}
env:
kind_version: v0.11.1
kind_config: .github/kind-config.yaml
cilium_cli_version: v0.1[2](https://github.com/cilium/cilium/actions/runs/3468759658/jobs/5794941960#step:12:2).4
QUAY_ORGANIZATION: cilium
QUAY_ORGANIZATION_DEV: cilium
✨ [kind-chart-testing] Creating namespace cilium-test for connectivity check...
✨ [kind-chart-testing] Deploying echo-same-node service...
✨ [kind-chart-testing] Deploying DNS test server configmap...
✨ [kind-chart-testing] Deploying same-node deployment...
✨ [kind-chart-testing] Deploying client deployment...
✨ [kind-chart-testing] Deploying client2 deployment...
✨ [kind-chart-testing] Deploying echo-other-node service...
✨ [kind-chart-testing] Deploying other-node deployment...
⌛ [kind-chart-testing] Waiting for deployments [client client2 echo-same-node] to become ready...
⌛ [kind-chart-testing] Waiting for deployments [echo-other-node] to become ready...
⌛ [kind-chart-testing] Waiting for CiliumEndpoint for pod cilium-test/client-7bdbddd7b-bc67l to appear...
⌛ [kind-chart-testing] Waiting for CiliumEndpoint for pod cilium-test/client2-6ff4656b7d-wjj8j to appear...
⌛ [kind-chart-testing] Waiting for pod cilium-test/client-7bdbddd7b-bc67l to reach DNS server on cilium-test/echo-same-node-77494b4cbd-shdlv pod...
⌛ [kind-chart-testing] Waiting for pod cilium-test/client2-6ff4656b7d-wjj8j to reach DNS server on cilium-test/echo-same-node-77494b4cbd-shdlv pod...
⌛ [kind-chart-testing] Waiting for pod cilium-test/client-7bdbddd7b-bc67l to reach DNS server on cilium-test/echo-other-node-c7cd7697d-sqdsb pod...
⌛ [kind-chart-testing] Waiting for pod cilium-test/client2-6ff4656b7d-wjj8j to reach DNS server on cilium-test/echo-other-node-c7cd7697d-sqdsb pod...
⌛ [kind-chart-testing] Waiting for pod cilium-test/client-7bdbddd7b-bc67l to reach default/kubernetes service...
⌛ [kind-chart-testing] Waiting for pod cilium-test/client2-6ff4656b7d-wjj8j to reach default/kubernetes service...
⌛ [kind-chart-testing] Waiting for CiliumEndpoint for pod cilium-test/echo-other-node-c7cd7697d-sqdsb to appear...
⌛ [kind-chart-testing] Waiting for CiliumEndpoint for pod cilium-test/echo-same-node-77494b4cbd-shdlv to appear...
⌛ [kind-chart-testing] Waiting for Service cilium-test/echo-other-node to become ready...
⌛ [kind-chart-testing] Waiting for Service cilium-test/echo-same-node to become ready...
⌛ [kind-chart-testing] Waiting for NodePort 172.18.0.2:[3](https://github.com/cilium/cilium/actions/runs/3468759658/jobs/5794941960#step:12:3)2177 (cilium-test/echo-other-node) to become ready...
⌛ [kind-chart-testing] Waiting for NodePort 172.18.0.2:30292 (cilium-test/echo-same-node) to become ready...
⌛ [kind-chart-testing] Waiting for NodePort 172.18.0.3:32177 (cilium-test/echo-other-node) to become ready...
⌛ [kind-chart-testing] Waiting for NodePort 172.18.0.3:30292 (cilium-test/echo-same-node) to become ready...
ℹ️ Skipping IPCache check
ℹ️ Cilium version: 1.12.90
🏃 Running tests...
[=] Test [no-policies]
...................................
[-] Scenario [no-policies/pod-to-pod]
[.] Action [no-policies/pod-to-pod/curl-0: cilium-test/client2-6ff[4](https://github.com/cilium/cilium/actions/runs/3468759658/jobs/5794941960#step:12:4)6[5](https://github.com/cilium/cilium/actions/runs/3468759658/jobs/5794941960#step:12:5)[6](https://github.com/cilium/cilium/actions/runs/3468759658/jobs/5794941960#step:12:6)b[7](https://github.com/cilium/cilium/actions/runs/3468759658/jobs/5794941960#step:12:7)d-wjj[8](https://github.com/cilium/cilium/actions/runs/3468759658/jobs/5794941960#step:12:8)j (10.244.1.237) -> cilium-test/echo-other-node-c7cd76[9](https://github.com/cilium/cilium/actions/runs/3468759658/jobs/5794941960#step:12:9)7d-sqdsb ([10](https://github.com/cilium/cilium/actions/runs/3468759658/jobs/5794941960#step:12:11).244.0.57:8080)]
[.] Action [no-policies/pod-to-pod/curl-1: cilium-test/client2-6ff4656b7d-wjj8j (10.244.1.237) -> cilium-test/echo-same-node-77494b4cbd-shdlv (10.244.1.233:8080)]
[.] Action [no-policies/pod-to-pod/curl-2: cilium-test/client-7bdbddd7b-bc67l (10.244.1.97) -> cilium-test/echo-other-node-c7cd7697d-sqdsb (10.244.0.57:8080)]
[.] Action [no-policies/pod-to-pod/curl-3: cilium-test/client-7bdbddd7b-bc67l (10.244.1.97) -> cilium-test/echo-same-node-77494b4cbd-shdlv (10.244.1.233:8080)]
[-] Scenario [no-policies/pod-to-hostport]
[.] Action [no-policies/pod-to-hostport/curl-0: cilium-test/client-7bdbddd7b-bc67l (10.244.1.97) -> cilium-test/echo-same-node-77494b4cbd-shdlv (172.18.0.3:40000)]
[.] Action [no-policies/pod-to-hostport/curl-1: cilium-test/client-7bdbddd7b-bc67l (10.244.1.97) -> cilium-test/echo-other-node-c7cd7697d-sqdsb (172.18.0.2:40000)]
[-] Scenario [client-egress-to-cidr-deny/pod-to-cidr]
[.] Action [client-egress-to-cidr-deny/pod-to-cidr/cloudflare-1001-0: cilium-test/client-7bdbddd7b-bc67l (10.244.1.97) -> cloudflare-1001 (1.0.0.1:80)]
[.] Action [client-egress-to-cidr-deny/pod-to-cidr/cloudflare-1001-1: cilium-test/client2-6ff4656b7d-wjj8j (10.244.1.237) -> cloudflare-1001 (1.0.0.1:80)]
[.] Action [client-egress-to-cidr-deny/pod-to-cidr/cloudflare-[11](https://github.com/cilium/cilium/actions/runs/3468759658/jobs/5794941960#step:12:12)11-0: cilium-test/client-7bdbddd7b-bc67l (10.244.1.97) -> cloudflare-1111 (1.1.1.1:80)]
❌ command "curl -w %{local_ip}:%{local_port} -> %{remote_ip}:%{remote_port} = %{response_code} --silent --fail --show-error --connect-timeout 5 --output /dev/null http://1.1.1.1:80" failed: command terminated with exit code 28
ℹ️ curl output:
curl: (28) Connection timeout after 5000 ms
:0 -> :0 = 000
📄 No flows recorded during action cloudflare-1111-0
📄 No flows recorded during action cloudflare-1111-0
[.] Action [client-egress-to-cidr-deny/pod-to-cidr/cloudflare-1111-1: cilium-test/client2-6ff4656b7d-wjj8j (10.244.1.237) -> cloudflare-1111 (1.1.1.1:80)]
❌ command "curl -w %{local_ip}:%{local_port} -> %{remote_ip}:%{remote_port} = %{response_code} --silent --fail --show-error --connect-timeout 5 --output /dev/null http://1.1.1.1:80" failed: command terminated with exit code 28
ℹ️ curl output:
curl: (28) Connection timeout after 5000 ms
:0 -> :0 = 000
📄 No flows recorded during action cloudflare-1111-1
📄 No flows recorded during action cloudflare-1111-1
ℹ️ 📜 Deleting CiliumNetworkPolicy 'allow-all-egress' from namespace 'cilium-test'..
ℹ️ 📜 Deleting CiliumNetworkPolicy 'client-egress-to-cidr-deny' from namespace 'cilium-test'..
[=] Test [health]
..
[=] Test [client-egress-l7-method]
............
[=] Test [client-egress-l7]
..........
[=] Test [client-egress-l7-named-port]
........
ℹ️ 📜 Applying CiliumNetworkPolicy 'client-egress-only-dns' to namespace 'cilium-test'..
ℹ️ 📜 Applying CiliumNetworkPolicy 'client-egress-l7-http-named-port' to namespace 'cilium-test'..
[-] Scenario [client-egress-l7-named-port/pod-to-pod]
[.] Action [client-egress-l7-named-port/pod-to-pod/curl-0: cilium-test/client-7bdbddd7b-bc67l (10.244.1.97) -> cilium-test/echo-other-node-c7cd7697d-sqdsb (10.244.0.57:8080)]
[.] Action [client-egress-l7-named-port/pod-to-pod/curl-1: cilium-test/client-7bdbddd7b-bc67l (10.244.1.97) -> cilium-test/echo-same-node-77494b4cbd-shdlv (10.244.1.233:8080)]
[.] Action [client-egress-l7-named-port/pod-to-pod/curl-2: cilium-test/client2-6ff4656b7d-wjj8j (10.244.1.237) -> cilium-test/echo-other-node-c7cd7697d-sqdsb (10.244.0.57:8080)]
[.] Action [client-egress-l7-named-port/pod-to-pod/curl-3: cilium-test/client2-6ff4656b7d-wjj8j (10.244.1.237) -> cilium-test/echo-same-node-77494b4cbd-shdlv (10.244.1.233:8080)]
[-] Scenario [client-egress-l7-named-port/pod-to-world]
[.] Action [client-egress-l7-named-port/pod-to-world/http-to-one-one-one-one-0: cilium-test/client-7bdbddd7b-bc67l (10.244.1.97) -> one-one-one-one-http (one.one.one.one:80)]
[.] Action [client-egress-l7-named-port/pod-to-world/https-to-one-one-one-one-0: cilium-test/client-7bdbddd7b-bc67l (10.244.1.97) -> one-one-one-one-https (one.one.one.one:443)]
[.] Action [client-egress-l7-named-port/pod-to-world/https-to-one-one-one-one-index-0: cilium-test/client-7bdbddd7b-bc67l (10.244.1.97) -> one-one-one-one-https-index (one.one.one.one:443)]
[.] Action [client-egress-l7-named-port/pod-to-world/http-to-one-one-one-one-1: cilium-test/client2-6ff4656b7d-wjj8j (10.244.1.237) -> one-one-one-one-http (one.one.one.one:80)]
❌ command "curl -w %{local_ip}:%{local_port} -> %{remote_ip}:%{remote_port} = %{response_code} --silent --fail --show-error --connect-timeout 5 --output /dev/null http://one.one.one.one:80" failed: command terminated with exit code [22](https://github.com/cilium/cilium/actions/runs/3468759658/jobs/5794941960#step:12:23)
ℹ️ curl output:
curl: (22) The requested URL returned error: 503
10.244.1.[23](https://github.com/cilium/cilium/actions/runs/3468759658/jobs/5794941960#step:12:24)7:52666 -> 1.1.1.1:80 = 503
📄 No flows recorded during action http-to-one-one-one-one-1
📄 No flows recorded during action http-to-one-one-one-one-1
[.] Action [client-egress-l7-named-port/pod-to-world/https-to-one-one-one-one-1: cilium-test/client2-6ff4656b7d-wjj8j (10.[24](https://github.com/cilium/cilium/actions/runs/3468759658/jobs/5794941960#step:12:25)4.1.237) -> one-one-one-one-https (one.one.one.one:443)]
[.] Action [client-egress-l7-named-port/pod-to-world/https-to-one-one-one-one-index-1: cilium-test/client2-6ff4656b7d-wjj8j (10.244.1.237) -> one-one-one-one-https-index (one.one.one.one:443)]
ℹ️ 📜 Deleting CiliumNetworkPolicy 'client-egress-only-dns' from namespace 'cilium-test'..
ℹ️ 📜 Deleting CiliumNetworkPolicy 'client-egress-l7-http-named-port' from namespace 'cilium-test'..
[=] Test [dns-only]
..........
[=] Test [to-fqdns]
........
📋 Test Report
connectivity test failed: 5 tests failed
❌ 5/30 tests failed (9/2[28](https://github.com/cilium/cilium/actions/runs/3468759658/jobs/5794941960#step:12:29) actions), 0 tests skipped, 0 scenarios skipped:
Test [no-policies]:
❌ no-policies/pod-to-cidr/cloudflare-1111-0: cilium-test/client-7bdbddd7b-bc67l (10.244.1.97) -> cloudflare-1111 (1.1.1.1:80)
❌ no-policies/pod-to-cidr/cloudflare-1111-1: cilium-test/client2-6ff4656b7d-wjj8j (10.244.1.2[37](https://github.com/cilium/cilium/actions/runs/3468759658/jobs/5794941960#step:12:38)) -> cloudflare-1111 (1.1.1.1:80)
Test [all-ingress-deny]:
❌ all-ingress-deny/pod-to-cidr/cloudflare-1111-0: cilium-test/client2-6ff4656b7d-wjj8j (10.2[44](https://github.com/cilium/cilium/actions/runs/3468759658/jobs/5794941960#step:12:45).1.237) -> cloudflare-1111 (1.1.1.1:80)
❌ all-ingress-deny/pod-to-cidr/cloudflare-1111-1: cilium-test/client-7bdbddd7b-bc67l (10.244.1.97) -> cloudflare-1111 (1.1.1.1:80)
Test [to-cidr-1111]:
❌ to-cidr-1111/pod-to-cidr/cloudflare-1111-0: cilium-test/client-7bdbddd7b-bc67l (10.244.1.97) -> cloudflare-1111 (1.1.1.1:80)
❌ to-cidr-1111/pod-to-cidr/cloudflare-1111-1: cilium-test/client2-6ff[46](https://github.com/cilium/cilium/actions/runs/3468759658/jobs/5794941960#step:12:47)56b7d-wjj8j (10.244.1.[237](https://github.com/cilium/cilium/actions/runs/3468759658/jobs/5794941960#step:12:238)) -> cloudflare-1111 (1.1.1.1:80)
Test [client-egress-to-cidr-deny]:
❌ client-egress-to-cidr-deny/pod-to-cidr/cloudflare-1111-0: cilium-test/client-7bdbddd7b-bc67l (10.[244](https://github.com/cilium/cilium/actions/runs/3468759658/jobs/5794941960#step:12:245).1.97) -> cloudflare-1111 (1.1.1.1:80)
❌ client-egress-to-cidr-deny/pod-to-cidr/cloudflare-1111-1: cilium-test/client2-6ff4656b7d-wjj8j (10.244.1.237) -> cloudflare-1111 (1.1.1.1:80)
Test [client-egress-l7-named-port]:
❌ client-egress-l7-named-port/pod-to-world/http-to-one-one-one-one-1: cilium-test/client2-6ff4656b7d-wjj8j (10.244.1.237) -> one-one-one-one-http (one.one.one.one:80)
Error: Process completed with exit code 1.
Metadata
Metadata
Assignees
Labels
area/CIContinuous Integration testing issue or flakeContinuous Integration testing issue or flakeci/flakeThis is a known failure that occurs in the tree. Please investigate me!This is a known failure that occurs in the tree. Please investigate me!