Skip to content

Conversation

fidencio
Copy link
Member

@fidencio fidencio commented May 1, 2025

When running Kata Containers on CSPs, the CSPs may enforce their clusters to be IPv4-only.

Checking the OCI spec passed down to container, on a GKE cluster, we can see:

    "sysctl": {
      ...
      "net.ipv6.conf.all.disable_ipv6": "1",
      "net.ipv6.conf.default.disable_ipv6": "1",
      ...
    },

Even with ipv6 being explicitly disabled (behind our back ;-)), we've noticed that IPv6 addresses would be received, but then as IPv6 was disabled we'd break on CreatePodSandbox with the following error:

Warning  FailedCreatePodSandBox  4s    kubelet            Failed to
create pod sandbox: rpc error: code = Unknown desc = failed to create
containerd task: failed to create shim task: "update interface: Failed
to add address fe80::c44c:1cff:fe84:f6b7: NetlinkError(ErrorMessage {
code: Some(-13), header: [64, 0, 0, 0, 20, 0, 5, 5, 19, 0, 0, 0, 0, 0,
0, 0, 10, 64, 0, 0, 2, 0, 0, 0, 20, 0, 1, 0, 254, 128, 0, 0, 0, 0, 0, 0,
196, 76, 28, 255, 254, 132, 246, 183, 20, 0, 2, 0, 254, 128, 0, 0, 0, 0,
0, 0, 196, 76, 28, 255, 254, 132, 246, 183] })\n\nStack backtrace:\n
0: <unknown>\n   1: <unknown>\n   2: <unknown>\n   3: <unknown>\n   4:
<unknown>\n   5: <unknown>\n   6: <unknown>\n   7: <unknown>\n   8:
<unknown>\n   9: <unknown>\n  10: <unknown>": unknown

A huge shoutout to Fupan Li for helping with the debug on this one!

Fixes: #11200

@katacontainersbot katacontainersbot added the size/small Small and simple task label May 1, 2025
@fidencio fidencio requested a review from lifupan May 1, 2025 10:46
Copy link
Member

@stevenhorsman stevenhorsman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@fidencio fidencio force-pushed the topic/agent-only-try-ipv6-if-stack-is-supported branch from b8ad13f to d38ecc1 Compare May 1, 2025 13:01
@fidencio fidencio requested a review from stevenhorsman May 1, 2025 13:01
@fidencio fidencio force-pushed the topic/agent-only-try-ipv6-if-stack-is-supported branch from d38ecc1 to 11d3a41 Compare May 1, 2025 21:09
@fidencio fidencio force-pushed the topic/agent-only-try-ipv6-if-stack-is-supported branch from 11d3a41 to 49dbf02 Compare May 2, 2025 07:08
When running Kata Containers on CSPs, the CSPs may enforce their
clusters to be IPv4-only.

Checking the OCI spec passed down to container, on a GKE cluster, we can
see:
```
    "sysctl": {
      ...
      "net.ipv6.conf.all.disable_ipv6": "1",
      "net.ipv6.conf.default.disable_ipv6": "1",
      ...
    },
```

Even with ipv6 being explicitly disabled (behind our back ;-)), we've
noticed that IPv6 addresses would be received, but then as IPv6 was
disabled we'd break on CreatePodSandbox with the following error:
```
Warning  FailedCreatePodSandBox  4s    kubelet            Failed to
create pod sandbox: rpc error: code = Unknown desc = failed to create
containerd task: failed to create shim task: "update interface: Failed
to add address fe80::c44c:1cff:fe84:f6b7: NetlinkError(ErrorMessage {
code: Some(-13), header: [64, 0, 0, 0, 20, 0, 5, 5, 19, 0, 0, 0, 0, 0,
0, 0, 10, 64, 0, 0, 2, 0, 0, 0, 20, 0, 1, 0, 254, 128, 0, 0, 0, 0, 0, 0,
196, 76, 28, 255, 254, 132, 246, 183, 20, 0, 2, 0, 254, 128, 0, 0, 0, 0,
0, 0, 196, 76, 28, 255, 254, 132, 246, 183] })\n\nStack backtrace:\n
0: <unknown>\n   1: <unknown>\n   2: <unknown>\n   3: <unknown>\n   4:
<unknown>\n   5: <unknown>\n   6: <unknown>\n   7: <unknown>\n   8:
<unknown>\n   9: <unknown>\n  10: <unknown>": unknown
```

A huge shoutout to Fupan Li for helping with the debug on this one!

Fixes: kata-containers#11200

Signed-off-by: Fabiano Fidêncio <fidencio@northflank.com>
@fidencio fidencio force-pushed the topic/agent-only-try-ipv6-if-stack-is-supported branch from 49dbf02 to 4ce00ea Compare May 2, 2025 07:10
@fidencio fidencio merged commit fbf7faa into kata-containers:main May 2, 2025
637 of 708 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ok-to-test size/small Small and simple task
Projects
None yet
Development

Successfully merging this pull request may close these issues.

gcp: ipv6: Error creating container due to EACCESS
4 participants