Skip to content

Conversation

brb
Copy link
Member

@brb brb commented Jun 19, 2019

This PR introduces a workaround for preventing the systemd 242+ from assigning MAC addrs for virtual devices after they have been created. The trick is to set a MAC addr when creating such device. This sets addr_assign_type to NET_ADDR_SET which makes the systemd to skip re-assigning of the MAC addr: https://github.com/systemd/systemd/blob/v242/src/udev/net/link-config.c#L292.

Note that we don't apply the same trick to ipvlan slave devices, as they inherit a MAC addr from their master, so they bypass the issue.

See #8304 for more details about the systemd feature and what it causes to Cilium.

Fix #8304.

Work around systemd 242+ trying to re-assign MAC addresses for Cilium interfaces which breaks endpoint connectivity.

This change is Reviewable

@brb brb added kind/bug This is a bug in the Cilium logic. pending-review area/datapath Impacts bpf/ or low-level forwarding details, including map management and monitor messages. labels Jun 19, 2019
@brb brb requested a review from a team June 19, 2019 13:09
@brb brb requested review from a team as code owners June 19, 2019 13:09
@brb
Copy link
Member Author

brb commented Jun 19, 2019

Added the needs backport labels as all Cilium versions are affected.

@brb brb force-pushed the pr/brb/gen-mac-addr branch from 8ebc717 to 6517b29 Compare June 19, 2019 13:10
@brb
Copy link
Member Author

brb commented Jun 19, 2019

test-me-please

@coveralls
Copy link

coveralls commented Jun 19, 2019

Coverage Status

Coverage increased (+0.004%) to 44.167% when pulling 828bc71 on pr/brb/gen-mac-addr into 9a5e31f on master.

Copy link
Contributor

@nirmoy nirmoy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@nirmoy
Copy link
Contributor

nirmoy commented Jun 19, 2019

test-me-please

@jrfastab
Copy link
Contributor

Bit surprised the veth pair isn't given a MAC already by whoever created it but OK. Patches LGTM.

@joestringer joestringer added the release-note/minor This PR changes functionality that users may find relevant to operating Cilium. label Jun 19, 2019
Copy link
Member

@joestringer joestringer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Please also add a release note to the PR description to explain that this fixes compatibility with systemd 242.

(http://docs.cilium.io/en/latest/contributing/#submitting-a-pull-request step 9 if you need the formatting)

Also, do we need to do this for IPVLAN devices?

@brb
Copy link
Member Author

brb commented Jun 19, 2019

CI failed due to:

[Pipeline] echo
[2019-06-19T15:03:45.899Z] Label 'area/k8s' is not part of the PR '[area/datapath, kind/bug, needs-backport/1.3, needs-backport/1.4, needs-backport/1.5, pending-review]'
[Pipeline] }
[Pipeline] // timeout
[Pipeline] }
[Pipeline] // stage
[Pipeline] stage
[Pipeline] { (Precheck)
Stage "Precheck" skipped due to earlier failure(s)
[Pipeline] }

Retrying.

@brb
Copy link
Member Author

brb commented Jun 19, 2019

test-me-please

@brb
Copy link
Member Author

brb commented Jun 19, 2019

Also, do we need to do this for IPVLAN devices?

Note that we don't apply the same trick to ipvlan slave devices, as they inherit a MAC addr from their master, so they bypass the issue.

@joestringer joestringer added the dont-merge/needs-rebase This PR needs to be rebased because it has merge conflicts. label Jun 19, 2019
@ianvernon
Copy link
Member

@brb needs rebase.

brb added 4 commits June 20, 2019 10:46
A generated MAC addr is unicast and locally administered.

Signed-off-by: Martynas Pumputis <m@lambda.lt>
To include the change which allows to specify veth peer mac address
(vishvananda/netlink#460).

Signed-off-by: Martynas Pumputis <m@lambda.lt>
systemd 242+ tries to set a "persistent" MAC addr for any virtual device
by default (controlled by MACAddressPolicy). As setting happens
asynchronously after a device has been created, ep.Mac and ep.HostMac
can become stale which has a serious consequence - the kernel will drop
any packet sent to/from the endpoint. However, we can trick systemd by
explicitly setting MAC addrs for both veth ends. This sets
addr_assign_type for NET_ADDR_SET which prevents systemd from changing
the addrs.

Signed-off-by: Martynas Pumputis <m@lambda.lt>
To work around the systemd 242+ feature which tries to assign
a persistent MAC address for any device by default (see commit
message of the previous commit for more details).

Signed-off-by: Martynas Pumputis <m@lambda.lt>
@brb brb force-pushed the pr/brb/gen-mac-addr branch from 6517b29 to 828bc71 Compare June 20, 2019 08:46
@brb
Copy link
Member Author

brb commented Jun 20, 2019

test-me-please

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/datapath Impacts bpf/ or low-level forwarding details, including map management and monitor messages. kind/bug This is a bug in the Cilium logic. release-note/minor This PR changes functionality that users may find relevant to operating Cilium.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

No connectivity to endpoints with systemd 242
9 participants