Skip to content

Conversation

georgibaltiev
Copy link
Contributor

@georgibaltiev georgibaltiev commented Feb 25, 2025

How to categorize this PR?

/area compliance
/area security
/kind enhancement

What this PR does / why we need it:
This PR sets securityContext.allowPrivilegeEscalation to false for every Seed and Garden cluster component container, which does not have securityContext.Privileged set to true or one of CAP_SYS_ADMIN/SYS_ADMIN capabilities added.

Which issue(s) this PR fixes:
Part of #11139

Special notes for your reviewer:
/ cc @AleksandarSavchev

Release note:

Garden and Seed cluster component containers, which do not require privilege escalations, now forbid privilege escalation explicitly.

Copy link
Contributor

gardener-prow bot commented Feb 25, 2025

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

@gardener-prow gardener-prow bot added do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. area/compliance Compliance related area/security Security related kind/enhancement Enhancement, improvement, extension labels Feb 25, 2025
@gardener-prow gardener-prow bot requested review from acumino and ary1992 February 25, 2025 15:31
@gardener-prow gardener-prow bot added cla: yes Indicates the PR's author has signed the cla-assistant.io CLA. size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Feb 25, 2025
Copy link
Member

@rfranzke rfranzke left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe this was already discussed, but why don't we do this via a webhook in GRM? We could augment https://github.com/gardener/gardener/tree/master/pkg/resourcemanager/webhook/seccompprofile (rename) to also handle this AllowPrivilegeEscalation field in pod specs (maybe even also other simple things from the component checklist)?
Doing this manually and expecting others to be aware of it will probably lead to deficiencies, don't you think?

/cc @ialidzhikov

@gardener-prow gardener-prow bot requested a review from ialidzhikov February 25, 2025 15:53
@AleksandarSavchev
Copy link
Member

It is not easy to determine when securityContext.allowPrivilegeEscalation can be set to false. The minimal requirement is for the securityContext.privileged to be set to false and CAP_SYS_ADMIN/SYS_ADMIN to not be part of securityContext.capabilities.add. However the container might still require securityContext.allowPrivilegeEscalation: true.

@gardener-prow gardener-prow bot added cla: no Indicates the PR's author has not signed the cla-assistant.io CLA. cla: yes Indicates the PR's author has signed the cla-assistant.io CLA. and removed cla: yes Indicates the PR's author has signed the cla-assistant.io CLA. cla: no Indicates the PR's author has not signed the cla-assistant.io CLA. labels Feb 26, 2025
@georgibaltiev georgibaltiev marked this pull request as ready for review February 26, 2025 08:37
@gardener-prow gardener-prow bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Feb 26, 2025
@gardener-prow gardener-prow bot requested a review from marc1404 February 26, 2025 08:37
@georgibaltiev
Copy link
Contributor Author

/test pull-gardener-e2e-kind-ipv6

@rfranzke
Copy link
Member

It is not easy to determine when securityContext.allowPrivilegeEscalation can be set to false. The minimal requirement is for the securityContext.privileged to be set to false and CAP_SYS_ADMIN/SYS_ADMIN to not be part of securityContext.capabilities.add. However the container might still require securityContext.allowPrivilegeEscalation: true.

Couldn't we add an opt-out option of the webhook with a label that can be set on Pods in order to not trigger it when it cannot be set to false? We have such an approach for other, similarly working webhooks

@ialidzhikov
Copy link
Member

Maybe this was already discussed, but why don't we do this via a webhook in GRM? We could augment https://github.com/gardener/gardener/tree/master/pkg/resourcemanager/webhook/seccompprofile (rename) to also handle this AllowPrivilegeEscalation field in pod specs (maybe even also other simple things from the component checklist)?
Doing this manually and expecting others to be aware of it will probably lead to deficiencies, don't you think?

For the seccomp profile field, the GRM webhook only covers the Seed/Garden cluster. For Shoots, we declaratively set the field. IIRC, the plan was even to drop the GRM webhook for the Seed/Garden cluster and use the corresponding kubelet field in the Shoot spec (for ManagedSeeds at least).

The advantage of a webhook is that there won't be need to adapt each and every container. However, exceptions still have to be done - most likely via label/annotation to instruct the webhook to do not mutate in special cases when the workload needs privilege escalation for example.
The work for the developer at the end is the same. Via the webhook or not, the workload has to be tested whether it works with allowPrivilegeEscalation=false or not.

There was a push on #7157 or a similar issue on why we, from Gardener Core side, didn't do anything so far on the topic. However, my reply back then was that:

  • The requirements to me are not clear. Do we want to enforce such security settings on the runtime (with webhooks) or are we fine only to have them declaratively.
  • There is not only 1 single setting - seccomp profile, run as non-root, allowPrivilegeEscalation=false, etc. If we do it via a webhook, then we might want to take care for every field.

We also have diki that performs the checks, so it won't be the case that we missed to adapt a component.

And, the webhook approach - it introduces a performance penalty (n milliseconds for the webhook call, network cost, etc) and yet another failure point in the system.

tl;dr: I don't see super big advantage of using a webhook if we don't have the requirement to enforce the fields on the runtime (kyverno-like). But I am open to discuss and see other points.

@georgibaltiev
Copy link
Contributor Author

/test pull-gardener-integration

1 similar comment
@georgibaltiev
Copy link
Contributor Author

/test pull-gardener-integration

@georgibaltiev
Copy link
Contributor Author

/test pull-gardener-e2e-kind-ipv6

2 similar comments
@georgibaltiev
Copy link
Contributor Author

/test pull-gardener-e2e-kind-ipv6

@georgibaltiev
Copy link
Contributor Author

/test pull-gardener-e2e-kind-ipv6

@georgibaltiev
Copy link
Contributor Author

Holding this PR until we have clarity on what approach to use for the containers.securityContext.allowPrivilegeEscalation field.
/hold

@gardener-prow gardener-prow bot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Mar 4, 2025
@georgibaltiev
Copy link
Contributor Author

/unhold

@georgibaltiev georgibaltiev force-pushed the remove-seed-component-privilege-escalations branch from ae01dfe to 36d55b9 Compare March 17, 2025 14:03
Copy link
Member

@ialidzhikov ialidzhikov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm

@gardener-prow gardener-prow bot added the lgtm Indicates that a PR is ready to be merged. label Mar 17, 2025
Copy link
Contributor

gardener-prow bot commented Mar 17, 2025

LGTM label has been added.

Git tree hash: 8b8f746249017fa869fbe81ae14a62955327ff4f

Copy link
Contributor

gardener-prow bot commented Mar 17, 2025

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: ialidzhikov

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@gardener-prow gardener-prow bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Mar 17, 2025
@gardener-prow gardener-prow bot merged commit 085ac49 into gardener:master Mar 17, 2025
19 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. area/compliance Compliance related area/security Security related cla: yes Indicates the PR's author has signed the cla-assistant.io CLA. kind/enhancement Enhancement, improvement, extension lgtm Indicates that a PR is ready to be merged. size/L Denotes a PR that changes 100-499 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants