Clear and concise description of the problem

I'm trying to run a container image build step in my pipeline that uses buildah. I'm running into a lot of permission errors like this:

time="2023-10-07T19:59:26Z" level=error msg="writing \"0 0 4294967295\\n\" to /proc/17/uid_map: write /proc/17/uid_map: operation not permitted"
time="2023-10-07T19:59:26Z" level=error msg="(Unable to determine exit status)"

After some experimentation with podman and buildah containers on my cluster, I have determined that I need to configure a few different settings at the Pod level for this to work:

• metadata.annotations: I need to be able to set container.apparmor.security.beta.kubernetes.io/<container_name> to use a custom apparmor profile for my buildah container. I would like to use the WOODPECKER_BACKEND_K8S_POD_ANNOTATIONS env variable, but since the container names are randomly generated I'm not sure how to make this work.
• spec.securityContext: The only way I have found to make buildah work is to run it as non-root. This means that I need to set runAsUser, runAsGroup, and fsGroup on the Pod level securityContext (not the container level).

Suggested solution

Adding support for securityContext seems like it should be fairly straightforward. We could just follow that same approach that is currently used for resources, nodeSelector, etc. documented here.

The AppArmor profile is trickier since the container name needs to be included in the annotation key. I'm imagining a configuration like backend_options.kubernetes.appArmorProfile that could take all of the possible values for the annotation documented here. Then, the pod generation logic would just look for this configuration and add the appropriate annotation if it's found.

Alternative

I could disable AppArmor entirely, but that's not really advisable for a production environment. I could also try to find a way to get the buildah container running as root (it should be possible), but again, that's not ideal from a security perspective.

Additional context

This is somewhat of a follow up from #2510. I'm trying to get Woodpecker to run on my kubeadm/CRI-O based cluster running on Debian which I believe makes me some kind of leprechaun/unicorn/lochness monster. There definitely aren't many of us out there.

Validations

• Checked that the feature isn't part of the next version already [https://woodpecker-ci.org/faq#which-version-of-woodpecker-should-i-use]
• Read the Contributing Guidelines.
• Read the docs.
• Check that there isn't already an issue that request the same feature to avoid creating a duplicate.