-
Notifications
You must be signed in to change notification settings - Fork 1.2k
genpolicy: add validation for storages #11248
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
f4b7546
to
895eca3
Compare
895eca3
to
9733a75
Compare
Closing and re-opening this PR to re-trigger some of the GHA jobs that had issues |
8d55146
to
b94cad3
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
src/tools/genpolicy/rules.rego
Outdated
i_storage.fstype == "overlay" | ||
i_storage.fs_group == null | ||
count(i_storage.options) == 0 | ||
p_storage := i_storage |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This means that the mount point is not verified at all, right? Shouldn't we rather check that the mount point is a container rootfs dir?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think enabling allow_storages() for the first time in the main branch, and cleaning up various issues related to it, is a reasonable enough goal for this PR.
I agree that the driver == "image_guest_pull" rules will likely still need improvements after this PR. Some ideas to make progress with this other type of work in future PRs:
- If someone can explain to me the rules for verifying that the image_guest_pull storage input is correct, I will try to implement these rules in genpolicy.
- If someone else already knows the rules for verifying that the image_guest_pull storage input is correct, maybe they can contribute these improvements to genpolicy.
- I believe pulling images on the Host has been discussed again recently for CoCo (while I was out from work for a few weeks), so maybe Host-pulled images will become at least as popular as image_guest_pull images?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We should add a TODO comment for the missing image_guest_pull rules though.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think enabling allow_storages() for the first time in the main branch, and cleaning up various issues related to it, is a reasonable enough goal for this PR.
Agreed and I'm happy about making progress here, but I think we should try not to leave a state that superficially seems to do the right thing, but actually doesn't. I would be ok with a warning comment, though.
1. If someone can explain to me the rules for verifying that the image_guest_pull storage input is correct, I will try to implement these rules in genpolicy.
There's actually not that much to verify: image and mountpoint. A simple implementation would just check that the input is exactly equal to what's configured here:
kata-containers/src/runtime/virtcontainers/fs_share_linux.go
Lines 597 to 613 in 3f5dc87
func forceGuestPull(c *Container) (*SharedFile, error) { | |
sf := &SharedFile{ | |
guestPath: filepath.Join("/run/kata-containers/", c.id, c.rootfsSuffix), | |
} | |
guestPullVolume := &types.KataVirtualVolume{ | |
VolumeType: types.KataVirtualVolumeImageGuestPullType, | |
ImagePull: &types.ImagePullVolume{ | |
Metadata: map[string]string{}, | |
}, | |
} | |
vol, err := handleVirtualVolumeStorageObject(c, "", guestPullVolume) | |
if err != nil { | |
return nil, fmt.Errorf("forcing guest pull virtual volume: %w", err) | |
} | |
sf.containerStorages = append(sf.containerStorages, vol) | |
return sf, nil | |
} |
However, when I implemented guest pull verification, I decided to only verify the manifest digest, not the registry/repo/tag, because that allows generating policies in one registry and pulling the images from another. Now, the alternative to pinning the image is to work with a tag and rely on image signatures, but in that case we'd need to verify registry and repository. Personally, I think that images used in CoCo should be pinned, if only because of reliability concerns. I don't mean to force that opinion onto people, though.
How about I open an RFC for this? Or is there an existing issue that I missed?
2. If someone else already knows the rules for verifying that the image_guest_pull storage input is correct, maybe they can contribute these improvements to genpolicy.
I can open a PR with my downstream implementation, and try to also cover the signed image case.
3. I believe pulling images on the Host has been discussed again recently for CoCo (while I was out from work for a few weeks), so maybe Host-pulled images will become at least as popular as image_guest_pull images?
Host-pulled with EROFS-snapshotter sounds very promising, it is definitely desirable when working with large images, and I think we can reuse most of the tardev policy for it. But there are good arguments for also supporting guest pull (simpler policy, confidential encrypted images), so we probably need to support both.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks @burgerdev - I agree with what you said! I just didn't want to let Great be the enemy of Good in this case. I agree that we need that kind of comment for the remaining missing features.
Perhaps verifying the registry/repo/tag raises the bar a bit depending on use case - but CoCo users would also have to worry about the confidentiality/integrity of their network stack, and of the image repository server. Otherwise, images with correct registry/repo/tag might be downloaded from a rogue server.
Verifying manifest digests and/or using trusted signatures seems useful to me.
Thanks for volunteering to help with the image_guest_pull implementation! If the community would like support for verifying manifest digests, I am not aware of good reasons to wait until the signature and/or registry/repo/tag verification get implemented too.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I wrote an RFC: #11354.
b94cad3
to
be48819
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks a lot, lgtm!
be48819
to
e83f02c
Compare
This patch fixes the rules.rego file to ensure that the policy is correctly parsed and applied by opa. Signed-off-by: Archana Choudhary <archana1@microsoft.com>
Description
This PR adds validation for storages in the genpolicy tool.
Test cases are added for: config map, and container images containing volumes
Test cases are improved for: empty_dir