Skip to content

Conversation

arc9693
Copy link
Contributor

@arc9693 arc9693 commented May 8, 2025

Description
This PR adds validation for storages in the genpolicy tool.
Test cases are added for: config map, and container images containing volumes
Test cases are improved for: empty_dir

@arc9693 arc9693 force-pushed the archana1/storages branch 3 times, most recently from f4b7546 to 895eca3 Compare May 21, 2025 14:53
@arc9693 arc9693 force-pushed the archana1/storages branch from 895eca3 to 9733a75 Compare May 22, 2025 15:21
@katacontainersbot katacontainersbot added the size/huge Largest and most complex task (probably needs breaking into small pieces) label May 22, 2025
@arc9693 arc9693 marked this pull request as ready for review May 22, 2025 16:42
@arc9693 arc9693 changed the title [WIP] genpolicy: add validation for storages genpolicy: add validation for storages May 22, 2025
@arc9693 arc9693 requested a review from danmihai1 May 27, 2025 11:49
@stevenhorsman
Copy link
Member

Closing and re-opening this PR to re-trigger some of the GHA jobs that had issues

@arc9693 arc9693 force-pushed the archana1/storages branch 2 times, most recently from 8d55146 to b94cad3 Compare May 29, 2025 15:50
@danmihai1 danmihai1 requested a review from burgerdev May 29, 2025 23:01
Copy link
Contributor

@burgerdev burgerdev left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The overall PR lgtm.

I think that ecf58cf, 9db0118, 4b19d45 and b94cad3 should be squashed into the appropriate parent commits.

i_storage.fstype == "overlay"
i_storage.fs_group == null
count(i_storage.options) == 0
p_storage := i_storage
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This means that the mount point is not verified at all, right? Shouldn't we rather check that the mount point is a container rootfs dir?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think enabling allow_storages() for the first time in the main branch, and cleaning up various issues related to it, is a reasonable enough goal for this PR.

I agree that the driver == "image_guest_pull" rules will likely still need improvements after this PR. Some ideas to make progress with this other type of work in future PRs:

  1. If someone can explain to me the rules for verifying that the image_guest_pull storage input is correct, I will try to implement these rules in genpolicy.
  2. If someone else already knows the rules for verifying that the image_guest_pull storage input is correct, maybe they can contribute these improvements to genpolicy.
  3. I believe pulling images on the Host has been discussed again recently for CoCo (while I was out from work for a few weeks), so maybe Host-pulled images will become at least as popular as image_guest_pull images?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should add a TODO comment for the missing image_guest_pull rules though.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think enabling allow_storages() for the first time in the main branch, and cleaning up various issues related to it, is a reasonable enough goal for this PR.

Agreed and I'm happy about making progress here, but I think we should try not to leave a state that superficially seems to do the right thing, but actually doesn't. I would be ok with a warning comment, though.

1. If someone can explain to me the rules for verifying that the image_guest_pull storage input is correct, I will try to implement these rules in genpolicy.

There's actually not that much to verify: image and mountpoint. A simple implementation would just check that the input is exactly equal to what's configured here:

func forceGuestPull(c *Container) (*SharedFile, error) {
sf := &SharedFile{
guestPath: filepath.Join("/run/kata-containers/", c.id, c.rootfsSuffix),
}
guestPullVolume := &types.KataVirtualVolume{
VolumeType: types.KataVirtualVolumeImageGuestPullType,
ImagePull: &types.ImagePullVolume{
Metadata: map[string]string{},
},
}
vol, err := handleVirtualVolumeStorageObject(c, "", guestPullVolume)
if err != nil {
return nil, fmt.Errorf("forcing guest pull virtual volume: %w", err)
}
sf.containerStorages = append(sf.containerStorages, vol)
return sf, nil
}

However, when I implemented guest pull verification, I decided to only verify the manifest digest, not the registry/repo/tag, because that allows generating policies in one registry and pulling the images from another. Now, the alternative to pinning the image is to work with a tag and rely on image signatures, but in that case we'd need to verify registry and repository. Personally, I think that images used in CoCo should be pinned, if only because of reliability concerns. I don't mean to force that opinion onto people, though.

How about I open an RFC for this? Or is there an existing issue that I missed?

2. If someone else already knows the rules for verifying that the image_guest_pull storage input is correct, maybe they can contribute these improvements to genpolicy.

I can open a PR with my downstream implementation, and try to also cover the signed image case.

3. I believe pulling images on the Host has been discussed again recently for CoCo (while I was out from work for a few weeks), so maybe Host-pulled images will become at least as popular as image_guest_pull images?

Host-pulled with EROFS-snapshotter sounds very promising, it is definitely desirable when working with large images, and I think we can reuse most of the tardev policy for it. But there are good arguments for also supporting guest pull (simpler policy, confidential encrypted images), so we probably need to support both.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @burgerdev - I agree with what you said! I just didn't want to let Great be the enemy of Good in this case. I agree that we need that kind of comment for the remaining missing features.

Perhaps verifying the registry/repo/tag raises the bar a bit depending on use case - but CoCo users would also have to worry about the confidentiality/integrity of their network stack, and of the image repository server. Otherwise, images with correct registry/repo/tag might be downloaded from a rogue server.

Verifying manifest digests and/or using trusted signatures seems useful to me.

Thanks for volunteering to help with the image_guest_pull implementation! If the community would like support for verifying manifest digests, I am not aware of good reasons to wait until the signature and/or registry/repo/tag verification get implemented too.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wrote an RFC: #11354.

@arc9693 arc9693 force-pushed the archana1/storages branch from b94cad3 to be48819 Compare June 2, 2025 14:59
Copy link
Contributor

@burgerdev burgerdev left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks a lot, lgtm!

This patch fixes the rules.rego file to ensure that the
policy is correctly parsed and applied by opa.

Signed-off-by: Archana Choudhary <archana1@microsoft.com>
@Redent0r Redent0r merged commit e7b9edd into kata-containers:main Jul 1, 2025
339 of 352 checks passed
@Redent0r Redent0r deleted the archana1/storages branch July 1, 2025 17:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ok-to-test size/huge Largest and most complex task (probably needs breaking into small pieces)
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants