-
Notifications
You must be signed in to change notification settings - Fork 1.2k
agent: clear log pipes if denied by policy #10818
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is a clever fix @burgerdev 😄
Please add you Sob to the commit message and this is good to go.
Oops - done. Thanks, @gkurz! |
@burgerdev , I think this is a good change - thank you! In addition to this change: did you also consider setting up the Write side of the pipe for non-blocking Writes? If that works, it should help a CoCo Guest work a little better if its untrusted Host decides to stop calling ReadStream. |
You mean setting set -e
report() {
ret=$?
printf "shell exit code: %d\n" "$ret" >&2
exit "$ret"
}
trap report EXIT
while true; do
echo foo
done
/bin/sh: line 12: echo: write error: Resource temporarily unavailable
shell exit code: 1 |
As well explained by @burgerdev , O_NONBLOCK won't help. If DoS is a concern, the agent should monitor the read end of the pipe, consume the data and discard it. ReadStream should just always return an empty response in this case. |
With respect to confidential computing, DoS is usually out of scope because the infrastructure provider can always decide not to run the workload (cf. https://www.redhat.com/en/blog/confidential-computing-primer). |
This is my understanding as well. I was just commenting on @danmihai1's suggestion. |
@burgerdev Do you think you could also add a test case for this? |
@burgerdev , I guess rebasing on the latest main code might help with this test failure: https://github.com/kata-containers/kata-containers/actions/runs/13076857575/job/36496222189?pr=10818 . (That's how we got that test to pass in #10811) |
Container logs are forwarded to the agent through a unix pipe. These pipes have limited capacity and block the writer when full. If reading logs is blocked by policy, a common setup for confidential containers, the pipes fill up and eventually block the container. This commit changes the implementation of ReadStream such that it returns empty log messages instead of a policy failure (in case reading log messages is forbidden by policy). As long as the runtime does not encounter a failure, it keeps pulling logs periodically. In turn, this triggers the agent to flush the pipes. Fixes: kata-containers#10680 Co-Authored-By: Aurélien Bombo <abombo@microsoft.com> Signed-off-by: Markus Rudy <mr@edgeless.systems>
I rebased and fixed the clippy warning.
A simple test would be to make the existing |
You could also have unit testing of |
I like the idea of unit testing this, but struggle to implement it. The closest example I see is |
@burgerdev You can take inspiration here on how to use the policy in an e2e test - other test cases also use a very similar approach bd6eedc#diff-8aefc3b3747a8e1e0fbc75e5ee84b20b7ccb4853a144c9102ea1999b4d2c3041 |
I'm not really convinced by the
Another suggestion is to add an |
This test verifies that, when ReadStreamRequest is blocked by the policy, the logs are empty and the container does not deadlock. Signed-off-by: Aurélien Bombo <abombo@microsoft.com>
Thanks for adding the test @sprt! |
Container logs are forwarded to the agent through a unix pipe. These pipes have limited capacity and block the writer when full. If reading logs is blocked by policy, a common setup for confidential containers, the pipes fill up and eventually block the container.
This commit changes the implementation of
ReadStream
such that it returns empty log messages instead of a policy failure (in case reading log messages is forbidden by policy). As long as the runtime does not encounter a failure, it keeps pulling logs periodically. In turn, this triggers the agent to flush the pipes.Fixes: #10680
Alternatives considered
ReadStreamRequest
. I don't like the added agent complexity of this solution.cc @sprt @gkurz @mythi