-
Notifications
You must be signed in to change notification settings - Fork 1.1k
Description
Is your feature request related to a problem? Please describe.
This feature request is raised in Kyverno slack, the use case here is to limit pods per k8s node. This can be achieved by adding a taint on k8s node when the pods' count reaches the threshold: whenever a pod.spec.nodeName
is set, the rule can check the pod count on that node and if it exceeds a threshold, set a taint / or cordon on the node. This policy should be triggered on the Pod creation, then patch k8s Node accordingly.
Apparently, the trigger resource(Pod) is different from the target resource(Node), so it is beyond the current mutate ability.
Describe the solution you'd like
There are a few approaches that can be considered to support this:
-
Enable mutate policy in the background: whenever there's a create/update on the mutate policy, the background controller applies the policy to the existing resource, similar to Mutate existing resource on policy update. With this, we can have a generate rule to generate a Kyverno mutate policy (which is triggered on the pod creation), the background controller will patch the node that the pod is scheduled on.
-
Add a different type of rule that can generate a
Job
, based on the admission request. You can have a script or even run the kubectl command to do some custom logic. For example,kubectl taint node ...
in this case. -
Support javascript in the mutate policy.
-
Support
apiPost
in Kyverno policy. Similar toapiCall
, which fetches data from the API server,apiPost
can send a Post or Put request to mutate resources.