-
Notifications
You must be signed in to change notification settings - Fork 789
Description
Fission/Kubernetes version
$ fission version client: fission/core: BuildDate: "2023-05-18T12:33:09Z" GitCommit: f955d118 Version: v1.19.0 server: fission/core: BuildDate: "2023-05-18T12:33:09Z" GitCommit: f955d118 Version: v1.19.0 $ kubectl version Client Version: version.Info{Major:"1", Minor:"27", GitVersion:"v1.27.2", GitCommit:"7f6f68fdabc4df88cfea2dcf9a19b2b830f1e647", GitTreeState:"clean", BuildDate:"2023-05-17T14:13:27Z", GoVersion:"go1.20.4", Compiler:"gc", Platform:"darwin/arm64"} Kustomize Version: v5.0.1 Server Version: version.Info{Major:"1", Minor:"27+", GitVersion:"v1.27.3-eks-a5565ad", GitCommit:"78c8293d1c65e8a153bf3c03802ab9358c0e1a14", GitTreeState:"clean", BuildDate:"2023-06-16T17:32:40Z", GoVersion:"go1.20.5", Compiler:"gc", Platform:"linux/amd64"}
Kubernetes platform (e.g. Google Kubernetes Engine)
EKS (eks.3)
Describe the bug
I observed that env builder pod (that has builder and fetcher container) keeps getting evicted because of over utlization of ephemeral storage. On checking the container, I found that it still has files related to the packages it built previously which are piling up (since I was building more than 100 functions together simultaneously).
To Reproduce
Create an environment (version 3) and build a lot of packages.
Expected result
Ideally once the package is built, all its files should be removed from the builder pod
Actual result
Builder Pod is keeping all the files of previously generated packages
Screenshots/Dump file
$ fission support dump
Additional context
On digging the fetcher code, it was confirmed in the UploadHandler
that even after a successful response from the ssClient.Upload
func, the files are not deleted.