Upgrade tests is installing and upgrading the same chart version #433
Description
Is this a request for help?:
No
Is this a BUG REPORT or FEATURE REQUEST? (choose one):
Yes, unless I am misunderstanding the docs
Version of Helm and Kubernetes:
helm version
version.BuildInfo{Version:"v3.8.0", GitCommit:"d14138609b01886f544b2025f5000351c9eb092e", GitTreeState:"clean", GoVersion:"go1.17.6"}
k version
Client Version: version.Info{Major:"1", Minor:"23", GitVersion:"v1.23.5", GitCommit:"c285e781331a3785a7f436042c65c5641ce8a9e9", GitTreeState:"clean", BuildDate:"2022-03-16T15:51:05Z", GoVersion:"go1.17.8", Compiler:"gc", Platform:"darwin/amd64"}
Server Version: version.Info{Major:"1", Minor:"23", GitVersion:"v1.23.3", GitCommit:"816c97ab8cff8a1c72eccca1026f7820e93e0d25", GitTreeState:"clean", BuildDate:"2022-01-25T21:19:12Z", GoVersion:"go1.17.6", Compiler:"gc", Platform:"linux/amd64"}
What happened:
We maintain some charts, we modified by mistake a selector between two minor versions and the CI running ct install --namespace ct --upgrade did not catched the issue
How to reproduce it (as minimally and precisely as possible):
Clone a repository with a chart, modify a deployment label selector updating the version, run the tool
No error will be catched, but if you install the old version and the upgrade you would get the classical:
Error: UPGRADE FAILED: cannot patch "nri-prometheus" with kind Deployment: Deployment.apps "nri-prometheus" is invalid: spec.selector: Invalid value: v1.LabelSelector{MatchLabels:map[string]string{"app.kubernetes.io/name":"nri-prometheus"}, MatchExpressions:[]v1.LabelSelectorRequirement(nil)}: field is immutable
Anything else we need to know:
I checked the code and I cannot understand the logic of the --upgrade function, but I checked and it is the very same of the first implementation, therefore I wonder how it is possible that it is a bug:
The tool states that is going to perform an upgrade test, but that is not the case! It upgrades to the same version ct_previous_revision2130172757
Testing upgrades of chart 'nri-prometheus => (version: "2.1.2", path: "charts/nri-prometheus")' relative to previous revision 'nri-prometheus => (version: "2.1.1", path: "ct_previous_revision2130172757/charts/nri-prometheus")'...
[...]
>>> helm install nri-prometheus-y2dumyevqw ct_previous_revision2130172757/charts/nri-prometheus --namespace ct --wait --values ct_previous_revision2130172757/charts/nri-prometheus/ci/test-lowdatamode-values.yaml
>>> helm upgrade nri-prometheus-y2dumyevqw ct_previous_revision2130172757/charts/nri-prometheus --namespace ct --reuse-values --wait
I checked the code and what I do not understand is why here it is not t.helm.Upgrade(newChart.Path(), namespace, release) instead.
I tried to perform the change, I tested it and now it cached the issue.
Testing upgrades of chart 'nri-prometheus => (version: "2.1.2", path: "charts/nri-prometheus")' relative to previous revision 'nri-prometheus => (version: "2.1.1", path: "ct_previous_revision78831704/charts/nri-prometheus")'...
[...]
>>> helm install nri-prometheus-avhvhn5q0a ct_previous_revision78831704/charts/nri-prometheus --namespace ct --wait --values ct_previous_revision78831704/charts/nri-prometheus/ci/test-lowdatamode-values.yaml
>>> helm upgrade nri-prometheus-avhvhn5q0a charts/nri-prometheus --namespace ct --reuse-values --wait
Error: UPGRADE FAILED: cannot patch "nri-prometheus" with kind Deployment: Deployment.apps "nri-prometheus" is invalid: spec.selector: Invalid value: v1.LabelSelector{MatchLabels:map[string]string{"app.kubernetes.io/name":"nri-prometheus"}, MatchExpressions:[]v1.LabelSelectorRequirement(nil)}: field is immutable
Is it a bug or I am missing something?