Use helm repo for integration tests instead of tarballs #42864

kfaseela · 2023-01-18T16:30:34Z

Signed-off-by: Faseela K faseela.k@est.tech

Make the helm integration tests use istio helm repo.
This is needed as part of helm graduation from Alpha to Beta (#42565)

howardjohn

I like the simplicity but am a bit concerned it goes the opposite direction of #35835. WDYT?

Since its GCS it may not be so bad since we already heavily depend on it

kfaseela · 2023-01-18T16:37:25Z

I like the simplicity but am a bit concerned it goes the opposite direction of #35835. WDYT?

Since its GCS it may not be so bad since we already heavily depend on it

Ohh!! I missed this discussion. I thought we wanted to get rid of tarballs based install everywhere as we discussed in the helm promotion task, so thought this was also needed!

Signed-off-by: Faseela K <faseela.k@est.tech>

jacob-delgado · 2023-01-18T16:57:10Z

If we don't test the helm repo then we need to figure out a way to make updating the tarballs and helm integration test. Right now it's a bit difficult to update them. It'd be nice if it was part of the release process, but it's too cumbersome to do for a release manager without understanding how they are packaged and tested.

Signed-off-by: Faseela K <faseela.k@est.tech>

keithmattix · 2023-01-18T22:19:29Z

pkg/test/helm/helm.go

@@ -35,6 +35,12 @@ func New(kubeConfig string) *Helm {
 	}
 }

+// InstallChartWithValues installs the specified chart with its given name to the given namespace
+func (h *Helm) AddRepo() error {
+	_, err := execCommand("helm repo add istio https://istio-release.storage.googleapis.com/charts")


Could we make this configurable so folks could use this test suite with their own chart repo?

Wonderful; thank you!

Signed-off-by: Faseela K <faseela.k@est.tech>

kfaseela · 2023-01-19T11:52:24Z

If we don't test the helm repo then we need to figure out a way to make updating the tarballs and helm integration test. Right now it's a bit difficult to update them. It'd be nice if it was part of the release process, but it's too cumbersome to do for a release manager without understanding how they are packaged and tested.

@howardjohn what do you think?

keithmattix · 2023-01-19T15:06:59Z

I like the simplicity but am a bit concerned it goes the opposite direction of #35835. WDYT?

Since its GCS it may not be so bad since we already heavily depend on it

With the configurable chart repo, a local web server that hosts the charts can likely be added later right?

howardjohn · 2023-01-19T15:48:54Z

yep lgtm (at ahigh level, will review the code in a bit)

howardjohn · 2023-01-19T15:53:43Z

pkg/test/helm/helm.go

@@ -35,6 +36,21 @@ func New(kubeConfig string) *Helm {
 	}
 }

+// Add Repo adds the specified helm repo to use for installing the charts
+func (h *Helm) AddRepo(t framework.TestContext) {
+	command := fmt.Sprintf("helm repo add istio %s", t.Settings().HelmRepo)


can we/should we just add --repo HelmRepo to each command? The current command will modify users global helm settings (and may conflict if they have istio defined. Especially DeleteRepo

Makes sense. Will fix

@howardjohn done

nshankar13 · 2023-01-19T21:09:40Z

@kfaseela as of now, what is the current way of running the helm e2e tests in /tests/integration/helm against helm chart tarballs that are built and packaged locally?

For instance, if we had the tarballs for base-1.16.1.tgz cni-1.16.1.tgz gateway-1.16.1.tgz istiod-1.16.1.tgz living in some /dir, how do we point the e2es to install / test against these charts?

Signed-off-by: Faseela K <faseela.k@est.tech>

kfaseela · 2023-01-23T18:01:30Z

@kfaseela as of now, what is the current way of running the helm e2e tests in /tests/integration/helm against helm chart tarballs that are built and packaged locally?

For instance, if we had the tarballs for base-1.16.1.tgz cni-1.16.1.tgz gateway-1.16.1.tgz istiod-1.16.1.tgz living in some /dir, how do we point the e2es to install / test against these charts?

currently the tarballs are hardcoded as far as i can understand, and there is no way to override. If you replace the tarballs with your own, under the testdata folder, and run the tests it will be picked up

nshankar13 · 2023-01-23T18:02:24Z

@kfaseela thank you!!

Update protobufs Add dep Derp Consider actually using TCP Hmm Fix this Missed one Add log message Logging Remove these Testing Missed this gawd Hmm Fix this Testing Ugh Whitespace Testing Invalid component Testing Fix logging Testt Add alt case Hmm Fix Fix 2 Fix routing Actually fix Fix this Don't cleanup gateways Aight I'm a goofus Tidy Testing Test Hmm Test Fix this Hmmm Oh Fix this Fix this test Hmm Test Hmm Hmm Cleanup Revert "Cleanup" This reverts commit 1f933b720c504dc82cdb5683e1e08e6d464f5f1a. Fix this Better comment Fix comment Automator: update proxy@master in istio/istio@master (istio#42865) Automator: update common-files@master in istio/istio@master (istio#42879) Automator: update istio/client-go@master dependency in istio/istio@master (istio#42880) improve wasm ecds comments (istio#42883) Signed-off-by: Rama Chavali <rama.rao@salesforce.com> Signed-off-by: Rama Chavali <rama.rao@salesforce.com> Change no server certificate verify as warning level (istio#42876) * Change no server certificate verify as warning level * update Use latest helm dependency (istio#42891) Automator: update proxy@master in istio/istio@master (istio#42898) Automator: update common-files@master in istio/istio@master (istio#42900) Automator: update istio/client-go@master dependency in istio/istio@master (istio#42901) Automator: update proxy@master in istio/istio@master (istio#42907) Cleanup unused metadata (istio#42903) Update BASE_VERSION to master-2023-01-20T04-29-21 (istio#42913) Add Locality info to debug interface (istio#42884) Switching revision in IstioOperator istio-gateway cache fix (istio#42529) * switching revision in IstioOperator istio-gateway cache fix Signed-off-by: Shubham Chauhan <shubham@tetrate.io> * include all components Signed-off-by: Shubham Chauhan <shubham@tetrate.io> Signed-off-by: Shubham Chauhan <shubham@tetrate.io> Automator: update proxy@master in istio/istio@master (istio#42923) Automator: update istio/client-go@master dependency in istio/istio@master (istio#42926) add e2e for MetricDefinition (istio#42563) (istio#42924) * add e2e for MetricDefinition * fix lint * send http traffic only * fix prom query Co-authored-by: zirain <hejianpeng2@huawei.com> tf: ensure Check is always set (istio#42908) * tf: ensure Check is always set Avoid cases where we do not test anything * Always set * a few more * fix Initial messing about for PROXY protocol tests Fix this Automator: update proxy@master in istio/istio@master (istio#42865) Automator: update common-files@master in istio/istio@master (istio#42879) Automator: update istio/client-go@master dependency in istio/istio@master (istio#42880) improve wasm ecds comments (istio#42883) Signed-off-by: Rama Chavali <rama.rao@salesforce.com> Signed-off-by: Rama Chavali <rama.rao@salesforce.com> Change no server certificate verify as warning level (istio#42876) * Change no server certificate verify as warning level * update Use latest helm dependency (istio#42891) Automator: update proxy@master in istio/istio@master (istio#42898) Automator: update common-files@master in istio/istio@master (istio#42900) Automator: update istio/client-go@master dependency in istio/istio@master (istio#42901) Automator: update proxy@master in istio/istio@master (istio#42907) Cleanup unused metadata (istio#42903) Update BASE_VERSION to master-2023-01-20T04-29-21 (istio#42913) Add Locality info to debug interface (istio#42884) Switching revision in IstioOperator istio-gateway cache fix (istio#42529) * switching revision in IstioOperator istio-gateway cache fix Signed-off-by: Shubham Chauhan <shubham@tetrate.io> * include all components Signed-off-by: Shubham Chauhan <shubham@tetrate.io> Signed-off-by: Shubham Chauhan <shubham@tetrate.io> Automator: update proxy@master in istio/istio@master (istio#42923) Automator: update istio/client-go@master dependency in istio/istio@master (istio#42926) add e2e for MetricDefinition (istio#42563) (istio#42924) * add e2e for MetricDefinition * fix lint * send http traffic only * fix prom query Co-authored-by: zirain <hejianpeng2@huawei.com> tf: ensure Check is always set (istio#42908) * tf: ensure Check is always set Avoid cases where we do not test anything * Always set * a few more * fix wasm: Send nack when there is not well-formed resource messages (istio#42928) * Send nack when there is an unmarshal error. Change-Id: I8143ba8bc8c292ce2de5abda0a4f0c70c111bf80 * change if structure to have more readability Change-Id: I076929d14dd1312b3717fe992352520223997354 * add unit tests Change-Id: I31831ea1ba87c99ffc0451a7f3a2cb02acc14d91 Automator: update proxy@master in istio/istio@master (istio#42931) Automator: update go dependencies@ in istio/istio@master (istio#42932) Fix TestDeltaEDS flake (istio#42925) improve appsv1 package import statement (istio#42946) Signed-off-by: xin.li <xin.li@daocloud.io> Signed-off-by: xin.li <xin.li@daocloud.io> Use the slices package for contains check (istio#42947) Automator: update common-files@master in istio/istio@master (istio#42948) reducing leaseduration, renewdeadline and retryperiod to shorten test time (istio#42920) add patch name to envoy filter crash log line (istio#42935) * add patch name to envoy filter crash log line Signed-off-by: Rama Chavali <rama.rao@salesforce.com> * log line change Signed-off-by: Rama Chavali <rama.rao@salesforce.com> Signed-off-by: Rama Chavali <rama.rao@salesforce.com> use authn filter constant every where (istio#42936) Signed-off-by: Rama Chavali <rama.rao@salesforce.com> Signed-off-by: Rama Chavali <rama.rao@salesforce.com> Automator: update istio/client-go@master dependency in istio/istio@master (istio#42949) Automator: update proxy@master in istio/istio@master (istio#42950) Automator: update proxy@master in istio/istio@master (istio#42953) Update master branch from 1.17 to 1.18 (istio#42875) * Update master branch from 1.17 to 1.18 * Remove older envoy filters * Fixup some version errors Update default k8s for test and buildkit (istio#42952) Use helm repo for integration tests instead of tarballs (istio#42864) * Use helm repo for integration tests instead of tarballs Signed-off-by: Faseela K <faseela.k@est.tech> * refactor Signed-off-by: Faseela K <faseela.k@est.tech> * configuration option for helm repo Signed-off-by: Faseela K <faseela.k@est.tech> * fix lint failure Signed-off-by: Faseela K <faseela.k@est.tech> * add --repo to individual commands Signed-off-by: Faseela K <faseela.k@est.tech> Signed-off-by: Faseela K <faseela.k@est.tech> Automator: update common-files@master in istio/istio@master (istio#42957)

kfaseela · 2023-01-24T18:20:01Z

/cherrypick release-1.17

istio-testing · 2023-01-24T18:20:42Z

@kfaseela: new pull request created: #42972

In response to this:

/cherrypick release-1.17

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

* Test * Add basic integration tests for PROXY protocol and gateway Update protobufs Add dep Derp Consider actually using TCP Hmm Fix this Missed one Add log message Logging Remove these Testing Missed this gawd Hmm Fix this Testing Ugh Whitespace Testing Invalid component Testing Fix logging Testt Add alt case Hmm Fix Fix 2 Fix routing Actually fix Fix this Don't cleanup gateways Aight I'm a goofus Tidy Testing Test Hmm Test Fix this Hmmm Oh Fix this Fix this test Hmm Test Hmm Hmm Cleanup Revert "Cleanup" This reverts commit 1f933b720c504dc82cdb5683e1e08e6d464f5f1a. Fix this Better comment Fix comment Automator: update proxy@master in istio/istio@master (#42865) Automator: update common-files@master in istio/istio@master (#42879) Automator: update istio/client-go@master dependency in istio/istio@master (#42880) improve wasm ecds comments (#42883) Signed-off-by: Rama Chavali <rama.rao@salesforce.com> Signed-off-by: Rama Chavali <rama.rao@salesforce.com> Change no server certificate verify as warning level (#42876) * Change no server certificate verify as warning level * update Use latest helm dependency (#42891) Automator: update proxy@master in istio/istio@master (#42898) Automator: update common-files@master in istio/istio@master (#42900) Automator: update istio/client-go@master dependency in istio/istio@master (#42901) Automator: update proxy@master in istio/istio@master (#42907) Cleanup unused metadata (#42903) Update BASE_VERSION to master-2023-01-20T04-29-21 (#42913) Add Locality info to debug interface (#42884) Switching revision in IstioOperator istio-gateway cache fix (#42529) * switching revision in IstioOperator istio-gateway cache fix Signed-off-by: Shubham Chauhan <shubham@tetrate.io> * include all components Signed-off-by: Shubham Chauhan <shubham@tetrate.io> Signed-off-by: Shubham Chauhan <shubham@tetrate.io> Automator: update proxy@master in istio/istio@master (#42923) Automator: update istio/client-go@master dependency in istio/istio@master (#42926) add e2e for MetricDefinition (#42563) (#42924) * add e2e for MetricDefinition * fix lint * send http traffic only * fix prom query Co-authored-by: zirain <hejianpeng2@huawei.com> tf: ensure Check is always set (#42908) * tf: ensure Check is always set Avoid cases where we do not test anything * Always set * a few more * fix Initial messing about for PROXY protocol tests Fix this Automator: update proxy@master in istio/istio@master (#42865) Automator: update common-files@master in istio/istio@master (#42879) Automator: update istio/client-go@master dependency in istio/istio@master (#42880) improve wasm ecds comments (#42883) Signed-off-by: Rama Chavali <rama.rao@salesforce.com> Signed-off-by: Rama Chavali <rama.rao@salesforce.com> Change no server certificate verify as warning level (#42876) * Change no server certificate verify as warning level * update Use latest helm dependency (#42891) Automator: update proxy@master in istio/istio@master (#42898) Automator: update common-files@master in istio/istio@master (#42900) Automator: update istio/client-go@master dependency in istio/istio@master (#42901) Automator: update proxy@master in istio/istio@master (#42907) Cleanup unused metadata (#42903) Update BASE_VERSION to master-2023-01-20T04-29-21 (#42913) Add Locality info to debug interface (#42884) Switching revision in IstioOperator istio-gateway cache fix (#42529) * switching revision in IstioOperator istio-gateway cache fix Signed-off-by: Shubham Chauhan <shubham@tetrate.io> * include all components Signed-off-by: Shubham Chauhan <shubham@tetrate.io> Signed-off-by: Shubham Chauhan <shubham@tetrate.io> Automator: update proxy@master in istio/istio@master (#42923) Automator: update istio/client-go@master dependency in istio/istio@master (#42926) add e2e for MetricDefinition (#42563) (#42924) * add e2e for MetricDefinition * fix lint * send http traffic only * fix prom query Co-authored-by: zirain <hejianpeng2@huawei.com> tf: ensure Check is always set (#42908) * tf: ensure Check is always set Avoid cases where we do not test anything * Always set * a few more * fix wasm: Send nack when there is not well-formed resource messages (#42928) * Send nack when there is an unmarshal error. Change-Id: I8143ba8bc8c292ce2de5abda0a4f0c70c111bf80 * change if structure to have more readability Change-Id: I076929d14dd1312b3717fe992352520223997354 * add unit tests Change-Id: I31831ea1ba87c99ffc0451a7f3a2cb02acc14d91 Automator: update proxy@master in istio/istio@master (#42931) Automator: update go dependencies@ in istio/istio@master (#42932) Fix TestDeltaEDS flake (#42925) improve appsv1 package import statement (#42946) Signed-off-by: xin.li <xin.li@daocloud.io> Signed-off-by: xin.li <xin.li@daocloud.io> Use the slices package for contains check (#42947) Automator: update common-files@master in istio/istio@master (#42948) reducing leaseduration, renewdeadline and retryperiod to shorten test time (#42920) add patch name to envoy filter crash log line (#42935) * add patch name to envoy filter crash log line Signed-off-by: Rama Chavali <rama.rao@salesforce.com> * log line change Signed-off-by: Rama Chavali <rama.rao@salesforce.com> Signed-off-by: Rama Chavali <rama.rao@salesforce.com> use authn filter constant every where (#42936) Signed-off-by: Rama Chavali <rama.rao@salesforce.com> Signed-off-by: Rama Chavali <rama.rao@salesforce.com> Automator: update istio/client-go@master dependency in istio/istio@master (#42949) Automator: update proxy@master in istio/istio@master (#42950) Automator: update proxy@master in istio/istio@master (#42953) Update master branch from 1.17 to 1.18 (#42875) * Update master branch from 1.17 to 1.18 * Remove older envoy filters * Fixup some version errors Update default k8s for test and buildkit (#42952) Use helm repo for integration tests instead of tarballs (#42864) * Use helm repo for integration tests instead of tarballs Signed-off-by: Faseela K <faseela.k@est.tech> * refactor Signed-off-by: Faseela K <faseela.k@est.tech> * configuration option for helm repo Signed-off-by: Faseela K <faseela.k@est.tech> * fix lint failure Signed-off-by: Faseela K <faseela.k@est.tech> * add --repo to individual commands Signed-off-by: Faseela K <faseela.k@est.tech> Signed-off-by: Faseela K <faseela.k@est.tech> Automator: update common-files@master in istio/istio@master (#42957) * Fix these * Lint fixes * Fix comments * Lint fixes * More lint fixes * Be slightly more efficient with gateway creation * Lints * Update tests/integration/pilot/common/routing.go Co-authored-by: Lin Sun <lin.sun@solo.io> --------- Co-authored-by: Lin Sun <lin.sun@solo.io>

kfaseela requested a review from a team as a code owner January 18, 2023 16:30

istio-policy-bot added area/test and release release-notes-none Indicates a PR that does not require release notes. labels Jan 18, 2023

istio-testing added the size/M Denotes a PR that changes 30-99 lines, ignoring generated files. label Jan 18, 2023

kfaseela changed the title ~~Use helm repo for integration tests instead of tarballs~~ WIP: Use helm repo for integration tests instead of tarballs Jan 18, 2023

istio-testing added the do-not-merge/work-in-progress Block merging of a PR because it isn't ready yet. label Jan 18, 2023

howardjohn reviewed Jan 18, 2023

View reviewed changes

kfaseela mentioned this pull request Jan 18, 2023

Helm: Promote to Beta - Status #42565

Closed

5 tasks

Use helm repo for integration tests instead of tarballs

010d614

Signed-off-by: Faseela K <faseela.k@est.tech>

kfaseela force-pushed the update-helm-upgrade-tests branch from ee6e256 to 010d614 Compare January 18, 2023 16:51

refactor

95e68be

Signed-off-by: Faseela K <faseela.k@est.tech>

istio-testing added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. and removed size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Jan 18, 2023

keithmattix reviewed Jan 18, 2023

View reviewed changes

kfaseela added 2 commits January 19, 2023 11:12

configuration option for helm repo

498c859

Signed-off-by: Faseela K <faseela.k@est.tech>

fix lint failure

bb21b6f

Signed-off-by: Faseela K <faseela.k@est.tech>

kfaseela changed the title ~~WIP: Use helm repo for integration tests instead of tarballs~~ Use helm repo for integration tests instead of tarballs Jan 19, 2023

istio-testing removed the do-not-merge/work-in-progress Block merging of a PR because it isn't ready yet. label Jan 19, 2023

howardjohn reviewed Jan 19, 2023

View reviewed changes

add --repo to individual commands

b8e7d63

Signed-off-by: Faseela K <faseela.k@est.tech>

howardjohn approved these changes Jan 23, 2023

View reviewed changes

istio-testing merged commit 8fd4f3b into istio:master Jan 23, 2023

istio-testing mentioned this pull request Jan 24, 2023

[release-1.17] Use helm repo for integration tests instead of tarballs #42972

Merged

kfaseela deleted the update-helm-upgrade-tests branch March 14, 2023 13:03

kfaseela mentioned this pull request Jun 16, 2023

Add kfaseela as Test and Release maintainer istio/community#1090

Merged

Use helm repo for integration tests instead of tarballs #42864

Use helm repo for integration tests instead of tarballs #42864

Uh oh!

Conversation

kfaseela commented Jan 18, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

howardjohn left a comment

Choose a reason for hiding this comment

Uh oh!

kfaseela commented Jan 18, 2023

Uh oh!

jacob-delgado commented Jan 18, 2023

Uh oh!

keithmattix Jan 18, 2023

Choose a reason for hiding this comment

Uh oh!

kfaseela Jan 19, 2023

Choose a reason for hiding this comment

Uh oh!

keithmattix Jan 19, 2023

Choose a reason for hiding this comment

Uh oh!

kfaseela commented Jan 19, 2023

Uh oh!

keithmattix commented Jan 19, 2023

Uh oh!

howardjohn commented Jan 19, 2023

Uh oh!

howardjohn Jan 19, 2023

Choose a reason for hiding this comment

Uh oh!

kfaseela Jan 19, 2023

Choose a reason for hiding this comment

Uh oh!

kfaseela Jan 23, 2023

Choose a reason for hiding this comment

Uh oh!

nshankar13 commented Jan 19, 2023

Uh oh!

kfaseela commented Jan 23, 2023

Uh oh!

nshankar13 commented Jan 23, 2023

Uh oh!

kfaseela commented Jan 24, 2023

Uh oh!

istio-testing commented Jan 24, 2023

Uh oh!

Uh oh!

kfaseela commented Jan 18, 2023 •

edited

Loading