Modified version of #41015 (from 1.16)

This issue aims to track all steps needed to prepare the release 1.17 branches/builds.

This issue was created early to help document the steps using some new automation as well as any ordering changes, etc. Please update/add additional steps taken here so we can document for future releases.

Note that the actual coding steps done by the automation steps are left in this issue for reference only. You only need to run the automated steps, and not the individual instructions. Also note that the reference steps may be INCOMPLETE as the automation is updated, not necessarily the reference steps.

For now, the code to run is in https://github.com/istio/release-builder master branch. From within that branch, one can do a make shell and then within the shell run REPO_ORG=myorg STEP=x ./release/branch.sh (you can update the STEP in the trigger-branch file and not set it manually). This generates the changes for review (trygit status or git diff). Adding DRY_RUN=false will cause the PRs to be created in the REPO_ORG.

Note that the automated steps 3 and on require the new release's branch in the various repos which are created by automated step 2.

• Set up release manager team - Make sure the new team has write access to the repositories. Istio_1.17_RMs

• Create a slack channel for the 1.17 release. Edit the description/topic of the channel to include RM's for the release, expected release date, etc.

• Send an announcement on Discuss pre-announcing branch cut (adding, but are branch cut notifications needed and where?)

• Send an announcement on Discuss announcing the start of branch cut and to stop merging of PRs (adding, but are branch cut notifications needed and where?)

• (Automation step=1) Update dependencies. This must be done before branching so that PRs can be easily backported later. PR

  • Run go get github.com/envoyproxy/go-control-plane@main to update (Added in 1.17 in Add additional item to the Step=1 branch cut process release-builder#1332).
  • Run the following command in istio/istio UPDATE_BRANCH=master ./bin/update_deps.sh; make gen (example)
  • Merge PR in istio/istio. Wait for it to complete.
  • Need to verify git code to allow commits and pulls to work for upcoming 1.18 branch cut.

• (Automation step=2) (new step because we are waiting on prior change to master before branching) Create a release-1.17 branch in every Istio repo - run commands export org=xyz ; export repo=xyz ; (git clone git@github.com:${org}/${repo}.git && cd $repo && git checkout -b release-1.17 && git push --set-upstream origin release-1.17).

  • istio/istio
  • istio/pkg
  • istio/api
  • istio/envoy (NOT done as part of release-builder) ([x] Should update code to include this again? It seems Step=4 needs it, so will add it back) PR to add it back
  • istio/proxy
  • istio/client-go
  • istio/tools
  • istio/common-files
  • istio/release-builder
  • istio/enhancements
  • Explicitly skipped: istio-releases/pipeline, istio/tests, istio/istio.io, istio/test-infra, istio/gogo-genproto (No longer used and not done as part of release-builder 1.15 PR)
  • No PRs to merge as the new branches were created by automation

• Set up branch protection. Modify prow/config.yaml in test-infra to add release managers as owners for a branch. 1.17 PR

• (Automation/Not very automated step=3) (doesn't work locally, appears to need credential to add a tag to an existing build-tools image)(new step because this might need to be done separately) Set up prow on release branches go run generate.go branch 1.17 (Note: Needed to run postsubmit in next step) (Contact Googler or ping in #test-and-release slack channel for help). 1.17 PR

  • Manual steps if Googler can't run automation:
    • Have Googler or someone with creds tag new build-tools, build-tools-proxy and build-tools-centos images, for example (replace hash with latest):
      • gcloud container images add-tag gcr.io/istio-testing/build-tools:master-81af2db75fc3fa7d1a27b08f6c2065aef10c5bb2 gcr.io/istio-testing/build-tools:release-1.17-81af2db75fc3fa7d1a27b08f6c2065aef10c5bb2
      • gcloud container images add-tag gcr.io/istio-testing/build-tools-proxy:master-81af2db75fc3fa7d1a27b08f6c2065aef10c5bb2 gcr.io/istio-testing/build-tools-proxy:release-1.17-81af2db75fc3fa7d1a27b08f6c2065aef10c5bb2
      • gcloud container images add-tag gcr.io/istio-testing/build-tools-centos:master-81af2db75fc3fa7d1a27b08f6c2065aef10c5bb2 gcr.io/istio-testing/build-tools-centos:release-1.17-81af2db75fc3fa7d1a27b08f6c2065aef10c5bb2
    • Create new branch off of latest test-infra master branch
    • Run make shell
    • You'll need to run a couple go get's for the tooling: go get github.com/kballard/go-shellquote go get istio.io/test-infra/tools/prowgen/pkg -- don't commit the modified go.mod and go.sum from these commands
    • Generate the jobs: go run tools/prowgen/cmd/prowgen/main.go branch 1.17
    • Generate the private jobs: go run ./tools/generate-transform-jobs/main.go branch 1.17
    • In the job configs prow/**/*-1.17.yaml, replace any image build-tools(-centos|-proxy):master with build-tools(-centos|-proxy):release-1.17
    • Verify there is no duplicated env entries of the following in the configs:

      - name: BUILD_WITH_CONTAINER
        value: "0"
      

    • Manually update testgrid/config.yaml manually, see last PR for entries to add and substitute the release (there is tooling to do this, just not sure where yet)
    • Manually update BASE_VERSION in the istio-{release} yaml from master to {release}
    • Run make gen
    • Update test-infra code to handle duplicate entries as well as master image changes done manually above for upcoming 1.18 branch cut.
  • Merge PR in istio/test-infra. Wait for it to complete.

• (Automation step=4) (new step since we want the automation from prior step to actually create the image after PR merges) PRS: 1 - istio/pkg 2 - istio/client-go 3 - istio/tools 4 - istio/common-files 5 - istio/enhancements 6 - istio/release-builder 7 - istio/istio 8 - istio/api 9 - istio/proxy 10 - istio/envoy

  • Updates istio/tools to build new release-1.17 build image (update BRANCH in docker/build-tools/build-and-push.sh. PR postsubmit will create new container images whose name will be used in the next step)
  • Update common-files in new release. You first have to manually update the common/Makefile.common.mk update-common target to point to the new release (but not in common-files). In Step=5, the prow automation will actually call make update-common in these repos to do the actual make update-common against the new common-files release branch
    • istio/common-files - skipped. Covered in next step
    • istio/istio
    • istio/pkg
    • istio/api
    • istio/client-go
    • istio/proxy
    • istio/release-builder
    • istio/tools
  • Update CODEOWNERS to contain only the release managers for this release. Command: export org=xyz ; export repo=xyz ; (git clone git@github.com:${org}/${repo}.git && cd $repo && git checkout release-1.17 && git checkout -b release-1.17-codeowners && echo '* @istio/release-managers-1.17' > CODEOWNERS && git add CODEOWNERS && git commit -m 'Set release managers as CODEOWNERS for release-1.17' && git push --set-upstream origin release-1.17-codeowners).
    • istio/common-files - skipped. Covered in next step
    • istio/istio
    • istio/pkg
    • istio/api
    • istio/proxy
    • istio/client-go
    • istio/tools
    • istio/release-builder
    • istio/enhancements
  • Stop publishing latest tags
  • Update istio/release-builder branch changes from master to new release in build.sh, publish.sh and manifests.
  • Merge PRs from STEP 4. Wait for the new build images to be created at https://gcr.io/istio-testing/build-tools.
    WARNING -- DO NOT MERGE As part of the STEP 4 PR post-submits, a new PR will be created in the istio/common-files repo with a title like: Automator: update build-tools image@release-1.17 in istio/common-files@release-1.17. The work in this PR is only a portion of Step 5, and merging it will cause issues with reverting the repos to the main branch common-files. CLOSE this PR.

• (Automation step=5) (new step since we need image from prior step)

  • Update istio/common-files to set release-1.17 build image (Update the UPDATE_BRANCH in files/common/Makefile.common.mk to be the new release name
  • Update IMAGE_VERSION in files/common/scripts/setup_env.sh to be the new build image from prior step (found at https://gcr.io/istio-testing/build-tools).
  • Also Update CODEOWNERS to contain only the release managers for this release. 1.17 PR
  • Merge PR from STEP 5. Wait until step 5 PR merges and all the postsubmit generated PRs (to update common files in repos) merge.
  • Fix code getting image name to not use the arm version. PR

Additional steps that still need automation (if possible)

• Add new label: cherrypick/release-1.17: PR

• Create a 1.17 milestone: PR

• Fix up proxy automated job to pull from correct envoyproxy release branch if possible (set UPDATE_BRANCH). example (** NOTE there may need to be a revert of a PR for updates in main branch post Envoy 1.23.**) 1.17 PR

• Also, a manual update can be done to fix up the proxy and istio repos to point to latest in that branch. example - Not needed as automation already picked this change up.

• Bump master to version 1.17 in the release-1.17 branches

  • BASE_VERSION should update when a new base image is created (vulnerabilities scan fails)
  • UPDATE_BRANCH in update_deps.sh
  • istio PR-1
  • istio PR-2 (change actually in update_deps.PR).
  • test-infra PR

• Bump 1.17 to 1.18 in the master branch

  • Update Istio: 1.18.0 to 1.18.0, 1.18-dev to 1.18-dev, etc: 1.16 PR
  • istio PR

• Send an announcement on Discuss announcing branch cut complete and PRs can be merged again (adding, but are branch cut notifications needed and where?)

• Ask istio.io team to run job to update to use the 1.17 branch (probably @ericvn) PR

• Publish a beta release (from https://docs.google.com/document/d/1ilwz-HGSeiRDCFwP5rm-5d_UY6OOYL_nflx7bNggek0/edit#heading=h.qex63c29z2to, builds on release branch are labeled beta. Updating below names to reflect that.

  • Run bin/update_deps.sh in istio/istio PR
  • Trigger a 1.17 beta.0 build PR
  • Some verification to verify build is good
    • You can download the build tar from the appropriate directory here: https://gcsweb.istio.io/gcs/istio-prerelease/prerelease/
    • untar the file and install, specifying the image repository. (ex: ./bin/istioctl install --set profile=demo -y --set hub=gcr.io/istio-prerelease-testing).
    • Run bookinfo from within the untar'd directory
  • Publish a 1.17 beta.0 release

• Send an announcement on Discuss announcing availability of beta. May wait and add something about starting the first round of community testing. (Done in Slack) Discuss

• Prepare for Community Testing

  • Update the community testing worksheet
  • Updating the worksheet includes verifying the priorities and checking the tests are automated or not

• Verify the min and max K8s version supported for the release.

  • RMs usually determine the range of k8s versions and it appears on the TOC agenda at times. The max version is usually is the current K8s version and the min version is 3 past versions.

• Run the long-running tests on the beta and rc builds as you can. Unfortunately, Sam Naser seems to be the only person that knows how and can do this. Plan to run the long running tests on the latest rc build before the actual release. It take almost 48hrs for the test to run, so factor the time.

• Prepare the release notes for the release. Even though a major part of this is automated, it still takes time to clean up, sort and group the release notes. It also takes time for reviewers to review the content and make sure it is accurate. So factor in the time. Sample release note PR for 1.17
  - To generate release notes, run the command ./gen-release-notes --notes ../../../istio --oldBranch 1.16.0 --newBranch release-1.17 (against the istio/istio and istio/api repos). Follow the instructions from here.

• Each time you publish a beta/rc/official release build, make sure that the build/helm charts/images are published correctly to Istio release repo/Helm repo/gcr repo.

• After official release, send out an announcement on announcement and release slack channel for Istio. Currently Craig Box handles all the twitter announcements related to Istio and he usually sends release related announcements on twitter. Coordinate with him. discuss, twitter

• Update the description in the release slack channel to reflect the official release date.