ref: #9598
This creates a new go module for the e2e code and the kubetest2 skeleton.
Most of the kubetest2 code was copied from sigs.k8s.io/kubetest2/kubetest2-gke.

Currently only building (make gcs-publish-ci) is in place.
I used test-infra/scenarios/kubernetes_e2e.py as reference, removing environment and make variables that are no longer needed.

Notes:

• This is a new go module so that anyone importing kops does not end up with transitive dependencies used only by e2e testing. (gopls users may need to update to 0.5.1 for multi-module workspace support)

• I elected not to vendor the dependencies as suggested by Justin

• I envision kops-specific e2e tests could live in tests/e2e/ as well as Cluster and IG template files from which we can launch clusters. (we'll need to templatize them to adjust the cluster name, region and other fields which are specific to each job run)

• For presubmits we'll compile kubetest2-kops with the same commit being tested, and install kubetest2 with the version defined in go.mod. Because periodics don't clone the kops repo we'll need a way to use a specific kubetest2-kops binary. A few ideas for this:

  • Build and publish the kubetest2 and kubetest2-kops binaries on postsubmit along with the other kops artifacts. This may get into a chicken/egg problem - periodics would need to pull the kops artifacts down before running any kubetest2 commands (I was originally assuming kubetest2-kops would download the artifacts)
  • Build and promote an image that we'll use for prow jobs - a postsubmit job will build & push the image and we'll manually bump the image tags in the test-infra job definitions whenever we want periodics to use a new version of kubetest2-kops.
  • Have the periodic jobs clone the kops repo using extra_refs and build kubetest2 and kubetest2-kops the same way that presubmits would.

  We'll need to keep in mind release branching and which versions of kubetest2-kops we use to test which release branches of kops.

Next Steps:

• Get our CI / verify jobs to play nicely with the new go module (i was envisioning we could eventually move tools.go into its own go module too)
• Setup an optional presubmit prow job that performs the instructions below
• Implement Up() and Down() for AWS with just kops create cluster and kops delete cluster. This will require determining all the environment variables and file paths that are in place for the prow jobs, including setting up the boskos client.

Instructions:

cd tests/e2e
go install sigs.k8s.io/kubetest2
go install ./kubetest2-kops
# runs `make gcs-publish-ci` and exits
kubetest2 kops --build --stage-location=gs://foobar/ --kops-root=../../

Help text:

kubetest2 kops --help
Usage:
  kubetest2 kops [Flags] [DeployerFlags] -- [TesterArgs]

Flags:
      --artifacts string   top-level directory to put artifacts under for each kubetest2 run, defaulting to "${ARTIFACTS:-./_artifacts}". If using the ginkgo tester, this must be an absolute path. (default "src/k8s.io/kops/tests/e2e/_artifacts")
      --build              build kubernetes
      --down               tear down the test cluster
  -h, --help               display help
      --run-id string      unique identifier for a kubetest2 run (default "78e3e602-c079-403f-a0cd-8319b7993cc2")
      --test string        test type to run, if unset no tests will run
      --up                 provision the test cluster

DeployerFlags(kops):
      --add_dir_header                   If true, adds the file directory to the header of the log messages
      --alsologtostderr                  log to standard error as well as files
      --kops-root string                 Path to root of the kops repo. Used with --build.
      --log_backtrace_at traceLocation   when logging hits line file:N, emit a stack trace (default :0)
      --log_dir string                   If non-empty, write log files in this directory
      --log_file string                  If non-empty, use this log file
      --log_file_max_size uint           Defines the maximum size a log file can grow to. Unit is megabytes. If the value is 0, the maximum file size is unlimited. (default 1800)
      --logtostderr                      log to standard error instead of files (default true)
      --skip_headers                     If true, avoid header prefixes in the log messages
      --skip_log_headers                 If true, avoid headers when opening log files
      --stage-location string            Storage location for kops artifacts. Only gs:// paths are supported.
      --stderrthreshold severity         logs at or above this threshold go to stderr (default 2)
  -v, --v Level                          number for the log level verbosity
      --vmodule moduleSpec               comma-separated list of pattern=N settings for file-filtered logging