Description

The k8s_test_setup rule can inadvertently use the wrong cluster if there is more than one kubeconfig defined in the WORKSPACE. It currently has the keyword _cluster, which cannot be overridden, as it's private. If there is more than one kubeconfig and it happens to have the name k8s_test it will use that instead of the new config that you may define.

The primary issue is that even if you wanted to have a kubeconfig with a different name other than k8s_test, it will not work as the _cluster attributes always assumes the kubeconfig will always use the label k8s_test.

Related Issue

Motivation and Context

1). Allow ability to use a Kubeconfig that doesn't necessarily need to be named k8s_test.
2). Allow ability to use multiple Kubeconfigs, where one may be named k8s_test.

How Has This Been Tested?

I have two kubeconfigs, one for our CI cluster, and one for my local cluster running on my Mac as follow:

kubeconfig(
    name = "k8s_test",
    cluster = "it-<REDACTED>",
    use_host_config = True,
)

kubeconfig(
    name = "local_test",
    cluster = "kind-kind",
    use_host_config = True,
)

This config was then used in k8s_test_setup as follows:

k8s_test_setup(
    name = "service_it.setup",
    kubeconfig = "@local_test//:kubeconfig",
    kubectl = "@local_test//:kubectl",
    cluster = "@local_test//:cluster",
    objects = [
       "<REDACTED>"
    ],
)

When running associated tests that use k8s_test_setup, I can see that the objects are deployed to the "kind-kind" cluster.

Types of changes

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

• I have signed the Adobe Open Source CLA.
• My code follows the code style of this project.
• My change requires a change to the documentation.
• I have updated the documentation accordingly.
• I have read the CONTRIBUTING document.
• I have added tests to cover my changes.
• All new and existing tests passed.