[GEP-28] gardenadm bootstrap: Deploy and connect to Bastion
#12407
Conversation
|
Skipping CI for Draft Pull Request. |
|
@timebertt: GitHub didn't allow me to request PR reviews from the following users: maboehm. Note that only gardener members and repo collaborators can review this PR, and authors cannot review their own PRs. In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. |
|
/test pull-gardener-unit pull-gardener-e2e-kind-gardenadm |
8975c2b to
4f95b42
Compare
|
/test pull-gardener-unit pull-gardener-e2e-kind-gardenadm |
4f95b42 to
e49ca4f
Compare
|
This PR is ready, now that #12366 has been merged. |
|
/assign |
| @@ -195,7 +196,18 @@ func run(ctx context.Context, opts *Options) error { | |||
| // TODO(timebertt): add b.Shoot.Components.Extensions.Worker.Wait when | |||
| // https://github.com/gardener/machine-controller-manager/issues/994 has been implemented | |||
|
|
|||
| deployBastion = g.Add(flow.Task{ | |||
| Name: "Deploying and connecting to bastion host", | |||
There was a problem hiding this comment.
Can you outline the next step/next iteration (now that we have a bastion and an SSH connection)?
There was a problem hiding this comment.
In the next step, we can connect to the first control plane machine via the Bastion connection. However, this requires gardener/machine-controller-manager#1007 (or a temporary hack for provider-local).
Once this has been established, we can prepare the ShootState, copy it along with the manifests to the machine via SCP, and then we can execute gardenadm init there.
There was a problem hiding this comment.
Perfect, thanks!
Do you plan to run gardenadm init from within gardenadm bootstrap? I thought, the idea would be to just prepare the infrastructure and the nodes via gardenadm bootstrap, and then let the human user SSH into it (via the bastion) to run gardenadm init themselves?
There was a problem hiding this comment.
Yes, in my vision, gardenadm init would be executed by gardenadm bootstrap (at least, by default).
Now that I checked the GEP section again, I found the sentence that I had forgotten:
machine-controller-manager will also take over the creation of the control plane nodes, however, bootstrapping and joining them must still be performed by the user (meaning that step 3 remains)
But it also says:
It still needs to be investigated how much automation by machine-controller-manager is possible for the control plane nodes
Because of this, I was under the impression that we strive for as much automation in gardenadm bootstrap as possible. It would be gardenadm bootstrapping/joining the control plane nodes instead of MCM, though.
But in the end, I don't see a reason for letting the user execute gardenadm init themselves.
gardenadm bootstrap already has the manifests, the SSH connection, etc. So it would be the simplest user experience to finish gardenadm bootstrap with a fully bootstrapped autonomous shoot cluster and output the kubeconfig.
WDYT? Do you have a case in mind where the user would need to run gardenadm init themselves?
There was a problem hiding this comment.
Sure, fair enough. We thought it might be easier to not couple this in the beginning, but left this open for the future to further automate this. If you think we are good to go down this path right away, even better.
Personally, I was wondering whether the user has to anyways manually SSH into the nodes (at least the control plane node). If they want to inspect the cluster (kubectl get nodes,pods or whatever) after bootstrapping, there is no way around it (as of today, at least, since we don't expose the API server or export the kubeconfig somehow (the latter could also be automated by gardenadm bootstrap eventually)).
Yet, there is nothing blocking the user from doing so, even if gardenadm bootstrap already calls init or join. They can always use the bastion and SSH into the nodes to inspect whatever they need (e.g., in case init or join fails), right?
TL;DR: Go ahead :)
There was a problem hiding this comment.
Great, I see we're on the same page :)
Yeah, we still need to clarify such details. I can imagine an option to keep the Bastion for troubleshooting/access purposes. I can also imagine that the Infrastructure adds a Load Balancer for the control plane.
|
LGTM label has been added. Git tree hash: 41b3616deaa3c5db3ab692df1463993ea5cb1fb9
|
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: rfranzke The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
How to categorize this PR?
/area ipcei
/kind enhancement
What this PR does / why we need it:
This PR continues #12267 and deploys a
Bastioningardenadm bootstrapfor connecting to the control plane machines.gardenadm bootstrapdeploys aBastion.extensions.gardener.cloud(not aBastion.operations.gardener.cloud) with a dedicated SSH key pair. Similar to gardenctl, the user's public IPs are detected using https://ipify.org and used as ingress restrictions for theBastion(can be overwritten using--bastion-ingress-cidr). Once theBastionis healthy, an SSH connection is opened.Which issue(s) this PR fixes:
Part of #2906
Special notes for your reviewer:
/cc @maboehm @ScheererJ @rfranzke
The
Bastioncomponent in this PR usespkg/utils/sshintroduced in #12366, and the e2e tests need theBastionimplementation in provider-local:Bastioncontroller in provider-local #12366 has been mergedRelease note: