Setting loadbalancer IP is not supported by the AWS cloud controller #688
Description
context
When creating a tenant control plane on AWS, we need to mention public IP in the tenant control plane spec.
issue
When applying, the kamaji controller will create a k8s service with spec.loadBalancerIP set. The AWS cloud controller will have the following warning:
Normal EnsuringLoadBalancer 8s (x2 over 13s) service-controller Ensuring load balancer
Warning EnsuringLoadBalancer 8s (x2 over 13s) service-controller Error creating load balancer (will retry): failed to ensure load balancer for xxx: LoadBalancerIP cannot be specified for AWS ELB
This is related to : kubernetes/cloud-provider-aws#48
The lb will be created with the warnings but the issue is that it will have an inconsistent updates ( for example, updating the target nodes ..)
Files for the tenant control plane creation:
---
apiVersion: kamaji.clastix.io/v1alpha1
kind: TenantControlPlane
metadata:
name: tenant-00
namespace: tenant-00
labels:
tenant.clastix.io: tenant-00
spec:
dataStore: default
controlPlane:
deployment:
replicas: 1
nodeSelector:
topology.kubernetes.io/zone: eu-west-3a
additionalMetadata:
labels:
tenant.clastix.io: tenant-00
extraArgs:
apiServer: []
controllerManager: []
scheduler: []
resources:
apiServer:
requests:
cpu: 250m
memory: 512Mi
limits: {}
controllerManager:
requests:
cpu: 125m
memory: 256Mi
limits: {}
scheduler:
requests:
cpu: 125m
memory: 256Mi
limits: {}
service:
additionalMetadata:
labels:
tenant.clastix.io: tenant-00
annotations:
service.beta.kubernetes.io/aws-load-balancer-backend-protocol: tcp
service.beta.kubernetes.io/aws-load-balancer-scheme: internet-facing
service.beta.kubernetes.io/aws-load-balancer-subnets: <public_subnet>
service.beta.kubernetes.io/aws-load-balancer-eip-allocations: eip-xxxx
service.beta.kubernetes.io/aws-load-balancer-type: nlb
external-dns.alpha.kubernetes.io/hostname: tenant-00.internal.kamaji.aws.com
serviceType: LoadBalancer
kubernetes:
version: v1.30.2
kubelet:
cgroupfs: systemd
admissionControllers:
- ResourceQuota
- LimitRanger
networkProfile:
address: <PUBLIC_IP_LINKED_TO_EIP>
port: 6443
certSANs:
- tenant-00.internal.kamaji.aws.com
serviceCidr: 10.96.0.0/16
podCidr: 10.36.0.0/16
dnsServiceIPs:
- 10.96.0.10
addons:
coreDNS: {}
kubeProxy: {}
konnectivity:
server:
port: 8132
resources:
requests:
cpu: 100m
memory: 128Mi
limits: {}