Add support for using existing VPC for ipv6 cluster #4398

aclevername · 2021-11-02T14:33:21Z

Description

Manual test:

---
apiVersion: eksctl.io/v1alpha5
kind: ClusterConfig

metadata:
  name: jk-existing
  region: us-west-2
  version: "1.21"

vpc:
  ipFamily: IPv6
  id: "vpc-0492baf606b68fcde"  # (optional, must match VPC ID used for each subnet below)
  subnets:
    # must provide 'private' and/or 'public' subnets by availibility zone as shown
    private:
      us-west-2a:
        id: "subnet-0937638d4685f935f"
      us-west-2c:
        id: "subnet-0f9bc36d056efab79"
      us-west-2d:
        id: "subnet-0d9d4e2b7f751bacf"
    public:
      us-west-2a:
        id: "subnet-054fc9c0692c17d47"
      us-west-2c:
        id: "subnet-0e1a368ce55b8623b"
      us-west-2d:
        id: "subnet-03d71f7ea27d5f53c"

addons:
  - name: vpc-cni
  - name: coredns
  - name: kube-proxy

iam:
  withOIDC: true

managedNodeGroups: []

output:

eksctl create cluster -f examples/29-vpc-with-ip-family.yaml
2021-11-03 13:30:34 [ℹ]  eksctl version 0.70.0-dev+884cf616.2021-11-03T13:18:52Z
2021-11-03 13:30:34 [ℹ]  using region us-west-2
2021-11-03 13:30:36 [✔]  using existing VPC (vpc-0492baf606b68fcde) and subnets (private:map[us-west-2a:{subnet-0937638d4685f935f us-west-2a 192.168.160.0/19} us-west-2c:{subnet-0f9bc36d056efab79 us-west-2c 192.168.128.0/19} us-west-2d:{subnet-0d9d4e2b7f751bacf us-west-2d 192.168.96.0/19}] public:map[us-west-2a:{subnet-054fc9c0692c17d47 us-west-2a 192.168.64.0/19} us-west-2c:{subnet-0e1a368ce55b8623b us-west-2c 192.168.32.0/19} us-west-2d:{subnet-03d71f7ea27d5f53c us-west-2d 192.168.0.0/19}])
2021-11-03 13:30:36 [!]  custom VPC/subnets will be used; if resulting cluster doesn't function as expected, make sure to review the configuration of VPC/subnets
2021-11-03 13:30:36 [ℹ]  using Kubernetes version 1.21
2021-11-03 13:30:36 [ℹ]  creating EKS cluster "jk-existing" in "us-west-2" region with
2021-11-03 13:30:36 [ℹ]  will create a CloudFormation stack for cluster itself and 0 nodegroup stack(s)
2021-11-03 13:30:36 [ℹ]  will create a CloudFormation stack for cluster itself and 0 managed nodegroup stack(s)
2021-11-03 13:30:36 [ℹ]  if you encounter any issues, check CloudFormation console or try 'eksctl utils describe-stacks --region=us-west-2 --cluster=jk-existing'
2021-11-03 13:30:36 [ℹ]  CloudWatch logging will not be enabled for cluster "jk-existing" in "us-west-2"
2021-11-03 13:30:36 [ℹ]  you can enable it with 'eksctl utils update-cluster-logging --enable-types={SPECIFY-YOUR-LOG-TYPES-HERE (e.g. all)} --region=us-west-2 --cluster=jk-existing'
2021-11-03 13:30:36 [ℹ]  Kubernetes API endpoint access will use default of {publicAccess=true, privateAccess=false} for cluster "jk-existing" in "us-west-2"
2021-11-03 13:30:36 [ℹ]  3 sequential tasks: { create cluster control plane "jk-existing", set AssignIpv6AddressOnCreation to true for public subnets, 5 sequential sub-tasks: { wait for control plane to become ready, associate IAM OIDC provider, no tasks, restart daemonset "kube-system/aws-node", 1 task: { create addons } } }
2021-11-03 13:30:36 [ℹ]  building cluster stack "eksctl-jk-existing-cluster"
2021-11-03 13:30:37 [ℹ]  deploying stack "eksctl-jk-existing-cluster"
2021-11-03 13:34:12 [ℹ]  waiting for CloudFormation stack "eksctl-jk-existing-cluster"
...
2021-11-03 13:46:49 [ℹ]  daemonset "kube-system/aws-node" restarted
2021-11-03 13:48:52 [ℹ]  creating role using recommended policies
2021-11-03 13:48:53 [ℹ]  deploying stack "eksctl-jk-existing-addon-vpc-cni"
2021-11-03 13:48:53 [ℹ]  waiting for CloudFormation stack "eksctl-jk-existing-addon-vpc-cni"
2021-11-03 13:49:10 [ℹ]  waiting for CloudFormation stack "eksctl-jk-existing-addon-vpc-cni"
2021-11-03 13:49:28 [ℹ]  waiting for CloudFormation stack "eksctl-jk-existing-addon-vpc-cni"
2021-11-03 13:49:30 [ℹ]  creating addon
2021-11-03 13:49:31 [ℹ]  successfully created addon
2021-11-03 13:49:31 [ℹ]  waiting for the control plane availability...
2021-11-03 13:49:31 [✔]  saved kubeconfig as "/home/jake/.kube/config"
2021-11-03 13:49:31 [ℹ]  no tasks
2021-11-03 13:49:31 [✔]  all EKS cluster resources for "jk-existing" have been created
2021-11-03 13:51:34 [ℹ]  no recommended policies found, proceeding without any IAM
2021-11-03 13:51:34 [ℹ]  creating addon
2021-11-03 13:51:35 [ℹ]  successfully created addon
2021-11-03 13:51:35 [ℹ]  no recommended policies found, proceeding without any IAM
2021-11-03 13:51:35 [ℹ]  creating addon
2021-11-03 13:51:35 [ℹ]  successfully created addon
2021-11-03 13:51:38 [ℹ]  kubectl command should work with "/home/jake/.kube/config", try 'kubectl get nodes'
2021-11-03 13:51:38 [✔]  EKS cluster "jk-existing" in "us-west-2" region is ready

inspecting resoures created:

ClusterSharedNodeSecurityGroup
ControlPlane
ControlPlaneSecurityGroup
IngressDefaultClusterToNodeSG-
IngressInterNodeGroupSG
IngressNodeToDefaultClusterSG
PolicyCloudWatchMetrics
PolicyELBPermissions
ServiceRole

Checklist

Added tests that cover your change (if possible)
Added/modified documentation as required (such as the README.md, or the userdocs directory)
Manually tested
Made sure the title of the PR is a good description that can go into the release notes
(Core team) Added labels for change area (e.g. area/nodegroup) and kind (e.g. kind/improvement)

BONUS POINTS checklist: complete for good vibes and maybe prizes?! 🤯

Backfilled missing tests for code in same general area 🎉
Refactored something and made the world a better place 🌟

pkg/cfn/builder/vpc_existing.go

Skarlso · 2021-11-08T13:34:13Z

Okay, so is this mostly refactoring? It's a bit hard to tell because of the moving code. The only thing relevant is the checking for ipv6 association, right?

aclevername · 2021-11-08T15:24:50Z

Okay, so is this mostly refactoring? It's a bit hard to tell because of the moving code. The only thing relevant is the checking for ipv6 association, right?

basically yeah. Before the existing VPC code was inside the ipv4 resource set, now its in a separate resource set to make it generic

nikimanoledaki

we should also check whether there's an integration test for creating a new cluster with an existing VPC for v4, and add one if not. imo we can choose to add a test for either v4 or v6 if one doesn't exist because it's pretty expensive so we don't need the same test for both v4 and v6.

pkg/cfn/builder/vpc_existing.go

nikimanoledaki · 2021-11-09T11:47:45Z

Everything looks good, but I checked and couldn't find an integration test for this feature at all with IPv4! :/ What do you think of adding a test as part of this PR that uses the VPC ID of a cluster created earlier on in a test suite, and then creates a new cluster with that VPC? Could be node-less for time efficiency!

aclevername · 2021-11-09T13:30:24Z

Everything looks good, but I checked and couldn't find an integration test for this feature at all with IPv4! :/ What do you think of adding a test as part of this PR that uses the VPC ID of a cluster created earlier on in a test suite, and then creates a new cluster with that VPC? Could be node-less for time efficiency!

I think thats a good shout! can this be a separate issue/PR? I think it might fall into the general testing areas improvements. It can be either ipv4 or ipv6 so shouldn't matter.

nikimanoledaki

(update) after discussing this we've agreed to add an integration test for this feature as part of this PR! 👍 🌟

pkg/apis/eksctl.io/v1alpha5/validation_test.go

cPu1 · 2021-11-12T08:22:42Z

pkg/apis/eksctl.io/v1alpha5/validation_test.go

+					cfg.VPC.IPv6Pool = "bar"
+					cfg.VPC.ID = "123"
+					err = cfg.ValidateVPCConfig()
+					Expect(err).To(MatchError("cannot provide VPC.IPv6Cidr when using a pre-existing VPC.ID"))


Before matching the error, a good practice is to precede it with Expect(err).To(HaveOccurred()):

Suggested change

Expect(err).To(MatchError("cannot provide VPC.IPv6Cidr when using a pre-existing VPC.ID"))

Expect(err).To(HaveOccurred())

Expect(err).To(MatchError("cannot provide VPC.IPv6Cidr when using a pre-existing VPC.ID"))

I think thats no longer the case? I believe MatchError gracefully handles a nill error response

yeah it handles it gracefully now:

It("uses the existing VPC", func() { Expect(addErr).NotTo(HaveOccurred()) Expect(addErr).To(MatchError("foo"))

• Failure [0.000 seconds] Existing VPC /home/jake/weave/eksctl/pkg/cfn/builder/vpc_existing_test.go:20 CreateTemplate /home/jake/weave/eksctl/pkg/cfn/builder/vpc_existing_test.go:53 uses the existing VPC [It] /home/jake/weave/eksctl/pkg/cfn/builder/vpc_existing_test.go:69 Expected an error, got nil /home/jake/weave/eksctl/pkg/cfn/builder/vpc_existing_test.go:71

I believe MatchError gracefully handles a nill error response

I didn't mean to suggest that MatchError doesn't handle nil errors. I more meant that it's good for readability to ensure that an error has occurred before making any assertions on it.

I don't really agree it improves readability tbh, in the past I used to do this pattern because MatchError didn't handle nil errors' gracefully, but since thats changed I don't think its worth it anymore

pkg/cfn/builder/vpc_existing.go

pkg/cfn/builder/vpc_existing_test.go

Co-authored-by: Chetan Patwal <cPu1@users.noreply.github.com>

aclevername · 2021-11-17T10:09:18Z

Green integration test:

[0] • [SLOW TEST:295.102 seconds]
[0] (Integration) [using existing VPC]
[0] /home/jake/weave/eksctl/integration/tests/existing_vpc/existing_vpc_test.go:43
[0]   supports creating managed and unmanaged nodegroups in the existing VPC
[0]   /home/jake/weave/eksctl/integration/tests/existing_vpc/existing_vpc_test.go:89
[0] ------------------------------
[0]
[0] JUnit report was created: /home/jake/weave/eksctl/test-results/__home__jake__weave__eksctl__integration__tests__existing__vpc__existing__vpc__test__go_1.xml
[0]
[0] Ran 1 of 1 Specs in 1169.281 seconds
[0] SUCCESS! -- 1 Passed | 0 Failed | 0 Pending | 0 Skipped
[0] PASS

stale

nikimanoledaki

awesome, couldn't have been easy to write this! I just added some suggestions mainly for readability.

integration/tests/existing_vpc/existing_vpc_test.go

nikimanoledaki

LGTM! 🎉 nice one!!

Add integration test as well for general existing-vpc support, not ipv6 specific.

* Add ipFamily to config * Generated sources and added some more tests * Change ip to IP and update schema description * Managed addons check (#4297) * Check if managed addons are set when ipv6 is enabled * Update userdocs/src/usage/vpc-networking.md Co-authored-by: Jake Klein <jakelarsj@gmail.com> Co-authored-by: Jake Klein <jakelarsj@gmail.com> * Add OIDC check for ipv6 cluster (#4298) * Check if managed addons are set when ipv6 is enabled * Add OIDC check for ipv6 cluster * Ipv6 version check (#4300) * Check if managed addons are set when ipv6 is enabled * Add OIDC check for ipv6 cluster * Add version check if ipv6 is defined * Service ipv4 cidr is not supported with IPv6 (#4314) * Check if managed addons are set when ipv6 is enabled * Add OIDC check for ipv6 cluster * Add version check if ipv6 is defined * Setting vpc.NAT is not allowed with ipv6 * serviceIPv4CIDR is not supported with ipv6 * Auto allocate ipv6 (#4315) * Check if managed addons are set when ipv6 is enabled * Add OIDC check for ipv6 cluster * Add version check if ipv6 is defined * Setting vpc.NAT is not allowed with ipv6 * serviceIPv4CIDR is not supported with ipv6 * AutoAllocateIPv6 is not supported together with ipv6 * Unmanaged nodegroups are not supported (#4316) * Check if managed addons are set when ipv6 is enabled * Add OIDC check for ipv6 cluster * Add version check if ipv6 is defined * Setting vpc.NAT is not allowed with ipv6 * serviceIPv4CIDR is not supported with ipv6 * AutoAllocateIPv6 is not supported together with ipv6 * Unmanaged nodegroups are not supported * Added a check to the nodegroup task as well * Nodegroup creation is not supported with unowned ipv6 clusters (#4322) * Check if managed addons are set when ipv6 is enabled * Add OIDC check for ipv6 cluster * Add version check if ipv6 is defined * Setting vpc.NAT is not allowed with ipv6 * serviceIPv4CIDR is not supported with ipv6 * AutoAllocateIPv6 is not supported together with ipv6 * Unmanaged nodegroups are not supported * Nodegroup creation is not supported with unowned ipv6 clusters * First iteration of modifying the interface for ownership determination * Changed but kept the check in nodegroup builder * Shifted the logic into the task building and removed the extra parameter from the builder * Update pkg/cfn/manager/tasks_test.go Co-authored-by: Jake Klein <jakelarsj@gmail.com> Co-authored-by: Jake Klein <jakelarsj@gmail.com> * use aws-sdk-go with ipv6 fields (#4368) * Implement default workflow to add IPv6 VPC resources - Add integration test for ipv6 - Renamed VPC files to IPv4 file, and added IPv6 VPC files. - Created new resource set for IPv6 - Added all of the resources to create IPv6 VPC to CF template - Resources being created: VPC, IPv4 CIDR, IPv6 CIDR, EOIGW, NAT GW, IGW, route tables, routes, private and public subnets, route table associations - Added outputs to CF template for VPC and public/private subnets - Added integration and unit tests Co-authored-by: Jake Klein <aclevername@users.noreply.github.com> Refactoring VPC template creation - Making addResources and addOutputs private - Creating new CreateTemplate function for both IPv4 and IPv6 * wait for addons to be healthy only if there are nodegroups * Passing IpFamily field to Control Plane CF template * Adding integration test set AssignIpv6AddressOnCreation to true for public subnets after the rest of the template. * Commenting ipFamily test and code due to CF bug - Revert me! * Adding AssignIpv6AddressOnCreation task after cluster creation due to CF bug - AssignIpv6AddressOnCreation also needs to be set on public subnets, but due to a current bug in CF, this cannot be set alongside MapPublicIpOnLaunch at create time. This means we need to add it "manually" by hitting the VPC API to update each public subnet after launch. - Added extra validation that NAT is nil * Revert "Commenting ipFamily test and code due to CF bug" This reverts commit dc2c50a. * Refactoring integration test to wait for Service to eventually exist * Updating goformation with ipFamily changes - Also pass unit tests and remove commented code * Implement review feedback Update integration/tests/ipv6/ipv6_test.go Co-authored-by: Chetan Patwal <cPu1@users.noreply.github.com> set example ipv6 region to us-west-2 * refactor nodegroup and cluster related functions out ov vpc_ipv4.go (#4376) * Use string instead of string pointer for IPFamily (#4379) * add support for setting extraIPv6CIDRs (#4412) Co-authored-by: Niki <18622989+nikimanoledaki@users.noreply.github.com> * User can create a new VPC with a custom IPv6 CIDR (#4378) * refactor out cidr block checker and re-use across ipv4/6 * add support for configuring vpc.ipv6cidr and vpc.ipv6cidrpool * remove unit test flake - maps aren't ordered * add validation logic * add test to check custom ipv4 cidr works * rename ipv6cidrpool to ipv6pool and add docs * Update pkg/apis/eksctl.io/v1alpha5/validation.go Co-authored-by: Niki <18622989+nikimanoledaki@users.noreply.github.com> * implent pr feedback Co-authored-by: Niki <18622989+nikimanoledaki@users.noreply.github.com> * Private IPv6 VPC (#4415) * Private IPv6 VPC * Fixed private network topography to actually work. * Fix the refactored code segment * Add support for using existing VPC for ipv6 cluster (#4398) Add integration test as well for general existing-vpc support, not ipv6 specific. * ensure vpc-cni version is atleast 1.10.0 when ipv6 is configured * Update pkg/apis/eksctl.io/v1alpha5/validation.go Co-authored-by: Niki <18622989+nikimanoledaki@users.noreply.github.com> * TODO: Revert this when 1.10 is default. default to 1.10.0 when no addon version is provided * update docs and examples * fix flakey test * Fix types for Ipv6CidrBlock and Ipv6Pool * Update goformation * set correct policies when creating vpc-cni for ipv6 cluster * update ipv6 integration check to create & test managed nodegroup * Ipv6 int test uses latest addon versions * create dns64 and nat64 resources for ipv6 clusters * Enable DNS64 only for private subnets * Update 29-vpc-with-ip-family.yaml * Merge `main` into `ipv6` (#4557) * pend failing test (#4293) * Parse task log to be more human readable (#4290) * Parse task log to be more human readable * Added new line breaks * pend failing test (#4320) * Add release notes for 0.70 (#4324) * wait for addons to be healthy only if there are nodegroups (#4332) * Prepare for next development iteration (#4326) * refactor: move from io/ioutil to io and os package (#4333) The io/ioutil package has been deprecated as of Go 1.16, see https://golang.org/doc/go1.16#ioutil. This commit replaces the existing io/ioutil functions with their new definitions in io and os packages. * Add flag to delete VPC resource controller * Enable Windows IPAM when creating a Windows nodegroup * Handle plan mode, deprecate command * Remove deprecated flag from integration test * Update docs and example * Add tests for IPAM, improve error messages * Revert "Enable Windows IPAM when creating a Windows nodegroup, deprecate `install-vpc-controllers`" (#4340) * Remove redundant func * Move package * Validate support for GPU instances * Add test for GPU instance support * Fix integration test * Bump go releaser (#4348) * bump goreleaser to 0.182.1 * bump build image * Bump mkdocs from 1.1 to 1.2.3 in /userdocs (#4349) Bumps [mkdocs](https://github.com/mkdocs/mkdocs) from 1.1 to 1.2.3. - [Release notes](https://github.com/mkdocs/mkdocs/releases) - [Commits](mkdocs/mkdocs@1.1...1.2.3) --- updated-dependencies: - dependency-name: mkdocs dependency-type: direct:production ... * Update aws-node * Add iamidentitymapping delete and get commands (#4346) * Add accounts to delete and get * Simplified and unified the display by extending the interface in an ugly way * Update pkg/authconfigmap/authconfigmap.go * Addressed comments * Fix integration test timeout (#4361) * Add release notes for 0.71 (#4362) * Adding release notes for 0.71.0 * Update docs/release_notes/0.71.0.md * Add breaking change * Prepare for next development iteration (#4365) * Use go:embed instead of go-bindata (#4370) * use go embed in authconfigmap * use go:embed in nodebootstrap * use go:embed in addons * use go:embed for schema * remove bindata from makefile & tools * update build image * fix linter/failing test * Add ARM support and fix AMI resolution for Ubuntu (#4367) * Add bootstrapper for native Bottlerocket * Pass new AMI type to MNG API * Add test for native Bottlerocket support * Add integration tests for native Bottlerocket support * Add test for AMI types * Improve integration tests for Bottlerocket * Update AWS SDK, generate mocks * Add release notes for 0.72 * Prepare for next development iteration * Clean up generation of aws-node manifest * update aws-node (#4391) * reduce API calls in upgrade cluster (#4366) * reduce API calls in upgrade cluster * Update pkg/eks/eks.go * clarify up-to-date nodegroup message (#4393) * make 1.21 default (#4394) * Add support for Windows Server 20H2 * Update schema * Update doc * Bump vpc-resource-controller/webhook dep to v0.2.7 (#4335) * Scale managed ng with --name flag * remove aws-cleanup and integration workflows (#4247) * remove aws-cleanup and integration workflows * remove integration check from release candidate workflow * remove unused slack action2 * Support CloudWatch log retention * Add integration test for CloudWatch log retention * Add test for validation * Generate mocks * Move package * Improve integration test * Improve validation tests * Add example for setting logRetentionInDays * Fix Windows integration test * fix panic in cloudwatch validation (#4409) * bump gjson (#4410) * Fix wait for managed nodegroups #3889 (#4401) * Fix wait for managed nodegroups #3889 * Refactored the long parameter list * Unpin and fix containerd integration test (#4411) * Update aws-node (#4408) * pend failing windows test (#4413) * Update to kops v1.21.2 (#4380) * Add notes for 0.73.0 * Prepare for next development iteration * Fix validation for GPU instance types when AMIFamily is not set (#4403) * Use NotTo instead of ToNot in tests (#4425) * Avoid writing temp ClusterConfig file (#4397) This changelist passes the ClusterConfig file via stdin using an io.Reader, instead of writing a temp file to disk and cleaning it up later. * Bump GoReleaser to 0.184.0 (#4422) * bump goreleaser to 0.184.0 * bump goreleaser to 0.184.0 * bump goreleaser to 0.184.0 * bump goreleaser to 0.184.0 * update build image (#4435) * Set volume size in data volume for Bottlerocket The volume size was being set for the OS volume, instead of the data volume for Bottlerocket nodegroups. * Add test for standard Bottlerocket nodegroup * Fix unit tests for ami.Use * Unpend and fix Windows integration tests The workload being deployed is not compatible with WindowsServer20H2CoreContainer and that was preventing the pod from starting. * Add support for Windows IPAM * Add release notes for 0.74 * Prepare for next development iteration * Document logs:PutRetentionPolicy in IAM policies doc * Fix update logic in iam service account (#4460) * Add option to skip endpoint creation completely (#4457) * Add config-file option to get nodegroup (#4465) * set default csi driver policies when creating the addon * Improve kube-proxy updating to always get the latest version&build #4344 (#4440) query EKS API to discover latest kube-proxy version * refactor set/get recommended policies * Add --config-flag to get labels (#4469) * Bump goreleaser * Update build image tag * Fixed the config naming check error (#4470) * remove flux1 & profiles support * update docs * add 0.75 release notes (#4472) * add 0.75 release notes * Update docs/release_notes/0.75.0.md * Update docs/release_notes/0.75.0.md * Update docs/release_notes/0.75.0.md * Prepare for next development iteration * Support scaling all nodegroups in config file * Rename consts and nodegroups for consistency * Also implement review feedback * Add unit tests for scaling ng * Update cluster-upgrade.md Add missing '=' * Add AttachPolicy to NodeGroupIAM configuration (#4331) * Add AttachPolicy to NodeGroupIAM configuration * Add NodeGroup AttachPolicy tests * fix * Add nodegroup IAM attachPolicy field to userdocs * Add assertion for attachPolicy policy document * Compare JSON encoded policy due to interface differences * Add config-file option to get cluster (#4468) * removed support for eks 1.17 * update docs for eks supported version * Fix panic in node affinity adder (#4491) * Add release notes for 0.76.0 * Prepare for next development iteration * Add integration test for `utils schema` (#4497) * Add integration test for `describe-stacks` (#4492) * Add integration test for describe-stacks * Update integration/tests/crud/creategetdelete_test.go * Deprecate golint and apply some changes from revive (#4489) * Separate VPC Networking into meaningful sections (#4490) * Separate VPC Networking into meaningful sections * Apply suggestions from code review * Renamed the file * Add `--config-file` to `set labels` (#4502) * Add config-file to set-labels * Updated to only update new labels, never to delete or update them * Added integration test * Proper schema. * Updated the tests and updated the wait time with a comment on why * pend failing integration test * Do not use GetLabels all the time (#4520) * added missing --region flags * New release 0.77.0 (#4535) * New release 0.77.0 * Update docs/release_notes/0.77.0.md * Prepare for next development iteration (#4537) * Add integration test for `associate identityprovider` * Add integration test for `get identityprovider` * Add integration test for `disassociate identityprovider` * Rename import alias and function * Increase timeout * Split Cognito setup func * Fix ignoring partitions on EBS controller (#4547) * Increase delete wait time (#4548) * Fix linter complaining about pkg name * Update dry-run integ test with IP family field * Move `ipFamily` config file field to `KubernetesNetworkConfig` (#4566) * Move ipFamily config file field to KubernetesNetworkConfig * Update pkg/apis/eksctl.io/v1alpha5/validation.go Co-authored-by: Chetan Patwal <cPu1@users.noreply.github.com> Co-authored-by: Chetan Patwal <cPu1@users.noreply.github.com> * Merge main into ipv6 (#4567) * pend failing test (#4293) * Parse task log to be more human readable (#4290) * Parse task log to be more human readable * Added new line breaks * pend failing test (#4320) * Add release notes for 0.70 (#4324) * wait for addons to be healthy only if there are nodegroups (#4332) * Prepare for next development iteration (#4326) * refactor: move from io/ioutil to io and os package (#4333) The io/ioutil package has been deprecated as of Go 1.16, see https://golang.org/doc/go1.16#ioutil. This commit replaces the existing io/ioutil functions with their new definitions in io and os packages. * Add flag to delete VPC resource controller * Enable Windows IPAM when creating a Windows nodegroup * Handle plan mode, deprecate command * Remove deprecated flag from integration test * Update docs and example * Add tests for IPAM, improve error messages * Revert "Enable Windows IPAM when creating a Windows nodegroup, deprecate `install-vpc-controllers`" (#4340) * Remove redundant func * Move package * Validate support for GPU instances * Add test for GPU instance support * Fix integration test * Bump go releaser (#4348) * bump goreleaser to 0.182.1 * bump build image * Bump mkdocs from 1.1 to 1.2.3 in /userdocs (#4349) Bumps [mkdocs](https://github.com/mkdocs/mkdocs) from 1.1 to 1.2.3. - [Release notes](https://github.com/mkdocs/mkdocs/releases) - [Commits](mkdocs/mkdocs@1.1...1.2.3) --- updated-dependencies: - dependency-name: mkdocs dependency-type: direct:production ... * Update aws-node * Add iamidentitymapping delete and get commands (#4346) * Add accounts to delete and get * Simplified and unified the display by extending the interface in an ugly way * Update pkg/authconfigmap/authconfigmap.go * Addressed comments * Fix integration test timeout (#4361) * Add release notes for 0.71 (#4362) * Adding release notes for 0.71.0 * Update docs/release_notes/0.71.0.md * Add breaking change * Prepare for next development iteration (#4365) * Use go:embed instead of go-bindata (#4370) * use go embed in authconfigmap * use go:embed in nodebootstrap * use go:embed in addons * use go:embed for schema * remove bindata from makefile & tools * update build image * fix linter/failing test * Add ARM support and fix AMI resolution for Ubuntu (#4367) * Add bootstrapper for native Bottlerocket * Pass new AMI type to MNG API * Add test for native Bottlerocket support * Add integration tests for native Bottlerocket support * Add test for AMI types * Improve integration tests for Bottlerocket * Update AWS SDK, generate mocks * Add release notes for 0.72 * Prepare for next development iteration * Clean up generation of aws-node manifest * update aws-node (#4391) * reduce API calls in upgrade cluster (#4366) * reduce API calls in upgrade cluster * Update pkg/eks/eks.go * clarify up-to-date nodegroup message (#4393) * make 1.21 default (#4394) * Add support for Windows Server 20H2 * Update schema * Update doc * Bump vpc-resource-controller/webhook dep to v0.2.7 (#4335) * Scale managed ng with --name flag * remove aws-cleanup and integration workflows (#4247) * remove aws-cleanup and integration workflows * remove integration check from release candidate workflow * remove unused slack action2 * Support CloudWatch log retention * Add integration test for CloudWatch log retention * Add test for validation * Generate mocks * Move package * Improve integration test * Improve validation tests * Add example for setting logRetentionInDays * Fix Windows integration test * fix panic in cloudwatch validation (#4409) * bump gjson (#4410) * Fix wait for managed nodegroups #3889 (#4401) * Fix wait for managed nodegroups #3889 * Refactored the long parameter list * Unpin and fix containerd integration test (#4411) * Update aws-node (#4408) * pend failing windows test (#4413) * Update to kops v1.21.2 (#4380) * Add notes for 0.73.0 * Prepare for next development iteration * Fix validation for GPU instance types when AMIFamily is not set (#4403) * Use NotTo instead of ToNot in tests (#4425) * Avoid writing temp ClusterConfig file (#4397) This changelist passes the ClusterConfig file via stdin using an io.Reader, instead of writing a temp file to disk and cleaning it up later. * Bump GoReleaser to 0.184.0 (#4422) * bump goreleaser to 0.184.0 * bump goreleaser to 0.184.0 * bump goreleaser to 0.184.0 * bump goreleaser to 0.184.0 * update build image (#4435) * Set volume size in data volume for Bottlerocket The volume size was being set for the OS volume, instead of the data volume for Bottlerocket nodegroups. * Add test for standard Bottlerocket nodegroup * Fix unit tests for ami.Use * Unpend and fix Windows integration tests The workload being deployed is not compatible with WindowsServer20H2CoreContainer and that was preventing the pod from starting. * Add support for Windows IPAM * Add release notes for 0.74 * Prepare for next development iteration * Document logs:PutRetentionPolicy in IAM policies doc * Fix update logic in iam service account (#4460) * Add option to skip endpoint creation completely (#4457) * Add config-file option to get nodegroup (#4465) * set default csi driver policies when creating the addon * Improve kube-proxy updating to always get the latest version&build #4344 (#4440) query EKS API to discover latest kube-proxy version * refactor set/get recommended policies * Add --config-flag to get labels (#4469) * Bump goreleaser * Update build image tag * Fixed the config naming check error (#4470) * remove flux1 & profiles support * update docs * add 0.75 release notes (#4472) * add 0.75 release notes * Update docs/release_notes/0.75.0.md * Update docs/release_notes/0.75.0.md * Update docs/release_notes/0.75.0.md * Prepare for next development iteration * Support scaling all nodegroups in config file * Rename consts and nodegroups for consistency * Also implement review feedback * Add unit tests for scaling ng * Update cluster-upgrade.md Add missing '=' * Add AttachPolicy to NodeGroupIAM configuration (#4331) * Add AttachPolicy to NodeGroupIAM configuration * Add NodeGroup AttachPolicy tests * fix * Add nodegroup IAM attachPolicy field to userdocs * Add assertion for attachPolicy policy document * Compare JSON encoded policy due to interface differences * Add config-file option to get cluster (#4468) * removed support for eks 1.17 * update docs for eks supported version * Fix panic in node affinity adder (#4491) * Add release notes for 0.76.0 * Prepare for next development iteration * Add integration test for `utils schema` (#4497) * Add integration test for `describe-stacks` (#4492) * Add integration test for describe-stacks * Update integration/tests/crud/creategetdelete_test.go * Deprecate golint and apply some changes from revive (#4489) * Separate VPC Networking into meaningful sections (#4490) * Separate VPC Networking into meaningful sections * Apply suggestions from code review Co-authored-by: Himangini <its_himangini@yahoo.com> * Renamed the file Co-authored-by: Himangini <its_himangini@yahoo.com> * Add `--config-file` to `set labels` (#4502) * Add config-file to set-labels * Updated to only update new labels, never to delete or update them * Added integration test * Proper schema. * Updated the tests and updated the wait time with a comment on why * pend failing integration test * Do not use GetLabels all the time (#4520) * added missing --region flags * New release 0.77.0 (#4535) * New release 0.77.0 * Update docs/release_notes/0.77.0.md Co-authored-by: Niki <niki.manoledaki@gmail.com> Co-authored-by: Niki <niki.manoledaki@gmail.com> * Prepare for next development iteration (#4537) * Add integration test for `associate identityprovider` * Add integration test for `get identityprovider` * Add integration test for `disassociate identityprovider` * Rename import alias and function * Increase timeout * Split Cognito setup func * Fix ignoring partitions on EBS controller (#4547) * Increase delete wait time (#4548) * Generate EC2 mocks * Add release notes for 0.78.0 * Prepare for next development iteration * Add g5 instance to list of GPU instance types (#4550) * Add g5 instance to list of GPU instance types * Add test cases for g5 instance * Check InstanceRoleARN exists before deleting authconfigmap reference for nodegroup (#4558) Co-authored-by: Chetan Patwal <cPu1@users.noreply.github.com> * Fix outdated link in docs * Add extra validation for kubernetesNetworkConfig to avoid panic Co-authored-by: Gergely Brautigam <182850+Skarlso@users.noreply.github.com> Co-authored-by: Jake Klein <jakelarsj@gmail.com> Co-authored-by: Jake Klein <jake@weave.works> Co-authored-by: cpu1 <patwal.chetan@gmail.com> Co-authored-by: Chetan Patwal <cPu1@users.noreply.github.com>

aclevername added the kind/feature New feature or request label Nov 2, 2021

aclevername changed the base branch from main to ipv6 November 2, 2021 14:33

aclevername marked this pull request as ready for review November 3, 2021 13:58

aclevername requested review from nikimanoledaki, Skarlso, cPu1 and Himangini November 3, 2021 13:58

aclevername changed the title ~~[wip] use existing VPC for ipv6 cluster~~ Add support for using existing VPC for ipv6 cluster Nov 3, 2021

Skarlso reviewed Nov 8, 2021

View reviewed changes

pkg/cfn/builder/vpc_existing.go Outdated Show resolved Hide resolved

Skarlso reviewed Nov 8, 2021

View reviewed changes

pkg/cfn/builder/vpc_existing.go Outdated Show resolved Hide resolved

Skarlso approved these changes Nov 8, 2021

View reviewed changes

nikimanoledaki reviewed Nov 8, 2021

View reviewed changes

pkg/cfn/builder/vpc_existing.go Outdated Show resolved Hide resolved

Jake added 5 commits November 9, 2021 09:56

add support for using existing VPC for ipv6

7252bc0

refactor tests

cd23e93

check ipv6 is enabled

810b9a5

fix failing unit test

3276a0c

use unsized array

d81debc

aclevername force-pushed the ipv6-existing branch from 8e20d6e to 7fe34a0 Compare November 9, 2021 09:59

validation ipv6Cidr isn't passed along with an existing VPC.ID

7fe34a0

aclevername requested a review from nikimanoledaki November 9, 2021 10:47

check VPC exists for both ipv4 and ipv6

193454d

nikimanoledaki previously requested changes Nov 10, 2021

View reviewed changes

cPu1 reviewed Nov 12, 2021

View reviewed changes

aclevername and others added 3 commits November 16, 2021 11:04

Update pkg/apis/eksctl.io/v1alpha5/validation_test.go

5ea6760

Co-authored-by: Chetan Patwal <cPu1@users.noreply.github.com>

Update pkg/cfn/builder/vpc_existing_test.go

10b9b7b

Co-authored-by: Chetan Patwal <cPu1@users.noreply.github.com>

Update pkg/cfn/builder/vpc_existing_test.go

58e0d6c

Co-authored-by: Chetan Patwal <cPu1@users.noreply.github.com>

aclevername and others added 2 commits November 16, 2021 11:11

Update pkg/cfn/builder/vpc_existing_test.go

eba1076

Co-authored-by: Chetan Patwal <cPu1@users.noreply.github.com>

refactor error msg

6b59c4c

aclevername force-pushed the ipv6-existing branch from de1dee5 to f30c5b3 Compare November 16, 2021 14:58

Jake added 2 commits November 16, 2021 14:58

add integration test

f30c5b3

add outputs to cluster stack when using existing vpc

6766559

aclevername requested review from cPu1 and nikimanoledaki November 17, 2021 10:08

cleanup cluster after test

3ce67f0

add unit test for outputs on cluster stack for existing vpc

5cc12f7

nikimanoledaki reviewed Nov 17, 2021

View reviewed changes

integration/tests/existing_vpc/existing_vpc_test.go Outdated Show resolved Hide resolved

integration/tests/existing_vpc/existing_vpc_test.go Show resolved Hide resolved

integration/tests/existing_vpc/existing_vpc_test.go Outdated Show resolved Hide resolved

aclevername requested a review from nikimanoledaki November 17, 2021 10:58

update integration test

866849d

nikimanoledaki approved these changes Nov 18, 2021

View reviewed changes

aclevername merged commit 5aafeaf into eksctl-io:ipv6 Nov 18, 2021

nikimanoledaki pushed a commit that referenced this pull request Jan 5, 2022

Add support for using existing VPC for ipv6 cluster (#4398)

ca5eddf

Add integration test as well for general existing-vpc support, not ipv6 specific.

jimmyraywv mentioned this pull request Jan 13, 2022

Use existing dual-stack VPC for IPv6 clusters #4628

Closed

	Expect(err).To(MatchError("cannot provide VPC.IPv6Cidr when using a pre-existing VPC.ID"))
	Expect(err).To(HaveOccurred())
	Expect(err).To(MatchError("cannot provide VPC.IPv6Cidr when using a pre-existing VPC.ID"))

Add support for using existing VPC for ipv6 cluster #4398

Add support for using existing VPC for ipv6 cluster #4398

Uh oh!

Conversation

aclevername commented Nov 2, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Checklist

BONUS POINTS checklist: complete for good vibes and maybe prizes?! 🤯

Uh oh!

Uh oh!

Uh oh!

Skarlso commented Nov 8, 2021

Uh oh!

aclevername commented Nov 8, 2021

Uh oh!

nikimanoledaki left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

nikimanoledaki commented Nov 9, 2021

Uh oh!

aclevername commented Nov 9, 2021

Uh oh!

nikimanoledaki left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

cPu1 Nov 12, 2021

Choose a reason for hiding this comment

Uh oh!

aclevername Nov 16, 2021

Choose a reason for hiding this comment

Uh oh!

aclevername Nov 16, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

cPu1 Nov 17, 2021

Choose a reason for hiding this comment

Uh oh!

aclevername Nov 17, 2021

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

aclevername commented Nov 17, 2021

Uh oh!

nikimanoledaki left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

nikimanoledaki left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

aclevername commented Nov 2, 2021 •

edited

Loading

nikimanoledaki left a comment •

edited

Loading

aclevername Nov 16, 2021 •

edited

Loading