Skip to content

Commit

Permalink
Address review comments
Browse files Browse the repository at this point in the history
  • Loading branch information
flostadler committed Sep 27, 2024
1 parent 18e9c91 commit 9bde177
Showing 1 changed file with 13 additions and 17 deletions.
30 changes: 13 additions & 17 deletions docs/eks-v3-migration.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# EKS v3 Migration Guide

AWS recently announced the deprecation of two features used by default in Pulumi EKS: the aws-auth ConfigMap and the AL2 operating system. EKS v3 tries to address the recent deprecations in EKS by AWS, improve maintainability of the provider and align it with EKS best practices.
AWS recently announced the deprecation of two features used by default in Pulumi EKS: the aws-auth ConfigMap and the AL2 operating system. Pulumi EKS v3 addresses these deprecations, enhances the maintainability of the provider, and aligns it with EKS best practices.

## New Features

Expand All @@ -12,7 +12,7 @@ AWS recently announced the deprecation of two features used by default in Pulumi
- **Default Instance Type Update: t3.medium**
The default instance type for node groups has been updated from `t2.medium` to `t3.medium`. The `t3.medium` instance type provides several advantages:

- **Higher Availability**: `t2.medium` is an older instance type that is becoming less common in AWS regions, which can lead to capacity issues during scaling. The newer `t3.medium` instance type offers better availability.
- **Higher Availability**: `t2.medium` is an older instance type that is being phased out in AWS regions, which can cause capacity issues during scaling. The newer `t3.medium` instance type offers better availability.
- **Improved Cost Efficiency**: `t3.medium` is marginally cheaper than `t2.medium` ($0.0416 vs $0.0464 per hour), providing better performance at a lower cost.


Expand All @@ -27,7 +27,7 @@ AWS recently announced the deprecation of two features used by default in Pulumi
This also simplifies management of clusters with private API endpoints because those components are managed via AWS APIs.

- **Support for Cluster Autoscaler and External Scaling**
Pulumi-EKS v3 introduces support for integrating with the Kubernetes cluster-autoscaler, which allows your cluster to scale dynamically based on resource needs. A new `ignoreScalingChanges` parameter has been added for node groups. This parameter allows Pulumi to ignore external scaling changes (e.g., changes to the desired size made by the cluster-autoscaler).
Pulumi EKS v3 introduces support for integrating with the Kubernetes cluster-autoscaler, which allows your cluster to scale dynamically based on resource needs. A new `ignoreScalingChanges` parameter has been added for node groups. This parameter allows Pulumi to ignore external scaling changes (e.g., changes to the desired size made by the cluster-autoscaler).

- **Cluster Version Tracking in `ManagedNodeGroup`**
`ManagedNodeGroup` now automatically tracks the EKS cluster version, ensuring smoother version upgrades and preventing version mismatches. Since EKS only allows one minor version skew between the cluster control plane and the nodes, this automatic tracking helps prevent upgrade failures due to version incompatibility.
Expand All @@ -48,13 +48,13 @@ The VPC CNI cluster component is now configured as an EKS addon as mentioned in
The `NodeGroup` component uses the deprecated AWS Launch Configuration (see [AWS docs](https://docs.aws.amazon.com/autoscaling/ec2/userguide/launch-configurations.html)). Launch Configurations do not support new instance types released after December 31, 2022 and starting on October 1, 2024, new AWS accounts will not be able to create launch configurations.
Its successor, the `NodeGroupV2` component is functionally equivalent and easier to operate because it does not use CloudFormation templates under the hood like `NodeGroup` did.

The default node group of the `Cluster` component was updated to use the `NodeGroupV2` component as well.
The default node group of the `Cluster` component has been updated to use the `NodeGroupV2` component as well.
Updates to the default node group will be done by first creating the new replacement nodes and then shutting down the old ones which will move pods to the new nodes. If you need to perform the update gracefully, please have a look at [Gracefully upgrading node groups](#gracefully-upgrading-node-groups).

### Default OS of node groups updated to AL2023

AWS recently deprecated the Amazon Linux 2 (AL2) operating system. It will reach end of life in June 2025, after which it will receive no more security and maintenance updates.
Up until now this was the OS of node groups created with the pulumi-eks provider.
Until now, this was the OS used by node groups created with the Pulumi EKS provider.
To align the provider with EKS best practices we’ve updated the default operating system to the new AWS-recommended option, Amazon Linux 2023 (AL2023).

You can either upgrade the OS right away, or intentionally configure AL2 as the desired operating system by using the `operatingSystem` parameter that’s available on all node groups.
Expand All @@ -63,23 +63,22 @@ Have a look at [Gracefully upgrading node groups](#gracefully-upgrading-node-gro

### Gracefully upgrading node groups

The `ManagedNodeGroup` component gracefully handles updates by default. For them EKS will:

The `ManagedNodeGroup` component gracefully handles updates by default. EKS will:
- boot the updated replacement nodes
- cordon the old nodes to ensure no new pods get launched onto them
- one-by-one drain the old nodes
- drain the old nodes one-by-one
- shut down the empty old nodes

The detailed update procedure can be seen in the [AWS docs](https://docs.aws.amazon.com/eks/latest/userguide/managed-node-update-behavior.html).

For self-managed node groups (i.e. the `NodeGroup` and `NodeGroupV2` components) you have two options:
For self-managed node groups (i.e., the `NodeGroup` and `NodeGroupV2` components) you have two options:

1. Update the node group in place. Pulumi does this by first creating the new replacement nodes and then shutting down the old ones which will move pods to the new nodes forcibly. This is the default behavior when node groups are updated.
2. Create a new node group and move your Pods to that group. Migrating to a new node group is more graceful than simply updating the node group in place. This is because the migration process taints the old node group as `NoSchedule` and drains the nodes gradually.

The second option involves the following steps:

1. Create the replacement node group side-by-side with the existing node group. When doing this you need to make sure that the two node groups are allowed to communicate with each other. You can achieve this the following way:
1. Create the replacement node group side-by-side with the existing node group. When doing this you need to make sure that the two node groups are allowed to communicate with each other. You can achieve this in the following way:

```ts
const oldNG = new eks.NodeGroupV2("old", {
Expand Down Expand Up @@ -134,19 +133,16 @@ Recently the aws-auth ConfigMap has been deprecated in favor of Access Entries.
You can start using Access Entries with your cluster by changing the `authenticationMode` to `API`.
A step-by-step migration guide can be found [here](https://github.com/pulumi/pulumi-eks/blob/master/docs/authentication-mode-migration.md).

At this time we recommend users to create all new clusters with the authentication mode `API.`
We currently recommend users create all new clusters with the `API` authentication mode.
More details about this can be found [here](https://docs.aws.amazon.com/eks/latest/userguide/grant-k8s-access.html).

## Nodejs SDK breaking changes

Up until this point the provider exposed a hand-written nodejs SDK instead of using the multi-language-component approach that was used for the other SDKs like Python. This caused inaccurate documentation as well as an increased maintenance effort.

As part of this release we’re aligning the nodejs SDK with the rest of the SDKs and leverage the multi-language-component to auto-generate it.
We’ve worked hard to minimize breaking changes this causes for nodejs users. These are the changes users see in their programs:
The Node SDK is updated to use state of the art Pulumi tooling, improving stability, documentation and security. The update requires the following changes to programs:

- Properties of the components are now outputs instead of plain types. Notable cases were you need to change your program are:
- Properties of the components are now outputs instead of plain types. Notable cases where you need to change your program are:
- The [`Cluster::getKubeConfig`](https://www.pulumi.com/registry/packages/eks/api-docs/cluster/#method_GetKubeconfig) method now returns an output.
- Creating an IRSA based IAM role now requires you to use `apply` for accessing the OIDC provider ARN and URL. An example of how this works can be found [here](https://github.com/pulumi/pulumi-eks/blob/release-3.x.x/examples/oidc-iam-sa/index.ts).
- `k8s.Provider` will be deleted if you are not referencing it (no impact, just will show up in the diff)
- `k8s.Provider` will be deleted if not referenced (no impact, but it will appear in the diff)
- `clusterOidcProvider` is an output now. `getKubeConfig` returns an output now
- The deprecated input property `deployDashboard` of the `Cluster` component has been removed from the nodejs SDK. This has already been removed from the other SDKs in the past. If you’d like to continue using it, you can adopt the existing code into your own program from [here](https://github.com/pulumi/pulumi-eks/blob/bcc170e72b802a78e7f0a99bc92316a5f8a62b0e/nodejs/eks/dashboard.ts).

0 comments on commit 9bde177

Please sign in to comment.