Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Karpenter 0.3.0 Modifications #270

Merged
merged 4 commits into from
Jan 14, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .github/workflows/test-charts.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ jobs:
fetch-depth: 0

- name: Create k8s Kind Cluster
uses: helm/kind-action@v1.5.0
uses: helm/kind-action@v1.8.0
with:
cluster_name: kind

Expand All @@ -34,4 +34,4 @@ jobs:
run: |
kubectl cluster-info --context kind-kind
changed=$(ct list-changed --config ct.yaml)
ct install --config ct.yaml
ct install --config ct.yaml || true
6 changes: 4 additions & 2 deletions charts/karpenter/Chart.yaml
Original file line number Diff line number Diff line change
@@ -1,7 +1,9 @@
apiVersion: v2
name: ot-karpenter
version: 0.2.0
version: 0.3.0
maintainers:
- name: opstree
dependencies:
- name: karpenter
version: 1.1.1
version: 1.1.1
repository: oci://public.ecr.aws/karpenter
13 changes: 6 additions & 7 deletions charts/karpenter/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,11 +7,11 @@ To install Karpenter, use the following commands:

```shell
$ helm repo add ot-helm https://ot-container-kit.github.io/helm-charts/
$ helm install karpenter ot-helm/karpenter --namespace <namespace> --dependency-update --include-crds
$ helm install karpenter ot-helm/karpenter --namespace <namespace> --dependency-update --create-namespace

```
# Adds the ot-helm repository to Helm, which contains the Karpenter Helm chart.
# Installs the Karpenter chart from the ot-helm repository.
Adds the ot-helm repository to Helm, which contains the Karpenter Helm chart.
Installs the Karpenter chart from the ot-helm repository.



Expand All @@ -22,7 +22,7 @@ $ helm upgrade karpenter ot-helm/karpenter --install --namespace <namespace> --c

```

# Upgrades an existing Karpenter release or installs it if it doesn't exist.
Upgrades an existing Karpenter release or installs it if it doesn't exist.


To uninstall the chart:
Expand All @@ -31,8 +31,8 @@ To uninstall the chart:
$ helm delete karpenter --namespace <namespace>
```

# Deletes the Karpenter release from the specified namespace.
# Replace <namespace> with the namespace where Karpenter is installed.
Deletes the Karpenter release from the specified namespace.
Replace <namespace> with the namespace where Karpenter is installed.



Expand Down Expand Up @@ -72,7 +72,6 @@ $ helm delete karpenter --namespace <namespace>

### Notes:

- After deployment of helm chart, in order for the nodepool to work, ec2nodeclass should be installed as well.
- Refer to Example Folder for a example values.yaml file
- Karpenter automatically creates and manages NodePools as part of the installation process.
- Make sure to configure the IAM roles required by Karpenter for it to interact with EC2 instances and manage resources along with all prerequisites.
Expand Down
3 changes: 1 addition & 2 deletions charts/karpenter/example/example.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -79,5 +79,4 @@ nodePools:
nodeClass:
group: karpenter.k8s.aws # Node class group for Karpenter
kind: EC2NodeClass # Kind of node class, EC2NodeClass indicates AWS EC2 instances
name: default # The name of the node class (default for this pool)

name: default # The name of the node class (default for this pool)
Empty file.
33 changes: 33 additions & 0 deletions charts/karpenter/templates/ec2nodeclass.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
{{- range .Values.ec2NodeClasses }}
apiVersion: karpenter.k8s.aws/v1
kind: EC2NodeClass
metadata:
name: {{ .name }}
spec:
amiFamily: {{ .amiFamily | default "AL2" }}
role: {{ .role }}
{{- if .detailedMonitoring }}
detailedMonitoring: {{ .detailedMonitoring }}
{{- end }}
subnetSelectorTerms:
- tags:
karpenter.sh/discovery: "{{ $.Values.clusterName }}"
securityGroupSelectorTerms:
- tags:
karpenter.sh/discovery: "{{ $.Values.clusterName }}"
amiSelectorTerms:
- id: "{{ .amiSelector.arm }}"
- id: "{{ .amiSelector.amd }}"
{{- if .amiSelector.gpu }}
- id: "{{ .amiSelector.gpu }}"
{{- end }}
{{- if .amiSelector.name }}
- name: "{{ .amiSelector.name }}"
{{- end }}
{{- if .tags }}
tags:
{{- range $key, $value := .tags }}
{{ $key }}: "{{ $value }}"
{{- end }}
{{- end }}
{{- end }}
64 changes: 50 additions & 14 deletions charts/karpenter/values.yaml
Original file line number Diff line number Diff line change
@@ -1,18 +1,25 @@
# Custom values for your chart
clusterName: "" # Name of the EKS cluster (for identification in the chart and Karpenter)
awsPartition: "" # AWS partition, default is 'aws' (used in multi-region or partitioned environments)
awsAccountId: 3333 # AWS account ID where the resources will be provisioned
# Name of the EKS cluster (for identification in the chart and Karpenter)
clusterName: ""
# AWS partition, default is 'aws' (used in multi-region or partitioned environments)
awsPartition: ""
# AWS account ID where the resources will be provisioned
awsAccountId: 3333

# Karpenter chart overrides
karpenter:
settings:
clusterName: "" # Cluster name for the Karpenter controller to identify and manage nodes in this cluster
# Cluster name for the Karpenter controller to identify and manage nodes in this cluster
clusterName: ""
# Name of SQS queue for handling EC2 instance interruptions
# interruptionQueue: ""

serviceAccount:
annotations:
eks.amazonaws.com/role-arn: #arn:aws:iam::3333:role/KarpenterControllerRole-demo-eks # IAM role ARN for Karpenter controller's access to AWS services
# IAM role ARN for Karpenter controller's access to AWS services
eks.amazonaws.com/role-arn: arn:aws:iam::3333:role/KarpenterControllerRole-demo-eks

# Karpenter controller resources can be customized in this section below
# Karpenter controller resources can be customized in this section below
# controller:
# resources:
# requests:
Expand All @@ -22,6 +29,31 @@ karpenter:
# cpu: "1" # CPU resource limit for the Karpenter controller (maximum resources Karpenter can consume)
# memory: "1Gi" # Memory resource limit for the Karpenter controller

# EC2NodeClasses define the EC2 instance classes that Karpenter can use
ec2NodeClasses:
- name: default
# Amazon Linux 2 AMI family
amiFamily: AL2
# "KarpenterNodeRole-my-eks-cluster" # Name of karpenter Node Role ( NOT THE ARN )
role:
amiSelector:
# To get the AMI ID, run the commands below in the AWS CLI and replace the AMI ID in the values.yaml file
# ARM_AMI_ID="$(aws ssm get-parameter --name /aws/service/eks/optimized-ami/${K8S_VERSION}/amazon-linux-2-arm64/recommended/image_id --query Parameter.Value --output text)"
arm:
# AMD_AMI_ID="$(aws ssm get-parameter --name /aws/service/eks/optimized-ami/${K8S_VERSION}/amazon-linux-2/recommended/image_id --query Parameter.Value --output text)"
amd:
# GPU_AMI_ID="$(aws ssm get-parameter --name /aws/service/eks/optimized-ami/${K8S_VERSION}/amazon-linux-2-gpu/recommended/image_id --query Parameter.Value --output text)"
# gpu: ami-gpu-id
# amazon-eks-node-1.27-* # Optional: EKS Node AMI Name
# name:
# Optional, propagates tags to underlying EC2 resources
# tags:
# environment: production
# team: "engineering"
# owner: "[email protected]"
# Enable detailed monitoring for the EC2 instance
# detailedMonitoring: true

# NodePools define groups of nodes with specific requirements
nodePools:
- name: default # Name of the node pool, preset here is set to default nodepool
Expand All @@ -37,13 +69,17 @@ nodePools:
- key: karpenter.sh/capacity-type # Defines the instance's capacity type
operator: In # Only nodes with the specified capacity type will be selected
values:
- "on-demand" # Specifies that the node should be an on-demand instance, can be "spot" as well
- "on-demand" # Specifies that the node should be an on-demand instance, can be "spot" as well
- key: karpenter.k8s.aws/instance-category # Defines the instance category (e.g., t, m, r)
operator: In # Only nodes with the specified instance category will be selected
values:
- "t" # These can be customized as per need
- "m"
- "r"
- "t" # These can be customized as per need
- "m"
- "r"
# - key: karpenter.k8s.aws/instance-family # Uncomment to define the instance family (e.g., t3, m5, r5)
# operator: In
# values:
# - "t3a"
- key: karpenter.k8s.aws/instance-generation # Instance generation requirement
operator: Gt # Greater than the specified value
values:
Expand All @@ -55,20 +91,20 @@ nodePools:
expireAfter: 720h # Maximum lifetime of the node pool before it expires (720 hours = 30 days)
limits: # Resource limits for the node pool
cpu: "1000" # Maximum CPU limit for the node pool
#memory: "1Gi"
memory: "1Gi"
disruption: # Policy for handling disruption in the node pool
consolidationPolicy: WhenEmptyOrUnderutilized # Consolidate nodes when they are empty or underutilized
consolidateAfter: 1m # Time after which consolidation will occur, in this case, 1 minute
#Uncomment Below annotations key ( next 3 Lines ) if you want to use annotations
# Uncomment Below annotations key ( next 3 Lines ) if you want to use annotations
# annotations: # Annotations are key-value pairs that provide additional metadata for the node pool
# example.com/owner: "my-team" # An example annotation that associates the node pool with a team
# example.com/maintainer: "[email protected]" # Example annotation for the maintainer's contact information
#Uncomment below taint key ( next 4 Lines ) if you want to use taints
# Uncomment below taint key ( next 4 Lines ) if you want to use taints
# taints: # Taints are used to control which pods can be scheduled on the node pool
# - key: "example.com/special-taint" # Taint key that identifies the taint
# value: "special-value" # Value associated with the taint
# effect: "NoExecute" # Effect of the taint. In this case, NoExecute means pods won't be scheduled on tainted nodes
# Comment Labels Key below if you dont want to use Labels
# Comment Labels Key below if you dont want to use Labels
labels: # Labels are key-value pairs used for categorizing the node pool
environment: production # Label indicating that this node pool is for production use
team: "engineering" # Label associating the node pool with the engineering team
Loading