Node Autoscaling

Node Autoscaling (karpenter) adds and removes nodes as demand changes. It runs Karpenter on Amazon EKS: it launches an instance for pods that fit nowhere, and removes the instance once it is empty. Node groups are fixed capacity; Node Autoscaling is for capacity that has to move, GPU capacity above all. It is Workload Autoscaling's counterpart: that one decides how many replicas run, this one provides the nodes.

When to use it

• Models need GPUs some of the time. Without it, an EKS managed node group is fixed capacity (minNode: 0 is refused there), so GPU nodes run and bill while idle. With the GPU pool on, the pool starts at zero nodes, launches one when a pod asks for a GPU, and removes it once it has stayed empty.
• A burst needs more nodes than the cluster has. Without it, new replicas stay Pending until someone adds nodes. With it, a node that fits them is launched in minutes, and removed when it empties.
• GPU spend needs a ceiling. GPU limit caps the GPUs the pool may run at once, and Allowed accelerators keeps the most expensive GPU models out of reach.
• Batch or experiments can run on cheaper machines. With Spot in GPU capacity types, the GPU pool buys Spot and drains a node before AWS reclaims it. Capacity reservations you hold are filled first.
• A CPU workload must not be interrupted. Set CPU capacity types to OnDemand to keep the default pool off Spot.

You don't need it when the cluster's load is steady and fixed node groups fit it, or when the cluster is not on EKS: GKE and AKS scale their own node pools, and EKS Auto Mode manages nodes itself.

Before you start

• An Amazon EKS cluster, created by AstroPulse or registered, on Kubernetes 1.30 or newer. astroctl infra k8s addons catalog <cluster> says whether this cluster can run it. It is refused on GKE, AKS, self-hosted Kubernetes and EKS Auto Mode (If it's refused); on EKS Auto Mode the cluster's own GPU pool settings do this (The GPU pool on EKS Auto Mode).
• GPU quota in the cluster's region for the GPU pool (Production guidance).
• A registered cluster only: the prerequisite stack in your AWS account (A registered EKS cluster).

In the console

After Version, its steps are Settings, AWS and Disruption, then Advanced and Resources. GPU capacity reservations and GPU disruption join them once Enable GPU node pool is on. Review runs the dry run, and Ask Nova at the top of the steps answers about the open step.

1. Open the cluster from the cluster list, select the Add-ons tab, then Install Add-on, and pick Node Autoscaling.

   Open zoom view

   Install Kubernetes Add-on, searched for Node Autoscaling

   

2. Turn on Enable GPU node pool if models will need GPUs. Optional: Allowed accelerators limits the GPU models it may launch, and GPU capacity reservations names reservations it fills first.

   Open zoom view

   Installing Node Autoscaling: Enable GPU node pool on and Allowed accelerators

   

3. Set the GPU limit, the most GPUs the pool may run at once. The other GPU settings, such as GPU node drain timeout, are optional.

   Open zoom view

   The GPU pool settings with GPU limit 4 and GPU node drain timeout

   

4. On an EKS cluster AstroPulse created, leave the AWS fields empty: AstroPulse creates the roles. IAM role ARN, Instance profile, Security group tags and Subnet tags are for a registered cluster. CPU limit caps the default pool.

   Open zoom view

   The AWS fields and CPU limit when installing Node Autoscaling

   

5. On Review, the dry run runs by itself. It checks the AWS connection and any reservations, and lists what the change does as notices (Dry run notices). Select Continue to deploy once it passes, then Deploy karpenter.

To change it later, open Node Autoscaling on the Add-ons tab, open Update and select Change configuration. Go to Review, select Continue to apply once the dry run passes, type karpenter, then select Update karpenter.

Open zoom view

Configure Node Autoscaling with the GPU node pool on

You should see

The add-on reads Deployed with its health checks passing, and Events has no rollout or AWS permission failure. A node is launched only when a pod cannot fit on the nodes there are.

Open zoom view

The Add-ons list with Node Autoscaling deployed and 1/1 checks passing

What AstroPulse sets up in your AWS account

On an EKS cluster AstroPulse created, the install creates the controller role, the node role, the node access entry, the Pod Identity association, the interruption queue and its event rules in the cluster's AWS account. You give no credential, role ARN, instance profile or template. A resync of Node Autoscaling sets these up again, so it needs the cluster's cloud connection.

Security: the controller signs in through the Pod Identity association with a role made for it alone, so no AWS key is stored on the cluster. On a cluster AstroPulse created, aws.iamRoleArn and aws.instanceProfile are refused, so the roles can't be swapped for wider ones.

Unless you set Security group tags, its nodes join the cluster's EKS security group, the one node group nodes use, so load balancers reach them. If an installed add-on's nodes sit in other security groups, apply or Resync it to move them. That replaces every Node Autoscaling node once, and the apply's dry run says so first, so pick a quiet time.

A registered EKS cluster

AstroPulse did not create a registered cluster's AWS resources and has no access to create them. Deploy the prerequisite stack once from astroctl cloud aws generate-karpenter, which writes a template you can review, then install Node Autoscaling with the stack's KarpenterControllerRoleArn in IAM role ARN. The install is refused without it. The cluster's subnets and security groups must carry the discovery tag karpenter.sh/discovery=<cluster>, or you enter the tags under Subnet tags and Security group tags. Set interruptionQueue: true when the stack's interruption queue is deployed; Spot needs it. Uninstalling the add-on leaves the stack in place.

The node pools Node Autoscaling creates

Node Autoscaling writes two Karpenter NodePools into the cluster. The general pool replaces every node after 30 days (expireAfter: 720h). The GPU pool's nodes do not expire unless you set gpuNodeExpiry, so a serving GPU node stays until it is empty. Consolidation removes at most 10% of a pool's nodes at once by default; that budget does not limit an expiry or a Spot reclaim.

	General pool	GPU pool
NodePool name	astro-default-nodepool	astro-gpu-nodepool (only with enableGpuNodePool: true)
Instances it may launch	Families c, m and r, generation 5 or later, never nano, micro or small	NVIDIA instances of families g and p, generation 4 or later (for example g4dn, g5, g6, g6e, p4d, p5), limited to allowedAccelerators when set
How it buys	cpuCapacityTypes; unset, Spot and On-Demand where the cluster has the interruption queue, otherwise On-Demand	gpuCapacityTypes: On-Demand unless you add Spot. Selected gpuCapacityReservations are filled first
Limit	cpuLimit vCPU (default 100) and memoryLimit memory (default 400Gi) across the pool, whichever is reached first. At the defaults, CPU binds first on c instances (2 GiB per vCPU), both at once on m (4 GiB per vCPU), and memory first on r (8 GiB per vCPU, about 50 vCPU)	gpuLimit GPUs across the pool
Removes a node	When it is empty or underused, 5 minutes after its pods last changed (disruption)	Only when it is empty, 10 minutes after its last pod left (gpuDisruption)
Node lifetime	Replaced after 30 days	gpuNodeExpiry, default Never
Drain bound	None: a node being removed waits for its pods	gpuNodeDrainTimeout, default 1h: a node being removed waits at most that long for its pods, then they are deleted
Taint and label	None	Taint nvidia.com/gpu:NoSchedule; label node.cluster.platform.astropulse.io/accelerator: nvidia
Root volume	The AMI family's default	100 GiB encrypted gp3, for model images and weights

Dry run notices

An apply never moves the GPU nodes you did not ask to move. Karpenter replaces every node whose template changes. The first apply with the GPU pool on stores the GPU settings it resolved (gpuCapacityTypes, gpuNodeExpiry, gpuNodeDrainTimeout), and a later apply that leaves any of them out keeps the stored value, so changing gpuLimit or another setting does not replace a serving GPU node. A dry run lists, as notices, each GPU node setting that would change and how many GPU nodes that replaces, by instance type, from the cluster's last node report:

Terminal

astroctl infra k8s addons apply -f karpenter.yaml --dry-run

Value	Label	Meaning
KarpenterGPUNodesReplaced	GPU nodes will be replaced	The apply changes GPU node settings, so Node Autoscaling replaces the GPU nodes it runs.
KarpenterNodesReplaced	Nodes will be replaced	The apply changes node settings, so Node Autoscaling replaces the nodes it runs.
KarpenterGPUSpot	GPU pool may buy Spot	The GPU pool may buy Spot capacity, which costs less but can be reclaimed at short notice.
KarpenterGPUExpiry	GPU nodes expire	Each GPU node is replaced after a set time, even while it serves.
KarpenterGPUCostCeiling	GPU pool cost ceiling	The most the GPU pool can cost per hour at its limit.
KarpenterGPUCostUnknown	GPU pool cost unknown	The GPU pool's cost at its limit could not be worked out.
KarpenterGPUIdleWait	Empty GPU nodes are billed until removed	How long an empty GPU node is kept before it is removed, and what it bills meanwhile.
KarpenterLimitChange	A node pool limit changes	The apply changes how much Node Autoscaling may launch in a node pool.
KarpenterDisruptionChange	Node removal settings change	The apply changes when Node Autoscaling removes or replaces nodes.
KarpenterIdleCost	Idle nodes stay billed	Idle nodes are kept, and billed, for longer before Node Autoscaling removes them.
KarpenterInterruptionQueueOn	The interruption queue is always on	The interruption queue is always on for a cluster AstroPulse created, so the setting changes nothing.
AddonUnchanged	Add-on unchanged	The add-on already runs the applied configuration, so nothing was started.
AddonConfigurationAdvice	Check this add-on configuration	Advice on the applied configuration.

A model with one replica goes offline while its GPU node is replaced. Costs use the catalog's prices for the region, and KarpenterGPUIdleWait is said on every dry run while the GPU pool is on. AddonConfigurationAdvice names what the apply could not check: the prerequisite stack it could not read, or the applications whose GPU capacity it can't tell.

The response's outcome is Proceed, or Blocked when a notice blocks the apply.

A dry run of the deployed configuration says "No changes". You can tell it apart from a change waiting to go out, and a real apply of the same file starts nothing. On a registered cluster AstroPulse compares the configuration alone. On a cluster AstroPulse created it also reads the AWS prerequisite stack. If that read fails, the answer is still "No changes", and a notice says the stack was not checked: "AstroPulse could not read Karpenter's prerequisite stack in your AWS account just now, so it was not checked and nothing was changed. Apply again to check it. If the read keeps failing, an add-on resync skips the check and repairs the stack." When an earlier apply is still running or failed, or the stack was deleted, rolled back or changed outside AstroPulse, a dry run reports the configuration as valid instead, and an apply runs again and repairs the stack.

Both pools launch only Linux nodes of the cluster's recorded CPU architecture (amd64 or arm64).

The GPU pool on EKS Auto Mode

Node Autoscaling is refused on an EKS Auto Mode cluster. There, the cluster's own autoMode.enableGpuNodePool and autoMode.gpuLimit settings create a GPU pool with the same NodePool name, astro-gpu-nodepool, but it behaves differently:

	Node Autoscaling GPU pool	EKS Auto Mode GPU pool
Set with	enableGpuNodePool, gpuLimit and the GPU settings above on this add-on	autoMode.enableGpuNodePool, autoMode.gpuLimit, autoMode.gpuCapacityTypes on the cluster
Instances	NVIDIA g and p, generation 4 or later, limited by allowedAccelerators	g and p, generation 4 or later, with no GPU-maker requirement
How it buys	gpuCapacityTypes, On-Demand by default, with selected reservations first	autoMode.gpuCapacityTypes: On-Demand on a new cluster unless you add Spot; an update that leaves it out keeps what the pool buys; no reservations
Node lifetime	gpuNodeExpiry, default Never	No lifetime set by AstroPulse; EKS Auto Mode replaces nodes after at most 21 days (EKS Auto Mode)
Removal	Empty for 10 minutes, or gpuDisruption.consolidateAfter; 10% budget	Empty for 10 minutes, set by AstroPulse; 10% budget
Disk	100 GiB root volume	100 GiB ephemeral storage

How the GPU pool picks an instance

A GPU node is launched only for a pod that is Pending because no existing node can hold it. For that pod, the pool:

1. Lists the instance types that fit. It keeps the g and p NVIDIA instances above whose GPUs, vCPUs and memory cover the pod's requests (nvidia.com/gpu, cpu, memory), and that match the pod's node selectors and affinity. A model application that names source.model.resources.accelerator.gpu.model carries a required affinity on that GPU model, so only instances with that GPU remain.
2. Picks how to buy it, from what gpuCapacityTypes allows, in this order: a capacity reservation or Capacity Block selected in gpuCapacityReservations, then Spot, then On-Demand. With the default gpuCapacityTypes: ["OnDemand"] that is a selected reservation, then On-Demand; Spot is used only when you list Spot. When AWS has no capacity of the preferred kind, Karpenter falls back to the next one (Karpenter NodePools). A model whose capacityType is OnDemand or Spot launches only nodes bought that way (see Pin a GPU model or let the pool pick).
3. Picks the instance. On-Demand takes the cheapest instance type that fits. Spot takes the instance from the pools AWS is least likely to reclaim, then the lowest price among them (the price-capacity-optimized strategy; see Using Amazon EC2 Spot Instances with Karpenter).
4. Checks gpuLimit. Once the GPUs on the pool's nodes reach gpuLimit, it launches no more GPU nodes until one is removed, and the pod stays Pending. A node counts all of its GPUs: one p5.48xlarge counts 8.

gpuLimit is the cost cap: the most the pool can cost per hour is about gpuLimit times the highest per-GPU hourly price among the instances it may launch. With no GPU model named, that includes the largest p instances (H100, H200, B200 and B300 nodes of 8 GPUs). Read the current prices for your region before choosing the limit:

Terminal

astroctl cloud aws accelerators list -r us-west-2

Name a GPU model on your workloads, or limit the pool with allowedAccelerators, to make the ceiling predictable. A dry run of the add-on states the ceiling for your region as the KarpenterGPUCostCeiling notice when the GPU pool is added or what it may run changes.

Consolidation costs and saves. The GPU pool removes a node only when it is empty, and only 10 minutes after its last pod left, so each scale-down bills at most a sixth of an hour of an idle GPU instance per node. Ten minutes is still longer than one cold start (a new instance, its driver, a multi-gigabyte image pull and the model weights take 5 to 10 minutes), so deleting a model and applying it again, or a rollout, finds the node still there. To keep a warm node longer for faster restarts, raise gpuDisruption.consolidateAfter: a dry run of the add-on states what the wait bills as the KarpenterGPUIdleWait notice. A partly used GPU node is never repacked, so a running job is never evicted to save part of a card.

GPU nodes are not replaced while they serve, by default. With gpuNodeExpiry: Never, a GPU node leaves only when it is empty, when its settings change (a KarpenterGPUNodesReplaced notice in the dry run says so first), or when AWS reclaims a Spot node. Set gpuNodeExpiry (such as 30d) only when nodes must be refreshed on a schedule: in Karpenter v1, expiry is a forceful disruption. The NodePool's disruption budget does not delay it, and a pod's karpenter.sh/do-not-disrupt annotation does not prevent it from starting. Only PodDisruptionBudgets and gpuNodeDrainTimeout bound the drain (Karpenter disruption). A model with one replica then has a short outage at each expiry, while its replacement node launches and loads the model. Two or more replicas, with a PodDisruptionBudget that keeps one available, keep serving.

gpuNodeDrainTimeout (default 1h) is how long a GPU node being removed waits for a PodDisruptionBudget or a karpenter.sh/do-not-disrupt pod before its pods are deleted. Without a bound the node is billed for as long as the drain waits.

See what a model will launch before it does

A dry run of a model application on a cluster with the GPU pool shows the placement without launching anything:

Terminal

astroctl app apply -f qwen-chat.yaml --dry-run -o yaml

The placement block names the nodePool, the instanceTypes that can hold one replica (the one most likely to launch first, with its On-Demand and Spot price per hour), the capacityTypes in the order the pool buys them, the gpuLimit, the GPUs the model takes at gpusAtMinReplicas and gpusAtMaxReplicas, and the account quota one replica's node draws on.

Pin a GPU model or let the pool pick

The model application's source.model.resources.accelerator.gpu.model field decides this. For any other workload, a node selector on karpenter.k8s.aws/instance-gpu-name does the same.

Pin the GPU model (gpu.model: l4, a10g, l40s, h100, ...) when:

• the model needs a minimum of GPU memory per device. The pool chooses by GPU count, vCPU, memory and price; it does not read GPU memory, so an unpinned one-GPU replica can land on a 16 GB T4 (g4dn), usually the cheapest GPU it launches;
• the model or runtime needs a GPU generation, such as FP8 weights;
• you need repeatable latency and throughput, or a predictable cost ceiling under gpuLimit;
• you hold a reservation or Capacity Block for one instance type.

Let the pool pick (omit gpu.model) when the model fits on the smallest GPU the pool launches and any NVIDIA GPU from generation 4 on runs it, for example small embedding or classification models, tests and batch work where price matters more than speed.

qwen-chat.yaml (excerpt)

source:
  type: "Model"
  model:
    uri: "hf://Qwen/Qwen3-8B:<40-character commit sha>"
    resources:
      accelerator:
        gpu:
          model: "l4"          # pinned: only instances with an L4 GPU, such as g6, launch
          countPerReplica: 1
          capacityType: "OnDemand"   # never on Spot; reserved capacity counts

A pinned model that the pool cannot launch in the region, that allowedAccelerators leaves out, or that comes only in instances with more GPUs than gpuLimit, is refused at apply with the models the pool can launch, before any node is requested.

source.model.resources.accelerator.gpu.capacityType decides how the replica's GPU node is bought: OnDemand (never Spot; capacity reservations and Capacity Blocks count), Spot, or Any (the default, whatever the cluster offers). The replica gets a required node affinity on nodes bought that way: GPU node groups whose instanceType matches, and the GPU pool when its gpuCapacityTypes includes it. The dry run's placement prices and quota are then for that purchase option. When nothing on the cluster is bought that way, the apply is refused, naming capacityType and what the cluster offers. On a registered cluster only the GPU pool counts, because AstroPulse does not know how its own node groups are bought.

Production guidance

• On-Demand, or reservations, for serving. A Spot GPU can be reclaimed with a two-minute warning (Spot interruptions), and a serving replica on it restarts and reloads its model. The GPU pool buys On-Demand by default, and capacity you reserved with gpuCapacityReservations is filled first. When the pool also allows Spot, set capacityType: OnDemand on every serving model that must not restart; for an image, Helm or YAML workload, add the node selector karpenter.sh/capacity-type: on-demand to its pods. Keep gpuNodeExpiry: Never so a serving node is not replaced on a timer, and run two or more replicas with a PodDisruptionBudget for a model that must stay up.

• Spot for batch and training. Work that checkpoints and can restart saves the most on Spot. Add Spot to gpuCapacityTypes and set capacityType: Spot on those models. The cluster's interruption queue drains a node when AWS announces it will reclaim it; a registered cluster needs interruptionQueue: true, from the generate-karpenter stack, before Spot is accepted.

• Limit the GPU models. allowedAccelerators keeps the most expensive instances out of reach; gpuLimit stays the hard cap on GPU spend.

• GPU quota before the first launch. EC2 quotas count vCPUs, per region, and are separate for Spot and On-Demand. A new account starts at 0 for GPU families. g instances draw on "Running On-Demand G and VT instances" (L-DB2E81BA) and "All G and VT Spot Instance Requests" (L-3819A6DF); p instances on "Running On-Demand P instances" (L-417A185B) and "All P Spot Instance Requests" (L-7212CCBC). Ask for at least gpuLimit times the vCPUs per GPU of the instance you expect, in the cluster's region (EC2 instance quotas). List the codes for your region with:

  Terminal

  aws service-quotas list-service-quotas --service-code ec2 --region us-west-2 \
    --query "Quotas[?contains(QuotaName, 'G and VT') || contains(QuotaName, ' P ')].[QuotaName,QuotaCode,Value]" \
    --output table

• Reservations and Capacity Blocks for steady or scarce GPUs. An On-Demand Capacity Reservation holds capacity in one zone for as long as you keep it; a Capacity Block holds GPU instances for a fixed period you buy in advance, and AWS reclaims its instances when the period ends (Capacity Blocks for ML). AstroPulse never buys either. Select them with gpuCapacityReservations; prefer tags for a block you renew, because a renewed block has a new ID.

• Keep the AMI pinned. Leave amiAlias empty to take AstroPulse's reviewed AMI for the cluster's Kubernetes version. A new alias replaces every node. On a Kubernetes upgrade, a default image not published for the target moves, once the control plane runs the target and the cluster is READY, to the release AWS recommends for it, in one roll. The dry run says the add-on's nodes, GPU nodes included, are replaced. An amiAlias you set is never moved, and an upgrade it cannot follow is refused, naming the release.

Uninstalling removes its nodes. It deletes the node pools, then drains and terminates every node Node Autoscaling launched, waiting up to 15 minutes, so the pods on them move to your fixed nodes or wait. The controller itself runs on your fixed node groups, never on a node it launched, so keep at least one.

AI infrastructure notes

Node Autoscaling reacts to pending pods, not to incoming requests. A model's first replica after a quiet period waits for an EC2 instance to launch and join the cluster; keep fixed GPU capacity when that wait is not acceptable. How it works with Workload Autoscaling and Workload Queueing: How the three fit together.

The model's replica limit sets how many requests it serves at once; GPU limit sets the largest GPU footprint. Neither reserves AWS capacity or a GPU before the workload needs it. An empty GPU node stays for 10 minutes before it is removed (gpuDisruption.consolidateAfter), so GPU cost stops when the node leaves, not when the model's pod stops.

Turning the GPU pool off

Set Enable GPU node pool off and remove the GPU pool's settings from the manifest. A GPU setting still written is refused, and one refusal names all of them. gpuLimit may stay: it is ignored while the pool is off.

A change that turns the pool off or narrows it, for example with Allowed accelerators or a lower GPU limit, is refused on the apply and its dry run when a model application on the cluster fits the pool today and would have no node for its GPUs afterwards: its replicas would wait Pending. A GPU node group that can still hold the model counts as a node for it, and an application being deleted does not count. The check reads the GPUs a model application asks for (source.model.resources.accelerator.gpu); it does not see GPU workloads deployed another way, so check those yourself. On a registered cluster, or when the machine catalog can't be read, the apply goes ahead with an AddonConfigurationAdvice notice that names the applications.

How to debug Node Autoscaling

Start with the add-on's own state, then the pod, then the NodeClaim Karpenter created for it. The add-on runs in the kube-system namespace.

Terminal

astroctl infra k8s addons status <cluster> karpenter
astroctl infra k8s addons events <cluster> karpenter
astroctl infra k8s addons logs <cluster> karpenter
kubectl describe pod <pending-pod>
kubectl get nodeclaims -o wide
kubectl describe nodeclaim <nodeclaim-name>
kubectl get nodes -L karpenter.sh/nodepool,karpenter.sh/capacity-type,node.kubernetes.io/instance-type,karpenter.k8s.aws/instance-gpu-name

What you see	Why	What to do
A GPU pod is Pending and no NodeClaim appears	The GPU pool is off, or the pod can't match it: no nvidia.com/gpu toleration, or a selector the pool can't meet	Turn on Enable GPU node pool. Pods need a toleration for nvidia.com/gpu; model applications get it. The pod's events name the requirement it fails
Pod events say the instance types exceed the NodePool's limits	The GPU pool is at GPU limit, or the default pool at CPU limit or Memory limit	Raise the limit, or free capacity
A NodeClaim fails with VcpuLimitExceeded or MaxSpotInstanceCountExceeded	The account's On-Demand or Spot vCPU quota is used up, often 0 on a new account	Raise the quota the error names; the pool retries on its own
A NodeClaim fails with InsufficientInstanceCapacity	AWS has none of that instance in that zone right now	Karpenter tries other instance types and purchase options. For scarce GPUs, unpin the model, allow more zones, or reserve capacity
The pod stays Pending with a pinned gpu.model, and nodes of another GPU exist	The replica waits for its GPU model rather than land on another	Pin a model the pool can launch in the region, or remove the pin
The node launched is not the instance you expected	Without gpu.model the pool takes the cheapest instance that fits; with Spot allowed it buys Spot first	Pin gpu.model, or set capacityType: OnDemand on the model, and read the dry run's placement block
A node joins, but the pod does not start or sees no GPUs	The NVIDIA device plugin is not ready on the node, or one you installed competes with it	Read the add-on's events. Remove a device plugin you installed cluster-wide, or keep it off this pool's nodes
An empty GPU node is not removed	The 10-minute wait has not passed, a budget allows 0 nodes, a pod has karpenter.sh/do-not-disrupt: "true", a PodDisruptionBudget blocks eviction, or the node is a node group's	kubectl describe node <node> says why. Nodes without the karpenter.sh/nodepool label belong to node groups, which Node Autoscaling never removes
GPU nodes are replaced on a schedule, and a one-replica model briefly goes away	GPU node expiry is set, and expiry is not held back by budgets or do-not-disrupt	Set it to Never, or run two replicas with a PodDisruptionBudget that keeps one
GPU nodes were all replaced after an apply	The apply changed a GPU node setting	Dry-run first: KarpenterGPUNodesReplaced names each change. Leave a setting out to keep its stored value
A GPU node being removed is gone before its pods finished	GPU node drain timeout passed, and the pods left were deleted	Raise it, up to 24h, or set None to wait while the node stays billed

More cases: Karpenter troubleshooting.

Other ways in

• Nova: ask it, for example "install Node Autoscaling on eks-prod with a GPU pool of up to 8 GPUs". Nova shows the change and applies it after you approve.
• SDK and API: the same operations, in the API reference.

With astroctl

karpenter.yaml

apiVersion: platform.astropulse.io/v1
kind: K8sClusterAddon
metadata:
  name: karpenter
spec:
  clusterName: eks-prod
  configuration:
    karpenter:
      enableGpuNodePool: true
      gpuLimit: 8
      allowedAccelerators: ["l4", "l40s"]

Terminal

astroctl infra k8s addons apply -f karpenter.yaml --dry-run
astroctl infra k8s addons apply -f karpenter.yaml

astroctl infra k8s addons template <cluster> karpenter writes every setting with what it does.

Settings

Its own settings, under karpenter, plus availability and resources. Every one is optional. The GPU settings need Enable GPU node pool, and an installed GPU pool keeps a GPU node setting you leave out.

Enable GPU node poolkarpenter.enableGpuNodePool

Lets Node Autoscaling launch GPU instances. GPU nodes are tainted nvidia.com/gpu:NoSchedule, so pods need a matching toleration. AstroPulse manages the NVIDIA device plugin on the pool's nodes.

Default

Off.

GPU limitkarpenter.gpuLimit

The most GPUs the GPU pool may run at once, counted across the pool. It stays the hard cap on GPU spend. 0 is refused while the pool is on; turn the pool off instead.

Accepts

1 to 1000.

Default

10.

GPU capacity typeskarpenter.gpuCapacityTypes

How the GPU pool buys nodes. Spot costs less, but AWS can reclaim a node at two minutes' notice. Spot needs the interruption queue: always there on a cluster AstroPulse created, and on a registered one with interruptionQueue: true.

Accepts

OnDemand, Spot, or both.

Default

OnDemand for a new pool; an existing pool keeps its own.

Allowed acceleratorskarpenter.allowedAccelerators

The GPU models the GPU pool may launch, as the machine catalog names them, such as l4, a10g, l40s or h100; at most 32.

Default

Every model.

See

astroctl cloud aws accelerators list -r <region>

GPU instance familieskarpenter.gpuInstanceFamilies

Limits the GPU pool to these instance families, such as g6 or g5, and gpuInstanceSizes to sizes such as xlarge; at most 32 each.

GPU max vCPU per nodekarpenter.gpuMaxVcpuPerNode

The most vCPU one GPU node may have (1 to 1024), and gpuMaxMemoryGiBPerNode the most memory in GiB (1 to 32768).

Default

Unset: the smallest sizes that fit each model and the account's quota.

GPU node expirykarpenter.gpuNodeExpiry

How long a GPU node runs before it is replaced, even while it serves.

Accepts

Never, or whole days or hours from 1h to 365d, such as 30d.

Default

Never for a new pool.

GPU node drain timeoutkarpenter.gpuNodeDrainTimeout

The most time a GPU node being removed waits for its pods before they are deleted.

Accepts

1m to 24h, such as 1h30m, or None for no limit.

Default

1h for a new pool.

GPU disruptionkarpenter.gpuDisruption

When GPU nodes are removed to save cost: consolidationPolicy, consolidateAfter (5m to 720h, or Never) and budgets, as for the default pool below.

Default

WhenEmpty, 10m, one budget of 10%.

GPU capacity reservationskarpenter.gpuCapacityReservations

EC2 capacity reservations, Capacity Blocks included, that the GPU pool fills before Spot or On-Demand: ids (cr-...), or tags (key=value) a reservation must all carry. A tag keeps matching a renewed reservation. AstroPulse checks each one exists in the cluster's account and region, is active or scheduled, reserves Linux instances on default tenancy, is a g or p type of generation 4 or later, and is in the cluster's zones. Registered clusters are not checked. AstroPulse never buys reservations.

CPU capacity typeskarpenter.cpuCapacityTypes

How the default pool buys nodes. An apply that leaves it out keeps the value it was last set to.

Accepts

OnDemand, Spot, or both. Spot needs the interruption queue.

Default

Spot and On-Demand, Spot first, where the cluster has the interruption queue; On-Demand elsewhere.

CPU limitkarpenter.cpuLimit

The most vCPU the default pool runs at once across its nodes. Pods wait once it is reached.

Accepts

1 to 100000.

Default

100.

Memory limitkarpenter.memoryLimit

The most memory the default pool runs at once. The pool stops adding nodes at whichever of the two limits it reaches first.

Accepts

1Gi to 1000Ti.

Default

400Gi.

Disruptionkarpenter.disruption

When default pool nodes are removed to save cost. consolidationPolicy: WhenEmpty or WhenEmptyOrUnderutilized (also moves pods onto fewer nodes, restarting them). consolidateAfter: 30s to 720h, or Never. budgets[]: nodes, the most that may change at once, as a count or a percentage (0 stops removals, so idle nodes stay billed); schedule (five-field cron in UTC, or @daily and the like) with duration; and reasons (Empty, Underutilized, Drifted, Unhealthy). When several budgets apply, the smallest wins; at most 50.

Default

WhenEmptyOrUnderutilized, 5m, one budget of 10%.

AMI aliaskarpenter.amiAlias

The machine image nodes launch from, as family@version, such as al2023@v20260818. Changing it replaces existing nodes; a floating @latest is refused with the GPU pool on.

Default

AstroPulse's pinned image for the cluster's Kubernetes version.

Interruption queuekarpenter.interruptionQueue

Registered clusters only: the cluster has the interruption queue from the generate-karpenter stack, so nodes are drained before Spot reclaims them and Spot is allowed. On a cluster AstroPulse created it changes nothing.

IAM role ARNkarpenter.aws.iamRoleArn

Registered clusters only: the role the controller assumes, the stack's KarpenterControllerRoleArn. aws.instanceProfile (Instance profile) is the profile attached to the nodes it launches. Both are refused on a cluster AstroPulse created.

Subnet tagskarpenter.aws.subnetTags

key=value tags that select the subnets, and aws.securityGroupTags (Security group tags) the security groups, for the nodes it launches.

Default

A registered cluster: karpenter.sh/discovery=<cluster>. A cluster AstroPulse created: its private subnets and its EKS security group.

Service account namekarpenter.serviceAccountName

The Kubernetes service account the controller runs as.

Default

AstroPulse's default.

If it's refused

"Node Autoscaling is supported only on Amazon Web Services clusters"

Cause

The cluster is on GKE or AKS, or registered outside AWS.

Fix

Use GPU node groups there: GPU node groups.

"Node Autoscaling is supported only on Amazon EKS or registered clusters"

Cause

The cluster is self-hosted Kubernetes on AWS.

Fix

Use its node groups.

"Node Autoscaling is not installed on this cluster because its cloud already manages that for it"

Cause

The cluster is EKS Auto Mode.

Fix

Set the cluster's own autoMode.enableGpuNodePool and autoMode.gpuCapacityTypes (The GPU pool on EKS Auto Mode).

"Node Autoscaling needs Kubernetes <version> or newer; this cluster runs <version>"

Cause

The cluster's Kubernetes version is below what the add-on runs on.

Fix

Upgrade the cluster first.

"Karpenter AWS preflight failed before deployment: ..."

Cause

The cluster's AWS connection failed one or more checks; the message states each check and how to fix it.

Fix

Do what each check says, then apply again. The dry run runs the same checks and changes nothing.

"The field 'gpuLimit' has invalid value '0'. Set enableGpuNodePool to false to turn the GPU pool off."

Cause

gpuLimit: 0 with the GPU pool on.

Fix

Set GPU limit to 1 or more, or turn the GPU pool off.

"<settings> require enableGpuNodePool: set it to true, or remove them to turn the GPU pool off"

Cause

GPU pool settings are still written with the pool off. The refusal names all of them.

Fix

Remove the settings it names, or turn the pool on. gpuLimit may stay.

"gpuCapacityTypes lists spot, which needs Karpenter's interruption queue so a reclaimed node is drained first: ..."

Cause

A registered cluster without the interruption queue. The same refusal names cpuCapacityTypes for the default pool.

Fix

Deploy the stack astroctl cloud aws generate-karpenter writes and set interruptionQueue: true, or remove Spot.

"allowedAccelerators lists <models>, which no GPU instance the GPU pool can launch in <region> carries; use one of <models>"

Cause

A listed GPU model is not carried by any GPU instance in the cluster's region.

Fix

Use one of the models the refusal lists; astroctl cloud aws accelerators list -r <region> shows them with prices.

"The GPU pool would no longer launch the GPUs application <name> runs on, so its replicas would wait Pending until the pool launches them again. ..."

Cause

The change turns off or narrows the pool, and the named model application has no other node for its GPUs.

Fix

Delete or move that application first, or keep the pool able to launch those GPUs, then apply again.

"AstroPulse creates and manages Node Autoscaling's roles on a cluster it created. Remove <settings> from the add-on configuration and apply again"

Cause

aws.iamRoleArn or aws.instanceProfile is set on a cluster AstroPulse created.

Fix

Remove what the message names, then apply again.

"gpuCapacityReservations selects reservations the GPU pool cannot use: ..."

Cause

The message states each problem: a reservation or tag that matches nothing in the account and region, or one that is not active, not Linux, not on default tenancy, not a GPU type the pool launches, or outside the cluster's zones.

Fix

Remove it, or name a reservation the pool can use, then apply again.

"The cloud account's GPU quota cannot launch one replica"

Cause

A model application's dry run: the account's quota for that instance family is below one node's vCPUs, for every purchase option the pool may use.

Fix

Request the quota the message names in AWS Service Quotas, in the cluster's region.

"AstroPulse could not read Karpenter's prerequisite stack in your AWS account just now, so it was not checked and nothing was changed. ..."

Cause

A notice beside "No changes": the read of the stack failed, so it was not checked.

Fix

Apply again to check it. If it keeps failing, astroctl infra k8s addons resync <cluster> karpenter skips the check and repairs the stack.

"AstroPulse couldn't check or move the Node Autoscaling node image for the new Kubernetes version; the upgrade itself finished. ..."

Cause

A cluster upgrade finished, but the node image could not be checked or moved. New nodes may not launch.

Fix

Run astroctl infra k8s addons resync <cluster> karpenter, and set AMI alias if the answer says to.

Next

• Workload Autoscaling
• GPU node groups
• Manage add-ons