Batch Workload Optimization

Batch Workload Optimization reduces temporary over-provisioning in NodePools that run Jobs and other run-to-completion Pods. It combines two independent controls:

• Minimum active nodes directs new batch Pods toward a smaller active set so that extra nodes can drain naturally.
• Provisioning delay waits before creating capacity for unschedulable batch Pods, giving running work time to finish and release existing capacity.

Each control is optional. Leaving a field empty disables only that control; leaving both fields empty disables Batch Workload Optimization for the NodePool.

Availability

Batch Workload Optimization is available for AWS, GKE, and AKS NodePools with CloudPilot Agent v1.21.0 or later. It is not available for Alibaba Cloud NodePools.

Use this feature for NodePools dedicated to run-to-completion workloads. An eligible Pod has one of these restart policies:

restartPolicy: Never

or:

restartPolicy: OnFailure

This includes:

• Job and CronJob Pods
• one-time bare Pods
• batch Pods created by systems such as Argo Workflows, Spark operators, or other custom controllers

Pods with restartPolicy: Always, including typical Deployment and StatefulSet Pods, are not eligible.

Configure a NodePool

In the CloudPilot console:

1. Open Node Autoscaler and select Node Pools.
2. Add a NodePool or edit an existing NodePool.
3. Open Batch workload optimization.
4. Configure either or both fields, then save the NodePool.

The equivalent NodePool wrapper fields are:

jobNodePoolOptimization:
  minActiveNodes: 2
  provisioningDelay: 30s

jobNodePoolOptimization is a CloudPilot NodePool setting alongside nodePoolSpec; it is not part of the native Karpenter NodePoolSpec.

Minimum active nodes

minActiveNodes must be a positive whole number. It is the minimum number of nodes that CloudPilot prefers for new batch Pods, not a reservation of CPU or memory.

This setting does not create nodes to satisfy the configured count and does not prevent Karpenter from deleting empty nodes. It only selects the preferred active subset from the eligible nodes that currently exist.

After the NodePool has been stable for 10 minutes, CloudPilot periodically ranks its nodes and keeps the first minActiveNodes nodes active. Other eligible nodes receive this soft taint:

node.cloudpilot.ai/job-nodepool-draining=draining:PreferNoSchedule

The soft taint changes placement preference for new Pods. It does not:

• evict running Pods
• cordon the node
• delete the NodeClaim directly
• override Karpenter disruption budgets

Existing work continues until it completes. When a soft-draining node becomes empty, the NodePool’s normal Karpenter consolidation settings determine when it is deleted. Because PreferNoSchedule is best effort, a soft-draining node can still accept a burst when the active nodes do not have enough capacity.

For safe natural drain behavior:

• Keep the NodePool dedicated to run-to-completion Pods and DaemonSets.
• Add karpenter.sh/do-not-disrupt: "true" to active batch Pods. If a running batch Pod does not have this annotation, CloudPilot does not soft-drain the NodePool.
• Do not add broad tolerations that tolerate node.cloudpilot.ai/job-nodepool-draining; such tolerations cancel the placement preference.
• Keep node.cloudpilot.ai/managed=true on nodes managed by CloudPilot. CloudPilot uses this marker to avoid modifying externally managed or Auto Mode nodes.

Provisioning delay

provisioningDelay must be a positive, finite Go duration such as 30s, 1.5m, or 2m. It applies only when a Pod is unschedulable and new capacity would otherwise be provisioned.

The delay does not hold a Pod that can already fit on an existing node. The flow is:

1. Kubernetes first attempts to place the Pod on current active or soft-draining nodes.
2. If the Pod remains unschedulable, CloudPilot starts measuring its Pending age.
3. If existing capacity becomes available before the delay expires, Kubernetes schedules the Pod without creating a node.
4. If the oldest delayed Pod reaches the configured duration, all currently pending Pods for that NodePool enter the normal provisioning solve together.

To use provisioning delay, a Pod must resolve to exactly one target NodePool through either:

• nodeSelector["karpenter.sh/nodepool"], or
• required node affinity whose terms all select the same single NodePool.

Pods without a unique NodePool binding bypass this feature. Disruption replacement, interruption handling, repair, drift, static provisioning, and scheduled rebalance replacement also bypass the delay.

How the controls work together

When both controls are enabled:

1. New batch Pods use capacity on active nodes immediately.
2. If active nodes are full, soft-draining nodes remain available for bursts.
3. Only when all current nodes are full do Pods remain Pending and start the provisioning delay.
4. Capacity released during the delay is reused without creating a node.
5. If no capacity is released in time, normal provisioning creates the required capacity.
6. A newly created node restarts the fixed 10-minute stabilization window before CloudPilot recalculates the active and soft-draining sets.

Choose values

Start conservatively and measure both scheduling latency and node utilization.

Use a shorter delay or leave it empty for workloads with strict start-time objectives. A finite duration has no product-level maximum, so avoid values that could leave work Pending longer than the business process allows.

Operational considerations

• The fixed 10-minute stabilization window starts again when the NodePool gains a new node. During that window, the optimized pool can temporarily retain more nodes than a pool that consolidates immediately.
• Actual savings depend on arrival patterns, job duration, cloud instance startup time, available capacity, consolidateAfter, and disruption budgets.
• minActiveNodes controls a node count, not a minimum capacity target.
• Provisioning delay reduces unnecessary provisioning work but does not remove the cost of listing and validating Pending Pods.
• Clearing one field disables only that behavior. Clearing both fields removes the CloudPilot optimization annotations and any owned soft-drain taints.

Validation results

CloudPilot validated the combined behavior in a non-production GKE Standard cluster using fixed e2-standard-4 on-demand nodes. The treatment NodePool used minActiveNodes: 1 and provisioningDelay: 30s; the control NodePool left both fields empty. Both pools received the same Job, Never bare Pod, and OnFailure bare Pod traffic.

Only phases with complete continuous sampling are included below. Resource-seconds were calculated by integrating sampled Node and NodeClaim counts over time.

In the validation run:

• The control NodePool created an extra node for a short burst; the treatment NodePool reused capacity released within the 30-second delay and did not create that node.
• Ten mixed low-load Pods were distributed 5/5 across the two control nodes, while all ten treatment Pods used the single active node.
• Four additional tail Pods continued to use both control nodes but only the treatment active node.
• The treatment soft-draining node entered deletion 246 seconds before the first control node.
• All 16 measured Pods in each group were scheduled and started. All seven measured Jobs in each group completed, with no failed Jobs.
• The sampler, CloudPilot controller, and GCP provider recorded no errors during the measured phases; the maximum sample gap was 14 seconds.

Released-capacity timing determines whether an extra cloud node is avoided. These controlled results demonstrate the intended behavior; they are not a guaranteed savings rate for every cluster or traffic pattern.