Memory Extension

Memory Extension improves node memory resilience by moving inactive memory pages into cold memory backed by the node’s root disk. Frequently reused compressed pages can remain in cached memory for faster access. This gives workloads additional time and capacity during node-level memory pressure and can reduce avoidable application termination when actual memory use grows beyond Pod requests.

Memory Extension is intended for workloads with a meaningful inactive memory set, such as object caches, session data, sparse in-memory indexes, and Java heaps containing long-lived objects that are not accessed continuously. It is a resilience mechanism, not a replacement for correct Pod requests and limits.

Requirements

Memory Extension currently supports:

• Amazon EKS;
• AWS EC2NodeClass resources managed by CloudPilot AI;
• Amazon Linux 2023 node images; and
• nodes with at least 4 GiB of physical memory.

Enable Memory Extension in the console

1. Open the target cluster in CloudPilot AI.
2. Go to Node Autoscaler and open NodeClass.
3. Create or edit an AWS NodeClass.
4. In Image & OS, select an Amazon Linux 2023 image.
5. In Metadata & add-ons, enable Memory Extension.
6. Keep the defaults for the initial rollout, or adjust the settings using the guidance below.
7. Save the NodeClass.
8. Replace or rebalance nodes that use this NodeClass so the configuration is applied to newly created nodes.

Configuration options

The console applies the following defaults when Memory Extension is enabled:

The sizing mode is currently Auto. For each new node, CloudPilot AI:

1. calculates the cold memory target from physical RAM;
2. applies the configured minimum and maximum;
3. applies the root disk percentage limit; and
4. reduces the result when necessary to preserve root disk free space.

CloudPilot AI preserves at least 10% of the root disk or 5 GiB, whichever is larger. The actual cold-memory capacity can therefore be lower than the RAM-based target. If the node cannot satisfy Minimum cold memory while preserving disk headroom, Memory Extension is not activated on that node.

The reclaim behavior is currently fixed to Conservative. It favors moving inactive pages before memory pressure becomes critical while retaining safety margins for Kubernetes and the container runtime. This policy is not currently an editable console option.

Recommended starting configuration

Use the default configuration for the first rollout:

Cold memory target: 30%
Minimum cold memory: 2 GiB
Maximum cold memory: 16 GiB
Root disk limit: 20%
Cached memory limit: 20%

Start with one non-critical NodePool, replace a small number of nodes, and observe workload behavior before expanding the rollout.

When to increase cold memory

Consider increasing Cold memory target or Maximum cold memory when all of the following are true:

• the node has sufficient root disk capacity and disk performance;
• workloads have a large inactive memory set;
• application tail latency remains acceptable when inactive data is accessed; and
• nodes remain healthy under sustained pressure.

A larger target does not guarantee an equal increase in usable workload memory. The benefit depends on how much memory is inactive, how well cached pages compress, the amount of disk headroom, and the rate at which workloads revisit cold data.

When to decrease cold memory

Decrease Cold memory target or Maximum cold memory when:

• cold-data access causes unacceptable latency;
• garbage collection or application pauses increase significantly;
• the node root disk has limited capacity, throughput, or IOPS;
• container images, logs, or temporary files compete for root disk space; or
• workloads continuously scan or rewrite most of their memory.

When to change cached memory

Increase Cached memory limit only when compressed cached pages provide clear latency benefits and active workloads still have sufficient RAM. Decrease it when active working sets need more RAM or when cached-memory growth does not improve application latency.

Pod requests and limits

Memory Extension does not change Kubernetes scheduling semantics:

• Set a non-zero memory request for every participating container. A container’s request determines its share of cold-memory capacity under pressure.
• Keep total Pod requests within Node Allocatable so the scheduler can place the Pods on the node.
• Set memory limits above the expected working set. Memory Extension does not bypass a container memory limit; a container can still be terminated when it reaches its own limit.
• BestEffort containers do not receive a guaranteed cold-memory allocation because they have no memory request.

The common use case is therefore: total requests fit on the node, but actual use grows above requests while remaining below individual Pod limits. Memory Extension can move inactive pages out of active RAM so the node has more room to absorb that growth.

Operational considerations

Node Allocatable and safety margins

Enabling Memory Extension applies memory safety reservations for the operating system, Kubernetes, and the container runtime. Node Allocatable may change after the node is replaced. Compare scheduling capacity using the new node’s reported Allocatable value rather than physical memory alone.

The current policy keeps a 512 MiB available-memory safety threshold, reserves 256 MiB and 100 millicores for Kubernetes, reserves another 256 MiB and 100 millicores for system services, and protects the Kubernetes and container runtime services during pressure. These guardrails are automatic rather than editable Memory Extension options. When the feature is disabled, CloudPilot AI restores the NodeClass values that were present before it managed these settings; running nodes still require replacement to receive the restored configuration.

Root disk capacity and performance

Cold memory uses the root disk. Keep additional headroom for the operating system, container images, writable layers, logs, and temporary files. Access to cold data is slower than access to active RAM, so disk throughput, IOPS, and latency directly affect application tail latency under pressure.

If Image Accelerator is enabled on the same NodeClass, both features can operate together. They do not overwrite each other’s NodeClass configuration, but container image data and cold memory may both depend on node storage capacity. Size the root disk for both workloads and keep the Root disk limit conservative.

Workload behavior

Memory Extension works best when part of the workload memory is inactive for meaningful periods. Benefits are smaller for workloads that:

• continuously scan their entire memory set;
• frequently rewrite most pages;
• keep mostly non-compressible data; or
• require consistently low latency for every memory access.

For Java workloads, long-lived inactive objects can benefit, while full-heap scans and allocation-heavy workloads can increase disk activity and garbage-collection pauses. Validate with the application’s real traffic pattern instead of relying only on synthetic memory allocation.

Monitoring

During rollout, monitor:

• node Ready and MemoryPressure conditions;
• root disk usage, throughput, IOPS, and latency;
• application P95 and P99 latency, especially cold-data paths;
• Java garbage-collection pause time where applicable;
• Pod termination, restart, and rescheduling events; and
• the difference between Pod requests, actual use, and Node Allocatable.

Disable or change the configuration

To disable Memory Extension, edit the NodeClass, turn off Memory Extension, and save it. To tune the configuration, edit the fields and save the NodeClass.

In both cases, running nodes keep their existing configuration. Replace or rebalance those nodes to apply the updated NodeClass. Plan the rollout so Pod disruption budgets, replicas, and capacity are sufficient during replacement.