Optimizing swap performance for databases: zswap, swappiness and SSD strategies

Optimizing swap performance for databases is not a theoretical exercise but an operational concern with direct consequences for latency, stability and recovery. The focus keyword is set early: optimizing swap performance for databases means aligning kernel policy, compression and storage so that swap serves as a controllable safety net rather than a trigger for domino effects. This guide expands basic diagnostics with measurement methods, load tests, Kubernetes specifics, I/O schedulers and concrete implementation and rollback steps.

Optimizing swap performance for databases in practice

Good decisions are based on correlations, not on isolated observations. Remember: swap utilization alone is useless; what matters are activity, queue length and application metrics.

Supplementary tools and metrics:

• iostat -xz: queue depth (avgqu-sz), await and svctm indicate saturation.
• sar -B: provides swap-in/out trends over longer intervals (historically important).
• perf top / eBPF: shows whether compression/decompression is CPU-bound.
• Database metrics: 99th-percentile latencies, fsync latency, checkpoint duration, repl lag.
• PSI (Pressure Stall Information): measures actual stall time due to memory/IO pressure.

# Extended monitoring snippet (30s intervals for 30 minutes)
for i in {1..60}; do
  date +"%Y-%m-%d %H:%M:%S"
  vmstat 1 2 | tail -n1
  iostat -xz 1 2 | awk 'NR>6{print; exit}'
  [ -f /proc/pressure/memory ] && cat /proc/pressure/memory
  sleep 29
done

Tip: Store these measurement series in an artifact (Compressed Log) and push them to a central observability endpoint. Only then can runbooks be triggered automatically.

Load tests: producing reproducible memory pressure

Before you change kernel parameters permanently, simulate memory pressure in a controlled way. For databases it is sensible to use real workloads (replay OLTP workloads) or to work synthetically with stress-ng. The goal is not to destroy the system but to reproduce typical peaks.

# Example: stress-ng VM load 75% of RAM size over 5 minutes
# Caution: on production systems only in a maintenance window
total_kb=$(awk '/MemTotal/{print $2}' /proc/meminfo)
alloc_kb=$((total_kb*75/100))
sudo stress-ng --vm 1 --vm-bytes ${alloc_kb}K --vm-keep --timeout 300s --metrics-brief

During the test monitor vmstat, iostat, PSI and your DB metrics. Good tests show if and when swap-in begins, how quickly iowait rises, and whether zswap smooths the I/O spike.

Concrete kernel and storage configurations

Recommendations are context-dependent. The following examples are conservative starting points and should be validated in your environment.

• vm.swappiness=10 for dedicated DB hosts, 20–30 in mixed environments.
• zswap.enabled=1, zswap.compressor=lz4, zswap.max_pool_percent=10–20.
• Swap on a separate NVMe device or LUN; if not possible, set I/O limits via cgroups.
• For NVMe: check the scheduler (mq-deadline or bfq) and control queue limits.

Example: set and check swappiness and zswap:

# Set swappiness permanently
echo 'vm.swappiness=10' | sudo tee /etc/sysctl.d/99-db-swap.conf
sudo sysctl --system

# Check zswap (enabled usually via kernel cmdline, but check runtime parameters)
cat /sys/module/zswap/parameters/enabled
cat /sys/module/zswap/parameters/compressor
cat /sys/module/zswap/parameters/max_pool_percent

I/O scheduler and NVMe tuning

On older SATA drives the I/O‑scheduler was a lever; with modern NVMe solutions the controller dominates. Still, settings can be improved on many systems:

# Beispiel: Scheduler setzen (Device anpassen)
echo mq-deadline | sudo tee /sys/block/nvme0n1/queue/scheduler
# Queue depth anpassen (vorsichtig testen)
echo 128 | sudo tee /sys/block/nvme0n1/queue/nr_requests

BFQ (Budget Fair Queueing) can help latency-sensitive workloads under mixed load, but it is not always available. Test changes under load.

Kubernetes: Deep Dive and practical How‑to

Kubernetes has historically treated swap RESTrictively. From certain K8s versions onwards, operation with swap is possible, but only if Kubelet is explicitly configured for it. Key points:

• kubelet does not start by default when Swap is active (the parameter –fail-swap-on=false allows operation with swap).
• cgroup v2 provides memory.swap.max: you can set Memory and swap limits for a cgroup separately. That enables granular policies for DB‑Pods.
• DB‑Pods should ideally have Requests == Limits (QoS Guaranteed). That makes memory behavior predictable.

Example: set the Kubelet flag (systemd Unit Edit) and RESTart the Node:

# systemd Kubelet override (Beispielpfad kann variieren)
sudo mkdir -p /etc/systemd/system/kubelet.service.d
cat <<'EOF' | sudo tee /etc/systemd/system/kubelet.service.d/10-swap.conf
[Service]
Environment="KUBELET_EXTRA_ARGS=--fail-swap-on=false"
EOF
sudo systemctl daemon-reload
sudo systemctl RESTart kubelet

cgroup v2 example: set memory.swap.max in a cgroup (caution: affects the system globally):

# cgroup Pfad anpassen; Beispiel für system.slice
echo 1073741824 | sudo tee /sys/fs/cgroup/system.slice/memory.swap.max  # 1 GiB

In Kubernetes you can use Pod‑QoS and StatefulSet‑specifications to ensure DB‑Pods do not swap unpredictably. Document node policy and backup/failover behavior in your runbook.

Runbook: Stepwise implementation with metric gates

A safe approach reduces risk. Use this stepwise checklist:

1. Baseline: collect 24–72h of metrics (DB‑metrics + Node‑metrics).
2. Configuration change 1: set vm.swappiness to 10; observe for 1–2 maintenance windows.
3. Configuration change 2: enable zswap (boot parameter) with lz4 and max_pool_percent 10–15; observe for 1–2 maintenance windows.
4. Storage: if possible, move swap to a separate NVMe or configure QoS.
5. Production test: perform a simulated peak (workload replay) and check metric gates.
6. Live rollout: staggered rollout across host groups (Canary Nodes) with automatic rollback on exceeding predefined alert thresholds.

Example metric gates (implementable as AlertRules): Wenn innerhalb 5 Minuten sowohl (a) Swap‑In > 0 KB/s konstant, (b) iowait > 20 %, und (c) 99‑Perzentil DB‑Latenz > 2x baseline → Rollback oder Node drainen.

Rollback commands and emergency measures

In case of failure you must roll back quickly and safely. Here are the most important commands:

# Swappiness zurücksetzen
sudo sysctl -w vm.swappiness=60
sudo rm /etc/sysctl.d/99-db-swap.conf || true
sudo sysctl --system

# zswap deaktivieren: Kernel-Parameter entfernen und Reboot (sauberer Weg)
# Achtung: reboot in Wartungsfenster oder redundante Hosts verwenden
# Swap temporär deaktivieren
sudo swapoff -a
# Swap wieder aktivieren nach Korrektur
sudo swapon -a

Note: Immediate swapoff can trigger OOMs if RAM is insufficient. Use swapoff only when you are sure memory is available or workloads are stopped.

Typical troubleshooting scenarios

Three real cases and how to approach them:

Case 1: Sudden latency after nightly backup

Cause: Backup process temporarily consumes RAM, kernel starts swapping. Solution: throttle the backup, set limits, enable zswap, run swap on a separate device.

Case 2: DB pod in Kubernetes swaps despite requests

Cause: Inconsistent node policy (nodes with and without swap), kubelet with –fail-swap-on=false not uniform. Solution: consistent node policy, adjust pod QoS, set up alerts.

Case 3: Swap I/O competes with WAL on NVMe

Cause: Shared device queue interference. Solution: isolate swap or test BFQ/I/O‑QoS; if necessary, provision a separate NVMe or storage LUN.

Conclusion

Optimizing swap performance for databases is a balancing act between robust fault tolerance and minimal latency. With a measurement-based approach, conservative swappiness, targeted use of zswap and a clear storage strategy you achieve controlled degradation instead of unpredictable failures. In Kubernetes, you also need consistent node policies and strict requests/limits to make behavior reproducible. Implement automated tests and clear rollback gates, and treat swap tuning as an operational feature with maintenance windows, observability and documented runbooks.

Architecture, virtualization and operational aspects that are often overlooked

When optimizing swap performance for databases, the work does not end with swappiness or zswap. Many problems arise at the interface between architecture, virtualization and storage topology. The following practical points help uncover hidden risks and establish operational reliability.

NUMA, process placement and remote-memory penalties

On multicore systems with NUMA (Non-Uniform Memory Access), swapping across NUMA boundaries can lead to dramatic latency increases: a page located on a different node generates not only swap I/O but also remote memory accesses. This is particularly true for large, monolithic DB processes.

# Check NUMA topology
numactl --hardware
# Bind DB process to local node (example)
numactl --cpunodebind=0 --membind=0 /usr/lib/postgresql/…

Recommendation: For DB hosts verify whether processes and their memory allocations reside on the same NUMA node. In many cases static CPU/memory binding and HugePages improve predictability under memory pressure.

Swap in virtualized environments: ballooning and host swap

Virtual machines often mask the actual cause of memory pressure. The KVM/virt-balloon drives memory allocation dynamically — which can lead to the hypervisor itself starting to swap. Check the actual memory status on both the hypervisor and guest sides.

# Domain memory statistics (libvirt)
virsh dommemstat 
# On the hypervisor check whether swapping is occurring
sudo cat /proc/meminfo | grep -E 'Swap|MemAvailable'
sudo iostat -xz 1 3

If hypervisor-side swapping is detected, solutions are: increase RAM, disable ballooning for critical VMs or place workloads on dedicated hosts.

Storage topology: queue isolation, NVMe multipathing and SWAP placement

Placing swap on the same device as WAL or data files leads to queue interference. Strategies:

• Physically separate NVMe/LUN for swap, ideally with its own controller queues.
• If separation is not possible: test I/O QoS (BFQ, cgroup-io) to prioritize WAL I/O.
• For NVMe multipathing, verify that the multipath configuration is evenly distributed (round-robin) and queue-friendly.

Wear, lifetime and operating costs for SSD/NVMe

Swap generates many write cycles. Over time this affects the remaining lifespan of consumer SSDs. Operationalize SMART monitoring and device telemetry:

# SMART-Report für NVMe/SSD
sudo smartctl -a /dev/nvme0n1
# Device-IO-Bytes überwachen (Prometheus node exporter oder iostat)
iostat -x 1 3

For production DB hosts, prefer enterprise NVMe with higher DWPD, sufficient overprovisioning and a swap strategy that minimizes writes (zswap, compression profiling, write limiting during backups).

Automatic alerting and action logic (Prometheus example)

Automatic actions should not react solely to swap usage. Combine indicators: swap-in rate, disk I/O wait time and DB 99th-percentile latency. Example Prometheus AlertRule:

groups:
- name: swap.rules
  rules:
  - alert: SwapWithHighIOWait
    expr: |
      increase(node_vmstat_pswpin[5m]) > 0
      and (avg by(instance)(irate(node_cpu_seconds_total{mode="iowait"}[1m])) > 0.2)
    for: 2m
    labels:
      severity: critical
    annotations:
      summary: "Swap activity with high iowait on {{ $labels.instance }}"
      description: "Sustained swap-in combined with elevated iowait. Consider draining node or triggering failover."

Link alerts to playbooks: on trigger automatically start a canary drain or send a webhook to a runbook orchestrator tool, not direct automatic reboots.

Integration note: backup, HA and failover policy

Decide whether swap degradation or fast failover better serves your RTO/RPO. In clustered architectures, a fast failover (short RESTart) can be the better option because prolonged swap degradation can cause replication or consistency issues.

Conclusion: Do not plan swap in isolation as a kernel parameter, but as part of the overall architecture. Document placement decisions (NUMA, VM policies, swap placement), automate metric gates and integrate hardware telemetry into your monitoring. That way swap becomes a controlled operational variable instead of a hidden failure source.

Swap governance, testing and lifecycle management

Technical measures are of little use if changes are not backed by change control, infrastructure code and real tests. Place kernel-cmdline, /etc/sysctl.d files and zswap settings in your Git-based IaC repo and enforce reviews. Run automated CI jobs that validate configuration syntax, kernel modules and a short smoke test under a staging NUMA topology.

Operationalize rollouts: canary groups (e.g., 5–10% of the DB hosts), automatic metric gates and an orchestrated drain playbook reduce risk. Add procurement rules: NVMe spec (DWPD, overprovisioning) and replacement cycles must account for swap loads, especially for custom enterprise software with strict latency SLAs. Also document escalation paths, maintenance windows and responsibilities in the runbook so fast, controlled rollbacks are possible.

For this topic, Linux swap and swap on SSD are also important. The article places these aspects into context in a comprehensible way and shows what matters in day-to-day operations.