Kernel tuning for databases: THP, vm.swappiness and I/O scheduler in practice

For stable production databases, targeted Kernel tuning for databases is often more effective than pure scaling up. Three kernel controls influence performance and latency particularly strongly: Transparent Hugepages (THP), vm.swappiness and the I/O‑Scheduler. This practical guide explains how these mechanisms work, what impact they have on operations, monitoring and security, how to measure changes, roll them out safely and roll them back — including specific notes for containers (Docker/Podman) and virtualization.

Kernel tuning for databases: Before tuning: goals, metrics and change plan

Before you change kernel parameters, define measurable goals (e.g. p95/p99 reduction, more stable fsync times) and acceptable risks. Set KPIs you can collect automatically: query latencies from the database, host-side iostat/iowait, vmstat (si/so = Swap-In/Swap-Out), PSI (Pressure Stall Information — a kernel mechanism for measuring CPU/memory/IO pressure), and kernel logs (dmesg/journal). Without clear metrics, drawing conclusions is difficult.

Quick baseline checks

# Systemübersicht
uname -r; cat /etc/os-release; systemd-detect-virt || true

# Speicher, Swap und PSI
free -h; swapon --show; vmstat 1 5; sudo cat /proc/pressure/memory

# I/O
lsblk -o NAME,TYPE,SIZE,ROTA,MOUNTPOINTS
iostat -xz 1 5 || true

# THP-Status
for f in /sys/kernel/mm/transparent_hugepage/enabled /sys/kernel/mm/transparent_hugepage/defrag; do
  [ -f "$f" ] && echo "$f: $(cat $f)"
done

Document version information (kernel, distribution, storage drivers), because behavior can vary between kernel versions.

Transparent Hugepages (THP): mechanics, risks and practice

THP attempts to reduce TLB misses (TLB = Translation Lookaside Buffer, a cache for virtual→physical addresses) by aggregating many small pages into large ones (typically 2 MiB). That helps numerically-intensive workloads, but can cause unexpected pauses in transactional databases: kernel defragmentation and copy operations when merging large pages cause short CPU or memory stalls that can significantly increase p99 latencies.

When THP is likely to cause problems

• Sudden p99 spikes without an identifiable I/O trigger.
• Kernel defrag or kswapd activity under high memory utilization.
• Reproducible latency spikes with certain query profiles.

Testing, disabling and persistence

Changes can be tested at runtime via sysfs; in production, persistent, roll-backable mechanisms should be used (systemd unit, GRUB command line or initramfs modification).

# Laufzeit-Test: THP deaktivieren
echo never | sudo tee /sys/kernel/mm/transparent_hugepage/enabled
echo never | sudo tee /sys/kernel/mm/transparent_hugepage/defrag

# Kontrolle
cat /sys/kernel/mm/transparent_hugepage/enabled
cat /sys/kernel/mm/transparent_hugepage/defrag

Example of a short, safe systemd unit that disables THP at boot. Create the unit file with the following content:

sudo tee /etc/systemd/system/disable-thp.service >/dev/null <<'EOF'
[Unit]
Description=Disable Transparent Huge Pages
After=network.target

[Service]
Type=oneshot
ExecStart=/bin/sh -c 'echo never > /sys/kernel/mm/transparent_hugepage/enabled'
ExecStart=/bin/sh -c 'echo never > /sys/kernel/mm/transparent_hugepage/defrag'
RemainAfterExit=yes

[Install]
WantedBy=multi-user.target
EOF

sudo systemctl daemon-reload
sudo systemctl enable --now disable-thp.service

Alternatively, you can set the kernel parameter transparent_hugepage=never in GRUB (see the next section). Important: test across multiple load profiles and measure p95/p99 before and after the change.

vm.swappiness: Understanding, tuning and risks

vm.swappiness is a kernel parameter (0–100) that describes how aggressively the kernel swaps anonymous memory to disk. A lower value reduces swapping under normal operation — often sensible for database hosts, because swap latencies can severely slow down queries. Still, not automatically vm.swappiness=0 is the best choice: on systems with limited RAM or mixed workloads, aggressively avoiding swap can increase page reclaim activity, which raises CPU and I/O load.

Setzen und persistent machen

# Aktuellen Wert prüfen
sysctl vm.swappiness; cat /proc/sys/vm/swappiness

# Laufzeit setzen
sudo sysctl -w vm.swappiness=10

# Dauerhaft (sysctl.d)
sudo tee /etc/sysctl.d/99-db-tuning.conf >/dev/null <<'EOF'
# DB-Tuning: Swap-Verhalten
vm.swappiness = 10
EOF
sudo sysctl --system

Besonderheiten bei Containern und cgroups

In container environments inappropriate swapping is often a secondary effect: a container can hit its cgroup limit even though the host shows free RAM. cgroup v1 and v2 differ in parameters and behavior; relevant settings include memory.swap.max and memory.high (cgroup v2). Orchestrators (Kubernetes, Docker Engine) sometimes apply defaults that affect container behavior.

# Docker: Memory-Limits prüfen und Container-Startbeispiel
docker inspect --format '{{.Name}}: Memory={{.HostConfig.Memory}} MemorySwap={{.HostConfig.MemorySwap}}' postgres

# Starten ohne zusätzliches Swap (Host verhindert swap usage durch MemorySwap)
docker run -d --name postgres 
  --memory=8g --memory-swap=8g 
  -v /srv/dbdata:/var/lib/postgresql/data 
  my-postgres-image

Recommendation: set container and host limits consistently, test OOM behavior (kernel OOM or systemd-oomd), and monitor dmesg/journal for OOM or OOM-killer events.

I/O‑Scheduler: Auswahl nach Storage-Typ und Schleifenfallen

The I/O scheduler affects ordering, prioritization and queueing of block I/O. Modern Linux kernels use Multi-Queue (blk-mq). Typical rules:

• Local NVMe: none (device-side scheduler is often better than Linux queuing).
• Virtual disks (VirtIO, VMware): mq-deadline reduces latency outliers.
• HDDs or interactive desktop workloads: bfq can be beneficial for fairness and latency sensitivity.

Important: for layers such as LVM, device-mapper (dm‑crypt) or Multipath you must set the scheduler at the correct layer — often on the physical device, not on the LVM device.

Prüfen, setzen und persistieren

# Current schedulers
for d in /sys/block/*/queue/scheduler; do
  echo "${d%/queue/scheduler}: $(cat $d)"
done

# Runtime change (example for nvme0n1)
echo none | sudo tee /sys/block/nvme0n1/queue/scheduler

# Persistence with udev rule
sudo tee /etc/udev/rules.d/60-io-scheduler.rules >/dev/null <<'EOF'
ACTION=="add|change", KERNEL=="nvme*", ATTR{queue/scheduler}="none"
ACTION=="add|change", KERNEL=="sd[a-z]", ATTR{queue/scheduler}="mq-deadline"
EOF
sudo udevadm control --reload-rules; sudo udevadm trigger --type=devices --action=change

Note cloud instances (e.g., AWS EBS gp3, Azure Managed Disks): the underlying behavior can be virtualized; test with real workload. For multipath: set schedulers on the backend devices, not only on /dev/mapper/…

Measurement methodology: reliable tests and tools

Kernel tuning only makes sense if you measure its effect cleanly. Use combined host and DB tests:

• DB workload tools: pgbench for PostgreSQL, sysbench for MySQL, native benchmark suites. These produce more realistic access patterns than synthetic I/O tools alone.
• Host tools: iostat (await, %util), blktrace for deep block I/O analysis, fio for controlled I/O profiles.
• Kernel observability: /proc/pressure/* for PSI, eBPF profiling (bcc, bpftrace) to make stalls and queues visible.

# Example: fio for fsync stress (synchronous write + fsync)
fio --name=fsync-test --filename=/srv/dbdata/testfile --size=4G 
  --bs=4k --iodepth=1 --numjobs=1 --rw=write --direct=1 --fsync=1 --runtime=300

# Simplified pgbench run example
docker exec -it postgres pgbench -c 50 -T 120 -r mydb

Important: Run tests over a representative time window (not just briefly) to capture p99 behavior.

Troubleshooting: Why tuning has no effect

If changes do not bring improvement, check:

• Is the database file actually on the tuned device? (bind mounts, LVM, dm-crypt, multipath).
• Does latency originate from the storage array or network (NAS/SAN/cloud)?
• Is CPU time consumed inside the DB itself (locking, checkpoints) rather than in the host I/O path?
• Are container cgroups, systemd-oomd, or orchestrator policies the cause?

Often kernel tuning is only part of a broader set of measures: storage optimization, tuning DB parameters (checkpoint intervals, background writer) and I/O path analysis should be considered together.

Rollback strategy and example script

Always prepare automatable rollbacks. Save current values before changes and provide a simple restoration.

#!/bin/bash
# rollback-tuning.sh - simple rollback routine
set -euo pipefail

echo "Saving current values..."
mkdir -p /root/kernel-tuning-backup
cat /sys/kernel/mm/transparent_hugepage/enabled > /root/kernel-tuning-backup/thp_enabled
cat /sys/kernel/mm/transparent_hugepage/defrag > /root/kernel-tuning-backup/thp_defrag
cat /proc/sys/vm/swappiness > /root/kernel-tuning-backup/swappiness
for d in /sys/block/*/queue/scheduler; do
  dev=${d%/queue/scheduler}
  cat $d > "/root/kernel-tuning-backup/$(basename $dev)_scheduler"
done

# Rollback steps (read example values from backup)
echo "Rollback: reset THP"
[ -f /root/kernel-tuning-backup/thp_enabled ] && cat /root/kernel-tuning-backup/thp_enabled | sudo tee /sys/kernel/mm/transparent_hugepage/enabled
[ -f /root/kernel-tuning-backup/thp_defrag ] && cat /root/kernel-tuning-backup/thp_defrag | sudo tee /sys/kernel/mm/transparent_hugepage/defrag

echo "Rollback: reset swappiness"
[ -f /root/kernel-tuning-backup/swappiness ] && sudo sysctl -w vm.swappiness=$(cat /root/kernel-tuning-backup/swappiness)

# Note: Scheduler rollback may require a reboot, depending on udev/driver

echo "Rollback completed - check logs and KPIs"

Document the original values in your ticketing system and perform a validation check of the KPIs after the rollback.

Special focus: Docker and cgroup v2

In container scenarios: most kernel settings take effect at the host level; container privileges and cgroup configuration can, however, alter behavior. Under cgroup v2 the relevant settings under /sys/fs/cgroup are: memory.high (soft limit), memory.swap.max (swap limit) and memory.max (hard limit). Kubernetes uses QoS classes (Guaranteed/Burstable/BestEffort) — set requests and limits cleanly so containers operate predictably.

# Example: check values in cgroup v2 (path may vary by engine)
cat /sys/fs/cgroup//memory.high
cat /sys/fs/cgroup//memory.swap.max

# Kubernetes: PodSpec (excerpt)
# ensure requests/limits are defined to avoid unexpected swap

Create test cases for OOM scenarios and verify how your orchestrator policies (Eviction thresholds, PodDisruptionBudget) respond.

Practical operational rules and recommendations

• Change only one kernel variable per test window so effects are clearly attributable.
• Automate backups of previous values and rollback scripts.
• For cloud or SAN storage, first check whether the latency source is outside the host.
• Use eBPF tools to make short kernel stalls visible that monitoring tools often miss.
• Keep persistence rules (sysctl.d, udev, systemd) versioned in the configuration repository.

Conclusion

Kernel tuning for databases is effective but precise work: Transparent Hugepages are a frequent cause of p99 spikes, vm.swappiness must align with RAM planning and cgroup policy, and the I/O scheduler should be chosen according to the storage type. Work methodically: baseline, targeted change, measurement, persistence test (reboot) and a clear rollback plan. Only then will you achieve reproducible improvements in bare-metal, VMs and container environments.

This runbook is intended as an operational reference: adapt values to your infrastructure, test with real workloads and record all steps with auditability in your change management.

Operational, upgrade and integration aspects

In addition to targeted parameter adjustments it is worthwhile to embed kernel tuning into an operational context: kernel and driver upgrades, NVMe/firmware levels, NUMA topology and database configurations interact strongly and can mask or amplify changes. Before a broad rollout, verify compatibility with storage drivers (nvme, virtio, multipath) and firmware releases; some latency changes are driver- or firmware-related and cannot be remedied via sysctl.

Architecture notes

• NUMA: Place DB threads, page cache and data files preferably on the same NUMA domain; incorrect affinity increases cross-node traffic.
• DB configuration: Tune shared_buffers/innodb_buffer_pool so host page cache and DB cache do not conflict; this reduces unnecessary reclaims.
• Cloud/Managed DB: With managed services kernel settings are often not available — instead choose appropriate instance classes, Provisioned IOPS or specialized storage options.

Operations and CI integration

Automate tests in your pipeline: run fio/pgbench as part of release pipelines on a small canary fleet and integrate result comparison (p50/p95/p99) into CI reporting. Version sysctl.d, udev rules and systemd units in the configuration repository and deploy idempotently via Ansible/Chef.

Monitoring checklist

• Alert for PSI‑Memory > defined threshold (e.g., 100ms) and for sustained swap-outs.
• Trend monitoring: p99 latencies, iowait, await and kernel OOM events.
• Audit: check kernel parameters and udev rules via configuration drift scans.

These operational perspectives ensure that kernel tuning remains part of a reliable change-management process for your individual enterprise software landscape and does not cause unexpected side effects.

For this topic, disabling Transparent Hugepages and the I/O schedulers mq-deadline, bfq and none are also important. The article places these aspects in context and shows what matters in day-to-day operation.