Web performance

QEMU VM Memory Tiering: How Red Hat Plans to Save DRAM Using NVMe SSDs and MADV_PAGEOUT

Presented by Red Hat's Peter Xu at KVM Forum 2026 and covered by Phoronix on November 14, 2026, QEMU VM memory tiering aims to offload cold guest pages onto NVMe SSDs without tanking live migration performance. Explore the three competing designs, why mincore() + MADV_PAGEOUT leads, and how to audit swap/MGLRU behavior today.

By 0xScorPionSon

8 min read

QEMU VM Memory Tiering: How Red Hat Plans to Save DRAM Using NVMe SSDs and MADV_PAGEOUT

QEMU Memory Tiering DRAM and NVMe SSD cold tier architecture with MADV_PAGEOUT

Executive Summary

At KVM Forum 2026 in Vancouver, Canada (November 12, 2026), Red Hat virtualization engineer Peter Xu presented QEMU memory tiering, outlining how hypervisors can leverage fast NVMe SSDs as a secondary memory tier to reduce expensive DRAM consumption for QEMU/KVM virtual machines. As reported by Phoronix on November 14, 2026, the effort is driven directly by high DRAM pricing—pushing cloud and enterprise virtualization teams to increase VM density without sacrificing workload responsiveness.

Because Linux has maintained a mature two-tier memory hierarchy via swap and memcg (memory cgroups) for decades, QEMU can already offload cold guest memory onto NVMe-backed swap and benefit automatically from kernel advances such as Multi-Gen LRU (MGLRU). However, Xu's KVM Forum 2026 abstract highlights a critical catch: because QEMU itself is currently tier-unaware, operations like live migration can trigger severe performance regressions when touching paged-out cold memory.

To solve this, Red Hat is evaluating three architectural paths: introducing a new maccess() system call, implementing full user-space tiering inside QEMU, or combining existing Linux APIs (mincore() + MADV_PAGEOUT) with a targeted kernel fix for asynchronous reclaim swap overshoot. Right now, the existing-kernel-API approach (mincore() + MADV_PAGEOUT) is the lead contender because it requires minimal QEMU changes and avoids duplicating kernel page-tracking overhead. Public latency and throughput numbers from Xu's talk slides are not included in the published abstracts, so quantitative benchmarks remain unavailable until upstream patches or slide decks are posted.


Why Naive Linux Swap Hurts Tier-Unaware QEMU VMs

Using Linux's existing swap subsystem and memcg per-VM limits appears deceptively simple: the kernel tracks page hotness via MGLRU, demotes cold guest pages to an NVMe SSD, and faults them back into DRAM when the guest touches them.

The breakdown happens when QEMU's user-space hypervisor threads—not the guest OS—scan guest memory:

  • Live Migration & Bulk Memory Scans: During live migration pre-copy, QEMU reads every guest physical page to transmit it over the network. If QEMU is tier-unaware, reading a cold page that the kernel previously swapped out to NVMe forces a synchronous page-in into DRAM—thrashing the host's page cache, spiking memcg pressure, and triggering swap overshoot as the kernel races to reclaim memory back out.
  • Duplicated Hotness Tracking: Conversely, if QEMU tries to manage the slow tier entirely in user space using userfaultfd and idle-page tracking, both QEMU and the Linux kernel end up tracking page hotness independently, creating CPU overhead and conflicts (for example, MGLRU may not yet work cleanly alongside legacy page-idle tracking).

Technical Mechanics & Architecture

Comparing Red Hat's Three QEMU Memory Tiering Approaches

Per Peter Xu's KVM Forum 2026 session and Phoronix's November 14, 2026 report, Red Hat is weighing three distinct designs:

Approach Mechanism Kernel Changes Needed Key Trade-Offs
1. New maccess() Syscall Introduces a new Linux syscall for tier-aware user-space memory access High (new syscall ABI + MM plumbing) Clean semantics for hypervisors, but requires long upstream MM review
2. Full User-Space Tiering QEMU owns the slow NVMe tier via userfaultfd + idle page tracking Low/None (uses existing userfaultfd) Duplicates kernel hotness tracking; conflicts with MGLRU
3. mincore() + MADV_PAGEOUT (Lead) QEMU queries residency with mincore(), migrates, and hints reclaim via MADV_PAGEOUT None initially; small optional patch for swap overshoot Avoids duplicated tracking; leverages MGLRU & memcg
+-----------------------------------------------------------------------+
|                    QEMU / KVM HOST (LINUX + MGLRU + MEMCG)            |
|                                                                       |
|  [ QEMU Guest VM Process ]                                            |
|   * Live Migration / Memory Scan                                      |
|   * Checks page residency via mincore()                               |
|   * Hints immediate reclaim via madvise(..., MADV_PAGEOUT)            |
+-------------------+-------------------------------+-------------------+
                    |                               |
          Hot Pages |                               | Cold Pages
          (In RAM)  v                               v (Reclaimed)
+-------------------------------+       +-------------------------------+
|       DRAM (FAST TIER)        |       |     NVME SSD (SLOW TIER)      |
|  * Active guest working set   |       |  * High-speed NVMe swap/tier  |
|  * Managed by MGLRU + memcg   |       |  * Avoids swap overshoot via  |
|  * Low-latency vCPU access    |       |    targeted MADV_PAGEOUT fix  |
+-------------------------------+       +-------------------------------+

How mincore() + MADV_PAGEOUT Fixes Live Migration

As documented in the Linux madvise(2) manual page (and process_madvise(2)), MADV_PAGEOUT (introduced in Linux 5.4) instructs the kernel to immediately reclaim a specified range of pages, freeing their DRAM frames while preserving their contents in swap/backing storage.

In Red Hat's lead contender workflow:

  1. Residency Check (mincore()): Before touching or migrating a guest memory range, QEMU checks which pages are currently resident in DRAM versus already swapped out to the NVMe slow tier.
  2. Controlled Reclaim (MADV_PAGEOUT): After QEMU reads and transmits a cold page during migration, it can issue madvise(..., MADV_PAGEOUT) so the page does not linger in DRAM and inflate the host's active working set.
  3. Fixing Async Reclaim Swap Overshoot: Xu noted in his KVM Forum 2026 abstract that mincore() + MADV_PAGEOUT eliminates most migration-induced performance degradation, and that a relatively small kernel patch can eliminate the remaining issue—specifically swap overshoot caused by asynchronous reclaim of MADV_PAGEOUT.

Integration With memcg, MGLRU, and zswap

Because the lead contender keeps the Linux kernel in charge of page-hotness tracking, virtualization operators retain full compatibility with modern kernel memory-management features:

  • Per-VM Isolation via memcg: Placing each QEMU instance in its own cgroup v2 (memory.high / memory.max / memory.swap.max) gives hypervisor schedulers deterministic per-VM DRAM vs. NVMe tier budgets.
  • MGLRU Page Aging: Multi-Gen LRU continues to age cold anonymous guest pages without interference from user-space idle-page-tracking bit-clearing.
  • Optional Compression Layer (zswap): Hosts with fast hardware compression (such as Intel IAA via iaa_crypto documented in Linux Kernel Docs) can even interpose compressed DRAM before spilling to NVMe swap.
  • Avoiding MADV_PAGEOUT Pitfalls on Special Mappings: Per madvise(2), memory-advice operations must respect special VM mappings (VM_IO, VM_PFNMAP, or memfd_secret(2) regions) and mlock(2) boundaries. In virtualized hosts using VFIO PCI passthrough (where guest DMA pages are pinned in DRAM), QEMU must cleanly skip pinned ranges so mincore() and MADV_PAGEOUT only target pageable anonymous guest RAM.

Evidence & Readiness Assessment

  • Problem & Contender Status (High Confidence): Peter Xu's KVM Forum 2026 talk and Phoronix's November 14, 2026 coverage confirm that mincore() + MADV_PAGEOUT is currently the lead contender, requiring no mandatory kernel changes initially and small QEMU changes, with a follow-up kernel patch under exploration for MADV_PAGEOUT async reclaim swap overshoot.
  • Open Upstream Design Decision (Moderate Confidence): Xu explicitly notes in his abstract that "It is still an open question on how QEMU should implement its tiering functionality," meaning upstream QEMU mailing-list consensus will determine the final patch series.
  • Quantitative Benchmarks (Unavailable): Although Xu's talk abstract mentions sharing numbers to illustrate live-migration slowdowns and mincore() + MADV_PAGEOUT improvements, those specific figures are not published in the public Pretalx abstract or Phoronix summary.

Hypervisor Memory Tiering Audit & Tuning Playbook

For KVM/QEMU operators and Linux systems engineers preparing NVMe-backed memory tiering hosts, use this five-step operational checklist today:

  1. Verify MGLRU and cgroup v2 Readiness: Confirm Multi-Gen LRU (/sys/kernel/mm/lru_gen/enabled) is active on your Linux host and that QEMU VMs run inside isolated cgroup v2 slices with explicit memory.high and memory.swap.max limits.
  2. Benchmark NVMe Swap Latency & Queue Depth: Tiered cold memory is only viable on low-latency enterprise NVMe SSDs; avoid spinning rust or network-attached block storage for VM swap backing.
  3. Monitor pswpin / pswpout During Live Migration: Track /proc/vmstat (pswpin, pswpout) and per-cgroup memory.stat (pgfault, pgmajfault) during virsh migrate tests to spot tier-unaware migration thrashing early.
  4. Watch MADV_PAGEOUT & process_madvise(2) Capabilities: Remember that process_madvise(2) on external processes requires PTRACE_MODE_READ_FSCREDS and CAP_SYS_NICE (process_madvise(2)), whereas in-process QEMU madvise(2) requires no extra privileges.
  5. Maintain Safe Rollback Controls: Keep live-migration bandwidth caps (--bandwidth) and memory.swap.max = 0 overrides ready in libvirt/systemd unit templates for latency-SLA VMs that must remain pinned in DRAM.
  6. Avoid Mixing MGLRU With Legacy Idle Page Tracking: If you benchmark user-space tiering prototypes that rely on /sys/kernel/mm/page_idle/bitmap, note Xu's warning that MGLRU may conflict with idle-page tracking; keep production hosts on kernel-managed MGLRU + memcg until mincore() + MADV_PAGEOUT patches land upstream.

Below is a safe, read-only Python diagnostic script for Linux KVM hosts to inspect MGLRU status, vmstat swap activity, and cgroup v2 memory/swap limits for running qemu-system processes:

#!/usr/bin/env python3
'''Read-only audit of MGLRU, swap counters, and QEMU cgroup v2 memory limits.'''
from pathlib import Path

def read_text(path: Path, default: str = "N/A") -> str:
    try:
        return path.read_text().strip()
    except OSError:
        return default

def main() -> None:
    mglru = read_text(Path("/sys/kernel/mm/lru_gen/enabled"))
    print(f"MGLRU (/sys/kernel/mm/lru_gen/enabled): {mglru}")

    vmstat = Path("/proc/vmstat").read_text().splitlines()
    for key in ("pswpin", "pswpout", "pgmajfault"):
        match = [line for line in vmstat if line.startswith(key + " ")]
        print(match[0] if match else f"{key} N/A")

    cgroup_root = Path("/sys/fs/cgroup")
    for mem_curr in sorted(cgroup_root.rglob("memory.current"))[:10]:
        cg_dir = mem_curr.parent
        if "qemu" in cg_dir.name.lower() or "machine" in str(cg_dir).lower():
            sw_curr = read_text(cg_dir / "memory.swap.current")
            sw_max = read_text(cg_dir / "memory.swap.max")
            print(f"{cg_dir.name}: mem={mem_curr.read_text().strip()}B | swap={sw_curr}B / max={sw_max}")

if __name__ == "__main__":
    main()

Frequently Asked Questions (FAQ)

Why is Red Hat working on QEMU VM memory tiering?

With DRAM prices remaining high, Red Hat engineers are exploring ways for QEMU/KVM VMs to use cheaper, fast NVMe SSDs as a secondary cold-memory tier to conserve DRAM while maintaining acceptable performance (Phoronix).

Why does standard Linux swap cause issues for QEMU live migration?

Because QEMU is currently tier-unaware, operations like live migration scan all guest memory pages, pulling swapped-out cold pages back into DRAM and causing severe performance regressions and swap overshoot (KVM Forum 2026).

Which QEMU memory tiering approach is currently leading?

According to Phoronix, using existing Linux kernel APIs (mincore() + MADV_PAGEOUT) is the lead contender because it requires small QEMU changes, no mandatory kernel changes initially, and can be further improved with a small kernel patch for MADV_PAGEOUT async reclaim swap overshoot.

What are the other two approaches Red Hat evaluated?

Red Hat also evaluated adding a new maccess() system call for tier-aware user-space memory access, and implementing full user-space tiering inside QEMU using userfaultfd and idle page tracking—though user-space tiering duplicates kernel work and can conflict with MGLRU (KVM Forum 2026, Phoronix).

Are benchmark numbers available for QEMU mincore() + MADV_PAGEOUT tiering?

Not publicly in the written summaries. While Peter Xu's KVM Forum 2026 talk included slides comparing approaches, exact benchmark numbers are not available in the published Pretalx abstract or Phoronix article.


Sources

  1. Phoronix — Red Hat Working To Improve Memory Tiering For QEMU VMs To Save RAM (November 14, 2026)
  2. KVM Forum 2026 — QEMU memory tiering, Peter Xu (November 12, 2026) (Speaker Abstract)
  3. Linux Manual Page — madvise(2) (MADV_PAGEOUT)
  4. Linux Manual Page — process_madvise(2)

More on Web performance

Tell us what is not working

A thirty-minute call, no deck and no obligation. You will leave with a straight answer about whether this is worth building - including if that answer is no.

Start a project

Direct line

hello@codebumble.net
Dhaka, Bangladesh  :   local timeCalls in English or Bangla