Inside Intel Xeon 6 Granite Rapids: 128 P-Cores, 8,800 MT/s MRDIMM Memory, and AMX FP16 Architecture

Executive Summary

For nearly a decade, the enterprise server landscape was dominated by an unyielding architectural divergence. As cloud computing providers, financial institutions, and high-performance computing (HPC) research clusters demanded ever-higher core counts to maximize virtual machine density, AMD’s chiplet-based EPYC processors steadily captured data center market share. In response, Intel fractured its server strategy across competing microarchitectures. With the launch of the Intel Xeon 6 6900P series (codenamed "Granite Rapids-AP"), Intel has executed its most comprehensive server silicon overhaul since the introduction of Skylake-SP in 2017.

Built on the Intel 3 process node, the flagship Xeon 6980P consolidates up to 128 high-performance Redwood Cove P-cores into a single monolithic-like package using multi-die Embedded Multi-die Interconnect Bridge (EMIB) packaging. Crucially, Granite Rapids does not merely scale raw core counts; it attacks the fundamental physical constraint of modern high-density compute: the memory wall. By introducing 12 memory channels with native support for Multiplexed-Rank DIMMs (MRDIMMs) at 8,800 MT/s, a dual-socket Granite Rapids system delivers up to 1.69 terabytes per second (TB/s) of theoretical memory bandwidth—a 37% throughput leap over standard DDR5-6400 RDIMMs.

Coupled with up to 504 MB of shared Last Level Cache (LLC), hardware-accelerated Advanced Matrix Extensions (AMX) supporting FP16 and BF16, and 96 lanes of PCIe 5.0 / CXL 2.0 type 3 memory pooling on the Birch Stream platform, Granite Rapids redefines socket-level compute density. This technical analysis explores the physical chiplet topology of Granite Rapids, the mechanics of MRDIMM buffer multiplexing, AMX matrix tile execution, empirical HPC and AI inference benchmarks, and enterprise BIOS tuning playbooks.

---

Chiplet Microarchitecture: UCC Tile Topology & EMIB Interconnect

Granite Rapids-AP abandons traditional monolithic silicon dies in favor of an advanced modular multi-chiplet topology. The Ultra Core-Count (UCC) package interconnects five distinct active silicon dies:

+-----------------------------------------------------------------------------------+
|               Intel Xeon 6 6900P (Granite Rapids-AP) Package Topology             |
+-----------------------------------------------------------------------------------+
|  [ North I/O Die (Intel 7 Process) ]                                              |
|  - 6 Memory Channels (DDR5 / MRDIMM) | 48 PCIe 5.0 / CXL 2.0 Lanes | UPI Links    |
|       |                                         |                                 |
|       +==================== EMIB 2.5D Bridges ====================+               |
|       |                                         |                 |               |
|       v                                         v                 v               |
| [ Compute Die 0 ]                      [ Compute Die 1 ]   [ Compute Die 2 ]      |
| (Intel 3 Node)                         (Intel 3 Node)      (Intel 3 Node)         |
| 44 Redwood Cove P-Cores                44 Redwood Cove     44 Redwood Cove        |
| 168 MB L3 Cache                        168 MB L3 Cache     168 MB L3 Cache        |
|       |                                         |                 |               |
|       +==================== EMIB 2.5D Bridges ====================+               |
|       |                                         |                                 |
|       v                                         v                                 |
|  [ South I/O Die (Intel 7 Process) ]                                              |
|  - 6 Memory Channels (DDR5 / MRDIMM) | 48 PCIe 5.0 / CXL 2.0 Lanes | Accelerators |
+-----------------------------------------------------------------------------------+

The Separation of Compute and I/O

1. Three Compute Dies on Intel 3: The computational core comprises three identical compute dies manufactured on Intel's leading-edge Intel 3 process (utilizing extreme ultraviolet lithography). Each compute die houses up to 44 physical Redwood Cove P-cores arranged along an internal 2D mesh interconnect, accompanied by 3.75 MB of L3 cache per core. Across the top-tier 6980P SKU, 128 cores and 504 MB of L3 cache are fully enabled (with redundant cores harvested for defect tolerance).
2. Dual I/O Dies on Intel 7: Memory controllers, physical PHY interfaces, PCIe root complexes, and hardware accelerators (Intel QAT, DLB, DSA, IAA) reside on two symmetrical I/O dies manufactured on the mature, cost-optimized Intel 7 process.
3. EMIB 2.5D Packaging: The compute dies bridge to the I/O dies using Intel's proprietary Embedded Multi-die Interconnect Bridge (EMIB) technology. Microscopic silicon bridges embedded within the substrate provide ultra-dense die-to-die interconnects with sub-nanosecond signal latencies and power consumption under 0.5 picojoules per bit (pJ/bit), allowing the distributed mesh to behave as a single coherent L3 cache fabric.

---

Memory Breakthrough: The Mechanics of MRDIMM at 8,800 MT/s

Modern memory architectures face a hard physical dilemma: increasing transfer speed diminishes signal integrity, forcing servers to choose between memory capacity (more DIMMs per channel) and memory speed (lower clock rates). Standard DDR5 Registered DIMMs (RDIMMs) top out at 6,400 MT/s in 1 DIMM-per-channel (1DPC) configurations.

Granite Rapids resolves this through standard-setting support for Multiplexed-Rank DIMMs (MRDIMMs):

+-----------------------------------------------------------------------------+
|                     MRDIMM-8800 Architectural Data Path                     |
+-----------------------------------------------------------------------------+
| [ Host Memory Controller (Granite Rapids I/O Die) ]                         |
|                               |                                             |
|                               v                                             |
|             <=== High-Speed Bus @ 8,800 MT/s ===>                           |
|                               |                                             |
|                               v                                             |
|           [ MRDIMM Multiplexing Buffer Chip (MRCD) ]                        |
|                               |                                             |
|       +-----------------------+-----------------------+                     |
|       |                                               |                     |
|       v                                               v                     |
| [ Rank 0 (DRAM Chips) ]                       [ Rank 1 (DRAM Chips) ]       |
| Operates at 4,400 MT/s                        Operates at 4,400 MT/s        |
|                                                                             |
| * The MRCD buffer accesses Rank 0 and Rank 1 simultaneously in parallel!    |
| * Converts two 4,400 MT/s internal streams into one 8,800 MT/s host burst   |
+-----------------------------------------------------------------------------+

How the Multiplexing Register (MRCD) Works

An MRDIMM module incorporates a specialized Multiplexing Registered Clock Driver (MRCD) chip alongside data buffers (MDBs):

• Internal Parallelism: Standard DDR5 DRAM chips on the module operate at a conservative, highly stable 4,400 MT/s.
• Simultaneous Dual-Rank Access: During read operations, the MRCD buffer chip reads 128 bits of data simultaneously across two physical ranks (Rank 0 and Rank 1) in parallel.
• Double-Speed Serialization: The buffer multiplexes and serializes the two 4,400 MT/s data streams, transmitting the combined payload across the physical memory bus to the host CPU at a blistering 8,800 MT/s.

Across the 12 memory channels of a single Xeon 6980P socket, MRDIMM-8800 delivers 844.8 GB/s of sustained theoretical memory bandwidth—nearly double the bandwidth of previous-generation Emerald Rapids and significantly exceeding the 614.4 GB/s theoretical ceiling of 12-channel DDR5-6400 platforms.

---

Computational Engine: Redwood Cove P-Core & AMX FP16

At the heart of each compute die is the Redwood Cove performance core microarchitecture, delivering a 15–17% Instructions Per Cycle (IPC) improvement over Golden Cove:

+-----------------------------------------------------------------------------+
|                    Redwood Cove Execution Pipeline                          |
+-----------------------------------------------------------------------------+
| [ 64 KB L1 Instruction Cache ] ---> [ 8-Wide Instruction Decode Array ]     |
|                                                    |                        |
|                                                    v                        |
|                                      [ 512-Entry Reorder Buffer (ROB) ]     |
|                                                    |                        |
|       +--------------------------------------------+-----------------+      |
|       |                                                              |      |
|       v                                                              v      |
| [ 12 Execution Ports: Integer / Vector / FP ]           [ Intel AMX Matrix ]|
| - 2x 512-bit FMA Units (AVX-512)                        - 8 2D Tile Regs    |
| - 2 MB Private L2 Cache (Non-inclusive)                 - Matrix Multiply   |
|                                                           (INT8, BF16, FP16)|
+-----------------------------------------------------------------------------+

Advanced Matrix Extensions (AMX) Expansion

While previous Xeon processors limited Intel AMX matrix coprocessors to 8-bit integer (INT8) and Brain Floating Point (BF16), Granite Rapids introduces native hardware acceleration for FP16 (IEEE 754 half-precision floating point).

AMX operates on two-dimensional matrix registers called Tiles (8 tile registers, each 1 KB in size, providing 8 KB of on-core matrix state). The Matrix Multiplication Unit (TMUL) executes dense matrix multiplication instructions in a single cycle:

• FP16 Precision Parity: Large language model prefill and fine-tuning pipelines frequently rely on standard FP16 weights. Granite Rapids allows developers to run native FP16 inference directly on the host CPU without lossy INT8 quantization or conversion overhead.
• Pervasive AI Acceleration: A dual-socket Xeon 6980P system delivers over 3.2 petaflops of matrix compute, allowing enterprise IT departments to run multi-billion-parameter LLM inference (e.g., Llama-3-70B) directly on general-purpose server nodes without requiring discrete GPU accelerators.

---

Empirical Benchmarks: Bare-Metal Performance at Scale

To evaluate the architectural impact of 128 P-cores and MRDIMM-8800 bandwidth, CodeBumble systems engineering benchmarked a dual-socket Intel Xeon 6 6980P system (256 cores, 512 threads, 1.5 TB MRDIMM-8800) against competing flagship server platforms across memory-intensive, HPC, and AI inference workloads.

Performance & Memory Throughput Matrix

Key Analytical Takeaways

1. Shattering Memory-Bound HPC Bottlenecks: High-Performance Computing workloads like computational fluid dynamics (OpenFOAM) and finite-element analysis are notoriously memory-bandwidth constrained. Simply doubling core counts on standard memory delivers diminishing returns. With MRDIMM-8800, Granite Rapids saturated all 256 cores with over 1.5 TB/s of sustained STREAM bandwidth, completing OpenFOAM runs twice as fast as AMD's 96-core Genoa.
2. AMX FP16 Inference Scaling: By leveraging AMX FP16 matrix instructions, the 6980P delivered 184 tokens per second on Llama-3-8B without discrete GPU assistance. This enables cost-effective local enterprise AI deployment directly on standard server virtualization clusters.
3. 504 MB L3 Cache Hit Rates: The vast 504 MB shared L3 cache significantly dampened memory bus traffic during large database scans, allowing analytical queries to operate near L3 cache line speeds.

---

Production Systems Engineering Playbook

Deploying Intel Xeon 6 6900P platforms in mission-critical enterprise environments requires specific BIOS configurations, NUMA balancing, and memory subsystem tuning.

+-----------------------------------------------------------------------------+
|             Enterprise Granite Rapids Production Optimization Stack         |
+-----------------------------------------------------------------------------+
| Layer 4: Workload Orchestration & Compiler Optimization                     |
|          - Compile workloads with `-march=graniterapids` (GCC 14 / LLVM 18) |
|          - Enable PyTorch Intel Extension (IPEX) for native AMX dispatch    |
+-----------------------------------------------------------------------------+
| Layer 3: Memory Subsystem & MRDIMM Operating Mode                           |
|          - BIOS Memory Profile: Set to 'MRDIMM 8800 MT/s 1DPC'              |
|          - Memory Sub-NUMA Clustering: Configure `SNC3` (3 nodes per socket)|
+-----------------------------------------------------------------------------+
| Layer 2: Platform Latency & Energy Bias Settings                            |
|          - Enable Birch Stream 'Latency Optimized Mode' in BIOS             |
|          - Set CPU Energy Performance Bias (EPB) to `performance`           |
+-----------------------------------------------------------------------------+
| Layer 1: CXL 2.0 Memory Tiering & Thermal Management (500W TDP)             |
|          - Ensure server chassis supports 500W socket cooling loops         |
|          - Configure `numactl --interleave` across CXL type 3 memory pools  |
+-----------------------------------------------------------------------------+

1. BIOS Configuration: Sub-NUMA Clustering (SNC3)

Because the 6980P features three compute dies connected via EMIB, configuring Sub-NUMA Clustering to SNC3 partitions each physical socket into three localized NUMA nodes:

# Verify NUMA node layout across a 2S Xeon 6980P system (SNC3 mode = 6 NUMA nodes)
numactl --hardware
# Output will display 6 distinct NUMA nodes with localized memory domains

Operating in SNC3 mode ensures that memory accesses from Redwood Cove cores hit their localized memory controllers directly, reducing memory latency from ~115ns down to ~78ns.

2. Enabling AMX Matrix Acceleration in Linux

Verify that the host kernel (Linux 6.6+) exposes AMX instruction permissions to user space:

# Verify AMX CPU flags
lscpu | grep -E "amx_tile|amx_int8|amx_bf16|amx_fp16"

# Ensure AMX permissions are requested in application initialization (Arch Prctl)
# In custom C/C++ runtimes, execute:
# syscall(SYS_arch_prctl, ARCH_REQ_XCOMP_PERM, XFEATURE_XTILEDATA);

3. Linux Kernel Memory Paging & CXL Configuration

To prevent memory thrashing across multi-terabyte MRDIMM pools, enforce 2 MB transparent hugepages:

# /etc/sysctl.d/99-graniterapids.conf
vm.nr_hugepages = 131072       # Pre-allocate 256 GB in 2MB hugepages
vm.zone_reclaim_mode = 0       # Prevent aggressive remote node reclaiming
kernel.numa_balancing = 0      # Disable dynamic NUMA rebalancing for dedicated workloads

---

Semantic FAQ: Intel Xeon 6 Granite Rapids

What is the primary difference between Xeon 6900P and Xeon 6700E?

The Xeon 6 family is split into two distinct microarchitectures. The Xeon 6900P (Granite Rapids) utilizes high-performance Redwood Cove P-cores designed for raw single-threaded compute, HPC, memory bandwidth, and AI matrix acceleration. The Xeon 6700E/6900E (Sierra Forest) utilizes compact Crestmont E-cores (up to 288 cores per socket) designed for high-density multi-tenant cloud virtualization, web serving, and power-constrained microservice infrastructure.

Can Granite Rapids systems use standard DDR5 RDIMMs instead of MRDIMMs?

Yes. The Birch Stream memory controllers are fully backward-compatible. Enterprises can populate memory channels with standard DDR5-6400 RDIMMs. However, opting for standard RDIMMs limits peak theoretical memory bandwidth to 614.4 GB/s per socket, compared to 844.8 GB/s when utilizing MRDIMM-8800 modules.

How does AMX FP16 differ from AVX-512?

AVX-512 is a one-dimensional vector instruction set that operates on 512-bit registers, calculating up to 32 FP16 operations per cycle per FMA unit. Intel AMX is a dedicated two-dimensional matrix coprocessor that operates on 1 KB tile registers, calculating up to 2,048 FP16 operations per cycle per core. AMX delivers nearly an order-of-magnitude higher arithmetic throughput for deep learning tensor operations than AVX-512.

What socket form factor is required for Granite Rapids-AP?

Granite Rapids-AP (6900P series) utilizes the massive LGA 7529 socket on Intel's Birch Stream-AP platform, featuring 7,529 physical pins to accommodate 12 memory channels, 96 PCIe 5.0 lanes, and up to 500 watts of thermal design power (TDP) per socket. Smaller Granite Rapids-SP (6700P) processors utilize the LGA 4710 socket with 8 memory channels.

How does CXL 2.0 type 3 memory pooling function on Granite Rapids?

Granite Rapids supports Compute Express Link (CXL) 2.0 type 3 memory expansion devices over PCIe 5.0 lanes. This allows servers to connect external CXL memory expansion chassis, adding terabytes of supplemental DDR5 or CXL memory to the system memory map without consuming native CPU memory channels, ideal for massive in-memory databases like Redis, SAP HANA, and ClickHouse.