Multicore Systems: SMP, NUMA & Amdahl | Draftsbook

Multi-Socket Architectures & Parallel Scaling Limits

In the mid-2000s, semiconductor physics struck a historic barrier: Dennard Scaling collapsed. Increasing CPU clock frequencies beyond 4 GHz resulted in unsustainable thermal power dissipation—commonly termed the Power Wall. Chip designers could no longer make single execution threads run faster by cranking clock speeds; the only viable path forward was packing multiple autonomous processor cores onto the same silicon die. In this concluding capstone article, we analyze Symmetric Multiprocessing (SMP), Non-Uniform Memory Access (NUMA), on-chip mesh interconnects, and the theoretical scaling boundaries established by Amdahl's Law and Gustafson's Law.

1. Symmetric Multiprocessing (SMP) & Uniform Memory Access (UMA)

In an SMP System, two or more identical CPU cores are connected to a single centralized physical memory system over a shared bus:

Uniform Memory Access (UMA): Every processor core experiences identical memory latency and bandwidth when accessing any physical address in RAM.
The Scalability Ceiling: As core counts increase (beyond 8 to 16 cores), all cores contend for the exact same physical memory bus, causing severe bus saturation and idling. SMP cannot scale to modern 64-, 128-, or 256-core server topologies.

2. Non-Uniform Memory Access (NUMA) Architecture

To scale to hundreds of cores, modern enterprise servers partition processors and memory into distinct physical clusters designated as NUMA Nodes:

Local Memory Access (~40–60 ns)

A core accesses DRAM attached directly to its own local memory controller on the same socket or chiplet. Maximum bandwidth and lowest latency.

Remote Memory Access (~100–160 ns)

A core accesses DRAM attached to a neighboring socket. The request must traverse coherent point-to-point interconnect fabrics (Intel UPI or AMD Infinity Fabric), incurring a 2x to 3x latency penalty.

Modern operating systems deploy NUMA-aware memory allocators and thread affinity pinning (e.g. numactl, pthread_setaffinity_np) to guarantee that worker threads execute on the specific CPU core physically adjacent to their data structures.

3. Theoretical Parallel Scaling: Amdahl's Law vs. Gustafson's Law

Amdahl's Law (Fixed Workload)

Formulated by Gene Amdahl in 1967:

\text{Speedup}(S) = \frac{1}{(1 - p) + \frac{p}{N}}

Where $p$ is the parallel fraction and $N$ is processor cores. If just 5% of a program is strictly serial ($1-p = 0.05$), the maximum theoretical speedup is capped at $\frac{1}{0.05} = 20\times$, even if the server possesses 10,000 CPU cores!

Gustafson's Law (Scaled Problem Size)

Formulated by John Gustafson in 1988:

\text{Scaled Speedup} = N - (1 - p) \times (N - 1)

Argues that when engineers have more computing power, they do not run the same tiny problem faster; they scale problem size (e.g., higher-resolution weather modeling or larger LLM models), yielding near-linear scalability.

4. Modern On-Chip Interconnects: Networks-on-Chip (NoC)

In modern silicon dies containing 64 to 128 cores (such as AMD EPYC Bergamo or Intel Xeon Granite Rapids), cores are no longer wired via shared buses. Instead, they communicate over packet-switched Networks-on-Chip (NoC) utilizing 2D Torus or 2D Mesh topologies with dedicated on-chip routers, providing aggregate bisection bandwidth exceeding Terabits per second.