NVIDIA NVLink Explained: High-Speed GPU Interconnect for AI and HPC

Modern computing increasingly depends not just on the power of a single processor or graphics card, but on the speed of data exchange between them. While one GPU can perform a tremendous amount of parallel operations, training large neural networks, conducting scientific simulations, and working with models containing hundreds of billions of parameters now requires dozens or even hundreds of accelerators. NVIDIA NVLink is a high-speed interconnect technology designed to enable rapid data exchange between GPUs and other compute components, reducing latency and significantly increasing bandwidth compared to standard PCIe connections. This allows multiple accelerators to be combined into a tightly integrated computing system, where data constantly moves between GPUs.

What Is NVIDIA NVLink and Why Is It Needed?

NVLink can be thought of as a dedicated ultra-fast highway connecting graphics processors. While standard systems use PCIe as a universal communication channel between the CPU, graphics cards, storage, and other devices, NVLink is specifically designed for intensive data exchange between compute accelerators.

This is especially critical for workloads that cannot be efficiently handled by a single GPU. For example, a large neural network might not fit entirely within the memory of one accelerator. Its parameters are then distributed across multiple GPUs, and during operation, devices must constantly share intermediate results.

Without a fast interconnect, performance is lost while waiting for data. Even if each GPU can perform trillions of operations per second, it may sit idle while waiting for information from another accelerator. Thus, as you scale up your compute system, the speed of interconnect becomes almost as important as the GPU performance itself.

NVLink addresses this by enabling direct, high-speed data transfer between accelerators. Data can move from one GPU to another without always relying on the CPU as an intermediary, reducing latency and freeing up PCIe bandwidth.

However, NVLink does not literally merge multiple graphics cards into a single GPU. The operating system and applications still see separate GPUs, each with its own memory and compute resources. To leverage them together, software must be capable of distributing computations and data across multiple accelerators.

So, when we talk about "combining GPUs," it doesn't mean creating a single physical device, but rather organizing such fast communication that multiple processors can efficiently work on a shared task. This is the principle behind modern AI servers and large NVIDIA compute clusters.

How NVLink Works and How Multiple GPUs Are Combined

NVLink functions as a dedicated communication channel between compute processors. Rather than sending data over the system bus, GPUs can directly exchange large volumes of information via high-speed NVLink lanes.

Each NVLink lane transfers data between connected devices, and multiple lanes can be used in parallel. More available connections mean higher total bandwidth between GPUs. This is especially important for AI workloads, where accelerators constantly exchange tensors, model segments, and intermediate results.

Direct GPU-to-GPU Connection

In smaller systems, several GPUs can be directly linked. Data then moves from one accelerator to another without always passing through the central processor.

Imagine a large neural network split between two GPUs. The first accelerator processes part of the model and must instantly send the result to the second. The faster this transfer happens, the less time the second GPU spends waiting.

In standard setups, such exchanges often depend on PCIe and server memory architecture. NVLink provides a much faster path, improving parallel computing scalability as more accelerators are added.

However, direct connections become increasingly complex as the number of GPUs grows. Linking two or four accelerators is straightforward, but in systems with dozens of processors, it's impractical to create a direct physical connection for every pair.

NVSwitch: When Two GPUs Aren't Enough

For large-scale systems, NVIDIA employs NVSwitch-a specialized, high-speed switch for NVLink. Its function is similar to a network switch, but instead of ordinary computers and Ethernet packets, it connects GPUs and manages their internal data traffic.

Multiple accelerators connect to the NVSwitch, which routes data between them. This allows GPUs to interact not only with direct neighbors but with any other accelerator within the NVLink infrastructure.

Such setups can create near all-to-all topologies, where every GPU has high-speed access to the rest of the system. This is much more convenient for software than a complex web of individual connections, which could otherwise result in variable transfer speeds depending on physical layout.

Large server solutions often use several NVSwitches together, forming an NVLink fabric-an internal network tying dozens of GPUs into a single compute domain.

This is where the idea of "one giant computer" becomes tangible. Physically, the system is still made up of many independent GPUs, each with its own cores and memory. But the fast inter-GPU exchange allows a massive task to be split among all accelerators, making far more effective use of their combined performance.

NVLink Speed and How It Differs from PCIe

The main advantage of NVLink is its bandwidth. In modern AI systems, the volume of data exchanged between accelerators is so large that a typical device interface can become a bottleneck, even with the most powerful GPUs.

The difference becomes even clearer as the technology evolves. Fourth-generation NVLink (Hopper architecture) provided up to 900 GB/s per GPU. Fifth-generation (Blackwell) doubled that to 1.8 TB/s, while NVLink 6 for the Rubin platform pushes up to 3.6 TB/s. These figures refer to the total bidirectional bandwidth of all NVLink connections for a single accelerator.

PCIe is much more versatile. It connects GPUs, network adapters, NVMe storage, and many other devices to the processor. NVLink, on the other hand, solves a much narrower task-enabling intense data exchange between compute processors where the standard system bus is insufficient.

Therefore, it's not entirely accurate to call NVLink "just a faster PCIe." These interfaces were designed for different roles: PCIe links a computer's components to the system, while NVLink creates a high-speed computing fabric between GPUs.

Not only peak speed, but also latency is critical. In distributed computing, accelerators may exchange small data chunks very frequently. If each operation is slow, delays add up and reduce overall system performance.

NVLink and PCIe are not mutually exclusive technologies. A server can use PCIe to connect GPUs to the CPU and peripherals, while NVLink enables high-speed exchanges directly between accelerators.

For example, modern Blackwell accelerators use PCIe to interface with the host system, while simultaneously employing fifth-generation NVLink with up to 1.8 TB/s bandwidth for GPU-to-GPU connections. Rubin increases this to 3.6 TB/s.

This is why NVLink speed becomes critical at scale. Adding another powerful GPU is relatively easy. What's much harder is making sure dozens of accelerators can continuously exchange data without sitting idle. In large compute systems, interconnect performance becomes a key factor in overall speed.

Why NVLink Matters for Neural Networks and Supercomputers

The larger a neural network, the harder it is to fit on a single GPU. The limiting factor isn't just compute power, but also video memory size. If a model occupies hundreds of gigabytes or even terabytes, its parameters must be split across multiple accelerators.

In this setup, GPUs are constantly exchanging data. One accelerator processes its part of the model and passes intermediate results to the next, which continues the computation. If the interconnect is slow, even powerful GPUs will idle while waiting for data.

NVLink reduces these idle times with its high bandwidth and low latency. This is especially noticeable when training neural networks, where gradients, parameters, and intermediate tensors regularly synchronize between accelerators. The more GPUs involved, the higher the demands on exchange speed.

This is also important for Mixture of Experts architectures, where different parts of the model run on different accelerators and requests are dynamically routed to the relevant experts. Large volumes of data move within the system, and performance depends directly on the quality of the GPU interconnect.

Learn more about how these models work in the article Mixture of Experts: How Neural Networks Choose the Right Experts.

NVLink isn't just used for training. During inference with large models, parameters may also be distributed across several accelerators, enabling neural networks that simply cannot fit into the memory of a single GPU.

A similar situation arises in scientific computing and HPC. Modeling climate, materials physics, aerodynamics, molecular processes, and other complex systems is often split into many parallel tasks. Individual GPUs handle their own data segments but must regularly exchange results.

That's why large-scale compute systems need more than just additional accelerators. If the links between them are slow, adding new GPUs brings diminishing returns. NVLink addresses precisely this issue: it helps systems scale not only by processor count, but also by the speed of interconnect.

NVLink Today: From Two Graphics Cards to Compute Racks

The first NVLink versions were often associated with workstations and systems featuring just a few graphics cards. Over time, however, the technology's purpose has shifted dramatically. NVIDIA has moved NVLink away from consumer products and into the realms of data centers, supercomputers, and AI infrastructure.

Today, NVLink is better understood not as a "bridge between two GPUs," but as a full-fledged high-speed network within a compute system. This is especially evident with the Blackwell generation. The fifth-generation NVLink can connect up to 72 Blackwell GPUs in a single NVLink domain, with each accelerator supporting up to 1.8 TB/s bandwidth. By comparison, Hopper-based systems typically featured eight GPUs per NVLink domain, each with up to 900 GB/s.

This shift has fundamentally changed the scale of the technology. Instead of a few accelerators inside one server, NVIDIA now builds entire compute racks where dozens of GPUs are linked with NVSwitch and work together in a unified, high-speed fabric.

The next generation-NVLink 6 for the Vera Rubin platform-raises per-GPU bandwidth to 3.6 TB/s. In the Vera Rubin NVL72 system, 72 Rubin GPUs and 36 Vera CPUs are connected, and the total NVLink fabric bandwidth within a rack reaches around 260 TB/s.

These systems demonstrate why the phrase "giant computer" is becoming less of a metaphor. Individual GPUs retain their own compute blocks and HBM memory, but NVLink and NVSwitch create such a fast communication channel between them that software can distribute a massive model across the entire rack.

This approach is especially in demand for today's models with huge parameter counts and Mixture of Experts architectures. When the necessary data constantly resides on different accelerators, performance depends not only on Tensor Core speed, but on how quickly information flows between GPUs.

NVLink continues to evolve. NVIDIA now views it as a scale-up network-a fabric for scaling computing within one large domain. The NVLink Fusion roadmap anticipates expanding such domains up to 1,152 accelerators using new connection methods, including optical technologies.

Thus, modern NVLink has become part of a broader architectural vision: the data center is no longer just a collection of servers, but is transforming into a unified compute complex where CPUs, GPUs, memory, and networking are co-designed.

This concept is especially apparent in NVIDIA's next-generation architecture. Learn more in the article NVIDIA Vera Rubin: A New Architecture for Scalable AI.

Conclusion

NVIDIA NVLink addresses one of the key challenges in modern compute systems-slow data exchange between many powerful GPUs. As neural networks and scientific workloads grow, performance increasingly depends not just on accelerator speed, but on how quickly they can interact.

Together with NVSwitch, the technology allows dozens of GPUs to be combined into a single, high-speed compute environment. NVLink does not replace PCIe or turn multiple accelerators into one physical graphics card. Its purpose is to enable fast data movement within distributed computing, minimizing GPU idle time.

For an average gaming PC, NVLink is largely irrelevant today. Its main domain is AI infrastructure, HPC, and large data centers, where models and computations can no longer fit on a single accelerator. Here, interconnect speed is one of the main factors for system scalability.

The evolution of NVLink reflects the direction of modern computing: instead of endlessly increasing a single processor's power, the industry is now combining many specialized chips into collective compute complexes. And as these systems grow, the speed of data exchange becomes just as critical as raw compute performance.