How CoWoS Packaging is Powering Next-Gen AI Accelerators

CoWoS was once a highly specialized term in semiconductor manufacturing, but with the rise of artificial intelligence, this technology has become a crucial element in modern AI accelerators. The performance of a single compute die is no longer sufficient: it must be paired with massive amounts of fast memory and ultra-low-latency data transfer.

This is especially evident in NVIDIA accelerators, where compute dies work together with several HBM memory stacks. Simply placing these components on a standard PCB introduces limitations in connection density, power consumption, and bandwidth. As a result, manufacturers are adopting more advanced packaging methods.

What is CoWoS and Why is This Chip Packaging So Important?

CoWoS stands for Chip-on-Wafer-on-Substrate. Despite the complex name, the main idea is simple: several separate semiconductor components are placed extremely close together and interconnected inside a single package much more densely than possible on a conventional motherboard or server board.

It's important to distinguish between chip manufacturing and packaging. First, transistors are created on a silicon wafer, forming the compute die. Afterward, the die must be connected to memory, power, and other components, and then the finished module is mounted on a PCB. This is where advanced packaging comes into play.

In a traditional setup, the processor and memory may be separate chips, with data traveling along PCB traces. The higher the required bandwidth, the more limiting the trace length, pin count, and electrical losses become. For modern AI accelerators, which constantly move huge amounts of data between GPU and memory, these limitations are especially pronounced.

CoWoS solves this problem with an additional layer called an interposer. This interposer can host the compute die or several chiplets alongside HBM stacks, connecting them with a large number of short conductors. Physically placing the components close together allows for much wider data interfaces. TSMC states that CoWoS enables the integration of several SoCs, logic chiplets, and HBM stacks into a single system.

This approach is known as 2.5D packaging. Unlike full 3D integration, where active compute dies are stacked, traditional CoWoS places the main components side by side on a shared intermediary structure. This results in much higher connection density than mounting individual chips on a PCB.

For AI chips, this is critical. Adding more compute units is only effective up to a point: if the GPU can't receive data from memory fast enough, those extra resources sit idle. The advancement of modern accelerators depends not only on process technology and transistor count, but also on how effectively compute dies and memory are integrated into a unified system.

How CoWoS Works: GPU, Interposer, HBM, and Substrate

At first glance, a CoWoS module may look like a single large processor, but inside are several components. The compute die or chiplets are positioned next to high-speed memory stacks, all sitting atop an interposer that links everything into one system.

The simplified structure:

• GPU or compute chiplets →
• HBM →
• Interposer →
• Package substrate →
• PCB

Each layer has a role. The GPU processes data, HBM stores and rapidly transfers it, the interposer creates a dense network of connections, and the substrate delivers power and connects the module to the rest of the system.

The Interposer: The Foundation of the System

The interposer is like a highly sophisticated miniature board placed right under the compute dies and memory. Its main advantage over a regular PCB is a dramatically higher connection density.

This allows for a huge number of conductors between the GPU and HBM. Instead of a relatively narrow interface to separate memory chips, the accelerator gets a very wide bus, capable of transferring much more data simultaneously.

The short length of these connections is particularly important. The shorter the distance an electrical signal travels between components, the easier it is to maintain high speeds and acceptable power efficiency. In CoWoS, the compute die and memory are literally side by side.

The interposer itself does not perform computations. Its job is to create an infrastructure through which different semiconductor components can function almost as parts of a single large chip.

Why HBM Is Placed Next to the Compute Die

For an AI accelerator, high compute power only makes sense if the GPU can access the necessary data quickly. Training and running large neural networks requires constant access to model weights, intermediate results, and other data arrays.

While conventional memory like GDDR can deliver high speeds, HBM takes a different approach: several memory dies are stacked vertically and connected with through-silicon vias. This enables a very wide memory interface within a compact footprint.

This is why HBM4, the next generation of memory for AI and HPC, continues to push the envelope on bandwidth-an essential feature for modern AI accelerators.

CoWoS allows these stacks to be placed directly next to the GPU. Data flows through the interposer via many parallel connections, letting the processor access memory much faster than if it were located far away on a PCB.

This design also saves space: several HBM stacks and one or more compute dies can be assembled into a single massive package. For server accelerators, this is crucial, as each module needs to pack in ever more compute and memory.

However, the complexity of such packaging increases dramatically. Large dies and HBM must be precisely placed, thousands of connections made, power delivered, and heat dissipated-all within a dense package. Modern AI accelerators are no longer just powerful GPUs, but intricate systems whose performance depends heavily on the efficiency of their internal connections.

Why CoWoS Is So Important for AI Chips

For a long time, AI accelerator performance was mostly tied to transistor counts, new process nodes, and more compute blocks. But modern neural networks require such intense data exchange that the bottleneck is no longer just the GPU, but also memory bandwidth.

When training large models, the accelerator must execute countless matrix operations while constantly fetching model weights. If memory can't supply data fast enough, compute blocks sit idle. Thus, for AI systems, memory bandwidth is almost as important as raw compute power.

CoWoS enables several HBM stacks to be placed next to the GPU and connected with ultra-wide interfaces. TSMC explicitly states the technology is designed to integrate multiple compute components and HBM in one package, boosting both available compute and memory bandwidth.

Processor architecture is also evolving. Instead of trying to make a single monolithic die ever larger, manufacturers increasingly combine multiple specialized components. This approach is explored in detail in the article Chiplets in Processors: The Modular Revolution Reshaping CPUs: breaking a large chip into chiplets helps scale compute systems but raises the bar for efficient interconnects.

For AI accelerators, this is especially challenging, as several large elements-compute dies, interface blocks, multiple HBM stacks-must fit within a single package. The more components, the harder it is to ensure fast communication without excessive power draw.

At this point, advanced packaging is no longer just a final manufacturing step. The quality of the package directly affects how many compute dies and how much memory can be combined in one accelerator.

TSMC continues to scale up CoWoS. In 2026, the company announced packages up to 5.5 reticles in size, with a 14-reticle version for 2028, targeting roughly ten large compute dies and twenty HBM stacks in one solution. This points to the future: AI systems will need ever more silicon and memory inside a single package.

Scaling up introduces new engineering challenges: maintaining high-speed signal integrity, delivering power to many components, and removing huge amounts of heat. The bigger the package, the harder it is to ensure mechanical stability and high manufacturing yields.

Thus, AI scaling now occurs on many levels. Manufacturers are improving transistors, boosting GPU performance, advancing HBM, and simultaneously developing more complex packaging technologies. Without this last step, assembling high-performance AI accelerators would be much harder.

Rising interest in CoWoS reflects this shift. According to TSMC, demand for the technology spiked after the generative AI boom at the end of 2022. The company is expanding its platform to accommodate more compute and memory within each package.

As a result, CoWoS has become an infrastructure element of the AI industry. It doesn't directly increase GPU operations, but enables the kind of compute-memory configurations needed to feed those operations efficiently.

How NVIDIA Uses CoWoS in Modern AI Accelerators

NVIDIA exemplifies why advanced packaging is now so vital to the AI industry. Its latest accelerators are no longer just single large GPUs: the package must integrate several huge compute dies with high-speed memory, and enable massive data exchange between them.

This is clear in the Blackwell architecture. NVIDIA states that a single Blackwell GPU consists of two large dies connected by a 10 TB/s interface. This split allows for a GPU with 208 billion transistors without attempting to fabricate an even larger monolithic die.

Compute dies must be tightly coupled with HBM-this is where CoWoS is key: TSMC's technology allows several logic dies and high-speed memory stacks to be assembled on a common interconnect system. The result is a set of independent dies operating as a single compute module.

For a detailed look at this generation of accelerators, see NVIDIA B200 and the Blackwell Architecture: A New Standard for AI Accelerators.

The scale is further illustrated by the GB200 Grace Blackwell system: one such superchip combines an NVIDIA Grace CPU and two Blackwell GPUs, while the GB200 NVL72 configuration includes 36 Grace CPUs and 72 Blackwell GPUs. NVIDIA claims the entire rack delivers 13.4 TB of HBM3E memory and a total bandwidth of 576 TB/s.

These numbers highlight how developer priorities have shifted. Designing a fast compute die is no longer enough-it must be paired with memory capable of constantly supplying huge data volumes. If memory is too slow, GPU compute blocks can't realize their full potential.

Why Packaging Is Nearly as Important as the Process Node

Previously, new processor generations were mostly associated with advanced process nodes. Shrinking transistors enabled more compute elements in the same area, reduced power consumption, or raised clock speeds.

This trend continues, but transistor scaling alone is no longer sufficient. Die size is limited by photolithography, and making extremely large monolithic chips is increasingly challenging and expensive. Manufacturers now split systems into multiple dies, which then must be reconnected at such high speed that the split is almost imperceptible in operation.

Blackwell illustrates this transition well. Two large dies inside a single GPU are connected by a massive bandwidth interface and placed next to HBM. Performance growth now depends on several technologies at once: advanced process nodes, GPU architecture, high-speed memory, and advanced packaging.

CoWoS enables this last level of scaling. TSMC is evolving the platform to accommodate ever more logic dies and HBM stacks in one package. The company directly links the continued expansion of CoWoS package size to the growing compute and memory demands of AI systems.

As a result, packaging is becoming one of the defining factors for next-generation accelerators. Even if NVIDIA designs a more powerful GPU, it still must be physically integrated with enough memory, power, and thermal management for all components in an extremely dense package.

Modern AI chip development increasingly resembles designing an entire compute system in a single package. NVIDIA handles accelerator architecture and the overall platform, memory vendors advance HBM technology, and platforms like CoWoS bring all elements together. Without such integration, scaling up AI accelerator size and performance would be far harder.

CoWoS-S, CoWoS-R, and CoWoS-L: What's the Difference?

CoWoS is not a fixed design but a family of packaging technologies. TSMC develops several platform variants to cater to accelerators with different size, connection density, and layout requirements for compute dies and HBM. The main versions now are CoWoS-S, CoWoS-R, and CoWoS-L.

CoWoS-S

CoWoS-S is the classic version. It uses a large silicon interposer to host logic dies and HBM stacks. Silicon allows for very dense conductor networks and is ideal for high-speed data exchange between components.

This achieves high connection density, but has a limitation: the larger the interposer, the harder and more expensive it is to produce. According to TSMC, CoWoS-S supports silicon interposers up to about 3.3 reticles (around 2,700 mm²). For even larger packages, CoWoS-R or CoWoS-L are recommended.

CoWoS-R

CoWoS-R replaces the large monolithic silicon interposer with a structure based on RDL (Redistribution Layer)-multiple layers of redistributed connections made from polymer materials and copper conductors.

This allows for larger packages while maintaining high-speed links between SoC and HBM. RDL also offers more mechanical flexibility versus massive silicon, helping reduce stresses from different thermal expansion rates. CoWoS-R has been in mass production since 2023.

Connection density is lower compared to local silicon structures, so CoWoS-R is especially useful when maximizing package area and efficiently routing many components is more important than peak density.

CoWoS-L

CoWoS-L combines the strengths of both approaches. It uses a large RDL-based interposer as the base, but adds small embedded silicon LSI (Local Silicon Interconnect) elements where extremely high connection density is needed.

This hybrid approach eliminates the need for a massive all-silicon interposer. High-density silicon connections are created only between selected components-such as two compute dies or between logic and HBM-while the rest of the signals and power pass through the larger RDL structure.

This is especially suited for massive AI accelerators. TSMC started mass production of 3.5-reticle CoWoS-L in 2024 and continues to expand package size so more compute silicon and HBM can fit within a single module.

The differences between CoWoS variants show how the packaging challenge has transformed. It's no longer just about mounting a finished GPU on a substrate. Manufacturers now design a full high-speed compute system inside a single enclosure, comprising many separate dies.

As AI models grow, this approach becomes ever more essential. If next-generation accelerators require even more compute dies and HBM stacks, traditional silicon interposer sizes will quickly become a bottleneck. That's why TSMC develops CoWoS-S, CoWoS-R, and CoWoS-L in parallel: each offers a different balance between connection density, package size, and manufacturing complexity.

Conclusion

CoWoS demonstrates how much modern AI processor design has changed. AI accelerator performance now depends not just on GPU architecture and process node, but also on how effectively compute dies can be combined with high-speed memory in a single package.

TSMC's technology enables GPUs, chiplets, and multiple HBM stacks to share a high-density interconnect system. This reduces the distance between components, boosts memory bandwidth, and makes it possible to build much larger compute modules.

For NVIDIA and other AI accelerator developers, this is critical: as neural networks grow, both compute and memory demands rise. Simply growing a single monolithic die is no longer enough, so the industry is shifting to systems of closely integrated components.

The ongoing development of CoWoS-S, CoWoS-R, and especially CoWoS-L shows the direction of the field. Packages are getting bigger, containing more compute dies and HBM, and the technology for connecting these elements is becoming a key factor in scaling AI hardware.

Thus, CoWoS is not just a way to mount a GPU on a substrate. It is the infrastructure inside a modern AI accelerator, without which combining immense compute power and memory bandwidth in a single module would be far more difficult.