Webinar Review, Part 1: A Packaging Revolution Powering the Next Era of AI Compute

Estimated reading time: 3 minutes

The rapid rise of artificial intelligence has fundamentally reshaped the demands placed on semiconductor technology, and nowhere is it more evident than in advanced electronic packaging. A recent episode of the Global Electronics Association’s Executive Pulse webinar series, featuring Dr. Hemanth Dhavaleswarapu of AMD as the first of two esteemed presenters, provided a comprehensive overview of chip-level packaging.

He outlined the evolution of AI workloads in forcing a rethinking of everything from chip architecture to data center design, with packaging now playing a central role in enabling continued performance scaling.

Large language models, he said, are at the heart of the challenge, having become some of the most compute- and memory-intensive workloads in existence. As models grow larger, incorporate richer modalities such as images and video, and rely on longer context lengths and techniques like retrieval-augmented generation, their appetite for memory capacity and bandwidth surges.

New approaches, including lower-precision data types, Dr. Dhavaleswarapu continued, help manage memory footprints, but they do not reduce the need for fast, low-latency access to vast quantities of data. At the same time, he emphasized that emerging trends like agentic AI, reinforcement learning, and mixture-of-expert models introduce new communication demands between GPUs, making high-speed interconnects a critical system-level requirement.

These pressures have pushed GPUs to the forefront of data center computing, driving dramatic increases in power consumption. While CPU power has largely plateaued due to air-cooling constraints, GPU power has risen sharply, especially following the surge in interest in generative AI. Individual GPUs now operate at kilowatt-class power levels, while racks are approaching or exceeding hundreds of kilowatts. This escalation has made liquid cooling not just advantageous, but essential, and has introduced new challenges related to flow rates, pressure drops, inlet temperatures, and the mechanical weight of fully populated racks.

Against this backdrop, advanced packaging has become a key enabler of continued performance growth. Dr. Dhavaleswarapu traced AMD’s journey from early multi-chip approaches to today’s sophisticated 2.5D and 3.5D architectures. He said that by integrating multiple dies and high-bandwidth memory into a single package, these approaches enable massive increases in compute density and memory capacity. In fact, packaging-driven integration has delivered orders-of-magnitude growth in total compute and memory area that would be impossible through silicon scaling alone, effectively extending and surpassing the traditional expectations of Moore’s Law.

Looking ahead, he said future systems will push integration even further, incorporating domain-specific accelerators, co-packaged optics, and other components into ever-larger modules. These designs aim to deliver higher bandwidth at lower energy per bit, but they also demand new manufacturing approaches. As chip modules grow beyond the practical limits of wafer-level processing, panel-level manufacturing becomes increasingly attractive. If offers improved throughput and cost efficiency for large-format packages.

He emphasized that three-dimensional interconnect technologies are advancing rapidly to support these trends. Hybrid bonding, finer interconnect pitches, and wafer-to-wafer bonding are enabling denser vertical integration, while backside power delivery networks promise improved power efficiency through nanoscale through-silicon vias (TSVs). Together, these innovations are essential for managing the power and signal integrity challenges of trillion-transistor-class packages.

Substrates, he added, represent another critical bottleneck. As packages grow in size and complexity, substrates must fan out ever-increasing numbers of signals while reliably delivering power and maintaining mechanical stability. This is driving demand for ultra-large substrates with higher layer counts, low-loss dielectric materials, smoother copper surfaces for high-speed signaling, and improved thermal and mechanical properties to manage warpage and stress.

Throughout the presentation, Dr. Dhavaleswarapu emphasized that these challenges cannot be solved in isolation. Thermal performance, structural integrity, power delivery, and signal integrity are tightly coupled, and optimizing one often impacts the others. Addressing this complexity requires advanced multiphysics simulation and co-design tools that span silicon, package, board, and system levels. Yet today’s toolchains remain fragmented to some extent , he said, relying on custom workflows and data translations, and highlighting the need for standardized formats and more integrated EDA solutions.

Finally, we learned that the scale and complexity of advanced packaging have significant implications for the supply chain. Innovation is required across every element of the ecosystem, from materials and substrates to inspection, test, and cooling technologies. Because different suppliers advance at different rates, supply chain resiliency has become a strategic concern, particularly as AI-driven demand accelerates and new fabrication capacity comes online. Supporting this growth will require sustained investment in workforce development to ensure the talent needed to design, build, and operate next-generation semiconductor infrastructure is available.

It was a lot to pack into 20 minutes, but Dr. Dhavaleswarapu’s presentation made clear that advanced electronic packaging is no longer a supporting technology, but a central pillar of AI system performance, something the Global Electronics Association’s CTO Matt Kelly has been speaking about for many months. As AI workloads continue to scale, packaging innovation will be essential to delivering the compute, memory, power efficiency, and reliability that modern data centers demand.