Several companies presenting at Hot Chips 2026 described a widening gap between compute performance and memory bandwidth in AI systems. Micron's HBM Design Architecture Fellow, Raghu Sreeramaneni, said compute capability is scaling roughly 3x every two years while memory bandwidth is scaling less than 2x in the same period, a trend Micron calls the 'memory wall.' Micron noted that in a typical GPU package with four 12-high HBM stacks, memory silicon accounts for about 90% of total silicon, and cited Meta's Llama 3 training paper, which attributed 17% of unintended training interruptions to HBM issues.

One proposed alternative to conventional High Bandwidth Memory (HBM) comes from d-Matrix, which unveiled a technology called Raptor 3D DRAM. The design stacks a logic die (built on TSMC's 4nm process) on top of custom DRAM using face-to-face stacking, and is claimed to deliver over 100 TB/s of bandwidth at 0.37 picojoules per bit—compared to an 18 TB/s, 2-3 pJ/bit estimate d-Matrix gives for a 192 GB HBM4 solution. According to d-Matrix, this amounts to roughly 5.6x higher bandwidth, 5-8x lower energy, and 7x denser I/O than HBM4, positioning 3D DRAM as a middle ground between SRAM's speed and HBM's capacity. The company also compared Raptor to an Nvidia Rubin R200 platform with HBM4, claiming 23.4x higher bandwidth density and 13.5x better power efficiency, though it acknowledges challenges around I/O power, bank-to-channel mapping, redundancy, and thermal management that require faster refresh cycles and error-correction schemes.

Other memory makers are pursuing incremental but substantial upgrades to HBM itself. SK Hynix, presenting its advanced packaging roadmap, highlighted current HBM4 specifications (up to 36 GB capacity, 2048 GB/s bandwidth) and outlined future moves toward hybrid bonding technology to enable stacks beyond the current maximum of 16 layers, along with a hotspot-mitigation technology called I-HBM. SK Hynix's slides also referenced Intel's EMIB packaging technology alongside existing options like CoWoS-L, CoWoS-R, and CoWoS-S, and the company said it is looking toward eventually stacking HBM directly on top of accelerators. Separately, Samsung described a three-phase roadmap that migrates the HBM base die to advanced logic nodes, first reclaiming chip area and adding functions like a Heat Path Block for thermal management, then expanding capacity and adding processing elements, and ultimately arriving at a fully 3D design called zHBM that stacks DRAM directly on the compute die without a 2.5D interposer. Samsung claims zHBM could offer roughly 0.5 pJ/bit I/O power, more than double the bandwidth, about 70% lower DRAM power, and up to 100 W of thermal headroom compared to standard HBM4E.

Enabling many of these 3D and advanced-packaging ambitions requires new manufacturing capabilities. Rapidus outlined a packaging roadmap targeting eight-reticle-size interposers and 600 x 600 mm panel-level processing by around 2030, arguing that panel-scale processing offers substantial yield advantages over wafer-based approaches used by rivals such as TSMC. Rapidus said its full packaging portfolio will eventually include 2.5D interposers, panel-level redistribution layers, bridge-based solutions, and 3D stacking with hybrid bonding, aiming to combine sub-2nm logic, HBM, chiplets, and optical interconnects into a single platform by 2030.

While these technical roadmaps play out, the near-term memory squeeze is already affecting AI hardware pricing. Bloomberg reported that Nvidia has told some of its largest customers that AI server prices will rise more than 15% in many cases, with increases taking effect on Grace Blackwell and Vera Rubin systems shipping early next year. The size of increases will depend on chip generation and memory configuration, and companies that build servers for operators including Microsoft, Google, and Oracle have already notified their own customers. The report ties the increases to a broader DRAM supply crunch, noting that conventional DRAM contract prices were projected to climb 58-63% quarter-over-quarter in Q2 2026 following a 90-95% surge in Q1, as memory suppliers shifted capacity toward HBM and server products. SK Hynix said in October it had sold out its entire 2026 memory production capacity, and both Samsung and SK Hynix raised 2026 HBM3E supply prices by close to 20% before the year began. Nvidia's Rubin GPU ships with up to 288GB of HBM4 per package, and a full NVL72 rack-scale system combines 72 such GPUs, putting more than 20TB of HBM in a single rack.