Breaking the Memory Wall: GPU · HBM Optical Interconnect

From 2.5D Co-Packaging to Decoupled Optical Architecture — Overcoming the Shoreline Bottleneck

Today: 2.5D Co-Packaging (Shoreline Constrained) Silicon Interposer (limited area) GPU Die Next-Gen AI Accelerator ~800 mm² compute capacity >> memory bandwidth HBM Stack 1 12-16 layers HBM Stack 2 12-16 layers Shoreline Limit: 4 HBM max HBM Stack 3 NO ROOM HBM Stack 4 NO ROOM electrical interconnects: ~1 TB/s per stack Pain Points: • Stack height capped at ~16 layers (physical limit) • GPU perimeter (shoreline) limits HBM count • JEDEC height specs relaxed but not solved • Vertical stacking hits process inflection point • Memory bandwidth cannot keep pace with compute EVOLUTION Optical Decoupling Future: Decoupled Optical Interconnect · Independent Packaging GPU Board (no shoreline constraint) GPU Die Independently Packaged No HBM adjacency req. CPO / optical I/O tiles Optical Bridge CPO / PIC silicon photonics HBM Zone (any layout) HBM 1 12-16L HBM 2 12-16L HBM 3 12-16L HBM 4 12-16L HBM 5 12-16L HBM N ... ≈ Key Advantages: • No shoreline constraint → any number of HBM stacks • HBM placed anywhere on board (lateral scaling) • Several times more total memory capacity • Dramatically expanded data bandwidth • Independent GPU & HBM packaging = modular Timing: rack-to-rack optical first → server-to-server → chip-to-chip "Nothing confirmed as official roadmap yet — preliminary research phase" Legend GPU / Compute HBM Memory Optical / Photonic Constraint / Bottleneck Optical Signal Path Board / Boundary

The Memory Wall Problem

  • • GPU compute grows exponentially each generation
  • • Memory bandwidth fails to keep pace
  • • HBM stacking hits physical limits at 12-16 layers
  • • GPU shoreline restricts HBM to max 4 stacks
  • • JEDEC relaxing height specs signals ceiling reached

Optical Interconnect Solution

  • • Decouple GPU and HBM into separate packages
  • • Connect via silicon photonics (CPO/PIC)
  • • Eliminates shoreline constraint entirely
  • • HBM can be placed anywhere on the board
  • • Several times more HBM capacity possible

Key Beneficiaries

  • • CPO / Silicon Photonics: MRVL, LITE, COHR
  • • HBM Memory Makers: MU, Samsung, SK Hynix
  • • Advanced Packaging OSATs: ASE, Amkor
  • • GPU Makers: NVDA, AMD (system architects)
  • • Timing: rack-level first, chip-level later

GPU-HBM Optical Interconnect Evolution · Based on ZDNet Korea report (May 22, 2026) · Discussion thread by @jukan05