High Bandwidth Memory

Technology

High Bandwidth Memory (HBM) delivers faster data transfer than DDR4 or GDDR5, consumes less power, and takes up much less physical space. It achieves this by stacking as many as 32 DRAM layers, sometimes with an optional base layer that holds buffer circuits and test logic. The stack typically connects to a GPU or CPU memory controller through a silicon interposer, though it can also be placed directly on top of the processor chip. Inside the stack, the layers communicate vertically using through-silicon vias (TSVs) and microbumps. HBM is conceptually similar to Micron’s Hybrid Memory Cube (HMC) interface, but the two are not compatible.

The HBM memory bus is exceptionally wide compared to DDR4 or GDDR5. A single HBM1 stack with four DRAM layers (4Hi) provides two 128-bit channels per layer, giving eight channels and a total width of 1024 bits. A graphics card equipped with four such stacks would have a 4096-bit-wide memory bus. For context, GDDR memories use a 32-bit bus width, and a card with a 512-bit interface has 16 channels. HBM1 packages could hold up to 4 GB of memory.

Because HBM requires many more connections than DDR4 or GDDR5, a new way of linking it to the processor was needed. Both AMD and Nvidia have used custom-built interposers—specialized semiconductor devices—to connect the HBM stacks to the GPU die. This approach also forces the memory and processor to sit very close together, shortening the data paths. However, fabricating these interposers costs far more than making a standard printed circuit board, which raises the overall price of the final product.

HBM DRAM is tightly integrated with the host processor through a distributed interface. This interface splits into independent channels that operate completely separately and do not need to be synchronized with one another. The wide-interface design enables high speed and low power consumption. In HBM1, the DRAM used a 500 MHz differential clock (CK_t and CK_c), with commands registered on the rising edges of both signals. Each channel had a 128-bit data bus running at double data rate (DDR). This setup allowed HBM1 to achieve a transfer rate of 1 GT/s per pin, resulting in a total package bandwidth of 128 GB/s.

History

Die-stacked memory first appeared in the flash memory market. In April 2007, Toshiba released a NAND flash chip with eight stacked dies; five months later, Hynix Semiconductor followed with a 24-die stack. The move to 3D-stacked RAM using through-silicon via (TSV) technology came from Elpida Memory, which built the first 8 GB DRAM chip from four stacked DDR3 SDRAM dies in September 2009 and put it on the market in June 2011. That same year, SK Hynix shipped a 16 GB DDR3 module using TSV on a 40 nm process, Samsung Electronics introduced a 32 GB DDR3 stack on 30 nm in September, and both Samsung and Micron Technology announced their TSV-based Hybrid Memory Cube (HMC) in October.

The JEDEC standard for Wide IO memory, a forerunner to HBM with four 128-bit channels and single data rate clocking, was published as JESD229 in December 2011 after years of work. The first official HBM standard, JESD235, arrived in October 2013. AMD started developing High Bandwidth Memory in 2008 to tackle rising power consumption and the physical size of computer memory. Over the following years, a team led by Senior AMD Fellow Bryan Black worked out solutions for die-stacking issues. AMD brought in partners with relevant expertise: SK Hynix, a Korean memory maker experienced in 3D stacking; UMC, a Taiwanese interposer specialist; and packaging firms Amkor Technology and ASE.

Development wrapped up in 2013 when SK Hynix produced the first HBM chip. That October, JEDEC adopted HBM as standard JESD235, based on a proposal AMD and SK Hynix had submitted in 2010. High-volume manufacturing started at SK Hynix’s facility in Icheon, South Korea, in 2015. The first graphics card to use HBM was the AMD Fiji GPU, which powered the AMD Radeon R9 Fury X when it launched in June 2015. In January 2016, Samsung Electronics began early mass production of HBM2, and JEDEC accepted it as standard JESD235a that same month. The Nvidia Tesla P100, announced in April 2016, was the first GPU to use HBM2. Intel followed in June 2016 with a family of Xeon Phi processors that packed eight stacks of HCDRAM, Micron’s take on HBM.