Ep. 030 - Long Live the Short King: Why 4-HI HBM Wins (Memory) | Myron Xie, Jordan Nanos

And Nvidia, when they originally previewed Rubin Ultra to people at GTC last year, they said that the Rubin Ultra would have 1TB of HBM per package for chip.

it's pretty clear to tell if a given site is targeted for pre-training or not [...] if you're going to have more than 100,000 GPUs in an individual building [...] that's a very different data center design compared to, you know, 2,000, 4,000, 8,000 GPUs spread across four data halls

they charge you for the capacity. So customers pay effectively on a dollar per gigabyte basis for HBM. That's how they price it. So you can have, but the bandwidth is same. So if you're paying for more capacity with 8-HI or 12-HI, you're paying much more, you're almost double or triple what you paid for the same cube of 4-HI HBM.

The whole Rubin having 192MB instead of 1T HPM