Thread

1. fooker · 1789929721 · focus · HN ↗

   What's the main blocker (other than the current inflated cost) for using HBM instead of DRAM as the primary memory for consumer electronics?

   1. HarHarVeryFunny · 1789935593 · focus · HN ↗

      Why would you want/need to?

      The advantage of HBM over regular non-stacked DRAM is memory bandwidth, which also requires a super-wide memory bus - 2048 bits wide for HBM4. Compare that to the 128 bit wide bus of a modern CPU.

      So to take advantage of it on the desktop, or anywhere else, you need that 2048 bit wide bus, and a processor capable of consuming 2-3 TB of data per second!

      These are not normal requirements, other than for a GPU.

      1. fooker · 1789935898 · focus · HN ↗

         SIMD (and especially the modern matrix extensions) can use as much bandwidth you can throw at it.

         1. b112 · 1789936580 · focus · HN ↗

            Without further clarification, that statement seems impossible. "As much" being unbounded and all. You should expand what you mean.

            1. fooker · 1789938993 · focus · HN ↗

               This will blow your mind, but it actually is pretty close to being unbounded. :)

               Consider the 'MMA N matrices' primitive modern CPUs are starting to support. For the current generation of CPUs, N is a constant like 16 or 32, but there's nothing preventing it from being 1024 or larger if we have more memory bandwidth.

               All this with a single instruction.

               1. articulatepang · 1789947110 · focus · HN ↗

                  Surely something prevents it being 1 quadrillion bits per instruction? Since that’s well within “unbounded”.

                  1. pixl97 · 1790004382 · focus · HN ↗

                     Most likely the chip running at the core temperature of the sun.

                     We'll have to figure out how to read and right to the surface of a black hole to get speeds that high.

               2. imtringued · 1789985766 · focus · HN ↗

                  Nah his point is a bit simpler. Vector units can process a limited amount of data per time unit. Memory can load a limited amount of data per time unit.

                  If you have infinite memory bandwidth you just move the bottleneck back to compute so both have to grow simultaneously in lockstep.

                  What you should have said is that CPUs have so much compute headroom for matrix vector multiplication that simply adding more memory bandwidth would make them faster so every improvement in memory bandwidth is welcome.

                  1. fooker · 1789986186 · focus · HN ↗

                     Agreed.

                     The "move the bottleneck back to compute" bit is changing rapidly though. The first time a major hardware company ships a PIM chip, you can push for a few orders of magnitude more data through without being compute bound.

Unofficial Hacker News client; not affiliated with Y Combinator.