Thread

Memory executives expect RAM shortage to continue through 2028

44 points · 66 comments · el_duderino

Invictus0 · 1790948631 · focus · HN ↗
Can someone please explain to me how the recent llm cache breakthrough doesn't alleviate this memory shortage issue? <a href="https://intl.cloud.baidu.com/en/article/8937874" rel="nofollow">https://intl.cloud.baidu.com/en/article/8937874
1. gortok · 1790948715 · focus · HN ↗
  Not everyone is using baidu?
2. Grimeton · 1790949259 · focus · HN ↗
  I have this wild theory that this has not just todo with AI but with weapons production overall.
  1. palmotea · 1790954091 · focus · HN ↗
    > I have this wild theory that this has not just todo with AI but with weapons production overall.
    Why? Weapons are low-volume compared to commercial products, and they probably use trailing edge or weird bespoke components.
    I was digging around the other day, and at least the initial versions of the Tomahawk cruise missile had the computing power of an original IBM PC. That just shows how little computing power is needed for a lot of advanced weapons capabilities.
    naishoya · 1790955704 · focus · HN ↗
    > how little compute power used to be needed ...
    ... for what were then 'advanced' weapons capability in high-cost units produced and expended in relatively small numbers with high yield payloads.
    The modern drone warfare version of expendable, high volume, high technology small units with two way communication via fibre-optic and RF jamming resistant multiple channels, and the ability to do on-device video object identification and terminal flight control when active electronic counter measures degrade operator control...
    That seems like exactly the kind of gear using commodity DDR4/5 sodimm and dimm modules and literally burning them up at then end of nearly every sortie. Even if they are using much smaller units (smartphone grade RAM) it's still cooking the limited output of the global memory fab capacity, and from the fields covered in fibre-optic blankets, I think it's safe to say there is a significant volume of electronics equipment being consumed.
    So yeah, AI might be ordering memory and putting it in datacenter, but eventually that material will be available for e-waste recovery (repurpose first, then recycle later). Both are competing for the same limited fab capacity.
    Grimeton · 1790970328 · focus · HN ↗
    Not just that. Everybody is rearming and the run for the first AI enabled fully autonomous robot is also going on.
    <a href="https://m.youtube.com/watch?v=TYaj0QnZx9s" rel="nofollow">https://m.youtube.com/watch?v=TYaj0QnZx9s
    naishoya · 1791001438 · focus · HN ↗
    Exactly. It might actually be a long term benefit to humanity that memory fab production is constrained at this juncture.
3. selbyk · 1790949356 · focus · HN ↗
  Why use less memory when you can just have more LLM?
  1. Invictus0 · 1790953968 · focus · HN ↗
    This is not a very useful comment
    post-it · 1790954278 · focus · HN ↗
    Yes it is, they're talking about induced demand. There are lots of resources where, when usage of the resource becomes more efficient, people end up using more rather than less.
4. linuxftw · 1790949642 · focus · HN ↗
  If training and inference is hardware constrained, and you can train and server better and bigger models on the same hardware with memory optimizations, that's exactly what I would expect companies to do.
5. s0ss · 1790949694 · focus · HN ↗
  Demand outstrips supply. Efficient software is great, but it doesn’t directly resolve insufficient supply of hardware.
6. moribvndvs · 1790950779 · focus · HN ↗
  See Jevons paradox. <a href="https://en.wikipedia.org/wiki/Jevons_paradox" rel="nofollow">https://en.wikipedia.org/wiki/Jevons_paradox
7. lostmsu · 1791196067 · focus · HN ↗
  This might regress intelligence.

Open on Hacker News to reply ↗

Unofficial Hacker News client; not affiliated with Y Combinator.