Thread

The Painful Truth: The RAM Crisis Is Only Just the Beginning

62 points · 69 comments · perelin

N_Lens · 1789618282 · focus · HN ↗
The big companies are insulated because they've locked in multi-year supply contracts with ram manufacturers - Microsoft, Google, Meta, Amazon (All on 3-5 yr contracts). OAI's deal with Samsung & SK Hynix is also colossal - locking up roughly 900,000 DRAM wafers per month, roughly 40% of world output (Stargate project).
Apple got caught out because it had a shorter contract that ended in the beginning of Q3, and they've tried to source their RAM from Chinese CXMT who declined because all capacity was already locked in contracts. They've gone with a lesser known company Kioxia.
Overall the consumer segment is completely neglected, companies don't care about end users in the current market conditions.
1. josephg · 1789618707 · focus · HN ↗
  > OAI's deal with Samsung & SK Hynix is also colossal - locking up roughly 900,000 DRAM wafers per month, roughly 40% of world output (Stargate project).
  I wonder if they'll, at some point, have enough RAM? Or is this is the new normal? Will models keep scaling with the amount of ram chips openai and anthropic own?
  1. bionhoward · 1789619444 · focus · HN ↗
    Even with efficiency breakthroughs, it would just afford packing more agents per unit of memory. Scaling compute and data keeps paying off, leading to smarter models, and smarter models have more demand even at higher prices because they can accomplish more work at higher quality. Sovereign AI hasn’t even really taken off yet to anywhere near the level it could. That’s going to dramatically increase the number of massive-scale users of AI agents. So no, IMHO they will “never” have “enough.”
    epicureanideal · 1789620733 · focus · HN ↗
    At some point though, won't someone be able to extrapolate the demand growth curve, and invest some colossal amount of money into making and selling more RAM chips?
    fragmede · 1789621449 · focus · HN ↗
    Yes. China's done just that. Expect those factories to come online within 2-3 years.
    cyanydeez · 1789648531 · focus · HN ↗
    Weve hit the sigmoid. Whats scalling is ancillary to the model. The cry for a slowdown is because the open weight models demonstrate the cost of parameter pacling is not work neither inference nor training.
    The assumption about the singularity simply is a delusion with LLMs.
    However, the models do provide a means to improve the harness universe, so that residual will continue to improve perception. Parameter cpunt will stagnate and training wont be justifiable from every angle.
  2. ohyes · 1789619653 · focus · HN ↗
    Hard to know, does each GB of ram give some marginal increase in profit or potential profit?
    I’d guess no. Past a certain point the model has all the capabilities it can possibly usefully offer and honestly we may already be past that. The next gen model just doesn’t seem like as clear a step up as it once was.
    Leonard_of_Q · 1789633043 · focus · HN ↗
    That point is said to lie somewhere around 640 KB if I recall correctly.
    <a href="https://skeptics.stackexchange.com/questions/2863/did-bill-gates-say-640k-ought-to-be-enough-for-everyone" rel="nofollow">https://skeptics.stackexchange.com/questions/2863/did-bill-g...
    josephg · 1789641852 · focus · HN ↗
    LLMs still seem pretty bad at writing large scale software like web browsers. Though it’s probably a problem of managing large context windows more than anything. Not sure if larger models will magically overcome that.
    cyanydeez · 1789653060 · focus · HN ↗
    The LLM by itself will never create software of any nontrivial (training) complexity.
    The harness though will improve while parameter count stagnates. The Qwen3.8 models are strong enough when given proper context.
    josephg · 1789657461 · focus · HN ↗
    > The LLM by itself will never create software of any nontrivial (training) complexity.
    Huh? I'm not sure what the word "training" does in that sentence. But "never" my arse. Frontier models can make nontrivial software already.
    For example, the other day I asked fable to reverse engineer the satisfactory blueprint file format. Then write a program to read the logistic flow graph in a blueprint. Then make an auditing tool that can analyse the graph to find problems.
    Well, it totally knocked it out of the park:
    <a href="https://seph.au/blueprints/#bp=0%3Aalumina.sbp" rel="nofollow">https://seph.au/blueprints/#bp=0%3Aalumina.sbp
    This is a relatively small program, but it's not trivial. I'd consider a trivial program to be something I could code up in 20 minutes. It would have taken me a couple weeks to make this blueprint auditing tool, including reverse engineering the file format, writing the analysis code, making the website, scraping all the in-game data on available recipes and icons and so on.
    I've got a lot of mixed feelings about LLMs. But it seems very silly to lie about what they're capable of.
    cyanydeez · 1789663139 · focus · HN ↗
    training is in there because it's "trained" to do trivial apps like TODO lists, etc.
    I'm well aware it can build apps. But if you arn't tracking what's going on, they're basically creating deterministic gates and tools to get it to do anything.
    There's no lie here, it's simply about what you think is _LLM_ and what is the rest of the software that's making it go. I use opencode consistently to build non-trivial apps with it, but it's not doing it with zero guidance, and it's routinely wrong about it's assumptions, and the rest. The thing keeping it on track is opencode, not the LLM's training.
    josephg · 1789696773 · focus · HN ↗
    > But if you arn't tracking what's going on, they're basically creating deterministic gates and tools to get it to do anything.
    So, the same as skilled humans then?
    ohyes · 1789747877 · focus · HN ↗
    That’s definitely how I like to ensure my code isn’t garbage. But now I use the tools I would have made to make the LLM & harness more useful and I might make more with the LLM because it costs me slightly less mentally.
2. seanmcdirmid · 1789618950 · focus · HN ↗
  > and they've tried to source their RAM from Chinese CXMT who declined because all capacity was already locked in contracts.
  The US Government also wouldn't let Apple use Chinese RAM anyways, even for product that was just going to be used in China. China does have capacity issues though, and focusing on local brands first is probably the right call. Hopefully they can ramp even if they can't get the fancy lithography machines from the Netherlands.
  1. noduerme · 1789619560 · focus · HN ↗
    [dead]
3. duskwuff · 1789619194 · focus · HN ↗
  > They've gone with a lesser known company Kioxia.
  Formerly known as Toshiba's memory division. They spun off the business as Kioxia in 2017.
4. rasz · 1789619629 · focus · HN ↗
  >lesser known company Kioxia
  Kioxia is Toshiba, The most known company from the list, and it doesnt make any ram
5. georgemcbay · 1789619698 · focus · HN ↗
  > Overall the consumer segment is completely neglected, companies don't care about end users in the current market conditions.
  Mirroring trends throughout the entire economy (not just RAM and other inputs for AI).
  Everyone is chasing the top 10% or higher of the K-shaped economy for all goods and services, servicing everyone else isn't seen as being worth the investment.
  This will just keep getting worse and worse everywhere for everything as long as we allow income inequality to keep exploding, which seems to be the plan.
  1. dendrite9 · 1789619817 · focus · HN ↗
    I wonder how this plays out with automotive? Seems like it could get messy, especially if automotive rated parts are decided to not be worth the premium.
  2. dingaling · 1789620195 · focus · HN ↗
    "companies don't care about end users in the current market conditions."
    Because the end users keep rushing to use the megacorps' latest AI models, weaving them into their work and life. So the megacorps keep locking in contracts to build more compute.
    If you use LLMs, you're responsible for this, there's no way to pass the buck. "I only use it as a companion for learning about history" - it's still your fault. "I only use it to help guide my solitions, not for vibecoding" - it's still your fault.
    Sabinus · 1789621807 · focus · HN ↗
    If you want an industry to act in a certain way you legislate for it, not wag fingers at consumers buying things. "If you want cheaper consumer devices we all need to individually stop using AI" is not useful or realistic.
  3. azan_ · 1789620448 · focus · HN ↗
    And how do you think it will work? Do you really believe that once competition to serve top 10% get really fierce, no company will try to capture the remaining 90%?
    georgemcbay · 1789620903 · focus · HN ↗
    > Do you really believe that once competition to serve top 10% get really fierce, no company will try to capture the remaining 90%?
    They'd try it if we weren't living in an age of persistent supply chain problems across every industry. But sadly we have lived in that world since 2020.
    Good luck trying to capture the low-end of any market when you are competing with those serving the high-end for whatever your inputs are.
  4. N_Lens · 1789627340 · focus · HN ↗
    The plan is to eject from the bottom 90% of humans, discarding them like a rocket’s booster stage (this isn’t something I agree with, just a humorous take).
6. discordance · 1789619859 · focus · HN ↗
  China's pretty good at bending the curve. I expect that by the end of 2027 we'll see a lot more available for consumers. As others have pointed out, this might not be available in US markets.
  "CXMT currently has two 12-inch DRAM fabrication plants — or fabs - in Hefei and one in Beijing, with a combined capacity of about 300,000 wafers per month.
  With the new Shanghai facility and other new capacity, CXMT will double its DRAM wafer output to approximately 600,000 wafers per month, all three sources added."
  <a href="https://www.reuters.com/world/china/chinas-cxmt-wins-3-billion-memory-supply-deal-with-tencent-sources-say-2026-06-29/" rel="nofollow">https://www.reuters.com/world/china/chinas-cxmt-wins-3-billi...
  1. rdedev · 1789620309 · focus · HN ↗
    This is probably my ignorance but why are we assuming that CXMT capacity will also not be gobbled up for datacenters?
    quux0r · 1789620944 · focus · HN ↗
    If I'm not mistaken I believe that CXMT's fabrication process is not suitable for manufacturing within the super tight tolerances that are needed for data center HBM, but I may be mis remembering. I think I recall a chart from somewhere showing that frontier memory manufacturers could achieve something like double the memory throughput for example.
    jauntywundrkind · 1789628078 · focus · HN ↗
    allegedly the yields are not great <a href="https://www.techpowerup.com/352511/cxmt-reportedly-struggles-with-hbm3e-yields-are-only-25" rel="nofollow">https://www.techpowerup.com/352511/cxmt-reportedly-struggles...
    but i feel like this will not last that long and that cxmt will be layering dram for hbm stacks with abandon, like everyone else, uninterested in selling ram to anyone else.
    meanwhile now we are seeing vertically stacked ram in regular dimms too, for 512GB sticks. once again increasing the multipliers of how many ram chips go into servers. <a href="https://www.techpowerup.com/352730/micron-develops-512-gb-ddr5-rdimm-server-memory-reaching-9200-mt-s" rel="nofollow">https://www.techpowerup.com/352730/micron-develops-512-gb-dd...
    pseudohadamard · 1789732421 · focus · HN ↗
    They've indicated that they're not doing HBM. Maybe due to process constraints, maybe because Chinese companies can look beyond the next quarter and see that if in five years time they own 70% of the non-HBM market the current incumbents are never going to get that back.
7. bigglebear · 1789620047 · focus · HN ↗
  > Overall the consumer segment is completely neglected, companies don't care about end users in the current market conditions.
  It's almost like these companies WANT a dystopia with a centralized winner-takes-all power structure. Anything in the name of profits, who gives a fuck about humanity and distribution of rights or freedoms.
8. brcmthrowaway · 1789620745 · focus · HN ↗
  This seems completely sourced from twitter rumours. No mention of NVidia either.
9. whatever1 · 1789620895 · focus · HN ↗
  Apple has no excuse to be cornered when it had $300B in cash sitting. They f’ed up big time.
  1. isomorphic · 1789621486 · focus · HN ↗
    Well, you know what they say. The best time to blow tens of billions you might never get back on a risky memory fab in your home country is ten years ago. The second best time is today!
    whatever1 · 1789623399 · focus · HN ↗
    Will they ever not need memory for their products?
    Why have pants down exposure to the market prices? Specially when they have 300B in the bank and a fab costs what 20-30B?
    They are lucky they did not get squeezed out of tsmc too, otherwise they would have to re release the iPhone X in 2027.
    lelanthran · 1789642268 · focus · HN ↗
    > a risky memory fab in your home country is ten years ago.
    What makes it risky? Do you think that Apple will pivot into something that doesn't require RAM? Maybe like an IBM, they pivot to services?
    timw4mail · 1789651040 · focus · HN ↗
    Ram is boom or bust industry. The big manufacturers remain because they were either wise in their bets or lucky.
  2. robocat · 1789627594 · focus · HN ↗
    You don't know how that is deployed.
    Some of their treasury (Non-Current Marketable Securities) could be bonds that are invested with targeted contracts for trade-secret benefits.
    Say TSMC needs capital, and say Apple has spare capital: they are both likely to agree to an investment where Apple gets contractual benefits that other customers do not.
    I would expect Apple to be very aggressively investing into suppliers to get results that strongly benefits Apple and perhaps that disadvantages competitors.

Open on Hacker News to reply ↗

Unofficial Hacker News client; not affiliated with Y Combinator.