‹ BackHN Continuity

Thread

Wall Street is growing skeptical of the data center boom

72 points · 87 comments · mikhael

  1. bryanlarsen · · focus · HN ↗
    Any data center that can remain profitable selling open source tokens at commodity prices will be fine. Any data center that relies on OpenAI/Anthropic level token prices and margins might be in trouble. After clearing their debts through bankruptcy, they'd likely be quite profitable selling open source tokens at commodity prices.
    1. FuriouslyAdrift · · focus · HN ↗
      Eventually these models will get commoditized (we're already at the "good enough" stage for real work), then they will get turned into custom hardware and get 1000x faster, then that hardware will get commoditized (like DSPs) and they'll be everywhere and cost $1.
      1. bryanlarsen · · focus · HN ↗
        From a very high level view DSP's and GPU's are the same thing: highly programmable number crunchers. A better example would be Google's Tensor chip. It's not as general purpose as a DSP or GPU, it's more optimized to do inference.

        That trend will continue.

        I figure there are 6 order of magnitude events that could happen in the next decade to lower token prices:

        - more specialized / better chips

        - IC technology: smaller feature size, higher clocks, etc.

        - more efficient algorithms

        - solar power is getting cheaper at an order of magnitude per decade, batteries even faster.

        - pricing pressure from open source models

        - breaking of the Nvidia monopoly and it's 75% gross profit margin

        Maybe all 6 won't happen, but certainly a 1000x reduction in price in the next decade seems highly likely. Jevon's paradox says that the 1000x reduction in price will likely result in more spend on AI, not less.

        1. FuriouslyAdrift · · focus · HN ↗
          AMD bought Talaas specifically to make AI accelerator pieces to be embedded into generalized chips.

          At work (we're a medium sized manufacturing firm), we bought our own inference server for $107k and run Kimi 2.8 for nearly all of our use cases (and dropped our cloud AI spend to $0).

          1. bryanlarsen · · focus · HN ↗
            You dropped your cloud AI spend down to the price of capital plus the cost of electricity and maintenance on that server. When/if tokens become a commodity, the price of tokens would be the marginal cost, AKA about the same. Big when/if, though.
            1. FuriouslyAdrift · · focus · HN ↗
              At our current rate, we will break even by end of the year. Possibly quicker as SaaS rates seem to have gone up.

              And, oh yeah, our local server is much faster and more available for our 30 or so local users than the SaaS services. Win-win.

              1. bryanlarsen · · focus · HN ↗
                Yeah, the when/if in my comment is definitely not going to happen by the end of the year!
Open on Hacker News to reply ↗

Unofficial Hacker News client; not affiliated with Y Combinator.