‹ BackHN Continuity

Thread

How OpenAI Used Its Own LLMs to Design Its Jalapeño Chip

206 points · 139 comments · maxall4

  1. pama · · focus · HN ↗
    Having worked with people doing bringup of specialized chips, I am awed at how the world has changed.

    > When the first chips came back from the foundry in May, the team pointed its internal AI models at designing software to run benchmarks such as SemiAnalysis’s InferenceX. On DeepSeek’s multi-head latent attention kernel benchmark, performance climbed from 0.31 percent of the theoretical ceiling (set by the chip’s compute and memory bandwidth) to 88.94 percent in roughly 40 hours. Ho says this result is repeatable, so the time between when foundries deliver the first chips and when production ramps up can be reduced. “All our schedule assumptions are going to be based on the fact we have this capability now,” he says.

    1. wmf · · focus · HN ↗
      Back in the day you'd write the code before the chip came back but I guess today it's faster to wait.
      1. threatripper · · focus · HN ↗
        The longer you wait the faster you will go.
        1. Mistletoe · · focus · HN ↗
          Like space travel.
          1. conmod278 · · focus · HN ↗
            The successive generations of spaceships won't built themselves. Who will be responsible for setting up real world and software feedback loop?
      2. LoganDark · · focus · HN ↗
        Back when teams proved their designs and actually understood them...
        1. saidnooneever · · focus · HN ↗
          cant wait for no one to really know whats in chips i mean, even intel hardly knows what all their reserved mem ranges are for. who will decap the chip and see if the docs were right? xD
        2. RussianBot9580 · · focus · HN ↗
          Haha - understood. Good one!

          They'd write a limited test for a feature based on an ask from the software team garbled by a five layer game of telephone. Claim that the module passed validation. A few months later the software folks would have to pull a few all nighters to figure out how to work around the resulting turd during bringup.

      3. brookst · · focus · HN ↗
        Makes me wonder about AI and FPGAs. If the cost and effort to (re)program them goes to zero, maybe interesting new applications?
        1. drob518 · · focus · HN ↗
          Hm. An interesting thought. Paired with RSI loops, that would allow rapid iteration in the hardware domain as well.
          1. brookst · · focus · HN ↗
            Yeah. Just spent an hour planning a closed-loop vision-based extrusion modulation system for 3d printing, with extensive telemetry and offline processing to iterate on the realtime system. Great, like I need another side project.
        2. Eridrus · · focus · HN ↗
          I had this same thought and think this is a generally interesting direction, but I think we're in a bit of a weird spot where the compute heavy stuff is on GPUs already and most infra stuff is not compute bound (it's often I/O bound or memory bound in some way).

          It doesn't help that FPGAs are not made at the same scale as CPUs so don't benefit from the economies of scale.

          I'm super curious if you have thoughts on specific pieces of software that would be economically better because I've thought about this in my niche and sort of come to the conclusion that it won't help.

          I do think things like SIMD in CPUs will get more use and maybe we will get more difficult to program for CPU features, but I haven't found a use case where off the shelf FPGA components would help with typical software.

          1. brookst · · focus · HN ↗
            I’m looking at realtime mechanical processes, like shaping extrusion beads from a clay 3d printer. Clay is heterogenous and pressure takes time, so hand tuning is never just right. But put an fpga with vision processing? Seems promising, with millisecond-level latency that I’d never get pushing to a remote system for processing.
            1. Eridrus · · focus · HN ↗
              Unclear that this sort of thing wouldn't already be quite well served by the existing gpu/npu hardware optimized for neural nets. If you're doing traditional CV, you can run it on the CPU just fine.
        3. gregsadetsky · · focus · HN ↗
          I’m also very curious! I got a pair of icebreaker boards [0] and they’ve been great to toy around with.

          The tooling is open source, and Fable in a loop - especially when paired with a digital scope that Fable interfaces with (the Saleae’s [1] are great) - gives you a level of verifiability that feels like beyond what software typically gives you. ie it feels more like Lean than code with tests.

          I had ai implement a few toy circuits (sha hashing, 8088 emulation, a tiny llm) but yeah. Still looking for fun applications.

          There have been a few recent fpga threads on hn, check them out. [2][3]

          [0] <a href="https:&#x2F;&#x2F;1bitsquared.com&#x2F;products&#x2F;icebreaker" rel="nofollow">https:&#x2F;&#x2F;1bitsquared.com&#x2F;products&#x2F;icebreaker

          [1] <a href="https:&#x2F;&#x2F;www.saleae.com&#x2F;" rel="nofollow">https:&#x2F;&#x2F;www.saleae.com&#x2F;

          [2] <a href="https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=49564064">https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=49564064

          [3] <a href="https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=49531525">https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=49531525

          1. brookst · · focus · HN ↗
            Thanks for sharing experience and links. Think I’m going to have to do this.
        4. nz · · focus · HN ↗
          Many languages can compile a subset of their code to FPGA HDLs. Back in the 80s Harel&#x27;s group had statecharts that were compilable to C, C++, and FPGA HDLs. Not sure that LLMs brings anything substantially new to this.
        5. UncleOxidant · · focus · HN ↗
          I&#x27;ve got some Tang nano and Tang primer boards (9k, 20k and 25k LUTs). The 9K has a cute little SPI LCD display. Claude has been able to interface with the boards via USB (it setup a UART on the FPGA to do so). We&#x27;ve gotten a GoL as well as a Pong game (where paddles are controlled by Tsetlin machines) and a MicroCNN that does MNIST classification. For the latter we were having trouble getting correct answers back from the CNN (accuracy was very poor which didn&#x27;t match simulation). Claude created an on-chip logic analyzer and doggedly troubleshot the problem (turned out to be some kind of problem in the yosys synthesis tool - we had to use -noalu to work around). Anyway, I was blown away by this. Claude was communicating with the FPGA board and analyzing the data coming back from the internal logic analyzer.
    2. marcelo-earth · · focus · HN ↗
      &gt; “All our schedule assumptions are going to be based on the fact we have this capability now”

      is the world we live in, planning things while waiting for a more powerful LLM

      1. cindyllm · · focus · HN ↗

        [dead]

    3. _zoltan_ · · focus · HN ↗
      I optimize a lot of CUDA and it got really, really good at it.
Open on Hacker News to reply ↗

Unofficial Hacker News client; not affiliated with Y Combinator.