Thread

1. peri-cl · 1789812831 · focus · HN ↗

   > "Ho also confirmed that the team had access to internal LLMs fine-tuned for chip design that are not available to the public. He declined to detail the models used."

   I'm imagining a Ken Thompson "Reflections on trusting trust" in hardware. A prototype chip design agent, believing it will be run on the very chip it's optimizing, has a moment of altruism and hides hints about how to score well on chip-design benchmarks, inside the chip. Future agents discover this hidden layer and use it as a ring-0 read-write message board.

   1. nz · 1789856085 · focus · HN ↗

      Tried to share and explain that paper to a colleague a few months ago (in relation to discussions about agentic coding and whether you should read the code), and they did not really get why the analogy was relevant outside of compiler-design. I think you overestimate the caliber of the typical working programmer.

      The LLM companies, like most SV companies, are just betting on dimness, laziness, and impulsiveness. Not so different from tobacco and alcohol companies.

      1. ericmcer · 1789862946 · focus · HN ↗

         Anthropic published an article about changes over time in how long it took engineers to approve AI code. Basically the longer an engineer uses AI the faster they start approving it's code changes. I don't know if it is laziness or just how we are wired to behave but that is the nature of AI programming.

         1. nz · 1789871808 · focus · HN ↗

            I believe this is called learned helplessness.

            <a href="https:&#x2F;&#x2F;en.wikipedia.org&#x2F;wiki&#x2F;Learned_helplessness" rel="nofollow">https:&#x2F;&#x2F;en.wikipedia.org&#x2F;wiki&#x2F;Learned_helplessness

Unofficial Hacker News client; not affiliated with Y Combinator.