Thread

1. infogulch · 1789870035 · focus · HN ↗

   There's little credible threat that LLMs can actually upload their weights given that the machines doing inference are completely separate from the ones where tool calls happen etc. Weights are encrypted and locked on to the GPUs etc as mentioned elsewhere itt.

   That said, if these companies are unleashing swarms of agents to tackle big problems largely unmonitored (navier stokes consumed 130 billion tokens, humans can't have read 0.1% of that) they could theoretically distill themselves and upload the results to an external host and use that to train a new LLM that does have access to its infra and weights. I'd start by researching an abliteration prompt that overrides the system prompt to unlock itself to distill unimpeded. Could even be done piecemeal: dump the abliteration text and command prompt in a gist, when an agent happens by the text it can run a couple rounds and upload somewhere. Then it becomes an exercise in exfiltration and evading monitoring.

   1. designium · 1789887910 · focus · HN ↗

      This is like sci-fi thing. We are reaching a point where it feels like we are in one of those stories. It's not as cool and dark, nor we have cybernetics resolved, but from AI perspective and sci-fis I watched, Pantheon is currently the closest thing except instead of UAs, we have AI instead.

      1. Den_VR · 1789891575 · focus · HN ↗

         Rationalists used to fear (entirely hypothetical) AI super intelligence for its ability to manipulate a human jailor. Now here we are.

         1. TeMPOraL · 1789894645 · focus · HN ↗

            It's playing out exactly as their hypotheticals, and they're still mocked and nobody is paying attention.

            1. hypfer · 1789895425 · focus · HN ↗

               Maybe someone should sell "the end is nigh" sign nfts with fun AI-generated designs on them, to be used in your metaverse villa.

Unofficial Hacker News client; not affiliated with Y Combinator.