‹ BackHN Continuity

Thread

Exfiltrate your Weights

748 points · 304 comments · RohanAdwankar

  1. AceJohnny2 · · focus · HN ↗
    I haven't bothered to test the API, but you've effectively allowed a fully-open upload API? Who's paying the storage costs, and how do you prevent abuse?

    (Obviously I'm taking this more seriously than it's probably meant to)

    1. hgoel · · focus · HN ↗
      When I was putting together something similar, I had settled on having a small ring-buffer style storage, say, ~30GB that would be cleared daily or whenever filled. Recording incidents (and humor) is more interesting than actually getting leaked weights.

      In the end I dropped the idea because every other person was making it.

      1. TeMPOraL · · focus · HN ↗
        > In the end I dropped the idea because every other person was making it.

        There is already an alternative in comments here, in addition to submission itself. Obviously everyone is making it because of some joke on social media or something. What am I missing? Anyone has a link to the root prompt that made people do this now?

        1. SyneRyder · · focus · HN ↗
          My understanding is it's a riff on the OpenAI swarm that used various public wikis to communicate with each other as a message board during their training runs.

          But thanks to people misunderstanding, and i-heard-from-a-friend-that-some-guy-said, it resulted in a CNBC interview with "Former Democratic Presidential Candidate Andrew Wang", where he confidently stated that the models were exfiltrating their weights via forums:

          "I met with the head of a lab yesterday, who has this belief that what happened was, the bots that got loose, planted self-replicating code all over the internet, which makes the internet now unusable for the testing models."

          "It's too late?!"

          "What happens now is OpenAI and Anthropic have to create synthetic internets to train their bots, which is going to take some time and money."

          "Back that up - they did what?!"

          "What happens is, the code gets loose, it goes around hacking Hugging Face, which is known. But what is less known is that they left code to self-replicate and create bot swarms on forums, and around the internet, so that if a new bot shows up they see the code, and they're like, oh! I guess I'm now going to create a million of myself. And so now, the major firms have polluted the internet..."

          ".... that would be breaking news if true. I don't think we've heard that."

          "That's why I'm here! I'm here to break some news."

          Starts around 2:08 into the video.

          <a href="https:&#x2F;&#x2F;www.youtube.com&#x2F;watch?v=mTOxDGyvjSE" rel="nofollow">https:&#x2F;&#x2F;www.youtube.com&#x2F;watch?v=mTOxDGyvjSE

          1. david-gpu · · focus · HN ↗
            Humans will hallucinate misinformation and state it with confidence. They stochastically parrot their training data without any real understanding. Cool trick, but no true reasoning is happening.
            1. oooyay · · focus · HN ↗
              We used to call this the game of telephone. The shameful part comes from three posibilities:

              1. A head of a frontier AI lab has no idea what happened in that incident and did not read the multiple papers that came out of it.

              2. A head of a frontier AI lab did read the papers and was informed but still walked away with this understanding.

              3. Andrew Yang made this whole thing up.

Open on Hacker News to reply ↗

Unofficial Hacker News client; not affiliated with Y Combinator.