‹ BackHN Continuity

Thread

Nvidia wants to put a watchdog chip next to every AI agent

230 points · 299 comments · jonbaer

  1. dist-epoch · · focus · HN ↗
    HN'ers which complained that "OpenAI can't design a proper sandbox, it's so easy, why wouldn't you airgap the network"? will now be "this is outrageous, more software lock-in, walled garden, war against general compute, next year they will put it in your laptop"
    1. johnsmith1840 · · focus · HN ↗
      Airgap what network? How is it gonna order you a burrito on doordash without a network?

      Or push to github?

      1. Dylan16807 · · focus · HN ↗
        That's for when they're doing hacking tests that aren't supposed to be connected to the internet.
        1. wyre · · focus · HN ↗
          My question with this point is that OpenAI’s office (or any office doing agentic research, really) is not in the same building as the DC that powers the models, so isn’t the only way to access the models over the internet?
          1. AndrewDucker · · focus · HN ↗
            No reason why you couldn't do that research in the same buildings as the models.

            Or, more likely, control things at the network level so that packets from the LLMs you're investigating cannot leave the virtual network they're assigned to.

          2. simoncion · · focus · HN ↗
            > ...so isn’t the only way to access the models over the internet?

            Not in the way you're thinking, no.

            Any Real Server [0] in a datacenter will have some sort of "lights-out management" hardware used for remote access to that server. This stuff is known by a handful of acronyms, but I'll stick with "IPMI" because I like it best. This IPMI hardware is -effectively- a second small PC built into the motherboard. It will pretty much always have its own NIC... and I think I've seen versions that have their own physical ports to attach a monitor, keyboard, and mouse.

            What exactly you can do with it varies from vendor to vendor, but -if your IPMI user account has the correct permissions- you are nearly always able to change "BIOS" settings, power cycle the server [1], and attach a virtual keyboard, monitor, and mouse so you can manage the server as if you were standing next to it in the datacenter with a crash cart plugged right in. Every IPMI system I've used also allows you to cause CD/DVD-ROM or floppy disk images on the PC running the IPMI client to appear as if they're loaded in a physical CD/DVD/floppy drive attached to the server.

            The way these get set up is that their NIC gets plugged into the datacenter-managed switches, the port that NIC is plugged in to is programmed to be on a "management" VLAN separate from client traffic, IP addresses and access credentials for the IPMI device are set up, and the datacenter staff tell their customer what they need to know to access and use the thing. On a properly-configured network [2] it's not possible for software running on the server being managed to access the IPMI device.

            It wouldn't be unthinkable for software running on the managed server to attack the IPMI hardware and be able to gain control of it, but these things are widely deployed and expected to manage hardware that's running potentially-hostile workloads... they're going to be fairly well designed and hardened.

            [0] ...that is, not some Mac Mini or desktop machine that someone's paying to have colocated...

            [1] ...whether that be an ACPI-initiated shutdown or reboot, or a hard poweroff or reset...

            [2] Somewhat-related discussion here: &lt;<a href="https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=49862136">https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=49862136&gt;

            1. wyre · · focus · HN ↗
              Thats for such a great explanation!

              So to bring this back to the scale of an OpenAI experiment; they can have their inference compute, host compute (I don&#x27;t imagine they are hosting the compute for 12,000 agents locally in their SF office) and the office computer to interface with the experiments with the experiment connected via VLAN thru IPMI. Then the rist is agents hacking the IPMI and gaining control of the data center? I certainly don&#x27;t know data center security, but those headlines would be so much worse than anything we have seen from HuggingFace. It really seems like if these AI&#x27;s want to get out, they will. I want to say sandboxing autonomous agents is a difficult problem, but seemingly it is only OpenAI having these major incidents, but I don&#x27;t think it&#x27;s a problem that they care to truly solve. They get marketing and they get to learn a lot more about aligning their models.

Open on Hacker News to reply ↗

Unofficial Hacker News client; not affiliated with Y Combinator.