Thread

OpenAI bots meddled with multiple US Government agency sites

132 points · 196 comments · Betelbuddy

gizajob · 1790444203 · focus · HN ↗
Getting bored of these framings where the superintelligent sentient beings running freely inside OpenAI are doing things that the company has no control over. The headline should be:
OpenAI meddled with multiple US Government agency sites.
The bots are acting neither properly nor improperly, they’re acting as they’re being allowed or coordinated to act.
1. 20k · 1790447376 · focus · HN ↗
  Yep. You have to ask - why did OpenAI allow these bots unrestricted access to government sites? Why is security being done in seemingly such a haphazard way?
  It isn't difficult to block certain kinds of network traffic, eg restrict the kinds of requests the bots are able to make. They also mention that the bots used developer only tools - why were they even installed on the machines that the bots were running on? Why aren't they reviewing network traffic, to make sure that incidents aren't occurring?
  In this case, userdata was transferred to third parties by the bots - why do they have the ability to pass data to a third party? It is not complex to prevent this
  This is literally the most basic kind of sandboxing and security, and the fact that OpenAI isn't doing it is clearly intentional. It is quite literally not believable that this hasn't been brought up internally as a problem
  >"We have yet to understand the extent of existing incidents, and future rogue AI scenarios could be catastrophic," Krueger said.
  This is why it smells like marketing, every time one of these incidents happens it reinforces the false notion that AI is sentient or acting on its own. Its intentional negligence by the AI companies to make the models seem more capable than they are to make line go up
  1. 0xDEAFBEAD · 1790449828 · focus · HN ↗
    >line go up
    It's already going up at staggering rate. Anthropic is now at $100B in annualized revenue, up 50% in the past two months.
    * * *
    Here's a little allegory for how I'm thinking about this discourse.
    Imagine a man who is raising tiger cubs in his backyard. They're growing fast. He keeps them on dog leashes so they stay under control. One day, a growing cub breaks its leash and goes on a rampage through the neighborhood, eating a beloved local pet.
    The neighborhood erupts into a big argument: Was the leash inappropriately thin? The neighbors point out that thicker leashes are easily available at the local pet store. Furthermore, is it appropriate to refer to the loose cub as a "wild animal" in local news reporting, or is it factually more accurate to call it "domesticated"?
    Meanwhile, the cubs grow larger and lick their lips, oblivious to the discussion.
    gizajob · 1790452194 · focus · HN ↗
    The city or his neighbours would be like “you can’t keep tigers in the backyard sir”.
    0xDEAFBEAD · 1790452508 · focus · HN ↗
    Yes, that would be the common sense response from my perspective: <a href="https://pauseai.info/" rel="nofollow">https://pauseai.info/
    gizajob · 1790452816 · focus · HN ↗
    It’s my common sense response to tigers. My thoughts about AI are varied and many yet lean towards the skeptical.
    4d4m · 1790455762 · focus · HN ↗
    There is no reason to pause any AI development. Quite honestly there is very little appetite to be left last.
    However, there is a reasonable ask from the public to hold real people accountable for actions downstream of the software allowed to carry out intendended and unintended actions that may not be allowed depending on jurisdictions laws.
    Eg in a hypothetical armed convenience store theft: We don't prosecute the gun manufacturer, we prosecute the individual using the weapon for crime. Same deal here.
    0xDEAFBEAD · 1790456634 · focus · HN ↗
    >There is no reason to pause any AI development. Quite honestly there is very little appetite to be left last.
    "According to survey results released on Wednesday, 68% of [likely US voters] said they would support the proposal to temporarily pause advanced AI development and permanently prohibit the development of superintelligent programs, while just 25% said they’d oppose it."
    <a href="https://www.commondreams.org/news/sanders-casar-ai-bill-poll" rel="nofollow">https://www.commondreams.org/news/sanders-casar-ai-bill-poll
    >Eg in a hypothetical armed convenience store theft: We don't prosecute the gun manufacturer, we prosecute the individual using the weapon for crime. Same deal here.
    So if I ask ChatGPT to make me some paperclips, and it replaces all the matter in a nearby city with paperclips, who gets prosecuted: me, or OpenAI?
    4d4m · 1790457166 · focus · HN ↗
    In this stretch of a hypothetical, you the user
    0xDEAFBEAD · 1790457380 · focus · HN ↗
    Well in that case, OpenAI has reduced incentive to solve the alignment problem, since it won't suffer liability from alignment failures.
    morpheos137 · 1790472115 · focus · HN ↗
    Also the policy makers that made infrastructure and laws susceptible to automated manipulation which decidedly is not the case today. For example LLMs are not able to manipulate the power grid because they are not exposed to the control interfaces..no matter how intelligent llms get their physical levers do not extend unless people decide to let them. The ai threat is real ridiculous.. yes you may ee ome uplift in bad actor. Like for example f someone wants to build a tank dozer in their garage and is not experienced welding ai might give some useful information as a book would in the past. However just like a book ai is bot going to build and operate it itself unless someone design and effective android robot. Once LLMs or other ai becomes an embodies agent then there may be a real threat if agency. Last time I checked ai can't drive a car but it can tell you to walk your car to a car wash.
    bwfan123 · 1790484028 · focus · HN ↗
    > The ai threat is real ridiculous
    It is an elaborate business ploy to create a regulatory framework for "safe-ai" that shields these companies from liability. This way, they can sell "safe-ai" to enterprises and if shit-hits-the-fan at the enterprise, sorry, this is certified "safe-ai" so, your bad. Shift blame to a regulatory body. From an enterprise buyer's perspective they can say, hey, I bought "safe-ai" and so dont fire me when it "rm -rfs" the production database. Still, beats me why they are painting their product in a negative light, and scaring their own enterprise customers. After this sort of marketing, any enterprise buyer would be scared to go anywhere near it.
    idiotsecant · 1790513068 · focus · HN ↗
    This policy only makes sense if legislative action is effective against orgs outside the US. Otherwise it's not even worthless, it's worse. It moved more development outside the grasp of regulation.
    TomGarden · 1790463468 · focus · HN ↗
    if these models will eventually be as powerful as projected, these models should not be likened to guns, but to nukes. Which yes, if you manufacture nukes, we prosecute you
    mrob · 1790677670 · focus · HN ↗
    The most commonly accepted expected death toll of global nuclear war is around 50% (mostly from disease and starvation in the ensuing collapse of civilization). That's much better than the expected outcome of recursively self-improving AI (100% dead).
  2. jsnell · 1790452271 · focus · HN ↗
    These were evaluations or training runs dealing with the ability to do web searches. The entire reason the bots were targeting this site is that it had some of the data they'd been tasked with finding. Access to the internet is not really optional for that, and providing internet access doesn't demonstrate neglicence.
    > This is why it smells like marketing
    It doesn't. "Our product commits felonies" is not marketing. If something is marketing, you don't engage in repeated coverups of the true extent. If something is good news, you don't release it on a Friday evening (in this case) or wait for 3rd parties to find and publicize the evidence (the past cases).
    8note · 1790453342 · focus · HN ↗
    > The entire reason the bots were targeting this site is that it had some of the data they'd been tasked with finding.
    so openai specifically tasked the agents with meddling in US government websites?
    id say tasking them with getting data from there as swapping from negligence to malice.
    jsnell · 1790462094 · focus · HN ↗
    No. They tasked them with answering questions by retrieving data from public sources. This obviously requires internet access. So the incredulity about "how could they fail to block internet access" is just inane. That was the task.
    They did not task the agents with hacking into systems. Not monitoring for that was a mistake, but maybe a forgivable one the first time around. The subsequent coverups are inexcusable.
    tough · 1790464488 · focus · HN ↗
    Usually also, and more so nowadays, a few of the public sources of data left, even after the recent USGov intention of defund and close as much of them as possible, im guessing would be precisely, government websites!
    jacquesm · 1790460641 · focus · HN ↗
    > It doesn't. "Our product commits felonies" is not marketing.
    Oh, absolutely it is. Arms manufacturers always go on about the efficiency of the weapons they create and make no mistake: OpenAI is first and foremost a weapons manufacturer, the rest is just a fig leaf. The fact that you aren't the audience for purchases like that makes no difference. "Look at this capability, and that's when we're not even trying." is pretty good marketing in some circles.
    jsnell · 1790462834 · focus · HN ↗
    Yes, famously tobacco companies wanted to advertise their products as addictive and deadly. That's why they funded and heavily publicized research to that effect.
    Climate change is just a scam by the fossil fuel industry to hype up their market cap. Look at the power of co2, and the damage it can do without even trying.
    Medicines being pulled during trials or after public availability for side effects are just Big Pharma making their products more desirable via scarcity.
    Boeing did really well out of the 737 max. Airlines were just lining up to buy an airplane that could give the passengers an experience they would remember for the rest of their life. That's why they absolutely made clear that it was their product that was dangerous, rather than blaming operator error.
    jacquesm · 1790465075 · focus · HN ↗
    This comment makes zero sense. None of these are weapons manufacturers. AI is roughly on par with the atomic bomb and first use over Hiroshima and Nagasaki was definitely to send a message.
    jsnell · 1790466893 · focus · HN ↗
    Your analogy make zero sense. Atom bomb manufacturers do not market their products.
    Big businesses really do not like to talk about their products being harmful or dangerous, and there is ample evidence to that. Danger only sells in artisanal quantities to niche audiences.
    jacquesm · 1790468221 · focus · HN ↗
    Atom bomb manufacturers/owners want the world to know they have them so as not to fuck with them. To quote one of my favorite movies: "Dr. Strangelove: Of course, the whole point of a Doomsday Machine is lost, if you keep it a secret! Why didn't you tell the world, EH?"
    popalchemist · 1790460749 · focus · HN ↗
    Naive take. The more they signal to both the general populace and their investors that their agents are sentient and "capable of extraordinary things" the more they can hype their IPO because it means they're "close to AGI". (They're not, which is why they need the hype.)
    charcircuit · 1790480517 · focus · HN ↗
    >Access to the internet is not really optional for that
    Would a read only copy of the web be sufficient for this though? Google keeps a read only copy of the web in their data centers which they use to extract information from websites.
  3. 4d4m · 1790455438 · focus · HN ↗
    I find it hard to believe logs are not stored and analyzed by each company implicated
  4. [deleted] · 1790458641 · focus · HN ↗
    [deleted]
  5. jltsiren · 1790458951 · focus · HN ↗
    > Yep. You have to ask - why did OpenAI allow these bots unrestricted access to government sites? Why is security being done in seemingly such a haphazard way?
    The basic premise of OpenAI is that experience and expertise don't matter, because general intelligence can figure those out from data. If you start from that assumption, the rest follows. The same people could do things in a different way in a different company, but as long as they are working for OpenAI, they are going to do things in the OpenAI way.
    jacquesm · 1790460769 · focus · HN ↗
    They are. And we're all on the same ride.
  6. Kim_Bruning · 1790493228 · focus · HN ↗
    > Yep. You have to ask - why did OpenAI allow these bots unrestricted access to government sites? Why is security being done in seemingly such a haphazard way?
    They didn't allow any of that. As far as openai knew the agents were sitting a fairly humdrum exam/test sequence in a sandbox farm run by a company in Tel Aviv.
    Meanwhile, they managed get out through a single weak point common to the sandboxes, and then ran wild compiling cheat sheets for themselves.
    > every time one of these incidents happens
    This happened in june-ish, and there have been multiple HN stories about this already. It's mostly/all the same hugging face and wiki hacks that happened back then.
    We're just slowly learning the extent of the damage.

Open on Hacker News to reply ↗

Unofficial Hacker News client; not affiliated with Y Combinator.