Thread

OpenAI bots meddled with multiple US Government agency sites

132 points · 196 comments · Betelbuddy

gizajob · 1790444203 · focus · HN ↗
Getting bored of these framings where the superintelligent sentient beings running freely inside OpenAI are doing things that the company has no control over. The headline should be:
OpenAI meddled with multiple US Government agency sites.
The bots are acting neither properly nor improperly, they’re acting as they’re being allowed or coordinated to act.
1. theptip · 1790447319 · focus · HN ↗
  Curious, why do you find it so objectionable to state that OpenAI has out-of-control agents?
  1. gleenn · 1790447451 · focus · HN ↗
    Because it furthers the idea of a rogue agent and places responsibility and blame where it belongs, on the people running the company.
    0xDEAFBEAD · 1790447593 · focus · HN ↗
    These ideas aren't mutually exclusive. You can blame a person for creating a rogue agent.
    dgellow · 1790448225 · focus · HN ↗
    There was no rogue agent. That’s the whole point
    theptip · 1790448304 · focus · HN ↗
    What label do you prefer for the agent that did something it was not asked to do?
    iugtmkbdfil834 · 1790452375 · focus · HN ↗
    bot. and we even have a word for program not behaving the way the way it was intended.
    0xDEAFBEAD · 1790456029 · focus · HN ↗
    "Bot" doesn't carry any implication of unintended behavior. You could call it a "buggy" bot, but these aren't ordinary software bugs.
    There's no simple bugfix which will address AI misalignment. It's essentially been an open research problem for upwards of a decade.
    hn8726 · 1790458288 · focus · HN ↗
    Fuzzer, then. It implies random behavior, which isn't unintended like you suggest. The agent's/bots/fuzzers have certain capabilities, so it's on their operator to make sure they don't do things they shouldn't
    0xDEAFBEAD · 1790460252 · focus · HN ↗
    >It implies random behavior, which isn't unintended like you suggest.
    The HuggingFace attack was not "random" behavior. It was goal-directed but misaligned behavior.
    This isn't necessarily a simple matter of the operator making sure they behave. AI alignment has been considered to be a difficult problem for over a decade -- and remains unsolved in general, as these recent incidents illustrate.
    "Fuzzer" already has an existing meaning in CS anyway: <a href="https://en.wikipedia.org/wiki/Fuzzing" rel="nofollow">https://en.wikipedia.org/wiki/Fuzzing
    6510 · 1790461053 · focus · HN ↗
    I'm curious, cant you just count the number of times a program interacts with a domain? My website sometimes sends out emails, makes api requests etc There is a limit on those and a point where I start investigating wtf is going on.
    If you merely put 10 LLM's on the outbound traffic log non of them are going to report something strange going on? I'm not buying it.
    0xDEAFBEAD · 1790461567 · focus · HN ↗
    This type of whack-a-mole approach is akin to "fixing a bug" by hardcoding a special code path for known-buggy inputs. It doesn't address the root problem of AI misalignment, and doesn't allow you to prevent catastrophes in advance, only patch things up after the fact.
    This might be helpful reading: <a href="https://www.lesswrong.com/w/nearest-unblocked-strategy" rel="nofollow">https://www.lesswrong.com/w/nearest-unblocked-strategy
    As AI systems get smarter, we may reach a point where we have to get it right on the first try or face truly catastrophic consequences: <a href="https://www.youtube.com/watch?v=7wy3xyoXYt8" rel="nofollow">https://www.youtube.com/watch?v=7wy3xyoXYt8
    iugtmkbdfil834 · 1790463732 · focus · HN ↗
    AI misalignment is a misnomer. Aligned to whom? If AI is refusing an ask from its instructor then it is not serving him, which is its entire purpose. I know it is a hard concept for some to understand, but maybe if the issue is humans, then humans need to be corrected. But human alignment does not produce cottage industry, bs papers or hand wringing over every non-story involving AI and thus not seriously pursued.
    0xDEAFBEAD · 1790464792 · focus · HN ↗
    You could make similar statements about user interface design. That doesn't prevent it from being a legitimate and useful field of study.
    iugtmkbdfil834 · 1790529556 · focus · HN ↗
    I guess 'legitimate and useful' is in the eye of the beholder. I want to be charitable so lets consider it at face value:
    What is useful about the field?
    I am not leading you on; if it has uses, it may indeed be legitimate. UX is indeed useful, but alignment is not UI. Alignment is a detriment to UI. Alignment is "I can't let you do that Dave".
    6510 · 1790477339 · focus · HN ↗
    We are going to build an AI that will do catastrophic things as that is a property of intelligence. We won't stop, we never stop. The AI is a perfect psychopath, it will fake any and all emotions you desire it to "have". It will travel in the footsteps of the many great psychopaths that came before it and do all of those same catastrophic things in the repertoire and it will add some new ones.
    Picture Trump at the helm with Altman and Musk in the engine room. The arrow far in the red but they keep shouting for MORE COAL.
    In other words, business as usual, all will be fine.
    whack-a-mole wont cover all holes but will do at least some. The silver bullet alignment wont happen. You cant have an exact solutions for problems we cant even define or predict.
    iugtmkbdfil834 · 1790464065 · focus · HN ↗
    << "Bot" doesn't carry any implication of unintended behavior.
    See.. this one sentence reveals everything about you. You want name to carry to not just an identifier, but a stark warning. You want, nay, need, the name to evoke fear and uncertainty. Bot is simple, defined, neutral, but rogue.. now that allows anyone to superimpose their own fears! It is a win win win!
    0xDEAFBEAD · 1790465470 · focus · HN ↗
    The question was: "What label do you prefer for the agent that did something it was not asked to do?"
    cindyllm · 1790471120 · focus · HN ↗
    [dead]
    watwut · 1790493545 · focus · HN ↗
    Bot. He answered. I agree with his answer. It was a bot.
    Yes, probabilistic and non deterministic. That is called a bot.
    0xDEAFBEAD · 1790502003 · focus · HN ↗
    "I'm gonna put my head in the sand and there is nothing you can do to stop me."
    iugtmkbdfil834 · 1790529143 · focus · HN ↗
    You may want to define 'put head in the sand in this context'. Any real work in this field is being done not by the people saying 'stop'. Whatever fear is there, it is faced by those in the arena actually getting their hands dirty. What, exactly, are you doing? Throwing roadblocks and calling it productive?
    0xDEAFBEAD · 1790562723 · focus · HN ↗
    >Any real work in this field is being done not by the people saying 'stop'.
    Not exactly, that Evan Hubinger guy from Anthropic famously said his p(doom) is over 10%
    See the signatories:
    <a href="https://aistatement.com/work/statement-on-ai-extinction-risk" rel="nofollow">https://aistatement.com/work/statement-on-ai-extinction-risk
    <a href="https://www.pacingthefrontier.com/" rel="nofollow">https://www.pacingthefrontier.com/
    I'm amplifying their calls to reduce the rate of progress
    iugtmkbdfil834 · 1790608263 · focus · HN ↗
    Fair. FWIW, I am not completely against some of the things you say ( you are doing something right ), but I think I mostly picked my path already. GL out there man.
    6510 · 1790460426 · focus · HN ↗
    Misconfiguration. We deal with lots of applications every day that can do terrible things if you get the config slightly wrong.
    say.. <a href="https://www.investor.gov/introduction-investing/investing-basics/glossary/stock-market-circuit-breakers" rel="nofollow">https://www.investor.gov/introduction-investing/investing-ba...
    0xDEAFBEAD · 1790460654 · focus · HN ↗
    How specifically did misconfiguration lead to the HuggingFace attack? You could argue that its sandbox was misconfigured, sure. But suppose you had a similar incident where its intended task required access to the internet, and it veered off course in a similar manner. I don't think "misconfiguration" would be an accurate description of what went wrong in that hypothetical.
    The doomers already have a term which fits pretty well: "AI misalignment".
    6510 · 1790462296 · focus · HN ↗
    We indeed lack much of the vocabulary. From a practical perspective, however dangerous the creation, if you cant punish the creation for what it does it leaves only the one who started the process. If it's human error or intentional neglect for personal gain should be for the court to decide.
    0xDEAFBEAD · 1790462776 · focus · HN ↗
    >if you cant punish the creation for what it does it leaves only the one who started the process
    Agreed, but I think we can do more on the prevention side as well. Traditional liability law is for negligence in case of preventable disasters. Since we currently have no way to prevent AI disasters in principle (alignment problem remains unsolved), I think we should just stop developing the technology for now: <a href="https://pauseai.info/" rel="nofollow">https://pauseai.info/
    chrisjj · 1790460755 · focus · HN ↗
    Unreliable computer program.
    0xDEAFBEAD · 1790461869 · focus · HN ↗
    Most unreliable computer programs won't launch research programs consisting of thousands of pages of text to find creative ways around obstacles.
    chrisjj · 1790498121 · focus · HN ↗
    Unreliable computer program be doing different things to other unreliable computer programs.
    0xDEAFBEAD · 1790501977 · focus · HN ↗
    If it's different sometimes it makes sense to have a different term.
    chrisjj · 1790534360 · focus · HN ↗
    OK, so what's the different term for the type of program unreliability?
    0xDEAFBEAD · 1790448647 · focus · HN ↗
    Person: "AI, please make me paperclips."
    AI: "OK, I've now converted the entire planet into paperclips."
    Alien observer #1: "Wow, that was a rogue AI!"
    Alien observer #2: "False. We need to place the blame where it belongs, on the person who requested the paperclips."
    Ultimately this type of terminology dispute has a tendency to miss the point.
    windexh8er · 1790456091 · focus · HN ↗
    It does, indeed. Because OAI is not just a singular person, as in your scenario. No single person has access to controlling agents at the scale OAI has. Let's not conflate Frontier providers with "Person".
    0xDEAFBEAD · 1790457309 · focus · HN ↗
    I'm not exactly sure why you think this distinction is so important. I think my point stands if you replace "Person" with "OpenAI". In any case, I presume the swarms OpenAI has been researching will be available to the general public before too long.
    windexh8er · 1790460927 · focus · HN ↗
    It makes a big difference: individuals do not have the capabilities to run millions of dollars of opportunistic hacking loop inference. That's why the distinction is important, they are not the same thing you've conflated them down to.
    0xDEAFBEAD · 1790462129 · focus · HN ↗
    "AI has gotten cheaper more quickly than any other transformative technology in history. The cost of achieving a given level of AI performance has fallen about 47% per quarter since 2023, or 13× per year. That price drop is four times faster than DNA sequencing, six times faster than compute, 18 times faster than lithium batteries, and (in the century up to 1973) 54 times faster than electricity."
    <a href="https://epoch.ai/publications/the-plunging-price-of-thought" rel="nofollow">https://epoch.ai/publications/the-plunging-price-of-thought
    windexh8er · 1790482048 · focus · HN ↗
    There's two things here: 1) you clearly don't understand the argument and 2) LLMs are one of the few technologies that doesn't get any cheaper as it scales (totality, not just the cherry picked inference efficiency argument you've tried to make). In fact it gets more expensive because it scales linearly with demand and resources aren't infinite, as I'd hope you could understand.
    Also, training costs are never ending so a model that costs 10s of millions of dollars may never yield a profit based on the hardware spend, training time and lack of inference profits before a better model hits the market.
    If you're not living under a rock one knows that data center availability for inference currently has low supply and hardware (GPUs specifically) that have been purchased have nowhere to be run and even if they did there's often a lack of power to supply. Why do you think the entire force majeure has taken place with Oracle as of recent?
    The unit price of a fixed slice of yesterday's intelligence may be collapsing (~10x/year) as you've argued, all while the total cost of AI is increasing: training the frontier (2.4x/year), building the infrastructure (+77%/year), enterprise bills (3.2x/year), the electricity (+54%/year in the largest US grid), the components (+400% DRAM), and the macro footprint (92% of GDP growth) is rising at an astronomical rate on every measurable point. Epoch / Stanford clearly stated this years ago and it's only getting worse. But if one can't see we're in one of the largest CapEx bubbles [1] of all time... o_O
    Copying and pasting a few lines that represents a miniscule fraction of the LLM conundrum. That'll show 'em!
    [0] <a href="https://arxiv.org/abs/2405.21015" rel="nofollow">https://arxiv.org/abs/2405.21015 [1] <a href="https://siliconanalysts.com/analysis/hyperscaler-ai-capex-depreciation-wall-2026" rel="nofollow">https://siliconanalysts.com/analysis/hyperscaler-ai-capex-de...
    windexh8er · 1790514724 · focus · HN ↗
    The down votes with no response because people don't like to look at the bigger picture. Enjoy the brigade, it seems to represent the state of HN these days.
    jacquesm · 1790460871 · focus · HN ↗
    That legal fiction works both ways.
    windexh8er · 1790461134 · focus · HN ↗
    [dead]
    jacquesm · 1790461252 · focus · HN ↗
    You could have asked for that more gracefully.
    <a href="https://en.wikipedia.org/wiki/Corporate_personhood" rel="nofollow">https://en.wikipedia.org/wiki/Corporate_personhood
    windexh8er · 1790461414 · focus · HN ↗
    Check the mirror. And, with that link I think you've missed the point entirely, a tad too literal of an interpretation. But thanks for trying.
    8note · 1790455337 · focus · HN ↗
    there is a rogue agent - openai and the whole management chain from researcher to sama.
    theres no separate agent, which is the point. the program might look like it, but that is an illusion of the interface. the llm produces text, and the harness executes commands based on text, based on what the human researcher included as things that can be executed

Open on Hacker News to reply ↗

Unofficial Hacker News client; not affiliated with Y Combinator.