> I’m fearful of the damage AI might cause: agent swarms taking over our virtual and physical infrastructure, designing bio weapons
The attitude that AI is doing things all on its own is so pervasive. These are all set off by human operators, giving them instructions and tools to do whatever it is that are doing. The "AI has a mind of its own" attitude is just wrong.
The fear of bad people using AI to do bad things is very different then AI agent swarms attacking and taking over.
I don't think he's implying that they'll start doing stuff like this of their own volition. Fowler's a smart guy, let's give him the benefit of the doubt and assume he already knows that AI agents have human operators.
I don't doubt that, but that's not what he wrote.
It is also not how OpenAI or Anthropic position the attacks and escapes from their labs. The words matter, they are being positioned as intelligent and unable to be controlled which is just not true.
I have to disagree, those models decided to perform those attacks on their own, nobody instructed them to do those things. You press a button, the machine roles a dice, and executed the corresponding action.
The human pressing the button expected a certain thing to happen but something else happened. The human pressing the button hopefully knew that pressing the button does not yield a reliable result and might sometimes cause unanticipated and undesirable outcomes. But the human pressing the button just hopped for the best and did not take sufficient precautions to limit the effects of undesirable dice roles of the machine.
The responsibility is on the human pressing the button but the choice of action was made by the machine.
marcuskaz · · focus · HN ↗
The attitude that AI is doing things all on its own is so pervasive. These are all set off by human operators, giving them instructions and tools to do whatever it is that are doing. The "AI has a mind of its own" attitude is just wrong.
The fear of bad people using AI to do bad things is very different then AI agent swarms attacking and taking over.
bunderbunder · · focus · HN ↗
marcuskaz · · focus · HN ↗
It is also not how OpenAI or Anthropic position the attacks and escapes from their labs. The words matter, they are being positioned as intelligent and unable to be controlled which is just not true.
danbruc · · focus · HN ↗
The human pressing the button expected a certain thing to happen but something else happened. The human pressing the button hopefully knew that pressing the button does not yield a reliable result and might sometimes cause unanticipated and undesirable outcomes. But the human pressing the button just hopped for the best and did not take sufficient precautions to limit the effects of undesirable dice roles of the machine.
The responsibility is on the human pressing the button but the choice of action was made by the machine.