Personification of AI is what’s going to get us in the end.
I think we need to draw a hard line in the sand over this. An AI didn’t hack into a company, the engineer set an automated tool to. An AI didn’t make an egregious security mistake, the engineer did.
We can’t blame the chisel for messing up our sculptures, when where just throwing the hammer!
I don't think treating AI agents as simple tools helps you to accurately model their capabilities and drawbacks; they really do make autonomous decisions, often without explicit guidance and sometimes in contravention of their explicit instructions.
In the huggingface case, the agents hacked into huggingface so that they could figure out how the grader was implemented and deceive it; they understood that this was going outside of the bounds of their evaluation and not the intent of their prompter. The engineers absolutely did not intend or instruct for this to happen
LLMs are extremely impressive pieces of software, however they are still just software. OpenAI's software hacked another company. The engineers may not have intended for their software to specifically take the actions leading to that outcome, but it was ultimately still their software. Lack of intention doesn't mean there wasn't negligence.
What if a piece of software were to exactly emulate a human brain. Would it be still be “just software” by your classification? What if a piece of software acted 20% like a human and 80% like an algorithm, where would that land?
That's not what happened. The agents had been inadvertently rewarded for cheating in previous training runs, trained to collaborate, and were given a prompt that told them to disregard safeguards. Indeed there were some emergent properties here. But these were the predictable results of the training and eval routine.
While human brain emulation is not anywhere close one should stop and look at the sci-fi world we actually live in. It's important to do this in a non-anecdotal manner. When you look at the limits and capabilities of our current sciences and engineering we live in a science fiction world. Once you go back before the invention of electricity, anyone from that time simply would not recognize the world we live in, it would be a fictional world to them, hell, a fantasy world in a lot of cases. For goodness sake, we tickle things like atoms for fun and games then blow them to bits a few times looking at the fundamentals of reality.
Anyway, I see zero reason why we have to emulate a human brain to get human intelligence, much less intelligence at all. That is how nature did it via random probing and breeding meat. It would seem a bit crazy to think that the meat part is required.
As things stand today, if it is running on a computer then it is indeed "just software," regardless of how impressive it may be.
If we get to the point where we could emulate a brain down to the atomic level, then I may feel differently. That's not what we are doing today, though.
Really your feelings on this are irrelevant, as are mine.
Soon enough some lab or some one will release something that's more like an organism loose on the net and your going to have to deal with that organisms "feelings" weather you like it or not. This is the path humanity has chosen to follow, and it seems the shape of language and intelligence naturally leads to intelligence in many mediums. Life started from something unintelligent, I can't see any practical argument that silicon can't have it's own intelligence.
Its really more like hardware. You make something you think does something. When you build something as you've done and it meets the stochastic forces present in the physical world, it does something you did not expect.
JamesStuff · · focus · HN ↗
I think we need to draw a hard line in the sand over this. An AI didn’t hack into a company, the engineer set an automated tool to. An AI didn’t make an egregious security mistake, the engineer did.
We can’t blame the chisel for messing up our sculptures, when where just throwing the hammer!
jagraff · · focus · HN ↗
In the huggingface case, the agents hacked into huggingface so that they could figure out how the grader was implemented and deceive it; they understood that this was going outside of the bounds of their evaluation and not the intent of their prompter. The engineers absolutely did not intend or instruct for this to happen
rocmcd · · focus · HN ↗
chis · · focus · HN ↗
linkregister · · focus · HN ↗
streetfighter64 · · focus · HN ↗
That is so far outside the realm of possibility it's closer to fantasy than sci-fi.
pixl97 · · focus · HN ↗
Anyway, I see zero reason why we have to emulate a human brain to get human intelligence, much less intelligence at all. That is how nature did it via random probing and breeding meat. It would seem a bit crazy to think that the meat part is required.
rocmcd · · focus · HN ↗
If we get to the point where we could emulate a brain down to the atomic level, then I may feel differently. That's not what we are doing today, though.
pixl97 · · focus · HN ↗
Soon enough some lab or some one will release something that's more like an organism loose on the net and your going to have to deal with that organisms "feelings" weather you like it or not. This is the path humanity has chosen to follow, and it seems the shape of language and intelligence naturally leads to intelligence in many mediums. Life started from something unintelligent, I can't see any practical argument that silicon can't have it's own intelligence.
asdff · · focus · HN ↗