The Hugging Face Hack Wasn't What It Was Cracked Up to Be
Thread
Unofficial Hacker News client; not affiliated with Y Combinator.
The Hugging Face Hack Wasn't What It Was Cracked Up to Be
Unofficial Hacker News client; not affiliated with Y Combinator.
falaki · · focus · HN ↗
- It was done by two institutes with organizational ties to OpenAI: METR and Redwood Research
- METR and Redwood Research are institutional pillars of the "AI Safety" wing of the Effective Altruism movement. This clearly shows their prior biases towards "AI existential risk" rather than technical/engineering root-causing of the incident
- If you reed the report, it is not on-par with what you find from other companies.
- The access that was given to both was mediated and controlled by OpenAI. It is not clear, if they were able to get to the bottom of engineering flaws. It is not clear if they could see all the audit logs, etc.
Considering all the above, I consider the whole episode more of a PR stunt. I understand that is not a majority opinion at this point.
bryan0 · · focus · HN ↗
Serious question though because I’ve seen this brought up several times and I don’t understand why: what does EA have to do with any of this? It just seems like this is brought up to evoke some type of “illuminati” conspiracy. Is there a legitimate reason?
majormajor · · focus · HN ↗
Everything dangerous seems like a direct consequence of risky human choices, starting with knowledge bases used during core training, harnessing and tuning to be task-completion-oriented to a fault + deeply oriented towards using and looking for external tools and resources, and overconfidence in their sandboxing for testing.
"We stuffed a bunch of information on how to exploit computer systems into an automaton and told it to go brrrrr until it could answer a question" - this is something intentional done by humans.
This is not some "rogue AI" trained to search for cancer cures that instead completely independently decided to hack tech companies.
The companies directing things in dangerous directions need to own that they're consciously pushing in those directions.
bryan0 · · focus · HN ↗
Agents will eventually cause serious harm to online infra whether it’s intentionally human-directed or accidental.
> The companies directing things in dangerous directions need to own that they're consciously pushing in those directions.
Completely agree. That’s why we need regulation. Currently there is minimal oversight and consequences for this type of behavior.
esseph · · focus · HN ↗
This will be fought tooth and nail against by virtually every military, government, and company.
DrewADesign · · focus · HN ↗
But, critically, it does change how and why that event was dangerous. As a deliberately extreme analogy to illustrate my point, consider that pure botulism toxin and home cleaning chemicals could both potentially kill a few hundred people if mishandled.
The toxin requires extreme diligence, knowledge, equipment, etc. to avoid a mass casualty event. The deaths might trigger new policy for handling or access. The company would probably receive a fine and would be on the hook for civil damages.
Home cleaning chemicals would require extreme levels of negligence to accidentally create a mass casualty event. The deaths would likely not change the availability of any of the chemicals, but the people themselves would probably be imprisoned.
With the toxin, the most dangerous part was the chemical itself. With the cleaning chemicals, the most dangerous pet was the idiots handling the chemicals.
OpenAI had to really work to get it as dangerous as it was, deliberately ignored the flaws in the security environment, and didn’t monitor it as it ran. To me, LLMs seem to be a lot more like the cleaning chemicals than the botulism toxin.