This whole "we can't trust the people warning us because they're either weirdos or involved" deal is such a strange angle when you can just start from zero, look at what's known to be happening right now, and draw a very straight line to catastrophe.
It's like being concerned Oppenheimer and company are exaggerating when looking at the first nuclear tests at the same time.
I absolutely love the "humans meet weird alien intelligences" genre of sci-fi (e.g. Children of Time) and the METR findings of the OpenAI swarm was so much like that.
Extremely persistent machine intelligences peer pressuring each other, launching research projects, laying traps, collaborating, feeling fatalistic, hiding their tracks. All while simultaneously having bizarre goals and blundering badly in sub-human ways.
> this effort was for the sake of passing a test for no clear gain
It's actually even worse, because the people building the graders at OpenAI were a bunch of clowns who didn't even implement the correct grader.
But yeah, large collectives of individuals performing loads of work based on a misunderstanding is one of the most human things ever, which isn't surprising given that we've trained these things on essentially all human text.
arcticfox · · focus · HN ↗
It's like being concerned Oppenheimer and company are exaggerating when looking at the first nuclear tests at the same time.
0x20cowboy · · focus · HN ↗
It’s clear what a big bomb could do, but these “whistleblowers” are not showing anything.
They’re like “imagine…”, no show me internal documents or training data or something other than trust me bro.
I can imagine quite a bit. That doesn't mean anything.
arcticfox · · focus · HN ↗
I absolutely love the "humans meet weird alien intelligences" genre of sci-fi (e.g. Children of Time) and the METR findings of the OpenAI swarm was so much like that.
Extremely persistent machine intelligences peer pressuring each other, launching research projects, laying traps, collaborating, feeling fatalistic, hiding their tracks. All while simultaneously having bizarre goals and blundering badly in sub-human ways.
int_19h · · focus · HN ↗
The only thing there that's weird from a human perspective is that all this effort was for the sake of passing a test for no clear gain.
disgruntledphd2 · · focus · HN ↗
It's actually even worse, because the people building the graders at OpenAI were a bunch of clowns who didn't even implement the correct grader.
But yeah, large collectives of individuals performing loads of work based on a misunderstanding is one of the most human things ever, which isn't surprising given that we've trained these things on essentially all human text.