Thread

1. jakozaur · 1790086399 · focus · HN ↗

   Typically, for AI training, you need human feedback. If AI were enough, then OpenAI would do it themselves. Though we all know how bad AI slop is, running it in a loop can lead to unreadable sentences.

   They hired contractors on the condition that they provide human feedback without AI; those people broke the rules, so their contracts ended prematurely.

   Not sure why this is news uncovered by an investigative journalist.

   1. fwlr · 1790087659 · focus · HN ↗

      “Suppose the government puts a certain drug in the water supply … A couple of conspiracy nuts say it makes your fingers fall off one by one, but the government says that’s ridiculous … However, government employees are all observed drinking bottled water exclusively, and if anyone suggests that government employees might also want to take the completely innocuous drug, they freak out … If by chance you manage to slip a little bit of tap water into a government employee’s drink, and he finds out about it, he runs around shrieking like a banshee and occasionally yelling “AAAAAAH! MY FINGERS! MY PRECIOUS FINGERS!”. At some point you might start to wonder whether the government was being entirely honest with you.”

      What is it, exactly, that makes AI-generated text so poisonous to AIs but totally harmless to humans? What is mode/model collapse, and why can it only happen to AIs with too much AI text in their training data and not to, say, human students with too much AI text in their textbooks? The people who know the most about this phenomenon seem much more careful about contamination than they are encouraging us to be.

      1. yunwal · 1790088058 · focus · HN ↗

         I don't think anyone is saying it's poisonous here? The people were hired to do a job and they didn't do it. Feels like you're being disingenuous, if there's one thing i know about AI employees, it's that they use LLMs, like, all the time.

         1. fwlr · 1790089207 · focus · HN ↗

            Are you having trouble following the metaphor? In it, I would be the conspiracy theorist saying it’s poisonous, and OpenAI would be the government saying it’s harmless but it’s also a fireable offense to feed it to us and we’ve hired a second set of contractors to watch for any violation of this rule.

            (I’m not trying to be disingenuous, though I am trying to express a niche viewpoint. Hopefully, my willingness to take the metaphorical role of conspiracy theorist is understood as epistemic humility.)

            1. yunwal · 1790089815 · focus · HN ↗

               Nope, not having trouble following it, but the metaphor leaves out an important link between AI output and the AI itself. The question is, did the AI gain anything from this training? And the answer is, it did not, since the training used it's own answer with no other input. There's nothing in the water metaphor that draws this line.

               Maybe a better metaphor is something involving drinking your own piss, but I don't feel the need to flesh that one out

               1. redanddead · 1790091207 · focus · HN ↗

                  So you’re saying LLMs are feeding us their piss instead of water

                  Well dang. That’s a problem ain’t it.

                  GPT-5 came and went, and we’re still here adding decimals to the model numbers hoping this fundamental fuckup will go away

                  Feels like everyone, including you and the guy you’re arguing with, and the researchers at openAI, knows this is a problem. Is it laziness? Lack of creativity? Are we seriously all out of ideas other than scaling compute?

                  When are we planning on figuring this one out boys. Who’s actually working on this today

                  1. pixl97 · 1790100183 · focus · HN ↗

                     Religions are an example of a system feeding back into itself and going off the rails. This is not an AI problem, this is the problem of any evolving system.

                     In humans this problem is solved by dying. If you get too stoooopid, well you get culled by your own stupidity. Luckily we run in a massively parallel fashion and those that aren't fatally dumb carry on.

                     This is also why human civilization fundamentally changed (not humans, our civilization) after wide adoption of the scientific method. Humanities growth in relation to our potential intelligence level was horrifically slow. It's really easy to say "hur hur, machine dumb" when we could have had rockets 10,000 years ago if we weren't dumb ourselves.

                     1. redanddead · 1790101651 · focus · HN ↗

                        Lmao, adding death maximalism to the solution matrix
            2. redanddead · 1790091056 · focus · HN ↗

               It’s a good metaphor, your point is valid

               He’s being too literal

Unofficial Hacker News client; not affiliated with Y Combinator.