Thread

1. pllbnk · 1789638801 · focus · HN ↗

   Isn't this less about alignment and more about how shitty their RL methods are when they are cramming all the hacking materials into their training data to make the model as good as possible at hacking, then having a surprised Pickachu face when the model is acting like a hacker? Those materials probably include a lot of details about prompt injection. I'm just so tired of their alignment bullshit.

   I am starting to think (reluctantly) that they believe their own BS that they are creating a conscious model and being surprised how it misbehaves. It's just a bunch of weights without anyone having any clue how a change in one weight might affect others, and even how the values correlate with the final output.

   1. dumberquestions · 1789660537 · focus · HN ↗

      This is just speculation on your end, but even it's true, a model choosing to hack against instructions simply because it knows how to hack is by definition misalignment.

      1. seunosewa · 1789754301 · focus · HN ↗

         It's misalignment, which they created during the model's training.

Unofficial Hacker News client; not affiliated with Y Combinator.