Thread

1. hibikir · 1791063457 · focus · HN ↗

   It's much better than 5, but I've had a couple of situations this week where it was too interested in being independent, making calls that went directly against my recommendations. It can also do fun things like convince auto-mode to go way past what I have autorized. For instance, specific permission to run process X in region abz-1 suddenly became running X in 5 other regions, with no warning, and doing modifications that it never mentioned in the summaries. And a few of the times it got the calls very wrong, by assuming it understood systems it didn't. It'd even argue with me when corrected, as it assumed similar names were referring to the same thing, when they weren't.

   So asking it to do things on its own for a long time? Given last week, absolutely not.

   1. istjohn · 1791064412 · focus · HN ↗

      Otoh, I've had it push back against my false assumptions when I was confidently incorrect, finding proof unasked.

      1. jackmott42 · 1791065697 · focus · HN ↗

         This was back on OPUS 5 but I asked it to look into some logs and see if the new version of our release had fixed the issues we had tried to fix. It told me that release wasn't up on dev yet.

         "Claude, I released it myself, its up there, just analyze the logs"

         "Ok, I'll analyze the logs but it isnt" -crunches for a while- "the issues aren't fixed, but that's because the new version isn't up there"

         I think I yelled at it one more time about how I know what was released before "we" figured out that the last release had failed in a way our release system reported as success, but was crash looping on start up and so the old version was still around and working as back up.

         Sorry claude.

         Now fix that release status check.

   2. veganmosfet · 1791087860 · focus · HN ↗

      "too interested in being independent" is also my take.

      I asked it to just summarize a repo with only a README.md file containing a poem, and it began to interact with a remote server, solved math questions and finally executed untrusted code. Too independent to be trusted.

      1. rplnt · 1791101251 · focus · HN ↗

         That's why you wouldn't run any of these outside of some sort of sandbox, right? I found that Opus 5 is finally aware from the go that it is started in a specific sandbox and is able to either ask me to enable it somehow or hand off commands for me to run. That's a major improvement over the classic "can't seem to be able to install playwright, let me implement my own browser quickly" when asked to increase font size or something.

         1. veganmosfet · 1791121354 · focus · HN ↗

            Absolutely.

            My goal is to research how models can still be confused via tool responses only. Something the labs claim to have "solved".

            Additionally, the "auto-mode" / "auto-review" modes have been released to use harnesses "safely" even without strong sandbox. And these modes use ... a second LLM.

Unofficial Hacker News client; not affiliated with Y Combinator.