"Sonnet 5.5’s cyber capabilities are a large improvement over Sonnet 5’s, so we’re deploying it with safeguards similar to those on Opus 5.5. Users can still find and fix bugs in their code as part of routine software development, but higher-risk cybersecurity tasks will visibly fall back to Sonnet 5
Sounds like at least for Anthropic models we reached peak cyber capabilities with Opus 4.8. Everything after that falls back to worse models
Where this gets painful, is that I was working on my own OSS project today, and this is what happened;
1. Opus 5.5 noted some concerns.
2. I asked it to write tests to safely test the concerns and write up a remediation plan
3. Opus 5.5 flagged and reset the conversation to Opus 4.8, murdering my usage quota and (possibly?) doing a sub-optimal set of tests.
NGL, it would have been at least polite for it to either:
1. Sanity check if I was the only/main committer on the repo (I'm the only committer, it's my side project) and then decide whether it was 'responsible' to help me fix it. (after all, I'm wanting to correct the problem, not exploit it!)
2. Warn me before just YOLOing the context to another model leading me to have to do a grace reset.
wongarsu · · focus · HN ↗
Sounds like at least for Anthropic models we reached peak cyber capabilities with Opus 4.8. Everything after that falls back to worse models
to11mtm · · focus · HN ↗
1. Opus 5.5 noted some concerns.
2. I asked it to write tests to safely test the concerns and write up a remediation plan
3. Opus 5.5 flagged and reset the conversation to Opus 4.8, murdering my usage quota and (possibly?) doing a sub-optimal set of tests.
NGL, it would have been at least polite for it to either:
1. Sanity check if I was the only/main committer on the repo (I'm the only committer, it's my side project) and then decide whether it was 'responsible' to help me fix it. (after all, I'm wanting to correct the problem, not exploit it!)
2. Warn me before just YOLOing the context to another model leading me to have to do a grace reset.