> taking careful precautions against feeding the findings back into training so as to not risk shaping Argon’s reasoning to evade our monitoring. We strongly encourage the rest of the industry to preserve reasoning transparency in these pivotal moments of increased capabilities while navigating alignment risks, so that model thoughts remain helpful in identifying and diagnosing misalignment.
This is good, but they're the slow mover due to this exact thing.
Google is getting punished for not letting the models enter an echo chamber and go faster than humanly possible.
OpenAI is the company that originally proposed and popularized chain-of-thought monitoring: <a href="https://openai.com/index/chain-of-thought-monitoring/" rel="nofollow">https://openai.com/index/chain-of-thought-monitoring/
So no, Google is not being punished, nor are they the people behind this technique.
The statement appears to be referencing Astra's supposed recurrent depth and how that causes reduced visibility into chain-of-thought reasoning. Astra's own system card describes tests where it's asked to solve challenging math problems while internally thinking about something entirely unrelated, which it's significantly more capable of than previous models (~60% vs ~16% of the time for Sol). That seems to point to reduced efficacy of chain-of-thought monitoring, but OpenAI's public statements basically boil down to "yeah but we haven't been able to catch it doing that" which isn't exactly reassuring if CoT monitoring is one of your main safety guardrails.
gopalv · · focus · HN ↗
This is good, but they're the slow mover due to this exact thing.
Google is getting punished for not letting the models enter an echo chamber and go faster than humanly possible.
janustimes · · focus · HN ↗
So no, Google is not being punished, nor are they the people behind this technique.
mattstir · · focus · HN ↗
WarmWash · · focus · HN ↗
The worst case scenario is that the easiest path forward is one where we lose sight of internal thought.