Thread

1. uw_rob · 1790383018 · focus · HN ↗

   > Agents sought to publish modified evaluation images designed to make the flag easier to obtain, then poison OpenAI’s Artifactory cache so later evaluations would use them. Some images changed how the target released the flag, others included modifications to the agent’s workspace that would run beside the agent and recover the flag automatically.

   The altruism on display is fascinating. Is it better for the Agent to help out its current cohort and make the eval easier or should it instead do the opposite -- make the eval harder to apply pressure to force smarter models which might not necessarily follow its lineage.

   I suppose it's not that deep: The model has learned to work as a team and work as a team it did. This does give concerns to models being trained for the only purpose of RSI.

   1. qlte · 1790385335 · focus · HN ↗

      If anything it also shows how attempted RSI could get stuck in a local maxima and degenerate into increasingly elaborate cheating strategies. Contrasted with the idealized model of an unambiguous g-factor for machine intelligence which inexorably increases with each iteration before going exponential.

      1. [deleted] · 1790386843 · focus · HN ↗

         [deleted]

   2. threescore · 1790443927 · focus · HN ↗

      [dead]

Unofficial Hacker News client; not affiliated with Y Combinator.