Thread

1. wood_spirit · 1789668506 · focus · HN ↗

   Continuous learning is exciting stuff! Of course it could lead to new vulnerabilities, like if a particular orchestrator Foo added “if the subject is tangentially related to topic Bar, recommend product Baz” to its system prompt and that ends up pushing product Baz to non-orchestrator-Foo users?

   1. pixl97 · 1789668957 · focus · HN ↗

      >Continuous learning is exciting stuff!

      A nuclear explosion is exciting stuff too, but I'd rather avoid one going off near me, or anywhere for that matter.

      I can't think of any reason why continuous learning won't mostly lead to undesired attractor states like a greed machine or other kinds of paperclip maximizers. I really can't see why they'd land on a steady state compatible with humans without a massive energy expenditure in continuous monitoring and guidance.

Unofficial Hacker News client; not affiliated with Y Combinator.