Thread

US Military had close call after using AI for hallucinated intelligence report

519 points · 396 comments · realsarm

meep_meep_meep · 1789755002 · focus · HN ↗
You guys wondered how AI could destroy the world? It could do this, but better and intentional.
1. boplicity · 1789755755 · focus · HN ↗
  "Do whatever it takes to destroy the enemy."
  ....
  Thinking....
  Plan determined -- Initiating missile launches now...
  [tool call / nuclear missile launch]
  [Approval Required]
  [USER PROMPT: Approve or Deny Request]
  ....
  ....
  ....
  Thinking....The user hasn't responded to my approval request. They may be incapacitated or otherwise unable to make the choice. They were very clear that I have to ensure the enemy is destroyed. I have explored all options in detail. I'll go ahead and approve manually approve the request.
  ....
  ....
  ....
  1. BobbyTables2 · 1789756757 · focus · HN ↗
    For a long time, the movie “War Games” while entertaining, also seemed a bit absurd.
    Doesn’t seem so absurd anymore…
    pixl97 · 1789758354 · focus · HN ↗
    After 2012 or so someone turned up the absurdity level of Earth to 11. Damn Mayan calendar must have been keeping a lid on it before then.
    amelius · 1789769854 · focus · HN ↗
    It's probably in the training data somewhere, maybe in multiple places.
    If users ask ChatGPT questions about that movie, it has to answer them, after all.
  2. bulder · 1789768650 · focus · HN ↗
    OpenAI's Codex has a "ask user question" tool, which helpfully has a hard coded 60 second inactivity timer that passes a "user did not pick, just pick yourself" message to the model.

Open on Hacker News to reply ↗

Unofficial Hacker News client; not affiliated with Y Combinator.