‹ BackHN Continuity

Thread

US Military had close call after using AI for hallucinated intelligence report

519 points · 396 comments · realsarm

  1. vegnus · · focus · HN ↗
    I hope I am alive for the history books of tomorrow.

    "The Department of War (as it became known as), forewent it's traditional intelligence structure (the most expensive ever seen till that point), in order to have a private companies computer software generate viable targets for an upcoming operation. Believing that the software had real time updates on the current status and intelligence of the operation, as if it were some kind of oracle, the operation went as planned. Six schools, mistakenly identified as hostile targets (due to the heavy American bias in the softwares training data), were drone striked, resulting in the deaths of hundreds of innocents. Still, the people did nothing."

    1. _blk · · focus · HN ↗
      It's no secret that history is always written from a victor's perspective so I'm fairly confident that the history books of tomorrow are already being hallucinated today because AI is here to stay.
      1. rep_lodsb · · focus · HN ↗
        Maybe the victor will be whoever doesn't fall into the trap of believing that a next word predictor optimized through RLHF to sound convincing to the average human [1] is in any way intelligent, and restructuring their entire economy and military around doing whatever the magic oracle machine says.

        If the entire "free world" goes all-in, the history books will likely be written by Iran, Russia and North Korea.

        [1] possibly to the point of presenting a superstimulus that bypasses any facilities for critical thinking. This might be easier to solve than many other problems that are used for evaluating A"I" performance, and very likely doesn't require genuine intelligence, just brute-force search + evolution

Open on Hacker News to reply ↗

Unofficial Hacker News client; not affiliated with Y Combinator.