‹ BackHN Continuity

Thread

Humans missed 1 in 3 threats approving AI agent commands across 40k game runs

210 points · 169 comments · Wirbelwind

  1. jascha_eng · · focus · HN ↗
    1 in 3 is not terrible you just need a few more humans in the loop to reduce the error rate meaningfully. Combined with other classifier models and heuristics you can get good results. Humans can probably also perform better if they don't have to judge every single command but just suspicious ones our attention is limited after all.
    1. crabbone · · focus · HN ↗
      1 in 3 is end of the line awful... Back when I was in college (former USSR), we had a subject roughly translated as "integration with industrial processes". USSR industry was highly regimented. Various norms, tolerances, recipes etc. were described in GOSTs (a kind of arsenal of industry standards). There were also some common knowledge / statistical bits that went into making these GOSTs. I mention this because this system dealt in great detail with quantifying human error (as well as errors resulting from equipment use etc.).

      One of the core assumptions was that outside of extraordinary circumstances, the expected rate of human error is about 5%. However, the course also provided examples where error rates were significantly lower (eg. nurses in maternity wards would have a much lower than 0.1% error rate when pairing mothers with newborns).

      The error rates, of course, also depended on human ability to measure the difference. Since I was studying typography, the printing process was of particular interest. A GOST for offset printing required that the color intensity for each ink of CMYK, for example, should be within +-2.5% range of the intended intensity. This is difficult for someone who doesn't have a lot of experience operating an offset printing machine to spot, but experienced printers have no problem with that.

      Most importantly. There was never an acceptable error rate of 33%. Not for anything. If people were likely to make that many errors (eg. because the measurement was too difficult), that product would never have been allowed into production.

      1. cedilla · · focus · HN ↗
        1/3, but under unreasonable time pressure, and with no prior vetting. For example, I played a few times, and I'm not a JS developer. I had to just suss out if npm whatever is dangerous or not. I'm very happy with my personal 25%.
Open on Hacker News to reply ↗

Unofficial Hacker News client; not affiliated with Y Combinator.