Thread

One Piece of Flock Camera Data Put This Innocent Woman in Jail for 13 Days

257 points · 145 comments · HotGarbage

freddealmeida · 1790398947 · focus · HN ↗
Flock did not put the woman in jail. The police and DA put her in Jail. This is not a failure of the technology (do they have confidence levels?). This is a failure of how police are outsourcing their reasoning and decision making to an AI camera service that seems to fail at the tail.
1. CrendKing · 1790401755 · focus · HN ↗
  But Flock gives police overconfidence to commit actions instead of having due diligence. It is like the best student in your classroom tells you 2 + 2 = 5, and you naturally feel compelled to believe him, even there is clear evidence against the claim.
  This is also comparable to self-driving car: Waymo's system has achieve the level of "less error than human on average" long time ago, but they are still only doing slow expansion. Why? Because when AI is involved, human naturally expect them to be almost 100% correct, not just barely better than human, otherwise why bother use AI (and all the troubles to legalize for AI) in the first place? So in this case, I guess the police just blindly trust the AI being 99.99% correct.
  1. nrmitchi · 1790403041 · focus · HN ↗
    Flock isn’t an “AI” though? It doesn’t put together a case and say “yes, this is the criminal”. It’s not a crime-solving AI.
    It’s a camera system, and data can be queried by police. They have to search for something. So the cops searched for a vehicle matching the description in the general area, seemed to take the first one, and called it a day.
    That’s not an AI-uncanny-valley. That’s the equivalent of (pre-flock/cameras) looking at vehicle registration list and assuming the one at the top of the list must be it.
    The automation-trust is a legitimate concern but it’s not applicable here.
    givinguflac · 1790420335 · focus · HN ↗
    Ahem, reality would like a word about flock not being AI.
    <a href="https://cybernews.com/privacy/flock-ai-os-investigate/" rel="nofollow">https://cybernews.com/privacy/flock-ai-os-investigate/
    newtonsmethod · 1790421434 · focus · HN ↗
    Flock is not an AI. That they have AI features on certain products does not mean "Flock" is an AI. That's the fallacy of composition.
    Flock does have AI components (this one seems LLM based). Basically every ANPR system will also use some level of AI.
    Also, on "reality would like a word". Why not just speak normally instead of using what seems like a high-school grade comeback?
    The main Flock related issue here is that officers treat results that come out of the systems as gospel. The same thing has happened with face recognition systems. There is also the broader issue of access controls to Flock, 5A, and whether places should be building up a technology so powerful (surveillance infrastructure can easily be used for evil).
    givinguflac · 1790509546 · focus · HN ↗
    I said that because your position is absurd, and clearly false. Flock is an AI company, period. They use a multitude of different ai types in their “AI OS”.
    And the issue is absolutely not just cops taking what they say as gospel; that’s a teeny tiny itty bitty fraction of the issue, which is in itself reductive. This is a massively multifaceted issue and to state otherwise means you don’t understand the scope.
    Perhaps you should actually read TFA. Here’s another one. <a href="https://cybernews.com/privacy/flock-ai-os-investigate/" rel="nofollow">https://cybernews.com/privacy/flock-ai-os-investigate/
    newtonsmethod · 1790595061 · focus · HN ↗
    I have made no comments on this thread before the one above.
    So saying you said something because my non-existent position is "absurd" and "clearly false" indicates massive levels of confusion.
    You then go on to imply I haven't read the article, using an acronymised swear (I'm not a pious person but clearly directing a swear to someone makes a message come off as aggressive), even though your understanding of what I said is based off non-existent messages. (This is quite ironic).
    Flock is a company. Companies are a type of organisation: it makes no sense to describe a company as AI, as AI is a type of computer system. It may make sense to say Flock is an AI-based company, but calling them "AI" in itself makes no sense.
    I can't find anything indicating that an LLM was used in this case, and this county seems to mainly have Falcon cameras (<a href="https://www.scribd.com/document/983922612/250730110407Flock-Safety-INV-63909-2025-05-09" rel="nofollow">https://www.scribd.com/document/983922612/250730110407Flock-...). Reverse engineering seems to indicate these are powered by non-LLM AIs like Tensorflow vision NNs. These have been around for ages, and notably no-one is alleging these models hallucinated etc.
    People also (perhaps wrongly) use "Flock" to refer to the camera system. CrendKing explicitly calls "Flock" a "camera system". A system of cameras is not AI, and the point of the comment seems to be that it's inappropriate to bring up LLM related AI fears about this system.

Open on Hacker News to reply ↗

Unofficial Hacker News client; not affiliated with Y Combinator.