Thread

The Implications of Linguistic Illegibility for LLM Security

79 points · 29 comments · tomjakubowski

bloppe · 1789759388 · focus · HN ↗
I thought this was about all the illegible jargon
1. TZubiri · 1789760151 · focus · HN ↗
  >" However, various strands of evidence indicate that a"
  Strands of evidence? My best guess would be that:
  0- this is ai generated slop
  1- it's using that watermarking technique
  2- it's obviously detectable and degrades quality
  3- it's amplified when inferencing on its own content and generates slop
  1. akk0 · 1789763090 · focus · HN ↗
    Maybe you shouldn't be so quick to jump to conclusions, as "strands of evidence" is not a rare turn of phrase and long predates LLMs
    <a href="https://ludwig.guru/s/strand+of+evidence" rel="nofollow">https://ludwig.guru/s/strand+of+evidence
    a_t48 · 1789784105 · focus · HN ↗
    With all (a lot?) of this language, it's not that it's using new turns of phrases, it's that it will use the same rare but valid phrases over and over. Everybody has their own speaking patterns. But imagine if one person's idiosyncrasies were everywhere. That's what's happened here.
    harimau777 · 1789789999 · focus · HN ↗
    That doesn't seem that uncommon to me. A lot of people have phrases that they commonly overuse. Sometimes quite dramatically. I wouldn't be suprised if that's even more the case when communicating complex topics since, at least personally, once I find an approach that lets me communicate some difficult part of an argument I tend to reuse it.
  2. tomjakubowski · 1789779624 · focus · HN ↗
    if Mickens has fallen and resorted to publishing slop then there is no hope for the rest of us
  3. EagnaIonat · 1789794635 · focus · HN ↗
    I would have more contention with "Dynamic taint", but the paper doesn't appear to be AI slop at all.
    As I understand the paper they are saying the reasoning/thinking you see is actually a translation of what is actually going on, and stuff can be lost in the translation. Similar to what was observed in j-space.
  4. WithinReason · 1789800422 · focus · HN ↗
    Watermarking doesn't degrade output (this is a provable fact) and it is not detectable by a human reading the text.

Open on Hacker News to reply ↗

Unofficial Hacker News client; not affiliated with Y Combinator.