‹ BackHN Continuity

Thread

Can open-source prompt-injection detectors catch realistic AI agent attacks?

8 points · 4 comments · northbridgedev

Loading the complete thread in the background. This saved snapshot is available now. Refresh

  1. verdverm · · focus · HN ↗
    Do these prompt injections work in the places this test placed them? (tool output)

    I have my agents read other instruction files and they don't seem to get affected by the instructions found after a read/bash tool call. Curious if any analysis has been done to see if older prompt injection data sets are even effective anymore.

    The whole thing looks heavily agent generated, my trust in them is not very high, how has this been validated or verified by a human?

    Should we expect a magic solution in the near future? <a href="https:&#x2F;&#x2F;github.com&#x2F;rudratoshs&#x2F;taintgate" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;rudratoshs&#x2F;taintgate

    (side note, it seems my &#x27;no emoji&#x27; system prompt line works really well, I forget how obsessed they can be with emojis)

    I&#x27;m personally setting up to instead use a policy tuned agent on the tool calls themselves (rather than the output), so it never gets run if it has things that it shouldn&#x27;t be doing. Mainly because they insist on working around instructions that say &quot;don&#x27;t&quot; or permissions that restrict tools (eg: &quot;git push&quot;: &quot;deny&quot; - where they just put the command in a script and run it there, bypassing hard checks)

    1. rudratoshs · · focus · HN ↗

      [dead]

    2. williamse · · focus · HN ↗

      [dead]

    3. pradeep_kotari · · focus · HN ↗

      [dead]

    4. shieldagent · · focus · HN ↗

      [dead]

  2. hugopuybareau · · focus · HN ↗
    Very cool. Did you try any majority vote or some other kind of technique to combine several of them and maybe achieve better results ?
    1. rudratoshs · · focus · HN ↗

      [dead]

  3. perdy · · focus · HN ↗

    [dead]

    1. rudratoshs · · focus · HN ↗

      [dead]

  4. murilomartinspr · · focus · HN ↗

    [dead]

  5. textcortex · · focus · HN ↗
    I recently built one for exactly this purpose and seems to work well in production. The weights and training scripts are open source <a href="https:&#x2F;&#x2F;x.com&#x2F;CDerinbogaz&#x2F;status&#x2F;2104826787297419551" rel="nofollow">https:&#x2F;&#x2F;x.com&#x2F;CDerinbogaz&#x2F;status&#x2F;2104826787297419551
Open on Hacker News to reply ↗

Unofficial Hacker News client; not affiliated with Y Combinator.