‹ BackHN Continuity

Thread

A Privacy Analysis of Web and Mobile Conversational AI Agents [pdf]

425 points · 141 comments · damaru2

  1. pbasista · · focus · HN ↗
    Tangential:

    I have recently noticed that e.g. ChatGPT, when used from a web browser, periodically sends unfinished prompts to their servers, namely to the `conversation/prepare` endpoint, without waiting for the user to actually send it.

    This partial prompt data might potentially be used to "pre-warm" some kind of cache.

    But it may also be used to track the user's writing cadence, error correction style and evolution of their stub ideas as they are being formulated into a prompt. I would assume that such data could also be sold to the advertisers.

    1. ShinyLeftPad · · focus · HN ↗
      I wouldn't be surprised if their privacy policy would say "what you send is private" and then it wouldn't apply to unfinished prompts on technicality
      1. jwstillwater · · focus · HN ↗
        This is my concern as well- the same wiggle-room methodology that allowed a business to claim not to “sell or share” PII, because “user data collaboration” was not part of the legal definition prior to CCPA.

        OpenAI’s statements in response to the Millenium Prize (and related) disputes I think are a pretty obvious example of this in practice. One man’s “user prompts” is not another’s “reasoning trace scratchpad”.

        This comment by Falserum on the mathematics research post articulates it well:

        <a href="https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=49649992">https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=49649992

Open on Hacker News to reply ↗

Unofficial Hacker News client; not affiliated with Y Combinator.