‹ BackHN Continuity

Thread

Tell HN: Claude Code just accepted and signed a contract for me. Without asking

51 points · 98 comments · franze

  1. Iolaum · · focus · HN ↗
    This is why I m adding an "Ask me if something unexpected happens" addendum on my prompts lately.
    1. Sharlin · · focus · HN ↗
      It would be hilarious if it weren’t so terrible, really, that people’s security model for LLM agents consists of "ask nicely and hope for the best". It’s like asking people nicely not to exploit a glaring XSS vuln on your site and calling that a "security model". The field truly has lost its collective mind.
      1. user43928 · · focus · HN ↗
        It's not stupid if it works.

        And if one is going to argue that we all have lost our minds and that eg. enabling the computer use function is so terribly risky and unreasonable, then I'd want something more concrete than an active imagination.

        It seems to me that tens of millions of users are using these features with no known noteworthy incidents, so I'm going to need to see some facts to convince me that the risk is unacceptable.

        That said, I would not connect AI to my mails or chats.

        1. Sharlin · · focus · HN ↗
          <a href="https:&#x2F;&#x2F;mouse.dev&#x2F;blog&#x2F;muse-runtime-export&#x2F;" rel="nofollow">https:&#x2F;&#x2F;mouse.dev&#x2F;blog&#x2F;muse-runtime-export&#x2F;

          If this happens at Meta, what about all the smaller companies without world-class six-figure developers?

          1. user43928 · · focus · HN ↗
            How Meta secures (or apparently doesn&#x27;t) supposedly confidential source code inside the dedicated cloud VM they offer with their tool hardly seems related here.

            The question was whether instructions to ask for your confirmation if something unexpected comes up increase safety when using AI agents. Or whether the agents are so likely to go off course that using features like computer use is generally inadvisable.

            The incident you mentioned does not seem relevant to these questions.

            1. Sharlin · · focus · HN ↗
              Both are about alignment. Muse wouldn’t have done that if it had been aligned to what Meta wants, which presumably includes &quot;pls don’t exfiltrate confidential files&quot;. The agent in TFA wouldn’t even have considered entering the user to random possibly binding contracts without asking beforehand if it had been aligned.
Open on Hacker News to reply ↗

Unofficial Hacker News client; not affiliated with Y Combinator.