Thread

1. Madmallard · 1790461270 · focus · HN ↗

   Sounds like you got scammed

   Hope this gets some visibility idk why it's flagged guess the PR guys for those companies are doing it

   Should spread this around

   1. lorenzomassaro · 1790461401 · focus · HN ↗

      honestly is also more about how dangerous this is, idk for the flag either

      1. verdverm · 1790471342 · focus · HN ↗

         humans remain responsible, agents don't go rogue, should put some billing controls in place, that's like the first thing to do

         1. blooalien · 1790472090 · focus · HN ↗

            > humans remain responsible, agents don't go rogue

            ^^^ 100% this ^^^ - It's either a serious flaw in the agent/harness software, or a user error in usage/configuration or prompting. Either way, it's a human somewhere responsible for these outcomes.

            > should put some billing controls in place

            At the very least, yes! These things should never be running without any limits on what they can do without some human signoff on important/dangerous actions. Not only should they have controls on those actions, but those controls should absolutely have some sane default settings.

            1. verdverm · 1790472341 · focus · HN ↗

               we only selected vendors that had billing limits, if they didn't, instant disqualification

               most are not as granular as we'd like, but seem to be headed in that direction finally, regardless, there are card limits and alerts

            2. lorenzomassaro · 1790491931 · focus · HN ↗

               There was never a user prompt requesting the creation of 823 agents nor a task assigned that would require 2130 billion tokens and the UX never provided any feedback on what codex was doing nor consumption metrics which would allow to notice. It was all hidden to me.

               1. verdverm · 1790527079 · focus · HN ↗

                  I don't think it likely hidden from you, more likely you didn't put the effort in, that's what the pattern looks like to outsiders reading your accounting of what happened here

                  we'd need to know more details to evaluate your botnet claims

                  1. lorenzomassaro · 1790585323 · focus · HN ↗

                     I would be happy to provide as many details as could help. I have supplied OpenAI with my trace logs, running tasks ids and all info available. What would you like to see more to give me some help ?

                     1. verdverm · 1790605681 · focus · HN ↗

                        what happened within the agents does not matter

                        what we would like to see is your settings and configuration, maybe the task(s) you gave them and any code/scripts around them, where did they run from (your laptop vs cloud vm)

                        why did they even have credentialed access to change credit cards? Sounds like you didn't do the basics for isolation

                        the agents go on side quests, all the time... super frustrating, but I suspect between that tendency and asking about a UI (image), you racked up a bill with legitimate requests

                        There is a reason some of us preach "stay in the loop" and I hope you now understand why we do

               2. blooalien · 1790537174 · focus · HN ↗

                  > ... and the UX never provided any feedback on what codex was doing nor consumption metrics which would allow to notice.

                  At the very least the system should (at least the first time it happens) immediately pause activity with an email'd warning to the "responsible human in-the-loop" about "unexpected usage levels" at some sane activity warning level by default, and give the user the opportunity to set their own custom warning level right then and there.

                  This is why I say it's either "user error" (totally possible/plausible) or a badly designed agent/harness software (also highly likely/plausible) with serious foundational flaws in how it works "under the hood". The models themselves can only "run-amok" if the agentic harness is designed in a way that specifically allows and/or enables such "rogue" behavior, either by design or by negligence on the part of it's designers.

                  1. lorenzomassaro · 1790585224 · focus · HN ↗

                     Agreed. This was not an user error because everything ran without any feedback to the user. No third party agents were in use. This was all within Codex.

                     1. verdverm · 1790605533 · focus · HN ↗

                        user errors can happen before you write the first prompt or hit enter to start the hacking test, eg. not even doing the basics for isolation / airgap

Unofficial Hacker News client; not affiliated with Y Combinator.