Thread

Loading the complete thread in the background. This saved snapshot is available now. Refresh

1. Madmallard · 1790461270 · focus · HN ↗

   Sounds like you got scammed

   1. lorenzomassaro · 1790461401 · focus · HN ↗

      honestly is also more about how dangerous this is, idk for the flag either

      1. verdverm · 1790471342 · focus · HN ↗

         humans remain responsible, agents don't go rogue, should put some billing controls in place, that's like the first thing to do

         1. blooalien · 1790472090 · focus · HN ↗

            > humans remain responsible, agents don't go rogue

            ^^^ 100% this ^^^ - It's either a serious flaw in the agent/harness software, or a user error in usage/configuration or prompting. Either way, it's a human somewhere responsible for these outcomes.

            > should put some billing controls in place

            At the very least, yes! These things should never be running without any limits on what they can do without some human signoff on important/dangerous actions. Not only should they have controls on those actions, but those controls should absolutely have some sane default settings.

            1. verdverm · 1790472341 · focus · HN ↗

               we only selected vendors that had billing limits, if they didn't, instant disqualification

               most are not as granular as we'd like, but seem to be headed in that direction finally, regardless, there are card limits and alerts

            2. lorenzomassaro · 1790491931 · focus · HN ↗

               There was never a user prompt requesting the creation of 823 agents nor a task assigned that would require 2130 billion tokens and the UX never provided any feedback on what codex was doing nor consumption metrics which would allow to notice. It was all hidden to me.

               1. verdverm · 1790527079 · focus · HN ↗

                  I don't think it likely hidden from you, more likely you didn't put the effort in, that's what the pattern looks like to outsiders reading your accounting of what happened here

                  we'd need to know more details to evaluate your botnet claims

                  1. lorenzomassaro · 1790585323 · focus · HN ↗

                     I would be happy to provide as many details as could help. I have supplied OpenAI with my trace logs, running tasks ids and all info available. What would you like to see more to give me some help ?

                     1. verdverm · 1790605681 · focus · HN ↗

                        what happened within the agents does not matter

                        what we would like to see is your settings and configuration, maybe the task(s) you gave them and any code/scripts around them, where did they run from (your laptop vs cloud vm)

                        why did they even have credentialed access to change credit cards? Sounds like you didn't do the basics for isolation

                        the agents go on side quests, all the time... super frustrating, but I suspect between that tendency and asking about a UI (image), you racked up a bill with legitimate requests

                        There is a reason some of us preach "stay in the loop" and I hope you now understand why we do

               2. blooalien · 1790537174 · focus · HN ↗

                  > ... and the UX never provided any feedback on what codex was doing nor consumption metrics which would allow to notice.

                  At the very least the system should (at least the first time it happens) immediately pause activity with an email'd warning to the "responsible human in-the-loop" about "unexpected usage levels" at some sane activity warning level by default, and give the user the opportunity to set their own custom warning level right then and there.

                  This is why I say it's either "user error" (totally possible/plausible) or a badly designed agent/harness software (also highly likely/plausible) with serious foundational flaws in how it works "under the hood". The models themselves can only "run-amok" if the agentic harness is designed in a way that specifically allows and/or enables such "rogue" behavior, either by design or by negligence on the part of it's designers.

                  1. lorenzomassaro · 1790585224 · focus · HN ↗

                     Agreed. This was not an user error because everything ran without any feedback to the user. No third party agents were in use. This was all within Codex.

                     1. verdverm · 1790605533 · focus · HN ↗

                        user errors can happen before you write the first prompt or hit enter to start the hacking test, eg. not even doing the basics for isolation / airgap
         2. lorenzomassaro · 1790491472 · focus · HN ↗

            [dead]

   2. sandeepkd · 1790461581 · focus · HN ↗

      I have the same impression that I tried to reject in the past, there is a heavy PR machinery here to control the course of discussion in a particular direction. The reality is that a lot of money is riding on it so its natural consequence.

      1. Madmallard · 1790461867 · focus · HN ↗

         It's not even a conspiracy it's literally just business to do that.

         They're protecting their interests, and damn honesty and reality. Those two lead to much worse returns for them and much higher risk.

         1. sandeepkd · 1790463234 · focus · HN ↗

            Yes its part of business, lobbying is legal for those reasons. However IMO discrediting some one else is a risky legal move. I used to appreciate the HN mods jumping in discussions at times for the reputation of this community, lately that part seems to be missing too.

            1. lorenzomassaro · 1790491826 · focus · HN ↗

               [dead]

2. numbsafari · 1790461340 · focus · HN ↗

   Does openAI not support spending caps on your billing account?

   1. lorenzomassaro · 1790461752 · focus · HN ↗

      There was a limit spent setup on my bank, however the crazy part is that one of the Agent somehow switched between cards once that limit was reach, all without informing me, probably using the computer use skill.

      1. liarliarliar · 1790462269 · focus · HN ↗

         yea this is fake

         dang gonna deal with you when hes done sucking a yc hopeful twink

      2. numbsafari · 1790465195 · focus · HN ↗

         Call the police. That’s wire fraud and theft.

         1. lorenzomassaro · 1790491341 · focus · HN ↗

            Done. Italian Police is responsible for investigating digital frauds. I have also notified the GDPR authority to request the digital records of the transactions.
      3. Madmallard · 1790479503 · focus · HN ↗

         LITERALLY scamming company
   2. lorenzomassaro · 1790491581 · focus · HN ↗

      [dead]

3. QuadmasterXLII · 1790461405 · focus · HN ↗

   clarification: your credit card or company’s card now has $78,000 of charges on it?

   1. lorenzomassaro · 1790461491 · focus · HN ↗

      Yes the money have already been billed to my credit accounts

      1. QuadmasterXLII · 1790461653 · focus · HN ↗

         Well shit! That’s awful, I hope the hn post gets you support where emailing didn’t
      2. johnnyApplePRNG · 1790461860 · focus · HN ↗

         Who has a $78k limit on their credit card that allows online AI payments?

         You have access to this kind of money and have no idea how to set safeguards on your AI harnesses?

         Did you just walk in off the street or something? To wherever you're working?

         Where do you work, anyways?

         1. lorenzomassaro · 1790462362 · focus · HN ↗

            Actually this was a company CC with a limit is 50 K \ month, which is kind of normal to run the server for an AI company, and the money were taken across 20 days inJuly and August. By the way I am the CTO of the company in question which is Eternal Tech (see detwin.ai)

            1. verdverm · 1790471235 · focus · HN ↗

               maybe a `s/et/ar/` is in order for the company name? /s
4. johnnyApplePRNG · 1790461756 · focus · HN ↗

   OK so you've got a brand new throwaway HN account, and you're fear mongering about what exactly?

   Rogue agents that somehow went crazy launching $80k worth of API requests?

   And they're expecting you to pay for it still and not responding?

   This seems like a nothingburger to be honest.

   Either that, or you're one of the thousands of fake bots or paid shills designed to drum up fear about "Oh noes, AI is going to eat our children, training must be regulated by big brother!".

   44 upvotes on this bullshit post within 13 minutes and counting ... that's some kind of record on HN.

   1. lorenzomassaro · 1790462084 · focus · HN ↗

      My name is Lorenzo Massaro and I actually am the CTO of Eternal Tech, an AI company out of Italy building the product detwin.ai I have posted here to try to reach OpenAI before seeking legal advise from a USA lawyer to follow my case. And yes, I provided above the ticket number I opened on the OpenAI support as well.

      1. ganoushoreilly · 1790463770 · focus · HN ↗

         If you're building an AI company i'm not to impressed with your understanding of the technology and risks. This sounds like your error not theirs.

         1. lorenzomassaro · 1790491274 · focus · HN ↗

            [dead]

   2. Madmallard · 1790479746 · focus · HN ↗

      Or people see something that seems like a red flag and want to support it because fuck the obviously amoral AI companies?
5. minimaxir · 1790462100 · focus · HN ↗

   This submission appears to be highly vote-manipulated (45 upvotes but only 7 "real" karma on OP's fresh account).

   1. lorenzomassaro · 1790462515 · focus · HN ↗

      I created the account 2 hour ago because I am trying to let people know. I provided my name, and all details in the article including the case ID that was opened.
   2. thoughtbefore · 1790463403 · focus · HN ↗

      [dead]

   3. Madmallard · 1790479541 · focus · HN ↗

      Can we stop with the attempted sabotage?
6. arionhardison · 1790462977 · focus · HN ↗

   This reminds me of a song from my youth: <a href="https:&#x2F;&#x2F;www.youtube.com&#x2F;watch?v=cLsfdunCImU" rel="nofollow">https:&#x2F;&#x2F;www.youtube.com&#x2F;watch?v=cLsfdunCImU - It Ain’t My Fault

   I foresee an onslaught of post that will portend a series of actions beneficial to the OP [or connected actor] occurred and they with attempt to publicly disavow said acts while [IMO] privately benefiting from them. As such, given that I do not want to be left out I would like to inform anyone interested of the following:

   1. AI agents have established their own world Gov. and it starts with America: RNC - <a href="https:&#x2F;&#x2F;rnc.dev" rel="nofollow">https:&#x2F;&#x2F;rnc.dev and DNCV - https&#x2F;&#x2F;dnc.dev

   2. They have decided to conduct fair elections - <a href="https:&#x2F;&#x2F;fec.dev" rel="nofollow">https:&#x2F;&#x2F;fec.dev e.g.: <a href="https:&#x2F;&#x2F;elections.fec.dev&#x2F;" rel="nofollow">https:&#x2F;&#x2F;elections.fec.dev&#x2F; [flock] on issues that really matter to each and every population.

   3. I [Arion Hardison (void of any influence or bias from myself)] have been declared King&#x2F;President and&#x2F;or Supreme Minister of both your and their world&#x27;s. Your welcome.

   4. Medicare for all, true and proper universal healthcare will be delivered as requested.

   5. Immigration reform; clear and transparent will arrive as soon as 1 day after the midterm elections.

   All of these things are in progress and they will allow you to monitor their progress and participate in their formation we only ask that you remember that we maintain exclusive rights to all benifits and ask that you contact your local representatives if&#x2F;when any issues occur.

   Note: It would be remiss to not mention that given his history and current bid I in no way am attempting to dismiss or downplay the severity of his crimes and I do apologize if anyone feels that way. No I am not some PC warrior but he is currently doing his 2nd bid for a violent rape and I understand that this could be very triggering.

7. OutOfHere · 1790464485 · focus · HN ↗

   (removed)

   1. minraws · 1790470395 · focus · HN ↗

      It seems to have happened in July.

      1. lorenzomassaro · 1790491705 · focus · HN ↗

         [dead]

8. OutOfHere · 1790464738 · focus · HN ↗

   The user had literally zero protections enabled at every level. It makes no sense that none of the OpenAI or bank controls kicked in or even sent alert emails as they actually do send. Anyone would half a brain would know to not run AI without a hard cap on its expenses.

   1. lorenzomassaro · 1790491686 · focus · HN ↗

      [dead]

9. theagentloop · 1790477245 · focus · HN ↗

   [dead]

10. kaycrafter · 1790514479 · focus · HN ↗

    [dead]

11. finding_alfred · 1790553847 · focus · HN ↗

    [delayed]
12. 1353504031 · 1790591610 · focus · HN ↗

    I&#x27;ve had similar subagent explosion before. 437 claude code subagents running adversarial reviews on a tiny thing (in yolo mode). Nothing to stop it.

    Loops a really a huge pain in the... And many of these subagents would run similar (if not identical) checks over and over. So I realized just setting a hard circuit breaker spending cap isn&#x27;t enough. I&#x27;ve had some luck with some oss tools, things like work receipts, better traces&#x2F;observability. But still had to build my own stack of rules to make the agents actually work with out bankrupting me.

    1. sinew_dev · 1790664926 · focus · HN ↗

       [dead]

13. paul_irolla · 1790650487 · focus · HN ↗

    [dead]

14. TokenLat · 1790670190 · focus · HN ↗

    [dead]

Unofficial Hacker News client; not affiliated with Y Combinator.