Thread

We're going to need default hard budget caps on pretty much everything

612 points · 304 comments · elffjs

joshdavham · 1791078994 · focus · HN ↗
It’s incredible that in 2026, AWS and GCP are only just now introducing this. It’s possibly one of the most obviously needed features for a cloud provider.
Also, does anyone know why it’s taken this long? I suspect it’s a technical reason. While one could be cynical, I doubt it’s an intentional business/product decision. Hard spending caps are both excellent product differentiators and could possibly save these providers money as they don’t have to forgive their users when they accidentally over-use a service.
1. Anon1096 · 1791080413 · focus · HN ↗
  It's one part technical, one part a product decision. The technical part is that billing is not actually instant. As a most basic example, a VM reports its billing units every X period of time it is active. If there is some network blip but it's still running, then that billing data could be delayed.
  The product level decision is that "shut down everything" is something the customers you want to target don't actually want. Are we including deleting RDS data? S3? Glacier storage? If so then the headline will just change from "Hobbyist got charged XXXXXX on AWS" to "Business literally had all their data deleted because a hacker took over their VM and mined bitcoin". The only people who really want this are hobbyists and it's not a market segment that's worth chasing. Easier to do the status quo of forgive afterwards then even open the can of worms of deleting all of a business's data and all their backups just because they had a 100k overrun.
  1. Gigachad · 1791081242 · focus · HN ↗
    I think there is a middle ground between deleting data and allowing 5000 VMs to be created to mine bitcoin. Obviously there are a lot of different scenarios to consider but the explosive costs seem to be constrained mostly to a couple of features which would be fairly safe to cap.
    asdfaoeu · 1791082917 · focus · HN ↗
    AWS and similar already have service quotas that cap these.
  2. cj · 1791082098 · focus · HN ↗
    > The only people who really want this are hobbyists and it's not a market segment that's worth chasing. Easier to do the status quo of forgive afterwards then even open the can of worms of deleting all of a business's data and all their backups
    Hit the nail on the head.
    Nearly all businesses would prefer a cost overrun than services going offline.
    simonw · 1791083425 · focus · HN ↗
    Today's hobbyist is tomorrow's decision maker at work over which service to use.
    bryanrasmussen · 1791085806 · focus · HN ↗
    and if they're going to be a good decision maker they need to put their personal feelings from hobbyist times aside and realize stuff is different in the two scenarios.
    necovek · 1791088486 · focus · HN ↗
    If your peak monthly cost for AWS services as a business is $100k, do you not think setting a $200k spending limit is reasonable?
    Obviously, the system should provide ample time by warning in advance of reaching it (and could even offer suggestion to keep it at N times your peak from M months ago).
    If as a business you set your spending limits so tight that you frequently run into them and it's not some unusual activity, the problem is not that spending limits are available :)
    It is mostly about protecting from the unknown, likely unbounded attack on your infrastructure, where your spend might grow 100x: even if you can take $100k, you might not be able to take $10M in a month.
    miki123211 · 1791095133 · focus · HN ↗
    And if you use enough services, a global per-account spending limit can't distinguish between peak usage versus one service being abused.
    Imagine you spend $100k on average each month, but spend around Christmas rises to $1m because of the specifics of your industry. With a global spending limit, you can't distinguish between $200k of general spend increases due to Black Friday versus $200k in fraudulent 2FA SMS to South Sudan.
    bryanrasmussen · 1791097868 · focus · HN ↗
    I agree but I think also, as a programmer, surely this is not an insurmountable problem. The idea that pops into my head immediately when hearing this problem is
    1. of course spending limits are monthly or even weekly basis. You set a yearly limit, probably most people will do something like November to January 4 times the limits set rest of year.
    2. as you near limit calls go to people on your team to tell them you are getting near your limit. Estimate is 2 hours, what do you want. Double Limit for this time period? Triple Limit for This Time Period? Remove Limit Entirely, you will get back to us with potential new limit? It's the beginning of Christmas, the next time your limit resets is the 3rd of January, remove limit entirely and you will get back to them. Why do you decide to remove limit entirely, because you have info that Amazon doesn't, specifically your assassin Nisse doll has gone viral for this Christmas season! The shit is making bank!!
    3. When setting limit you say "expected low usage", "expected high usage", "limit". Limit should be significantly above expected high usage. Service informs you - you have been over your expected high usage by 20% last three time periods. Would you like to increase limit and high usage by 20%? Please Look at your settings otherwise.
    4. Phone calls when there is an unexpected peaking in usage, like one hour we are 300 thousand which is very high for you, next hour it is 1.5 million.
    Obviously none of this stuff helps hobbyists but even the worst run businesses I've worked at would handle this. Otherwise they deserve to be hobbyists, there's no reason to be an organization if you're not organized.
    Obviously these things do not stop fraudulent attacks abusing your service, but it does make it harder for them, at the same time making it more difficult for your stuff to just get shut off without you knowing.
    Of course, as a programmer I am aware that all these services are created by programmers as effective or more effective than I, and who have undoubtedly thought about it more than I, so I must also assume there are reasons why my off the cuff suggestions are ludicrously unhelpful, but I lack the knowledge as to why this should be so.
    lazyasciiart · 1791100550 · focus · HN ↗
    The systems are hopefully created with the help of finance professionals who are familiar with a massive variety of unpredictable cost scenarios.
    zufallsheld · 1791090587 · focus · HN ↗
    How would they know if the service is any good when they didn't try the service as a hobbyist because of spending fears?
    chii · 1791092284 · focus · HN ↗
    They would ask around, as well as perform a proper evaluation based on their current needs, rather than their experienced needs as a hobbyist?
    dingaling · 1791093604 · focus · HN ↗
    Meanwhile while in the real business world, Cloud decisions are shaped by marketing campaigns and it's the hobbyists and old grey-beards who provide the empirical push-back and reality checks.
    tomjen3 · 1791095684 · focus · HN ↗
    You are correct; however, you're also putting a lot of faith in people's ability to make decisions.
    Another thing (that does not really apply at AWS anymore), is that todays's enthusiasts are going to be the future CTOs, and the easiest time to recruit them to your service is when they are still an enthusiast who gets to make decisions on their own because there's exactly one decision maker you have to appeal to and that person really likes to try new stuff.
    That's why you can get a free fly.io and why we all use Tailscale. And it works too — if I was in charge and needed it, I would immediately go with Tailscale for a business; I know it and I use it.
    Symbiote · 1791085878 · focus · HN ↗
    I removed some pay-per-use APIs from our (business) public website after a surprise $4000 bill, caused by an LLM company scraping the site.
    Some were replaced with a competing service which has a limit, others replaced by a self-hosted alternative.
    I think many small businesses would prefer to be offline or have a degraded service than pay $X000.
    fcarraldo · 1791087405 · focus · HN ↗
    yeah, I don’t know why anyone thinks otherwise.
    for personal/hobby accounts sure. for a business, it’s much better to negotiate around billing or adjust systems/processes post-facto than it is to have service cut off unexpectedly.
    debts are easier to manage when you have an active (ideally growing) customer base. you don’t have customers anymore if your cloud account takes down your service for the rest of the month due to spending limits.
    necovek · 1791088094 · focus · HN ↗
    If you are actually a business, a common warning that you are near the limit should mostly resolve it. It might only be tricky because the estimated time remaining is really short if it's a huge recent spike: eg. nobody is looking forward to a notice of "you'll use up your spending limit in 4h" on the weekend.
    This type of warning should give you enough time to investigate if the warning is real and adjust the spending limits.
    But then again, even if you hit them and your services get paused, you'd be increasing the spending limits and restoring services after you are back at work and notice they are down, so it mostly comes down to your incident response times.
  3. 10000truths · 1791083045 · focus · HN ↗
    > Are we including deleting RDS data? S3? Glacier storage?
    If you're billing per GB of storage, then you can put hard caps on storage capacity, and then hard-reject any operation that would take the total stored size over that capacity.
  4. simonw · 1791083269 · focus · HN ↗
    Amazon's new feature for this specifically says that it won't delete any of your data for 90 days:
    > If you take no action within 90 days of your project being paused, AWS permanently deletes your project data.
    From <a href="https://docs.aws.amazon.com/accounts/latest/reference/create-spend-limit.html#spend-limit-reached" rel="nofollow">https://docs.aws.amazon.com/accounts/latest/reference/create...
    eddythompson80 · 1791084407 · focus · HN ↗
    Can I store few petabytes, and pay for it for one day every 90 days to reset the timer?
    sillysaurusx · 1791084793 · focus · HN ↗
    What surprised me about GCP was that yes, you could load all of your training data and model weights (terabytes) into a free starter account, then create a new account after 30 days and transfer the billing obligation from account A to account B.
    It was such a blessing for hobbyists, back in ye olde 2019.
    radicalbyte · 1791088896 · focus · HN ↗
    Given that the cloud providers just put a 0 or two on the cost to determine their prices then yes, you could try.
    They'll ban you after a year because it will be against their TOS.
    But sure, go for it.
    DangitBobby · 1791091653 · focus · HN ↗
    Presumably you'd have to back pay the 90 days but it's your life
  5. cortesoft · 1791085188 · focus · HN ↗
    Yeah, you can't just implement it as a pure "stop all services immediately once I hit a set amount"
    It needs to be more like "don't allow spinning up additional services after you hit this amount", although that still allows you to go over the limit by a lot, since most services are billed hourly.
    It really is difficult to implement a spending cap that doesn't risk shutting down important things.
    onion2k · 1791087367 · focus · HN ↗
    It really is difficult to implement a spending cap that doesn't risk shutting down important things.
    That's a checkbox decision for the customer. There needs to be the option of "This is important, never turn it off and I'll pay for any overages." versus "I want an entirely predictable bill up to $xxx, so stop my stuff as soon as possible over that."
    It's not up to a cloud service to decide my website is more important than my money for me. That's my decision to make.
    chii · 1791092194 · focus · HN ↗
    > That's a checkbox decision for the customer.
    and they would still complain if they got it wrong - it's always the platform/company's fault.
    Look at banks and fraudulent transfers that customers themselves get phished into doing. The bank in the end usually take the hit (after the customer complains long enough). That's why there's all sorts of hoops and such to prevent customers from failing - and that causes friction for people regularly.
    Therefore, the cloud company's decision to default safer is more correct from this perspective.
    throwaway27448 · 1791098773 · focus · HN ↗
    > and they would still complain if they got it wrong - it's always the platform/company's fault.
    I mean, it's really not unless they don't build it in the first place—that really is the platform's fault.
    tomjen3 · 1791095132 · focus · HN ↗
    It's more complicated than that.
    You might be perfectly okay with having certain systems shut down, but you probably still want to pay for the archival storage of your important files.
    That archival storage might be in several places, including one S3 bucket, whereas there might be another one that, contains copies of scraped Craigslist for X where you'd actually be happy that it just shut down.
    This makes it far more complicated to do correctly, and as others pointed out, mostly relevant for hobbyist — this is not something you are going to make a lot of money from.
    Better to spend engineering hours making an MCP for the dashboard or improving your Databricks setup.
  6. amluto · 1791086606 · focus · HN ↗
    A provider could, in theory, calculate the cost of storage for X days and prevent you from uploading an object that would push you over the limit.
  7. Dylan16807 · 1791087389 · focus · HN ↗
    For S3 a reasonable option is a data limit as the primary limit. If you set it a TB over your real needs you only waste one dollar per day after it locks. And there's no need for any other paused service to charge more than that for idle data.
    You're right that nobody wants deletion. Spending limits do not imply deletion.
    judge2020 · 1791087614 · focus · HN ↗
    I think a more sensible default is S3/etc locking access to data for 30 days, maybe even as little as 7 days, while you're able to still list and DELETE said data as you wish, without being able to get or put.
  8. [deleted] · 1791088244 · focus · HN ↗
    [deleted]
  9. Doohickey-d · 1791094497 · focus · HN ↗
    Regarding hobbyists: some clouds, and also the bigger clouds, do target hobbyists quite a lot, presumably with the intention that some of those hobbyists will eventually grow and stick with them. So I don't think "not wanting hobbyists" is entirely true.
    But yes, the amount of money they spend is less, so it makes less sense to implement features that only hobbyists want.
  10. miki123211 · 1791094788 · focus · HN ↗
    A lot of billing systems are organized around event delivery. The system does what it does and reports usage. This reporting is asynchronous and can be done E.G. via cron jobs running on a 24 hour cadence in certain cases. There's an internal guarantee that billing records for a given period are delivered by a certain time. Nobody checks whether the user has enough money to do what they're trying to do, just whether they're authorized to access the system in the first place. Shutting down accounts due to non-payment is more of an abuse / fraud concern, and happens long after the bill is delivered.
    Joker_vD · 1791108305 · focus · HN ↗
    Wow, I wish my my cell provider worked like that! Instead, they somehow manage to track and bill me literally in real time. How do they do it? Do the telcos have some magic pixie dust or what?
  11. beAbU · 1791095032 · focus · HN ↗
    There is a middle ground here: Make this setting configurable, off by default, and with all the associated warnings of what will happen if you switch it on. Better yet, make it settable on each billable service. Keep Route 53 going, but halt and delete any VMs that exceed some limit.
  12. tancop · 1791097253 · focus · HN ↗
    Storage is almost never the main thing racking up bills so it should be handled in a different way. If you hit a limit you can't store any more data but everything you have stays there and readable.
    This is more about compute, VMs, LLM inference and services like hosted database. These are all safe to stop if the system triggers a normal shutdown when costs hit a limit.
    gpt5 · 1791097946 · focus · HN ↗
    Yeah, the storage point made the whole case weaker.
    zbentley · 1791099829 · focus · HN ↗
    I dunno, the largest accidental AWS overages I’ve triaged in my career (at very different companies) were all storage. Dangling EBS, forgotten S3 accumulating data after a deletion cron broke, incompletely retention-policy’d CloudWatch buckets, Aurora snapshots…the list is pretty long.
    “Large” is relative to the business of course, but the biggest storage-related overages I’ve personally triaged are in the high $100ks to low $millions per month. Colleagues have heard of orders of magnitude more costly.
    lazyasciiart · 1791100446 · focus · HN ↗
    That’s not storage costs, that’s activity adding to storage used. The question is whether it would be cheap enough for them to keep all your existing data frozen until you paid the bill.
    hnlmorg · 1791100922 · focus · HN ↗
    You’re literally agreeing with the GP but phrasing it like they’re wrong.
    gpt5 · 1791135527 · focus · HN ↗
    The solution above (no new storage) would solve that problem.
    hnlmorg · 1791100717 · focus · HN ↗
    I’ve recently worked somewhere where S3 read access was the second largest AWS cost.
    I’m not saying it’s normal. Just that AWS is complicated, and people use it in a plethora of ways, thus billing caps aren’t easy to get right either.

Open on Hacker News to reply ↗

Unofficial Hacker News client; not affiliated with Y Combinator.