‹ BackHN Continuity

Thread

How Uber Protects Against Retry Storms

120 points · 49 comments · iscmt

  1. aftbit · · focus · HN ↗
    I'd be interested to hear other strategies in this space. I've done the naive thing of allowing retries everywhere, and gotten into retry storms. When I was next presented with the problem, I tried the other naive thing of only allowing retries from the very top level service, which led me to redoing absolutely tons of work for each failure. What's a nice middle path that doesn't add too much complexity?
    1. otterley · · focus · HN ↗
      <a href="https:&#x2F;&#x2F;aws.amazon.com&#x2F;blogs&#x2F;developer&#x2F;introducing-retry-throttling&#x2F;" rel="nofollow">https:&#x2F;&#x2F;aws.amazon.com&#x2F;blogs&#x2F;developer&#x2F;introducing-retry-thr... (2016 -- 10 years ago!)

      <a href="https:&#x2F;&#x2F;builder.aws.com&#x2F;content&#x2F;3EumjoZascWd1oZiEgL8ORlv3qE&#x2F;timeouts-retries-and-backoff-with-jitter" rel="nofollow">https:&#x2F;&#x2F;builder.aws.com&#x2F;content&#x2F;3EumjoZascWd1oZiEgL8ORlv3qE&#x2F;... (originally published 2020, republished 2026)

      <a href="https:&#x2F;&#x2F;docs.aws.amazon.com&#x2F;sdkref&#x2F;latest&#x2F;guide&#x2F;feature-retry-behavior.html#retry-quota-token-bucket" rel="nofollow">https:&#x2F;&#x2F;docs.aws.amazon.com&#x2F;sdkref&#x2F;latest&#x2F;guide&#x2F;feature-retr...

      <a href="https:&#x2F;&#x2F;aws.amazon.com&#x2F;blogs&#x2F;developer&#x2F;announcing-updated-retry-behavior-for-aws-sdks-and-tools&#x2F;" rel="nofollow">https:&#x2F;&#x2F;aws.amazon.com&#x2F;blogs&#x2F;developer&#x2F;announcing-updated-re... (2026)

Open on Hacker News to reply ↗

Unofficial Hacker News client; not affiliated with Y Combinator.