Thread · HN Continuity

We're a Swiss managed hosting provider and run our own network. In August one of our customers was hit by a UDP amplification attack that we estimate peaked at 500 to 600 Gbit/s across all our links combined, several times what our uplinks can carry. Because we provide that customer's internet connection, the impact hit our network directly, and roughly three hours later the attack was broadened to our own services as well.

The write-up is mostly about what we got wrong: our automated detection only covered our own prefixes, not customer prefixes we announce on their behalf. Blackholing at the IXs we connect to only took effect via route servers, not on direct peerings. And we had no way to withhold a prefix from one specific upstream, so we had to build that while under full load. On top of that, our own website runs on the same shared platform as customer apps, so when it was targeted, unrelated applications were affected too.

What eventually ended it was moving exposed applications behind a CDN with DDoS protection. The full timeline is in the PDF postmortem linked from the post. Happy to answer questions.