Thread

Telstra outage: The night a network decided the year was 2006

93 points · 34 comments · TMWNN

vesinisa · 1789708269 · focus · HN ↗
Is it really so that Time Division Multiplex timing in cellular networks is based on NTP time? I stopped reading after the article implied this - it doesn't sound plausible at all.
1. bigfatkitten · 1789709054 · focus · HN ↗
  No, it absolutely is not.
  That timing requires a very tight time, frequency and phase reference. It’s derived from local high stability GNSS-disciplined time references.
  1. Maxious · 1789738253 · focus · HN ↗
    [dead]
2. [deleted] · 1789712182 · focus · HN ↗
  [deleted]
3. hdgvhicv · 1789719105 · focus · HN ↗
  I would assume it’s similar to dab etc, and it’s ptp that carried that data on IP. In the broadcast tv world we use ptp, or more traditionally black and burst, or word clocks in audio.
  The source is still the same - your grandmaster (ptp) or sync generator (b+b) is fed from a good timing source - often GPS, with one or two inputs, but sometimes local atomic clocks.
  I’ve seen issues before with 40ms audio gaps when a GPS antenna has died and the coders can’t cope with frame drift so they incentive a 40ms silence in the audio every 40 minutes or so as the source and destination clocks are different speeds by about 1 part in 60,000.
  In DAB the tolerances are far lower, there was a problem about 10 years ago due to a GPs satellite being a few microseconds off
  <a href="https://www.radioworld.com/trends-1/gps-anomalies-caused-problems-for-dab-around-the-uk" rel="nofollow">https://www.radioworld.com/trends-1/gps-anomalies-caused-pro...
4. kcexn · 1789719898 · focus · HN ↗
  No. The official report seems to suggest that software in IMS core (the subsystem that controls VOIP signalling in the mobile network) bugged out after the time was shifted back 20 years. That doesn't narrow down the specific failure that stopped calls from going through very much. IMS core is full of timers, but it could also have been a license server issue for all the report said about it.
  1. SulphurCrested · 1789727239 · focus · HN ↗
    I haven’t read the report, but I did read at the time that it was a certificate validation failure: clients thought the current time was before the "Not Valid Before" date. I forget now what essential service became unavailable because it was disbelieved.
    (Source would either have been the Whirlpool forums – where Australian ISP and telco engineers can be found – or local Australian mass media.)
    kcexn · 1789729509 · focus · HN ↗
    That's also plausible.
    vesinisa · 1790097173 · focus · HN ↗
    This the most common failure mode I have seen with clocks being widely out of sync. You can't access many online services as the certificates are either dated to future or expired.

Open on Hacker News to reply ↗

Unofficial Hacker News client; not affiliated with Y Combinator.