Thread

1. kqgnkqgn · 1790019316 · focus · HN ↗

   Building multiple diverse paths and monitoring for fiber cuts is not hard. Having 2 fiber paths is insufficient for even only moderately important workloads at a tech company. Overlapping fiber cuts happen. For something with significant economic and safety impact this is just crazy.

   Sometimes the level of incompetence / lack of care in organizations like this astounds me. I understand issues like this can be complicated and systemic but it honestly makes me think very poorly of the technologists building these systems in government.

   1. woodruffw · 1790019872 · focus · HN ↗

      They had two diverse paths, but apparently did not have a (reliable?) process for ensuring the backup path was functional during fail-over. I imagine there will be some soul-searching over that.

      Nothing I've ever seen or experienced with FAA indicates a lack of care; the parsimonious explanation is almost always that people who care a great deal are working within complex systems that don't always have a consistent or externally legible set of priorities. Or more intuitively, the failures we see are the "acceptable" ones versus the unacceptable ones (like planes falling out of the sky).

      1. OutOfHere · 1790020319 · focus · HN ↗

         The problem is with considering the second path a backup path. It is a common problem with things that are considered to be a backup. In contrast, both paths should ideally have been in regular use. As a general policy, for resiliency, one should always exercise all of one's routes/suppliers/vendors, even if some are suboptimal, not just the primary one.

         1. woodruffw · 1790020726 · focus · HN ↗

            I completely agree. My comment is not to suggest there wasn't a problem or failure (clearly there was one!), but that we shouldn't assume incompetence versus conflicting priorities under limited resources.

Unofficial Hacker News client; not affiliated with Y Combinator.