‹ BackHN Continuity

Thread

Book review: Is parallel programming hard, and, if so, what can you do about it?

149 points · 66 comments · ahelwer

  1. hoistway · · focus · HN ↗
    Spent days tracking down a deadlock that only manifested under specific load. Definitely hard, even with good tooling.
    1. kccqzy · · focus · HN ↗
      What kind of tooling were you using? IMO, deadlocks are some of the easiest concurrency bugs to diagnose. If you can see the thread stacks, it is easy to see threads are blocked from acquiring a lock. If you can attach a debugger, it is easy to see which locks are involved. Then you can pretty much figure things out using the straightforward guideline that if multiple locks are involved, they must be acquired in the same order in all code paths.

      I don't want to devalue your experience, but I am surprised to hear that. Livelock is harder to debug. Silent data corruption caused by missing or wrong synchronization is way harder to debug.

      1. afdbcreid · · focus · HN ↗
        They are, if you can attach a debugger. If only one thread is stuck and in production... Not so much (but there are still much harder bugs).
        1. gpderetta · · focus · HN ↗
          ah, that's probably a missed wakeup problem! Much nastier. Still seeing where your thread is blocked might hint you to where the missing signal should have been.
          1. afdbcreid · · focus · HN ↗
            I didn't mean literally a single thread, just that not the whole application is hang.
            1. gpderetta · · focus · HN ↗
              So I actually had this exact issue where we suspected a deadlock, but the application was limping along and we couldn't justify attaching a debugger. I managed to confirm my suspicions with judicious use of perf and /proc/<pid>/task/<tid>/wchan .
        2. kccqzy · · focus · HN ↗
          Just drain the traffic (make the load balancer temporarily not send traffic to it) and attach a debugger. I hope your load balancer has that feature because health checking also needs it.
Open on Hacker News to reply ↗

Unofficial Hacker News client; not affiliated with Y Combinator.