‹ BackHN Continuity

Thread

AI coding has made CI a bottleneck, so we reworked ours to keep up

317 points · 408 comments · julian_digital

  1. azkalam · · focus · HN ↗
    If you can pay the setup cost, Bazel will get you build times ~10s with a warm cache for even massive projects.
    1. manquer · · focus · HN ↗
      Can't comment on Bazel specifically, but having worked with both nx and turbo, the bottleneck was usually network and disk IOPS rarely compute.

      Even fully cached outputs needs to fetched and read from a remote server[1]. A step n-1 outout fetched from remote cache server need to written to disk and then again read by step n[3] - all disk I/O and network bound operations.

      10s may be achievable/realistic goal in the Java/C++ world where Bazel normally seen. In TS eco-system most people would be over the moon to get into ballpark of 1-2m for a decently large monorepo.

      We should define Build more clearly here, if you mean running just transpile/compile steps or the full series of steps that includes tests (as the linear post here is talking about). It is hard to see even a small sub-set of a large suite of test that require a virtual DOM or a real browser can run in 10s or less.

      [1] Typical for say managed CI setup .

      [3] Common run-of-the-mill frontend + backend stacks in different languages etc.

      1. criemen · · focus · HN ↗
        If you don't need to rebuild anything, bazel can fetch only the final artifact (not the intermediates) from the remote cache.

        Also, if you have persistent CI workers with a persistent bazel instance, you save on some network roundtrips, but that's obviously harder to set up and make bulletproof.

        1. rienbdj · · focus · HN ↗
          Bazel can even skip the final artefact!
          1. criemen · · focus · HN ↗
            well for test runs you kinda need the binary to run, but you're correct, if the job is just "does this build" no download necessary.
Open on Hacker News to reply ↗

Unofficial Hacker News client; not affiliated with Y Combinator.