‹ BackHN Continuity

Thread

Saving another 100TB of RAM

488 points · 123 comments · f311a

  1. officialchicken · · focus · HN ↗

    [dead]

    1. agosta · · focus · HN ↗
      We can tell you didn't read the post because it is definitively NOT garbage. Very interesting write up by the Cloudflare team - the man literally did calculus to improve something. When's the last time any of us did Calculus to improve anything? Bang up job Kevin and everyone!!
    2. cyberpunk · · focus · HN ↗
      Anyone have an idea how it behaves differently from google's jump hash algorithm? The cool thing about google's one is it's so short I can include it in a HN comment:

          int32_t JumpConsistentHash(uint64_t key, int32_t num_buckets) {
      int64_t b = 1, j = 0;
      while (j < num_buckets) {
        b = j;
        key = key * 2862933555777941757ULL + 1;
        j = (b + 1) * (double(1LL << 31) / double((key >> 33) + 1));
      }
      return b;
    }
      <a href="https:&#x2F;&#x2F;arxiv.org&#x2F;pdf&#x2F;1406.2294" rel="nofollow">https:&#x2F;&#x2F;arxiv.org&#x2F;pdf&#x2F;1406.2294
      1. cyberpunk · · focus · HN ↗
        Well I looked it up; nginx, apparently, uses ketama -- it&#x27;s a ring-style hash probably works better for web backends than the jch above, as when given [0,1,2,3] and replacing the server in slot 1 you&#x27;re going to have a lot of hash moves. With ketama, you&#x27;d only have the &#x27;1&#x27; hashes moving. You can&#x27;t really beat google&#x27;s for brevity, though.
      2. prirun · · focus · HN ↗
        I have used Google&#x27;s jump hash. As I recall, one of the main differences is that jump hash doesn&#x27;t have a mechanism to remove targets, eg, a server dies and you don&#x27;t want to route requests to it. Traditional consistent hashing can do that. I guess if you had 4 servers, server #4 dies, then you can go back to 3 servers by just changing num_buckets from 4 to 3. But if server 1 dies, you can&#x27;t.

        Jump hash does allow adding more targets and preserves the property that most request targets stay the same when adding a new target, so if you had 3 targets and add a fourth, ~8% of the requests that would have been sent to targets 1-3 are sent to target 4, evenly chosen from servers 1-3.

    3. globnomulous · · focus · HN ↗
      If you have a real, actual, substantive critique of either the post or package itself, I&#x27;d be interested in reading that. What you posted doesn&#x27;t provide that. I&#x27;m not sure who you&#x27;re talking to or what you expect your comment to accomplish.
    4. terabyteoff · · focus · HN ↗
      You got me! I keep meaning to do this.

      <a href="https:&#x2F;&#x2F;github.com&#x2F;cloudflare&#x2F;pingora&#x2F;issues&#x2F;1014" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;cloudflare&#x2F;pingora&#x2F;issues&#x2F;1014

      I have now solved the problem once and for all

Open on Hacker News to reply ↗

Unofficial Hacker News client; not affiliated with Y Combinator.