‹ BackHN Continuity

Thread

AMD's random number generator can't generate a 0?

288 points · 220 comments · BruceEel

  1. jstanley · · focus · HN ↗
    This is not the first RNG bug on Zen 2, I recall after I first got mine that some application or other would quit immediately at startup because rdrand always returned -1, i.e. all 1s. It was fixed with a microcode update.

    Do we now learn that they fixed "always generate all 1s" with "never generate all 0s"??

    EDIT: I've been unable to reproduce the problem on my CPU, FWIW. It's a Ryzen 5 3600.

    EDIT2: OK, update, I can reproduce it with rdrand16, rdrand32 is fine but rdrand16 can never generate all 0s. So my CPU does have this problem!

    1. yk · · focus · HN ↗

          return 4 # Determined by fair dice roll.
      1. Gander5739 · · focus · HN ↗
        <a href="https:&#x2F;&#x2F;xkcd.com&#x2F;221&#x2F;" rel="nofollow">https:&#x2F;&#x2F;xkcd.com&#x2F;221&#x2F; for those not in the know
        1. lathiat · · focus · HN ↗
          And for the full fail story behind it, fail0verflow hacking the PS3 presentation is great and covers the bug: <a href="https:&#x2F;&#x2F;youtu.be&#x2F;DUGGJpn2_zY" rel="nofollow">https:&#x2F;&#x2F;youtu.be&#x2F;DUGGJpn2_zY

          Most of the console hacking talks are great, both informative and entertaining.

          1. einsteinx2 · · focus · HN ↗
            This comic predates that presentation, in fact they use it in their slide deck at 39:00 in your linked video.

            That presentation is awesome though, worth a watch either way!

            1. adastra22 · · focus · HN ↗
              The comic postdates the PS3 DRM bug and jailbreak by many, many years.
              1. Liquid_Fire · · focus · HN ↗
                That seems impossible (unless maybe you meant to write &quot;predates&quot;, and even then it seems to only be about 3 years).

                The comic was published on 9 February 2007 [0].

                The PS3 was first released in November 2006. I haven&#x27;t watched the video yet, but its description says &quot;2010 saw the first hacks for the Playstation 3&quot;.

                [0] <a href="https:&#x2F;&#x2F;xkcd.com&#x2F;221&#x2F;info.0.json" rel="nofollow">https:&#x2F;&#x2F;xkcd.com&#x2F;221&#x2F;info.0.json

        2. Betelbuddy · · focus · HN ↗
          <a href="https:&#x2F;&#x2F;i.imgur.com&#x2F;bwFWMqQ.png" rel="nofollow">https:&#x2F;&#x2F;i.imgur.com&#x2F;bwFWMqQ.png
          1. [deleted] · · focus · HN ↗

            [deleted]

        3. matja · · focus · HN ↗
          CVE-2008-0166 (Debian OpenSSL Predictable PRNG Vulnerability) inspired xkcd&#x2F;221 but this sort of thing happens a lot :)
      2. rbanffy · · focus · HN ↗
        I always think of <a href="https:&#x2F;&#x2F;www.reddit.com&#x2F;r&#x2F;ProgrammerHumor&#x2F;comments&#x2F;5yhl93&#x2F;random_number_generator&#x2F;" rel="nofollow">https:&#x2F;&#x2F;www.reddit.com&#x2F;r&#x2F;ProgrammerHumor&#x2F;comments&#x2F;5yhl93&#x2F;ran...
    2. RandomOnyx · · focus · HN ↗
      Does rdrand32 and then taking the lowest 16 bits of its result yield any zeroes?

      Basically I&#x27;m wondering if it&#x27;s a bug in the version of the instruction that writes to a 16-bit reg, or a bug in the underlying RNG

      1. jstanley · · focus · HN ↗
        Yes it does. rdrand32()%65535 was my first attempt, and generated zeroes at about the expected rate, that&#x27;s why I initially erroneously thought my CPU did not have this problem.
        1. RandomOnyx · · focus · HN ↗
          How about* rdrand32()%65536? Taking the remainder by 65535 doesn&#x27;t take the lowest 16 bits after all

          *: missed a word the first time around

        2. goalieca · · focus · HN ↗
          You should be using &amp;0xFFFF for masking. Your mod is off by 1 too.
          1. jstanley · · focus · HN ↗
            You&#x27;re right, the code was correct but my comment above is wrong.
    3. JdeBP · · focus · HN ↗
      You probably recall <a href="https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=19848953">https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=19848953 .
    4. rbanffy · · focus · HN ↗
      Even if you reproduce the issue, it is not a proof it can&#x27;t generate a zero - just that it&#x27;s very unlikely.

      To prove it, we&#x27;d need to examine the chip and its microcode.

    5. peri-cl · · focus · HN ↗
      Zen 4 reporting in. I&#x27;m unable to reproduce it (7840U).

         $ .&#x2F;a.out | rg &#x27;\b\-?\d\b&#x27; | sort -n | uniq -c
         15281 -2
         15192 -1
         15273 0
         15243 1
         15269 2
      
      I used the GCC intrinsic ( _rdrand16_step ),

          #include &lt;immintrin.h&gt;
          
          short rdrand16() {     &#x2F;&#x2F; gcc -mrdrnd
              short ret;
              while (1 != _rdrand16_step(&amp;ret)) { }    
              return ret;
          }
    6. 0x000xca0xfe · · focus · HN ↗
      I can reproduce it too with rdrand16 on Zen2.

      But it looks like the rdrand16 instruction can produce zeros just fine, it just sets CF=0 erroneously (indicating an error and that the user program should retry).

      So keep that in mind when you try to reproduce it too and use some abstraction that could implement retries internally.

      1. dooglius · · focus · HN ↗
        Good observation, that seems like the most likely explanation. Do you ever see &quot;true&quot; CF=0 (with nonzero arg) or did they just take the lazy approach?
        1. 0x000xca0xfe · · focus · HN ↗
          No, CF=0 occurences seem to be happen frequently and uniformely distributed like valid results at ~1&#x2F;65536, not clustered. Under a minute-long all-core load CF=0 always produces zero, but that&#x27;s to be expected according to the manual.

          Here are some stats:

              Rounds (N): 1000000000
              Failed (F): 15312
              Valid  (V): 999984688
              N&#x2F;65536: 15258.789
              V&#x2F;65536: 15258.555
              Failed, result was zero: 15312
              Failed, result non-zero: 0
              Bucket value for      0: 15312
              Bucket value for      1: 15290
              Bucket value for  65535: 15223
              Min bucket value: 14670
              Max bucket value: 15835
          
          I used this C program to collect them:

              #include &lt;stdio.h&gt;
              #include &lt;stdint.h&gt;
              #include &lt;stdbool.h&gt;
          
              const size_t N = 1000000000; &#x2F;&#x2F; 1e9
          
              struct rdrand16_result {
                  uint16_t n;
                  bool ok;
              };
          
              static inline struct rdrand16_result rdrand16()
              {
                  struct rdrand16_result result;
                  __asm__ __volatile__( &quot;rdrand %0&quot; : &quot;=r&quot; (result.n), &quot;=@ccc&quot; (result.ok) );
                  return result;
              }
          
              int main()
              {
                  size_t buckets[0xFFFF + 1] = { 0 };
                  size_t notok = 0, notok_zero = 0, notok_nonz = 0;
                  for (size_t i = 0; i &lt; N; ++i) {
                      struct rdrand16_result result = rdrand16();
                      ++buckets[result.n];
                      if (! result.ok) {
                          ++notok;
                          notok_zero += result.n == 0;
                          notok_nonz += result.n != 0;
                      }
                  }
                  size_t max = 0, min = N;
                  for (size_t i = 0; i &lt;= 0xFFFF; ++i) {
                      size_t n = buckets[i];
                      min = n &lt; min ? n : min;
                      max = n &gt; max ? n : max;
                  }
                  printf(&quot;Rounds (N): %zu\n&quot;, N);
                  printf(&quot;Failed (F): %zu\n&quot;, notok);
                  printf(&quot;Valid  (V): %zu\n&quot;, N - notok);
                  printf(&quot;N&#x2F;65536: %.3f\n&quot;, (double)N &#x2F; 65536);
                  printf(&quot;V&#x2F;65536: %.3f\n&quot;, (double)(N - notok) &#x2F; 65536);
                  printf(&quot;Failed, result was zero: %zu\n&quot;, notok_zero);
                  printf(&quot;Failed, result non-zero: %zu\n&quot;, notok_nonz);
                  printf(&quot;Bucket value for      0: %zu\n&quot;, buckets[0]);
                  printf(&quot;Bucket value for      1: %zu\n&quot;, buckets[1]);
                  printf(&quot;Bucket value for  65535: %zu\n&quot;, buckets[0xFFFF]);
                  printf(&quot;Min bucket value: %zu\n&quot;, min);
                  printf(&quot;Max bucket value: %zu\n&quot;, max);
                  return 0;
              }
          1. eigenform · · focus · HN ↗
            &gt; but that&#x27;s to be expected according to the manual

            Confusingly, the AMD programming manual (Rev. 3.38 - July 2026) only explicitly states this (&quot;that the result is always zero when CF=0&quot;) in the description of RDSEED, but the Intel SDM mentions this in the description of both instructions.

      2. ComputerGuru · · focus · HN ↗
        Funny. Look up errata AMD-SB-7055: RDSEED Failure on AMD “Zen 5” Processors.

        Zen 5 rdrand16&#x2F;32 return zero with CF=1 on entropy exhaustion and their recommended approach directly leads to the issue you observed: treat all-zero result of rdseed as if cf=0 (failure) and re-roll the dice, effectively recreating the zen 1&#x2F;zen 2 issue all over again!

        They say this might be addressed by a future microcode update… meaning there’s a chance they’ll just patch it to do just that in software. Maybe that’s how they got into this mess in the first place?

        Also, am I a complete idiot or is asserting the relative distribution of a mere 64k possible results a rather easy black box validation test that I would’ve assumed they’d be doing? When I used to write cycle-accurate emulators in the past, that would have been an obvious test to include. This isn’t some arcane instruction no one uses or a really complicated case with deep dependency and&#x2F;or timing issues; it’s like getting rdtsc wrong.

        1. pseudohadamard · · focus · HN ↗
          I was going to suggest exactly that, if you&#x27;re got an RNG, or pretty much anything else for that matter, you need the ability to return some sort of things-went-wrong-somewhere indicator value, and presumably AMD is using 0 to do this. Yes, there&#x27;s also the CF, but the caller may not be checking that, particularly if it&#x27;s being done from a HLL.

          Has anyone checked whether it can return ~0, (signed) -1, the traditional error-return value?

          1. Someone · · focus · HN ↗
            &gt; Yes, there&#x27;s also the CF, but the caller may not be checking that, particularly if it&#x27;s being done from a HLL.

            You can’t call a CPU instruction from a high-level language. You would either use inline assembly or call a library function.

            Either way, not handling CF=0 would be a bug (in your code or in the library function)

      3. shawn_w · · focus · HN ↗
        So it sets too many 0&#x27;s
    7. jamesponddotco · · focus · HN ↗
      If I remember correctly, we had a setting in every Linux server we owned to remove CPU as a RNG seeder for the kernel because of those bugs with AMD CPUs.

      I.e., we had `random.trust_cpu=off nordrand` in `GRUB_CMDLINE_LINUX`.

      1. knorker · · focus · HN ↗
        Adding bad randomness can&#x27;t degrade good randomness, can it?

        I thought the kernel would not replace anything just because it adds a potentially bad source.

        E.g. if you have rand source A, and xor it with rand source B, then you get, at worst, the best of A and B,

        1. jamesponddotco · · focus · HN ↗
          As far as I know that is correct; the kernel was written in a way such that one bad source doesn’t poison the pool. Still, if you know one source is bad, might as well take it out.
          1. adastra22 · · focus · HN ↗
            Why? That would actually reduce randomness. The value of adding sources to the entropy pool has a floor of zero. Worst case scenario, it just provides no extra entropy.
          2. wahern · · focus · HN ↗
            You don&#x27;t know if it&#x27;s bad. Microcode updates might fix it, or break it for that matter. Revision history can be difficult if not impossible to comprehensively catalog.

            What it is is unreliable. And that&#x27;s fine so long as you have other entropy sources. OpenBSD is really good about this. Quite a few drivers for various chipsets and cards exist just to read their RNGs, not actually use them for their primary function (which can be a bummer if you want to use the the device, get your hopes up when you see the driver exists in the tree, then discover the only capability it supports is reading the RNG). If you have a CPU with a known bad rdrand, odds are OpenBSD is still sourcing strong randomness from some other chip in your system (PSP, NIC, etc). And because feeding bad (as opposed to malicious[1]) entropy is harmless[2], they don&#x27;t have to maintain a pile of conditions. Nobody is worse off, and overall everybody is better off, including having stronger getrandom&#x2F;getentropy output, by not trying to be clever.

            [1] <a href="https:&#x2F;&#x2F;blog.cr.yp.to&#x2F;20140205-entropy.html" rel="nofollow">https:&#x2F;&#x2F;blog.cr.yp.to&#x2F;20140205-entropy.html

            [2] Presuming nothing is relying on an entropy estimator. I can&#x27;t remember if Linux finally moved past the entropy estimator nonsense. IIRC they did add a software jitter RNG that runs early to try to set a minimum entropy floor, regardless of hardware sources.

            1. knorker · · focus · HN ↗
              &gt; that&#x27;s fine so long as you have other entropy sources

              Well, if you literally have nothing else, then you don&#x27;t have an option anyway, so the whole question is moot.

              Except yeah if literally the only way to collect entropy in your system is the platform&#x27;s opaque RNG, then sure this means your risk assessment should list that as a SPOF. But by definition these cases only have that option, so you can&#x27;t do anything else.

              In reality, you can probably do something else in all but the most extreme embedded environments.

          3. knorker · · focus · HN ↗
            &gt; if you know one source is bad, might as well take it out.

            Yes and no. Mostly no.

            In a simplified model, it&#x27;s only useless if it adds zero bits of entropy. But if a source that&#x27;s supposed to add 128 bits of entropy only adds 16, well, it&#x27;s still 16.

            I would never trust RDRAND on its own. If nothing else because it&#x27;s always subject to a microcode backdoor. But if I already have something I&#x27;m happy with the entropy of, sure, I&#x27;d XOR it with RDRAND output. It cannot make it worse.

        2. teravor · · focus · HN ↗
          this is generally true however if an adversary is able to control a source it becomes dangerous if they can preview the results or inspect the other sources.
          1. adastra22 · · focus · HN ↗
            Not to the kernel random pool, no.
            1. teravor · · focus · HN ↗
              this is a well known attack...

              if your algorithm controls a source of entropy and can inspect the other sources, it can craft its source to bias the result. a fanciful attack but it means you should at least discriminate what you put into the pool.

              1. tptacek · · focus · HN ↗
                Can you link to the paper you&#x27;re thinking of? Maybe people are just talking past each other here. A biased random source can&#x27;t bias the kernel random pool in any straightforward kind of way.
                1. teravor · · focus · HN ↗
                  I explicitly said

                      &gt; it becomes dangerous if they can preview the results or inspect the other sources
                  
                  because the malicious source can just precompute the hash for the bias it wants.

                  <a href="https:&#x2F;&#x2F;blog.cr.yp.to&#x2F;20140205-entropy.html" rel="nofollow">https:&#x2F;&#x2F;blog.cr.yp.to&#x2F;20140205-entropy.html

                  1. adastra22 · · focus · HN ↗
                    You&#x27;re assuming a hash preimage attack, which would be a complete break of the cryptosystem. (Your link only works on the toy implementation given.)
                    1. teravor · · focus · HN ↗
                      there is no preimage attack involved.

                      while you cannot take control over the hash output you can bias it because you have multiple tries. that&#x27;s how bitcoin mining works too...

                      for cryptographic applications any bias can be engineered to be fatal in one way or another.

          2. knorker · · focus · HN ↗
            Right, so your starting point is that the attacker has read-only access to ALL entropy sources, and in that scenario it&#x27;s worse if the attacker has read-write access to one entropy source.

            Yes. I don&#x27;t find this a particularly interesting scenario, though. Sure, we can come up with stuxnet-like airgap attacks where we on-device, but not remotely, can read entropy sources. AND we can modify the output of RDRAND. And there keys have been generated for data we can later intercept. But despite that control (potentially on a CPU microcode level) we are unable to stegonographically leak it?

            Sure. Possible. Has it ever happened?

        3. edelbitter · · focus · HN ↗
          Careful, there is two different things going on here:

          a) whether you use the maybe-entropy provided by the CPU (and&#x2F;or the bootloader)

          b) whether you credit that maybe-entropy towards your tracking of whether the pool should be considered sufficiently seeded

          random.trust_cpu&#x2F;random.trust_bootloader configures b).

          nordrand has been removed from the kernel as it had become overloaded by meaning both a) and b)

          Under most circumstances, a) is harmless. You mostly want that off when the CPU exhibits some performance hiccups when asked.

          Under some circumstances, b) is outright dangerous. Some applications can work without seeded pool at some slightly reduced performance, but could be made to fail miserably if they had been made to believe that the pool was seeded yet it was not. This happens with hash tables when you skip some of the accounting because it seems no longer relevant. It really would not be relevant, once even a determined attacker should be unable to reliably trigger the worst-case-performance.

          1. mitxela · · focus · HN ↗
            Note that this issue doesn&#x27;t make rdrand useless for entropy. It&#x27;s still as useful as always if passing through any whitening or mixing algorithm.
        4. leni536 · · focus · HN ↗
          &gt; E.g. if you have rand source A, and xor it with rand source B, then you get, at worst, the best of A and B,

          With the assumption that sources A and B are independent from each other.

          1. knorker · · focus · HN ↗
            Sure. In general this is a very important factor.

            In the context of this topic, it&#x27;s a bit pedantic.

            1. leni536 · · focus · HN ↗
              Even if the assumption is reasonable, it is worth explicitly spelling out.
Open on Hacker News to reply ↗

Unofficial Hacker News client; not affiliated with Y Combinator.