Thread

1. MarkusWandel · 1790365137 · focus · HN ↗

   Did he though? TMK Shannon did specifically not give a way to deal with noisy channels, no. He invented a way to quantify what could be sent on a noisy channel if you figure out the optimal way to do it. This is very similar to "no matter what you figure out about faster travel, you can't go faster than light in a vacuum".

   Calculating the limit is easy with Shannon's theorem. Approaching it in practice is hard.

   1. PunchyHamster · 1790365373 · focus · HN ↗

      > Calculating the limit is easy with Shannon's theorem. Approaching it in practice is hard.

      Many modern modulations already operate basically on Shannon's limit for a given band/SNR. Well, on raw data, the encoding almost always use some kind of error correction so the decoded bitrate is few % lower than the wire one

      1. tverbeure · 1790371467 · focus · HN ↗

         That’s understating the importance of the error coding. The modulation scheme is almost a side show, it’s the modern error coding algorithms (LDPC or Turbo coding) that allows you to get arbitrary close to the Shannon limit.
   2. ironqcold · 1790366601 · focus · HN ↗

      Fair distinction
   3. kragen · 1790438256 · focus · HN ↗

      If I recall correctly, he invented the optimal way to do it and published it in the original paper: use an arbitrarily long random code. The difficulty is that decoding a random code in the obvious way (compare the received codeword against each codeword in the codebook and decode as the one with the lowest Hamming distance) requires an exponentially large amount of both memory and computation. As I understand it, the advances since then have all been about how to get closer to the Shannon limit with reasonable amounts of computation by using codewords that aren't truly random.

Unofficial Hacker News client; not affiliated with Y Combinator.