Writing is fundamentally the transfer of information from your brain to my brain. If you have 1000 bits of semantic information you want to transfer, you can't give 300 bits of semantic information to an LLM and have it fill in the remaining 700, because it doesn't know what those 700 bits are. If it's able to guess those 700 bits correctly, then they aren't true semantic information, and you really only have 300 bits you want to transfer. You might as well transfer those bits to me directly, rather than having the LLM add on an extra superfluous 700 bits that I then have to filter out.
I’m not disagreeing with your preference for human authored writing, but I don’t know that I agree with the content of your argument.
If you consider that what humans are doing during conversation is a form of compressed encoding / decoding from some latent representation through a quantized signal then if you interpret it that as a compressed sensing problem you absolutely can infer to a very close approximation the original latent representation using far fewer than those 1k bits.
From an information-theoretic perspective, if I can effectively transfer a latent concept to you in 300 bits of quantized communication, then by definition the latent concept itself isn't more than 300 bits. Any bit over 300 in the communication I send to you is fluff.
hatthew · · focus · HN ↗
Writing is fundamentally the transfer of information from your brain to my brain. If you have 1000 bits of semantic information you want to transfer, you can't give 300 bits of semantic information to an LLM and have it fill in the remaining 700, because it doesn't know what those 700 bits are. If it's able to guess those 700 bits correctly, then they aren't true semantic information, and you really only have 300 bits you want to transfer. You might as well transfer those bits to me directly, rather than having the LLM add on an extra superfluous 700 bits that I then have to filter out.
FuckButtons · · focus · HN ↗
If you consider that what humans are doing during conversation is a form of compressed encoding / decoding from some latent representation through a quantized signal then if you interpret it that as a compressed sensing problem you absolutely can infer to a very close approximation the original latent representation using far fewer than those 1k bits.
hatthew · · focus · HN ↗