Writing is fundamentally the transfer of information from your brain to my brain. If you have 1000 bits of semantic information you want to transfer, you can't give 300 bits of semantic information to an LLM and have it fill in the remaining 700, because it doesn't know what those 700 bits are. If it's able to guess those 700 bits correctly, then they aren't true semantic information, and you really only have 300 bits you want to transfer. You might as well transfer those bits to me directly, rather than having the LLM add on an extra superfluous 700 bits that I then have to filter out.
While I acknowledge that this is meant to be framed in the context where the message is fully compressed, I think particularly when the medium is language (usually pretty compressable), then actually anti-aliasing the transfer of information is still 100% relevant! After all, AA is really about making better use of the information that you actually transmit. Compressed or otherwise, 1000 bits of aliasy garbage is not in any way the same as 1000 bits of perfectly antialiased signal.
I like to think of this idea of transfer of information (via words, let's say) from speaker/writer to listener/reader, as actually a fairly normal signal processing situation. One where the constructor of the message during it's 'rasterisation' from continuous thoughts to discrete words, can do the job well (ie: band-limit and 'antialias' the message, fully considering the target sampling domain), or badly (ignore the target domain and just speak/write from the source context, leading to 'aliasing' in the listener's received message).
I know this isn't how many people think about words, but I think the idea of antialiasing is relevant and something that people should think more about (including with respect to AI-generated text).
hatthew · · focus · HN ↗
Writing is fundamentally the transfer of information from your brain to my brain. If you have 1000 bits of semantic information you want to transfer, you can't give 300 bits of semantic information to an LLM and have it fill in the remaining 700, because it doesn't know what those 700 bits are. If it's able to guess those 700 bits correctly, then they aren't true semantic information, and you really only have 300 bits you want to transfer. You might as well transfer those bits to me directly, rather than having the LLM add on an extra superfluous 700 bits that I then have to filter out.
danwills · · focus · HN ↗
I like to think of this idea of transfer of information (via words, let's say) from speaker/writer to listener/reader, as actually a fairly normal signal processing situation. One where the constructor of the message during it's 'rasterisation' from continuous thoughts to discrete words, can do the job well (ie: band-limit and 'antialias' the message, fully considering the target sampling domain), or badly (ignore the target domain and just speak/write from the source context, leading to 'aliasing' in the listener's received message).
I know this isn't how many people think about words, but I think the idea of antialiasing is relevant and something that people should think more about (including with respect to AI-generated text).