‹ BackHN Continuity

Thread

UTF-8000: Unlimited UTF-8

134 points · 122 comments · vismit2000

  1. OutOfHere · · focus · HN ↗
    Perhaps we can use a generalized Unicode-like encoding space for LLM text tokens. A text tokenization scheme uses up a few hundred thousand entries, with say one thousand new entries added annually. These can be called amojis, meaning AI mojis.
Open on Hacker News to reply ↗

Unofficial Hacker News client; not affiliated with Y Combinator.