Thread

1. k__ · 1789999982 · focus · HN ↗

   Half-OT:

   Is Jev a decoder (e.g., BERT) or is it some kind of encoder (e.g., GPT) that just happens to be trimmed down to only outputting a handful of tokens for the answers and their probability?

   1. Alpha3031 · 1790000596 · focus · HN ↗

      You've got encoder and decoder reversed. BERTs and other models that primarily convert text (or other input) into latent representations are encoders. Models that convert their internal representations back into outputs are the decoders (in the case of GPT et al., autoregressive decoders because they perform this decoding based on past tokens).

Unofficial Hacker News client; not affiliated with Y Combinator.