Thread

1. E-Reverance · 1790026814 · focus · HN ↗

   I get that this is for explaining GPT-2, but I really hope laymen don't use it as an example of how modern models work (ex. absolute positional encoding is no longer used)

   edit: I know that it mentions its not modern, but these kinds of details have major implications in terms of the representations a model can learn, which is in many ways the most important part!

   1. foobazgt · 1790035072 · focus · HN ↗

      This is how pedagogy works. You must simplify something.

      1. E-Reverance · 1790035609 · focus · HN ↗

         I'm talking about bringing up a specific outdated details, not about simplifying it
      2. ViscountPenguin · 1790038017 · focus · HN ↗

         I think a better simplification would be to remove the position-encoding altogether, and just discuss transformers as set models.

Unofficial Hacker News client; not affiliated with Y Combinator.