Thread · HN Continuity

I get that this is for explaining GPT-2, but I really hope laymen don't use it as an example of how modern models work (ex. absolute positional encoding is no longer used)

edit: I know that it mentions its not modern, but these kinds of details have major implications in terms of the representations a model can learn, which is in many ways the most important part!