‹ BackHN Continuity

Thread

Context Language Models

176 points · 51 comments · emersonmacro

  1. Bolwin · · focus · HN ↗
    The biggest discovery might actually be that they ignored regular caching rules and kept invalid cache suffixes and it didn't hurt performance
    1. TeMPOraL · · focus · HN ↗
      I wonder how bad the performance would be if they plain ignored the whole rotary encoding dance and just back-filled precisely the parts of the cache that changed directly. Would it break the model? Confuse the model? Or would the model internally correct for it?
      1. imtringued · · focus · HN ↗
        You can apply rotary encoding on the file level to keep relative positioning within a file. Then there would be no global order of files (which could be an issue) but you can now add or remove whole files as you like.
Open on Hacker News to reply ↗

Unofficial Hacker News client; not affiliated with Y Combinator.