Thread · HN Continuity

I have read somewhere that Transformer architecture has a quadratic cost [0] (which explains the high costs associated with LLMs and the difficulty for constant improvement without state size pockets).

For what I understand PSSA belongs to a line of research for LLMs with scalable architecture because you don't need to load the full KV in memory to generate a single token:

[0] <a href="https://aclanthology.org/2023.findings-emnlp.936/" rel="nofollow">https://aclanthology.org/2023.findings-emnlp.936/

<a href="https://arxiv.org/abs/2503.00392" rel="nofollow">https://arxiv.org/abs/2503.00392

<a href="https://papers.nips.cc/paper_files/paper/2023/hash/6ceefa7b15572587b78ecfcebb2827f8-Abstract-Conference.html" rel="nofollow">https://papers.nips.cc/paper_files/paper/2023/hash/6ceefa7b1...