Thread

1. vatsachak · 1790797844 · focus · HN ↗

   I wonder what the "waves" in LLMs are; The global firing patterns that can help us interpret what the weights are doing en-masse

   1. randomImmigrant · 1790809346 · focus · HN ↗

      Why would you assume waves with LLMs? What about the structure or function of an artificial neuron prompts you to hypothesize they generate continuous waves in a bunch of nested frequency ranges?

      1. peddling-brink · 1790827106 · focus · HN ↗

         > "waves"

         OP was postulating that there might be higher order patterns we can see arise out of underlying LLM mechanics.

         1. HarHarVeryFunny · 1790855921 · focus · HN ↗

            Brain waves are due to feedback paths and dynamics - oscillations etc, but LLM connectivity is far simpler. In an LLM there is a strict bottom-up one-way flow of information, with no looping. There is no way for a higher up layer to affect a lower down layer, whether directly or indirectly.

Unofficial Hacker News client; not affiliated with Y Combinator.