Infinite-Parameter LLMs: Generating and Adapting Weights from Live Data
Thread
Unofficial Hacker News client; not affiliated with Y Combinator.
Infinite-Parameter LLMs: Generating and Adapting Weights from Live Data
Unofficial Hacker News client; not affiliated with Y Combinator.
lubujackson · · focus · HN ↗
Putting attribution/privacy issues to the side, imagine if any individual could try new approaches to solve a problem/make a discovery and any micro-advancement gets integrated into the model itself, dynamically. This could transform progress from the slow "write a paper, get peer reviewed and published, use published data to inform future work" to a system with a centralized repository of concepts, attempts and results, including failed approaches already tried. How much work do humans waste replicating failed approaches?
Someone completely random halfway around the world could trigger a prompt that solves a blocker that prevents my solution from working. Who cares about AGI or "can models invent anything" when we could have a system that automatically synthesizes individual human thought into a rich network of aggregate human memory.
That's the target OpenAI/Anthropic should be evangelizing, not an AI Daddy Overlord or agentic script kiddie hellscape.
Retr0id · · focus · HN ↗
unrented7977 · · focus · HN ↗
You'd want an enormous vector database at minimum. Text is just completely wrong for models at this scale, you must work in the latent space directly.
unholiness · · focus · HN ↗
Anyway, I'm much more optimistic about the worlds with AIs that make the effort to properly explain themselves.