Thread

Launch HN: Magnitude (YC S25) – Self-optimizing inference engine for agents

194 points · 99 comments · anerli

NKosmatos · 1790847312 · focus · HN ↗
Nice one! Tried it and unfortunatelly there are no small models that can fit my 16GB RAM or GTX1650 4GB GPU. Yeah, I know that this configuration is not meant to be used for AI/LLM work, but it would be good to provide support for some smaller models so that us plebeians can also play a bit with what you techbros are used to ;-) There are many small/very small models out there and I'm sure you could add a couple just for playing around and experimenting.
1. anerli · 1790877324 · focus · HN ↗
  We have a couple bugs we are patching where we are reserving too much memory overhead, you should actually be able to fit a couple different models on there!
  Plus in the near future, we'll add ways to automatically utilize your RAM as well (such as expert-streaming).
  Our goal is to make the best use of whatever hardware you have, even if its not high end!

Unofficial Hacker News client; not affiliated with Y Combinator.