Thread

1. JCharante · 1785957691 · focus · HN ↗

   I have done my own testing and found that smaller models can beat their larger siblings on fact retrieval from documents. I haven’t investigated it in depth with a large enough dataset but my guess is that larger models overthink it while smaller ones just do it. I would like if they compared this with 5.6 Luna instead.

   1. seahyinghang8 · 1785982383 · focus · HN ↗

      founder of castform here again - slightly unrelated to retrieval but on the topic that folks are discussing here, i was actually collecting benchmarking various coding traces for the purpose of training a model router and surprisingly, luna held up very well against sol and terra. it was able to solve close to >95% the that sol can handle at a fraction of the cost. have not benchmarked the OSS models yet but will add the popular ones to the list like Deepseek Flash and Kimi k3 to see how they fare.

      will share the full results soon!

Unofficial Hacker News client; not affiliated with Y Combinator.