‹ BackHN Continuity

Thread

Jeff – Jev-compatible 0.8B decision models, trained at home, ~30 ms

575 points · 225 comments · firelex

  1. imranq · · focus · HN ↗
    Everyone saying you could replace Jev or decision type models with an LLM with bolted schema output constraining are missing the point completely. Its about extreme speed and cost effectiveness with high quality, neither of which you are going to get with LLMs even with these KV-cache tricks
    1. computerex · · focus · HN ↗
      There is no free lunch. You cannot get high quality on open domain and have it be fast.
Open on Hacker News to reply ↗

Unofficial Hacker News client; not affiliated with Y Combinator.