‹ BackHN Continuity

Thread

Jeff – Jev-compatible 0.8B decision models, trained at home, ~30 ms

575 points · 225 comments · firelex

  1. vektormemory · · focus · HN ↗
    Can someone remove the extra LLM and just have an embedder do the classifier work?

    It's turning into pimp my llm...

    1. nojs · · focus · HN ↗
      Wait until you hear about support vector machines!
      1. abhgh · · focus · HN ↗
        Its funny - I was going to leave a similar comment - and I have, earlier, on a different thread. If people need fast classification, on a fairly scoped problem, it is very fruitful to start with an off-the-shelf embedding model like ModernBERT (which Laya uses) and stick a classifier in front - like a Support Vector Machine (SVM). For starters just tune the SVM, you don't even have to fine-tune the embedder - often it works very well, esp. given the compute needed. Plus you can get reliable confidence scores and generate explanations if you want them (using something like SHAP).
Open on Hacker News to reply ↗

Unofficial Hacker News client; not affiliated with Y Combinator.