‹ BackHN Continuity

Thread

Kev: Tiny Jev-like family of decision models built on top of Qwen3.5

462 points · 211 comments · tosh

  1. nico · · focus · HN ↗
    If you only need classification, and you can provide some training data, you can ask Codex/Claude to build an embeddings + logistic classifier model for you

    For emails, I get 95% accuracy with this method, with only 50-100 examples for training

    Training the model takes less than 5 minutes on a CPU

    The resulting model is <1MB, and inference is sub 100ms

    Some other cool things about this approach:

    * the model doesn’t train on some “ideal” or general classification, instead it learns your preferences

    * the model runs on pretty much any mobile device and can be retrained online on the device

    * privacy, the whole training and inference is 100% local, no data goes anywhere (except whatever you feed codex/claude while building the model)

    Note: to do a more general test, I made a classifier for the Banking77 dataset. The model is <10MB, trains in <30s on CPU and gets 94.5% accuracy, which puts it in the top 5?models by accuracy for that set (the best one is at 94.86%, but it’s 350MB in size and takes hours to train on a GPU).

    1. rgbrgb · · focus · HN ↗
      love this idea. did you try comparing to jev?
      1. nico · · focus · HN ↗
        Yes, I ran some benchmarks. This architecture seems to match or beat Jev and Laya in all basic classification tasks (datasets tested: AG News, Emotion, MASSIVE Intent, Banking77)

        The type of task in which it does really well, especially against Laya, is classification with >50 classes

        But this architecture has no “reasoning”, so it performs rather poorly on tasks that require it, like the ones from the XLNI dataset (Jev/Laya do a lot better on this one)

        For the latter cases, you could probably enhance the architecture with a lightweight LLM, something like a Gemma model. Or even some basic MLP

    2. samuel · · focus · HN ↗
      Do you realize people is using LLM's as classifiers, right? For lots of companies and developers reaching an API is feasible, while running a training pipeline, no matter how simple, is not. I know that they should still be gathering data for evaluation and they could use it to train a model instead. But they won't do it, for a variety of reasons.

      This is the same route but WAAAY faster and cheaper. And you can modify it like you do with code or prompts. It's really appealing, TBH.

    3. 0x457 · · focus · HN ↗
      I built a whole thing that collects data, trains classifiers, exports models and dataset just for that. Claude writes me a terraform file that contains shape of the classifier and dataset. For images it can create datasets based of another dataset (crop this region from images that have these labels).

      Originally it was so I can label data to fine-tune a VLM, but now a few tiny classifiers that run in milliseconds on cpu.

      Now its collecting data to make a domain specific BERT and do what Jev does.

      1. nico · · focus · HN ↗
        Very cool. What kinda of classifications are you running? How big are the models/training sets?

        Also curious about if you plan on doing some sort of routing for the requests. Like detecting the type of task to decide which model to route the request to

        1. 0x457 · · focus · HN ↗
          Some classifiers are tiny - like 2k params, maybe even less.

          This whole thing started because I wanted something to help me play Dune Imperium. Even relatively large models with vision encoders couldn't reliably extract the full state of the board. Now that I have ~2k labeled screenshots, I want to train heads on top of SigLIP2 to extract all of that data in one go.

          That's how it started. Now the thing supports multiple kinds of datasets:

            Images - currently the Dune Imperium and Bolatro screenshots, with SigLIP2 heads being the next step.

  STT - my self-hosted Linux dictation tool feeds this dataset. I run Nemotron ASR tuned for my voice.

  TTS - for Piper TTS, trained to speak like SHODAN. Trained from data generated by Qwen3-tts + original video games files.

  Text pairs - for a 1.2B model that converts normal text into "what would SHODAN say?"


  FastApply - a Qwen3.5-4B LoRA adapter for doing fast edits.

  Chat threads - all agent/chat threads get saved too, so eventually I can turn the useful ones into a dataset and train a LoRA for a really good Rust-specialized version of Qwen3.8-27B.

  Tool calls (extracted from chat threads) - this is where I want something Jev-like, mainly to add an auto-approval mode to my agent harness.
          A model router isn't planned because I'm trying to gear everything toward self-hosting, and there just isn't that much to route between. I’ll probably build something Jev-like for smart-home control, though.

          The FastApply dataset is already ~20k entries, with the majority of outputs being 8k–16k tokens. The STT dataset is roughly 30 hours and growing.

          Basically, the whole thing has turned into a Collect -> Distill -> Train pipeline for whatever I happen to need.

          1. nico · · focus · HN ↗
            > Basically, the whole thing has turned into a Collect -> Distill -> Train pipeline for whatever I happen to need

            Amazing, thank you for sharing your setup. Very cool applications

          2. exographicskip · · focus · HN ↗
            > TTS - for Piper TTS, trained to speak like SHODAN

            I didn't know I wanted this! Can just imagine asking how to make a Trinidad Sour and being called an insect lmao

            1. 0x457 · · focus · HN ↗
              It still requires some post-processing to add glitches and metal sound, final result is like this this: <a href="https:&#x2F;&#x2F;drive.google.com&#x2F;file&#x2F;d&#x2F;1CJkK9nHQNBPMr6sIxVmlq8gaxDGwaB0b&#x2F;view?usp=sharing" rel="nofollow">https:&#x2F;&#x2F;drive.google.com&#x2F;file&#x2F;d&#x2F;1CJkK9nHQNBPMr6sIxVmlq8gaxDG...

              I had a version that on validation dataset nearly 1:1 across entire spectrum, but I ended up linking this revision more.

              Also had qwen3-tts version, but no matter what I do it sounded more like brat Cortana. The way piper-tts work under the hood make it sound more machine-like.

              1. andai · · focus · HN ↗
                Nice. I haven&#x27;t played System Shock but this reminded me of the Adjutant from StarCraft.
      2. swyx · · focus · HN ↗
        &gt; Now its collecting data to make a domain specific BERT and do what Jev does.

        this misses the point of jev somewhat - the point is that this is a foundational, general purpose classifier model - see some good sources <a href="https:&#x2F;&#x2F;x.com&#x2F;mparakhin&#x2F;status&#x2F;2101683565520199887?s=12" rel="nofollow">https:&#x2F;&#x2F;x.com&#x2F;mparakhin&#x2F;status&#x2F;2101683565520199887?s=12

        1. 0x457 · · focus · HN ↗
          Missing the point that they marketing, but so far I saw it perform good at some tasks, and pretty bad on others. I don&#x27;t think it&#x27;s all that general purpose they claim it to be.
    4. mjyoke1111 · · focus · HN ↗

      [dead]

    5. constantlm · · focus · HN ↗
      I did this yesterday and it works incredibly well. I finetuned ModernBERT to classify documents. With zeroshot it achieved around ~30% accuracy, which jumped to 98.2% with finetuning, and latency of around 150ms on my Macbook. Just incredible!
    6. tk90 · · focus · HN ↗
      I&#x27;m working on exactly this! I&#x27;m building a small model that classifies the correct DOM node containing an HTML&#x27;s article content&#x2F;title&#x2F;date&#x2F;author (given a raw html with a lot of noise&#x2F;chrome). A fun learning exercise :)

      30KB model, 40-50ms inference. Pretty happy with the results so far!

      I can see an entire industry of tiny models like this, now that we have AI to help us do the grunt setup work (validation&#x2F;training data creation, data cleaning, etc). Or just use a general classifier like Jev&#x2F;Kev ha

      1. ozozozd · · focus · HN ↗
        30kb model is super impressive.

        What’s the model architecture?

      2. bicepjai · · focus · HN ↗
        Is this like for cleaning html data from say common crawl ?
      3. Flere-Imsaho · · focus · HN ↗
        This. I see LLMs building smaller bespoke models on-the-fly to achieve a desired goal.
    7. e12e · · focus · HN ↗
      &gt; For emails, I get 95% accuracy with this method, with only 50-100 examples for training

      I&#x27;m assuming you only need to consider a single language for your emails?

      1. nico · · focus · HN ↗
        Mostly 2 languages

        Haven’t tried with more

        Do you have a specific use case?

        1. e12e · · focus · HN ↗
          I would need at least two languages, probably three. I was guessing this approach might be less multilingual than LLMs.

          In my case Norwegian, English and Japanese.

          1. dotancohen · · focus · HN ↗
            Are you in the Salmon business?
    8. psadri · · focus · HN ↗
      Now that the models are smarter &#x2F; know more than most of us, a new bottleneck is finding out what you don&#x27;t know. If you don&#x27;t know about logistic classifiers and how they could be applied to your problem, you will not ask for it.

      These days, I tend to start my coding sessions by the high level problem I&#x27;m trying to solve vs the prescriptive, specific solution I may have in mind. It often surfaces ideas and approaches that I did not know about.

      1. dotancohen · · focus · HN ↗

          &gt; It often surfaces ideas and approaches that I did not know about.
        I&#x27;d love to hear some examples. I do agree that tech is moving at a pace that I can no longer keep up, however that absolutely does not mean that I want a dependency on some project that didn&#x27;t even exist this time last year.
    9. roseway4 · · focus · HN ↗
      If you don’t see good performance with LRs, you may want to try RBF SVMs. We’ve found they work super well for our use cases with the embeddinggemma model as they can better separate classes in the non-linear embedding space.

      Our resulting RBF models are tiny and fit in L1 cache, with microsecond inference latency.

    10. foofoobar · · focus · HN ↗
      I&#x27;m interested in the Banking77 example you gave which I want to reproduce. Can you give some details for this run? Also, does the &lt;10 MB size include the embedding encoder? Would appreciate any configuration or code to look at!
      1. nico · · focus · HN ↗
        Here&#x27;s a gist with code you can use to test the Banking77 dataset:

        <a href="https:&#x2F;&#x2F;gist.github.com&#x2F;nicobrenner&#x2F;056a5aaff5d0119c0032ecdad5029557" rel="nofollow">https:&#x2F;&#x2F;gist.github.com&#x2F;nicobrenner&#x2F;056a5aaff5d0119c0032ecda...

        The &lt;10MB does not include the embeddings encoder

        The gist uses BAAI&#x2F;bge-large-en-v1.5, which is 1.2GB approx. You can replace it for all-MiniLM-L6-v2 (91 MB @ fp32 or 45 MB quantized fp16) small enough for mobile&#x2F;edge. With all-MiniLM-L6-v2 it still gets 93.0% on Banking77, only 1.3 points behind bge-large at 15x smaller

Open on Hacker News to reply ↗

Unofficial Hacker News client; not affiliated with Y Combinator.