‹ BackHN Continuity

Thread

Kev: Tiny Jev-like family of decision models built on top of Qwen3.5

462 points · 211 comments · tosh

  1. nico · · focus · HN ↗
    If you only need classification, and you can provide some training data, you can ask Codex/Claude to build an embeddings + logistic classifier model for you

    For emails, I get 95% accuracy with this method, with only 50-100 examples for training

    Training the model takes less than 5 minutes on a CPU

    The resulting model is <1MB, and inference is sub 100ms

    Some other cool things about this approach:

    * the model doesn’t train on some “ideal” or general classification, instead it learns your preferences

    * the model runs on pretty much any mobile device and can be retrained online on the device

    * privacy, the whole training and inference is 100% local, no data goes anywhere (except whatever you feed codex/claude while building the model)

    Note: to do a more general test, I made a classifier for the Banking77 dataset. The model is <10MB, trains in <30s on CPU and gets 94.5% accuracy, which puts it in the top 5?models by accuracy for that set (the best one is at 94.86%, but it’s 350MB in size and takes hours to train on a GPU).

    1. foofoobar · · focus · HN ↗
      I'm interested in the Banking77 example you gave which I want to reproduce. Can you give some details for this run? Also, does the <10 MB size include the embedding encoder? Would appreciate any configuration or code to look at!
      1. nico · · focus · HN ↗
        Here's a gist with code you can use to test the Banking77 dataset:

        <a href="https:&#x2F;&#x2F;gist.github.com&#x2F;nicobrenner&#x2F;056a5aaff5d0119c0032ecdad5029557" rel="nofollow">https:&#x2F;&#x2F;gist.github.com&#x2F;nicobrenner&#x2F;056a5aaff5d0119c0032ecda...

        The &lt;10MB does not include the embeddings encoder

        The gist uses BAAI&#x2F;bge-large-en-v1.5, which is 1.2GB approx. You can replace it for all-MiniLM-L6-v2 (91 MB @ fp32 or 45 MB quantized fp16) small enough for mobile&#x2F;edge. With all-MiniLM-L6-v2 it still gets 93.0% on Banking77, only 1.3 points behind bge-large at 15x smaller

Open on Hacker News to reply ↗

Unofficial Hacker News client; not affiliated with Y Combinator.