Thread

1. nico · 1790006937 · focus · HN ↗

   If you only need classification, and you can provide some training data, you can ask Codex/Claude to build an embeddings + logistic classifier model for you

   For emails, I get 95% accuracy with this method, with only 50-100 examples for training

   Training the model takes less than 5 minutes on a CPU

   The resulting model is <1MB, and inference is sub 100ms

   Some other cool things about this approach:

   * the model doesn’t train on some “ideal” or general classification, instead it learns your preferences

   * the model runs on pretty much any mobile device and can be retrained online on the device

   * privacy, the whole training and inference is 100% local, no data goes anywhere (except whatever you feed codex/claude while building the model)

   Note: to do a more general test, I made a classifier for the Banking77 dataset. The model is <10MB, trains in <30s on CPU and gets 94.5% accuracy, which puts it in the top 5?models by accuracy for that set (the best one is at 94.86%, but it’s 350MB in size and takes hours to train on a GPU).

   1. rgbrgb · 1790009713 · focus · HN ↗

      love this idea. did you try comparing to jev?

      1. nico · 1790010875 · focus · HN ↗

         Yes, I ran some benchmarks. This architecture seems to match or beat Jev and Laya in all basic classification tasks (datasets tested: AG News, Emotion, MASSIVE Intent, Banking77)

         The type of task in which it does really well, especially against Laya, is classification with >50 classes

         But this architecture has no “reasoning”, so it performs rather poorly on tasks that require it, like the ones from the XLNI dataset (Jev/Laya do a lot better on this one)

         For the latter cases, you could probably enhance the architecture with a lightweight LLM, something like a Gemma model. Or even some basic MLP

Unofficial Hacker News client; not affiliated with Y Combinator.