Thread

1. nico · 1790006937 · focus · HN ↗

   If you only need classification, and you can provide some training data, you can ask Codex/Claude to build an embeddings + logistic classifier model for you

   For emails, I get 95% accuracy with this method, with only 50-100 examples for training

   Training the model takes less than 5 minutes on a CPU

   The resulting model is <1MB, and inference is sub 100ms

   Some other cool things about this approach:

   * the model doesn’t train on some “ideal” or general classification, instead it learns your preferences

   * the model runs on pretty much any mobile device and can be retrained online on the device

   * privacy, the whole training and inference is 100% local, no data goes anywhere (except whatever you feed codex/claude while building the model)

   Note: to do a more general test, I made a classifier for the Banking77 dataset. The model is <10MB, trains in <30s on CPU and gets 94.5% accuracy, which puts it in the top 5?models by accuracy for that set (the best one is at 94.86%, but it’s 350MB in size and takes hours to train on a GPU).

   1. samuel · 1790010451 · focus · HN ↗

      Do you realize people is using LLM's as classifiers, right? For lots of companies and developers reaching an API is feasible, while running a training pipeline, no matter how simple, is not. I know that they should still be gathering data for evaluation and they could use it to train a model instead. But they won't do it, for a variety of reasons.

      This is the same route but WAAAY faster and cheaper. And you can modify it like you do with code or prompts. It's really appealing, TBH.

Unofficial Hacker News client; not affiliated with Y Combinator.