Thread · HN Continuity

> difference between an instruct based re-ranker and laya/jev I just don't see it

Main difference is that laya/jev/et-al give you a zero-shot classifier that requires no training. You can prompt engineer your way to a quick fairly reliable cheap enough decision engine that you can use to iterate quickly (by prompt engineering).

Right now a lot of people are doing this with LLMs and it's too slow and expensive.

Imo the right iterative approach to productionizing these systems is something like:

    1. Build it with an LLM. Iterate on the prompt
    2. Start building a real-world dataset
    3. When the prompt works, turn it into a clear rubric for Jev or similar
    4. Keep iterating until desired accuracy achieved
    5. Use the real-world evals you&#x27;ve built to train a custom classifier fine-tuned to your needs

You now have a system that has produced useful results in production from the very beginning and by the end it's a reliable super cheap classifier that can make thousands of decisions per second.