‹ BackHN Continuity

Thread

Ollaya – Ollama for open-source, Jev-style decision models

618 points · 145 comments · Ardakilic

  1. alex7o · · focus · HN ↗
    Guys I have a real q, what is the difference between an instruct based re-ranker and laya/jev I just don't see it.

    Edit: One is that jev/laya are tuned to have better probabilities, but a reranker can be fine tuned to do that as well. And jev/laya use RLCD?

    1. Swizec · · focus · HN ↗
      > difference between an instruct based re-ranker and laya/jev I just don't see it

      Main difference is that laya/jev/et-al give you a zero-shot classifier that requires no training. You can prompt engineer your way to a quick fairly reliable cheap enough decision engine that you can use to iterate quickly (by prompt engineering).

      Right now a lot of people are doing this with LLMs and it's too slow and expensive.

      Imo the right iterative approach to productionizing these systems is something like:

          1. Build it with an LLM. Iterate on the prompt
    2. Start building a real-world dataset
    3. When the prompt works, turn it into a clear rubric for Jev or similar
    4. Keep iterating until desired accuracy achieved
    5. Use the real-world evals you've built to train a custom classifier fine-tuned to your needs
      You now have a system that has produced useful results in production from the very beginning and by the end it's a reliable super cheap classifier that can make thousands of decisions per second.
      1. janalsncm · · focus · HN ↗
        I don’t think that’s it. I sincerely doubt most developers are doing side by side comparisons of calibration quality.

        OpenAI has a section on their embeddings model api page for zero shot classification. Of course you can choose an open weights embedding too if you’d like.

        <a href="https:&#x2F;&#x2F;developers.openai.com&#x2F;cookbook&#x2F;examples&#x2F;zero-shot_classification_with_embeddings" rel="nofollow">https:&#x2F;&#x2F;developers.openai.com&#x2F;cookbook&#x2F;examples&#x2F;zero-shot_cl...

        I think Jev wins on marketing and convenience. Most SWEs don’t want to talk about embeddings, cosine similarity, or precision&#x2F;recall tradeoffs. They want something which plausibly works and is easy to use.

        1. Swizec · · focus · HN ↗
          Yes it turns all that work of building a classifier into an api call. This is hugely valuable for prototyping and while you iterate on what the product should even do.
Open on Hacker News to reply ↗

Unofficial Hacker News client; not affiliated with Y Combinator.