Thread

1. sharih · 1790687184 · focus · HN ↗

   What is the point of this, if it is p90 17 seconds? Might as well use an LLM. The beauty of Jev is that it is dirt cheap and insanely fast.

   1. zihotki · 1790687732 · focus · HN ↗

      I would hold your horses to paint it as dirt cheap.. In my cases for spam detection Luna was 20% cheaper due to prompt caching, although not as fast.

      1. calebhwin · 1790693618 · focus · HN ↗

         How are you benefiting from prompt caching for simple classification?

         1. zihotki · 1790694205 · focus · HN ↗

            There are two parts in the data you supply to Jev for classification - the prompt describing your classification and the data. The data can be quite small - a simple chat message. And prompt part could be considerable since you need to describe your rubrics well.

            With Jev you each time pay for your prompt, you can't cache it.

            1. sarkarghya · 1790701333 · focus · HN ↗

               I mean, it sounds like it's only ideal for cases with significant system prompt overhead. I don't think Jev was built to have a large well described prompt setup. To me its more like a happy go lucky small label classification tool with important decisions left to stronger agentic models or yk humans.

Unofficial Hacker News client; not affiliated with Y Combinator.