Thread

A single function Jev-like wrapper for LLMs, including vision models

157 points · 45 comments · allanrbo

bicsi · 1790407101 · focus · HN ↗
Of course it works, Jev is nothing but an API breakthrough
1. dist-epoch · 1790408925 · focus · HN ↗
  Jev is rumored to be a 30B model, and it's input price is MUCH cheaper than similarly sized models. The maker is also heavily focused on having a profitable product, so it's unlikely to be subsidizing the cost, especially since they say they have more demand than what they can serve.
  1. jampekka · 1790413514 · focus · HN ↗
    You can get Gemma 4 26B A4B at the exact same token input price of $0.042/M. GPT-5 nano is not much more expensive at $0.05.
    <a href="https://openrouter.ai/google/gemma-4-26b-a4b-it" rel="nofollow">https://openrouter.ai/google/gemma-4-26b-a4b-it
    bicsi · 1790415219 · focus · HN ↗
    Exactly, imo it’s not even that cheap if you look into perspective and consider the fact that providers could subsidize the cost of cached input tokens to virtually zero if they would allow for a more flexible API (e.g. tree of message blocks instead of chain). Most of the cost is the infrastructure around keeping KV caches, estimating their lifetimes, etc. When mist people just want to run one context block with multiple subsequent variants of a second block in parallel. I still stand by my statement.
    imtringued · 1790418837 · focus · HN ↗
    That's an interesting point, if you send a batch with a shared prefix you basically only end up paying for the sequence length difference effectively.
    There is still some minor memory bandwidth issue on outputting more tokens, but the truth is that if you process e.g. 16 messages at once you wont end up being much slower than Jev even though you have to perform several autoregressive passes.
    bicsi · 1790441643 · focus · HN ↗
    "you wont end up being much slower than Jev" -- I would even go as far as saying that "you will end up being Jev".
    _flux · 1790423221 · focus · HN ↗
    With this cost, does it perform the same quality and speed as Jev?
    I'm quite interested in this; my current understanding is though that Jev is great when scored with response quality and latency metrics.
    jampekka · 1790424682 · focus · HN ↗
    Seems to be slightly higher quality and substantially faster, although this compares remote API vs local deployment.
    The prevalent idea of Jev's superiority in price, speed and accuracy seems to come from TypeSafe's marketing and their, I'd say even bad faith, benchmarking. In independent benchmarks the relative numbers tend to be very different.
    <a href="https://github.com/Mushroom-Systems/lichen" rel="nofollow">https://github.com/Mushroom-Systems/lichen

Open on Hacker News to reply ↗

Unofficial Hacker News client; not affiliated with Y Combinator.