Thread

1. vulture916 · 1790884845 · focus · HN ↗

   Jev = $0.042/m input, output free Clef = $0.24/m input, no output price listed

   At 300 tokens per call, you'd get:

   One million decisions on Jev cost about $12.60. One million decisions on Clef cost about $72.

   Would probably make sense to self-host Clef, if you have the capability/resources. If not...

   1. jampekka · 1790886148 · focus · HN ↗

      Output for decisions have so few output items (not really tokens here) that they are negligible anyway. Jev hyping "free output" is almost lying by omission.

      1. verdverm · 1790895439 · focus · HN ↗

         framing it as "free output" is a decision /s
   2. scronkfinkle · 1790886603 · focus · HN ↗

      > no output price listed

      It's weird to think of these kinds of models as having "output tokens". Cross-encoder approaches like Laya add a [MASK] marker per option, but nothing is generated the way an autoregressive transformer generates. It's one bidirectional pass over your input, then a small head scores each option, so you wouldn't really pay for output as much as only input

   3. strangescript · 1790896546 · focus · HN ↗

      Because Clef is a larger model, larger context, can process images. Plus its open weights, so you can run it yourself.

      Since it benches better than Jev, Jev is probably smaller and easier to host. They could also be losing a lot of money.

      1. wongarsu · 1790932242 · focus · HN ↗

         Considering Jev is named after Jevon's Paradox (making things cheaper increases spending) I'm willing to bet that it's the former. Smaller model, possibly an inference pipeline optimized for a Jev-like workload (large shared context that is used for multiple questions in parallel)

         And while Jev has a lot of latency, that latency stays very flat with larger inputs. So it's probably related to their inference pipeline, not model size

         1. andrewingram · 1790942102 · focus · HN ↗

            It's anecdata, but I ported a Jev-based prototype i've been working on to Clef, and it ended up being significantly slower and the results were worse.
   4. Havoc · 1790926921 · focus · HN ↗

      >Would probably make sense to self-host Clef

      For privacy perhaps but on pricing you're unlikely to come out ahead versus datacentres with scale and industrial power pricing

      1. otabdeveloper4 · 1790933458 · focus · HN ↗

         That's what they originally said about AWS too.

         Turns out AWS is actually about 10 times more expensive than renting your own datacenter compute.

         1. Havoc · 1790948419 · focus · HN ↗

            You’re not really paying for just the compute in AWS though. You’re paying for compute that comes with 200 integrated ready to go other products and stuff like compliance and certifications.

            It’s like comparing pricing of a ribeye at the butcher and on restaurant menu. It’s apples and orange.

            Certainly AWS is making good money too though. And yeah if you just need a VM then AWS isn’t the way

Unofficial Hacker News client; not affiliated with Y Combinator.