This sounds good but so far all claims just sound like marketing terms. I'd love to see real proof. e.g. "RLCD" and "parallel sampling" have nothing to back it up.
also "70-500ms vs 3-329 seconds" are apples-to-oranges unless the LLM baseline is doing comparable work (e.g., long chain-of-thought). If Jev is skipping generation entirely for a narrow structured task, of course it's faster.
Nonetheless i want this to be true, so I'm looking forward to Jev
Edit: I really have to say that I like their manifesto <a href="https://typesafe.ai/manifesto" rel="nofollow">https://typesafe.ai/manifesto
> If Jev is skipping generation entirely for a narrow structured task, of course it's faster
I think this is reasonable if people are actually using LLMs to solve this type of narrow structured task, which they are. The evidence is that every LLM provider has some method of forcing the output to conform to a json schema in their documentation.
ramon156 · · focus · HN ↗
also "70-500ms vs 3-329 seconds" are apples-to-oranges unless the LLM baseline is doing comparable work (e.g., long chain-of-thought). If Jev is skipping generation entirely for a narrow structured task, of course it's faster.
Nonetheless i want this to be true, so I'm looking forward to Jev
Edit: I really have to say that I like their manifesto <a href="https://typesafe.ai/manifesto" rel="nofollow">https://typesafe.ai/manifesto
yunwal · · focus · HN ↗
I think this is reasonable if people are actually using LLMs to solve this type of narrow structured task, which they are. The evidence is that every LLM provider has some method of forcing the output to conform to a json schema in their documentation.