‹ BackHN Continuity

Thread

A single function Jev-like wrapper for LLMs, including vision models

157 points · 45 comments · allanrbo

  1. CROON_tv · · focus · HN ↗
    What I'd want to see next to accuracy is tail latency. In a real-time use, deciding when a spoken sentence is finished, a general LLM with the same prompt was slower and more hesitant for us than Jev, even though both cost about the same.
Open on Hacker News to reply ↗

Unofficial Hacker News client; not affiliated with Y Combinator.