Thread

Decision models like Jev don't beat LLM-as-a-judge or traditional classifiers

96 points · 34 comments · tomncooper

bjord · 1791194461 · focus · HN ↗
am I missing something or is there an incredible amount of title editorialization here? on the page itself (and within the slug), the title is:
"Benchmarking AI decision models against traditional guardrails"
1. kwinkunks · 1791200090 · focus · HN ↗
  Agree. In fact Jev does reasonably well in both tests. In the article, the assertion is hedged, and followed up with a sentiment I liked:
  > decision models like Jev do not reliably outperform [other] models in speed or accuracy. However, they rightly refocus industry attention on lightweight, task-specific inference paradigm that more closely resembles predictive machine learning
  (I also noticed some issues with bolding in Table 4 that downplay Jev's performance a little.)

Unofficial Hacker News client; not affiliated with Y Combinator.