Thread

1. amluto · 1790881961 · focus · HN ↗

   I’ll go out on a limb and suggest that I don’t think a Jev-like model is particularly useful unless you can fine tune it. The Jev API has zero ability to pass in a prior [0], and, if you can neither pass in a prior nor fine tune for your system, you will get an output that may be almost meaningless.

   I’d love to see someone build a model of this sort that can actually accept priors and do something intelligent with them.

   [0] You can feed Jev a prior as text. I’ve tried it. It works poorly.

   1. sheepscreek · 1790882774 · focus · HN ↗

      Also one of the more interesting features of Jev is the confidence rating that hardly any Jev-cc talks about.

      1. amluto · 1790884182 · focus · HN ↗

         It seems interesting to me only in the sense of being useless. From the horse’s mouth:

         > Confidence is derived from the probabilities

         <a href="https:&#x2F;&#x2F;docs.typesafe.ai&#x2F;confidence" rel="nofollow">https:&#x2F;&#x2F;docs.typesafe.ai&#x2F;confidence

         (Why is it much easier to find AI-slop websites quoting this than it is to find the actual documentation?)

         My inner Bayesian would like for Jev to provide something resembling “evidence”, although I admit that one might ask Jev questions that are somewhat awkward to treat as typical Bayesian questions. If I ask “will this PR be merged”, it’s kind of strange to contemplate the probability of a PR conditioned in that PR being merged in the future. But I bet there is a way to formalize a prior-free classifier in a way that makes Bayesians and non-Bayesians happy, possibly involving actual learned probabilities and confidence levels. If you read the literature on scoring rules, you will find that classifier scores do somewhat naturally decompose into a few interpretable terms.

Unofficial Hacker News client; not affiliated with Y Combinator.