I have to be honest. While this is obviously a smart and useful idea, it misses one of the core features of Jev: its confidence scores. Partial confidence could easily be mapped to fractional spaces, using unicode characters like U+2009: THIN SPACE. As it stands, this package is not harnessing the full power of Jev.
Yeah I keep getting this weird sense that Jev is kinda poorly reinventing ML. I guess the graphs don't lie and theoretically I can replace luna with it, but I don't really use luna anyway.
What is the use case for a classifier that works 90% of the time...? I feel like if I'm classifying something, I probably care enough that 90% ain't gonna cut it...
I guess the answer is just agential stuff that effectively gets double checked by the LLM in the driver seat, anyway? That tracks, though it means that jev is mostly just for the people making harnesses. Which is all of us but still!
I mean when you get your bloodwork done to check for an illness, the test you get will give the right result 90% of the time - and depending on the result, you doc might order more tests, which could be more expensive but no mrpe reliable than the first - but they are going to be statistically independent, and after 2 more, he can be 99.9% sure.
Which begs the question, can Jev retest until it gets the right result? Can it tell how corellated two of its results are? 90% correct makes for a wonderful iterator, but a poor oracle.
Yup, you articulated what I was trying to say but much more clearly (thanks!). I suppose I haven't tried just setting n=3 or something, but presumably the Jev docs would mention that if it were enough to get it from 1 figure to 2 (i.e. 90% -> 99%). I agree that there are places where 90% certainty can help, but it really needs to be an agential system; when a doctor runs more tests, they are experimentally engaging with empirical reality in a context-appropriate way.
I guess, in the end: I think it'll end up being fantastically useful for artificial engineers with their vastly superior ability to keep track of fine details and rapidly context switch, but fairly niche for any of us organic engineers that are left.
All that doesn't apply to low stakes stuff like games, though -- can't wait for the first truly open world game, NGL. A silver lining to the cobalt cloud?
sethaurus · · focus · HN ↗
danieltanfh95 · · focus · HN ↗
bbor · · focus · HN ↗
What is the use case for a classifier that works 90% of the time...? I feel like if I'm classifying something, I probably care enough that 90% ain't gonna cut it...
I guess the answer is just agential stuff that effectively gets double checked by the LLM in the driver seat, anyway? That tracks, though it means that jev is mostly just for the people making harnesses. Which is all of us but still!
torginus · · focus · HN ↗
Which begs the question, can Jev retest until it gets the right result? Can it tell how corellated two of its results are? 90% correct makes for a wonderful iterator, but a poor oracle.
bbor · · focus · HN ↗
I guess, in the end: I think it'll end up being fantastically useful for artificial engineers with their vastly superior ability to keep track of fine details and rapidly context switch, but fairly niche for any of us organic engineers that are left.
All that doesn't apply to low stakes stuff like games, though -- can't wait for the first truly open world game, NGL. A silver lining to the cobalt cloud?