Thread

1. amluto · 1790881961 · focus · HN ↗

   I’ll go out on a limb and suggest that I don’t think a Jev-like model is particularly useful unless you can fine tune it. The Jev API has zero ability to pass in a prior [0], and, if you can neither pass in a prior nor fine tune for your system, you will get an output that may be almost meaningless.

   I’d love to see someone build a model of this sort that can actually accept priors and do something intelligent with them.

   [0] You can feed Jev a prior as text. I’ve tried it. It works poorly.

   1. AnthusAI · 1790893651 · focus · HN ↗

      You can't fine-tune Jev itself but you can train an ML model that uses outputs from Jev as inputs. Which does enable you to 'fine-tune' your overall model.

      You can also improve your Jev classifications based on your ongoing data if you're labeling it continuously, especially if you're explaining the reasoning in the feedback labels. You can identify new elements of the rubric and add them to the list of classifications that Jev produces, and then those become new features for your ML model.

      Two levers of control for using data to make a Jev-based classifier model continuously better-aligned.

      1. AnthusAI · 1790893687 · focus · HN ↗

         Example of that: <a href="https:&#x2F;&#x2F;anth.us&#x2F;blog&#x2F;fine-tuning-jev&#x2F;" rel="nofollow">https:&#x2F;&#x2F;anth.us&#x2F;blog&#x2F;fine-tuning-jev&#x2F;

Unofficial Hacker News client; not affiliated with Y Combinator.