Looking at the example Jev use cases, it almost feels like Jev's incredible cost/task can make it competitive as a generalized "poor man's ranking" algorithm that can be useful for lean startups or any fast paced development org.
I need to rank 1000 articles and pick the 5 most relevant for the user? Jev.
I need to audit and strip out content because my user is affected by regional privacy laws (without hallucinating)? Jev.
I need to surface the 3 funniest media comments that match the user's sense of humour? Jev.
I'm sure someones working on this as we speak using an open-weight LLM base (Qwen or something would be a perfect fit).
This sort of task is a perfect fit for a very small model capable of semantic parsing. You can get away with a LOT less parameters without all the autoregressive generation and long-context reasoning.
you don't say - <a href="https://huggingface.co/harshatheg/Qwen-2.5-1B-RLCD" rel="nofollow">https://huggingface.co/harshatheg/Qwen-2.5-1B-RLCD
<a href="https://x.com/harshagundal/status/2100044305536889015" rel="nofollow">https://x.com/harshagundal/status/2100044305536889015 tweet by the author
latteren · · focus · HN ↗
I need to rank 1000 articles and pick the 5 most relevant for the user? Jev.
I need to audit and strip out content because my user is affected by regional privacy laws (without hallucinating)? Jev.
I need to surface the 3 funniest media comments that match the user's sense of humour? Jev.
nullbio · · focus · HN ↗
I need to ...? -> Open-weight model.
I'm sure someones working on this as we speak using an open-weight LLM base (Qwen or something would be a perfect fit).
This sort of task is a perfect fit for a very small model capable of semantic parsing. You can get away with a LOT less parameters without all the autoregressive generation and long-context reasoning.
rana3g · · focus · HN ↗
latteren · · focus · HN ↗
smusamashah · · focus · HN ↗
DavCreator · · focus · HN ↗