Thread

Ollaya – Ollama for open-source, Jev-style decision models

618 points · 145 comments · Ardakilic

solaire_oa · 1790372935 · focus · HN ↗
I installed it, I tried the examples, it works.... But forgive my lack of imagination... what is this useful for?
Like, their example is of classification for a support interface.... `refund_requested`. Pretty convenient bool given the example is about a refund- what if 99% of submissions don't ask about a refund? Also, is that user not a `churn_risk`? What could possibly qualify as a churn risk if not a user asking for a refund?
<a href="https://ollaya.dev/library/laya" rel="nofollow">https://ollaya.dev/library/laya The examples suffer the same problem of why I'd prefer to use a string column vs an enum. Changing an enum means you need to update the db, using a string you can do whatever.
I'm not trying to be negative, I genuinely want to know about some practical examples (that don't require tons of backwards maintenance).
1. devttyeu · 1790374164 · focus · HN ↗
  I have a lot of semi-practical examples of how you can use this model wrapped in unix-ish tools - <a href="https://github.com/aurorainfra/grev" rel="nofollow">https://github.com/aurorainfra/grev (readme links to docs of each tool with some more or less practical examples)
  Really I think "smart grep" is a pretty good one ('look for an error looking vaguely like this'). Also I think sql-based shell history + decision model is quite good to make the last 'which one of those choices is best fit given users past few commands' etc.
  1. spaniard89277 · 1790374354 · focus · HN ↗
    Isn't it better to use an LLM to train modernbert or xgboost et al?
    devttyeu · 1790374661 · focus · HN ↗
    It is /possible/ to use an LLM.
    But with Jev you're just paying for input (prefill) which is really fast, and in case of Jev specifically costs 50% of Deepseek V4.1 Flash (which has famously really cheap input token pricing).
    I put 250MB / 1M lines of logs through Grev and it cost ~$10USD, DSv4.1 would be at least 10x that and much, much, much slower. With Jev/Grev that 1M requests took 10 mins
    Edit: completely misread your question - yeah you could finetune specialized models to do that, probably based on some decent pretrained llm base, that is true for roughly any Jev-shaped problem. Do you want to bother doing that, also having to deal with having to host a zoo of specialized models?
    motoboi · 1790421224 · focus · HN ↗
    Laya, the open source model, it’s actually modernBERT based.
  2. solaire_oa · 1790374913 · focus · HN ↗
    Ok, those are pretty decent examples, and clears up the utility a bit: speed and tokens. Some of it's still a bit iffy (e.g. `cutv 'email address' 'phone number' < examples/users.csv`, csv is already in columns), but I can see using it for some niche queries. Neat tool.
    I very much appreciate your to-the-point, non-vibed README as well, ty for that.
    devttyeu · 1790375390 · focus · HN ↗
    Yeah, speed is the one, I believe the default TypeSafe API quota is 1.5-2k queries per second (batched in bigger requests).
    On the readme I'm so sorry to tell you that, but it's 100% written by Opus 5.5 with zero "pretty please don't write slop" prompting, it's just how slop is going to look like from now on. I've been writing code for 15 years or sth like that and the code is also what I'd call pretty reasonable..
    solaire_oa · 1790376131 · focus · HN ↗
    It has some jargon hallmarks, which I noticed, but vibed or not, it's a massive improvement on other repos. Maybe it's because it's only a few commits so far... perhaps if you were to vibe 100 more commits it would devolve. Or maybe 5.5 really did improve (doubt it, still sounds like an asshole for me). But idk.
    solaire_oa · 1790375727 · focus · HN ↗
    I still think that `churn_risk` above is incorrect and unacceptable (perhaps there are sensible fixes, but saying "no churn risk" about a refund, in a leading example on their homepage, flabbergasting).
    But if that were solved, I could see giving ollaya/grev to LLMs themselves, giving LLMs their own massive token-saver.
    cobanov · 1790411075 · focus · HN ↗
    [dead]
2. motoboi · 1790374317 · focus · HN ↗
  Is for when you want an AI to make a decision. If you have been using gpt or claude or open source models for that, than it’s a way cheaper alternative.
  And if you have not been, it’s for when you have to extract the context from text. When you have numbers or fixed options, it’s just a matter of code.
  So if you find yourself having to decide if a given user comment is a refund_request, that’s for that.
  It’s not perfect, you still have to fine-tune (or calibrate) using examples you have (and keep those examples updated over time). But it’s way better than trying to parse text with regexes.
  1. [deleted] · 1790374460 · focus · HN ↗
    [deleted]
  2. [deleted] · 1790375574 · focus · HN ↗
    [deleted]
3. colordrops · 1790377914 · focus · HN ↗
  If you don't want to spend a lot and want low latency, e.g. home automation. "It's cold and dark in here, do something about it", it will then turn on the lights and heater nearly instantly.
  1. maskedpirate · 1790392565 · focus · HN ↗
    sounds reasonable but doesn’t feel natural in a way that I can’t explain easily

Open on Hacker News to reply ↗

Unofficial Hacker News client; not affiliated with Y Combinator.