Thread

Ollaya – Ollama for open-source, Jev-style decision models

618 points · 145 comments · Ardakilic

pradn · 1790371022 · focus · HN ↗
I'm not sure what this means for AI startups if their innovations can be copied by OSS so quickly (what, like 2 weeks?). There's "consumer surplus" for everyone, to borrow an economic concept. But we do ideally want some of the surplus to flow to the innovator, too. I know there were precursors, but that's fine - it's hard to have a totally novel idea in such a popular field. I don't know what the end game is for TypeSafe - they'd need to demonstrate perpetually better results, or compete in another axis: UX, support, custom solutions, etc. So much of the time, someone proving a concept, or it simply getting enough publicity, is enough for a "Cambrian explosion" of follow-ups and copies. Famously, that was true for "Attention is All You Need", and the general idea of "next-token prediction" being so powerful.
We've stumbled into general differentiable models..
1. redox99 · 1790374331 · focus · HN ↗
  Because what they did is kinda trivial. Its basically like the Dropbox comment really[0], except here you don't need petabytes of storage and infinite VC pockets.
  After chatgpt everything in AI mostly became LLMs and building wrappers around them. It's like people forgot how to do ML.
  To those of us who actually trained models back in the day, its kind of cute to see people wowed by a classifier. Yes, this is 0 shot and doesn't need training (most people wanting this would've used structured output, this is cool because it's cheaper and faster). But anyone with basic ML knowledge could've built this in a few hours.
  The question is mostly why wasn't this productized. And it's interesting indeed that it took this long to become a finished product.
  [0] <a href="https://news.ycombinator.com/item?id=9224">https://news.ycombinator.com/item?id=9224
  1. nickstinemates · 1790383209 · focus · HN ↗
    It wasn't until recently that demand for classifiers at this scale existed. Jev exists because LLM's exist. Without them it wouldn't be (as) useful
    jmalicki · 1790388917 · focus · HN ↗
    That's not at all true.
    There have been tons of applications for this. People were using earlier LLMs like BERT for classifiers long before LLMs became viable chatbots.
    janalsncm · 1790390091 · focus · HN ↗
    I think GP has a point, though. BERT models have existed for a while but OpenAI made classification via LLM convenient and accessible for regular developers.
    People didn’t know they wanted classifiers until OpenAI gave them a taste.
    calebkaiser · 1790403704 · focus · HN ↗
    I don't know if that first part is true? Classifiers were/are one of the dominant applications of classic ML and neural networks, especially in production. Even today, image classification, object recognition, language detection, segmentation models etc are still super common.
    I think the hype with Jev is just that, while structured generation is great, LLM judges tend to kind of suck for precise classification. And the more powerful the base model, the more accurate they can get, but they get increasingly expensive/impossible to finetune. It was specifically the latency/price point Jev offered vs. the general accuracy it claimed that generated all the excitement. Plus the promise of cheap calibration (tuning).
    "Jev exists because LLMs exist" is kind of a truism, as Jev apparently is literally a Transformer model.
    nickstinemates · 1790442647 · focus · HN ↗
    How many people were doing ML work pre LLMs? And how many are using LLMs now?
    calebkaiser · 1790448352 · focus · HN ↗
    I think the answers are "a lot" and "a lot more"? But what I'm saying is that Jev's virality isn't because people didn't have access to classifiers before it. Jev's general claims about capability and performance vs. cost would have been a very big deal 5 years ago too. In a vacuum, the idea that you can get a general classifier that is very accurate across any domain and on any modality with minimal latency and a very low price point is wild.
    In early 2016, Clarifai's core product was basically just an image classifer exposed via an API. And at that point, they'd raised $40 million--the same amount as TypeSafe.ai/Jev--and they were experiencing viral growth among developers + signing contracts with a bunch of flashy logos. The demand was so high that Amazon launched Rekognition and Google launched their similar APIs to compete.
    The AI hype cycle and the number of people thinking about using AI certainly puts more wind at Jev's back, but even in an alternative universe where we don't have contemporary LLMs, Jev's core claims would be remarkable and there would be a big market for it.

Open on Hacker News to reply ↗

Unofficial Hacker News client; not affiliated with Y Combinator.