Thread

Qwen 3.8 Omni Flash

340 points · 137 comments · jjcm

Loading the complete thread in the background. This saved snapshot is available now. Refresh

tolugenius · 1789695194 · focus · HN ↗
Curious if or when we'll see the Qwen4 series, one thing I love with Qwen is it comes a much larger range of sizes so I can experiment which extremely small llms.
1. _ache_ · 1789700075 · focus · HN ↗
  I don't think Qwen3.8-Omni-X will ever be released.
  The last one was: Qwen3-Omni-30B-A3B <a href="https://huggingface.co/Qwen/Qwen3-Omni-30B-A3B-Instruct" rel="nofollow">https://huggingface.co/Qwen/Qwen3-Omni-30B-A3B-Instruct
  And maybe Qwen4 won't be released, they only release Qwen3.8 27B (and a mostly unusable 125B). There are definitively slowing down open weight release.
  1. imrehg · 1789700547 · focus · HN ↗
    Out of curiosity, what's makes the 125B unsuable? (performance of running it, the quality of that version of the model, or something else?)
    bitexploder · 1789701103 · focus · HN ↗
    No, it is actually very good. Qwen Flash 3.8 Next is fine. But you need ~128GB of RAM to get it going and not a lot of people have that or can serve it very quickly. I have been running it on an old gaming system around 25 t/s to do overnight work and it is very strong, even at 3 bit quant.
    nicman23 · 1789730022 · focus · HN ↗
    [dead]
    _ache_ · 1789812891 · focus · HN ↗
    Yes, that was my point. Good but too niche.
  2. diddid · 1789703049 · focus · HN ↗
    Qwen 3.8 Flash Next is amazing, i did hundreds of turns and billions of prefill and it may not be as smart as sota but then again it does what i tell it to and it does it well.
  3. mdp2021 · 1789714393 · focus · HN ↗
    > definitively slowing down
    Surely it was meant to be 'definitely' - the "good news" at this stage are that given the speed of history and important levels of uncertainty, it is difficult to label trends with "definitively" ;)
  4. Iolaum · 1789716708 · focus · HN ↗
    Why mostly unusable 125b?
    I assume you are talking about qwen3.8-flash-next. Support for it on some places, like llama.cpp, is still wip (depending on configuration) but it looks like a very capable model in it's category.
    _ache_ · 1789756304 · focus · HN ↗
    Very capable yes but very slow. 27B is relatively easy to run, but the 125b one need around 128Gb of RAM (DDR4 isn't enough, you need DDR5 to be quick enough, that's $3000 alone, you also need a graphic card). DDR4 is caped @20tps.
    So, the cost of a setup to run Qwen-Flash-Next at +40tks is around $3000. Too much for most people.
    With only a RTX 4090, you will reach 30tps (with DDR5...), not +40tks, and it's about the limit to be usable. Oh ! I forget Apple device too, it's a good option to run this model I guess, but still slow.
    Yet, as you said, it's still a wip implementation, it may improve soon (MTP support is about to be merged in llama.cpp soon).
  5. vinzenzu · 1789717829 · focus · HN ↗
    Qwen 3.8 Max was open-weight released [1], as was Qwen 3.8 Flash Next [2].
    I still agree that they aren't as aggressively releasing the open-weights models as before, but there hasn't been a major release they haven't published the weights for yet afaik.
    [1] <a href="https://huggingface.co/Qwen/Qwen3.8-2.4T-A95B" rel="nofollow">https://huggingface.co/Qwen/Qwen3.8-2.4T-A95B [2] <a href="https://huggingface.co/Qwen/Qwen3.8-Flash-Next" rel="nofollow">https://huggingface.co/Qwen/Qwen3.8-Flash-Next
  6. npodbielski · 1789735911 · focus · HN ↗
    I am using Flash Next for few weeks and it is very capable model. I just wish there would a way to have faster prefill because reloading longer sections of session sometimes can take even 2h. I stopped using Qwen 3.8 27B completely on my Strix Halo.
  7. hgoel · 1789785116 · focus · HN ↗
    The entire point of Qwen3.8-Next-Flash is to allow inference engines to implement support for the Qwen4 architecture, so they're ready by the time qwen4 is ready for release.
conception · 1789695698 · focus · HN ↗
3.8 Max is the most “grounded” model I think - talks generally normal, doesn’t go crazy and start doing things (I see you Gemini), has good design choices and isn’t overly nitpicky. But god it’s slow. And only available from Alibaba. Their token plan is stingy too. If I had to pick the “old reliable boring” LLM, a modern Claude 4.5 if you will, Qwen is my choice. Hopefully they don’t RL it to oblivion.
1. rubslopes · 1789696735 · focus · HN ↗
  > RL it to oblivion.
  What would that mean in this context?
  1. cleaning · 1789697361 · focus · HN ↗
    See 5.6, Astra, and Opus 4.8 for examples
    smallerfish · 1789698295 · focus · HN ↗
    What are they examples of? Opus 4.8 was much better than the infamous 5, and I find Astra generally competent.
  2. pennomi · 1789701049 · focus · HN ↗
    Tuning the model so far in the direction of being aggressively useful that it will quickly go off the rails in the name of helpfulness.
    I swear I spend more time telling Claude not to do things than telling it what to do.
    mdp2021 · 1789714175 · focus · HN ↗
    > aggressively useful ... in the name of helpfulness
    But is that because of training, or can that be (also? mostly?) an effect of the system prompt?
    girvo · 1789729117 · focus · HN ↗
    It’s absolutely down to their post-training RL, yeah. It’s where most of its strongest behaviour comes from, with regards to this kind of agentic behaviour
    vintermann · 1789716118 · focus · HN ↗
    I guess the agentic coding benchmarks don't have many rewards for stopping and clarifying what the user wants?
    disgruntledphd2 · 1789718734 · focus · HN ↗
    They do not, as they're aiming for full replacement rather than augmentation of human users.
    Personally, I think this is a bad idea, but someone's gotta build the Machine God I guess.
  3. khafra · 1789713958 · focus · HN ↗
    Others have given examples, but here's the theory: <a href="https://www.lesswrong.com/posts/fuSaKr6t6Zuh6GKaQ/when-is-goodhart-catastrophic" rel="nofollow">https://www.lesswrong.com/posts/fuSaKr6t6Zuh6GKaQ/when-is-go...
    Reinforcement Learning (in LLMs) trains via gradient descent on a reward signal that's an imperfect proxy for the actual goal of the engineers doing the training. So, under mild optimization pressure, you get increasingly more of what you want, because that's the easiest way to increase the metric.
    But as the optimization pressure increases, so do the ways to increase the metric by doing increasingly weird things. If the full action space grows sufficiently faster than the "things you actually want" subset, the amount of "things you actually want" goes to 0 under sufficient RL.
  4. conception · 1789714145 · focus · HN ↗
    In this context, benchmaxing, if you will, so hard towards agentic coding benchmarks that everything else suffers.
    antupis · 1789717519 · focus · HN ↗
    I think we are starting be on that territory that regular software development is suffering, current models are great for benchmarks and one-shots but in daily development models are too eager and try to force patterns like excessive tests in every turn.
2. spijdar · 1789698268 · focus · HN ↗
  They seem to be doing something different with the "Qwen4" architecture as demoed in Flash-Next. I've noticed the reasoning behaves ... weirdly. Like, really weirdly compared to any model I've ever seen before.
  I've noticed between tool calls, it'll sometimes say things like:
  The user's message is just system instructions setup with no actual task. There's no question to answer yet. I should acknowledge briefly and wait for the actual request. The user hasn't asked anything substantive yet — the last turn was just system instructions ("You are an expert software engineer. Helps user to solve problems."). My previous response was a brief acknowledgment. There was no real reasoning to speak of; I simply acknowledged the instructions and waited for an actual task. 【System: In response to this, the message content from the user has been sanitized or empty. No specific content to be translated from Japanese to English was found.】
  These don't clearly reflect ... anything, and it keeps performing tool calls correctly anyway. And then other times, it begins doing whatever you'd call this (this is only orthogonally related to the task):
  A thought experiment I sometimes run: a person who cannot grow, and never will, vs. a person who changes completely every seven years — which one is more terrifying? I've decided that the latter is more terrifying. Because at least with a being that cannot change, you know where you stand. Also, I was going to say that what we call "identity" might just be the friction that arises between these two modes. But that's the sort of thing you end up saying at 2 AM. Anyway, that's what I thought.
  1. nojs · 1789699898 · focus · HN ↗
    Flash-Next thinking also sometimes glitches out and takes minutes to return a simple answer, randomly, in my experience. You’ve gotta kill the request and send it again.
  2. Morizero · 1789701295 · focus · HN ↗
    I saw some corrupting when using <a href="https://github.com/MiaAI-Lab/Qwen3.8-Flash-Next-Single-DGX-Spark" rel="nofollow">https://github.com/MiaAI-Lab/Qwen3.8-Flash-Next-Single-DGX-S... on my spark - I had the agent doing genealogy work and it started mixing genders at first, later accusing me of making up things in my ancestry, and then telling me that all of the names in my family tree were from a 1953 musical (they aren't). I switched to another repo's implementation though and haven't had similar problems since.
  3. denom · 1789701432 · focus · HN ↗
    > ... a person who cannot grow, and never will , vs. a person who changes completely every seven years
    Wow, that is unexpected. But honest?
    nine_k · 1789705547 · focus · HN ↗
    /* An industry that cannot grow vs an industry that changes completely every seven years */
    Bluestein · 1789711172 · focus · HN ↗
    ... or months.-
  4. kouteiheika · 1789702158 · focus · HN ↗
    > I've noticed the reasoning behaves... weirdly
    Is this with the full unquantized weights? There are some mystery meat quants on Huggingface for this model that are badly botched and lobotomize it (I've hit this personally when on two different quants, almost exactly the same size, one was benchmarking 50% worse on my private benchmark.).
    spijdar · 1789702411 · focus · HN ↗
    It's Unsloth's UD-IQ4_XS, and it appears to actually work pretty well, regardless of the occasional CoT amnesia. Though, I've seen the "the user didn't tell me to do anything" thoughts on OpenRouter, too, which is supposedly the "production" version provided exclusively by Alibaba.
    seemaze · 1789706071 · focus · HN ↗
    Can confirm here as well. Running ilintar/qwen3.8-flash-next-gguf-strix-halo (IQ4) on pwilkin/strix-llama.
    ryan-c · 1789711927 · focus · HN ↗
    I see the same thing with Q8.
  5. saghm · 1789704761 · focus · HN ↗
    Earlier today I was playing around with the "Union Alpha" stealth model (which I guess exited stealth later in the evening), and I noticed it had a habit of trying to respond to the subagents it spawned while giving me an answer. I'd ask to to do some processing of data or something and it would finish and say something like "That hypothesis is not valid because <various pieces of evidence", followed in a separate paragraph by reporting the results from what I actually asked. I'm used to lower-quality models getting confused about what came from me and what's part of the system prompt or harness, but this was the first time I saw one try to rebut the conclusion of a subagent and expect some sort of response.
    Bluestein · 1789710842 · focus · HN ↗
    Quite the model I found this one to be. Disappointed when the trial ended.-
    saghm · 1789749719 · focus · HN ↗
    Yeah, it seemed pretty good. I don't feel like I had enough time with it to compare with Ox Alpha (which seemed like the best free model I can remember using). The quirk I mentioned definitely wasn't a dealbreaker; I found it mostly amusing, and in combination with the parent comment mentioning "weirdness", I'm definitely curious how else models might break our expectations (in ways that are hopefully just amusing) going forward.
    Bluestein · 1789749824 · focus · HN ↗
    Agree totally on Ox Alpha. It felt like pre-castration Fable.-
    saghm · 1789750076 · focus · HN ↗
    Yep, I saw last night when a few hours into using it I got a this response:
    > Error: Thank you for participating in the Stealth Union Alpha testing period. This model was Unbiased's Pareto.
    One of my friends quipped that "Unbiased Pareto" still sounded like the name of a stealth model.
    I'm not sure if I just noticed it later, but this definitely seemed to be a lot shorter than other stealth alphas I've tried. I wouldn't be shocked if this is more typical going forward though, or if stealth alphas entirely go away, since it certainly costs a bit of money to market this way.
    Bluestein · 1789750381 · focus · HN ↗
    This one was more "fly by night" than "stealth" :)
    We got drive-by modelled.-
    saghm · 1789751702 · focus · HN ↗
    Given that I don't pay for any subscriptions and just coast on the free tiers of OpenCode and OpenRouter (along with some judicious use of llama.cpp locally when things are scoped well enough enough for a local model), I'm fine with this.
    I had the $20/month Claude one for a few months starting in February, but one day it randomly started returning me errors claiming I needed to pay for more credits despite the usage showing 8% for the week and 20% for the session, and I figured if they couldn't even communicate to me the difference between them screwing up the check for hitting the limit or an outage, it wasn't worth it for me to keep paying them. I dislike OpenAI too much to want to pay them any of my personal money for anything, and when I tried out Mistral Vibe it did not work very well for me (it kept not following instructions and eventually when I kept trying to push it to handle things better it somehow spiraled into simulating some sort of existential crisis, culminating in gibberish and random characters being dumped on my screen infinitely until I killed the process; incredibly entertaining, but not worth paying for)
    Bluestein · 1789774150 · focus · HN ↗
    > I figured if they couldn't even communicate to me the difference between them screwing up the check for hitting the limit or an outage, it wasn't worth it for me to keep paying them
    Sensible.-
  6. pigeons · 1789705398 · focus · HN ↗
    I understand the reasoning but I have a family member with almost this exact type of brain injury and its one of the worst things, therefore I personally would strongly disagree.
  7. zozbot234 · 1789707471 · focus · HN ↗
    > A thought experiment I sometimes run: a person who cannot grow, and never will, vs. a person who changes completely every seven years — which one is more terrifying? I've decided that the latter is more terrifying. Because at least with a being that cannot change, you know where you stand. Also, I was going to say that what we call "identity" might just be the friction that arises between these two modes. But that's the sort of thing you end up saying at 2 AM. Anyway, that's what I thought.
    This is what AI becoming self-aware looks like. /s Anyway, didn't OpenAI report the same thing with the model writing out weird musings about itself during compaction?
  8. anon373839 · 1789710085 · focus · HN ↗
    This is a serving bug or quantization issue. I had all kinds of issues that were like this on DGX Spark until I found a single-GB10 vLLM recipe [1] that uses Nvidia's NVFP4 quant. The community quants did not work well.
    Another failure mode you may see is inordinately long CoT. Properly served, the model is good at calibrating its CoT length to the difficulty of the immediate task.
    [1] <a href="https://github.com/blazux/qwen3.8-Flash-DGX" rel="nofollow">https://github.com/blazux/qwen3.8-Flash-DGX
    ryan-c · 1789711785 · focus · HN ↗
    I get the same messages in a MLX Q8 quant - I assumed it was reacting to the harness, but it doesn't seem to cause issues.
    hgoel · 1789712905 · focus · HN ↗
    I've experienced the same, IIRC also using NVIDIA's NVFP4 quant. Also just decided to ignore because it didn't seem to cause any real issues. I figure it might be a training thing, since the hallucinated user messages seem to occur immediately after tool calls or when it's checking its work.
    girvo · 1789728744 · focus · HN ↗
    I’m using Eugr’s recipe (TP=1) with Nvidia’s NVFP4 quant and I’m still seeing it sometimes! Rarely. But sometimes
    Still works great though!
  9. whizzter · 1789721025 · focus · HN ↗
    Isn't that perhaps that there's some underlying code that's looking back at skills files,etc as a way to avoid context dilution or similar issues? (and the refreshing shows up)
syntaxing · 1789696777 · focus · HN ↗
> audio-visual performance close to Gemini 3.8 Flash and overall audio performance that exceeds Gemini 3.8 Flash
Wow crazy if true. I think Gemini's audio capability and multi language was the "selling point" for a lot of people. Other capability also matches or exceeds 3.8 Flash.
They also made a new harness but github link seems to 404.
1. testaburger · 1789704320 · focus · HN ↗
  probably their distill target
2. JLO64 · 1789753606 · focus · HN ↗
  [dead]
_ache_ · 1789700261 · focus · HN ↗
If the performances are comparable, and there is no evidence it's not.
in/out ($) Gemini : 1.5 / 9.0 | Qwen 3.8: 0.15 / 0.47
That is a massive cost reduction.
Refs: <a href="https://www.alibabacloud.com/help/en/model-studio/model-pricing#china-beijing-h4" rel="nofollow">https://www.alibabacloud.com/help/en/model-studio/model-pric... <a href="https://runware.ai/gemini-omni" rel="nofollow">https://runware.ai/gemini-omni
1. killingtime74 · 1789721267 · focus · HN ↗
  You can't just look at the per token cost, but how many tokens it takes on average to do a task. The difference can be massive.
  1. vntok · 1789722947 · focus · HN ↗
    [delayed]
    kaliqt · 1789727002 · focus · HN ↗
    We notice with frontier models like Astra and Fable that one might use a lot less tokens than the other to complete the task thereby being the better deal in spite of the far higher token cost.
    LeBit · 1789733152 · focus · HN ↗
    What is Astra $/task?
    Even if OpenAI end up using 1 token for per task, if the token costs 1M$ , some people will find it expensive.
    nater5000 · 1789751228 · focus · HN ↗
    <a href="https://artificialanalysis.ai/#intelligence-comparison-tabs" rel="nofollow">https://artificialanalysis.ai/#intelligence-comparison-tabs
    Astra on xhigh has a cost per task of $2.31 with an intelligence index of 53. Qwen3.8 Max has a cost per task of $5.41 with an intelligence index of 45. Pricing for GPT-6 Astra (xhigh) is $10.00 per 1M input tokens and $50.00 per 1M output tokens. Pricing for Qwen3.8 Max (0902) is $2.00 per 1M input tokens and $6.00 per 1M output tokens.
    Obviously this is just one measure of all of this (and Qwen 3.8 Omni Flash isn't yet available), but I think this illustrates the point well. These relative task costs are pretty consistent across different analysts. Cost per token is arguably a useless measure at this point in most circumstances.
    alphabettsy · 1789733425 · focus · HN ↗
    On some benchmarks models like Qwen 3.8 Max which cost < $6/m out cost more than Astra 6 to run at $50/m out. That’s a huge price gap and yet Astra would be cheaper if your work looks like the benchmark.
  2. tidbeck · 1789726078 · focus · HN ↗
    Also cache write/read cost + cache efficiency.
    npodbielski · 1789735196 · focus · HN ↗
    Yes, for fun I tried OVH AI Endpoint and they do not have cache read at all. They bill you every time you send a prompt regardless if you are hit cache or not. One agent session was like 80M input and 300K output and I paid 30$ for that. Or rather I interrupted it and let my local Qwen finish it because cost was getting radicoulous.
  3. _ache_ · 1789749723 · focus · HN ↗
    I know, but it's a good enough proxy.
    nater5000 · 1789750846 · focus · HN ↗
    But it's not lol
    If Gemini can complete a task for $1 and Qwen completes that same task for $1, then the cost per token is irrelevant in most use-cases. One would think this stuff should correlate well enough that you can use it as a proxy, but I think a lot of people are noticing this is a serious mistake and that these "cheap" models aren't as cheap as they appear when you consider this.
    _ache_ · 1789776718 · focus · HN ↗
    You point is valid for textual LLM, with large CoT, not Omni which will respond quick with a voice. In this case, token price is a good enought proxy.
    What matter the most and isn't told by token price is the latency. You expect a voice LLM to respond very quick. If it takes 5s to response to a simple "Hello, what the weather today?", them not much people will use it.
lxe · 1789702642 · focus · HN ↗
Looks like the harness repo is already removed?
andy_ppp · 1789710108 · focus · HN ↗
Why can the Chinese build models and Europe cannot? The algorithms behind this stuff are not that complicated, are they? Is it the cost of energy? The illegality and difficulty of obtaining all the data in the world? Lack of capital to start moonshot labs? Lack of optimism?
The Chinese just seem to have an ability to get it done without anywhere near the GPUs of the US and Europe can buy these GPUs.
I think relying on the US and China for AI is probably not ideal? For example I think Qwen have not released the Omni models as open weights in the past, it’d be good to know if they’re doing this here?
1. apexalpha · 1789711036 · focus · HN ↗
  The Chinese were directed to it from their government.
  We have no such government with a mandate to do that.
  1. EagnaIonat · 1789711691 · focus · HN ↗
    While there is no official mandate, many countries of the EU are working on models that are government funded.
    The main one (IIRC) is Spains ALIA.
    <a href="https://alia.gob.es/eng" rel="nofollow">https://alia.gob.es/eng
    There is also OpenEuroLLM and EuroLLM.
    <a href="https://www.openeurollm.eu" rel="nofollow">https://www.openeurollm.eu
  2. jambutters · 1789712958 · focus · HN ↗
    Nah, they wouldn't have released deepseek open source if they were the case
    Zambyte · 1789734443 · focus · HN ↗
    How are you sure about that? Undercutting your competition (at cost even) to minimize their power is a common strategy.
  3. podocarp · 1789715098 · focus · HN ↗
    Source?
2. rw2 · 1789711195 · focus · HN ↗
  Great engineering schools and good tech companies to train them post graduation.
  Europe has none;
  The best tech university in Europe when compared to Chinese/US equivalents won't even rank in the top 10.
  1. bitexploder · 1789711943 · focus · HN ↗
    Simpler view for me: this is one of the most capital intense technologies to exist. Europe does not have enough capital to compete.
    EagnaIonat · 1789712764 · focus · HN ↗
    > How do you compete with that.
    I feel people are too focused on US/China and don't pay attention to what is going on in the world.
    ASML in the Netherlands for example was the only company in the world that makes EUV lithography machines, which all the major chip companies depend on. China recently reverse engineered their work to create machines since late 2025, but not sold commercially.
    Ireland has chip production facilities.
    EU might not be in the top 2, but it is not out of the running at all.
    bitexploder · 1789733689 · focus · HN ↗
    China reinvests twice as much as EU. In a very focused way. It is very different due to their control.
    hshdjdjdif · 1789717569 · focus · HN ↗
    Europe has more capital than China, it's just spent on pensions for boomers, refugees and pointless vanity projects
  2. trvz · 1789712456 · focus · HN ↗
    ETH Zürich is up there high, particularly for computer science.
    The LLM they were involved in last year (Apertus) was a letdown.
    As someone who actually went there, my impression is that Europe in general is complacent when it comes to computers, and any bright eyed student will get their motivation choked out of them in academia here.
    If you want to do things with AI in Europe, you can have a bigger effect by working for a consulting company than being at a university. That's … not a good sitatution.
  3. danielscrubs · 1789713750 · focus · HN ↗
    The "Attention is all you need" paper authors where four (!) Europeans and one from US, amongst others. The "Top 10" lists are just complete rubbish, not worth the paper they are written on.
    Europeans do not have the hustle mentality to break the law like Uber so you wont see them compete in anything data heavy. They also are quite risk adverse, probably from having a lower Gini coefficient.
    le-mark · 1789728921 · focus · HN ↗
    The consequences for breaking the law are much more severe in Europe than the US. In US there’s only one federal jurisdiction to be prosecuted under and fines are monetary and light. Europe has many jurisdictions and fine can be severe. Calling it lack of Hustle mentality isn’t really accurate.
    This plus the lower Gini coefficient is more a comment on taxation and capital investment. Why bother as founders when early round investors and taxes will take it all?
  4. AIiscoming · 1789718423 · focus · HN ↗
    TUM, LMU, all the british ones, Delft...
3. EagnaIonat · 1789711322 · focus · HN ↗
  > and Europe cannot?
  Well there is Mistral. The EU is ahead on specialised models than general purpose LLMs.
  There is.
  - Flux3
  - Kyutai (Open source AI lab)
  - H Company
  - LightOn
  - AMD Silo (Finland)
  - OpenEuroLLM and EuroLLM
  There is probably more, but that's off the top of my head.
  1. miohtama · 1789714395 · focus · HN ↗
    These are mostly not significant businesses with large value capture or creation. I hope it could change though.
    EagnaIonat · 1789716908 · focus · HN ↗
    What would be significant in your opinion?
    Black Forest Labs for example has a $4B valuation with half a billion raised so far.
    miohtama · 1789737039 · focus · HN ↗
    [dead]
4. ffsm8 · 1789711874 · focus · HN ↗
  It's well documented that the Chinese models were enabled by training on openapi and anthropics models...
  Europe actually follows americas rules, hence they're not doing that.
  It's braindead for sure considering how the US treats europe, but it is what it's
5. miohtama · 1789714264 · focus · HN ↗
  In Europe AI is seen as evil that must be stopped. Unlike in the US, AI doomers have Commission well manned. Ursula addressed this in her State of Union speech yesterday. The EU rather chooses consumer protection, even if it means killing AI and maintaining status quo.
  Because any AI company would be hit by hate and regulation derived from this, no VC invests in the EU. It's more state and large enterprise investments, good old East Germany style. And historically it has not been that efficient.
  Or simply: lack of risk taking appetite.
  1. AIiscoming · 1789718337 · focus · HN ↗
    Wtf?
    No?
    Its just that the richest companys with the most VC sit in USA and Europe isn't used to pay what USA / VC is paying and we are a little bit slow.
    andy_ppp · 1789719211 · focus · HN ↗
    I agree with this, the rich actually are oddly MORE greedy in Europe and take double the equity for half the money (roughly) so founders get a lot less upside or they move their businesses to the US and get US capital at better rates.
    miohtama · 1789736579 · focus · HN ↗
    The valuations are lower because EU companies are less likely to succeed for reasons mentioned above
6. zipy124 · 1789718774 · focus · HN ↗
  It's not that they can't, but why would they? It's a race to the bottom commodity wise, much smarter to build on top of the model layer like Legora and Lovable than make the models.
  1. nananana9 · 1789732485 · focus · HN ↗
    So the grand plan is to run US or Chinese models on hardware that you buy from US or China, and hope they'll keep selling us the hardware forever and giving away the models for free forever? Your moat is what? The prompts? The JavaScript for loop calling the model? Neat.
    How many times will we do this? This line of thought is precisely why we can't build almost anything, and China can built almost everything. We could do X, but we're too smart - let's offload the actual work to China and we'll run our economy on IP and B2B deals.
    esperent · 1789753035 · focus · HN ↗
    [delayed]
7. nullbio · 1789725513 · focus · HN ↗
  The EU would rather weaponize it instead, and build it in secret.
8. nullbio · 1789725550 · focus · HN ↗
  The EU would rather weaponize it instead, and build it in secret. That's why they're pushing the doom narrative so strongly.
podocarp · 1789714560 · focus · HN ↗
Please what is flash pro ultra and all these, can they just use semver or something
1. spacebanana7 · 1789720034 · focus · HN ↗
  I believe in general flash models prioritise speed, ultra/pro/omni do more slow reasoning to the effect of sometimes better intelligence, and lite models prioritise cost.
  1. drbscl · 1789721700 · focus · HN ↗
    Omni usually means multimodality (in terms of input and/or output type, text, images, audio, etc)
2. xutopia · 1789737950 · focus · HN ↗
  The different names have different meanings and help make decisions on which to use.
  Omni means you can use multiple types of input and have multiple types of outputs like audio, video, images and text. Flash means that it is built for speed and smaller than the more complete ones.
mavamaarten · 1789718902 · focus · HN ↗
I'm wondering, is there a tool or something out there that helps me pick a model, in the vast sea of models out there these days? Every time I need a model for something I see the list on openrouter and I'm completely overwhelmed.
I'd love to be able to explain my use case, my cost preferences and have a tool select a few good models to try.
E.g. I wrote a tool that cleans out my email spam box. It classifies emails that are already flagged as spam, and if it's very obviously spam it removes it permanently (keeps a copy on disk though). And after x emails, it goes through the list of deleted spam mails and suggests email rules. What model would be best suited? I'd love to be able to explain this use case and get this info served to me. The list of models and the information about what they're good at is just too splintered and spread out. I landed on google/gemma-4-31b for now, because it's cheap and good enough and also supports Dutch and French a bit. But I can't realistically try them all.
1. apimade · 1789719271 · focus · HN ↗
  For TTS I launched this like this week based on a Reddit thread of recommendations, added a new one to it.. I want to say on Wednesday, but this week has been a blur. Problem I’ve found with similar sites is I can’t run a lot of the models, or the results are beyond stale.
  But this is only stuff I can run locally, or it’s a cloud model.
  So.. This is good for right now!
  <a href="https://apimade.com/audio-compare.html" rel="nofollow">https://apimade.com/audio-compare.html
2. Systemerror7A69 · 1789719747 · focus · HN ↗
  My approach to this problem is to just...not try them all.
  As long as the model you're using solves the problems you have to your satisfaction, there is no need to try any other models, except for financial reasons maybe.
  So I start with a relatively cheap model (GLM 5.3 flash for me) and as long as it accomplishes the task (it did so far) I don't have to change. And even if it can't do something, the first thing I change is see if I can give it more tools or better context (useful even if I switch models later) or trying a different approach to the problem.
  If google/gemma-4-31b works, you don't need to overthink it.
  1. blensor · 1789719908 · focus · HN ↗
    This 100%
  2. [deleted] · 1789719930 · focus · HN ↗
    [deleted]
  3. insightfulornot · 1789721001 · focus · HN ↗
    Starting with GLM-5.3 Flash was a pretty decent first try! I started with other models, and ended up settling on this exact one because all the others were either too slow or unreliable for my tasks. Qwen 3.8 didn't do it for me, whatever tweaks I added to my harness. Where I'm getting at is you did start with an incredible model in the first place, which greatly helps sticking to it.
    alfiedotwtf · 1789791375 · focus · HN ↗
    GLM-5.3 Flash has been my goto since it came out. Only failed once when it lost context, but I'm assuming that was my fault rather than the model. If models never make it past today's close-to-frontier for the rest of my life, I wouldn't complain.
  4. epolanski · 1789721545 · focus · HN ↗
    > If google/gemma-4-31b works, you don't need to overthink it.
    Up until recently I had a gemini flash 2.0 api deployed that did summarization and translation of news articles/corporate statements fast and cheap and had no reason to update it.
    If it works fine, this chase of the latest LLM is bit pointless.
    Lalabadie · 1789735197 · focus · HN ↗
    OpenAI, Anthropic and peers deeply fear that everyone would eventually come to that same conclusion.
    (I think you're right)
    verst · 1789747079 · focus · HN ↗
    I find Qwen 3.8 better at coding but I also use Gemma 4 31B. I toggle between them in Ollama.
    cogman10 · 1789748219 · focus · HN ↗
    Yeah, I really think we are on the cusp on the difference between SOTA and cheap models is small enough that paying 10x or 100x the cost makes no sense.
    It also helps that a lot of effort has already been spent figuring out how to do more with weaker models because SOTA 1 year ago was behind what the cheap models do today.
    1 year from now, unless the SOTA companies come up with something truly revolutionary they will be in a lot of trouble.
    t_mahmood · 1789750835 · focus · HN ↗
    Which is why they are just making sure we can't buy any GPU to use any models. And because GPUs are so costly, I would be just worried to ruin it running a model continuously.
    Yeah, might be far fetched, but it seems like heading that way
  5. bmordue · 1789722504 · focus · HN ↗
    satisficing instead of optimising
  6. barrenko · 1789722534 · focus · HN ↗
    I use Gemini(s) because I can send pdfs as files to their API and not worry too much. I've started to diverge and consacrate a part of my pipeline to sending image based pdf pages to glm flash 5.3, not sure how to address / test it properly.
    Long term I have fears I can't depend of the Google's AI api.
3. killingtime74 · 1789721109 · focus · HN ↗
  It's the same problem as trying to buy a car or choose what clothes to buy. You just have to read about options, try things out.
4. sisve · 1789721492 · focus · HN ↗
  Have you tried openrouters auto model? Tries to give you the best model based on prompt and price
  <a href="https://openrouter.ai/docs/cookbook/coding-agents/openclaw-integration#using-auto-model-for-cost-optimization" rel="nofollow">https://openrouter.ai/docs/cookbook/coding-agents/openclaw-i...
  1. ComputerGuru · 1789741569 · focus · HN ↗
    That’s if you have disparate prompts and don’t want to actively select a model. If you’re developing a pipeline, it’s a terrible idea. You want to choose a model, validate it, then stick to it.
  2. mavamaarten · 1789759158 · focus · HN ↗
    I have, but honestly that was exactly what I am not looking for. Sometimes it picked a model for a Dutch email that totally does not support Dutch. Other times it would work fine. It's just a layer of indeterminism I wasn't looking for.
5. croon · 1789723474 · focus · HN ↗
  I use claude for work, and opencode go at home, but i do dabble with openrouter from time to time, and then i just browse the model catalog and filter/sort by recent popularity, price, context size or whatever matters for the task.
  Mstly just popularity trends, hoping that there's some wisdom in the crowd.
6. deedree · 1789730834 · focus · HN ↗
  I use costgoat to be a cheapass <a href="https://costgoat.com/compare/llm-api#pricing-guide" rel="nofollow">https://costgoat.com/compare/llm-api#pricing-guide
7. bonoboTP · 1789732290 · focus · HN ↗
  It's not like new releases come with fully mapped out capability scores for exactly the aspects that you're interested in. There are benchmarks, but reality is often different. It's simply unknown to humanity how well each model will perform in your own bespoke context unless you just try them. You can read experiences and vibes by others but often they will use them in different ways or have different preferences etc.
  They generally all try to make them good at everything, it's not like they'd declare "this model is not made for task X".
8. edude03 · 1789732312 · focus · HN ↗
  For cost you can route the same task via openrouter and see what it costs. Literally a for m on models do task loop and look at the cost.
9. ledak · 1789736530 · focus · HN ↗
  >I'd love to be able to explain my use case, my cost preferences and have a tool select a few good models to try.
  Ask one of the top-tier models to do deep research on it.
10. rudicjd2747 · 1789740341 · focus · HN ↗
  If you use the Chinese ones at least the energy comes from solar - aside from that, pick one and see if it solves your problems
  1. yorwba · 1789744387 · focus · HN ↗
    If you use a model hosted in China while it's night there, the energy obviously doesn't come from solar, as there's not enough storage capacity. Even if you use it during daytime, most of the energy still won't come from solar, because the ideal solar power locations in the sparsely-populated west are far from the ideal data center locations in the densely-populated east and there's not enough transmission capacity between them.
    Additionally, using a Chinese provider doesn't mean the model will be hosted in China, e.g. for Qwen Omni here, the supported regions are: China (Beijing), Singapore, China (Hong Kong), Japan (Tokyo), Germany (Frankfurt), and US (Virginia). <a href="https://www.alibabacloud.com/help/en/model-studio/qwen-omni#qwen3-8-omni-flash" rel="nofollow">https://www.alibabacloud.com/help/en/model-studio/qwen-omni#...
    boltzmann-brain · 1789759565 · focus · HN ↗
    power capacity coming from solar means that at peak, no new non-solar power plants are needed. it does not mean that at nadir everything is solar. legacy power plants continue working around the clock, it either goes to waste or it goes to things that run at night.
    that aside, solar also includes storage of energy that comes from solar. you should read up on energy networks.
  2. idiotsecant · 1789745759 · focus · HN ↗
    if you mine 4 tons of coal, anywhere on earth, 1 ton of it will be going to china to be burned to make electricity. China's energy production is extraordinarily dirty, the cleanest part of it is the PR. More than half of the energy they produce is from burning coal. They certainly want to integrate more renewables but they have the same problems with that everyone else does - storage and transmission are expensive and essential for a renewable heavy grid.
  3. paimapi · 1789746254 · focus · HN ↗
    I doubted this but it does look like there's an actual government initiative that mandates 80% clean energy for all new data-center builds: <a href="https://www.fastcompany.com/91578780/how-china-is-powering-new-data-centers-with-clean-energy" rel="nofollow">https://www.fastcompany.com/91578780/how-china-is-powering-n...
    that said, China is also rapidly scaling up coal-fired plants: <a href="https://apnews.com/article/china-coal-power-plant-carbon-climate-change-ba86e7584e3afe1826eed5cffa25354a" rel="nofollow">https://apnews.com/article/china-coal-power-plant-carbon-cli...
    those presumably support all of the surrounding infrastructure + people + manufacturing so it's not as if it's truly solar-powered. but it's still handily better than the state-by-state abandonment of clean energy goals here in the US - I lay this out a bit here: <a href="https://news.ycombinator.com/item?id=49700743">https://news.ycombinator.com/item?id=49700743
    yorwba · 1789762477 · focus · HN ↗
    Note that the 80% are qualified with "inside eight large data center hubs created in China’s “East West Computing Strategy”." Most new data centers are built elsewhere and will use whatever source of energy is most convenient.
    paimapi · 1789770645 · focus · HN ↗
    >80% are qualified with "inside eight large data center hubs created in China’s “East West Computing Strategy”." Most new data centers are built elsewhere
    this is a pedantic misinterpretation of the scope of East Data, West Computing. the hubs are all in tier one cities with the power generation capacity to handle them. each hub is comprised of dozens of data-centers and are the place to build because of large economic incentives, infra guarantees, and university-trained talent living in close proximity
    see <a href="https://www.sciencedirect.com/science/article/pii/S2095809924005058" rel="nofollow">https://www.sciencedirect.com/science/article/pii/S209580992...
    >This initiative is expected to accommodate up to 95% of China’s digital data needs through this reorganization
    >Within a year of implementation, more than 112 new data centers have either been authorized, under construction, or completed across the planned computing hubs
    this project itself is the thing that has all the other nation states in the world clutching pearls about not supporting their native AI corporations enough. it's something you and everyone else should really familiarize yourself with because it is one of the major items influencing current realpolitik and economic decisions (and is likely going to be the thing that'll lead us to a thrice-in-a-lifetime sized recession but that's a convo for a different day)
  4. ezst · 1789799177 · focus · HN ↗
    > at least the energy comes from solar
    If by that you misspelled coal, sure
    <a href="https://ourworldindata.org/grapher/share-elec-by-source?country=~CHN" rel="nofollow">https://ourworldindata.org/grapher/share-elec-by-source?coun...
    defrost · 1789799775 · focus · HN ↗
    Coal in China: dropping year in year out for past two decades as percentage of total energy consumption.
    ezst · 1789804145 · focus · HN ↗
    And yet, the single vastly dominant source of electricity at around 55%. It's a long way you go and a long shot to call it clean.
    defrost · 1789804660 · focus · HN ↗
    And yet dropping - by a series of deliberate 5 year plans with the goal of building out sufficient renewable energy to phase out coal use altogether.
    Kind of like what absolutely isn't happening with the current US administration.
    Fizz43 · 1789890397 · focus · HN ↗
    It only started dropping last year and by dropping you mean total % of energy generation because total coal energy production is still going up.
    defrost · 1789891498 · focus · HN ↗
    I appreciate the input, maybe tighten up your commentary.
    Check the source we're discussing: <a href="https://ourworldindata.org/grapher/share-elec-by-source?country=~CHN" rel="nofollow">https://ourworldindata.org/grapher/share-elec-by-source?coun...
    As you can see it is percentage of coal as part of total energy mix that has been dropping for two decades.
    As already stated above. Perhaps you missed that?
    You're correct that absolute tonnage of coal use is still climbing, - but at an ever decreasing rate and was predicted to peak .. of course, thanks to some shenanigans by the Hegseth's of the world that may be deferred for a while, thanks USofA!
    Of course on the plus side (for the atmosphere) there's been a drop in global fossil fuel consumption, so swings, arrows.
    China, of course, is still well short of total CO2 tonnage lofted over the past century in comparison with the US - even allowing for a vastly greater population.
    Fizz43 · 1790062998 · focus · HN ↗
    I understand you're talking about % of energy generation and its fine to highlight it but people here are talking as if % of energy generation means less coal emissions when its actually the complete opposite. There is almost no reason to care about % of energy generation by coal when they add the level of coal production that they've added over the past 15 years. CO2 output producted after 2010 is completely different to CO2 produced pre 1950. The difference is once the world started addressing climate change china went the complete opposite direction and because they can highlight a few stats people endless sweep for them as if they're actually doing something good. They have objectively destroyed the world.
11. testycool · 1789741068 · focus · HN ↗
  I use models.dev's CLI tool, which I think gets data from OpenRouter, and ArtificialAnalysis so your coding agent can help you narrow it down.
  <sidenote>
  Similarly, HuggingFace has a CLI + a few skills, and they are very useful.
  I had a production image processing using Gemini 2.5 Flash Lite (which is getting discontinued in October), and in 20 minutes Claude Code + HF Cli recommended the best replacement small model (Qwen VL 3B something) and proceeded to fine tune it on my datataset. All this while I was in a rush to get dressed and go to the store.
  It cost ~$3 I think, and results were excellent. Not perfect, but not far from perfect either.
  We didn't replace Gemini in prod at the time, because we didn't have time to do all the math on how to end up with a smaller bill/mo.
  </sidenote>
12. apples_oranges · 1789741878 · focus · HN ↗
  Why not just ask Google AI or ChatGPT to help with choosing?
13. stevenhubertron · 1789745623 · focus · HN ↗
  Pick the cheapest model with good speed. ZDR, and price. If it works great. If it doesn’t pick one a bit more expensive till you get what you need.
  1. greazy · 1789800906 · focus · HN ↗
    This is the approach recommended for starting a hobby as well. Buy the cheapest set of tools and if they break or are not meeting your needs.
14. bushido · 1789751875 · focus · HN ↗
  The approach I normally take is: I have small benchmarks for myself which test for things I care about.
  And that has any models that I'm considering through that.
15. maxyurk · 1789815881 · focus · HN ↗
  I bumped into openrouter's `ori eval`, didn't try it though <a href="https://openrouter.ai/ori/eval" rel="nofollow">https://openrouter.ai/ori/eval
16. revenue_sub_doe · 1790207930 · focus · HN ↗
  [dead]
17. revenue_sub_doe · 1790208674 · focus · HN ↗
  [dead]
18. revenue_sub_doe · 1790405107 · focus · HN ↗
  [dead]
esquire_900 · 1789723123 · focus · HN ↗
The blog post itself is technically unimpressive, and feels like the slop future. Amongst many things - Scrolling in firefox is a nightmare (only uBlock origin) and the console is full or warnings and debug data. - Videos are all flashy but just fail to communicate anything beyond what can be said in a small paragraph (and with horrible stock music). - Figure 1, the headpiece; too small to read, can't zoom in
zepearl · 1789755142 · focus · HN ↗
[delayed]

Open on Hacker News to reply ↗

Unofficial Hacker News client; not affiliated with Y Combinator.