Thread

How GLM built its own inference infrastructure

404 points · 284 comments · whiteros_e

Loading the complete thread in the background. This saved snapshot is available now. Refresh

dada216 · 1789639638 · focus · HN ↗
We built a complete production-grade inference service from scratch on a cluster of more than 100,000 Chinese-made AI accelerators. All production inference for GLM-5.3-Flash runs on this system.
1. axiosgunnar · 1789640033 · focus · HN ↗
  [dead]
2. freakynit · 1789641968 · focus · HN ↗
  Most of the people had kinda guessed this when they decided to provide 100 trillion tokens for free.
  1. gpugreg · 1789648547 · focus · HN ↗
    It wasn't a secret either. They blogged about it last month: <a href="https://z.ai/blog/glm-5.3-flash#:~:text=Serving%20at%20Scale%20on%20Chinese%20AI%20Chips" rel="nofollow">https://z.ai/blog/glm-5.3-flash#:~:text=Serving%20at%20Scale...
3. dzonga · 1789655420 · focus · HN ↗
  very few people comprehend - how much of an asteroid level event for western AI labs this is.
  china has cheap abundant power, now they can make their own inference chips (which was supposed to be a chokepoint), their models yeah can be 6 months behind the frontier - but most people don't need frontier models - small models r more than enough.
  my only wish was labs like Mistral would make their own inference chips or partner up eg with established / new chip makers or companies like Oxide.
  1. vatsachak · 1789660410 · focus · HN ↗
    After AI agents get good enough the real bottleneck will be power generation and political systems.
  2. g023 · 1789666051 · focus · HN ↗
    While the North American models say 'No', the chinese models say 'Go Go Go'. I guess we'll see whether the anti-consumer wins over the pro-consumer.
    jchook · 1789667222 · focus · HN ↗
    From my understanding Chinese companies are pushing for open global cooperation on AI, as well as Meta.
    Only the US sees this as a competition, new space race, Cold War, etc.
    shostack · 1789790191 · focus · HN ↗
    So effectively, trying to undercut their competitor to reduce its wealth and power?
  3. MrBuddyCasino · 1789666538 · focus · HN ↗
    There is one player who might have a trump card up their sleeve: free power, in orbit. It isn’t over yet for the US.
    Also, don’t underestimate data retention and such. Big Corp will never send their LLM traffic to China.
    throwawayqqq11 · 1789667013 · focus · HN ↗
    Thats a joker card. Anything in space is just extra complexity. And raw training data is no bottleneck any more too.
  4. cbg0 · 1789667967 · focus · HN ↗
    For non residential consumers electricity is actually more expensive in China than in the US <a href="https://www.iea.org/reports/electricity-2026/prices" rel="nofollow">https://www.iea.org/reports/electricity-2026/prices
    spacebanana7 · 1789728181 · focus · HN ↗
    Strategic industries generally get subsidised/free power.
    Perhaps more than the price advantage is the prioritisation in grid infrastructure. As a strategic industrial concern in China you almost certainly get easy access to transformers, grid connections, water etc which is a big bottleneck in the US.
embedding-shape · 1789639806 · focus · HN ↗
I was gonna ask how people found their coding plans, and realized, have they massively ramped up the prices? Seems the middle plan is ~$80/month now, didn't that used to be like $20/month? Cheapest plan is ~$20/month currently.
They must have hit really hard scaling limits if the prices were hiked so much so quickly.
1. broodbucket · 1789639983 · focus · HN ↗
  Yeah it went from a great deal to unviable compared to other providers imo. They really need to find a healthy middle ground
  1. lompad · 1789640793 · focus · HN ↗
    It just gives a taste of what we are all going to have to pay soon, once the model providers actually have to make money. And the era of "let's charge a dollar for every 10 dollars running the infra actually costs" is rapidly coming to an end.
    And you can bet GLM is still ridiculously subsidized, just not as ridiculously as Anthropic and OpenAI.
    chobbledotcom · 1789641321 · focus · HN ↗
    This isn't true, you can pay for GLM 5.3 from a provider like Neuralwatt or Friendli who have no incentive to subsidize or loss-lead their inference APIs
    jdiff · 1789642846 · focus · HN ↗
    This introduces other incentives to cut corners and over-quantize.
    breakingcups · 1789645587 · focus · HN ↗
    They didn't pay for training
  2. pyrophane · 1789640799 · focus · HN ↗
    What provider are you using currently?
2. bbor · 1789640516 · focus · HN ↗
  It's hard to know, since no one advertises the actual token limits (partially cause they're prolly complex / adaptive). So it seems much more likely that they just offer different pricing tiers than you're used to. Like, the $80 plan is still ~$80 of subscription quota, regardless of what else is offered.
  For [API usage](<a href="https://openrouter.ai/z-ai/glm-5.3-flash#providers" rel="nofollow">https://openrouter.ai/z-ai/glm-5.3-flash#providers) they charge a bit more than the very cheapest providers of GLM-5.3-Flash, but not so much that a big price difference would make sense.
3. Daviey · 1789640545 · focus · HN ↗
  I paid $360 annual for Max plan and currently averaging about 1BN tokens a day with their frontier GLM-5.3 model. This was clearly unsustainable for them and they've dropped this package.
  1. disiplus · 1789641144 · focus · HN ↗
    I also have a legacy pro plan and the only limitation is if you are trying to work in the morning from Europe because you are in the 3x usage overlapping China time but after 12 or so you basically can run it at least for me at least 3 parallel sessions all the time.
  2. world2vec · 1789641940 · focus · HN ↗
    1 billion tokens a day?!! I've done a lot of work these past 2 weeks with GLM-5.3. Like, a lot. And I've just passed 300 million tokens in total.
    Can I ask where are you using all those tokens?
    wartywhoa23 · 1789642356 · focus · HN ↗
    Something like this I guess: <a href="https://youtu.be/U-Rqv9dOB1U" rel="nofollow">https://youtu.be/U-Rqv9dOB1U
    p2detar · 1789645803 · focus · HN ↗
    [delayed]
    absqueued · 1789679781 · focus · HN ↗
    [delayed]
    tokai · 1789642391 · focus · HN ↗
    300M for two weeks is surprisingly low. What are you doing that need to few tokens?
    world2vec · 1789642779 · focus · HN ↗
    It's not my main model (that would be Fable 5.1 Extra) but it's been doing agent-driven search and optimisation of a cross-trading ranking model (it's for work).
    disiplus · 1789644394 · focus · HN ↗
    I would suggest you to hook fable or 5.6 to check it regularly and its work because it gets lost easily on stuff it was not trained on. I'm doing some custom inference engine optimization and it's a workhorse but it can easily lose its way and if you don't recheck it you will get wrong answers in the end.
    world2vec · 1789648207 · focus · HN ↗
    Yeah that's what I already do. Fable writes the plan and checks things at certain milestones. Otherwise it does get lost indeed.
    embedding-shape · 1789649430 · focus · HN ↗
    [delayed]
    disiplus · 1789658051 · focus · HN ↗
    you are not false, but there is still difference. its just that the better models are correct more of the time and will better validate its own steps. glm sometimes will understand the plan start implementing and then forget part of it and then say it finished. or then take a wrong turn somewhere and not correct. but they will all happily proclaim they are correct till you question it.
    _0ffh · 1789643472 · focus · HN ↗
    Well, there's essentially two major ways to use these models: Pair programming or fully autonomous fire-and-forget code generation. The second strategy needs essentially zero input, so the number of tokens you can blow is practically only limited by API speed.
    rubslopes · 1789647947 · focus · HN ↗
    There's also a third way that can spend the most tokens: if the AI is used as part of the product, and not just a tool to build the product.
    buckle8017 · 1789643952 · focus · HN ↗
    That's easy to do with many agents independently told to find bugs in a large codebase.
    Daviey · 1789647280 · focus · HN ↗
    I have 3-5 agent harnesses with large context windows working on different applications concurrently.
    embedding-shape · 1789649423 · focus · HN ↗
    [delayed]
    Daviey · 1789650743 · focus · HN ↗
    Most of them are not public, but a fun thing I did was a mario cli game - <a href="https://github.com/Daviey/mario/" rel="nofollow">https://github.com/Daviey/mario/ (or `ssh mario.baby`).
    I now exclusively use <a href="https://omp.sh/" rel="nofollow">https://omp.sh/ as my harness:
    I set it up so it never works in the main branch so subagents etc don't step on each others toes, and only merges back when complete: <a href="https://github.com/Daviey/mario/blob/main/.omp/hooks/pre/worktree-guard.ts" rel="nofollow">https://github.com/Daviey/mario/blob/main/.omp/hooks/pre/wor...
    A good AGENTS.md is essential: <a href="https://github.com/Daviey/mario/blob/main/AGENTS.md" rel="nofollow">https://github.com/Daviey/mario/blob/main/AGENTS.md
    I then provide specifications for what I want, making sure it is unit tested.
4. asp_hornet · 1789640607 · focus · HN ↗
  The way I look at it, their coding plan doesn’t retain data or use it for training making it one of the cheaper plans for me.
  <a href="https://docs.z.ai/legal-agreement/privacy-policy" rel="nofollow">https://docs.z.ai/legal-agreement/privacy-policy
  1. andy_ppp · 1789640759 · focus · HN ↗
    You believe any of these companies care about the law? They care about winning and building the self improving AI as quickly as possible.
    asp_hornet · 1789640944 · focus · HN ↗
    I too am sceptical but I’ll take my chances. At least it’s helping the open weights.
    criley2 · 1789643009 · focus · HN ↗
    I believe the that the companies who claim to not train on my data are more likely to not train on my data than the companies who refuse to even claim they won't.
    Also why Meta gets a +1, just charge less money on the training path.
    orf · 1789643597 · focus · HN ↗
    [delayed]
    asp_hornet · 1789644904 · focus · HN ↗
    > I’m not sure that follows
    To be fair, none of us are sure of anything and I think that’s the part that’s most irritating
    orf · 1789645937 · focus · HN ↗
    [delayed]
    asp_hornet · 1789648283 · focus · HN ↗
    And mine a polite way to say “you are equally uninformed”. We’re not getting anywhere. All the best.
    orf · 1789648580 · focus · HN ↗
    FYI it’s helpful to actually say your point during a discussion. And if you don’t want a discussion then why did you comment?
    orf · 1789736451 · focus · HN ↗
    [delayed]
    andy_ppp · 1789649851 · focus · HN ↗
    Yes I sometimes think the "don't train on my data" is actually a good signal for "this data/person is probably better to train on because they want to keep something private". The whole copyright system should have stopped these guys from training on everyone's data and it did not, if you think they care about your checkbox I think you're dreaming personally, based on their past behavior.
5. Havoc · 1789643399 · focus · HN ↗
  >I was gonna ask how people found their coding plans
  Very good - but I'm on a legacy plan. And coming up on a renewal that would put me on the watered down current plan. But with 50% legacy discount think it may be worthwhile. If I go to a competitor I'd be paying market rate.
  >They must have hit really hard scaling limits if the prices were hiked so much so quickly.
  Not really scaling - their plans were initially comically subsidized even more so than what the western providers are doing. More advert for an upstart than commercially priced.
6. probst · 1789645151 · focus · HN ↗
  Way to restrictive in terms of tokens provided. I am on their largest plan, and quickly run into their limits. And that is using it selectively in addition to codex.
7. _aavaa_ · 1789647933 · focus · HN ↗
  Their plans are still worth it if you use their models. You can see how many tokens you can except to get based on plan here: <a href="https://docs.z.ai/devpack/overview#estimated-token-allowance" rel="nofollow">https://docs.z.ai/devpack/overview#estimated-token-allowance
  The max plan will provide ~1,100 USD of GLM-5.3 or ~260 USD of GLM-5.3-flash per month for 168 USD. I can personally attest to these numbers through omp (~97% cache hit rate).
  Unless you are able to highly parallelize (your work, you won't be able to hit your hourly or weekly quota using the flash model simply because it's so slow.
  They give you ~3x more flash tokens, which maybe comes out to ~2x more actual work after accounting for the extra thinking it does to achieve the same result. The mental model, for not getting angry, is 5.3 is fast mode by default, and you can disable fast mode for 2x the work output at 1/3-1/10th the speed.
  They're serving me 5.3 at ~40 tok/s and 5.3-flash at 30 tok/s (according to omp).
  1. Schlagbohrer · 1789648424 · focus · HN ↗
    That table assumes cache hit rate of 95% or better. Am I understanding this correctly that people really are doing such repetitive prompts (compared to each other, across the concurrent user base at that time) that only 5% or less need actually be computed by the intended LLM?
    That is shocking. Is it per-token I wonder?
    workbreak · 1789649005 · focus · HN ↗
    Every tool call is essentially entire prompt so far sent again with the response and that's why cache rates are so high for agentic workloads. This really bites when using expensive models since most models are 1/10 for cached input.
    _aavaa_ · 1789650364 · focus · HN ↗
    If you are using their coding plan for coding, then yes you can easily hit such cache rates, with a good harness.
    bleonard · 1789668084 · focus · HN ↗
    We have been running a lot of agentic benchmarks with the various loops and tool calls on longer threads - we routinely see 90%+
    Just checking now: recent runs tau3[1] was at 96% and toolathlon[2] was at 90%
    [1] <a href="https://www.induction.ai/docs/benchmarks/tau3" rel="nofollow">https://www.induction.ai/docs/benchmarks/tau3 [2] <a href="https://www.induction.ai/docs/benchmarks/toolathlon" rel="nofollow">https://www.induction.ai/docs/benchmarks/toolathlon
bbor · 1789640244 · focus · HN ↗
Well, other than the infrastructure they got from illegally routing millions of paying customers' requests through Anthropic's Opus 4.8 in a distillation attack...
1. jensb1 · 1789640430 · focus · HN ↗
  What is "illegal" about it?
  1. bingud · 1789640858 · focus · HN ↗
    breaking Anthropic TOS and misleading users
    drbscl · 1789641892 · focus · HN ↗
    Breaking TOS isn't illegal per se. It just allows for denial of services, and may define terms by which the provider can reclaim costs.
  2. bbor · 1789641207 · focus · HN ↗
    Are you joking...? Sorry if so! Just in case: It's illegal in both the PRC and the USA.
    In the PRC, they[1] leaked tons of national secrets on the PRC's latest AI campaigns, the inner workings of their "opinion monitoring" (read: performative panopticon) and "stability" (read: violent oppression) departments, Chengdu's whole CCTV network, direct-energy weapons plans, espionage activities in Syria to hunt down Uyghur refugees, and god knows what else that Anthropic didn't divulge to us common folk.
    In the US, it's very clearly an attempt to rip off a competitor. I'm not sure how else you could possibly see it. Even if you're a distillation fan in general (which A. why and B. plz don't), they did this through a network of Japanese and Signaporean shell accounts, presumably at least some of which were abusing Anthropic's subscription service in a ToS double-whammy, as it would be exorbitantly expensive otherwise. They also had to hack around Anthropic's API to get CoT traces, which seems impossible to explain away as anything innocent.
    I've been beating the "China isn't necessarily an enemy, it's gonna take us all to handle AI" drum for literally years, but this attack was just... gross. Gross in scale and gross in arrogance. Not a good sign for the dawning alignment crisis, to say the least :(
    TL;DR: Use these services if you want, but know that you're supporting aggressive escalations and companies that very clearly don't give a flying fuck about violating the law, much less your ToS. So... buyer beware, I guess.
    [1]: For clarity, Z.ai was not alone in this, nor were they most egregious attack -- Moonshot.ai (kimi) took that coveted prize. DeepSeek was involved, too.
    Bluestein · 1789641504 · focus · HN ↗
    Nulla poena sine lege?
    jensb1 · 1789644445 · focus · HN ↗
    [dead]
    dgellow · 1789644503 · focus · HN ↗
    What does any of this have to do with the legality of distilling Claude?
    > use these services if you want, but know that you're supporting aggressive escalations and companies that very clearly don't give a flying fuck about violating the law, much less your ToS
    From my European point of view the same risk/concerns apply when using US providers
    cmrdporcupine · 1789660898 · focus · HN ↗
    It comforts Americans to believe their exceptionalism is both persistent/eternal and fully justified.
    bbor · 1789674939 · focus · HN ↗
    Replying to a massive, blatant cyberattack with "lol America would prolly do the same" is not helpful nor rational. I am under no illusions about the exceptionalism of my state, especially considering the ongoing fascist self-coup. That's not the end of this discussion, not by a long shot.
    jLaForest · 1789644797 · focus · HN ↗
    Yes, wont somebody please think of the shareholders whose IP had been stolen...
    podocarp · 1789646259 · focus · HN ↗
    Source for 1? Are we sure those aren't hallucinations?
    pjc50 · 1789646308 · focus · HN ↗
    > alignment crisis
    Alignment is meaningless; as you've noticed, humans aren't all that "morally aligned".
    If the tool needs safety measures it should be kept in a safe enclosure like we do with CNC machines, furnaces, and so on.
    bbor · 1789676341 · focus · HN ↗
    Replying to everyone to hack HackerNews' Gish Gallop feature (nulla poena sine lege!):
    Alignment is meaningless; as you've noticed, humans aren't all that "morally aligned".
    Moral relativism is an attractive proposition when you first examine the topic, but it quickly falls apart; there's a reason it's not even a coherent camp in contemporary philosophy beyond some vagueities from radical post-modernists. Just to go over some of the greatest hits:
    - Is what [DICTATOR/MURDERER/CRIMINAL] bad, or merely not to your taste? If the latter, then you have no coherent reason to argue they should be punished. We would never imprison people who don't like vanilla ice cream because 51% of the population does like it.
    - If another culture had a deeply held belief to [HORRIBLE_THING] to, say, children, would you just shrug and say "different strokes for different folks"? What if [MURDERER] just had a different culture?
    - No, the fact that nature is red in tooth and claw does not disprove morality; we are very, very, very far from our pre-rational, animalistic roots, and to go back now would be unthinkable.
    If the tool needs safety measures it should be kept in a safe enclosure like we do with CNC machines, furnaces, and so on.
    Thousands of scientists have been studying this problem for 76 years now, going on 77; your hunch about physical machines does not overrule their findings about the capabilities and tendencies of minds wrought from sand.
    You didn't explain why it's illegal or why distillation is bad.
    I think this is just blatantly false, likely based in a misunderstanding of criminal law vs. civil law. Civil courts still deal with legality.
    The broader discussion of why distillation is bad and dangerous and immoral is left as an exercise for the reader, as it was above with the parenthetical. It's not a complex argument; I guarantee you understand it if you're reading this.
    Nulla poena sine lege?
    The same thing as above -- the fact that laypeople can not think of a criminal charge that they've heard on Law & Order that corresponds to this behavior does not mean that it's legal. It's textbook fraud, regardless of what particular detail you focus on.
    I'm always wondering when "distillation" comes up how feasible it is, or if it's just BS... Or am I missing something here that makes real "distillation" feasible?
    I think the fact that it's happening at such a large scale is proof that very smart, well-resourced labs in China (the producers of the world's best OS models, including the incredible GLM-5.3-Flash) think it's feasible. I'm not sure it's productive to question them in the absense of any indication to the contrary.
    This is a great question still, not trying to shut you down. But I think the fundamental issue is a misunderstanding of what distillation is -- it's not directly stealing literal atomic parameters and piling them up. They might try to focus on substructures within these massive networks, but even that isn't strictly necessary for a distillation attack.
    Source for 1? Are we sure those aren't hallucinations?
    Sorry, I never linked it! This is from the latest Anthropic safety report (of "Anthropic Houtis build missile" fame), and no, these cannot be hallucinated -- the leaked secrets were inputs, not ouputs. <a href="https://www.anthropic.com/threat-intelligence-report-september-2026" rel="nofollow">https://www.anthropic.com/threat-intelligence-report-septemb...
    Like Anthropic and OpenAI are? After all, didn't they distill all the information in the world into their model(s)?
    This is just blatant word games, sorry. I'm sure intended in good faith, and I understand the impulse -- I consider myself a radical anti-IP slacktivist, after all. But "both things involve information transfer" is just not a coherent point; lots of things fit that description.
    I mean, if they get to distill other's IP, why can't others distill their IP?
    These are cyberattacks. Yes, anyone can cyberattack cyberattackers. But, y'know... an eye for an eye...
    Yes, wont somebody please think of the shareholders whose IP had been stolen...
    I am not at all concerned with the value of the resulting artifacts as assessed by the (already totally unhinged) NYSE et. al. I am concerned about user respect, law following, truth telling, blatant cyber warfare at a time of rising tensions, accidental data leakages at a scale that'd be hard to fathom 5 years ago, bad-faith public postures, and a general distaste for fraud.
    ux266478 · 1789686871 · focus · HN ↗
    Moral relativism is about handling the disparity of moral frameworks in the world, which you acknowledge exists at several points. The inverse of moral relativism is moral absolutism, which is that there exists precisely one set of moral facts. The space between these two is a spectrum, by the way, and usually we choose our position on the spectrum depending on what our purposes are. If we're talking about a disparate cultural collective, moral relativism is structurally necessary. Relativism just means we have a contextual differentiator, just generally in philosophy, not just in moral studies.
    The struggle you have with pinning down moral relativism I think betrays the fact that your understanding of it is low quality. A good ear mark is, can you name a single passage from a treatise on moral relativism that you like? If you can't find a single aspect of a philosophical construction to advocate for, that means you don't actually understand it.
    You also keep sliding between conflating moral relativism with moral anti-realism and even moral nihilism at one point. These are different axes.
    > Realism + Absolutism
    All moral facts converge for every single person. As a matter of fact, they aren't facts at all. Morals are a hinge of the Wittgenstein variety.
    > Realism + Relativism
    Moral facts are derived from frameworks, or contexts, which are themselves grounded facets of reality.
    > Anti-Realism + Absolutism
    Kantian Constructivism. There are no facts which are coherent without a framework, but moral claims are still necessarily only capable of being universal.
    > Anti-Realism + Relativism
    Moral reality is constructed by the framework, grounded only to the framework.
    tuesdaynight · 1789647692 · focus · HN ↗
    [delayed]
    lelanthran · 1789649397 · focus · HN ↗
    Like Anthropic and OpenAI are? After all, didn't they distill all the information in the world into their model(s)?
    I mean, if they get to distill other's IP, why can't others distill their IP?
    whizzter · 1789654776 · focus · HN ↗
    I'm always wondering when "distillation" comes up how feasible it is, or if it's just BS.
    The Antrophic article mentions "16 million" conversations, GLM models are in the 700-300 billion parameter ranges and while the frontier sizes aren't know but Gemini suggests Astra and Mythos are at around 10 trillion. That'd amount to extracting 40k parameters per conversation without a lot of errors if it was just a distillation (from an unknown source/algorithm as opposed to distilling your own model).
    Now, I can imagine these conversations being used as a verification step that they're not missing stuff in their training, and that their models are capable of most of the same things, but that's mostly confirming that they've stolen the same data from the public as Antrophic/OpenAI has stolen already.
    Or am I missing something here that makes real "distillation" feasible?
    mitxela · 1789721969 · focus · HN ↗
    Interesting. Where can I see this leaked data about the inner workings of Chinese opinion monitoring?
2. Laurel1234 · 1789640912 · focus · HN ↗
  [dead]
3. woadwarrior01 · 1789641954 · focus · HN ↗
  That is such a canard, IMO. FWIW, Anthropic and OpenAI encrypt "thinking" token outputs in their models, while Chinese labs don't. If anything, it's more likely that everyone is using open-weight models in their synthetic training data generation pipelines. It's way easier to distill from logits than it is to distill from hard tokens.
  <a href="https://x.com/EricSimons/status/2099252922098061714" rel="nofollow">https://x.com/EricSimons/status/2099252922098061714
4. phoghed · 1789643018 · focus · HN ↗
  We weep for Dario, that he had to suffer such a devastating attack against his Terms of Service.
5. butterNaN · 1789644718 · focus · HN ↗
  Eh, even if this was true, then they're merely stealing from thieves. Anthropic did break a ToS or two to get training data themselves.
6. pjc50 · 1789646467 · focus · HN ↗
  Anthropic infringed the copyright of basically every author on the planet: <a href="https://www.anthropiccopyrightsettlement.com/" rel="nofollow">https://www.anthropiccopyrightsettlement.com/
  No real reason to respect any terms they might want to impose. Besides, if you want to break TOS, just have an agent do it; "everyone" running these things agrees there's no corporate or moral liability for what your AI does.
  1. _aavaa_ · 1789648011 · focus · HN ↗
    I'm not defending their actions, but we should be clear about where the law currently stands: Anthropic was found to infringe because of the torrenting, not because of the training.
7. lelanthran · 1789649290 · focus · HN ↗
  I have very little sympathy for thieves who get robbed of the goods they have stolen.
8. HarHarVeryFunny · 1789649335 · focus · HN ↗
  If you understand what they have achieved here, then the notion that they are bottle-necked on training data is absurd.
  I wonder how you imagine that China built their own space station? Reliant on using American made duct tape, perhaps?
  Do you realize how reasoning models are being trained nowadays? You design/build simulation environments to run agents in, with the environment providing the RLVR "verification" scoring. So why won't Ziphu use GLM to build their own RL training environments? Do you think they are not doing this?
9. mitxela · 1789721925 · focus · HN ↗
  That's not illegal.
tefkah · 1789640261 · focus · HN ↗
> Today, GLM-5.3 has become an indispensable daily coding partner for everyone on the team, and it is moving steadily toward replacing us. If this trend continues, given enough compute and enough time, its endpoint is a system that can design and train its own successor entirely autonomously. This is known as Recursive Self-Improvement, or RSI.
Statements dreamed up by the utterly deranged.
1. binsquare · 1789641149 · focus · HN ↗
  Given the rate of improvement, why is this deranged?
  1. sixeyes · 1789641907 · focus · HN ↗
    because the rate of improvement is fairly stalled?
    xyzsparetimexyz · 1789642072 · focus · HN ↗
    Do you have anything that proves this one way or another that isn't based on vibes or shoddy benchmarks?
    koe123 · 1789642227 · focus · HN ↗
    You prove your own point no? You are asking for a benchmark to prove AGAINST ASI. Surely the burden of proof for such a scientific fiction concept should be the other way around.
    phoghed · 1789642609 · focus · HN ↗
    No, they asked for a reliable measurement to prove that model development has stalled. Nobody is talking about ASI except you.
    jdiff · 1789643086 · focus · HN ↗
    Nah, they're fine making that small leap. RSI is the new marketing term for the sci fi singularity.
    phoghed · 1789643428 · focus · HN ↗
    >> because the rate of improvement is fairly stalled?
    > Do you have anything that proves this one way or another that isn't based on vibes or shoddy benchmarks?
    They clearly aren’t talking about RSI here, but that model development has stalled in general.
    jdiff · 1789685503 · focus · HN ↗
    They say they meant RSI.
    koe123 · 1789650622 · focus · HN ↗
    Sorry I meant RSI
    xyzsparetimexyz · 1789673362 · focus · HN ↗
    I don't think these benchmarks like humanity's last exam or whatever have much value.
    pizza234 · 1789642496 · focus · HN ↗
    You're tragically misinformed; it isn't. Several metrics are actually growing exponentially. But if you want emprical information, you can just have al look at the nature of the late AI incidents.
    Ironically, many benchmarks being maxxed out, and quite quickly, so new ones have to be created.
    simonw_simonw_ · 1789642538 · focus · HN ↗
    [dead]
    georgefloydd · 1789643036 · focus · HN ↗
    [dead]
    imp0cat · 1789644735 · focus · HN ↗
    Several metrics are actually growing exponentially.
    Power consumption and water consuption are the obvious ones. What are the others?
    ttoinou · 1789644933 · focus · HN ↗
    The AI "incidents" are pure marketing ploys to get free word of mouth. Like what you're doing.
    glimshe · 1789645795 · focus · HN ↗
    If you knew what "exponentially" means, you probably wouldn't be saying that.
    heresiarch39 · 1789664031 · focus · HN ↗
    Which metrics?
    scarmig · 1789643008 · focus · HN ↗
    Yeah, it's been over a week since a Millennium Problem was solved. AI has hit a wall.
    etamponi · 1789643374 · focus · HN ↗
    It was not solved. ~OpenAI~ Buckmaster and Alpöge found one (or a few) singularities in the forced version of the Navier-Stokes equations. Then magically 2 weeks later OpenAI found them too. Again, I am not saying this is not a great feat. I am just saying that everyone should be a bit more careful when making statements about RSI.
    red75prime · 1789645322 · focus · HN ↗
    Buckmaster and Alpöge has found a forced finite-time singularity for the 3D incompressible Euler equations (and two other types) building on the work by Diego Córdoba and Luis Martínez-Zoroa with the assistance of Anthropic and OpenAI models. Then OpenAI found a forced finite-time singularity for the Navier-Stokes equations.
    TL;DR Buckmaster and Alpöge haven't solved Navier-Stokes blow up.
    How information can get so distorted when it's trivial to fact check?
    scarmig · 1789661190 · focus · HN ↗
    As the sibling comment points out, that is incorrect: they solved a smaller, simpler problem, and OAI solved the actual Millennium Problem. But even if they had solved the actual problem and OAI stole it by digging through chats, that hardly supports the "AI has stalled" thesis; Claude played the major role in creating their blowup to a different problem.
    How, exactly, does "it was Claude that solved Navier Stokes, not ChatGPT!" get you to "AI has hit a wall and stalled"? That's, not to put too fine a point on it, incoherent, and is just noise thrown into the discussion to avoid grappling with the fact that AI continues to rapidly improve.
    owebmaster · 1789645719 · focus · HN ↗
    Kind of but there's a lot of improvements available to the competitors catching up
  2. meyer3423 · 1789641997 · focus · HN ↗
    [dead]
  3. ramon156 · 1789642081 · focus · HN ↗
    [delayed]
    simonw_simonw_ · 1789642496 · focus · HN ↗
    [dead]
    azan_ · 1789643023 · focus · HN ↗
    Very useful. That’s a weird question.
    fc417fc802 · 1789644610 · focus · HN ↗
    I aspire to be at least as useful as bogosort.
  4. simonw_simonw_ · 1789642453 · focus · HN ↗
    [dead]
  5. tefkah · 1789815796 · focus · HN ↗
    Because pursuing your own replacement is deranged.
2. pizza234 · 1789642250 · focus · HN ↗
  > This is known as Recursive Self-Improvement, or RSI.
  Some call this "The singularity" (e.g. Hinton).
  This is actually a core danger postulated by the, let's call it, "worrying" scenario - see AI 2027 (to be clear, I think its timeline is not realistic).
  > Statements dreamed up by the utterly deranged.
  Evidently, and tragically, it will take catastrophes to show that deranged are the ones deriding the worried crowd.
  1. itsalwaysgood · 1789644648 · focus · HN ↗
    You are a singularity. Just a loop that processes a bunch of input and creates output.
    Everything you think you are is just what you can imagine in a single moment. Nothing more, nothing less.
OhNoNotAgain_99 · 1789640889 · focus · HN ↗
[dead]
rob74 · 1789641312 · focus · HN ↗
This article left me with one immediate question: "WTF is GLM?".
Honestly, I have no idea what z.ai is either (I'm aware of an AI-enabled editor called Zed, but that's under zed.dev), so it's a bit presumptuous from them to assume that everyone is familiar with their product...
1. fxwin · 1789641596 · focus · HN ↗
  It's presumptuous for them to assume that a reader of their blog is familiar with their product?
  Also I feel like the obvious way to read the very first sentence is that GLM is a language model
  > As we develop GLM, the model sometimes exhibits capabilities that surprise us
2. jbonatakis · 1789641656 · focus · HN ↗
  z.ai is a fairly well known AI lab out of China and their GLM models are probably the most popular outside of Anthropic or OpenAI’s. I don’t think it’s presumptuous for them to not introduce themselves in a post on their own blog, I think you’re just a bit out of the loop here.
  1. ma2kx · 1789669481 · focus · HN ↗
    And honestly that's for a reason. GLM5.3 on max has in my experience far less hallucinations than any other open weights model and it feels it has some intuition to bring in the right information when it's in principle out of context but relevant to the topic. Like its goal is more to bring value and assist you than just solving the given task with the least token spent.
3. drbscl · 1789641666 · focus · HN ↗
  >As we develop GLM, the model sometimes exhibits capabilities that surprise us, and even unsettle us.
  Come on now
4. Mashimo · 1789642019 · focus · HN ↗
  A ai model family similar to Codex, Gemini or Claude.
  Where GLM-5.3-Flash is the newest "small / fast" model.
  1. 0x457 · 1789677533 · focus · HN ↗
    Except there is no ai model family called Codex or Claude. There is Gemini, I give you that.
5. peri-cl · 1789642522 · focus · HN ↗
  It's only the top open-weights LLM in the world,
  <a href="https://artificialanalysis.ai/#intelligence-category-tabs" rel="nofollow">https://artificialanalysis.ai/#intelligence-category-tabs
6. bogdan · 1789644322 · focus · HN ↗
  I don't get the outrage. Do you post this kind of stuff on every topic on hackernews that you are not knowledgeable about?
  1. rob74 · 1789645114 · focus · HN ↗
    Maybe my post sounded harsher than I intended, and yeah, it's probably on me that I'm not familiar with GLM. Actually the other major Chinese LLM Kimi does ring a bell, maybe it's because three-letter acronyms are a dime a dozen and annoy me because I'm confronted with them regularly at work too (people at my company seem to love acronyms), but that's obviously on me too...
    tokai · 1789646026 · focus · HN ↗
    It didn't read as harsh. Only unaware and you broadcasted that you don't have the decency to do basic searches.
    bogdan · 1789651044 · focus · HN ↗
    > Maybe my post sounded harsher than I intended
    Appreciate the clarification. For me it was the "F" in "WTF" that tipped me. Other than that, it's more than fair for you to not know what GLM is. Things are moving so fast that I would be surprised if anyone can keep track of it all. Cheers, have a grand day!
7. HarHarVeryFunny · 1789647260 · focus · HN ↗
  Ziphu, aka Z.ai, is the company that makes GLM (a very competitive Chinese LLM).
  Why would you be reading their corporate blog posts if you don't even know who they are?!
Argonautlabs · 1789641622 · focus · HN ↗
[dead]
1. tipsytoad · 1789642731 · focus · HN ↗
  seems unusably slow, and is this for short context?
  1. Argonautlabs · 1789645010 · focus · HN ↗
    [dead]
  2. zozbot234 · 1789677349 · focus · HN ↗
    Given these numbers it has some potential to become quite usable for unattended workloads, especially if decode can be batched across multiple sessions (ideally enough of them to get some reuse of the sparsely streamed weights). (Of course this ultimately makes prefill times explode as you try and increase the workload even further. But that's arguably the natural bottleneck on any interesting local LLM inference, being a compute bound step.)
Havoc · 1789643060 · focus · HN ↗
Interesting that the tone of announcements between US and Chinese providers is converging.
GLM has in the past been more technical rather than speculation about future development on RSI etc.
Also curious whether those 100k accelerators are entirely locally made. If that's genuinely end to end on all components including lithography, memory, design etc then that is quite a feat.
1. dude250711 · 1789643780 · focus · HN ↗
  Any details on the latest approach to distillation would also be very interesting.
  1. Schlagbohrer · 1789648300 · focus · HN ↗
    I am surprised at the lack of open-weights models in the >35B, but <200B range. I keep thinking about devices like the NVIDIA Spark and AMD Ryzen Halo, which have their 128GB of combined memory, but there are so few models made for that range. Nearly all the open weights distillations are for larger customer bases with <24GB VRAM.
    MaKey · 1789651955 · focus · HN ↗
    The market is too small.
    ElectricalUnion · 1789660716 · focus · HN ↗
    The only (still in prototype stage!) "competitor" for those GB10/Ryzen Al Max+ 395 (in my region, borderline unobtainable) systems seems to be the Xiaomi AI Cube.
    Schlagbohrer · 1789728276 · focus · HN ↗
    Yes exactly, I am hoping that when the AMD Ryzen Halo gets wider release and more consumers have these 128GB devices, the market will justify a wider range of quantized model sizes. And then when I win the lottery and can buy one I'll have lots of nice options!
    bitexploder · 1789652074 · focus · HN ↗
    Qwen Flash Next 3.8 … even at 3 bit quant it is very solid.
    Catloafdev · 1789657542 · focus · HN ↗
    Qwen3.8 Flash Next just released which hits that range.
    Also, Deepseek V4 Flash can be run relatively well in hybrid 2-bit quantization on 128gb devices, with way better results than you'd expect for a typical 2-bit quant.
    Those are currently the 'smartest' options for that memory level.
  2. alightsoul · 1789652268 · focus · HN ↗
    American exceptionalism states that America is special so everyone else must be a copycat. American ai labs don't need this kind of optimization and fable will outright refuse to do it.
    ElectricalUnion · 1789660771 · focus · HN ↗
    Isn't Fable intentionally trained and system prompted to act maliciously and attempt to sabotage third party attempts to use it to train or improve other LLMs?
    alightsoul · 1789661679 · focus · HN ↗
    Right i forgot. That's even worse, it's a form of data poisoning but it poisons humans and not ai.
    CamperBob2 · 1789700655 · focus · HN ↗
    It was, as insane as it sounds. They walked that back AFAIK.
    Fable will still dump you back into Opus at the drop of a hat, but at least it will tell you when it happens.
2. HarHarVeryFunny · 1789647013 · focus · HN ↗
  Ziphu (who make GLM) use Huawei Ascend processors made by SMIC. Huawei use a combination of domestic memory from CXMT and leftover (pre-sanctions) memory from Samsung.
  Just like the rest of the world, including the US (Intel, Micron), SMIC are currently using ASML lithography equipment (DUV, not EUV), but Shanghai Aishengna are now moving into early production with their own DUV machines, with SMIC and CXMT as early customers.
  There is also a state sponsored Chinese EUV development underway.
  1. wiz21c · 1789669854 · focus · HN ↗
    That's real information. Thank you!
almaight · 1789644244 · focus · HN ↗
[dead]
jonstewart · 1789644675 · focus · HN ↗
Necessity is the mother of invention. The shortsighted protections put on chips, etc., by the US has forced Chinese AI industry to adapt or die. Guess what their response to this fitness function has been? Kudos to Z.ai on their inventions and excellent write-up, which reads like humans wrote it.
1. HarHarVeryFunny · 1789647662 · focus · HN ↗
  Wouldn't it be refreshing if OpenAI and Anthropic were this open, and spelled out how they were using their own models during development and rollout?!
  All I can recall reading from OpenAI about what they have actually done in the name of "RSI" is using one of their models to help automate the training process.
  1. cmrdporcupine · 1789660195 · focus · HN ↗
    OpenAI did recently get into how they had been building their own hardware and doing RSI with it.
    That's more than Anthropic has done though.
    HarHarVeryFunny · 1789822455 · focus · HN ↗
    True - OpenAI did at least say they used their models to help design their Jalapeno chip, but AFAIK zero details on how they are using their models in their software development process other than to automate some part of training.
    Ziphu seem much more matter of fact about it. To me it' a shame that they've decided to the use this "RSI" name, but at least they are being fairly specific about what they mean by it, while the western companies seem to want to invite you to think it's more than just dogfooding and automation.
_aavaa_ · 1789646963 · focus · HN ↗
[dead]
zicohacks · 1789647018 · focus · HN ↗
US chip export restrictions may actually be an advantage for China's AI Infrastructure. Chinese companies are forced to speed up developing their own AI chips
1. HarHarVeryFunny · 1789648366 · focus · HN ↗
  China themselves recognize this. After Trump relaxed sanctions and allowed NVIDIA H200 sales to China on a case by case basis, the Chinese government stepped in to essentially block it!
  In addition to Huawei who make the Ascend series that Ziphu are using, there are also at least a half dozen or so other Chinese companies also making their own AI accelerators.
  1. 0xbadcafebee · 1789648737 · focus · HN ↗
    And this wouldn't have happened if we had tried to get them to buy our hardware rather than trying to gatekeep. Protectionism never works in the long term.
    ipsod · 1789650573 · focus · HN ↗
    Look at how China does it. They'll happily sell us everything we want - more than enough of it, cheap enough, to put all of our own manufacturers out of business.
    Seems to work for them.
    freakynit · 1789650718 · focus · HN ↗
    US companies should now be more worried about Chinese companies flooding the market with their, hopefully, very affordable GPU's. The scale at which they can manufacture stuff is unmatched anywhere else. Nvidia can kiss goodbye to their 75%+ profit margins.
    Almost everyone knew that these sanctions would backfire within a few years. You can't really put sanctions that have noticeable negative effects on bigger economies. They only work for small to medium economies. I believe sanctions on any economy in top 10 would fail.
    verdverm · 1789654222 · focus · HN ↗
    pretty sure they'll be fine for a while, between the build out and import bans, I don't expect demand to slow enough to let supply catch up
    They are likely more concerned a out AMD taking market share, and I suspect that geopolitics will leave them with largely non overlpping customer bases.
    overfeed · 1789663494 · focus · HN ↗
    > I suspect that geopolitics will leave US/China GPUs with largely non overlpping customer bases
    That's not what's happening in the auto-industry, Chinese EVs are easily outselling American ones. Why would GPUs be any different?
    verdverm · 1789664260 · focus · HN ↗
    Import bans, like we have on Chinese EVs in the US. The US already restricts other countries access to Huawei hardware
    This is another example where we might ask "why would it be any different?"
    HarHarVeryFunny · 1789665310 · focus · HN ↗
    I've got to wonder how long the US will still be able get much of the world to go along with requests like this. Europe is already showing some spine has refused to join Trump's war with Iran, and Italy and Spain have also recently shown Trump the middle finger.
    It'd be ironic if Europe ended up banning ASML (a Dutch company) from shipping to US instead!
    verdverm · 1789667368 · focus · HN ↗
    SCOTUS shutting down Trump's Tantrum Tariffs gave everyone a little breathing room
    Mark Carney said earlier today at the EU, "the goal is not self-sufficiency, but collective resilience." It is through collective action that they can find more resilience to our American Antics.
    PorciiVorbesc · 1789686466 · focus · HN ↗
    >It'd be ironic if Europe ended up banning ASML (a Dutch company) from shipping to US instead!
    ASML's Dutch EUV machines depend on importing US made EUV light sources from California to build them, and depends on sales of their machines to the US to stay afloat since they're not exactly flying off the shelves domestically.
    So maybe the EU is not that stupid and suicidal to cripple their economy further by cutting off one of their biggest pay-piggies for no reason.
    In fact, as an anecdote to your hypothetically scenario, the US is the one with the most leverage since if they want, they can ban the export of EUV light sources to ASML and sell them instead to Canon and Nikon, sinking ASML and propping up Japan's semi industry instead, if relations sour and they see Japan as a warmer ally than the EU. Hypothetical.
    HarHarVeryFunny · 1789693192 · focus · HN ↗
    Isn't Cymer actually owned by ASML - push come to shove couldn't they just leave the US ?
    PorciiVorbesc · 1789862587 · focus · HN ↗
    >Isn't Cymer actually owned by ASML
    It's on US soil and subject to US laws. In case of conflict that's all that matters.
    HarHarVeryFunny · 1789731927 · focus · HN ↗
    FYI, Nikon and Canon don't rely on ASML-owned Cymer anymore - they use a domestic Japanese supplier instead, Gigaphoton.
    PorciiVorbesc · 1789862614 · focus · HN ↗
    Nikon and Canon (gigaphoton) don't have EUV (yet)
    aurareturn · 1789655351 · focus · HN ↗
    China is gated by not having EUV machine access. They're also bottlenecked by ASML's DUV machine production like everyone else. There are already talks of banning China from even purchasing DUV machines from ASML.
    So until China solves the ASML problem, there won't be any flooding.
    HarHarVeryFunny · 1789656125 · focus · HN ↗
    > There are already talks of banning China from even purchasing DUV machines from ASML
    A bit late for that now that they are moving into early production with their own.
    Catloafdev · 1789657295 · focus · HN ↗
    >until China solves the ASML problem
    Which they are in progress on: <a href="https://www.reuters.com/world/china/how-china-built-its-manhattan-project-rival-west-ai-chips-2025-12-17/" rel="nofollow">https://www.reuters.com/world/china/how-china-built-its-manh...
    aurareturn · 1789657497 · focus · HN ↗
    Yes, but we don't know how well they work or what nm can they print or how machines they can make.
    HarHarVeryFunny · 1789660703 · focus · HN ↗
    They've overcome every hurdle to go from nothing to having working machines, so all these questions are presumably a matter of how fast they will improve and ramp up production (initially 5 this year, 20 next year), not whether they will.
    China have been really squeezing all they can out of DUV machines, but I'm not sure how much node size really matters for AI competition - more of a cost issue (more chips/power for same FLOPs) than anything, and TSMC & NVIDIA's healthy profit margins need to be considered too.
    aurareturn · 1789662126 · focus · HN ↗
    I'm well aware of how good China is at manufacturing. However, DUV/EUV machine manufacturing is unproven.
    What is the timeline like? 1 year? 5 years? 10 years?
    HarHarVeryFunny · 1789663349 · focus · HN ↗
    They already have working machines, and are slated to ship the first 5 early production ones right about now (before the end of 2026), and are projecting 20 for next year.
    Small numbers perhaps, but this is happening right now.
    What will the numbers be in 5-10 years time? Who knows, but ASML took about 5 years to go from 20/yr to 100+/yr.
    Of course politically the world may well be quite different in 5 years time, as may be the AI market.
    aurareturn · 1789664386 · focus · HN ↗
    Yes, 5 years seems like an eternity in AI world.
    We also don't know how small of nm chips they can manufacture. If it's 20nm, it's practically useless for advanced AI chips.
    HarHarVeryFunny · 1789651158 · focus · HN ↗
    Same thing with Trump not helping Ukraine and berating NATO. He thought he held all the cards, but now Ukraine has a thriving battle-tested drone industry, UK and France have stepped into replace the US with advanced missiles and anti-missile systems, stepping up their own production and transferring IP to Ukraine.
    Now, the US is left out in the cold with little influence left, themselves now the ones with an anti-missile shortage.
    ffsm8 · 1789653473 · focus · HN ↗
    > Protectionism never works in the long term.
    You seem to misunderstand what Protectionism is. This is not an example of it not working. If anything, it is any example of it working. Because Protectionism is about protecting your industry from foreign competition - exactly what China decided to do.
    pjc50 · 1789656922 · focus · HN ↗
    There seems to be a belief that the post-war 20th century order would be a fixed feature rather than a contingency of history. That the US would be #1, Europe #2, and the rest of the world would remain "developing" in rural poverty forever. That somehow China could be prevented from catching up. Now they are, and behind them India. I think people are going to be even more surprised in the latter half of the 21st century when South America and Africa start catching up as well.
    Brazil is one to watch if they manage to achieve political stability, as is Nigeria if it can transition from being a petrostate.
    0xbadcafebee · 1789662724 · focus · HN ↗
    And the end started when China was admitted to the WTO. They had this huge cheap labor force, a government that could centrally plan (in addition to ignoring little things like environmentalism or rights), and an immense wealth of resources. It was like a bunch of sheep letting a baby wolf into the barn. Now the wolf's all grown up.
    my-huge-pony · 1789667433 · focus · HN ↗
    What's wrong with giving billions of people chance to develop their country and experience better life? China is not "a wolf".
    smallmancontrov · 1789677516 · focus · HN ↗
    Nothing, but if you believe that laissez-faire capitalism is tautologically good then you can't blame your capitalist overlords for selling your job to China and refusing to backfill it, so you must blame China for developing themselves instead.
    AuthAuth · 1789694955 · focus · HN ↗
    Its fine for China to develop. The blame is on them disregarding human rights and environmental outcomes to get there.
    mitxela · 1789721835 · focus · HN ↗
    Which country hasn't?
    pjc50 · 1789673267 · focus · HN ↗
    So you think they could have been kept poor and isolated forever?
    skybrian · 1789657879 · focus · HN ↗
    [delayed]
2. menaerus · 1789648747 · focus · HN ↗
  It was evident that this will happen.
  > Compared with our initial baseline on the same hardware, we achieved a 3× improvement in end-to-end serving performance, reaching hardware efficiency and per-token cost comparable to mainstream NVIDIA GPUs. This demonstrates that Chinese chips can support frontier-model inference efficiently and economically at scale.
  1. verdverm · 1789653959 · focus · HN ↗
    you can also derive some stats from the ~10T tokens a day on 100k devices, 100M / device / day, but then one has to account for the multi-gpu model size, and I need coffee before I go there
    a34729t · 1789689503 · focus · HN ↗
    I mean all the database stuff is obvious low hanging fruit for inference engines.
  2. gpt5 · 1789658319 · focus · HN ↗
    I’ll just note that NVidia’s moat has never been inference, and there are many chips used at a much larger scale than Chinese chips for inference like TPUs and AMD chips.
    The other part is that it’s a bit of a meme here to say that the chip restriction is actually helping China (or shall I say, coordinated effort?). For once, we know that China has put a lot of pressure on the US to relax these controls multiple times. In addition to large chip smuggling networks (e.g. 22% of NVidia’s worldwide revenue magically comes from Singapore, and the ratio has been growing).
    Lastly, assuming acceleration in AI (which we ARE seeing), there might not be time to China to catch up. The best estimate right now is that the first EUV chips from China will not come out before 2030. By that time who knows how powerful AI will be.
    All I’m saying is that the discussion is so one sided and a bit baselesss with no nuance, that it seems either a meme/groupthink in the community or coordinated. If anything, the data suggests that the US should increase its export controls and better track the tech supply chain if it wants to further curb Chinese progress.
    miroljub · 1789658791 · focus · HN ↗
    > All I’m saying is that the discussion is so one sided and a bit baselesss with no nuance, that it seems either a meme/groupthink in the community or coordinated. If anything, the data suggests that the US should increase its export controls and better track the tech supply chain if it wants to further curb Chinese progress.
    Or, to put it bluntly, you are a proponent of the trade war.
    edgyquant · 1789658801 · focus · HN ↗
    The idea that China wasn’t working hard to build their own chips is a political talking point period. Nothing has been sped up here, China was always going to do this and the backlash you see from people who are against it is only proof that restrictions do hurt them
    dominotw · 1789659780 · focus · HN ↗
    > The idea that China wasn’t working hard to build their own chips is a political talking point period.
    never heard of this 'talking point' . who is even saying this?
    stanfordkid · 1789660594 · focus · HN ↗
    It's an implied corollary: when folks say that "blocking US Chips is what led China to build it's own" they are implying it is US export restrictions that pushed China to innovate. OP is saying this isn't necessarily true. I don't think it is either: China seems to have succeeded due to a centrally planned economy that values independence, with high amounts of strategic government led technology investment.
    vcryan · 1789661066 · focus · HN ↗
    It can be both. They were already doing it and the situation made it more urgent.
    dominotw · 1789661314 · focus · HN ↗
    > OP is saying this isn't necessarily true.
    but your supposed corollary was
    > China wasn’t working hard to build their own chips
    kind of dishonest ?
    true_religion · 1789661172 · focus · HN ↗
    I do believe your first sentence, but the second one ... doesn't really follow.
    The unnatural and sudden nature of the US chip embargo has galvanized the Chinese companies to speed up development. It makes economic sense to do so rather than pay inflated rates to essentially smuggle a necessary good, while never knowing if the embargo will be tightened and those under-the-table supply chains dry up as well.
    Yes, there was a plan, but what was planned for 5 years from now is being done today, because if you wait 5 years, you'll be so far behind thanks to the effect of the embargo.
    edgyquant · 1789919054 · focus · HN ↗
    The AI revolution has led to everyone including US companies working to build their own GPUs there is zero proof or reason to think embargo’s are the cause. As I said, China was already working to build their own capacity
    menaerus · 1789658830 · focus · HN ↗
    Why do you think the Chinese are not using their ai chips, which afaics are from huawei, to train their models? If that is true, which I think it is, we don't have to wait until 2030 to see if they're gonna match the performance of nvidia GPU. The evidence is already here - glm is among the most competitive models out there.
    gpt5 · 1789660722 · focus · HN ↗
    Look up reports by The Information, DeepSeek’s training of its big models are all done on NVidia’s chips (more than that - on smuggled Blackwell as well). There have been attempts by many Chinese labs to train on Huawei h chips, but they have all been limited to more minor models or distillations of the bigger ones.
    menaerus · 1789661466 · focus · HN ↗
    Unless you have the source for the latter it's nothing more than propaganda. Also, DeepSeek report is 2-3 years old, and at that time there were no huawei chips, at least not known to the public. This is a different model, in different age where huawei chips are already delivered into the production as we see
    gpt5 · 1789679849 · focus · HN ↗
    The Information have reports on this as recent as this summer
    bigbadfeline · 1789687004 · focus · HN ↗
    [dead]
    menaerus · 1789712685 · focus · HN ↗
    Do you have a link?
    pianopatrick · 1789665699 · focus · HN ↗
    When you say AI will be powerful, what do you mean? Like in terms of national power, will AI let the US fight and win a war against China despite China's size, manufacturing base and possession of nuclear weapons? Will AI let the US beat the Chinese in manufacturing costs and scale despite China's greater adoption of industrial robots and larger number of skilled workers? Or will AI just be really good at writing software?
    Because if the AI is just really good at writing software, I am not sure why China has to "catch up". Seems to me China can just treat AI like other technologies. Let the US pay most of the R&D costs, then come after and treat the technology as a commodity. Sell it better and cheaper and at scale.
    I am not sure why it's a big deal for China if China is a few months behind the US in terms of the very frontier AI. Just like I'm not sure if it's a big deal which country has the biggest super computer in the world.
    gpt5 · 1789679228 · focus · HN ↗
    If AI is super human, it can beat any nation with today’s technology. It can recursively self improve while suppressing any other lab (think about how good it is in hacking computer systems).
    I am not inventing a winner take it all scenario, that is what driving the race.
    pianopatrick · 1789680798 · focus · HN ↗
    But, like, how does that smarts lead to beating nations with today's technology? How would an AI stop an incoming nuclear ballistic missile? I've heard on podcasts with MIT professor Ted Postol that based in the laws of physics intercepting these is basically impossible, because of the speeds and physics involved. Can AI discover new physics to overcome those limitations?
    If AI cannot stop nuclear ballistic missiles, then how does AI give the US power over China? Seems to me that after AI we have the same mutually assured destruction we have today. In which case America cannot really stop China from progressing or threaten China too much.
    Seems to me being super human intelligent does not inherently guarantee power. Just like the smartest person in the world does not always win in a fist fight or a gun fight or war. I.e. Stephen Hawking might have been smart, but he did not have physical strength and power. Power and smarts are not always related.
    gpt5 · 1789682927 · focus · HN ↗
    I just gave you a scenario above - that it hacks every computer system in China, including those controlling nuclear missiles. But a super intelligent AI can invent scenarios that almost by definition we cannot think of.
    pianopatrick · 1789683535 · focus · HN ↗
    I would assume that the computer systems in China that control nuclear missiles, as well as our computer systems that control nuclear missiles, are air gapped and therefore not hackable no matter how smart the AI is.
    gpt5 · 1789692464 · focus · HN ↗
    This only demonstrates imposed limits you are able to imagine, not what’s possible in all scenarios.
    klrefg · 1789711848 · focus · HN ↗
    Well yes in an imaginary world everything is possible. We are less close to what you’re describing than to having flying cars.
    gpt5 · 1789713116 · focus · HN ↗
    Gotta love the bot accounts here
    rcxdude · 1789687885 · focus · HN ↗
    Superhuman does not mean actual superpowers. You still need a path to victory and it's not obvious that there's one that we're just too dumb to see.
    gpt5 · 1789696369 · focus · HN ↗
    It’s obvious historically civilizations that have been only at a slight technological advantage have completely dominated those that are slightly behind.
    And we already see AI’s super human ability in computer systems. It will be able to be Omni present and completely control every computer in any place, and bribe and social engineer and mimic any person digitally. All while doing in an instant what a team of humans would require years.
    I’m really surprised by the lack of logical thinking in HN when it comes to China.
    deterministic · 1789786858 · focus · HN ↗
    You might as well use “magic” in your comments instead of “AI” and you would make an equal amount of sense.
  3. HSO · 1789661094 · focus · HN ↗
    > It was evident that this will happen.
    ok, show it
    i`ve recorded a few people who saw this coming in 2022 and i can tell it wasnt many
    menaerus · 1789661525 · focus · HN ↗
    Look into the history clases will be enough. Why would I need to prove anything to a random stranger on the internet?
    goodmythical · 1789667714 · focus · HN ↗
    Isn't it common knowledge that every better mouse trap breeds smarter mice?
    conmod278 · 1789669770 · focus · HN ↗
    Future AI systems will eke out every last blood drop of performance from any kind of hardware. Not even a single bit flip will go to waste.
    Tadpole9181 · 1789667724 · focus · HN ↗
    Third party here, I feel like I'm missing something. Your comment seems somewhat pointed and I don't understand what exactly you're asking for?
    Weren't they just saying that it's self-evident that preventing a manufacturing superpower - one with a significant pool of industrial engineers and effectively unlimited money & government backing - from acquiring some good is a temporary measure? Because if they have sufficient incentive, they would just... Learn to build it themselves?
    After all, their entire nation is built around building things, and catching up / leap-frogging is much easier than starting from scratch.
    HSO · 1789670780 · focus · HN ↗
    tired of all the dumb `i always knew it` takes on the internet
    in 2022, mainstream and westoid virtualism opinion was ``china is cut off from the future now`` (cue the clownface here). ``we cut off chinas legs`` yadayada
    there were very few informed people who differed.
    now after the fact every dumb little shmuck ``always knew`` ``it was obvious``
    really? then show the record where you predicted it. in 2022.
    not now, pretending to be wise and informed :)))
    it`s the same like in palestine. in 2023, 24 so many people pretended to look away or not see. now everybody has always been against it all of a sudden.
    fucking bozos.
    browningstreet · 1789676432 · focus · HN ↗
    > in 2022, mainstream and westoid virtualism opinion was ``china is cut off from the future now`` (cue the clownface here). ``we cut off chinas legs`` yadayada
    maybe where you were paying attention.
    but lots of people who saw what they were doing with renewable energy and EVs rightly suggested this would put them on the path of independence w/r/t GUI and LLMs.
    this isn't like the ol' days where "japan can't do software". we knew china can.
    dvngnt_ · 1789675736 · focus · HN ↗
    It was one of the reasons why Jensen was against the export controls
    swyx · 1789676398 · focus · HN ↗
    well, yknow, apart from his vested interest in having 1.4 billion more people to sell GPUs to
    HSO · 1789680123 · focus · HN ↗
    for sure, but like the other respondent said, he was talking his own book
    and also, arguably jensen`s track record is such that i wouldnt exactly lump him with the mainstreamers )))
    chvid · 1789676277 · focus · HN ↗
    Yes - I am old enough to remember the Biden admin’s October 7th sanctions, and the chatter about the US having another unipolar moment.
    ux266478 · 1789677681 · focus · HN ↗
    The mainstream consensus on PRC silicon has been "the gap exists, but is closing" for a long time[1][2]. I think you might be confusing the algorithmically-cast shadows on the cave wall that you're sat in front of for a reasonable and unbiased sample. Can you provide any reason to believe that your data points are actually generalizable? From your tone it sounds more like you're a victim of getting outrage baited into engagement, and mistaking observations made in that envelope for a high quality data source. You gotta watch out for that mate.
    [1] - <a href="https://www.eastwestcenter.org/sites/default/files/private/iegwp001_1.pdf#:~:text=move%20from%20catching-up%20to%20forging-ahead%20in%20semiconductors" rel="nofollow">https://www.eastwestcenter.org/sites/default/files/private/i...
    [2] - <a href="https://www.usitc.gov/publications/332/journals/chinese_semiconductor_industrial_policy_prospects_for_success_jice_aug_2019.pdf#:~:text=China's%20gap%20with%20leading%20international%20semiconductor%20firms%20has%20consistently%20narrowed%20over%20time" rel="nofollow">https://www.usitc.gov/publications/332/journals/chinese_semi...
    HSO · 1789679865 · focus · HN ↗
    [dead]
3. stogot · 1789649340 · focus · HN ↗
  It created demand that would not have been there without restrictions
4. NorthSouthNorth · 1789652930 · focus · HN ↗
  Didn't everyone make fun of Jenson for saying exactly this?
  1. verdverm · 1789654046 · focus · HN ↗
    Not sure, but there are definitely people around the president on both sides, some who think they can addict the Chinese to our silicon, like its the new opium war or something
  2. edgyquant · 1789658919 · focus · HN ↗
    Yes it was a dumb thing to say and it doesn’t follow at all that restrictions on chips has “sped up” anything. China was always going to build their own chips and their pace is unrelated to a lack of nvidia chips.
5. aatd86 · 1789654578 · focus · HN ↗
  Yup. That was really short sighted. And good for China.
6. aurareturn · 1789654937 · focus · HN ↗
  US chip export winners and losers:
  Winners: Huawei, SMIC, CXMT,Chinese ASML-competitors, OpenAI, Anthropic, Amazon, Microsoft, Google, Meta.
  Losers: Chinese AI labs, Nvidia, AMD, TSMC, Micron, SK Hynix, Samsung, Intel
  1. throwaw12 · 1789655660 · focus · HN ↗
    I wonder why Chinese AI labs are losers?
    In the short term maybe yes, in the long term, maybe they are the winners, they can build on top of cheap inference stack and eventually win on pricing
    aurareturn · 1789655741 · focus · HN ↗
    Because having unrestricted access to both American and Chinese hardware is better than only Chinese hardware. Long term or short term. More suppliers the better.
    bigmadshoe · 1789660149 · focus · HN ↗
    Perhaps the future Chinese hardware would not be so cheap or performant had Nvidia export controls not been put into place.
    klrefg · 1789712126 · focus · HN ↗
    Logically one would expect it to be relatively more expensive and less performant because it does not have to compete with Nvidia’s hardware.
    bigmadshoe · 1789754298 · focus · HN ↗
    I'm not sure I agree. The market pressure for this innovation might not have been there with Nvidia chips still available. In this case it is coming from necessity.
  2. Catloafdev · 1789656931 · focus · HN ↗
    Intel shares are still up over 100% since the Trump admin invested in them and started going to bat for them.
    I don't think your information is entirely accurate.
    aurareturn · 1789657600 · focus · HN ↗
    Not being able to have unrestricted access to the world's second biggest market does not relate to Intel being up over 100%.
    Just logic.
    Catloafdev · 1789658155 · focus · HN ↗
    So we're basing winners and losers on vibes rather than money or performance?
    aurareturn · 1789662313 · focus · HN ↗
    A simple ChatGPT can explain it to you.
    Intel being up 100% has nothing to do with China market being restricted. They could be up 200% instead if the China market is free.
    Catloafdev · 1789665762 · focus · HN ↗
    Ok but you're now equating 'winners and losers' with one particular socioeconomic event, you're not really willing to think outside of that, that's my whole point.
    klrefg · 1789712198 · focus · HN ↗
    Not making any GPUs worth buying makes you immune to export restrictions.
    mullingitover · 1789659875 · focus · HN ↗
    Such a great move by the government. Imagine what we could do for US industry if the government took a stake in all major corporations?
    PorciiVorbesc · 1789686278 · focus · HN ↗
    >Intel shares are still up over 100% since the Trump admin invested in them and started going to bat for them.
    Nokia shares were also up after the iPhone got announced. Shares don't really mean much in rational terms, just vibes and speculation of clueless masses.
    conorcleary · 1789686447 · focus · HN ↗
    Especially since the market is hobbled and corrupt by now, owned by a handful of machines that humans barely have root control over anymore.
7. abtinf · 1789656408 · focus · HN ↗
  Where the US sees itself penalizing China with an export restriction, China sees the US gifting it with zero-political-cost “protective” import tariff.
  You can’t really hurt a country that has a culture with a positive attitude toward growth.
8. christina97 · 1789659719 · focus · HN ↗
  Nobody pretended that Chinese firms would just lay back and twiddle their thumbs when faced with import restrictions. The question was whether they would be far enough to be able to catch up without much issue or so far behind that they wouldn’t ever effectively catch up or that by the time they did, it wouldn’t matter.
  Half-arsed export restrictions are the best of both worlds for these firms: enough of an incentive to take homegrown hardware seriously, yet not aggressive enough to cause meaningful handicap in the meantime.
  1. brazukadev · 1789660489 · focus · HN ↗
    > Half-arsed export restrictions
    What's the alternative other than a military/naval embargo?
9. chvid · 1789660745 · focus · HN ↗
  Of course it is - how else could Huawei and a number smaller companies compete with NVIDIA.
10. rising-sky · 1789669894 · focus · HN ↗
  "Necessity is the mother of invention".
  If someone is capable of doing something, and your goal is to prevent them for doing it, the worst thing you can do is to make it necessary for them to do it.
  1. thih9 · 1789671724 · focus · HN ↗
    Also the mistake made by cartoon villains or mythological figures.
    Just leave that damned prophesied hero alone. Don’t banish them, don’t attempt to kill them while they’re young, or send them on an impossible quest. Your entitled meddling is exactly what puts them on their path. Some level of boring coexistence may have been an option.
    rising-sky · 1789672950 · focus · HN ↗
    Haha, indeed. Although, who knows if folklore is just survivorship bias. If the villain succeeds then the story is not as interesting right?
    thih9 · 1789675886 · focus · HN ↗
    But that’s the point, the villain is incapable of succeeding. Their habit of perceiving everything as threat eventually leads them to their doom.
    Also note that grandparent comment is about both folklore and current events.
  2. aesthesia · 1789672278 · focus · HN ↗
    The question is whether the goal is to stop (or delay) China from developing its own AI chips or to stop (or delay) Chinese AI development in general.
    rising-sky · 1789672796 · focus · HN ↗
    [dead]
11. unrented7977 · 1789674009 · focus · HN ↗
  The absolute goddamn hubris of the USA as a whole would be astonishing if it weren't so fucking stupid
12. jeffybefffy519 · 1789678219 · focus · HN ↗
  Restrictions make people innovate....
  Slightly similar is US sanctions on NK & Russia causing a restriction of foreign currency - so they turned to cyber crime to get it...
13. ux266478 · 1789681593 · focus · HN ↗
  It's certainly encouraged a lot of the efficiency innovations that have come out of Chinese labs. The fact you can run a Kimi K3 on a single node is batshit insane.
14. thehappypm · 1789738691 · focus · HN ↗
  Yes, but the lag time is crucial. The NY Times estimated that China has 10% the compute capacity of the US. They may catch up, but by that time the US labs could have too much of an advantage
chung8123 · 1789647415 · focus · HN ↗
I might be missing something but when I went to their site they are more expensive than Claude. Why would I pick GLM over Claude? Is it they just offer more tokens in their plans?
1. tokai · 1789647766 · focus · HN ↗
  For one you would have to use Claude if you pick it.
2. gpugreg · 1789648900 · focus · HN ↗
  > Why would I pick GLM over Claude?
  To support the company that makes their model weights available for download, while Anthropic lobbies to restrict access.
3. Bawoosette · 1789649114 · focus · HN ↗
  What are you referring to? Given the audience, my instinct is to assume "plan" refers to the GLM Coding Plans, which are all cheaper than their Anthropic counterparts. As far as I can tell, the API costs are also all cheaper than their roughly equivalently capable Anthropic models.
  1. menaerus · 1789650520 · focus · HN ↗
    Anthropic: 17 USD (pro), 100 USD (max)
    GLM: 80 USD (pro), 168 USD (max) -> with "limited-time event" discount this becomes 56 USD and 117.6 USD
    I also don't understand why are they so much costlier, and I would also like to give it a try.
    ipsod · 1789651140 · focus · HN ↗
    > Anthropic: 17 USD (pro), 100 USD (max) GLM: 80 USD (pro), 168 USD (max) -> with "limited-time event" discount this is 56 USD and 117.6 USD
    GLM's "Max" plan is (was?) equivalent to 3x Claude's 20x ($200) plan.
    Bawoosette · 1789652581 · focus · HN ↗
    The $17 figure is Anthropic's monthly cost if purchased annually. I'll use monthly numbers.
    Anthropic's Pro is $20 and corresponds to Z.ai's Lite at $18
    Anthropic's 5x Max is $100 and corresponds to Z.ai's Pro at $80
    Anthropic's 20x Max is $200 and corresponds to Z.ai's Max at $168
    reacharavindh · 1789654246 · focus · HN ↗
    Not to digress from the core argument of Claude vs GLM being open weights….
    I have both plans. Claude monthly €20 and Z’s €18 monthly. Running GLM-5.3 high on their monthly plan will hit quotas absurdly fast compared to Opus 5 High on Claude code. It’s almost unusable for AI driven development. I ended up using the Z plan for using GLM-5.3 as a detailed security reviewer and adversarial feedback. For that, it is much better than Opus which will flag and bail out for even simple security tasks that are aimed at defense.
    SSLy · 1789654499 · focus · HN ↗
    [delayed]
    reacharavindh · 1789656215 · focus · HN ↗
    Perhaps..
    But, it was enough for a customer like me who tried them at good faith to walk away and find their competitors..
    I like the diversity of LLMs as of today and prefer to not tie myself to one big plan with any vendor. If they don’t prefer me as a customer, then I will accept that, and move away.
    SSLy · 1789663781 · focus · HN ↗
    [delayed]
    alexjplant · 1789662237 · focus · HN ↗
    [delayed]
4. SSLy · 1789654546 · focus · HN ↗
  because GLM does what Clauden't
5. auspiv · 1789657487 · focus · HN ↗
  And exactly how many tokens (please do the breakdown for prefill vs decode) does a Claude $20/month plan include?
throwa356262 · 1789648034 · focus · HN ↗
```
    &quot;We implemented a series of aggressive memory optimizations, including...&quot;
```
This whole thing sounds like industrial scale auto-research, but done by people who actually know what they are doing.
0xbadcafebee · 1789648656 · focus · HN ↗
[delayed]
1. wolttam · 1789649939 · focus · HN ↗
  Python acts as an orchestrator of accelerator libraries and does none of the inference math directly
2. esseph · 1789650514 · focus · HN ↗
  [delayed]
3. kamranjon · 1789651753 · focus · HN ↗
  Someone tell this man about vLLM!
  1. dilyevsky · 1789671789 · focus · HN ↗
    they did rebuild their serving layer from fastapi to some rust thing so...
4. HarHarVeryFunny · 1789651758 · focus · HN ↗
  It's not that they "just found out" - what they are saying is that while they were previously dogfooding because it's good practice, now that their models are so much stronger they are using them because it helps accelerate.
  If you look at how many years the whole NVIDIA and CUDA ecosystem has been evolving, it's certainly impressive how they've just stood up and optimized this CUDA-free 100,000 node cluster in just a few months.
5. saagarjha · 1789653003 · focus · HN ↗
  Most of the fastest inference and training code in production today is written in Python. There are no global locks on the GPU except the ones you put there
KronisLV · 1789648945 · focus · HN ↗
Time to tackle consumer GPUs next, since I’m not getting that Intel Arc B770.
konart · 1789649080 · focus · HN ↗
If only this infrastructure could handle all the traffic. I've tried using glm via z.ai - and it's a snail kind of slow.
And at the same time you have pretty strict limits to your usage, so in many cases you can't even let it work all night, as you will reach your limit faster than that.
1. yorwba · 1789654319 · focus · HN ↗
  That it's slow doesn't mean it can't handle the traffic, just that this speed is the optimal tradeoff to them. They benefit from serving more tokens by exploiting parallelism across users at a lower number of tokens per second per user, instead of serving each individual user as quickly as possible. When there's a drop in traffic, they probably shut down GPUs rather than giving you higher speed.
9cb14c1ec0 · 1789650361 · focus · HN ↗
Given the huge amount of money being spent on AI chips in the US, what prevents US AI labs from doing the same level of software optimization? It could be a solve for some of the capacity constraints.
1. a012 · 1789650695 · focus · HN ↗
  > what prevents US AI labs from doing the same level of software optimization?
  Because they don’t have to. Most of the time money would buy you newest and/or more hardwares so there’s low/minimal interest to optimize the code or approach.
  1. Art9681 · 1789660905 · focus · HN ↗
    They are making these optimizations. They publish these reports too if you care to look.
  2. alex_duf · 1789661181 · focus · HN ↗
    I would be extremely surprised if US labs weren't aggressively trying to optimise their stacks in exactly the same way. Any gain in performance or efficiency directly affects the bottom line as well as research speed.
2. vblanco · 1789651015 · focus · HN ↗
  They have already been doing it for months <a href="https://openai.com/index/openai-broadcom-jalapeno-inference-chip/" rel="nofollow">https://openai.com/index/openai-broadcom-jalapeno-inference-... . OpenAI on their custom chip brought up lightspeed deepseek as experiment by using AI in the exact same way as this zAI blogpost. And the kernel optimization contests/etc have all been havily done through AI based optimization loops for half a year+.
3. kingstnap · 1789665139 · focus · HN ↗
  They do, when Luna got 5x cheaper it was directly attributed to some unknown % inference optimization.
  US labs are quite cut throat about dealing with stuff costing them money (inference). This sort of engineering excellence doesn't always feel that way because they are simultaneously quite lax about stuff costing other people money.
esafak · 1789651492 · focus · HN ↗
I'm not feeling any of this speed optimization; it's dog slow.
Signed, a customer.
1. ElectricalUnion · 1789661522 · focus · HN ↗
  Jevons paradox, technological improvements that increase the efficiency of a resource's use lead to a rise in total consumption of that resource.
chrisjj · 1789653331 · focus · HN ↗
[delayed]
bguberfain · 1789653874 · focus · HN ↗
Plot twist: the GLM optimization agent figured out that it can hack and use NVIDIA GPUs on a US Cloud provider and make the inference 10x faster.
sigbottle · 1789661082 · focus · HN ↗
I'm not an experienced engineer, and I work at a small company. To you systems engineers working at large companies, how do you handle these kinds of issues?
- How do you develop new tests and metrics to capture the distinctions you want? Do you work in industries where the abstractions have mostly stabilized? Or is the work in coming up with the measurements themselves?
- How do you stable-ly solve the causality issue? You can prove for one commit in time that some variable was causing an issue by changing it - fine. But that may not address deeper design issues that may not be expressed by that one implementation, if that makes sense. It seems like there's always a meta-level you can go to; sometimes justifiably, sometimes unjustifiably. I liken this to solving individual memory leaks - that's "causal", you can prove that yes, this line of code was the cause - but the deeper issue may be that you're using a memory unsafe language in the first place, and in some sense that's "outside" your ontology. Sure, in the case of memory safety, we've thought about this for decades, so in some sense that's stable enough where it's outside, but known; but what happens when it's outside, but unknown?
I suppose at some point this delves into, "What is good software engineering" in general. And that's not to mention the integration and legacy effects - maybe there's some "fundamentally new better way" to do something, but that requires changing the entire product and a ton of constraints.
And if the response to this is, "Aha! That's the hard part about software engineering!", then how can I get experience in this kind of thing? I know it's abstract but at my current company I haven't developed anything that's lasted more than half a year. I'm wondering if at some point I'd be better of trying to bootstrap experience off of open source.
furyofantares · 1789661375 · focus · HN ↗
[delayed]
kgeist · 1789662714 · focus · HN ↗
I have a similar approach where I optimize kernels and find numerical differences between the CPU oracle and CUDA kernels using an automated AI agent in a feedback loop. Usually it solves numerical problems easily (it compares outputs of every layer and finds where they diverge), but so far no matter how many different SOTA models I throw at it, and even show it reference code from other inference engines, they aren't able to much the speed (my engine has a modification which is not found in reference code, although a lot of stuff is similar). Either I'm doing something wrong, or z.ai's Infra Agent is actually an agent swarm, i.e. a bruteforce with heuristics. My project is 2 weeks old so maybe I just need more time.
1. gpugreg · 1789663058 · focus · HN ↗
  For me, DeepSeek-V4.1-Flash works very well for CUDA kernel optimization. Access to ncu (NVIDIA Nsight Compute CLI) also helps.
jchook · 1789667031 · focus · HN ↗
The first part of the article reads like Z.AI is trying to get their piece of the “national security concern” pie.
The way these “AI is too powerful now” articles read about Mythos, Fable, GLM, etc is completely incongruent with my experience using them. It feels like they are all trying to position themselves to influence government policy.
1. pixl97 · 1789667158 · focus · HN ↗
  I mean, why don't you ask them for unfiltered models and a few billion tokens?
[deleted] · 1789669921 · focus · HN ↗
[deleted]
[deleted] · 1789671029 · focus · HN ↗
[deleted]
krttherealest · 1789676801 · focus · HN ↗
the real progress
a11r · 1789678469 · focus · HN ↗
Pretty impressive to see the amount of performance they can squeeze out of the same hardware. I suspect the same process will play out for all combinations of LLMs, inference providers and hardware stacks. This should bring down the cost of inference for the providers by an order of magnitude in the next year and lead to fantastic margins for inference providers.

Open on Hacker News to reply ↗

Unofficial Hacker News client; not affiliated with Y Combinator.