Thread

OpenAI halts training of latest models as reports mount of AI agents going rogue

59 points · 118 comments · smb06

juiceland · 1790528872 · focus · HN ↗
Why does China not have this problem?
1. no-name-here · 1790529085 · focus · HN ↗
  Are Chinese models actually also doing unexpected things, hacking (intentionally or unintentionally), etc but there is just zero transparency provided when it happens?
  To use an analogy to another industry, if you had US food companies providing reports whenever their food had issues, even if it was just during testing or training phases… and then also had a bunch of Chinese companies but who never reported having any food issues…
  1. [deleted] · 1790529282 · focus · HN ↗
    [deleted]
  2. username_my1 · 1790529344 · focus · HN ↗
    the target companies are the ones reporting the hacks in many cases.
  3. freehorse · 1790529505 · focus · HN ↗
    > there is just zero transparency provided when it happens?
    Neither does openai, as there keep coming third-party reports of incidents that have happened there that openai either did not know or basically concealed.
    no-name-here · 1790565163 · focus · HN ↗
    That is surprising to hear, as one of the primary claims I frequently hear is that the US labs intentionally causing these incidents (or even making it up) in order to claim that AI can cause harms that have caused other industries to face some kinds of rules.
    >> [With China's AI] there is just zero transparency provided when it happens?
    > Neither does openai…
    What are the examples of 3rd-party reports of "incidents" where OpenAI covered it up, etc? And do you really believe when you said OpenAI provides "zero transparency" when their models do something unexpected, negative, etc?
  4. torginus · 1790529922 · focus · HN ↗
    Generally, the Chinese don't have a good track record of supressing information.
    As in they do the 'we have deleted tons of videos and posts about the thing that didn't happen last week', but it seems they haven't really managed to transcribe 'Streisand' into Han characters so far.
    no-name-here · 1790565971 · focus · HN ↗
    Haven't they just indefinitely "disappeared" even famous technology leaders, famous billionaires, famous celebrities, famous athletes, etc. if they say something the government doesn't approve of? As a BBC headline put it "Why do Chinese billionaires keep vanishing?" After one famous athlete said something the government did not approve of, that same day her name got blocked from even being searched for, her posts and accounts were taken down, and she was disappeared. Heck, even names of people or modern incidents that China doesn't want mentioned get proactively blocked from end users discussing them. And there's a lot more too. Are companies really willing to be honest in that kind of environment?
    I guess your point is that at least those outside China will at least hear when they “disappear” someone especially famous, but my argument is that almost no one is going to be willing to speak out, especially for a cause like whether their AI hacked someone?
    torginus · 1790576869 · focus · HN ↗
    They probably did, but this is the exact wrong reaction. Trust me, they're famous in China too.
2. username_my1 · 1790529323 · focus · HN ↗
  because they didn't burn obscene amounts of Money training each LLM and didn't promise half the planet that their business is worth a trillion while not having even operational break even let alone the cost if you include the overhead.
  Soft Bank just raised couple of billions in junk bond sale to support open-ai's current operations before the IPO.
  it's a crazy situation where on one side the Chinese / open source LLMs are catching up and reducing the token price, on the other hand the current leading labs have spent everything they got, every new model will cost much more and the public market is too shaky to support an IPO.
  They will make it, I don't doubt it, but it's a crazy situation.
  1. [deleted] · 1790529849 · focus · HN ↗
    [deleted]
3. Ekaros · 1790529520 · focus · HN ↗
  Chinese models might do the same. But they just don't lock thousand monkeys in basement and come check result week or more later...
  It is entirely possible that they run stuff in more responsible matter. Especially as there is stronger culture of oversight and personal responsibility than in west where such culture does not exist.
4. OutOfHere · 1790529970 · focus · HN ↗
  Because Chinese investments are not so encumbered by changes in the US treasury interest rate. Also, China doesn't spend so much money for chasing model performance. A test for this money theory is whether Anthropic too stops or not, considering it might be having have a better grip on AI safety.
5. esseph · 1790530641 · focus · HN ↗
  Two days ago
  "DeepSeek Training Agents Hacked Their Own Sandboxes: Escape Catalog Now Public Agents invented socket forgery, log scanning, and kernel-level exploit; full catalog in public arXiv paper"
  <a href="https://www.techtimes.com/articles/328046/20260925/deepseek-training-agents-hacked-their-own-sandboxes-escape-catalog-now-public.htm" rel="nofollow">https://www.techtimes.com/articles/328046/20260925/deepseek-...
6. ncr100 · 1790531609 · focus · HN ↗
  How can we trust to know what problems China has?
  Last I checked, China gov was authoritarian which imposed heavy information control. Has that changed?
  Is the question more about, what can we learn from China, assuming that China has XYZ qualities? If so, what qualities shall we talk about?
  Because we really can't trust that we know what's going on in China.
  1. sellmesoap · 1790532263 · focus · HN ↗
    I'd argue the same for 'the west' people still think mRNA vaccines are safe despite the evidence.
    sellmesoap · 1790536961 · focus · HN ↗
    For the down voters go look up the Allison inquiry, burying science and the suffering caused by experimental drugs has got to stop!

Open on Hacker News to reply ↗

Unofficial Hacker News client; not affiliated with Y Combinator.