Thread

OpenAI GPT–6 Astra breaks Enigma message that has resisted solution since 2005

738 points · 448 comments · sohkamyung

jtrn · 1790087529 · focus · HN ↗
The correct title: Researcher brakes one specific stubborn historic enigma message with good help from Astra.
Stubborn for a long time because the message used a completely different key from the rest of that day's traffic. Everyone assumed it shared the daily key. The original transcription had errors. The left rotor turned over at letter 72, which is rare and breaks standard crib attacks.
What is cool, if true, is that it was a 2 day collab between the Leffer and Astra. To me this shows the importance of human in the loop, was still all also showing how immensely power of llm tools. But I think it’s getting a bit silly how much anrticles ignores the driving force (the person) in breakthroughs like this.
1. exfalso · 1790087762 · focus · HN ↗
  Come on, the trivializations start to sound quite unfounded now. Yes, a human was needed, but no, it wasn't a "collaboration"
  1. jtmarl1n · 1790088044 · focus · HN ↗
    So the LLM would have done all of this on its own? Why is it ok to acknowledge the human was needed but it’s not a collaboration? Is there a defined percentage of ownership required to make the word collaboration valid?
    Macuyiko · 1790088172 · focus · HN ↗
    From the article:
    "However, the most astonishing thing about this break is that the GPT–6 Astra did it entirely on its own. Carter Leffer only directed GPT–6 Astra to see if it could break any of the unbroken Enigma messages published on the Crypto Cellar Research web page."
    I mean... I'm all for collaboration but I think this case is pretty clear, no?
    alerighi · 1790088863 · focus · HN ↗
    I mean, the LLM could do it even without all the HUMAN knowledge that was stealed during training about the Enigma machine?
    We are fooling to me, there is no intelligence in these models, they just apply methods that were invented by humans without any consciousness on what they are doing.
    allturtles · 1790089150 · focus · HN ↗
    You could say exactly the same about humans. Ex nihilo nihil fit. Every human depends on a vast corpus of prior human knowledge to be able to accomplish anything. This doesn't mean they have no intelligence.
    huty · 1790089466 · focus · HN ↗
    This is dumb.
    At the margin the innovative human matters.
    What that means for the rest of society is TBD.
    TeMPOraL · 1790090415 · focus · HN ↗
    At the margin innovation is just taking the obvious next step first.
    huty2 · 1790092721 · focus · HN ↗
    Says the person whose never produced a novel thought in his life!
    redsocksfan45 · 1790093567 · focus · HN ↗
    [dead]
    jstanley · 1790088726 · focus · HN ↗
    If you get someone to build you a house and they do it on their own, does that not count because they wouldn't have done it if you didn't pay them to do it?
    Technically you built it yourself and the builder was just a minor collaborator?
    [deleted] · 1790089273 · focus · HN ↗
    [deleted]
    cicko · 1790091284 · focus · HN ↗
    Correct. I do everything by myself.
    jfyi · 1790100537 · focus · HN ↗
    I appreciate the effort that went into the verbose version of the "I made this" meme for an hn audience.
    true_religion · 1790088979 · focus · HN ↗
    We focus on the tool because that is what makes it novel.
    Hearing a guy built his home in a week with the power of nails and a hammer would have been novel in era of mortise and tenon.
    I actually found an article about raving about how fast nail production was thanks to machining advances in 1790 and that it would bring great value: <a href="https://digital.libraries.psu.edu/digital/collection/pabooknews/id/1054/" rel="nofollow">https://digital.libraries.psu.edu/digital/collection/pabookn...
    It’s seems to me humans haven’t changed, just which machines we praise.
    radium3d · 1790091825 · focus · HN ↗
    Humans are also needed to code in C++ and compile it, then the computer does the rest.
    Now we just code in english and the computer does the rest.
    platevoltage · 1790107033 · focus · HN ↗
    English isn't code. Come on.
    radium3d · 1790184317 · focus · HN ↗
    It is much closer now. Just like how C/C++ devs used to say "JavaScript isn't real code" or "Python script kiddies". Code was literally designed to be just easy enough for people to understand, now it's just even easier to understand.
    It is not yet the same as C++ or Python, for two reasons:
    1) Ambiguity is still the default. Formal languages force you to resolve it up front.
    English lets you paper over it until the model or the compiler (the human) notices.
    2) The "compiler" (the LLM) is statistical and non-deterministic. Same prompt, different day, different bugs. A real language has a spec.
    The practical move is to treat English as a high-level specification language, keep the generated artifacts inspectable, and "still know enough of the lower layers to notice when the translation went wrong."^1
    [1] This is the key that is where humans can still be necessary, or at least another pass through the LLMs to decide on the best path, in the compiled code. Compilers for other languages do the same C -> Binary, etc.
    A conventional compiler is bound by an as-if rule. It can take many internal routes, but the observable behavior has to match the language spec. Same source, same defined semantics. If two gcc runs emit different binaries, the program is still supposed to compute the same answers on the same inputs. That is why people treat the source as the artifact and the binary as disposable.
    An LLM compiling English has no as-if rule unless you add one. "Sort the users by last active" can become a stable sort, an unstable sort, a SQL order by, an in-memory timsort, or a query that drops people with null timestamps. All of those can look like success. They are different programs. The model is not optimizing under a spec. It is filling in the parts you did not write.
    A human who can read the destination language still notices when the chosen path is the wrong program.
    A second model pass can compare paths, but only if you give it a way to score them: tests, types, invariant
    So the historical analogy still holds, with one correction. JavaScript and Python were dismissed for being too easy, but they already had grammars and evaluators. English is easier still, and the evaluator is a statistical translator that will invent a dialect if you let it. The practical move stays the same: treat English as the spec language, pin the generated artifacts behind tests, and keep enough fluency in the lower layer to see when the translation chose a different program than the one you meant. The human is not required because the computer is weak. The human is required because the source language still leaves room for more than one destination.
    platevoltage · 1790197461 · focus · HN ↗
    The difference is that those C/C++ devs were wrong. Claiming Javascript isn't code is just wrong. Even claiming HTML isn't code is wrong. How is it much closer now? Did English evolve to be more like code, or did code become more like English? And I'm not talking about how Python is more English-like than 6502 Assembly, I mean in the last few years.
    English is not code just because a technology was developed that could make educated guesses based on being trained with other code that people have written as to what the code generated should look like.
    Also, much of this reply looks LLM generated.
  2. speed_spread · 1790089899 · focus · HN ↗
    A cyclist pedaling up a mountain isn't a "collaboration" between a bicycle and a human. This is the same. You don't see feral bicycles roaming the land. All models are ultimately built and run by humans, with human-provided instructions. And as with any program, it's garbage in, garbage out.
    TeMPOraL · 1790090510 · focus · HN ↗
    More apt analogy here: a cyclist pushing a bicycle down the mountain and seeing it somehow get down the whole track without falling down, is not a collaboration between a bicycle and a human. The human was not involved beyond giving the initial push.
    speed_spread · 1790108944 · focus · HN ↗
    Cyclist still chooses the time, mountain and direction the bicycle gets pushed in. Bicycles have no agency and only go down because gravity. Bicycle will not "discover" tree or wall, any outcome solely the result of cyclist's decisions even if thrown bicycles don't have generally deterministic paths. Don't anthropomorphize the bicycle.
    NetMageSCW · 1790177617 · focus · HN ↗
    If I told you to pick an encrypted message from the web site and decrypt it, and you went and did it with no further input from me, would you okay with me calling it a collaboration to decrypt the message?
    speed_spread · 1790263472 · focus · HN ↗
    Probably not, but then again I have an ego, and I have other things going on in my life than decrypting random stuff for someone else's enjoyment. Both of which aren't true for an agent/LLM.
    It's a machine - when we attribute it human concepts such as victory, mistakes, ethics, we also shift responsibility away from the operator and soon enough bad people will use it as an umbrella. The agent ate my homework! The agent bombed a school! Bad agent, no!
    1attice · 1790092538 · focus · HN ↗
    The bicycle -- a simple method of transport powered entirely by humans -- used analogically to prove a point about [clears throat] automation.
    I think this sounds somewhat less silly in English because "automotive" and "automative" don't have the same hyper-visible affinity, but all the same; you may want to consider the car as a more viable analogand.
  3. UpsideDownRide · 1790094518 · focus · HN ↗
    What do you even mean it wasn't a collaboration. At any meaningful level LLMs just plain out suck when left unguided.
    The shortcomings should really be obvious by now to anyone honest. And the marketing distortion being oushed out is just tiresome and detrimental for all of us.
    anthonyrstevens · 1790180755 · focus · HN ↗
    What's tiring is the constant snide dismissiveness of the advances in technology by people who don't even RTFA, but yet show up on every AI thread and spout the same nonsense.
2. shuvrojit · 1790087840 · focus · HN ↗
  I agree with this. I think the researchers who's harnessing the llm's power should be credited more than the model itself. We also need to understand the thought process and the prompts that are given to the model so we can learn and collab to ensure humanity's progress as much as the llm itself.
  1. TeMPOraL · 1790090213 · focus · HN ↗
    > I think the researchers who's harnessing the llm's power should be credited more than the model itself.
    Even when the report literally says the LLM did it on its own?
    Let's not over-correct in the direction of knowing better than the first party.
    jfyi · 1790091164 · focus · HN ↗
    > the report literally says the LLM did it on its own
    Not mention it also says this...
    > We are still analysing the GPT–6 Astra logs to see exactly how it executed the break.
    [deleted] · 1790092198 · focus · HN ↗
    [deleted]
  2. ricksunny · 1790095509 · focus · HN ↗
    ‘knowing what problems are worth solving —- priceless”.
    For everything else, there’s Astracard
    0c3ca83z · 1790096978 · focus · HN ↗
    [dead]
    dev_tty01 · 1790098064 · focus · HN ↗
    It also decided which problem to solve:
    "After analysing the unbroken messages on the website, it decided that the most promising message was Nr. 172, MVUEH and it also quickly suspected that the plaintext of Nr. 173, SIPVX ..."
  3. dyauspitr · 1790143068 · focus · HN ↗
    Why? Sounds like they played no role in solving it besides presenting the problem.
3. PunchyHamster · 1790088384 · focus · HN ↗
  also the entirety of the research that went into breaking enigma in the first place is in the training dataset
4. WithinReason · 1790088924 · focus · HN ↗
  From TFA:
  "However, the most astonishing thing about this break is that the GPT–6 Astra did it entirely on its own."
  1. pixelesque · 1790089155 · focus · HN ↗
    Follow up bit adds more context:
    "Carter Leffer only directed GPT–6 Astra to see if it could break any of the unbroken Enigma messages published on the Crypto Cellar Research web page."
    [deleted] · 1790096664 · focus · HN ↗
    [deleted]
  2. gre · 1790089194 · focus · HN ↗
    keep going
    awesome! keep going
    great work! keep going
    ec109685 · 1790089428 · focus · HN ↗
    > to me this shows the importance of human in the loop
    [deleted] · 1790089815 · focus · HN ↗
    [deleted]
    TeMPOraL · 1790090132 · focus · HN ↗
    /goal See if you can break any of the unbroken Enigma messages published on the Crypto Cellar Research web page.
    vorticalbox · 1790093488 · focus · HN ↗
    does /goal also have time/token limits?
    eru · 1790094368 · focus · HN ↗
    Not as far as I can tell, when I've been using it.
    weiran · 1790096737 · focus · HN ↗
    Only when my usage runs out
    antii · 1790092557 · focus · HN ↗
    Now we just need this as a service. Another LLM that would encourage your agent like a cheerleader and provide emotional support and reassurance if necessary
    andriy_koval · 1790095740 · focus · HN ↗
    Make me proud is current SOTA
    nonethewiser · 1790106145 · focus · HN ↗
    Las Vegas Algorithm: A randomized, non-deterministic algorithm that is 100% accurate but has a variable runtime.
5. chrisjj · 1790089073 · focus · HN ↗
  > The correct title: Researcher brakes one specific stubborn historic enigma message with good help from Astra.
  That's not correct for the content.
  "However, the most astonishing thing about this break is that the GPT–6 Astra did it entirely on its own. Carter Leffer only directed GPT–6 Astra to see if it could break any of the unbroken Enigma messages published on the Crypto Cellar Research web page."
  1. [deleted] · 1790091941 · focus · HN ↗
    [deleted]
6. mrcwinn · 1790090483 · focus · HN ↗
  Your suggested title is more misleading, so what’s the value in using it?
7. josu · 1790090770 · focus · HN ↗
  8 months ago I got to the top of highload.fun using GPT-5 and Opus 4.5, and a lot of human interaction.
  Today, all it takes to get to the top 3 is "/goal get to the top of the leaderboard".
  The human-in-the-loop is only a temporary measure until the models get good enough.
  1. zahlman · 1790133690 · focus · HN ↗
    That appears to be against the site rules, though.
8. wolfi1 · 1790091487 · focus · HN ↗
  that the left rotor turned over at 72 - was that a bug or a feature?
9. serbuvlad · 1790091726 · focus · HN ↗
  Even if this is true, we must avoid falling into the trap of Kasparov of betting on Centaur Chess.
  Just like with Kasparov's Centaur Chess, the idea of a 'human in the loop' is just a necessity due to current limitations.
  There will hopefully (?) come a time one day when human beings provide only ultimate value judgments, and everything else is done by machines. Or it may not.
  But I don't think betting your ego on the idea that you will be useful in the loop for very long is very wise.
  1. skybrian · 1790093039 · focus · HN ↗
    This is a bit too future-oriented. Let's not mix up current capabilities and speculation about future capabilities. For the time being, collaboration works well. What the future brings is uncertain.
    ctoth · 1790093268 · focus · HN ↗
    > This is a bit too future-oriented.
    This? Still? After everything?
    Buddy, you're living in the future. In a science fiction novel. Please get used to it.
    skybrian · 1790093532 · focus · HN ↗
    It's good to think about where you're going, but you also have to keep track of where you are.
    (Also, getting people to think about the future rather than the present is a classic con. Looking at an empty field: "can't you just see the potential here?")
    pyrale · 1790095168 · focus · HN ↗
    > Please get used to it.
    The novel title: "Don’t Build The Torment Nexus".
    sifar · 1790100793 · focus · HN ↗
    If there is one thing I have realized after reading quite a few science fiction books, it is that I don't want to live in any one them - not one.
    Unfortunately, there are enough people in the world who think that is the future everyone should live in and are actively working to bring it about.
    And so, one must adapt. .
    myrmidon · 1790164875 · focus · HN ↗
    > If there is one thing I have realized after reading quite a few science fiction books, it is that I don't want to live in any one them - not one.
    I find this suprising kinda. If I got a choice right now, there's a bunch of dystopian Scifi that I would instantly go for (out of sheer curiosity), e.g. the Murderbot universe.
    Are there fantasy worlds that you would want to live in? I feel this is a bit of a suspect benchmark in the first place because books typically want some kind of tension/conflict which you won't get if everyone is just gratefully living their best life.
    anthonyrstevens · 1790180578 · focus · HN ↗
    Alan Dean Foster created some very appealing universes.
    serbuvlad · 1790095656 · focus · HN ↗
    I'm still in my 20's so I feel some necessity to be future-oriented.
    I do expect centaurs to outperform other systems for many types of tasks for years to come (and am kind of betting on this to keep getting paid).
    But what I'm talking about is ego. It your ego is tied up with (a) your intelligence or (b) your ability to perform task X; you will probably be humbled this century.
    goatlover · 1790104419 · focus · HN ↗
    Why do you want machines to replace human intelligence and ability? I find that dystopian. AI could have meant Augmented Intelligence (which was proposed a long time ago), not let's see when we can replace all human activity.
    One is humanist, the other is anti-human, (in the end goal at least).
    krisoft · 1790109677 · focus · HN ↗
    > Why do you want machines to replace human intelligence and ability?
    Where do you see the commenter say this?
    goatlover · 1790132386 · focus · HN ↗
    > There will hopefully (?) come a time one day when human beings provide only ultimate value judgments, and everything else is done by machines.
    In their previous grandparent post. They do reserve a place for human value judgement, but I doubt even that remains if everything else has been automated. The machines will decide what we value. We already see that to some degree with algorithmic engagement and targeted advertisements.
    serbuvlad · 1790148801 · focus · HN ↗
    The (?) is meant to express doubt that it is a good thing on net. But regardless of whether it is, it is what is likely to happen, and there will certainly be some good things that come out of it.
    dyauspitr · 1790143216 · focus · HN ↗
    Seems like machines being able to replace human intelligence and ability will be a dystopia in the short term, but it’s the only route to a long-term utopia.
  2. banannaise · 1790093549 · focus · HN ↗
    While this is a reasonable analogy, engines became better than humans in the late 1990s, and engines became better than centaurs in the early 2020s. Could AI-powered mathematics improve faster than the ~25 years it took for chess? The AI labs are certainly hoping it does, but that's far from a guarantee.
    myrmidon · 1790164146 · focus · HN ↗
    Might be a bad analogy, because early chess computers were purely algorithmic and worked despite their poor board state evaluation heuristics that humans used to be able to augment.
    Compare with go (boardgame): Centaur go was basically not ever a thing.
    Other applications behaved kinda similarly (AI Starcraft/Dota/...), where we had decent "human-like" heuristics from the get go and the Centaur concept could never really shine, much less for a decade or more.
    I'd also like to stress that past progress in this mainly happened for the love of the game, while the (economical) incentives to replace human office workers are... high.
  3. zahlman · 1790133536 · focus · HN ↗
    > hopefully (?)
    What you describe doesn't actually sound to me like the worthy goal it might initially come across as. Struggle helps us feel alive.
10. 1attice · 1790092136 · focus · HN ↗
  *breaks, and also, your conclusion ignores the words typed by the article's author in the piece you presumably read, where it is reported that Astra did it mostly on its own.
  Therefore Astra could also have done this comment better
11. mannyv · 1790093394 · focus · HN ↗
  What's amazing is this comment is complete bullshit, and yet is #1.
  Don't people actually read anymore?
12. HDThoreaun · 1790093849 · focus · HN ↗
  Your "corrected" title is much less descriptive of what actually happened
13. moffkalast · 1790095047 · focus · HN ↗
  If generating an image doesn't make you an artist, generating a solution doesn't make you a researcher.
  1. jfyi · 1790099601 · focus · HN ↗
    Well, it makes them an AI researcher maybe but not a cryptographer anyway.
    edit: I think that's going to be my go to on "you aren't an artist" from now on. "No! I'm an AI researcher!"
14. jtrn · 1790099272 · focus · HN ↗
  It seems I was wrong in this instance with regard to the "colab" part.
  I found that Leffen even said the explanatory website took about 99 times more effort than the codebreaking itself. And he said that he set the direction and pushed, and the model did the execution. How much steering "pushed forward" involved is not disclosed anywhere, but in this instance, it seems to be more a case of "Human pointed at hard task and AI did an awesome job mostly by itself." Tho how much he was a simple meat-ralph-loop is not entirely clear.

Open on Hacker News to reply ↗

Unofficial Hacker News client; not affiliated with Y Combinator.