Thread

Pop!_OS bans AI-generated code from much of its codebase

116 points · 167 comments · bundie

ItsMattyG · 1791054287 · focus · HN ↗
I don't see how this will survive the attacker/defender gap as ls get increasingly good at cyber security and finding 0 days... but maybe it's an obscure enough is it doesn't matter?
1. barbazoo · 1791054396 · focus · HN ↗
  I'm assuming they still use AI to find vulnerabilities, just not fix them?
2. PorciiVorbesc · 1791054482 · focus · HN ↗
  Probably because pop_os and Cosmic are so niche and their market share so insignificant, that they're irrelevant to attackers and bad actors, when those now have much bigger fish to fry to make their time and effort worth it. See the Arch AUR attacks, for perspective.
  I think even amongst the HN and Linux userbase, pop_os is still niche, let alone amongst normies who never heard about Linux. So they can afford take the high road and treat it like their personal sandbox, accepting only human written code.
  But larger and more important projects like Fedora and Debian are more pragmatic with the fact that they'll have to accept AI written(but human reviewed) code, if they wish to keep up with the real world development and threats, as expressed by Linus Torvalds himself.
  The thing is, the cat's out of the bag on this one now, especially in the field of pen-testing and reverse-engineering. AI can brute-force its way into projects in ways that beat even experienced researchers, so your only choice to keep up is to accept the use of AI generated fixes as a counter defense.
  1. DelightOne · 1791055578 · focus · HN ↗
    It also means security is not held as high and vulnerabilities not as much found. A simple 0-day may survive for years. Not much effort needed to have permanent access.
    PorciiVorbesc · 1791055789 · focus · HN ↗
    Sorry, I don't understand what you mean by this, can you elaborate pls?
    DelightOne · 1791058635 · focus · HN ↗
    Its easier for an LLM to find vulnerabilities in a project with less usage, and those vulnerabilities will stay open longer, making them much cheaper to attack to keep the door open.
    PorciiVorbesc · 1791059697 · focus · HN ↗
    What's the point of attacking projects that almost nobody uses?
    Do you think Netanyahu, Trump or Xi-Jinping are somehow secretly using Cosmic DE at home, to be worthy targets?
    Bad actors have limited time, lives of their own and mouths to feed as well, so they concentrate their efforts where "the fish are" if they want to PWN someone for profit.
    That's why Windows was the biggest target in the past for so long and why MacOS and Linux were ignored. Because most of the fish were on Windows.
    DelightOne · 1791060542 · focus · HN ↗
    If it costs you as good as nothing, you might as well do it.
    Previously, time was the most precious resource. Now its tokens, and more cheaply at at.
    PorciiVorbesc · 1791061145 · focus · HN ↗
    Offensive security employees, tokens, and peoples' time are still a finite resource that get allocated based on target priorities and operational end-goals, even by state actors.
    If you assume Mossad and NSA are Token-maxxing every single niche FOSS project out there to cast as large as possible fishnet on hacking all Average Joes on the planet just in case, then maybe using Mozilla and MacOS gets you hacked too, maybe even visiting HN and commenting here gets you hacked by some zero days you don't yet know.
    Where does this open-ended paranoia argument end?
  2. badc0ffee · 1791056116 · focus · HN ↗
    It does show up on top 5 lists for Linux desktop distros quite a lot, and COSMIC is quite unique, so I suspect a lot of people are at least trying it.
    (I wasn't able to make it work on a scrap Dell I tried it on because the GPU was too old. Booted the USB key and COSMIC greeter failed to start)
    PorciiVorbesc · 1791056340 · focus · HN ↗
    >It does show up on top 5 lists for Linux desktop distros quite a lot
    1) Firstly, your source plase? My research according to Google Gemini 3.8 Pro shows top 5 DEs are as follows:
    +---+---------------+------------+-------------------------------------+ | # | Environment | Est. Share | Primary Ecosystem / Defaults | +---+---------------+------------+-------------------------------------+ | 1 | GNOME | 45% - 50% | Ubuntu, Fedora, Debian, RHEL | | 2 | KDE Plasma | 25% - 30% | SteamOS, openSUSE, Kubuntu, Manjaro | | 3 | Cinnamon | 8% - 12% | Linux Mint flagship | | 4 | Xfce | 6% - 9% | MX Linux, Xubuntu, low-spec PCs | | 5 | MATE | 2% - 4% | Ubuntu MATE, Mint MATE (GNOME 2) | | - | Others / WMs | 3% - 5% | Hyprland, i3, Sway, LXQt, Budgie | +---+---------------+------------+-------------------------------------+
    So then COSMIC isn't even TOP 5, for this to be a major target by market share as originally claimed.
    2) Secondly, keep in mind that distrowatch is not representative of linux userbase. MX-Linux kept showing up at the top spot for many years despite being niche.
    3) And thirdly, TOP 5 DE isn't really an achievement when Linux DE market share is overwhelmingly dominated by KDE Plasma and Gnome as the majority shareholders, with XFCE and Cinnamon trailing. So Cosmic DE if it somehow made the no. 5 spot, would still be ignorable sub <1% market share, as I initially claimed, basically invisible to bad actors.
    yugoslavia4ever · 1791057817 · focus · HN ↗
    You're being way too defensive. GP replying to you was clearly trying to have a conversation, not attack your knowledge. Chill.
    PorciiVorbesc · 1791058583 · focus · HN ↗
    >You're being way too defensive.
    "Way too defensive" how? By asking and bringing data for my PoV?
    >GP replying to you was clearly trying to have a conversation
    As am I, except I ask for, and also bring data to back up my PoV, instead of vague opinions.
    >Chill.
    Where am I not being chill?
    kyubey · 1791062166 · focus · HN ↗
    To reflect some sentiment from another comment you posted - we're not your unpaid tone auditors. Learn to have normal discussions.
    For funsies, I threw two different DE-by-marketshare inquiries at Gemini 3.8 Pro in separate sessions and it ranked Cosmic 12th in the first and 7th in the second. Very trustworthy stuff.
    PorciiVorbesc · 1791062577 · focus · HN ↗
    >Learn to have normal discussions.
    Please show me what part of what I said before was not "a normal discussions" according to you?
    >it ranked Cosmic 12th in the first and 7th in the second. Very trustworthy stuff.
    And that disproves me how exactly? I originally showed "Cosmic is NOT a TOP5 DE", and the LLM data you posted also shows that IT IS indeed NOT a top 5 DE.
    cindyllm · 1791064354 · focus · HN ↗
    [dead]
    satvikpendem · 1791058142 · focus · HN ↗
    Don't post AI generated content, if needed post the actual source.
    PorciiVorbesc · 1791058473 · focus · HN ↗
    >Don't post AI generated content
    Do you have a better source than GP?
    >if needed post the actual source
    Define "actual source"? In good faith, I mean.
    Where else do you get this information that's, quote, "actual source"?
    satvikpendem · 1791058661 · focus · HN ↗
    As in the study or survey or analytics segment where the data was collected to produce the report.
    PorciiVorbesc · 1791059369 · focus · HN ↗
    "The analytics", is the publicly available training dataset of the LLMs that share the same common opinion on that DE market share.
    What do you expect exactly? Do you want me to now manually parse through terabytes of information at your whim for your own convenience? Sorry, but I'm not your personal unpaid servant.
    If you wish to disprove me in the comment section, then you need to do the manual work and show us that the my quoted LLMs statistics are wrong. I'm not your personal errand boy to do your bidding, 'massa'.
    satvikpendem · 1791060719 · focus · HN ↗
    Point at the public dataset then if it's so public. You really have no idea how the LLM is parsing the data and could easily be hallucinating. That you find being called out on this as some sort of insult is quite telling and frankly funny, what a strong reaction to someone asking for a basic source when the burden of proof is on you to prove (or at least show a source that) these stats are right, not on anyone else to disprove AI bullshit.
    PorciiVorbesc · 1791062000 · focus · HN ↗
    Sorry, it's not my job to provide for you the things that you demand from me at your whims, because the original comment I replied to with LLM data also did not bring peer-reviewable information as proof that COSMIC is a TOP 5 DE, and yet you did not demand proof from them in that case. Why is that?
    Did you just blindly trust the opinions of others on this topic, and yet I'm the one who has to provide peer-reviewable data for you to back-up mine? Sorry, I'm not your unpaid lackey. Try to formulate a better (counter)argument for why their baseless opinion is right, but my LLM backed up opinion is wrong, if you wish for a even-footed good-faith argument.
    satvikpendem · 1791065123 · focus · HN ↗
    I ask them for the same source (and looks like they provided it), your comment wasn't special. LLMs however are especially less trustworthy, that's why. It's not my job to educate you on why they are, as you say, and why people aren't trusting your comments.
    PorciiVorbesc · 1791066531 · focus · HN ↗
    >I ask them for the same source (and looks like they provided it)
    I also saw it now. That blog is not a representative ground truth, but just another opinion piece, which I can respect as an opinion of the blog's user base, but I can't take as an accurate real world statistic, same how aggregate opinions you read on HN are not representative of the actual real world.
    I hope you can understand my PoV. You can also disagree if you want, but you'll need to bring something more than "that's wrong because LLMs sometimes hallucinate" as proof that Gemini's data is wrong in this case.
    satvikpendem · 1791068136 · focus · HN ↗
    In theory if you brought a better source than that person then I'd agree with you, but,
    > you'll need to bring something more than "that's wrong because LLMs sometimes hallucinate" as proof that Gemini's data is wrong in this case.
    actually I can say it's wrong or likely to be wrong especially if it doesn't cite the sources it uses. And if it does, then just paste the sources here instead of the LLM output. It is also unknown where it got the info and as someone else said, you ask it two different times and it gave two different answers, thus it is unreliable.
    badc0ffee · 1791058758 · focus · HN ↗
    Sorry for being unclear. I meant it shows up as a top 5 distro recommended by reviewers. This kind of thing: <a href="https://linuxblog.io/best-linux-distro/" rel="nofollow">https://linuxblog.io/best-linux-distro/
    PorciiVorbesc · 1791058890 · focus · HN ↗
    OK, but what's the sample size of that and who's measuring it? I never heard of that blog or took part in that poll. So how is that blog link the yardstick but mine is not?
    nvme0n1p1 · 1791064680 · focus · HN ↗
    Because that blog actually talked to real people and did real work, and yours is just a hallucinated list from one of the me-too LLM vendors
    PorciiVorbesc · 1791065297 · focus · HN ↗
    >Because that blog actually talked to real people and did real wor
    How did you verify that those people from the blog are "real"?
    I also talked to real people for my own data, case in point, I asked my mom and dad which linux DE is most used, and the results came out different. Which "real people" are the ones representative for the ground truth of Linux DE sahre?
    > and yours is just a hallucinated list from one of the me-too LLM vendors
    How do you know it's hallucinated? Ask the LLM the population of your country? Is the answer mostly accurate or is it hallucinated in an inaccurate way?
    Aren't LLMs just outputting the highest statistical probability from the aggregate of their scraped data, which in this case would be including opinions on Reddit, and every website and blog on the entire internet (including that random one posted by badc0ffee) on the Linux DE uusage topic, making it a more accurate real-world representation than just a single random blog?
    You can call it "hallucinated" if you want, but that doesn't mean it's not accurate. I asked for proof that my answer was inaccurate, not that it was "hallucinated", those are two different things, and your argument didn't prove it was inaccurate nor did it prove it was hallucinated. Would you like to try again?
    satvikpendem · 1791065139 · focus · HN ↗
    Source on this?
    Edit: I see you posted elsewhere.
  3. iugtmkbdfil834 · 1791056958 · focus · HN ↗
    << Probably because pop_os and Cosmic are so niche and their market share so low, that they're irrelevant to attackers and bad actors, when those now have much bigger fish to fry.
    That is such a weird statement that I am not entirely certain where to begin. PopOS is hardly niche. Its base are all fairly common components by linux standards. And, more importantly, attackers and bad actors may other considerations in mind than sheer population size -- just to point out the glaringly obvious.
    << I think even amongst the HN and Linux userbase, pop_os is still niche
    I think rather than trying to disprove it, I think I should ask why you think that? If anything, PopOS annoys me because it is just a step before ubuntu ( and ubuntu is just windows at this point ). Maybe I am defensive, because my first real distribution ( that did not share disk with windows was popos )?
    PorciiVorbesc · 1791057142 · focus · HN ↗
    [dead]
  4. satvikpendem · 1791058103 · focus · HN ↗
    You only need one bad actor. For example, someone reading this thread could easily decide to start attacking it just because someone else said it wasn't worth it, as a personal challenge.
    PorciiVorbesc · 1791058764 · focus · HN ↗
    >You only need one bad actor.
    If that's your threat model then you shouldn't use any SW in exitance, FOSS or otherwise. In fact you shouldn't even go online, or even outside you own house, since one single bad actors exist everywhere. You can walk down the street and suddenly someone in a car runs you over(witnessed myself). And yet live goes on.
    satvikpendem · 1791058996 · focus · HN ↗
    Of course. I never said life doesn't go on or it's a worthwhile risk to care about, not sure how you got that from my comment.
    PorciiVorbesc · 1791059168 · focus · HN ↗
    > not sure how you got that from my comment.
    Because your comment didn't disprove that Cosmic DE isn't too niche for bad actors to get involved.
    satvikpendem · 1791061234 · focus · HN ↗
    It doesn't have to be niche (or not) to have bad actors. That doesn't make it likely of course but it is possible, and increasingly more so in the age of AI when you can simply point it at multiple projects in parallel.
    PorciiVorbesc · 1791062524 · focus · HN ↗
    >It doesn't have to be niche (or not) to have bad actors.
    Then they shouldn't reject AI aids to help them patch vulns found by bad actors faster, no?
    satvikpendem · 1791066091 · focus · HN ↗
    I never said they should. I think you're reading too much into my comment, the point was something being niche doesn't prevent it from being threatened.
  5. plqbfbv · 1791063149 · focus · HN ↗
    > Probably because pop_os and Cosmic are so niche and their market share so insignificant
    Consider that it's packaged for many well-known distros, so pop_os install base alone doesn't tell the whole story: <a href="https://system76.com/cosmic/download" rel="nofollow">https://system76.com/cosmic/download
3. vorticalbox · 1791054619 · focus · HN ↗
  The issue is “ai generated code” using ai to find bugs/0 day and manually writing a fix would (I assume) be allowed under the new rules.
4. altcognito · 1791054671 · focus · HN ↗
  Irony is that they will get the benefits from upstream projects (like the Linux kernel) that does accept AI inputs.
5. VCFundedGenYer · 1791055303 · focus · HN ↗
  AI finds a lot of "vulnerabilities" but most are fake, untested, or not actually vulnerabilities.
  Reminder that AI is quite stupid.
  1. dumberquestions · 1791056082 · focus · HN ↗
    I think this is just plain denial, AI has found many high severity vulnerabilities.
    zdragnar · 1791057259 · focus · HN ↗
    There's plenty more false positives than actual finds. There are still actual finds, but that doesn't change all the false positives.
    DaSHacka · 1791058222 · focus · HN ↗
    This is why you run a second agent to verify any assertions.
    zdragnar · 1791080226 · focus · HN ↗
    It'd be great if people did that. They don't. Open source projects with limited budgets shouldn't have to spend money from those limited budgets (or personal funds!) filtering through what amounts to spam. That's why there was a slew of open source projects that turned off issues on github and stopped taking outside contributions- they literally couldn't afford to keep up with the nonsense lazy bums were sending their way.
  2. novafunc · 1791060955 · focus · HN ↗
    It may have a high false positive rate, but at the speed AIs can review code, there's still plenty, of real vulnerabilites mixed in with the garbage.
    I'll trust the words of groups like curl (<a href="https://daniel.haxx.se/blog/2026/06/10/a-human-in-control/" rel="nofollow">https://daniel.haxx.se/blog/2026/06/10/a-human-in-control/), Linux, and even the infamously anti-AI Gnome (<a href="https://blogs.gnome.org/mcatanzaro/2026/10/02/the-era-of-software-quality-or-the-era-of-ostriches/" rel="nofollow">https://blogs.gnome.org/mcatanzaro/2026/10/02/the-era-of-sof...) that AI is finding real vulnerabilities and you're your project a disservice by ignoring them.
    Edit: Though Greg did recently have a talk (that I skimmed) where he was a little reserved on LLMs: <a href="https://www.youtube.com/watch?v=NnV_cWeoo5Q" rel="nofollow">https://www.youtube.com/watch?v=NnV_cWeoo5Q
6. layer8 · 1791056024 · focus · HN ↗
  Using AI to find vulnerabilities doesn’t mean that you need to use AI to generate the code that fixes them. And you can still ask AI whether it thinks the fix is okay, as a second opinion.
7. fwlr · 1791058340 · focus · HN ↗
  So you took every single line of open source code you could possibly get your hands on (using scrapers so violently dumb that they amount to a permanent low-grade DDoS) and spent billions of dollars to tune trillions of parameters, and the value you can offer is… “let us inundate you with bad code or else we’ll generate exploits for your software”.

Open on Hacker News to reply ↗

Unofficial Hacker News client; not affiliated with Y Combinator.