It's great to hear about $topic from someone no-nonsense and in-the-know like Greg KH. You can verify all of this too - since, well Linux kernel. (As opposed to what Microsoft or Apple claims to fix as far as LLM finds.)
Mythos may not be great today but it is not far fetched to imagine bug discovery, analysis and fixes can be made much quicker, accurate and even newly possible with specialized models trained on say Linux kernel specifics - with codemap/coding standards/threat models, good and bad coding patterns, tools to validate etc. an LLM can be much more relentless than humans and if it has the help to be accurate it will be worth the electricity burned. Oh and another model trained on triage data to validate the first one's findings would be good.
(I think Microsoft is doing this internally - different models trained internally alongside Mythos - there was some talk about it on the tubes, don't recall where exactly.)
> Mythos may not be great today but it is not far fetched to imagine...
I...
Look. Mythos was hyped up as the absolute best bug hunting tool ever made... no software was safe from its awesome bug-finding and exploit-writing capabilities. So strong was it that access _had_ to be limited to a select few pre-vetted entities, lest these awesome capabilities fall into the hands of Evildoers(!!!). Mythos' claimed capabilities were absolutely an important part of the "The LLM-based tools we're building are so dangerous that we must have new laws made to regulate us, or else all of humanity is likely to die!" story that the major LLM manufacturers have been building for a while and are telling now.
Now? Not even six months after release? "Well, yeah, okay, it's actually not that great. But imagine how great the next one could be!"... which is the story I've been hearing roughly every six months for what feels like five years now.
As an aside: I often wish we lived in a world where it was illegal for companies to use hype or any other types of emotional manipulation when advertising (or otherwise speaking in an official capacity) about tools that are to be used in a professional setting. Is it anything other than a bare statement of verifiable facts? Big fines, and repeat offenders get jail time. I know it's never going to happen, but it sure would be nice.
Just for the 'record' - I am totally with you on the hype and just in general the normalization of sleazy behavior surrounding it but I'm not sure we as normal people have any say anymore including where our money is going to be invested.
Don't lower yourself like this. The people working on this are "normal people" too. Don't put them on a pedestal. That's part of the problem here: lionizing people who are actively lying to others to protect their bags and reputation. Stop acting like a pawn for these people. Buying into their delusions of grandeur is what's perpetuating this mess. LLMs are a tool, not a monolith.
I am not lowering myself or lionizing anyone - I am being practical about this. If we as a society have a workable stance for/against LLMs that is beneficial to all of us I am very open to hearing about it - It would however be fair to say that I am pessimistic given a lot of our collective history. And on oop of that nobody seems to be talking about any real solutions either this time around. Just being dismissive or being all-in are equally bad approaches.
> ...nobody seems to be talking about any real solutions...
I guess you didn't bother to watch the video that is TFA. Greg K-H mentions that if you're going to use LLMs to do bug-finding, you should use open-weights models that run locally and "harnesses" built by members of the community You really should watch the video to hear his reasons for why.
There's nothing wrong with LLM-the-technology. There's everything wrong with the major LLM manufacturers.
I wasn't talking about fixing the tools or using open weight models - I don't need to watch anything to know that's an option. You are also now contradicting your earlier stance somewhat by saying what I was saying - LLMs can and will get better - you have now introduced an irrelevant open vs close element which doesn't really matter. There's no way only open weight models and harnesses get better - if you understand even a little, proprietary models with access to same open information but with billions to burn on research and hardware will get better faster.
Ugh stop posting links dude - answer what do you mean by your previous claim for fixing everything you were complaining about earlier - "you should use open-weights models that run locally and "harnesses" built by members of the community" - like how is that not contradicting what you are saying - if open weights and local harnesses can do it Anthropic and OpenAI will do it faster and better. That doesn't make sense at all in this conversation.
> ...like how is that not contradicting what you are saying...
A careful and honest reader notes that I've never claimed or suggested that there exists a software project that will never get better with enough time and appropriately-focused effort.
But, the single thing in this thread that statement by GK-H addresses is your claim [0] that
...nobody seems to be talking about any real [alternatives to using LLMs provided by the major LLM manufacturers]...
Of the things I've talked about in this thread, that's the least important one. However, I do understand that it is the easiest one for you to talk about while still staying vaguely "on message".
blinkingled · · focus · HN ↗
Mythos may not be great today but it is not far fetched to imagine bug discovery, analysis and fixes can be made much quicker, accurate and even newly possible with specialized models trained on say Linux kernel specifics - with codemap/coding standards/threat models, good and bad coding patterns, tools to validate etc. an LLM can be much more relentless than humans and if it has the help to be accurate it will be worth the electricity burned. Oh and another model trained on triage data to validate the first one's findings would be good.
(I think Microsoft is doing this internally - different models trained internally alongside Mythos - there was some talk about it on the tubes, don't recall where exactly.)
simoncion · · focus · HN ↗
I...
Look. Mythos was hyped up as the absolute best bug hunting tool ever made... no software was safe from its awesome bug-finding and exploit-writing capabilities. So strong was it that access _had_ to be limited to a select few pre-vetted entities, lest these awesome capabilities fall into the hands of Evildoers(!!!). Mythos' claimed capabilities were absolutely an important part of the "The LLM-based tools we're building are so dangerous that we must have new laws made to regulate us, or else all of humanity is likely to die!" story that the major LLM manufacturers have been building for a while and are telling now.
Now? Not even six months after release? "Well, yeah, okay, it's actually not that great. But imagine how great the next one could be!"... which is the story I've been hearing roughly every six months for what feels like five years now.
As an aside: I often wish we lived in a world where it was illegal for companies to use hype or any other types of emotional manipulation when advertising (or otherwise speaking in an official capacity) about tools that are to be used in a professional setting. Is it anything other than a bare statement of verifiable facts? Big fines, and repeat offenders get jail time. I know it's never going to happen, but it sure would be nice.
blinkingled · · focus · HN ↗
rglover · · focus · HN ↗
blinkingled · · focus · HN ↗
simoncion · · focus · HN ↗
I guess you didn't bother to watch the video that is TFA. Greg K-H mentions that if you're going to use LLMs to do bug-finding, you should use open-weights models that run locally and "harnesses" built by members of the community You really should watch the video to hear his reasons for why.
There's nothing wrong with LLM-the-technology. There's everything wrong with the major LLM manufacturers.
blinkingled · · focus · HN ↗
simoncion · · focus · HN ↗
Not even a little bit, no. Go back and read carefully. [0][1][2][3]
[0] <<a href="https://news.ycombinator.com/item?id=49943382">https://news.ycombinator.com/item?id=49943382>
[1] <<a href="https://news.ycombinator.com/item?id=49944036">https://news.ycombinator.com/item?id=49944036>
[2] <<a href="https://news.ycombinator.com/item?id=49951289">https://news.ycombinator.com/item?id=49951289>
[3] <<a href="https://news.ycombinator.com/item?id=49951466">https://news.ycombinator.com/item?id=49951466>
blinkingled · · focus · HN ↗
simoncion · · focus · HN ↗
A careful and honest reader notes that I've never claimed or suggested that there exists a software project that will never get better with enough time and appropriately-focused effort.
But, the single thing in this thread that statement by GK-H addresses is your claim [0] that
Of the things I've talked about in this thread, that's the least important one. However, I do understand that it is the easiest one for you to talk about while still staying vaguely "on message".[0] <<a href="https://news.ycombinator.com/item?id=49948725">https://news.ycombinator.com/item?id=49948725>