It may surprise some people here to see that Andrew is warming up to using LLMs to discover bugs (inspired by results from SQLlite) and considers it a tool on the path to getting to bug free software.
> It may surprise some people here to see that Andrew is warming up to using LLMs to discover bugs (inspired by results from SQLlite) and considers it a tool on the path to getting to bug free software.
Someone in the thread below says their bug was closed because of mentioning that they use AI to confirm the bug they had encountered. Is he going to go back and reopen all of those now that he learned what pretty much everyone else already knew?
Neither of those are my understanding of what happened: a human found a bug, and then had an LLM verify their theory of the issue. To me, that's just extra due diligence and pretty weird to use as grounds to ignore.
You are referring to this comment right? <a href="https://news.ycombinator.com/item?id=49938732">https://news.ycombinator.com/item?id=49938732
> that I had every LLM check it to confirm it's a bug
I'm primarily objecting the way he phrased it, as opposed to just saying "I've tested and confirmed and reproduced the bugs". Instead, it sounds uncertain and detached, like he asked the LLMs if it's indeed a bug without further verification.
audunw · · focus · HN ↗
<a href="https://youtu.be/zwi5b5xSsKA?is=PTjJJjSnVMdRuZag" rel="nofollow">https://youtu.be/zwi5b5xSsKA?is=PTjJJjSnVMdRuZag
It may surprise some people here to see that Andrew is warming up to using LLMs to discover bugs (inspired by results from SQLlite) and considers it a tool on the path to getting to bug free software.
saghm · · focus · HN ↗
Someone in the thread below says their bug was closed because of mentioning that they use AI to confirm the bug they had encountered. Is he going to go back and reopen all of those now that he learned what pretty much everyone else already knew?
nvlled · · focus · HN ↗
- using the LLM to find (possible) bugs and a human confirms it by testing, reviewing, etc.
- using the LLM to find and confirm the bug without the human confirming it
saghm · · focus · HN ↗
nvlled · · focus · HN ↗
> that I had every LLM check it to confirm it's a bug
I'm primarily objecting the way he phrased it, as opposed to just saying "I've tested and confirmed and reproduced the bugs". Instead, it sounds uncertain and detached, like he asked the LLMs if it's indeed a bug without further verification.