It may surprise some people here to see that Andrew is warming up to using LLMs to discover bugs (inspired by results from SQLlite) and considers it a tool on the path to getting to bug free software.
> It may surprise some people here to see that Andrew is warming up to using LLMs to discover bugs (inspired by results from SQLlite) and considers it a tool on the path to getting to bug free software.
Someone in the thread below says their bug was closed because of mentioning that they use AI to confirm the bug they had encountered. Is he going to go back and reopen all of those now that he learned what pretty much everyone else already knew?
Shouldn't everybody already know that while using AI to find bugs for oneself is amazingly efficient, using AI to submit bug reports for others is quite the opposite? Burden of verification and all.
Exactly, the main issue isn't the LLM creating or confirming the report. Rather, the maintainer has no idea how it was prompted, and the results may be completely wrong. Some LLMs also have a bad habit of trying to please the user, confirming their biases.
I don't care how it was prompted any more than I care about what exact thoughts went through your head before you realized it was a bug. The problem isn't usually a lack of information, but the abundance of non-information. Chatbots and the people who use them to pen bug reports frequently seem to have had "no time to make it shorter", as Blaise Pascal put it.
That you used an LLM to "confirm" the bug is an example of such non-information. The reproduction testcase and/or the reproduction steps and description of the expected outcome confirm the bug.
audunw · · focus · HN ↗
<a href="https://youtu.be/zwi5b5xSsKA?is=PTjJJjSnVMdRuZag" rel="nofollow">https://youtu.be/zwi5b5xSsKA?is=PTjJJjSnVMdRuZag
It may surprise some people here to see that Andrew is warming up to using LLMs to discover bugs (inspired by results from SQLlite) and considers it a tool on the path to getting to bug free software.
saghm · · focus · HN ↗
Someone in the thread below says their bug was closed because of mentioning that they use AI to confirm the bug they had encountered. Is he going to go back and reopen all of those now that he learned what pretty much everyone else already knew?
itishappy · · focus · HN ↗
dev-in · · focus · HN ↗
boomlinde · · focus · HN ↗
That you used an LLM to "confirm" the bug is an example of such non-information. The reproduction testcase and/or the reproduction steps and description of the expected outcome confirm the bug.