Comment by octocop
The models used have improved quite well since then, I guess his change of opinion shows that.
The models used have improved quite well since then, I guess his change of opinion shows that.
I think it's more about how people are using it. An amateur who spams him with GPT-5-Codex produced bug reports is still a waste of his time. Here a professional ran the tools and then applied their own judgement before sending the results to the curl maintainers.
The last time I was staffed on a project that had to do this, we were looking at many dozens per day, virtually all of them bogus, many attached to grifters hoping to jawbone the triage person into paying a nominal fee to get them to shut up. It would be weird if new tooling like LLMs didn't accelerate it, but that's all I'd expect it to do.
No, he's still dealing with a flood of crap, even in the last few weeks, off more modern models.
It's primarily from people just throwing source code at an LLM, asking it to find a vulnerability, and reporting it as-read, without having any actual understanding of if it is or isn't a vulnerability.
The difference in this particular case is it's someone who is: 1) Using tools specifically designed for security audits and investigations. 2) Takes the time to read and understand the vulnerability reported, and verifies that it is actually a vulnerability before reporting.
Point 2 is the most significant bar that people are woefully failing to meet and wasting a terrific amount of his time. The one that got shared from a couple of weeks ago https://hackerone.com/reports/3340109 didn't even call curl. It was straight up hallucination.