Comment by MattyRad
Seems like a catch-22. For codebases that I'm highly familiar with and regularly perform code review in, I'd say "thanks LLM, but I don't trust you, I'm more familiar with this codebase than you, and I don't need your help." For codebases that I'm not familiar with, I'm not really performing code review (at least not approving MR/PRs or doing the merging).
But still, this is very creative and a nice application of LLMs that isn't strictly barf.
Ok, I'll bite though, let's try it out as a non-maintainer.
I loaded https://0github.com/laravel/framework/pull/57499. Completely random, it's a PR in the last github repo I had open.
At 60%, it highlights significantly more test code than the material changes that need review. Strike one.
At no threshold (0-100) does it highlight the deleted code in UniqueBroadcastEvent.php, which seems highly important to review. The maintainer even comments about the removal in the actual PR! Strike two.
The only line that gets highlighted at > 50% in the material code diffs is one that hasn't changed. Strike three.
So, honest attempt, but it didn't work out for me.