Comment by addaon

Comment by addaon 6 days ago

9 replies

View on Hacker News

> if you work on a team most code you see isn’t yours.. ai code review is really no different than reviewing a pr… except you can edit the output easier and maybe get the author to fix it immediately

And you can't ask "why" about a decision you don't understand (or at least, not with the expectation that the answer holds any particular causal relationship with the actual reason)... so it's like reviewing a PR with no trust possible, no opportunity to learn or to teach, and no possibility for insight that will lead to a better code base in the future. So, the exact opposite of reviewing a PR.

arrowleaf 6 days ago

Are you using the same tools as everyone else here? You absolutely can ask "why" and it does a better job of explaining with the appropriate context than most developers I know. If you realize it's using a design pattern that doesn't fit, add it to your rules file.

Reply View 6 replies
  • addaon 6 days ago

    You can ask it "why", and it gives a probable English string that could reasonably explain why, had a developer written that code, they made certain choices; but there's no causal link between that and the actual code generation process that was previously used, is there? As a corollary, if Model A generates code, Model A is no better able to explain it than Model B.

    Reply View 3 replies
    • ramchip 6 days ago

      I think that's right, and not a problem in practice. It's like asking a human why: "because it avoids an allocation" is a more useful response than "because Bob told me I should", even if the latter is the actual cause.

      Reply View 2 replies
      • addaon 6 days ago

        > I think that's right, and not a problem in practice. It's like asking a human why: "because it avoids an allocation" is a more useful response than "because Bob told me I should", even if the latter is the actual cause.

        Maybe this is the source of the confusion between us? If I see someone writing overly convoluted code to avoid an allocation, and I ask why, I will take different actions based on those two answers! If I get the answer "because it avoids an allocation," then my role as a reviewer is to educate the code author about the trade-off space, make sure that the trade-offs they're choosing are aligned with the team's value assessments, and help them make more-aligned choices in the future. If I get the answer "because Bob told me I should," then I need to both address the command chain issues here, and educate /Bob/. An answer is "useful" in that it allows me to take the correct action to get the PR to the point that it can be submitted, and prevents me from having to make the same repeated effort on future PRs... and truth actually /matters/ for that.

        Similarly, if an LLM gives an answer about "why" it made a decision that I don't want in my code base that has no causal link to the actual process of generating the code, it doesn't give me anything to work with to prevent it happening next time. I can spend as much effort as I want explaining (and adding to future prompts) the amount of code complexity we're willing to trade off to avoid an allocation in different cases (on the main event loop, etc)... but if that's not part of what fed in to actually making that trade-off, it's a waste of my time, no?

        Reply View 1 reply
        • ramchip 6 days ago

          Right. I don't treat the LLM like a colleague at all, it's just a text generator, so I partially agree with your earlier statement:

          > it's like reviewing a PR with no trust possible, no opportunity to learn or to teach, and no possibility for insight that will lead to a better code base in the future

          The first part is 100% true. There is no trust. I treat any LLM code as toxic waste and its explanations as lies until proven otherwise.

          The second part I disagree somewhat. I've learned plenty of things from AI output and analysis. You can't teach it to analyze allocations or code complexity, but you can feed it guidelines or samples of code in a certain style and that can be quite effective at nudging it towards similar output. Sometimes that doesn't work, and that's fine, it can still be a big time saver to have the LLM output as a starting point and tweak it (manually, or by giving the agent additional instructions).

          Reply View 0 replies
  • JackFr 6 days ago

    Although it cannot understand the rhetorical why as in a frustrated “Why on earth would you possibly do it that brain dead way?”

    Instead of the downcast, chastened look of a junior developer, it responds with a bulleted list of the reasons why it did it that way.

    Reply View 1 reply
    • danielbln 6 days ago

      Oh, it can infer quite a bit. I've seen many times in reasoning traces "The user is frustrated, understandably, and I should explain what I have done" after an exasperated "why???"

      Reply View 0 replies
supern0va 6 days ago

>And you can't ask "why" about a decision you don't understand (or at least, not with the expectation that the answer holds any particular causal relationship with the actual reason).

To be fair, humans are also very capable of post-hoc rationalization (particularly when they're in a hurry to churn out working code).

Reply View 0 replies