Comment by godelski

  > this is a massive conflict of interests

I think everyone is aware of this.

But people like that they aren't shying away from negative results and that builds some trust. Though let's not ignore that they're still suggesting AI + manual coding.

But honestly, this sample size is so small that we need larger studies. The results around what is effective and ineffective AI usage is a complete wash with n<8.

Also anyone else feel the paper is a bit sloppy?

I mean there's a bunch of minor things but Figure 17 (first fig in the appendix) is just kinda wild. I mean there's trivial ways to solve the glaring error. The more carefully you look at even just the figures in the paper the more you say "who the fuck wrote this?" I mean like how the fuck do you even generate Figure 12? The numbers align with the grids but boxes are shifted. And Figure 16 has experience levels shuffled for some reason. And then there are a hell of a lot more confusing stuff you'll see if you do more than a glance...