I wish they had attempted to measure product management skill.
My hypothesis is that the AI users gained less in coding skill, but improved in spec/requirement writing skills.
But there’s no data, so it’s just my speculation. Intuitively, I think AI is shifting entry level programmers to focus on expressing requirements clearly, which may not be all that bad of a thing.
What AI development has done for my team is the following:
Dramatically improved Jira usage -- better, more descriptive tickets with actionable user stories and clearly expressed requirements. Dramatically improved github PRs. Dramatically improved test coverage. Dramatically improved documentation, not just in code but in comments.
Basically all _for free_, while at the same time probably doubling or tripling our pace at closing issues, including some issues in our backlog that had lingered for months because they were annoying and nobody felt like working on them, but were easy for claude to knock out.
I'd be willing to bet that your AI written issues, docs, etc look impressive initially but are extremely low signal to noise. You might be checking some boxes (docstrings, etc) but I do not envy anyone on your team that needs to actually read any of that stuff in the future to solve an actual problem.
> Dramatically improved Jira usage -- better, more descriptive tickets with actionable user stories and clearly expressed requirements. Dramatically improved github PRs. Dramatically improved test coverage. Dramatically improved documentation, not just in code but in comments.
> Basically all _for free_
Not for free, the cost is that all of those are now written by AI so not really vetted any longer. Or do you really think your team is just using AI for code?
> I wish they had attempted to measure product management skill.
We're definitely getting better at writing specs. The issue is the labor bottleneck is competent senior engineers, not juniors, not PMs, not box-and-arrow staff engineers.
> I think AI is shifting entry level programmers to focus on expressing requirements clearly
This is what the TDD advocates were saying years ago.