Comment by nichochar
The effort described in the article is maybe a couple hours of work.
I understand the "enjoy doing anyway" part and it resonates, but not using AI is simply less productive.
The effort described in the article is maybe a couple hours of work.
I understand the "enjoy doing anyway" part and it resonates, but not using AI is simply less productive.
You say that as if tech hasn't always been a moving target anyway. The skills I spent months learning a specific language and IDE became obsolete with the next job and the next paradigm shift. That's been one of the few consistent themes throughout my career. Hours here and there, spread across months and years, just learning whatever was new. Sometimes, like with Linux, it really paid off. Other times, like PHP, it did, and then fizzled out.
--
The other thing is, this need for determinism bewilders me. I mean, I get where it comes from, we want nice, predictable reliable machines. But how deterministic does it need to be? If today, it decides to generate code and the variable is called fileName, and tomorrow it's filePath, as long as it's passing tests, what do I care that it's not totally deterministic and the names of the variables it generates are different? as long as it's consistent with existing code, and it passes tests, whats the importance of it being deterministic to a computer science level of rigor? It reminds me about the travelling salesman problem, or the knapsack problem. Both NP hard, but users don't care about that. They just want the computer to tell them something good enough for them to go on about their day. So if a customer comes up to you and offers you a pile of money to solve either one of those problems, do I laugh in their face, knowing damn well I won't be the one to prove that NP = P, or do I explain to them the situation, and build them software that will do the best it can, with however much compute resources they're willing to pay for?
> but not using AI is simply less productive
Some studies shows the opposite for experienced devs. And it also shows that developers are delusional about said productivity gains: https://metr.org/blog/2025-07-10-early-2025-ai-experienced-o...
If you have a counter-study (for experienced devs, not juniors), I'd be curious to see. My experience also has been that using AI as part of your main way to produce code, is not faster when you factor in everything.
I've heard this mentioned a few times. Here is a summarized version of the abstract:
> ... We conduct a randomized controlled trial (RCT)
> ... AI tools ... affect the productivity of experienced
> open-source developers. 16 developers with moderate AI
> experience complete 246 tasks in mature projects on which they
> have an average of 5 years of prior experience. Each task is
> randomly assigned to allow or disallow usage of early-2025 AI
> tools. ... developers primarily use Cursor Pro ... and
> Claude 3.5/3.7 Sonnet. Before starting tasks, developers forecast that allowing
> AI will reduce completion time by 24%. After completing the
> study, developers estimate that allowing AI reduced completion time by 20%.
> Surprisingly, we find that allowing AI actually increases
> completion time by 19%—AI tooling slowed developers down. This
> slowdown also contradicts predictions from experts in economics
> (39% shorter) and ML (38% shorter). To understand this result,
> we collect and evaluate evidence for 21 properties of our setting
> that a priori could contribute to the observed slowdown effect—for
> example, the size and quality standards of projects, or prior
> developer experience with AI tooling. Although the influence of
> experimental artifacts cannot be entirely ruled out, the robustness
> of the slowdown effect across our analyses suggests it is unlikely
> to primarily be a function of our experimental design.
1. 16 people were randomly given tasks to do
2. They knew the codebase they worked on pretty well
3. They said AI would help them work 24% faster (before starting tasks)
4. They said AI made them ~20% faster (after completion of tasks)
5. ML Experts claim that they think programmers will be ~38% faster
6. Economists say ~39% faster.
7. We measured that people were actually 19% slower
This seems to be done on Cursor, with big models, on codebases people know. There are definitely problems with industry-wide statements like this but I feel like the biggest area AI tools help me is if I'm working on something I know nothing about. For example: I am really bad at web development so CSS / HTML is easier to edit through prompts. I don't have trouble believing that I would be slower trying to make an edit to code that I already know how to make.
Maybe they would see the speedups by allowing the engineer to select when to use the AI assistance and when not to.
it doesnt control for skill using models/experience using models. this looks VERY different at hour 1000 and hour 5000 than hour 100.
Minutes really, despite what the article says you can get 90% of the way there by telling Claude how you want the project documentation structured and just let it do it. Up to you if you really want to tune the last 10% manually, I don't. I have been using basically the same system and when I tell Claude to update docs it doesn't revert to one big Claude.md, it maintains it in a structure like this.
It's a couple of hours right now, then another couple of hours "correcting" the AI when it still goes wrong, another couple of hours tweaking the file again, another couple of hours to update when the model changes, another couple of hours when someone writes a new blog post with another method etc.
There's a huge difference between investing time into a deterministic tool like a text editor or programming language and a moving target like "AI".
The difference between programming in Notepad in a language you don't know and using "AI" will be huge. But the difference between being fluent in a language and having a powerful editor/IDE? Minimal at best. I actually think productivity is worse because it tricks you into wasting time via the "just one more roll" (ie. gambling) mentality. Not to mention you're not building that fluency or toolkit for yourself, making you barely more valuable than the "AI" itself.