Comment by ern

Comment by ern 2 days ago

3 replies

Even then, most of these coding tools aren't even close to automating meaningful coding tasks in practice, and while that doesn't mean they can't in the future, it's a long ways off!

Not related to your main point, but I've introduced Github Copilot to my teams, and, surprisingly, two of our strongest developers reached out to me independently, and told me it's been a huge boost to their productivity, one in refactoring legacy code, and another in writing some non-trivial components. I thought the primary use would be as a crutch for less capable developers, so I was surprised by this.

As a middle-manager whose day job previously robbed me of the opportunity to write code, I've used ChatGPT 4o to write complex log queries on legacy systems that would have been nearly impossible for me to do otherwise (and would have taken a lot of effort from my teams) and to turn out small (but meaningful) tasks, including learning Android dev from scratch to unblock another group and other worthwhile things that keep my team from being distracted and able to deliver.

I guess there's a "no true Scotsman" fallacy hiding there, about what constitutes "meaningful coding tasks in practice", but to me, investing in these tools has been money well spent.

phillipcarter 2 days ago

Oh, I completely agree with using tools like this. For example, the latest models are very good at being passed a description of a problem and its inputs, expected outputs, and some sample test cases, and then generating a very diverse set of additional cases that likely account for some edge cases you might have missed. Hugely productive for things like that.

However, these same coding assistants lack so much! For example, I can't have a CSV co-located in my directory as a jupyter notebook file, then start prompting+coding without having also done a call to df.head to get those results burned into the notebook file. The CSV is sitting right there! These tools should be able to detect that kind of context, but they can't right now. That's the sort of thing I mean when we have a long way to go.

  • salomonk_mur 2 days ago

    But still, a huge productivity boost. I think we can say that as of the lastest models, AI pairs are pretty great and save a ton of time.

fumeux_fume 2 days ago

My experience with Copilot has been the opposite of your devs'. It frequently screws up routine tasks, has poor UI and writes buggy code I would expect from someone who was brand new to programming. Sometimes it'll chance on a nice solution without requiring modifications, but not enough to fool me into any glowing reviews! I think I have higher standards than most though.