Comment by stingraycharles

It’s not delivering on magical stuff. Getting real productivity improvements out of this requires engineering and planning and it needs to be approached as such.

One of the big mistakes I think is that all these tools are over-promising on the “magic” part of it.

It’s not. You need to really learn how to use all these tools effectively. This is not done in days or weeks even, it takes months in the same way becoming proficient in eMacs or vim or a programming language is.

Once you’ve done that, though, it can absolutely enhance productivity. Not 10x, but definitely in the area of 2x. Especially for projects / domains you’re uncomfortable with.

And of course the most important thing is that you need to enjoy all this stuff as well, which I happen to do. I can totally understand the resistance as it’s a shitload of stuff you need to learn, and it may not even be relevant anymore next year.

tsimionescu 2 days ago

While I believe you're probably right that getting any productivity gains from these tools requires an investment, I think calling the process "engineering" is really stretching the meaning of the word. It's really closer to ritual magic than any solid engineering practices at this point. People have guesses and practices that may or may not actually work for them (since measuring productivity increases is difficult if not impossible), and they teach others their magic formulas for controlling the demon.

Reply View 6 replies

gtaylor 2 days ago

Most countries don’t have a notion of a formally licensed software engineer, anyway. Arguing what is and is not engineering is not useful.

Reply View | 3 replies
- tsimionescu 2 days ago
  
  Most countries don't have a notion of a formally licenses physicist either. That doesn't make it right to call astrology physics. And all of the practices around using LLM agents for coding are a lot closer to astrology than they are to astronomy.
  I was replying to someone who claimed that getting real productivity gains from this tool requires engineering and needs to be approached as such. It also compared learning to use LLM agents to learning to code in emacs or vim, or learning a programming language - things which are nothing alike to learning to control an inherently stochastic tool that can't even be understood using any of our regular scientific methods.
  
  Reply View | 0 replies
- llbeansandrice 2 days ago
  
  I think it's relevant when people keep using terms like "prompt engineering" to try and beef up this charade of md files that don't even seem to work consistently.
  This is a far far cry from even writing yaml for Github/Gitlab CICD pipelines. Folks keep trying to say "engineering" when every AI thread like this seems to push me more towards "snake oil" as an appropriate term.
  
  Reply View | 1 reply
  
  stingraycharles 2 days ago
  
  Prompt engineering is a real thing though, but it’s not related to markdown files etc.
  If you’re not benchmarking and systematically measuring the impact of your changes, it’s not prompt engineering, it’s just improving stuff.
  
  Reply View | 0 replies
stingraycharles 2 days ago

Well it requires a lot of planning, spec’ing, and validation.
The whole point is to get a process around it that works and gets the “ritual magic” out of it.
I labeled that as engineering.

Reply View | 0 replies
bdangubic a day ago

everything you wrote is true (and much worse) when humans are "engineering" shit

Reply View | 0 replies

[removed] a day ago

[deleted]

Reply View 0 replies

llbeansandrice 2 days ago

My issue is not with learning. This "tool" has an incredibly shallow learning curve. My issue is that I'm having to make way for these "tools" that everyone says vastly increases productivity but seems to just churn out tech-debt as quickly as it can write it.

It a large leap to "requires engineering and planning" when no one even in this thread can seem to agree on the behavior of any of these "tools". Some comments tell anecdotes of not getting the agents to listen until the context of the whole world is laid out in these md files. Others say the only way is to keep the context tight and focused, going so far as to have written _yet more tools_ to remove and re-add code comments so they don't "poison" the context.

I am slightly straw-manning, but the tone in this thread has already shifted from a few months ago where these "tools" were going to immediately give huge productivity gains but now you're telling me they need 1) their own special files everywhere (again, this isn't even agreed on) and 2) "engineering and planning...not done in days or weeks even

The entire economy is propped up on this tech right now and no one can even agree on whether it's effective or how to use it properly? Not to mention the untold damage it is doing to learning outcomes.

Reply View 0 replies

fpauser 2 days ago

>> [..] and it may not even be relevant anymore next year.

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Reply View 0 replies

giancarlostoro 2 days ago

Yeah I feel like on average I still spend a similar amount of time developing but drastically less time fixing obscure bugs, because once it codes the feature and I describe the bugs it fixed them, the rest of my times spent testing and reviewing code.

Reply View 0 replies

nineteen999 2 days ago

Learning how to equip a local LLM with tools it can use to interact with to extend its capabilities has been a lot of fun for me and is a great educational experience for anyone who is interested. Just another tool for the toolchest.

Reply View 0 replies