Comment by imoreno

The most interesting point in this is that people don't/can't fully utilize LLMs. Not exposing the system prompt is a great example. Totally spot on.

However the example (garry email) is terrible. If the email is so short, why are you even using a tool? This is like writing a selenium script to click on the article and scroll it, instead of... Just scrolling it? You're supposed to automate the hard stuff, where there's a pay off. AI can't do grade school math well, who cares? Use a calculator. AI is for things where 70% accuracy is great because without AI you have 0%. Grade school math, your brain has 80% accuracy and calculator has 100%, why are you going to the AI? And no, "if it can't even do basic math..." is not a logically sound argument. It's not what it's built for, of course it won't work well. What's next? "How can trains be good at shipping, I tried to carry my dresser to the other room with it and the train wouldn't even fit in my house, not to mention having to lay track in my hallway - terrible!"

Also the conclusion misses the point. It's not that AI is some paradigm shift and businesses can't cope. It's just that giving customers/users minimal control has been the dominant principle for ages. Why did Google kill the special syntax for search? Why don't they even document the current vastly simpler syntax? Why don't they let you choose what bubble profile to use instead of pushing one on you? Why do they change to a new, crappy UI and don't let you keep using the old one? Same thing here, AI is not special. The author is clearly a power user, such users are niche and their only hope is to find a niche "hacker" community that has what they need. The majority of users are not power users, do not value power user features, in fact the power user features intimidate them so they're a negative. Naturally the business that wants to capture the most users will focus on those.