Comment by bwfan123 a day ago
Finally some real pushback to the whole agentic mania - from an actor who is incentivized to push the narrative. Following the recent apple paper - some realism is being injected into the hype.
58% success rate on a task is close to a coin flip. and 35% success rate on multiturn. >80% success rate on workflows could make that a reasonable usecase (eg, form filling) with some human supervision.
It might depend how much this employee costs.
Your incentive to fire an employee who isn't great and costs $1 per day is much less than an incentive to fire one who isn't great and costs $1000 per day...
What is their incentive to share this data? I’m not really understanding
They’ve leaned so hard into AI and agentforce that it doesn’t make sense to shoot themselves in the foot.
Except that Hubspot, their main competitor on the SMB/MM/startup side recently announced a deep integration with ChatGPT. Still seems like a shot in the foot in an effort to undercut a growing competitor in a part of the market that theyd be better off exiting.
> 58% success rate on a task is close to a coin flip.
Why does a single-step task imply a coinflip to you?
There are more than two possible choices for an instruction like: "Lookup the status of order X".
If it were an employee it would have been fired already, unless it were a nepo hire, and in someways, it is.