Comment by onlyrealcuzzo a day ago
> 58% success rate on a task is close to a coin flip.
Why does a single-step task imply a coinflip to you?
There are more than two possible choices for an instruction like: "Lookup the status of order X".
You don't have a 50% chance of being right rolling an N-sided weighted die.
No, if there are 100 order statuses there are 99 wrong choices and 1 right choice.
Additionally, the distribution of the choices is not guaranteed to be equal.
If you assume equal distribution, you have a 1% chance of being right and a 99% chance of being wrong.
50% chance of being right is equivalent to a coin-flip.