Comment by James_K
It's still pretty simple. I think you are really underestimating the ability of humans to do rote activities. Especially since we could give the human a pen and paper in this case and let them write stuff down on it to give parity to the AI which can write words in its output.
It seems pretty clear to me that learning how to go about following an arbitrary set of rules is a part of general intelligence. There are lots of humans who have learned this skill, and many (mostly children) who have not. If the AI has failed to learn this ability during its extensive training, and critically if it cannot be taught this ability as a human could, then it's certainly not "generally intelligent" to anywhere near the human degree.
You have to understand these people are fundamentally misanthropic, and think everyone but them is an idiot. That's why they're so impressed by LLMs even when they fail miserably.