Comment by terribleperson

A real, actual doctor told my brother, who has a chronic headache disorder, to just keep taking OTC painkillers.

You very specifically should not do that; you'll develop a medication overuse headache and be worse off than you were.

It gets worse, though. I was able to ask them a few questions about their symptoms, compare them to entries in the International Classification of Headache Disorders, and narrow it down to, iirc, two likely possibilities.

One of them was treatable. The treatment works. They still have pain, but can do stuff.

An AI that makes stuff up and gets stuff wrong isn't any different from the doctors we already have, except you can afford to get a second opinion, and you have the time available to push back and ask questions.

Edit: to expound on quality of the doctor - diagnosis and proposing a treatment was the work of several hours for me, a layman. A doctor should have known the ICHD existed. They should have been able to, in several minutes, ask questions about symptoms, reference the ICHD to narrow down likely diagnoses, and then propose a treatment with a "come back if that doesn't help".