Comment by langcss
You need a lot more power. I found gpt4o struggles doing basic OCR of printed text by hallucinating alot, while tesseract engine (old skool) gets it perfect. You need the model to be powerful enough to do everything.
You can work around this by the way by sending the output through a checking stage.
So picture -> gpt4o -> out1, picture -> tesseract -> out2, out1,out2 -> llm.
Might work for sound too.
Interesting, I've actually been using gpt4o extensively for OCR and didn't encounter any significant issues - could I ask you to please give an example of an image of (otherwise legible) text that it hallucinates on?