Comment by theptip

Lots has been written on this subject, check out OpenAI’s papers on -O for example.

Latency is a big one due to batching. You can’t really interrupt the agent, which makes actual conversation more clunky. And yes, multimodal has better understanding. (I haven’t seen analysis of perception of emotions, has anyone seen analysis of this capability for GPT-O?)