I think it will always feel unnatural as long as 'AI Speech' is turn based. Right now developers used Voice Activity Detection to detect when the user has stopped talking.
What would be REALLY cool is if we had something that would interrupt you during conversation like talking with a real human.
I absolutely agree with your analysis wrt current tech - however, I suspect the person you're replying to is talking about "what would be really cool" in terms of it happening in a future where the relevant underpinnings had advanced to the point where it could actually manage the situational/nuance stuff properly.
I almost certainly wouldn't want to use something that tried to implement it now but it's a lovely dream and the state of the art keeps advancing at quite the speed (i.e. faster than I would have predicted, even when I do my best to take into account that it keeps advancing faster than I would have predicted ;).
I can see how interruptions would prove even more unnatural and annoying pretty quick. There's a lot of nuance in knowing how to interrupt properly and often, people that interrupt only do so quickly, then yield, allow person to finish then resume - very situational and tons of nuance. Otherwise, with current level of sophistication, you'd just have the AI talking over you the entire time, not allowing you to complete your thoughts/questions/commands/etc and people would quickly be more frustrated and just turn it off.