Comment by quotemstr
Great job! If you tug on this thread long and hard enough, you develop this enough and you get RDP (which you can try via xrdp, GNOME's remoting thing, etc.).
The reason the terminal ecosystem doesn't get much more sophisticated over time isn't just the herd-of-cats fragmentation, but also evaporative cooling: people who do really cool things with terminal come to realize that what they really want is remote desktop (perhaps rootless) and leave terminal stuff as-is while they invest in more sophisticated systems instead.