X.org considers hardware acceleration, with XRender, for example. Quite too often, buggy and incomplete graphic drivers and other problems makes it even worse than no acceleration at all. I don't really see these problems as X.org specific, is there any evidence that they are? I think it's more likely caused by very limited resources that are available, incomplete or proprietary (ati and nvidia tends to lag months behind new X server releases, would they ever catch up with a completely new system?) drivers, and other problems, and throwing out X means throwing out all effort spent to solve this problems.
But that doesn't mean that it is not a good idea to explore other possibilities and experiment with another systems in parallel. Like Wayland, for example. Of course it is. Just, don't hurry with burying X11, it's the best what you've got, and likely to remain so for a long time.
You kind of have to bury X, otherwise you're going to end up with two competing projects and then you're even worse off. See: Linux audio.
If X were to disappear from the face of the earth open source programmers would scramble to get something new into place, and that would in all likelihood be better at dealing the realities of graphics today.
Just because it's the best we got doesn't mean it's good enough.
(I'm not advocating we actually should bury X... just making the general point that sometimes it's worth it to stake a step back to take two steps forward later. I realize it's often not politically feasible.)
Well, I was hoping for many more than just two. I don't really see any other way than many competing projects, one eventually killing all others.
But it shouldn't be that painful. As you've said, it's mostly about toolkits, and it's easy to simply run X server atop of the new system (Mac OS X does that, Linux does that in some ways, Windows can do that, so anybody can). I guess that the terrible mess of Linux audio can be avoided (after all, you don't have X11 applications locking whole graphic system for just themselves, as is often the case in audio).
But that doesn't mean that it is not a good idea to explore other possibilities and experiment with another systems in parallel. Like Wayland, for example. Of course it is. Just, don't hurry with burying X11, it's the best what you've got, and likely to remain so for a long time.