My X Toolkit Plan

For a long time, I've been threatening to "work up the stack" from XCB. Indeed, this is behind many XCB design decisions. I want to eventually build apps on top of a framework designed bottom-up in a flexible manner.

This blog post describes my medium terms in this direction. By the way, this is a pretty different plan from what it was two months ago. This may mean that it's also different from what I'll eventually actually do. My mileage may vary…

Premise: Current toolkits (meaning practically everything) have lousy solutions to several important problems. As an example, let's imagine I'm going to build a chess program with an integrated GUI. Problems:

1. The chess engine wants to do its search without paying attention to the rest of the program, except when it has calculated some kind of update. In a single-threaded app, this means that the user may be locked out.
2. The user inputs are serialized by the window system into an event stream. In most toolkits, all other kinds of application inputs are serialized as well. This means that activities can't easily be performed out-of-order with the event stream, or concurrently, even when it would simplify or improve the application to do so.
3. Toolkits are typically set up to support a modularity module in which individual windows or drawing areas are mapped 1::1 to individual objects or structures, and in which handling I/O is mapped onto these structures. This makes it hard to support metaphors other than the standard desktop metaphor, and hard to get emergent behavior.
4. Toolkits typically make it hard to get robust, high-quality layout on the screen. Indeed, IMHO Keithp's and my Kgames Xt layout widget from the 80s is easier to use and works better than any of the standard solutions around today.
5. Toolkits typically make it hard to support direct manipulation; yet direct manipulation is arguably the easiest, most powerful GUI style possible. It is embarassing that the user interfaces for word processing and vector drawing today are, in my humble opinion, uniformly so much worse than those of MacWrite and MacDraw for the original Mac, but there we are.

So what's my solution? Four phases—you've seen the first two already.

1. Get the Xserver and client libraries to support reasonable rendering and typography. Done by Keithp and friends: see Xft/fontconfig, Render, Pango and Cairo.
2. Replace Xlib with a protocol binding that offers sensible direct protocol support, thread transparency, and latency hiding. Done: see XCB.
3. Build an X toolkit that supports a reasonably wide range of GUI structures robustly and simply. Current item, addressing 1-3 above.
4. uild X GUI libraries atop this toolkit to support a direct-manipulation non-desktop-metaphor graphical environment (while retaining compatibility with current X apps). Future work addressing 4-5 above.

Specifically, phase III involves trying new architectures for X application structuring. My first candidate is based on these observations:

1. Model-View-Controller is *almost* right. Certainly, the Model idea seems to be a good one. In our chess program, the engine would be responsible for maintaining the model. But in practice, MVC tends to degenerate into Model + ViewControllerThingy, and the communication between them tends to be a mess. We need some way of coupling the engine to GUI input and output that is easy to use and robust.
2. Concurrency and dynamic modularity means threads, but programmers have a hard time handling standard thread programming; debugging is hard, and it's easy to introduce races, deadlocks and livelocks and hard to detect them. If GUI apps are to be threaded, the thread processing and communication model must be so simple that it's easier than programming single-threaded event-driven applications. Fortunately, in my experience this isn't a very high bar.

OK, enough blather. Here's my proposed toolkit architecture, version 0.

     event thread ---> |            | -> drawing thread 
	               | blackboard | -> other output
     one thread per -> |            |    threads
      other input      |            |
                         ----------
                           |   ^
	                   v   |
			 application
			   "engine"

It is important that all threads are completely lockless, block waiting for inputs ("pull model"), and communicate only via channels and the blackboard.

What's a "blackboard"? It's an architectural data structure idea that has been around, especially in AI circles, for 40 years, and IMHO has been grossly underutilized. Basically, a blackboard is a large data structure segmented into components. Applications can block waiting to be triggered by combinations of data appearing on the blackboard. They can also poll the blackboard, and post data to the blackboard. Thus, the blackboard fills simultaneous roles of synchronization/rendezvous of threads and data sharing.

(Mental exercise—60 seconds: What blackboard has been a stock part of the X Window System since the beginning, and is used much as above? [Note that this does not mean that I'm planning to use the existing thingy as a blackboard; we'll build a custom blackboard from scratch.])

The theory is that with blackboard and library magic, I can provide a library that makes the threads essentially invisible (as e.g. Java AWT/Swing does), but still prevents or detects common threading bugs. 1 above is taken care of trivially. 2 can be handled by not serializing or even by de-serializing where appropriate, and by getting rid of the locks, condition variables, shared structures etc that make thread programming hard. Debugging becomes much more fun, because you can stop one or more threads at any point and examine the blackboard to figure out what's going on. 3 is handled by the fact that there's no necessary or real correspondence between threads or objects in the program and objects on the screen; this allows much more flexibility in tackling 4 and 5 in Phase IV.

At least this is the current theory. It awaits time and/or volunteers to try building it out. Any comments are greatly appreciated. (B)