Seth Nickell

Since he caught a glimpse of Kristian’s wobbly windows, Bryan has stalked Red Hat’s dark and hallowed halls, breathing fire, demanding his chance in the directorial seat. So it is that we bring you Monkey Hoot productions first, uh, production. Since a lot of people have asked, these videos show Luminocity running on two different laptops, both with fairly slow/old video cards (Intel i830 and ATI Radeon 7500 mobility) and open source drivers.

Luminocity

Theora | MPEG4

Kristian showing off his spring-modeled “wobbly windows” effect in Luminocity,Owen’s crack-tastic OpenGL based window/compositing manager. This is the only effect that requires GL hardware acceleration in Luminocity (and not even much at that, Kristian’s development machine uses an embedded Intel video card). Notice that menus and tooltips are also animated as they pop on and off the screen. The animation effects on window impulses are implementable in a modular manner, allowing anyone to write new effects. Monkey Hoot productions would like to thank “The Blair Witch Project” for its inspirational camera work and lighting, and apologize to our viewers.

Physics Models for Window Moving

Theora | MPEG4 | MJPEG

The wobbly window effect is mildly addictive. Kristian hasn’t gotten much work done since he wrote it. He (and now I) spends all day moving windows around and watching them settle. This video shows off the motion a little better. It also demonstrates Luminocity’s live workspace switcher (aka pager) which updates in synch with the screen. We were surprised by how much more tangible windows felt when they gave a little (i.e. less than in this video) as you moved them (like a real world object). Of course, we turned the effect on “high” for this demo so it’d be very visible.

Live Updating Workspace Switcher

Theora | MPEG4 | MJPEG

The workspace switcher in luminocity is updated in-synch with the window contents. Also notice that the workspace switcher renders each window rather than just “capturing” what each workspace looks like (this can be seen in the absence of a background in the pager), allowing us to do nice UI tricks in the future. Since its just re-using the existing window textures, applying them to a new (smaller) surface, the workspace switcher has basically no performance overhead when using hardware accel (other than a few new surfaces for your graphics card to render, no biggie for the card).
Its a little hard to see in this video, but Luminocity also has a nifty workspace switching animation. It zooms out as it pans down to the next workspace and then zooms back in. Of course, since its also the compositing manager, any on screen action doesn’t freeze as you switch workplaces. Watch for this as we switch into the 3rd workspace containing an animated circle-o-icons, the icons keep spinning as you switch.

Movies Still Play as the Window is Warped

Theora | MJPEG

A GStreamer movie pipeline rendering into Luminocity. Notice that its warping the movie as it plays without slowdown (and of course, updating the workspace switcher live, which is just re-using the same GL texture rendered onto a smaller surface).

OpenGL Accelerated Alpha Compositing

Theora | MPEG4 | MJPEG
> 

> 

JPEG

Luminocity uses GL for hardware accelerated alpha compositing. It works well with software GL implementations too. Of course, since Luminocity is a technology testbed, we use it for “unfocused windows” here, probably not a very good long term use ;-). In one of his earlier demos Owen hijacked the mouse scroll wheel to control window transperancy. Bad Owen! This video also has another nice demonstration of wobbly menus. They feel really nice, though they’ll probably need to be faster in a “real world” version. The screenshot shows fdclock rendering in Luminocity. fdclock, unlike the video, actually uses a 32-bit ARGB visual to specify where (and how much) transparency it wants. No videos because our camera man is tired (you can run it yourself with “fdclock -ts”).

Border/Contents Resize Synchronization

When you resize a window inside Luminocity it doesn’t redraw the borders until the application is done redrawing the window contexnts. This means that they feel like “one piece” instead of the staggered redraw you see on traditional window managers (where the border gets ahead of the contents, and then they catch up). The effect on the perceived “reality” of windows on the screen is excellent, i.e. windows feel more like solid real objects (the same sort of improvement as double-buffering of widgets gave). It also, ironically, makes window resizing feel smoother (though each redraw is slightly slower and not progressive).

Cairo

While not as sexy as the Luminocity videos, here (finally) are screenshots of GTK+ themes rendering with Cairo enhancements. Cairo both increases the rendering quality of GTK+ widgets, and allows for widgets that scale beautifully to different sizes (of course, we also have a Cairo driven SVG renderer, knock yourself out). When you get your 600 dpi monitor, we’ll be there

Dynamic Themes - each widget unique

In my last X rendering post I discussed dynamic theme rendering, where every time a widget is rendered it looks slightly different. By writing algorithmic renders rather than fixed pixbuf based widgets, we can increase how dramatic the visual effects are without driving people nuts. For example, the tiger stripe buttons have proved very reasonable for long term use. However, any single rendering of a tiger stripe button would get old very quickly when repeated all over the screen ad nauseum. Currently visual designers are extremely restricted in what they can do without a theme being unusable. That’s largely the reason all themes look basically the same. We hope dynamic themes will allow visual designers to increase the variety of their palette without producing themes that wear quickly. Of course, its still easy to go overboard, *grin*. By providing higher level drawing primitives, Cairo makes it much easier to implement dynamic themes.

Resolution Independent Rendering

A large checkbox rendered with Cairo. This would look fantastic as a checkbox “normal size” on a next-gen 600dpi display.

Cairo makes it easy to draw well rendered custom-widgets. Here’s an example of how the GTK+ color picker looked before and after Cairo integration.

Getting Luminocity

It took me about half an hour of work (and some compiling time) to get Luminocity running using jhbuild. Eventually we’ll add a jhbuild target for compiling Luminocity. Luminocity is not intended to turn into a real world window/compositing manager. Instead, its a technology test bed. We’re trying stuff out in Luminocity and will be rolling them into Metacity (and hence stock GNOME) as they mature. Don’t expect Luminocity to have the frills and smarts you’d expect from a normal window manager. You’ll need hardware GL acceleration enabled to have wobbly windows work, though you can try the other bits of luminocity without it. Emebedded Intel video cards (which have open source DRI drivers) will work just fine. ATI and NVidia cards, of course, work even better.

This section has been superceded by the Luminocity wiki page which has simpler more up to date build instructions

1. If you have not used jhbuild, get jhbuild from Gnome CVS module ‘jhbuild’. Then run jhbuild bootstrap to compile basic tools such as autoconf and automake (just agree with its defaults).
2. Run jhbuild build xserver Xcomposite Xdamage Xrender Xext Xcursor X11 Xtst. This will build the freedesktop.org xserver, including the damage and composite extensions, and the Xephyr/Xfake nested X servers.
3. Apply a small patch that evilly hacks around some issues with DAMAGE in the X server
4. Checkout module “luminocity” from Gnome CVS.
5. With the jhbuild buildroot at the start of your PATH (so you get autoconf, automake, etc from the buildroot): from the luminocity directory run ./autogen.sh –prefix=PATH_TO_JHBUILD_TREE, then make and finally make install. Alternatively, see README.jhbuild for instructions on adding a “luminocity” target to jhbuild (eventually we should just include this in jhbuild).
6. Now to get things running. Luminocity grabs windows from an existing X server and renders them in its own GL context. This technique is not intended to be particularly efficient, but it works surprisingly well for a development testbed. We will use “Xfake” as the X server. Xfake doesn’t display windows sent to it, so they only get rendered on screen once (by luminocity). Xfake is included in the “xserver” module built by jhbuild above. If you are running at 1024×768, run jhbuild run Xfake -ac -screen 1024×3072x32 :1 to start Xfake on display :1. Basically, use XRESULTIONx(4*YRESOLUTION)x32. This is because Luminocity starts with 4 workspaces by default.
7. Now lets display something on the Xfake display. From a new window, set DISPLAY to “:1″ (e.g. export DISPLAY=:1). Then run any program you want to use, e.g. gnome-terminal. Of course, you won’t see a window since its displaying to the fake X server.
8. Start Luminocity with luminocity -f :1 PATH_TO_BACKGROUND_IMAGE. Including a background image is important since a bug in the wobbly windows rendering code causes major performance problems when the background is missing. Luminocity should now be running fullscreen, display whatever application you launched earlier (in this example, gnome-terminal). Another bug in wobbly windows increases the animation timeout every time you open a new window. This means that for every window you open, wobbly windows get jerkier and jerkier. Oops! Don’t worry, this is a silly bug and not a sign that we’re overloading your card or something.
9. Sometimes windows start with their titlebars off screen. To move them onto the screen, you’ll need to drag them while holding down the super key (if you have a Windows key on your keyboard, try this). You may have to remap your super key to make this work, esp. if you have no Windows key, e.g. xmodmap -e ‘keycode 95=Super_L’, which will then allow you to move windows by dragging them while holding down the F11 key.

If you need help, or you’re interested in contributing to Luminocity etc, you can probably find some knowledgable people on #fedora-desktop on irc.gnome.org (naturally, you don’t have to be running fedora Eventually we’ll probably have a channel for this. People to look out for are: “owen”, “ssp” (Soeren Sandmann), “krh” (Kristian Ho/gsberg) and “seth” (Though I’m just a user, *grin*).

Update: I wrote a little more explaining how Luminocity relates to xcompmgr/metacity/Xgl in another blog entry

Update: People have been asking what sort of hardware this was done on. Videos were shot on a mix of an IBM thinkpad X30 (with a paltry Intel i830 video card using open source drivers) and an IBM thinkpad T41 (with a slightly beefier but still pretty old Radeon Mobility 7500, also using open source drivers). Everything we’re doing so far is light on hardware requirements. FYI, a locking bug was introduced in Luminocity that causes wobbly windows to get increasingly jerky as more windows are opened (or if there’s no background image present, go figure!). This is not related to its CPU or graphics card use, and should be easy to fix without major codebase changes.

Update: If you’re having build problems, I’ve updated the “jhbuild” line to include more luminocity dependencies than just xserver. Also added a note about “jhbuild bootstrap” for building the initial dev environment (auto*, etc).

Update: Build section now superceded by the Luminocity Wiki page

Just created a wiki page for Luminocity with improved build instructions. Should be a lot easier now, esp. thanks to all the people who have reported problems and found solutions on #fedora-desktop. Its basically “jhbuild build xserver luminocity” at this point, except that a patch has to be applied to xserver first.

Relation to Metacity

When it has proved itself, Luminocity’s compositing manager will probably be moved into Metacity (along with any effects / extra features we consider good and stable). We originally considered doing the work in Metacity itself, but didn’t want to destabilize it until various approaches were tested. Luminocity is, effectively, a testbed for Metacity. It provides a smaller/simpler codebase to test interesting rendering code with, and means we don’t have to worry about fucking up Metacity in the process. Soeren’s computer is (as of tonight, at least, that’s the first I saw of it) running a version of Metacity that’s apparently using the compositing manager code from Luminocity to render to a GL context.

Relation to xcompmgr

Luminocity has an internal compositing manager that performs the same function as xcompmgr. The compositing manager / window manager integration allows Luminocity to do things that an individual compositing manager or window manager couldn’t. Of course, Luminocity composites using OpenGL, unlike xcompmgr. This apparently can be an upside and a downside, but I don’t know anything about it so I’ll shut my trap.

Relation to Xgl

This is the complicated one :-). I’m loathe to stick my toes in these waters because I’m the wrong person to do it, but I’m also afraid that we’re going to end up with two rendering infrastructures down the road and no clarity for application developers as to which (if either) they can use. I don’t know if that’s where we’re headed, I hope not, but I have this vague (probably, hopefully unfounded) fear… The effect will be slow adoption and general suck. I should premise this by saying that I have no direct knowledge of the Xgl codebase. I have knowledgable sources, and I know what Xgl generally is, but I haven’t personally used Xgl, let alone looked at its codebase (I’ve barely looked at the Luminocity codebase either, for that matter).

Xgl is an X server implementation that, rather than directly accessing chip specific hardware drivers, does its low-level drawing using OpenGL calls. That means Xgl is functionally equivalent to a traditional X server, it just uses a different rendering path. Put another way, Xgl is to X11 as Glitz is to Cairo: it provides the same APIs rendered in a much smarter way.

Luminocity, on the other hand, is a compositing manager / window manager fusion that composites using OpenGL. Compositing and Window managing are all about what you do with client-rendered windows. Luminocity doesn’t know what’s inside windows, and it doesn’t care. Xgl, on the other hand, I would characterize as primarily being about how the contents of windows are drawn (in this case: quickly and with less CPU load, *grin*). Xgl can do some other non-inside-window things like drop shadows, but I’m going to argue later those are mostly expedient demos of cool technology and Xgl is probably not the place we want to be doing those things long term. From the perspective that Luminocity is mostly about rendering windows and Xgl is mostly about rendering window contents, they are theoretically complimentary. At the moment, they can not be used in conjuction with one another (since they both want to directly drive the GL hardware), but they’re goals are at least compatible.

Neither Xgl nor Luminocity are complete on their own. Xgl provides an X server and requires a window manager (and a compositing manager?) (and an X server for doing GL calls into, but see below, that will hopefully cease to be an issue eventually). Luminocity provides a window manager and a compositing manager but requires an X server (currently using Xfake or Xephyr, though supposedly there’s some plan for modifying the core fd.o X server so Luminocity will work using only the host X server?). With some hand waving (in particular there’s no way to hand OpenGL textures residing in the video card between processes), perhaps we could get Xgl to render windows into textures on the video card, and then use Luminocity to figure out what do with those textures. All graphics computations are done by the card, and data flows only once to the card. Perfect! Other than those niggly make-or-break technical details

As far as I know (and I’m pretty sure of this), there is no systematic way (besides GLX inside a running XFree86 / fdo.o X server) to access the “hardware accelerated OpenGL drivers”. That means that Xgl and Luminocity are currently forced to have a traditional host X server, open a fullscreen window on the host server and draw into it using OpenGL. Both Luminocity and Xgl are premised on OpenGL as the standard API through which vendors can provide graphics hardware acceleration (as opposed to, say, RENDER).

Update: Soeren, one of our X hackers, thinks that Xgl actually includes no cross-window stuff but just uses an existing compositing manager (and of course, accelerates its rendering). In that case, the next couple paragraphs are totally unnecessary. Like I said above, I don’t know anything about the Xgl codebase.

I would not take something I say here as authoritative! My knowledge of this stuff only scratches the surface. But many people have been saying even less informed things, so I wanted to get slightly more accurate info out there (esp. on online forum comments). Enjoy