You and Me and The War at The End Times

the actor tree is only the representation of what your user interface is composed of: textures, reactive items, containers to give them structure. if nothing is painted, though, it’s not much of a user interface.

with the current stable version of Clutter, if you want to get something on the screen you either use one of the pre-canned sub-classes of ClutterActor provided by Clutter itself — ClutterTexture, ClutterRectangle, and ClutterCairoTexture — or you end up using something written outside of Clutter — Mx widgets, for instance; or you write your own UI elements. if you chose to write something from scratch, you will have to sub-class ClutterActor and override the ClutterActor::paint virtual function. ((yes, it’s a signal, which means that you can also attach a handler to an existing actor and completely change how said actor paints itself; this is incredibly evil, and will not be possible in 2.0, so if you’re doing that, stop right now, or I will tell mom, and you won’t be invited to the Buffy marathon next time.))

what happens inside a paint virtual function implementation is usually this:

  • you use Cogl API to set up the state of the graphics pipeline;
  • you use Cogl API to submit geometry to the graphics pipeline;
  • you recursively call clutter_actor_paint() on your children, if any.

the third step can now be taken care of Clutter itself, as any actor now has access to the section of the actor tree with itself as the local root. the first and second steps are kind of tricky: you have to remember a lot of things about how to use the Cogl API. it would be nice if Clutter provided some convenience API to Do The Right Thing™ instead.

it would also be nicer if we could structure that API to build a representation of an entire frame, decoupling what gets painted from what is the logical unit of the user interface, because then we could keep it (either as parts or the entire thing) around between frames, manipulating the former without affecting the latter. the keen readers will already have said: you want a retained drawing model. good job! ten points for Gryffindor!

this new API is provided by paint nodes; each paint node is an element on the render tree — and every time you should paint an actor, you get passed a local root that you can use to attach your nodes. each paint node contains the pipeline state and the geometry to submit. once the tree has been built, Clutter will decide how to paint it. the base entry point for this operation is the ClutterActor::paint_node() virtual function. inside it, you’re supposed to paint only the actor — the children will be painted by Clutter itself. the paint_node() virtual should be considered orthogonal to the paint() one, though the latter will probably remain in 2.0 to give you control over which of your children you may decide to paint. Clutter provides predefined paint nodes for texture, solid colors, text, and even for custom pipelines; to each node you can add a rectangle (with or without texture coordinates), paths, or even Cogl primitive objects if you decide to use the experimental Cogl 2.0 API.

currently, given that we allow app and toolkit developers to use a direct drawing model, we still need to maintain that invariant, otherwise stuff will break; but we can already start migrating towards a retained drawing model, and have it fully in place by the time we hit 2.0. instead of setting up all the transformations and modifying the modelview and clip states, we’ll be able to attach transformation and clip nodes, and even be able to discard sections of the render tree without hitting the GPU. from there, we could go anywhere; we could hand off rendering into a separate thread and never block the main loop; or we could do the reverse, and thread everything else instead — event processing, layouting — without having application code left to deal with the fact that only one thread at a time can access a GL context.

obviously, most developers will not want to care about this; apps usually draw textures and text anyway, fancy as they may be. app developers should get a better API to achieve that, though, and the current model of sub-classes of ClutterActor is not serving them well enough; if you sub-class ClutterTexture you get a ton of behaviours, and most of them are really not at all nice, especially if all you want was to get some image data on the screen. I won’t even speak about ClutterCairoTexture, which encodes an implementation detail (that is not even true nowadays) into the class structure.

so, for 1.10 I introduced the ClutterContent interface, and a couple of trivial implementations for displaying image data and for drawing using Cairo. the classes are side-effect-free because they only know about one thing: painting some state. implementing a ClutterContent to paint something custom is easier than sub-classing ClutterActor, just like it’s easier to implement a ClutterLayoutManager to manage the layout policy of the children of an actor. all the cleverness on where and when to paint the content associated to an actor is where it belongs: inside the actor itself, leaving you to care just about what to paint.

Had a dream

as I wrote already in a previous blog post, if you try and compile an existing Clutter application with the current master, the first thing you will notice is that the compiler will start complaining about deprecated symbols.

a lot.

some of these symbols are little used API variants, or baggage we’ve been stringing along since the 0.x days and that we never managed to deprecate until now — and if you’ve been using them, then I’m not sure I want to know why, and you’ll have to trust me when I say that you’re better off not using them.

on the other hand, many of the deprecated symbols have been pretty central as to how a Clutter app is built and works; clutter_container_add_actor()? clutter_actor_raise()? ClutterBox? clutter_timeline_set_loop()?

in reality, not much has changed per se: the deprecation warnings will tell you how to port an app from the old API to the new one, and in basically all cases it’s juts a matter of changing functions or classes; plus, deprecated classes and functions will still work until we break API and release Clutter 2.0.

what really changed is how a Clutter app should be written from the get go — and this has more to do with the intent of the API, more than the actual form of the API.

case in point: if you look at the replacement for many of the deprecated functionality, as well as the new API introduced in 1.10, you will notice that it is overwhelmingly inside the ClutterActor class; the 1.9 development cycle stats for the clutter-actor.[ch] files alone are:

 git diff --stat clutter-1.8... clutter/clutter-actor.[ch]
 clutter/clutter-actor.c |20516 +++++++++++++++++++++++++++++------------------
 clutter/clutter-actor.h |  803 +-
 2 files changed, 13350 insertions(+), 7969 deletions(-)

so ClutterActor just gained around 5000 lines ((yes, I’m cheating a little bit, as a lot of documentation is inlined inside the source code, so that balloons the size of the delta)).

this growth reflects one of the fundamental tenets of how Clutter should be used:

it’s all about actors

Actors are not the base, abstract class for elements on the UI: they are now a concrete class that should be used directly, instead of through sub-classes. the ClutterActor class is the element of the scene graph; it holds children, it paints them, it lays them out. there is one API to learn, one class that holds the keys to all the behaviour, and one class to inherit from for custom controls.

obviously, ClutterActor cannot do everything by itself; but instead of creating sub-classes to specialize the behaviour right within Clutter, a delegation pattern was preferred. layout is now deferred to ClutterLayoutManager; painting is delegated to ClutterContent.

delegation means that composition is the approach for creating apps; Clutter is, now more than ever, like a set of blocks that can be composed together to achieve a compelling, dynamic user interface.

let’s start from an actual example: whereas until 1.8 you had to use the ClutterContainer interface as the public side for building and manipulating the actor tree, now ClutterActor owns the entire list of public entry points for adding, removing, and iterating children; this also applies to the implementation of container actors: the ClutterContainer interface is implemented directly by the ClutterActor class, so you can just subclass the latter and use the default implementation without having to care about internal children, or memory management, or mapping and unmapping. even painting and picking by default will do the right thing — so you don’t have to care any more. when you subclass the ClutterActor class you can now concentrate only on the things your class has to do — instead of having to care about Clutter internals.

moving all the API from within the containers to the actors also meant being able to add a bunch of convenience functions for manipulating and iterating the actor tree; usually, these functions had to be provided by container classes, and thus had to be re-implemented over, and over, and over again. now, instead, you can get the first and last child of an actor, and iterate over their siblings; or you can create a simple, stack allocated iterator structure, and safely iterate over the children of an actor. you can insert actors at a specific index in the paint and allocation sequence, or you can replace a child atomically. finally, you can remove, or destroy, all the children of an actor in one fell swoop.

implementing the Container interface inside ClutterActor also allowed us to seamlessly migrate the ClutterLayoutManager delegate over to ClutterActor; this means that every actor can now delegate the layout of its children to any layout manager without requiring weird incantations. ((the layout manager API still needs some adaptations, which will come in the 1.12 time frame))

building the actor tree, though, it’s just part of the job of creating a Clutter app; we’ll see in the next blog post what changes happened when you want to fill an app with content…

Calamity Song

I have written a bunch of blog posts about the Clutter Apocalypses that have landed in Git, as well as how I see the Clutter API evolving after this cycle and towards the 2.0 API break; they should appear in the next few hours, staggered a bit to avoid clogging the intertubes.

hopefully, I won’t bore you to death — but I’ll understand if you want to filter me out.

I’ll keep the comments section open, but feel free to use the clutter-devel mailing list: it’s much, much better than a comment in a blog.

From my Own True Love (Lost at Sea)

Third day of the GTK hackfest. They have taken the bridge and the second hall. We have barred the gates but cannot hold them for long. The ground shakes, drums… drums in the deep. We cannot get out. A shadow lurks in the dark. We can not get out… they are coming.

today, the topics discussed were two:

  • ClutterBenjamin, Matthias, and I discussed how to integrate Clutter with GTK in the future. the plan that is shaping up is to hollow out GtkWidget and make it the minimal semantic unit of the structure of an application; a button, a combo box, a text entry, etc. each widget would then be composed of internal elements — the background and the text for a button; the icons, the progress bar, and the text, for an entry, and so on, and so forth. each element maps the CSS box model (as far as it makes sense), and it’s backed by a ClutterActor. layout management of each element, and rendering, is deferred to Clutter. so, each GtkWidget has-a ClutterActor, that you can also access to use Clutter directly. implementing a GtkWidget would then be simplified to actually building the structure that has to be rendered, drawing with Cairo on provided surfaces if you have custom content instead of primitives like texture data or text, thus leaving the rest to the scene graph underneath. while this is not a radical change in how GTK+ works, we’re definitely considering bumping to the 4.0 API version — this time it wouldn’t be such a quantum leap like it has been for the 2.x to 3.x transition. all this work depends on the Clutter 2.0 effort, which will take the bulk of the next cycle for me. hopefully, with a little help, we’ll be able to do a GTK+ 4.0 in 12 months.
  • design — after lunch, we all sat down with the GNOME designers, and talked about the direction for the toolkit, especially with regards to enabling touch on widgets, and adding more widgets that are touch-oriented to be used when designing applications for touchscreen environments. we also discussed a lot about themeing, and how app developers should use or depend on themes like they do on ABI. then we went through the design whiteboards section of the wiki to see what tasks involve the toolkit — like printing, notification, selection, and content selection.

all in all, it was a very productive day; we’re churning through the agenda points at a fairly good pace, and hopefully we’ll soon be able to work on all the areas that have been identified, in order to make GNOME rock.

An Interlude

today I spent the morning at the Developer Conference: Matthias had an interesting talk on the changes that happened in GTK+ 3.x over the past year, as well as his experience in implementing the new GtkColorChooser after the design proposals.

then it was multi-touch time. after a recap of what happens on different platforms and different toolkits, we discussed and reviewed the API proposed and implemented by Carlos Garnacho; after a quick break for a late lunch at 4pm, the discussion resumed — it’s actually still going on to this very moment: Chase, Benjamin, and Matthias are now discussing about event controllers and gesture recognisers — and how to design event handling in the Brave New World® of touch events and gestures.

Youth and Beauty Brigade

first day of the GTK+ Hackfest.

while we waited for various participants to arrive, discussions started on a couple of topics:

  • accessibility — Benjamin reported on the various discussions and deliberations that happened during the latest Accessibility Hackfest; what we want to achieve is a state in which the accessibility team has fewer headaches working around issues in the toolkit, and can instead work on the toolkit itself. there is also a question of what kind of API to expose to application developers, not just for people writing AT apps, but also for people writing UI testing suites.
  • multi-touch — Chase, Benjamin, Ryan, and occasionally myself, caught up a bit on the overall issues of what kind of API to expose from GTK+ to applications, and what kind of approaches we should be using internally. the bulk of the discussion will resume either this evening, where alcohol will be involved in making them long and very much detailed; or tomorrow, where hangover will keep them focused and short.

plus, people were really interested in the correctness of the regular expression on my tee, which was initially amusing, up until the point a whiteboard and a state machine were involved. occupational hazards, I guess.

I also want to point out that at no point knives, or other weaponry, was involved in the various discussions.

I’d like to thank Red Hat (and the Red Hat office in Brno) for offering the conference room for the hackfest.

Bridges and Balloons

after a rather interesting, albeit not really eventful, journey that spanned cars, buses, a train, and a plane, I’m finally in the lovely Brno — with 2 degrees Celsius and a lot of snow — for the 2012 GTK+ hackfest.

unlike the previous two hackfests, and against my usual inability to write interesting stuff, I’ll try and live-blog the event as much as I can.

as you can see on the wiki page, the agenda is pretty dense, and it’s going to be an intense 6 days hackathon, filled with discussions as much as coding. my part? talking about rainbows and unicorns, obviously.

a lot happened in the months between the Desktop Summit in Berlin, where we first sat down and talked about what Clutter does, and what GTK needs; there has been various progress in the Clutter API towards a sane base for a 2.0 release. if you try and compile an existing application using Clutter with the 1.9 developers snapshots, you’ll notice the depth and breadth of deprecations and API additions. a lot more needs to change — and some of this work on getting from here to there is detailed on the Clutter wiki, under the not at all threatening name of Clutter Apocalypses. I’ll try and write other blog posts about how I envision those changes, and why I think they are necessary for a 2.0 API version.

Days of Elaine

oh man, it’s getting harder to find blog post titles…

clutter 1.8 — last Friday I rolled the first release of the new stable 1.8 cycle, and Rob rolled the first stable release of the now standalone Cogl. it’s been six months of intense development, with lots of contributions, and I’d like to thank everyone who submitted patches, opened bugs, added a translation, or even dropped by in the #clutter IRC channel to tell us that they were using Clutter for their awesome project.

clutter 1.9 — instead of resting after pushing 1.8 out of the door, we’re already hard at work on what will become 1.10 in six months. while Cogl is getting more and more refactoring to make the code base more understandable and reliable, as well as preparing the way for new features, Clutter is also getting some refactoring. today, for instance, I worked on building Clutter with multiple backends in the same shared object. one of my first major contributions to Clutter, back in the OpenedHand days, was to provide the ability to have multiple backends — and it worked pretty well: after the GLX backend we got a Windows backend, then a framebuffer backend, then an OS X backend. during the years, the implementation was amended here and there to avoid leaking the backend implementation all over the place, and we always resisted the temptation (and the outside pressure) to provide a full abstraction library for the windowing system akin to GDK. this allowed us to maintain the backend code lean and lightweight, to the point that it took me literally an hour this morning to go from the current HEAD of master to a test running with either GDK or GLX (depending on an environment variable) — and most of my changes were in the build system, not in the code.

as an example, this is a simple test program that checks the backend support at both compile and run time; using Clutter from the multi-backend branch, compiled with --enable-x11 and --enable-gdk, you can run the example on X11 using:

$ CLUTTER_BACKEND=x11 ./test-backend &
$ CLUTTER_BACKEND=gdk ./test-backend &
multiple backends, same great taste

and have two instances, one using the native X11 backend and the other using the GDK backend.

there’s still some work to do, namely fixing Clutter-GTK to work with a multi-backend Clutter, and making it use the GDK backend by default, if found. there’s also the issue of being able to use the GLX_EXT_texture_from_pixmap extension under the GDK backend on X11, if available, so that we can efficiently embed GTK widgets rendered off-screen like we can already do when using the Clutter X11 backend.

all in all, this work will help towards the goal of using Clutter and GTK together in a more organic fashion, and with fewer blocks and hacks in the middle.

Down by the Water

it’s been far too long since the last London GNOME Beer meetup, and we also have the excuse occasion of celebrating the imminent 3.2 release of GNOME. so if you are a GNOME user or developer, or if you just want to meet GNOME hackers here in Good Old London Town, go on the wiki page and sign up for tomorrow evening London GNOME Beer, version 3.2.