Vala delegates under the hood

On the Vala mailing list someone wondered if delegates are merely function pointers, not some higher-level concept.  This was in the context of a blog post earlier this year speculating (accusing?) Vala of infringing on Microsoft patents.  I won’t step into the legal morass of software patents under United States law, but it’s important to know: delegates are not merely function pointers.  Understanding how delegates differ from function pointers makes for a better understanding of their power.

Consider the common C++ dictum against taking pointers of non-static functions (that is, methods, which are different than pointers to member functions, C++’s mangled way of dealing with this problem).  A method pointer is near-useless without a corresponding this pointer (which must be manually passed on the stack).  And pointers to virtual methods bypass the vtable, which is itself dangerous for a bunch of reasons.

Delegates solve these problems in an elegant way.  In order to do this, a delegate bundles together a this pointer (which Vala calls the target and is null for a static delegate) and a function pointer.

However, the delegate’s function pointer is not merely pointing at your callback.  Instead, Vala passes a pointer to a hidden delegation function which calls the “real” function.  The delegation function re-sorts the callback’s parameters to move the user data pointer (which can be anywhere on the stack, and is often last in GLib) to the first parameter for the real callback (making it the this pointer).  If the delegate is for a virtual function, this “real” function in turn calls through the object’s vtable (which is a function pointer stored in the object’s instance structure).

These levels of indirection means delegates just work (with some notable exceptions).  Because this binding happens late (i.e. at run-time), you can pass delegates around in constructors before the child class has completed initialization (although there’s some danger there, obviously).  If Vala later adds deeper mixin support, late binding means delegates still work.

What’s cool about Vala is how this is all laid out visibly in C.  Perusing the C code makes it easy to learn how modern object-oriented features are implemented down at the metal.  In essence, Use the Source Luke has two meanings with Vala: you can glean language features by examining the compiler or the code the compiler generates.

Dave Neary on forking

As someone who witnessed Shotwell get forked over a dispute of two text labels, I have to chuckle and nod my head at Dave Neary’s apt analogy:

A fork is like a schoolyard football game. Tommy brings a ball to school, and gets to be captain of a team every time, picks the best players for himself, because if he doesn’t, he threatens to take his ball away. After a few weeks of this, the other players get annoyed, and everyone chips in to buy a ball for the group. At this point, Tommy has lost all power – he still has his ball, but he’s not team captain any more. He gets left standing against the wall till the very end to teach him a lesson in humility. So he sits out the games for a while in protest, starts a competing game at the other end of the yard with his ball and his rules. All the best players go to the other game, though, and some of the kids make fun of “Tommy’s” game – invitations to give up his game & join the new one seem more like taunts than honest efforts to include him. After a while, he realises that playing football with others is more fun, and more important, than playing with his football.

My only comment here is that Tommy in this situation represents a master trunk and the others are the fork.  There’s an additional football analogy here, that of Tommy wanting to be captain and bringing his own ball to the schoolyard and trying to get the others to play his game.  If they do, we admire the power of the open-source ecosphere.  If they don’t, it’s another branch that’s shriveled and died.

In this sense, open-source forking has made it into Urban Dictionary.

Those “realize” & “map” widget signals

I’ve always been a little fuzzy on the relationship between these important Gtk.Widget signals: “map”, “map-event”, “realize”, “show”, and “expose-event”.  I was working on a problem with Shotwell today and realized (no pun) that I needed to intercept at least one of these to solve a UI initialization problem.  Okay — which one?

They’re all related, of course.  Over time I’ve gathered some notion of their order (“realize” before “map” followed at some point by “expose-event”), but what is “map-event” versus “map”?  And why a “show” signal when all these other signals seem to be saying, in their own way, the same thing?

For the nth time I dug through DevHelp (no help) and the Xlib Programming Manual (which is written in some variant of Middle English — I recognize the language, fuzzily) looking for help.  I finally discovered this clue by none other than Havoc Pennington:

Realize means to create the GDK resources for a widget. i.e. to instantiate the widget on the display. This is more useful once we have multiple display support in GTK.

Map means to actually pop the widget’s window onscreen. It requires the widget to be realized, since the window is created in realize.

Show means the widget should be mapped when its toplevel is mapped, or in the case of a toplevel, should be mapped immediately.

Mapping is asynchronous though; that is, gtk_widget_map() and the map signal are emitted when the map is requested. When the map actually occurs you get a map_event (distinct from plain map). But you are not allowed to draw on widgets until you get the first expose, map_event is insufficient.

I’m not saying that clears everything up, but it gave me a starting point:

realize is what takes a data structure in memory (the Widget) and allocates GDK resources (i.e. a window) for it.  I’ve done a lot of network programming in my past, so I imagine this step as analogous to initializing a Session object.  The Session object itself is (usually) cheap to instantiate, but can’t do much until a network connection (i.e. a socket) is created — sockets being a system resource much scarcer than heap memory.

map, as Havoc says, “pops” the window onto the display.  But actually, it only requests the window be displayed.  I compare this to an asynchronous connect() call.  The request is initiated, but when the call returns, the circuit is not complete.

map-event is a GDK event.  This is called when the window is now on-screen, i.e. the connection is complete.  It’s like a callback.

expose-event is called when the window may be drawn upon, and is called thereafter whenever a portion of the window has gone dirty and must be redrawn.  In network parlance, this is like a signal some stacks offer to indicate when a connection is able to send().  (This is the roughest analogy of them all.)

This leaves unrealize, unmap, and unmap-event, which look like obvious complements — signals fired during tear-down to indicate the stages of destroying the underlying window object.

I built a test application in Vala to verify some of this as well as determine the ordering of these signals for a Gtk.Window.  (They might be in a different order for windowed widgets; I didn’t investigate.)  This is the ordering of the events after I called show_all() on the Gtk.Window and then closed the window by clicking on the dismiss button in the chrome:

  1. show
  2. realize
  3. map
  4. expose-event
  5. map-event
  6. unmap
  7. unrealize

Notice that “unmap-event” was never called; this may have to do with connecting the “destroy” signal to Gtk.main_quit().  The fact that the application can exit without it being properly called is enough for me to know not to rely on it during tear-down.

Also notice that “expose-event” is called in between the “map” call and the “map-event” callback.  This too is not what I expected.

And what about “show”?  I understand what the virtual method is doing, but why make it a signal?  Havoc explains in a follow-up:

It’s one of those weird historical things from pre-1.0.

Okay.  I can accept that.

(Postscript: I know that the “expose-event” signal is deprecated in GNOME 3.0 in favor of “draw”, but I still think the above information is useful.)

Vala and the comma operator

One of the more interesting parts of working on a Vala project is occasionally having to read through the C source the Vala compiler generates.  Sometimes it’s to investigate a bug (maybe your own, maybe the compiler’s) and sometimes it’s just out of curiosity.

Personally, I think the C code Vala produces is pretty darn good as far as machine-generated code goes.  (And I’ve seen some horrible machine-generated code, not including obfuscators and their ilk.  Which reminds me, have I ever told you about when I met Phil Katz?  Another time.)  Certainly I feel I could produce much prettier code by hand, but that’s the point: I would have to do it by hand.  A lazy programmer is a good programmer.

Take a look at this typical example of Vala’s generated C code:

gboolean dimensions_has_area (Dimensions *self) {
    gboolean result;
     gboolean _tmp0_ = FALSE;
     if ((*self).width > 0) {
         _tmp0_ = (*self).height > 0;
     } else {
         _tmp0_ = FALSE;
     }
     result = _tmp0_;
     return result;
}

If an interview candidate at Yorba put this up on the whiteboard, I imagine the first thing we’d ask would be, “Ah … anything you think you could do to clean up this function?”

This isn’t to knock Vala or the compiler or the wonderful work Jürg and Team Vala have done.  And yes, I know, gcc will optimize this down to machine code that would be pretty much like the machine code it would produce if I hand-coded this.  And, of course, unless this is in hot loop (which it’s not, last time I checked), none of this matters anyway.

In short, I lose zero sleep over this.

But consider this line, which is also not atypical:

scaled = (dimensions_floor ((_tmp2_ = (dimensions_init (&_tmp0_, (gint) round (scaled_width), (gint) round (scaled_height)), _tmp0_), &_tmp2_), (_tmp3_ = (dimensions_init (&_tmp1_, 1, 1), _tmp1_), &_tmp3_), &_tmp4_), _tmp4_);

Ah … okay, hold on.  Don’t run off.  It’s not as bad as you think.  It’s just that this makes much more sense to the C compiler than it does to you.

My advice to anyone who dives into Vala’s generated code is to study up on the comma operator.  I suspect most programmers (maybe 98% of them!) have never used the comma operator outside a for loop, and even then, rarely.  Most of them would say that the comma operator is only used to perform multiple statements (actually, “operands” in the parlance) on a single line of code.  That is, it’s a handy way when writing a for loop to squeeze in multiple statements inside each of its three parameters.

Which is true, save for one thing: With the comma operator, the statement as a whole evaluates to the final operand.  Thus, the initializers in this for loop evaluate to 42:

for (i = 0, j = 42; i < j; i++)

The initializer isn’t being assigned to anything, so we don’t notice, so we never learn.

Looking back over that C code up there, mentally weed out the _tmp0_, _tmp1_, etc. and see it for what it is: Initializing two dimensions structs (one goes into _tmp2_, the other into _tmp3_), which are passed to dimensions_floor as parameters, which puts its result into _tmp4_ (whose address is passed to dimensions_floor as its third parameter).  The final operand of the comma operator is, in fact, _tmp4_, which then is returned to scaled.  The comma operator makes all this happen.

Here’s that line in Vala, by the way:

Dimensions scaled = Dimensions((int) Math.round(scaled_width), (int) Math.round(scaled_height)).floor();

Shotwell 0.7.2 and gexiv2 0.2.1 released

Yorba has released Shotwell 0.7.2, an update to our digital photo
organizer.  This release includes crucial fixes and translation updates.  It’s highly recommended that all
users upgrade.

This release includes the following:

  * Fixed major startup problem when the user’s Pictures directory is actually
    a symbolic link.
  * Fixed potential crasher when the user’s Pictures directory contains a
    large number of subdirectories.
  * No longer asking if copying or linking if importing from Pictures directory
    (which will always be linked).
  * Fixed update problem when using an external editor on a photo that was
    edited externally in an earlier session.
  * Various bug fixes.
  * Updated translations.

In addition, Yorba has released an update to our Exiv2 GObject wrapper library, gexiv2 0.2.1.  (This library is a requirement for Shotwell.)  This update fixes a bug where a photo file with malformed EXIF could cause a crash.  It’s highly recommended that all users upgrade.

Download a source tarball from the Shotwell home page at:
http://www.yorba.org/shotwell/

Download a source tarball from the gexiv2 home page at:
http://trac.yorba.org/wiki/gexiv2

Binaries for Ubuntu Lucid and Maverick are available at Yorba’s Launchpad PPA:
https://launchpad.net/~yorba/+archive/ppa

Bruno Girin on importing photos from other apps to Shotwell

Bruno Girin, author of the F-Spot import subsystem now shipping in Shotwell 0.7, has written a fantastic blog post on his architecture and how to write import code for other photo applications.  He mentions importing from Picasa, which we certainly are interested in doing at some point.  Bruno has done quite a bit of database work and his thoughts and approach are worth a read for those insights alone.

JIm Nelson's blog + archives from Yorba Foundation's original blog