Category Archives: Hacking

GNOME and the missing one-fifth: part one

GNOME ParkingEarlier this year Lionel Driscot’s “The Last GUADEC” was slashdotted, redditted, and everthing else-itted that can happen to a viral blog post in the tech community.  It’s a provocative piece.  It also sparked for me the impetus to write this entry, which I’ve been mulling over for a few months.

My one nit with Driscot’s post are the questions he ends with, his call to action:

How can we evolve? Can we move the GNOME spirit into a web browser? How can we make use of our history and bring freedom to users instead of becoming just another web dev consultancy company?

These are not the questions I would have asked.  I do not believe the answers to these questions will yield appreciable fruit.

Here’s a portion of Emmanuele Bassi’s response to Driscot’s post (which I can no longer access, unfortunately):

GNOME is about building a whole operating system, and a whole ecosystem of apps, libraries, and tools, for people to create applications to consume and to create content.

This hovers closer to the questions I would have asked.

GNOME’s stated goal of “a complete free software solution for everyone” seems to me to be four-fifths met.  What GNOME offers is (1) free, (2) software, (3) a solution, and (4) for everyone.  I question the word complete.

I say this specifically as an application developer, not as someone maintaining a library or writing development tools.  Yorba isn’t even a cross-platform developer.  We target UNIX, and, frankly, we’re not proactive about any flavor of UNIX other than Linux.  Our requirements for a development platform aren’t particularly stiff.  Yet even slicing away Windows and Mac support, I still think GNOME is far from complete.

Here are the questions I would ask in place of Driscot’s:

  • How does the GNOME platform fit into a world of big and small interconnected devices accessing a variety of Internet services?
  • How is GNOME enticing developers to build upon — not build, build upon — that platform?

If I was a developer building Mac, iOS, Windows, or Android applications, I’d be able to answer those questions.  I honestly can’t answer those questions as GNOME application developer, at least not in full.

Thinking about those questions got me to thinking about another: What’s missing from the GNOME platform?  I’ll take that up in my next post.

The Garden of the Forking Paths

A thought experiment:

A software developer decides to write an open-source GNOME desktop application that interacts with any number of Internet services, such as Facebook, Gmail, Flickr, and so forth.  It’s not specific to any one service, however.  The app should take advantage of all the available desktop technologies, not only to leverage all that hard work, but also to be a good desktop citizen.

The first thing the application does is log in to an Internet service.  What should this developer do?

(a) Handroll the authentication code.  Most Internet services use a token-exchange system, but each service is different, so the developer will need to write custom code for all of them.  The user will want the application to save the password for next time, so better look into keyring solutions.  And there’s always the problem of where and how to store long-term tokens, since every service has its own Terms of Agreement.  And don’t forget to request Application Keys — each service has its own.  Some services have different entry points for their various “faces” (i.e. Yahoo! and Flickr) while some have a federated login system (i.e. Google).

Halfway through the process the developer thinks, “Why am I reinventing this wheel?  A lot of applications could use this code.”

Or (b) Use a system library.  The developer calls the library and gets back a token.  No messing around with passwords, keyrings, confirmation Web pages … all that’s handled by the system service.

The developer thinks, “Perfect!  Problem solved and we’re being a good desktop citizen because the user only has to enter their password once.”

(b) seems the obvious choice, right?

Then the developer learns that Ubuntu and most of its derivatives (Ubunten?) use Ubuntu Online Accounts (UOA).  The rest use GNOME Online Accounts (GOA).  UOA and GOA are not API/ABI compatible.

Suddenly (a) looks like an appealing prospect.

This is a poisonous situation for application developers.  It’s not terribly visionary of me to suggest that the number of programs that interact with Internet services — the magical “cloud” — is only going to grow over time.  GNOME and Ubuntu are both acutely interested in attracting more developers (or at least should be).  But new and curious developers arrive and are confronted with this situation.

I speak from experience.  Both Shotwell and Geary could use some kind of Online Account solution today.  Canonical has forked Shotwell to call UOA rather than our handrolled code, but those patches have not migrated upstream, nor could we take them as-is.  Requiring UOA would preclude Shotwell from running on non-Ubunten distros.

I’m not here to argue UOA vs. GOA, or GNOME Shell vs. Unity, or lament that there’s yet-another split on the desktop.  I accept that both sides have decided to go their own ways (here’s Debarshi Ray giving his reasons).  But we at Yorba want Shotwell and Geary to work well — no, work spectacularly — on all GNOME desktops.  That we can’t deal fluidly with something as low-level as server credentials seems absurd.

I once had a professor tell me, “All problems in computer science are solved with a layer of indirection.”  He was being snarky, but there’s a germ of wisdom in that observation.  It seems to me that the solution to the GOA/UOA divide could be solved with a translation or wrapper library.  I’m not thrilled about wrapper code, but sometimes that’s what you’ve got to use.

To throw the first dart at the dartboard, here’s my proposed interface for such a library:

  • Enumerate supported Internet services: Facebook, Yahoo!, Google, etc.
  • Query a particular service for details/capabilities/description/icon/etc.
  • Generate an authentication token for a particular service
  • Launch the login/account registration application (i.e. the GOA/UOA program that takes the user though the steps of logging in to a service and making it available to all applications)

Obviously it’s far more complicated than this, but that’s the gist.  I know GOA is DBus-based, and I believe UOA is as well, so the solution could simply be a DBus interface that both technologies implement, perhaps in the org.freedesktop namespace.  In fact, maybe this should be a freedesktop.org initiative.

We need something.  Asking an application developer (including Yorba) to pick UOA or GOA is a non-starter.  There seems to be a bit of bad blood between the GOA and UOA camps — I don’t know all the details — but I ask both of you to scratch together a bridge rather than fork the path.

Gedit class browser for Vala

 Here at Yorba most of us do our development work in Vala using Gedit.  Out of the box, Gedit doesn’t seem like much of an IDE.  But with a few plugins it starts to approach the feature set of a modern development environment.

One feature that’s lacking, though, is a good class browser.  With larger projects this is a must-have tool — scrolling through hundreds of lines of code to find a method seems silly when you can just point and click.

There already is a class browser plugin for Gedit. Caveat: it’s based on ctags.  While ctags is a great library, one language it doesn’t support is Vala.

So is all hope lost? No!  Here’s how to make the aforementioned class browser plugin support Vala.

  1. Install Anjuta. (You’ll see why in a second.)
  2. Install the plugin as described in its readme. Make sure you install the GSettings schema as described.
  3. Open Gedit and enable the plugin. It’s listed as “Source Code Browser.”
  4. In the same window, hit Preferences to open the plugin options.
  5. Next to “ctags executable” change the executable name from “ctags” to “anjuta-tags”.  Anjuta-Tags is a fork of ctags that adds Vala support, GIR support, etc.

Enable your Gedit sidebar (if it isn’t already) to see the class browser tab.  Open a Vala file and you’ll get a nicely formated hierarchical symbol list.

That’s all there is to it. Happy coding!

Avoiding the association by hash map anti-pattern with GObject

There’s a common anti-pattern in software design that I call “association by hash map.” Rather than explain this in words, let me illustrate it in Vala:

HashMap<MyClass, string> name_map = 
    new HashMap<MyClass, string>();

name_map.set(an_object, "Foo");
name_map.set(another_object, "Bar");

stdout.printf("an_object's name is %sn", 
    name_map.get(an_object));

“But wait,” you’re saying, “Couldn’t we just add a name field to MyClass and remove the hash map?”

YES! That is exactly what you should do — if you can.

But what if you can’t?  Adding a field isn’t always an option.  Perhaps MyClass is someone else’s API.  Or even if it’s contained entirely in your code, the name field might only make sense for a single use-case.  No reason to add an extra field to your class if it doesn’t truly belong there.

So what to do? Is there a better way? If your class is based on GObject, you’re in luck.

During a recent code review, Jim pointed out to me that GObject has methods for attaching arbitrary named data to an object.  In our example, we can use the simplest of these methods, gobject_set_data() and gobject_get_data() which use simple key/value pairs.

const string NAME_PROP = "name";

an_object.set_data(NAME_PROP, "Foo");
another_object.set_data(NAME_PROP, "Bar");

stdout.printf("an_object's name is %sn", 
    an_object.get_data(NAME_PROP));

Isn’t that better?  No more extraneous hash maps, all the data is stored right in the object itself where it belongs.  And you didn’t even have to modify the class!

In closing, if you’ve dealt with the association by hash map pattern before you now know a way to avoid it with GObject.  And if you haven’t, I envy you.

The 100,000 line question

Clinton Rogers (Yorba developer and all-pro Telestrations master) pointed out something interesting yesterday.  Ohloh now lists Shotwell as having achieved an odd milestone just ten days shy of its third birthday: Shotwell has reached 100,000 lines of code.  That number represents the work of 51 code contributors and 89 translators.  (It also represents blanks and comments — 100,000 lines of pure code is a ways off.)

It’s an odd milestone because there’s a rosy tradition in computer programming of honoring tight code and efficient algorithms.  The code placed on pedestals are not thousands of lines long, they’re short and sweet, like a witty joke or a clever haiku.  Duff’s Device is my favorite example; you probably have your own.  (Tom Duff’s history of his Device is a great short read and offers a basketful of concise observations on code optimization.)

Which is why reaching the 100,000 mark makes me simultaneously proud and a little uncomfortable.  Shotwell has grown quite a bit in three years — but is it really doing the work of 100,000 lines of code?  Ur-spreadsheet VisiCalc was also 100,000 lines of code, pretty lean compared to the Macbeth that knocked it off its throne, Lotus 1-2-3 (clocking in at 400,000 lines).  Compare that to the original Colossal Cave game, which was (gulp) 700 lines of FORTRAN and 700 lines of data.  It later grew to a whopping 4,800 lines of code and data that ran entirely in memory.  100,000 lines of code feels downright luxurious, even bourgeois, in comparison.

(I’m not claiming Shotwell should be placed alongside these landmarks.  It’s just interesting to consider what 100,000 lines of code represents.  I’m also aware that there’s a number of people who think line count is a misleading, or even useless, metric.  I do think lines of code provides some scale of complexity and size.  I’ve never seen a program grow in size and get simpler.)

There’s probably no reason to duck my head in shame.  Sure, there’s plenty of features we want to add and bugs we want to squash, but those 100,000 lines of code we have today are pulling a lot of collective weight.  They include the basic organizer, a nondestructive photo editor, a standalone image viewer (which also includes an editor), hierarchical tagging, stored searches, ten plug-ins, and plenty more.  Could we scrape away 1,000 lines of code and still have the same functionality?  Almost certainly.  10,000?  I can think of a few places where fat could be trimmed, but I don’t think it’s excessive.

Note that Ohloh is counting lines of Vala code, not the C generated by valac.  Although valac does not exactly produce sparse output, it’s worth mentioning that sloccount reports over 720,000 lines of C code generated by Vala.  If Vala is producing on average six times more C code than a competent human programmer (and I’m not asserting it does), that’s 120,000 extra lines.  Reducing that by the magic factor of six means Vala saved us from writing 20,000 lines of C code, a victory worth popping open a can of beer and celebrating over.

A few of my favorite Vala things: interfaces

In my prior post on Vala’s language features, I discussed enums and how I appreciated Vala’s implementation of them.  I feel that Vala’s enums straddle an interesting line of utility and pragmatism.  It took me a while to learn about their features, partially because documentation has been sparse (but is getting better) and partially because as a C / C++ / Java programmer, I’d had hammered into me a set of expectations about enums that Vala didn’t quite adhere to.  (I had a similar learning curve, for many of the same reasons, about Vala’s lock keyword.)

Learning interface in Vala was a similar experience.  Consider the Java Language Specification’s opening statement about interfaces, which Vala’s interfaces look to be a descendant of:

An interface declaration introduces a new reference type whose members are classes, interfaces, constants and abstract methods. This type has no implementation, but otherwise unrelated classes can implement it by providing implementations for its abstract methods.

Compare to this statement in the C# Programmer’s Guide:

Interfaces consist of methods, properties, events, indexers, or any combination of those four member types. An interface cannot contain constants, fields, operators, instance constructors, destructors, or types. It cannot contain static members. Interfaces members are automatically public, and they cannot include any access modifiers. … The interface itself provides no functionality that a class or struct can inherit in the way that base class functionality can be inherited.

What’s interesting about Java and C# interfaces is what they can’t provide: implementation.  Interfaces are often touted as a sane solution to the problems surrounding multiple inheritance, but it always struck me as odd (and a bit of a blind-spot on the part of their boosters) that interfaces provide no reusable code.  After all, isn’t that the name of the game?  Especially for a technique that’s replacing a form of inheritance, which is all about reusable code?

(I have a pet theory that if one was to study the history of the development of software development technologies — languages, tools, paradigms, all of it — it’s primarily a history of reusable code.  How much money and manpower has been dumped into this holiest of Holy Grails: write once, debug a bit more, reuse a billion times.)

Like enum and lock, Vala offers an interesting interpretation of interface with a couple of surprises.  I’m not sure how much of it is due to the mechanics of GTypeInterface, Vala’s influences from Eiffel, or simply a reflection of Jürg’s vision, but it’s cool all the same.

Let’s start with a simple Vala interface that looks familiar to any Java or C# programmer:

interface Agreeable {
    public abstract bool concur();
}

In-the-know Java programmers will say that the public and abstract keywords are allowed but not required; in C#, they’re simply not allowed.  But in Vala, neither are optional:

interface Agreeable {
    bool concur();
}

produces this compiler output:

error: Non-abstract, non-extern methods must have bodies

That seems to suggest that non-abstract, non-extern methods in interfaces can have bodies (“must implies can”, kind of a programmer’s variant of Kant’s argument from morality).

And sure enough, this will compile:

interface Agreeable {
    public bool concur() {
        return true;
    }
}

What’s going on here?  Simple: reusable code.

Vala interfaces are much more than hollow declarations of required methods.  Interfaces are full-fledged members of the object hierarchy, able to provide code to inheriting classes.  The reason an interface is not an abstract class is that an interface has no storage of its own, including a vtable.  Only when an interface is bound to a class (abstract or concrete) is storage allocated and a vtable is declared.  In other words, while the above is legal, this is not:

interface Agreeable {
    bool yep = true;

    public bool concur() {
        return yep;
    }
}

gets you this:

error: Interfaces may not have instance fields

Bingo.  That boolean is a member instance, which requires instance storage, which an interface does not have by itself.

So what good is allowing an interface to provide reusable code if it has no fields of its own?  There’s a number of patterns of use, in particular Facade-style methods.  For example, an interface could declare a handful of abstract primitive methods and offer a helper method that uses them in concert according to contract:

interface Agreeable {
    public abstract bool concur(int i);

    public abstract string explanation(bool concurred);

    public void process(int i) {
        if (concur(i))
            stdout.printf("Accepted: %d %sn", i, explanation(true));
        else
            stdout.printf("Not accepted: %d %sn", i, explanation(false));
    }
}

class OnlyOdds : Agreeable {
    public bool concur(int i) {
        return (i & 0x01) == 0x01;
    }

    public string explanation(bool concurred) {
        return concurred ? "is odd" : "is not odd";
    }
 }

class OnlyMultiplesOfTen : Agreeable {
    public bool concur(int i) {
        return (i % 10) == 0;
    }

    public string explanation(bool concurred) {
        return concurred ? "is a multiple of ten" : "is not a multiple of ten";
    }
}

First, note that the implementations of concur() and explanation() in the classes don’t use the override keyword even though you must use the abstract keyword in the interface.  I’m not sure of the reasoning, but so it goes.

Also know that virtual methods, signals, and virtual signals have their own peculiarities with interfaces.  I’ll deal with them in another post.

So, a pretty contrived and very silly example, but notice how process() understands Agreeable’s interface and contract and hides those details behind a single method call.  This is useful.

Going back to those language specifications earlier, remember that Java and C#’s interfaces cannot contain static methods.  In Vala they can:

interface Agreeable {
/* ... */
    public static void process_many(Agreeable[] list, int i) {
        foreach (Agreeable a in list)
            a.process(i);
    }
}

This allows for binding aggregator-style code with the interface itself, rather than building utility namespaces or classes.  Again, this is useful.

However, if the following code is written:

Agreeable[] list = new Agreeable[2];
list[0] = new OnlyOdds();
list[1] = new OnlyMultiplesOfTen();

Agreeable.process_many(list, 10);

you get this compiler error:

error: missing class prerequisite for interface `Agreeable',
add GLib.Object to interface declaration if unsure

What’s this about?

It’s due to another freedom in Vala that is lacking in Java and C#.  Vala classes don’t have to descend from a single master class (i.e. Object).  Unlike the other two languages, if a Vala class is declared without a parent, there is no implicit parent; Vala registers the class with GType as a fundamental type.  If you don’t know what that means, read this.  You probably still won’t know what that means, however.

Because Agreeable is declared without a prerequisite class, Vala can’t produce the appropriate code to store it in a tangible data structure, in this case, an array.  (Update: As Luca Bruno explains in the comments, this is because of Vala’s memory management features.)  This solves the problem:

interface Agreeable : Object {

What this means is that any class that implements Agreeable must descend from Object (i.e. GObject), meaning we need to change two other lines in the code:

class OnlyOdds : Object, Agreeable {

class OnlyMultiplesOfTen : Object, Agreeable {

Although Agreeable now looks to descend from Object, it does not.  It merely requires an implementing class to descend from Object.  (A subtle difference.)  Interfaces can also require other interfaces, and like classes, it can require any number of them:

interface Agreeable : Object, Insultable, Laughable {

Like requiring Object, this means that any class implementing Agreeable must also implement the other interfaces listed (Insultable, Laughable).  This does not mean that Agreeable must implement those interfaces’ abstract methods.  In fact, it can’t, one place where code reuse can’t occur.

Prerequisites also mean that Agreeable’s code can rely on those interfaces in its own code, and therefore can do things like this:

interface Agreeable : Object, Insultable, Laughable {
/* ... */
    public void punish(int i) {
        if (concur(i))
            laugh_at();
        else
            insult();
    }
}

… where laugh_at() is an abstract member of Laughable, insult() is an abstract member of Insultable, and of course concur() is its own abstract member.  In other words, because Agreeable knows it’s also Insultable and Laughable, it can treat itself as one of them.

It’s easy to go crazy with interfaces, prerequisites, and helper methods, but most great languages have their danger zones of excess and abuse — features that are the hammer that makes everything look like a nail.  Still, I think code reuse is the most important goal of any programming technology — language, tool, or paradigm — and I’m glad Vala has given it some thought in terms of interface.