A few days ago, I set out to get some experience with building an application as an xdg-app.  In this post, I’m collecting some of the lessons I learned.

Since I didn’t want pick a too easy test case, I chose terminix, a promising terminal emulator for GNOME. Terminix uses GTK+ and vte, which means that most dependencies are already present in the GNOME runtime.

However, terminix is written in D, and the GNOME sdk does not include D support.  So, the challenge here is to build a compiler and runtime for this language, and any other required language-specific utilities. The DMD D compiler is written in D (naturally), so some bootstrapping was required.

Build tools

xdg-app comes with low-level build support in the form of various xdg-app build commands. But you really want to use the newer xdg-app-builder tool. It is really nice.

xdg-app-builder downloads and builds the application and all its dependencies, according to a JSON manifest.  Thats par for the course for modern build tools, of course. But xdg-app-builder also has smart caching: It keeps git checkouts  of all sources (if they are in git), and only rebuilds them when they change. It also keeps the results of each modules’ build in an ostree repository, so reusing a previous build is really fast.

All the caches are kept in .xdg-app-builder/ in the directory where you run the build. If you have many dependencies, this hidden directory can grow large, so you might want to keep an eye on it and clean it out every now and then (remember, it is just a cache).

You can take a look at the JSON file I came up with.

Build API

Similar to the GNOME Continuous build system, xdg-app-builder assumes that each module in your JSON supports the ‘build api’ which consists of configure & make & make install. The world is of course more diverse than that, and rolling your own build system is irresistable for some.

Here is a way to quickly add the required build api support to almost any module (I’ve stolen this setup from the pitivi xdg-app build scripts):

Place a foo-configure script next to your JSON recipe that looks like this (note that Makefile syntax requires tabs that were eaten when I pasted this content in here):

#!/bin/sh

cat <<EOF >Makefile
all:
        ...do whatever is needed to build foo

install:
        ...commands to install foo go here

EOF

In the JSON  fragment for the foo module, you add this file as an extra source (we are taking advantage of the fact that xdg-app-builder allows multiple sources for a module):

"modules": [
    {
        "name": "foo",
        "sources": [
            {
                "type": "git",
                 "url": "http://foo.org/foo.git",
                 "branch": "master"
            },
            {
                "type": "file",
                "path": "foo-configure",
                "dest-filename": "configure"
            }
        ]
    }

I guess you could just as well have the Makefile as yet another source; this approach is following the traditional role of configure scripts to produce Makefiles.

Network access

As I mentioned already, the first step in my build was to build a D compiler written in D. Thankfully, the build script of the dmd compiler is prepared for handling this kind of bootstrapping. It does so by downloading a pre-built D compiler that is used to build the sources.

Downloading things during the build is not great for trusted and repeatable builds. And xdg-app’s build support is set up to produce such builds by running the build in a controlled, sandboxed environment, which doesn’t have network access.

So, in order to get the D compiler built, had to weaken the sandboxing for this module, and grant it network access.  It took me a little while to find out that the build-args field in the build-options does this job:

"modules": [
    {
        "name": "dmd",
        "build-options":
            {
                "build-args": ["--share=network"]
            },
        ...

Shedding weight

After navigating around other hurdles, I eventually succeeded in having a build of my entire JSON recipe run through the end. Yay! But I quickly discovered that the build directory was quite heavy. It came to over 200M, a bit much for a terminal.

xdg-app-builder creates the final build by combining the build results from all the modules in the JSON recipe. That means my terminix build included a D compiler, static and shared libraries for the D runtime, build utilties, etc.

To fix this, I added a couple of cleanup commands to the JSON. These are run after all the modules have been built, and can remove things that are no longer needed.

"cleanup-commands": ["rm -rf /app/lib",
                     "rm -rf /app/src",
                      rm -rf /app/share/nautilus-python",
                      "rm /app/bin/dmd",
                      ...

Note that the paths start with /app, which is the prefix that xdg-app apps are put in (to avoid interference with /usr).

After these cleanups, my build weighed less than a megabyte, which is more acceptable.

Trying it out

The best way to distribute an xdg-app is via an OSTree repository. Since I don’t have a good place to put one, and Terminix is developed on github, I decided to turn my xdg-app into a bundle, using this command:

xdg-app build-bundle ~/xdg-app-repos/terminix \
                     terminix.x86_64.xdgapp \
                     com.gexperts.Terminix \ 
                     master

Since github has a concept of releases, I’ve just put the bundle there:

https://github.com/matthiasclasen/terminix/releases/tag/2016-04-15

Enjoy!