xdg-app christmas update

Yesterday I released xdg-app 0.4.6 and I wanted to take some time to talk about what is new in this version what is happening around xdg-app.

libxdg-app and gnome-software integration

In the release, but disabled by default, is a new library called “libxdg-app”. It is intended for applications that want to present a user interface for managing xdg-app applications. We’re working on integrating this with gnome-software so that we can have graphical installation and updating of applications. This is work in progress, and the APIs are not yet stable, but it is very important progress that we will continue working on in the near future.

New xdg-app-builder tool

The basics of how to bundle and application with xdg-app is very simple. You initialize an application directory with build-init. For example:

$ xdg-app build-init appdir 
          org.gnome.Sdk org.gnome.Platform 3.18

This gives you an place where you can both run the build, and store the application being build. Typically you then go to your source directory and run something like:

$ xdg-app build appdir ./configure --prefix=/app
$ xdg-app build appdir make
$ xdg-app build appdir make install

At this point the application is mostly done, but you need to run build-finish in order to export things like desktop files and icons as well as configure some application metadata and permissions, and then export the directory to an ostree repository that your users can install it from:

$ xdg-app build-finish appdir
     --command=run-example --socket=x11
     --share=network --filesystem=host
$ xdg-app build-export appdir /path/to/repo

This is pretty easy, as long as all the tools you need to build your app are in the sdk, and all the dependencies the app needs are in the runtime. However, most apps need a few extra dependencies, which was a large pain point for  people experimenting with xdg-app.

I decided to write a tool that automates this, and thus xdg-app-builder was born. It builds on experience from the Gnome continuous integration system and the nightly xdg-app build work that I did a while ago. Its based on the build-api proposal from Colin Walters, and the idea is to push as much build-knowledge upstream as possible, so that all you need to do is list your dependencies.

Here is an example json manifest that describes the above steps, plus adds a dependency:

  "app-id": "org.example.ExampleApp",
  "version": "master",
  "runtime": "org.gnome.Platform",
  "runtime-version": "3.18",
  "sdk": "org.gnome.Sdk",
  "command": "run-example",
  "finish-args": ["--socket=x11", 
                  "--filesystem=host" ],
  "build-options" : {
    "cflags": "-O2 -g",
    "env": {
        "V": "1"
  "cleanup": ["/include", "*.a"],
  "modules": [
      "name": "some-dependency",
      "config-opts": [ "--disable-something" ],
      "cleanup": [ "/bin" ],
      "sources": [
          "type": "archive",
          "url": "http://someting.org/somethinbg-1.0.tar.xz",
          "sha256": "93cc067b23c4ef7421380d3e8bd7c940b2027668446750787d7c1cb42720248e"
      "name": "example-app",
      "sources": [
          "type": "git",
          "url": "git://git.gnome.org/gimp"

In addition to just building things this will also automatically download tarballs and pull git/bzr repos and clean up and strip things after install. It even has a caching system so that any module that did not change (in the manifest, or in the git repos) will have the results taken from the cache on consecutive builds, rather than rebuilding.

Some people have started using this, including the pitivi and glom developers, and I’ve converted the existing nightly builds of gimp and inkscape to use this instead of the custom scripts that was used before. If you’re interested in playing with xdg-app-builder those links should give you some examples to work from. There is also pretty complete docs in the manpages for xdg-app-builder.

Updated nightly builds

As I mentioned above the nightly builds were converted to xdg-app-builder, but I have also extended the set of builds with Darktable, MyPaint and Scribus, in addition to the old Gimp and Inkscape builds. The scribus build have some issues which I don’t understand (help needed), but the others seem to work well.

If you’re interested in using these, take a look at https://wiki.gnome.org/Projects/SandboxedApps/NightlyBuilds which has instructions on how to get builds of xdg-app for your distro and how to use it to test the nightly builds.

Updated runtime and sdk

Since more people have started testing the Gnome runtimes I’ve fixed quite a few issues that were found in them, as well as added some new tools to the sdk. If you installed the old one, make sure to update it.

Upcoming work

The basic functionality of xdg-app is pretty much there, at least for non-sandboxed applications. The main focus of the work right now is to finish the integration with gnome-software. But after that I will return to work on sandboxing, finishing the work on the file chooser portal and the other APIs required to run apps in a sandboxed fashion.

Native file choosers in Gtk+

Recently I have been working on support for a filechooser portal for sandboxed applications (in xdg-app). The way these work is that the application triggers a file chooser, then the actual user interaction happens outside the sandbox, and the application only recieves the file data after the user finished the interaction.

Ideally something like this would be completely hidden by the toolkit, and the application would just use the regular file chooser APIs. However, the Gtk+ filechooser APIs expose too much details about the file chooser dialog, which means it has to be a regular in-process widget. Unfortunately this means we can’t replace it by an out-of-process dialog.

What we need is a Gtk+ API for the file chooser that hides the details of how the dialog works. At this point I realized that this is something that has been requested a lot in a different context. Such an API would allow us to plug in platform-native file chooser dialogs.

So, I got to work, and today I landed support for native Windows file choosers in Gtk:

win32 file chooser in gtkWe will also look at implementing an OSX version of this to ensure that the APIs work for the common cases.

Now that we have this people can start porting their applications (which is quite trivial in the common case, as the APIs is very similar to the old APIs). And once the apps are ported they will automatically get support for the filechooser portal to make them work in a sandboxed environment.

So, anyone maintaining a Gtk 3 application that want to work better on other platforms, or when sandboxed should take a look at the new GtkFileChooserNative API. Right now it only exists in git, but once there is a new release it should show up in the API docs.

Nightly development builds using xdg-app

When reporting a bug in some software it is often common that the developer asks you to check if the bug is fixed in the latest development version. This makes a lot of sense from the perspective of a developer that gets a lot of bug reports. However, unless you are very experienced with building software this is prohibitively hard.

This is an area where xdg-app shines, because it allows you to create binary builds of desktop applications that work on any distribution. In order to demonstrate this I set up an automated build system that builds Gimp and Inkscape from the development branch every day and produces a new binary that you can easily install and run:

screenshot of app icons
Launching the apps

To make it easy to use I also created packages of xdg-app for some common distributions.

For more information about how to use these builds, see the nightly builds page.

Playing games with runtime extensions

One of the core ideas of xdg-app is that users should be running the same build of everything that the developers tested on. Not only does this mean that you can trust the testing that went into the app, but it also means that an app can run on multiple distributions, and on different version of the same distribution.

However, an application does not have bundle everything. Instead the app specifies a dependency on a runtime, which contains the base system libraries. I like to compare this to dynamic libraries, where xdg-app is “dynamically linked” to its runtime, whereas container systems (like docker) are “statically linked” (by shipping a complete runtime in each app).

This may seem weird and contrary to the first paragraph, but it turns out that this is pretty much a requirement. We want third parties to be able to produce a binary that will keep running “forever”, but the base system may need fixes or support for new hardware. We can’t expect every vendor to rebuild every application (for example some old game) each time something needs fixing on the lower levels. So, therefore we allow updates to the runtime separately from the app (although any update must be compatible).

An app can only depend on one runtime, and everything not in the runtime must be bundled with the application. There is (by design) no way to depend on multiple runtimes, or have runtimes depend on each other. However, there is something called runtime extensions. Extensions are a way to split off and make optional parts of the runtime and recombine them at runtime.

For instance, I’m working on a runtime called org.freedesktop.Platform that has the basic freedesktop libraries (X11, Mesa, DBus, etc). It has this snippet in the configuration:

[Extension org.freedesktop.Platform.Timezones]

This means that whenever another runtime called org.freedesktop.Platform.Timezones is installed its contents will replace the directory share/zoneinfo in the runtime. This is very useful as it allows the timezone info (which changes frequently) to be updated separately from the runtime.

It also has this:

[Extension org.freedesktop.Platform.Locale]

This means that if a runtime like org.freedesktop.Platform.Locale.sv is installed, it will replace the contents of share/runtime/locale/sv in the runtime. During the build all the locale data and translations are separated out into per-language runtimes which can be installed separately.

And finally, it has:

[Extension org.freedesktop.Platform.GL]

There is no (official) runtime with this name, but if one is installed it will appear in lib/GL, and the main runtime has been programmed to look into this directory for libGL.

The idea here is that if your system uses an OpenGL driver that does not ship with the regular runtime (i.e. Mesa), or needs a more recent version of it, then you can create your own runtime with this name and get your drivers into the runtime.

For a long time this has been a theoretical solution, but recently I acquired an NVidia card in order to test this. The result is this script, which takes an upstream nvidia driver release and converts it into a runtime that matches the (soon to be released) 1.2 version of the Freedesktop runtime.

To verify that it works I created an xdg-app bundle for the Unreal Editor. Here are some screenshots of a sandboxed version of unreal to show this working:

Launching the unreal editor
Launching the unreal editor
Editing the sample project
Editing the sample project

While runtimes can’t have dependencies on other runtimes, they can be build from the same base, and thus be compatible. For instance, the official Gnome runtime takes the Freedesktop runtime and adds the Gnome modules to it. Since we have the same ABI  we can reuse the same extensions. The two runtimes have different versions (Gnome is 3.18, Freedesktop is 1.2), so we have to specify the version (which is otherwise infered by the runtime version). Here is how the Gnome runtime config looks:

[Extension org.freedesktop.Platform.GL]

I hope to have stable builds out of the Freedesktop 1.2 and Gnome 3.18 runtimes shortly, so that other people can play with them. Unfortunately I’m not allowed to distribute the unreal editor app.

xdg-app moving to freedesktop.org

For anyone following the development of xdg-app, all development have now moved to freedesktop.org. Here is where things are happening now:

Testing rawhide apps using xdg-app

An important aspect of xdg-app is application sandboxing, which will require application changes to use sandbox-specific APIs. However, xdg-app is also a good way to deploy and run non-sandboxed (or partially sandboxed) regular applications.

A very interesting usecase for this is to have an image-based operating system, for instance a Workstation spin of Fedora Atomic. Such a system would have a basic workstation installation with a read-only /usr, and atomic updates/rollback. However, installing an application is painful, and customizing yor install in that way undoes many of the advantages of an image-based OS.

With xdg-app you can install apps into /var (or $HOME) and have them fully integrate with the system, while still being isolated from changes to the host. This makes for a great combination, just like atomic + docker is a good combination for the server space.

I’ve spent some time recently making a prototype runtime based on the Fedora packages, as reported on the desktop list. This is kind of interesting as it lets you test applications from rawhide on fedora 21 or 22. Just install xdg-app from fedora-updates and then install the runtime:

$ xdg-app add-remote --no-gpg-verify --user fedora http://fedorapeople.org/~alexl/repo/
$ xdg-app install-runtime --user fedora org.fedoraproject.Platform 23

And then you can try gedit 3.17.0:

$ xdg-app install-app --user fedora org.gnome.gedit
$ xdg-app run org.gnome.gedit

Or evince 3.17.2:

$ xdg-app install-app --user fedora org.gnome.evince
$ xdg-app run org.gnome.evince

Once installed you can also just start them from the desktop environment as usual.  They should be there like any regular application as the desktop files and icons are exported to the host.

Official GNOME SDK runtime builds are out

As people who have followed the work on sandboxed applications know, we have promised a developer preview for GNOME 3.16. Well, 3.16 has now been released, so the time is now!

I spent last week setting up an build system on the GNOME infrastructure, and the output of this is finally available at:


This repository contains the gnome 3.16 runtimes, org.gnome.Platform, as well as a smaller one that is useful for less integrated apps (like games) called org.freedesktop.Platform. It also has corresponding develoment runtimes (org.gnome.Sdk and org.freedesktop.Sdk) that you can use to create applications for the platforms.

This is a developer preview, so consider these builds weakly supported. This means I will try to keep them somewhat updated if there are major issues and that I will keep them API and ABI stable. I will probably also pick up at least some 3.16.x minor releases as they are released.

I also did the first official release of xdg-app. For easy testing this is available for Fedora 21 and 22 as a copr repo.

Testing the SDK

Using the repo above makes it really easy to test this. Just install the xdg-app package from copr, log out+in (needed update the environment for the session), then follow these instructions (as a regular user):

  1. Install the Gnome SDK public key into  /usr/share/ostree/trusted.gpg.d, (or alternatively, use –no-gpg-verify when you add the remote below).
  2. Install the basic Gnome and freedesktop runtimes:
    $ xdg-app add-remote --user gnome-sdk http://sdk.gnome.org/repo/
    $ xdg-app install-runtime --user gnome-sdk org.gnome.Platform 3.16
    $ xdg-app install-runtime --user gnome-sdk org.freedesktop.Platform 1.0
  3. Optionally install some locale packs:
    $ xdg-app install-runtime --user gnome-sdk org.gnome.Platform.Locale.se 3.16
    $ xdg-app install-runtime --user gnome-sdk org.freedesktop.Platform.Locale.se 1.0
  4. Install some apps from my repository of test apps:
    $ xdg-app add-remote --user --no-gpg-verify test-apps https://people.gnome.org/~alexl/test-apps/repo/
    $ xdg-app install-app --user test-apps org.gnome.gedit
    $ xdg-app install-app --user test-apps org.freedesktop.glxgears
  5. Run the apps! You should find gedit listed among the regular applications in the shell as it exports a desktop file. But you can also run them manually like this:
    $ xdg-app run org.gnome.gedit
    $ xdg-app run org.freedesktop.glxgears
  6. I also packaged the latest gnome builder from git. It requires the full sdk which takes a bit longer to download:
    $ xdg-app install-runtime --user gnome-sdk org.gnome.Sdk 3.16
    $ xdg-app install-app --user test-apps org.gnome.Builder

All the above install the apps into your home-directory (in ~/.local/share/xdg-app) . You can also run the commands as root and skip the –user arguments to do system-wide application installs.

Future work

With the basics now laid down to run current applications in a minimally isolated environment the next step is to work on the sandboxing aspects more. This will require lots of work, both in the system side (things like kdbus), the desktop (add sandbox aware APIs, make pulseaudio protect clients from each other, etc)  and in modifying applications.

If you’re interested in this, you can follow the work on the wiki.

Building your own apps

If you download the SDKs you have enough tooling to build your own applications. There are some documentations on how to do this here.

I also created a git repository with the scripts I used to build the test applications above. It uses the gnome-sdk-bundles repostory which has some tooling and specfiles to easily bundle dependencies with the application.

Building the SDK

If you ever want to build the SDK yourself, it is available at:


This repository contains the desktop specific parts of the SDK, which is layered on a core Yocto layer. When you build the SDK this will be automatically checked out and built from:


However, if you don’t want to build all of this you can download the pre-build images from http://sdk.gnome.org/images/x86_64/ and put them in the freedesktop-sdk-base/images/x86_64 subdirectory of gnome-sdk-images. This can save you a lot of time and space.

First fully sandboxed Linux desktop app

Its not a secret that I’ve been working on sandboxed desktop applications recently. In fact, I recently gave a talk at devconf.cz about it. However, up until now I’ve mainly been focusing on the bundling and deployment aspects of the problem. I’ve been running applications in their own environment, but having pretty open access to the system.

Now that the basics are working it’s time to start looking at how to create a real sandbox. This is going to require a lot of changes to the Linux stack. For instance, we have to use Wayland instead of X11, because X11 is impossible to secure. We also need to use kdbus to allow desktop integration that is properly filtered at the kernel level.

Recently Wayland has made some pretty big strides though, and we now have working Wayland sessions in Fedora 21. This means we can start testing real sandboxing for simple applications. To get something running I chose to focus on a game, because they require very little interaction with the system. Here is a video I made of Neverball, running in a minimal sandbox:

In this example we’re running a regular build of neverball in an environment which:

  • Is independent of the host distribution
  • Has no access to any system or user files other than the ones from the runtime and application itself
  • Has no access to any hardware devices, other than DRI (for GL rendering)
  • Has no network access
  • Can’t see any other processes in the system
  • Can only get input via Wayland
  • Can only show graphics via Wayland
  • Can only output audio via PulseAudio
  • … plus more sandboxing details

Yet the application is still simple to install and integrates nicely with the desktop. If you want to test it yourself, just follow the instructions on the project page and install org.neverball.Neverball.

Of course, there are still a lot to do here. For instance, PulseAudio doesn’t protect clients from each other, and for more complex applications we need to add new APIs to safely grant access to things like user files and devices. The sandbox details page has a more detailed list of what has to be done.

The road is long, but at least we have now started our journey!

Back from DevX hackfest

I’m now back from a week in Cambridge at the developer experience hackfest. This was a great event, it was a lot of fun to meet people again, and we got a lot of things done. I spent a lot of time talking to people about things related to xdg-app and sandboxed applications, both spreading information and actually implementing features.

I spent some time with Emmanuele, Ryan and Lars working on glib stuff, which resulted in the G_DECLARE_*_TYPE macros finally being merged. Additionally I reviewed the new list model abstraction which I hope we can land soon, and Ryan and I worked out a new fancy __attribute__(cleanup) approach that we hope to merge into glib soon.

We also worked a bit on Gtk+ OpenGL support. Based on feedback from early users we’re doing some changes in how GL contexts are created to allow you to configure them in more detail. We also decided that we want to completely drop support for legacy OpenGL contexts, as these had issues cooperating with Core 3.2 contexts, and because we don’t live in the 90s anymore. Carlos was working on converting GtkPopover to use (override redirect) toplevels on X11, and I gave him moral support and generally hated on ancient crappy X11 behaviour.

Props to Collabora and Philip for arranging a great event!

Introducing Gthree

I’ve recently been working on OpenGL support in Gtk+, and last week it landed in master. However, the demos we have are pretty lame and are not very good to show off or even test the OpenGL support. I’ve looked around for some open source demos that used modern GL that we could use, but I didn’t find anything that we could easily use.

What I did find though, was a lot of WebGL demos that used three.js. This looked like a very nice open source library for highlevel 3d rendering. At first I had some plans to bind OpenGL to gjs so that we could run three.js, but this turned out to be a hard.

Instead I started converting three.js into C + GObject, using the Gtk+ OpenGL support and the vector/matrix library graphene that Emmanuele has been working on recently.

After about a week of frantic hacking it is now at a stage where it may be interesting for others. So, without further ado I introduce:


It does not yet support everything that three.js can do, but it does support a meshes with most mesh matrial types and lighting, including a loader for the json model format of thee.js, which means that it is minimally useful.

Here are some screenshots of the examples that ships with the code:

Screenshot from 2014-10-24 15:04:47
Various types of materials
Screenshot from 2014-10-24 15:10:00
Some sample models from three.js examples
Screenshot from 2014-10-24 15:31:40
Some random cubes

This has been a lot of fun to work on as I’ve seen a lot of progress very fast. Mad props to mrdoob and the other three.js developers for creating three.js and making it free software. Gthree is a huge rip-off of their work and would never be possible without it. Thanks also to Emmanuele for his graphene library.

What are you sitting here for, go ahead and play with it! Make some demos, port some more three.js features, marvel at the fancy graphics!