Category Archives: Uncategorized

Flatpak in detail, part 2

The first post in this series looked at runtimes and extensions. Here, we’ll look at how flatpak keeps the applications and runtimes on your system organized, with installations, repositories, branches, commits and deployments.

Installations and repositories

An installation is a place on your filesystem where flatpak can install apps and runtimes. By default, flatpak has a system-wide installation in /var/lib/flatpak, and a user installation in $HOME/.local/share/flatpak.

It is possible to define additional system-wide installations by placing a key file in /etc/flatpak/installations.d. For example, this can be used to keep apps on a portable drive.

Part of the data that flatpak keeps for each installation is a list of remotes. A remote is a reference to an ostree repository that is available somewhere on the network.

Each installation also has its own local ostree repository (for example, the system-wide installation has its repo in /var/lib/flatpak/repo). You can explore the contents of these repositories using the ostree utility;

$ ostree --repo=$HOME/.local/share/flatpak/repo/ refs

Branches and versions

Similar to git, ostree organizes the data in a repository in commits, which are grouped in branches. Commits are identified by a hash and branches are identified by a name.

While ostree does not care about the format of a branch name, flatpak uses branch names of the form $KIND/$ID/$ARCH/$BRANCH to uniquely identify branches.

Here are some examples:


Most of the  time, it is clear from the context if an app or runtime is being named, and only one architecture is relevant. For this case, flatpak allows a shorthand notation for branch names omitting the $KIND and $ARCH parts: $ID//$BRANCH.

In this notation, the above examples shrink to:



Installing an app or runtime really consists of two steps: first, flatpak caches that data in the local repo of the installation, and then it deploys it, which means it creates a check-out of the branch from the local repo. The check-outs are organized in a folder structure that reflects the branch name organization.

For example, Inkscape will be checked-out in $HOME/.local/share/flatpak/app/org.inkscape.Inkscape/x86_64/stable/$COMMIT, where $COMMIT is the hash of the commit that is being deployed.

It is possible to have multiple commits from the same branch deployed, but one of them is considered active and will be used by default. Flatpak maintains symlink in the check-out directory that points at the active commit.

It is also possible to have multiple branches of an app or runtime deployed at the same time; the directory structure of checkouts is designed to allow that. One of the branches is considered current. Flatpak maintains a symlink at the toplevel of the checkout that points at the current checkout.

Flatpak can run an app from any deployed commit, regardless whether it is active or current or not. To run a particular commit, you can use the –commit option of flatpak run.

The relevance of being active and current is that flatpak exports some data (in particular, desktop files) from the active commit of the current branch, by symlinking it into ~/.local/share/flatpak/exports,  where for example gnome-shell will find it and allow you to run the app from the overview.

Note: Even though it is perfectly ok to have multiple versions of the same app installed, running more than one at the same time will typically not work, since the different versions will claim the app ID as their unique bus name on the session bus. A way around this limitation is to explicitly give one of the versions a different ID, for example, by appending a “.nightly” suffix.

Application data

One last aspect of filesystem organization to mention here is that every app that is run with flatpak gets a some filesystem space to use for permanent storage. This space is in $HOME/.var/app/$ID, and it has subdirectories called cache, config and data. At runtime, flatpak sets the XDG_CACHE_DIR, XDG_CONFIG_HOME and XDG_DATA_HOME environment variables to point at these directores.

For example, the persistent data from the inkscape flatpak can be found in $HOME/.var/app/org.inkscape.Inkscape.


Flatpak installations may look a bit intimidating with their deep diretory tree, but they have a well-defined structure and this post hopefully helps to explain the various components.

Flatpak in detail

At this point, Flatpak is a mature system for deploying and running desktop applications. It has accumulated quite some sophistication over time, which can make it appear more complicated than it is.

In this post, I’ll try to look in depth at some of the core concepts behind Flatpak, namely runtimes and extensions.

In the beginning: bundles

At its very core, the idea behind Flatpak is to bundle applications with their dependencies, and ship them as a self-contained unit.  There are good reasons that bundling is attractive for application developers:

  • There is a much bigger chance that the app will run on an arbitrary end-user system, which may have different versions of libraries, different themes, or a different kernel
  • You are not relying on all the different update mechanisms and policies of Linux distributions
  • Distribution updates to your dependencies will not break your app behind your back
  • You can test the same code that your users run

Best of both worlds: runtimes

In the age-old debate beween bundlers and packagers, there are good arguments on both sides. The usual arguments against bundling are:

  • Code duplication. If a library gets hit by a security issue you have to fix it in all the apps that bundle it
  • Wastefulness. if every app ships an entire library stack, this blows up the required bandwidth for downloads and the required disk space for installing them

With this in mind, Flatpak early on introduced the concept of runtimes. The idea behind runtimes is that many desktop applications use a deep library stack, but it is often a similar set of libraries. Therefore, it makes sense to take these common library stacks and distribute them separately as “GNOME runtime” or “KDE runtime”, and have apps declare in their metadata which runtime they need.

It then becomes the responsibility of the flatpak tooling to assemble  the applications filesystem tree with the runtimes filesystem tree when it creates the sandbox environment that the app runs in.

To avoid conflicts, Flatpak requires that the applications filesystem tree is rooted in /app, while runtimes have a traditional /usr tree.

Splitting off runtimes preserves most of the benefits that I outlined for bundles, while greatly reducing code duplication and letting us update libraries independently of applications.

Of course, it also brings back some of the risks of modularity: updating the libraries independently carries, once again, the risk of breaking the applications that use the runtime. So the team maintaining a runtime has to be very careful to avoid introducing problematic changes or incompatibilities.

Going further: extensions

As I said, shipping runtimes separately saves a lot of bandwidth, since the runtime has to be downloaded only once for all the applications that share it. But a runtime is still a pretty massive download, and contains a lot of things that may not be useful most of the time or are just optional.

A good examples for this are translations. It is not uncommon for desktop apps to be translated in 50 locales. But the average user will only ever use a single one of these. In traditional packaging, this is sometimes addressed by breaking translations out as “lang packs” that can be installed separately.

Another example is debug information. You don’t need symbols and other debug information unless you encounter a crash and want to submit a meaningful stacktrace. In traditional packaging, this is addressed by splitting off “debuginfo” packages that can be installed when needed.

Flatpak provides a mechanism  to address these use cases.  Runtimes (and applications too) can declare extension points, which are designated locations in their filesystem tree where additional runtimes can be mounted. These additional runtimes are called extensions. When constructing a sandbox for running an app, flatpak tooling will look for matching extensions and mount them at the right place.

Flatpak is not a generic solution, but tailored towards the use case of desktop applications, and it tries to do the the right thing out of the box: flatpak-builder automatically breaks out .Locale and .Debug extensions when building apps or runtimes, and when installing things, flatpak installs the matching .Locale extension. But it goes beyond that and only installs the subset of it that is relevant for the current locale, thereby recreating the space-saving effect of lang packs.

Extensions: infinite variations

The extension mechanism is flexible enough to cover not just locales and debuginfo, but all sorts of other optional components that applications might need. To give just some examples:

  • OpenGL drivers that match the GPU
  • Other hardware-specific APIs like vaaapi
  • Media codecs
  • Widget themes

All of these can be provided as extensions. Flatpak has the smarts built-in to know whether a given OpenGL driver extension matches the hardware or whether a given theme extension matches the current desktop theme, and it will automatically install and use matching extensions.

At last: the host OS

The examples in the previous paragraph are realized as extensions because the shared objects or theme components need to match the runtime they are used with.

But some things just don’t change very much over time, and don’t need exact matching against the runtime to be used by applications. Examples in this category are fonts, icons or certificates.

Flatpak makes these components from the host OS available in the sandbox, by mounting them below /run/host/ in the sandbox, and appending /run/host/share to the XDG_DATA_DIRS environment variable.


Flatpak does a lot of hard work behind the scenes to ensure that the apps it runs find an environment that looks similar to a traditional Linux desktop, by combining the application, its runtime, optional extensions and host components.

The Flatpak documentation provides more information about working with Flatpak as an application developer.

Fedora Atomic Workstation → Team Silverblue

I have been writing a long series of posts about my experience with
Fedora Atomic Workstation, which I’ve enjoyed it quite a bit. But all good things must come to an end. So, no more Atomic Workstation for me …since we’re renaming it to Team Silverblue.

Team Silverblue logoWith this project, we want to take the Atomic Workstation as it is today, and turn into something that is not just cool for the few who know it, but useful for everybody who has a need for a Workstation.

There is still some work to be done to reach that goal. We have some ideas what is needed, but as with all open projects, those who show up to build it get a chance to shape the end product.

So, come and join us in Team Silverblue!

Fedora Atomic Workstation: Getting comfortable with GNOME Builder

Note: Fedora Atomic Workstation has recently been renamed to Team Silverblue. Learn more here.

I am still going with my attempt to use Fedora Atomic Workstation fulltime as my main  development system. As part of that, I am figuring out how to do GTK+ development on an immutable OS, using GNOME Builder.

As I’ve explained in a previous post, one thing I figured out early on is that while the flatpak support in GNOME Builder is optimized for desktop applications, you can use it for other things. In my case, that means developing GTK+.

Build Configurations

GNOME Builder knows what and how to build via its build configurations. It can have multiple such configurations for each project – the Build Preferences page has a list of them. In the flatpak case, build configurations correspond more or less 1:1 to flatpak manifests that are stored as json files in the source tree. Last time, I created one that launches the  gtk4-demo application.

Currently, creating a new build configuration means copying an existing flatpak manifest to a new file, but in  3.28, GNOME Builder will support copying build configurations directly in the Build Preferences.


Influencing GTK+ at runtime for debugging purposes is often done via environment variables, such as GTK_DEBUG for controlling the debug output. Unfortunately, setting environment variables is not the most obvious in GNOME Builders flatpak support, currently.

There is an Environment section in the Build Preferences, but it sets environment variables at build time, not in the runtime sandbox that gtk4-demo will run in. GNOME Builder currently has no UI for setting up the runtime environment. We can still do it, by manually adding –env options in the finish-args section of the flatpak manifest:

"finish-args" : [

This is hardly elegant, and it has the big downside that any change to the flatpak manifest will cause GNOME Builder to rebuild the project, even if the change only affects the runtime setup, as is the case here. I hope that GNOME Builder will gain proper runtime setup UI at some point.


But maybe it would be more natural to get a command prompt and set environment variables as usual, while launching apps from there.

GNOME Bulder can actually give you a terminal, with Ctrl-Alt-Shift-T, but it is a terminal in the build sandbox, not the runtime sandbox.

Thankfully, we can help ourselves in the same way as before: Just create another build configuration, this time using “sh” as the command – that gives us a shell inside the runtime sandbox, and we can launch test apps from there while setting environment variables.


One aspect of GTK+ that I have worked on in this new setup is module loading. I’ve switched GTK+s printing support to use a GIO extension point, and of course, I wanted to test this before pushing it.

So I was a bit surprised at first that the print dialog in gtk4-demo did not trigger my new module loading code and yet seemed to work just fine. But then I remembered that we are working in a flatpak sandbox now, so GTK+s portal support kicked in and helpfully redirected the print operation to an out-of-process print dialog.

Fedora Atomic Workstation: Theming Possible

Note: Fedora Atomic Workstation has recently been renamed to Team Silverblue. Learn more here.

Today, I decided to try out how themes work with flatpak:ed apps on an Atomic Workstation.

To change themes, I need gnome-tweaks. I already had it installed from the gnome-nightly-apps repository. Unfortunately, it didn’t work, complaining about a missing python package in the runtime. Such is life with nightly builds.

Instead, I used package layering to install a stable version of gnome-tweaks:

$ rpm-ostree install gnome-tweaks

This worked better, and …I found that there are no themes on the Atomic Workstation install, apart from the default ones (Adwaita and HighContrast). So, back to package layering once more. I installed Adapta:

$ rpm-ostree install adapta-gtk-theme

Now I could switch the theme to Adapta in gnome-tweaks:After all this preparation, I was finally ready to see how flatpak:ed apps handle the new theme. But first I wanted to update them:

$ flatpak update
Looking for updates...
Installing for user: org.gtk.Gtk3theme.Adapta/x86_64/3.22 from flathub
[####################] 1 delta parts, 2 loose fetched; 336 KiB transferred in 1 seconds

Neat. Flatpak detected the current GTK+ theme (by reading the gsetting), found a matching runtime extension, and automatically installed it. Flatpak has a similar automatism built-in for matching the host GL drivers and for installing some hardware-specific libraries.

Of course, I could have just manually installed the theme, but this is much more convenient. To see what themes are available on flathub, use:

$ flatpak remote-ls flathub | grep theme

And now, when I run a flatpak app, for example GNOME Characters, it comes up with the Adapta theme:

Lessons learned from this quick experiment: Nightly apps don’t always work, but flatpak can handle themes just fine.

Update: It is probably worth a brief explanation why it is necessary to install the theme twice, on the host and for Flatpak.  With Flatpak, the apps can use different runtimes, and those may contain different version of libraries like GTK+. The host system may be much more stable (think RHEL). So, for every “OS extension” (like fonts, or themes, or drivers) we have to ask ourselves  – will this work across possibly different library versions ? For fonts, the answer we found is yes, font file formats have been stable forever, and it is very unlikely that a font you have on your system will not work with any app or runtime. Therefore, we make the host system fonts available inside flatpak sandboxes – you don’t have to install them again. For themes, we were not so confident, despite GTK+ 3 being stable now, we decided that it is better to have themes match the toolkit version in each runtime.

Fedora Atomic Workstation: Developer Tools, continued

Note: Fedora Atomic Workstation has recently been renamed to Team Silverblue. Learn more here.

Last time, I wrote about using flatpak-builder to do commandline development in a container (namely, in a flatpak sandbox). Flatpak-builder is a pretty versatile and well-documented tool.

Of course, it works well to build desktop apps that already have a flatpak manifest in their git tree. But I have also used it successfully to build and run anything from a library to a session service.

For this reason, I suggested that we should add it to the default Fedora Workstation installation – it is a nice tool to have around.  When the Workstation SIG discussed this idea, it was rightly pointed out that there are quite a few dependencies that flatpak-builder pulls in: git, bzr, svn, meson, autotools, … Not surprising for a meta-build-tool that supports diverse source control and build systems.  But it does make the default install quite a bit heavier.

Maybe some of this can be fixed by turning ‘fringe’ dependencies like svn or bzr into Recommends and make flatpak-builder handle the lack these tools gracefully. But there’s an easier solution here: Just use flatpak! It may not be the premier use case it is designed for, but flatpak can handle commandline apps just fine.

So we created a flatpak for flatpak-builder today, and made it available on flathub. You can get it with:

flatpak install flathub org.flatpak.Builder

As I’ve explained in an earlier post, you can get the familiar command name back by setting up a shell alias:

alias flatpak-builder=org.flatpak.Builder

And, voilá, flatpak-builder works as before! And all its dependencies are in the runtime that it uses (in this case, an SDK). My host system can stay lean and clean, as I like it.

At first I was surprised by this use of flatpak, but then I learned that there are already a few non-graphical applications on flathub. For example, you can install glxinfo from there (under the name org.freedesktop.GlxInfo), and vim is also available on flathub.

I suspect that we might see more commandline tools become available in this form in the future.

Fedora Atomic Workstation: Developer tools

Note: Fedora Atomic Workstation has recently been renamed to Team Silverblue. Learn more here.

A while ago, I wrote about using GNOME Builder for GTK+ work on my Fedora Atomic Workstation. I’ve done this with some success since then. I am using the nightly builds of GNOME Builder from the flatpak repository, since I like to try the latest improvements.

As these things go, sometimes I hit a bug. Recently, I ran into a memory leak that caused GNOME Builder to crash and burn. This was happening just as I was trying to take some screenshots for a blog post. So, what to do?

I figured that I can go back to using the commandline, without giving up the flatpak environment that I’m used to now, by using flatpak-builder, which is a commandline tool to build flatpak applications. In my opinion, it should come out-of-the-box with the Atomic Workstation image, just like other container tools. But  that is not the case right now, so I used the convenient workaround of package layering:

$ rpm-ostree install flatpak-builder

flatpak-builder uses a json manifest that describes what and how to build. GTK+ is shipping manifests for the demo apps in its source tree already, for example this one:

These manifests are used in the GNOME gitlab instance to build testable flatpaks for merge requests, as can be seen here:

This is pretty amazing as a way to let interested parties (designers, translators, everybody) test suggested changes without having to go through a prolonged and painful build process of ever-changing dependencies (the jhbuild experience). You can read more about it in Carlos‘ and Jordan’s posts.

For me, it means that I can just use one of these manifests as input to flatpak-builder to build GTK+:

$ flatpak-builder build \

This produces a local build in the build/ directory, and I can now run commands in a flatpak sandbox that is populated with the build results like this:

$ flatpak-builder --run build \
   build-aux/flatpak/org.gtk.WidgetFactory.json \

A few caveats are in order when you are using flatpak-builder for development:

flatpak-builder will complain if the build/ directory already exists, so for repeated building, you should add the –force-clean option.

The manifest we are using here is referring to the main GTK+ git repository, and will create a clean checkout from there, ignoring local changes in your checkout. To work around this, you can replace the https url pointing at the git repository by a file: url pointing at your checkout:

 "url": "file:///home/mclasen/Sources/gtk"

You still have to remember to create a local commit for all the changes you want to go into the build. I have suggested that flatpak-builder should support a different kind of source to make this a little easier.

Once you have the basic setup working, things should be familiar. You can get a shell in the build sandbox by using ‘sh’ as the command:

$ flatpak-builder --run build \
  build-aux/flatpak/org.gtk.WidgetFactory.json \

flatpak-builder knows to use the sdk as runtime when setting up the sandbox, so tools like gdb are available to you. And the sandbox has access to the display server, so you can run graphical apps without problems.

In the end, I got my screenshots of the font chooser, and this setup should keep me going until GNOME Builder is back on track.

A font update

At the end of march I spent a few days with the Inkscape team, who were so nice to come to the Red Hat Boston office for their hackfest. We discussed many things, from the GTK3 port of Inkscape, to SVG and CSS, but we also spent some time on one of my favorite topics: fonts.

Font Chooser

One thing Tav showed me which I was immediately envious of is the preview of OpenType features that Inkscape has in its font selector.

Clearly, we want something like that in the GTK+ font chooser as well. So, after coming back from the hackfest, I set out to see if I can get this implemented. This is how far I got so far, it is available in GTK+ master.

This really helps understanding which glyphs are affected by a font feature. I would like to add a preview for ligatures as well, but harfbuzz currently does not offer any API to get at the necessary information (understandably — its main focus is applying font features for shaping) and I’m not prepared to parse those font tables myself in GTK+. So, ligatures will have to wait a bit.

Another thing I would like to explore at some point is possible approaches for letting users apply font features to smaller fragments of text, like a headline or a single word. This could be a ‘font tweak’ dialog or panel. If you have suggestions or ideas for this, I’d love to hear them.

At the request of the Inkscape folks, I’ve also explored a backport of the new font chooser capabilities to the 3.22 branch, but since this involves new API, we’re not sure yet which way to go with this.

Font Browser

While doing font work it is always good to have a supply of featureful fonts, so I end up browing the Google web fonts quite a bit.

Recently, I stumbled over a nice-looking desktop app for doing so, but alas, it wasn’t available as a package, and it is written in rust, which I know very little about (I’m hoping to change that soon, but that’s a topic for another post).

But I’ve mentioned this app on #flatpak, and just a few days later, it appeared on flathub, and thus also in GNOME software on my system, just a click away. So nice of the flathub team!

The new flathub website is awesome, btw. Go check it out.

The best part is that this nice little app is now available not just on my bleeding-edge Fedora Atomic Workstation, but also on Ubuntu, Gentoo and even RHEL, thanks to flatpak.  Something we could only dream of a few years ago.


Fedora Atomic Workstation: Beta

Note: Fedora Atomic Workstation has recently been renamed to Team Silverblue. Learn more here.

The Beta release of Fedora 28 is out, and it contains the usual assortment of good stuff:

  • better battery life
  • Thunderbolt support
  • guest additions  make Fedora work better in VirtualBox
  • the latest GNOME release brings polish and new applications
  • updated versions of tons of popular software

The announcement was highlighting all the ways in which you can try it out: there are isos for Workstation, Server and Atomic Host. One thing it forgot to point out is that you can also try Fedora 28 Beta in the form of the Atomic Workstation.

How do you get it ?

To install Fedora Atomic Workstation from scratch, use this iso:

If you already have an Atomic Workstation installation of Fedora 27, you can jump to Fedora 28 Beta with the rpm-ostree rebase command:

# rpm-ostree rebase atomic:fedora/28/x86_64/workstation

This assumes you have already an ostree remote pointing at the Fedora atomic repository. If not, this command creates one:

# ostree remote add --set=gpgkeypath=/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-28-primary atomic
Whats new ?

Besides all the general Fedora 28 news, there are some improvements that are specific to the Atomic Workstation.

The version of GNOME software that is included in the Beta is now able to update the OS image. This is an outcome of our effort to close the feature gaps between the traditional and Atomic Workstation variants. I hope we can close a few more before Fedora 28 is released.

Where to learn more ?

The Project Atomic website has lots of useful information about Atomic and related technologies, such as this Introduction to Atomic Workstation.

And the #atomic irc channel on freenode is a good place to get answers to Atomic Workstation questions. Feel free to come by!

Update: Don’t forget that a fresh Atomic Workstation installation currently comes without any pre-configured flatpak remotes. You need to add one before GNOME Software can show you flatpak apps. The flathub remote can be added by clicking on this link:


Fedora Atomic Workstation: Almost fool-proof

Note: Fedora Atomic Workstation has recently been renamed to Team Silverblue. Learn more here.

I’ve had a little adventure with my Fedora Atomic Workstation this morning and almost missed a meeting because I couldn’t get to a desktop session.
I’ve been using the rawhide branch of Fedora Atomic Workstation to keep up to speed with the latest developments in Fedora. As is expected of rawhide,  recently, it would not get me to a login screen (much less a working desktop session). I’ve just booted back into my working image and ignored this for a few days.

The Adventure begins

But since it didn’t go away by itself, yesterday, I decided to see if I can debug it a bit. Looking at the journal for the last unsuccessful boot gave some hints:

gnome-shell[2934]: Failed to create backend: Failed to initialize renderer: Missing extension for GBM renderer: EGL_KHR_platform_gbm
gnome-session-binary[2920]: WARNING: App 'org.gnome.Shell.desktop' exited with code 1
gnome-session-binary[2920]: Unrecoverable failure in required component org.gnome.Shell.desktop

Poking the nearest graphics team member about this, I was asked to provide the output of eglinfo in this situation. Since I had an hour to spare before the meeting, I booted back into the broken image in runlevel 3, logged in on a vt, … and found that eglinfo is not in the OS image.

Well, thats easy enough to fix on an Atomic system, using package layering:

rpm-ostree install egl-utils

After that, I proceeded to reboot to get to the OS image with the newly added layer, and when I got to the boot prompt, I realized my mistake: rpm-ostree never replaces the booted image, since it (reasonably) assumes that the booted image is ‘working’.  But it only keeps two images around, so it had to replace the other one – which was the image which successfully boots to my desktop.

Now, at the boot prompt, I was faced with the choice between

  • the broken image
  • the broken image + egl-utils

Ugh. Not what I had hoped for. And my meeting starts in 50 minutes. Admittedly, this was entirely my fault. rpm-ostree behaved as it should and as documented. Since it is a snow day, I need to do the meeting from home and need a web browser for that.

So, what can be done? I remembered that ostree is ‘like git for binaries’, so there should be history, right? After some fiddling with the ostree commandline, I found the log command that shows me the history of my local repository. But sadly, the output was disappointing:

$ ostree log fedora/rawhide/x86_64/workstation
commit fa09fd6d2551a501bcd3670c84123a22e4c704ac30d9cb421fa76821716d8c20
ContentChecksum: 74ff34ccf6cc4b7554d6a8bb09591a42f489388ba986102f6726f9e662b06fcb
Date: 2018-03-20 10:27:42 +0000
Version: Rawhide.20180320.n.0
(no subject)

<< History beyond this commit not fetched >>

rpm-ostree defaults to only keeping the latest commit in the local repository, a bit like a shallow git clone. Thankfully, just like git, ostree is versatile, and bit more searching brought me to the pull command, and its –depth option:

# ostree pull --depth=5 onerepo fedora/rawhide/x86_64/workstation

Receiving metadata objects: 698/(estimating) 2.2 MB/s 23.7 MB

This command writes to the local repo in /sysroot/ostree/repo and thus needs to be run as root.

Now ostree log showed a few older commits. I had to bump the depth a few times to find the last working commit. Then, I made that commit available for booting into again, using the depoy command:

# ostree admin deploy 76723f34b8591434fd9ec0

where that hex string is a prefix of the commit ID of the last working commit.  This command also needs to be run as root.

Now a quick reboot, and… the boot loader menu had an entry for the working image again. I made it back to my desktop with 5 minutes to spare before the meeting. Phew!Update: Since you might be wondering, the output of eglinfo was:

eglinfo: eglInitialize failed