Phaedrus Leeds' GNOME Blog

Last month I was fortunate enough to be able to attend GUADEC 2022 in Guadalajara, Mexico, and overall it was a worthwhile experience where I learned a lot. Also it was the 25th anniversary of the GNOME project (give or take a month) and where better to celebrate than in the country where it all began?

My main contribution this year was to give a talk, “Integrating Progressive Web Apps in GNOME”, the recording of which you can find here. The attendance of it and engagement with it was pretty good so hopefully folks found it useful. Now with GNOME 43 around the corner, the revamped web app support should be coming to a distribution near you soon so you can see for yourself in case you haven’t been adventurous enough to try it out in GNOME OS Nightly!

A few days later into the conference and kind of at the last minute (just the day before) I organized a Flatpak/Portals BoF that took place on Saturday. It was hybrid both online and in-person and pretty well-attended in both mediums. As with past such meetings, there’s always much more to discuss than there’s time for, and perhaps that means we should do more of them. But in any case there were some productive discussions on camera portals and some triaging of open issues and pull requests of Flatpak, which is imminently due for a 1.14.0 release.

Other than that I enjoyed several of the presentations; here are some highlights:

• Sebastian Wick’s talk: “HDR? in GNOME?” which taught me a few things about color science and how complicated the problem is of rendering colors properly.
• Federico Mena Quintero’s talk: “Paying technical debt in our accessibility infrastructure” which included some really neat graphs showing codebase evolution (and stagnation) over time.
• Rob McQueen’s talk: “The case for Internet autonomy (and how GNOME can help)”
• Robin Tafel’s keynote: “Peeling Vegetables and the Craft of (Software) Inclusivity”
• Philip Withnall’s talk: “Reworking threading in gnome-software” which indirectly inspired a conversation in the #epiphany channel about the pros and cons of different languages Epiphany might use, the benefits of good support for asynchronous patterns at the language level, and the need for memory safety.
• Tobias Bernard’s talk: “Post-Collapse Computing”, in which there’s a mention of how useful it could be to have support for offline app updates and installs in upstream GNOME Software (since perhaps we won’t always be able to rely on good Internet connections).
• David King’s talk: “Cheese strings: Webcams, PipeWire and portals” in which I learned about the complexities of handling multiple webcams on one device, and that webcams can also have infrared video feeds not just visible light.
• (and many more)

I also was really happy to meet several people I had before only interacted with online; perhaps that’s even the best thing about conferences.

Finally, thank you to the GNOME Foundation for sponsoring my attendance.

In the code review cycles for some of the PWA project PRs I’ve put out, the question has been raised about the proper way to version D-Bus APIs, since I am adding a new D-Bus API to GNOME Web. I learned a couple things and thought I’d share in case it’s useful.

As D-Bus APIs evolve over time and new functionality is added or existing functionality is removed, in most cases you don’t have the luxury of ensuring that all clients are updated in lockstep, so it’s important to think about how existing clients can continue to work without modification, and how clients which are aware of the new functionality can take advantage of it. So there are two types of compatibility to worry about:

1. Backwards compatible changes to the API, which are changes that do not cause existing client code to break. For example, if a method accepts a dictionary of options, each a string key mapped to a value, adding a new supported option key does not cause existing clients to stop working; they just won’t be able to take advantage of the new option. That’s assuming the behavior of the D-Bus service with the new option omitted is the same as it was before the new option was added.
2. Backwards incompatible changes to the API, which are changes that would cause existing client code to break if it’s not updated. For example, changing the name of a method, changing the parameters of a method, or removing a method.

The de facto way to handle backwards compatible API changes, at least in xdg-desktop-portal as well as org.gnome.Mutter.ScreenCast, is to increment an integer Version property when such changes are made (in the portal code the property is not capitalized but the spec recommends capitalization). This allows clients to check at runtime what functionality is available, and only make use of new functionality when the running service has it. This is especially useful for Flatpaks since a Flatpak app needs to be able to run regardless of the version of xdg-desktop-portal or its backends installed on the host; a major benefit of Flatpaks is that they don’t have version requirements for software installed on the host system. This scheme seems to work pretty well for the portal code and its many clients. Here’s an example of how to check the available version of a portal interface in C code.

Per the D-Bus API Design Guidelines, the way to handle backwards incompatible API changes is to leave the existing interface as it is, continue to support it as before, and make a new interface with an incremented number on the end like com.example.MyService2 and use that in the service name, interface name, and object path. While the spec says to add a 1 to the end of the original service name, in practice that is often left off since com.example.MyService2 can just as well follow com.example.MyService as com.example.MyService1. An example of this is the interface org.gnome.ShellSearchProvider2 provided by gnome-shell.

That’s all for now, happy hacking

For various reasons, desktop applications sometimes need to know whether they are running under a sandbox made by a technology such as Flatpak or Snap. Some portal APIs, such as the file chooser dialog, are used transparently so that the application code doesn’t need to make any distinction between the sandboxed and unsandboxed cases, and if you ask me that’s a pretty impressive magic trick on its own. Other portal APIs such as the screencast one are used by both sandboxed and unsandboxed apps thanks to the secure architecture of Wayland compositors. But still other portal APIs are used conditionally depending on whether the app is running sandboxed; this is the case for the OpenURI portal used by Epiphany.

 

It’s also useful for applications to know when they are sandboxed in order to disable features that don’t work (yet) under Flatpak, as Epiphany does for web apps, or to access host resources via sandbox holes, as in the case of GNOME Builder.

 

Currently apps can check for the existence of a /.flatpak-info file to check if they are in a Flatpak sandbox, but this is not good enough to know if they should use portals! Snaps also use the same portals as Flatpaks, and for this reason xdg-desktop-portal has code to detect if the calling process is running as a Snap by checking cgroups membership. However since this check is not trivial it is not ideal for every app to keep a separate copy of it, so I submitted a patch to add API to libportal with some helper functions. Once that is merged apps can use it to easily check their sandboxed status.

 

As a side note, the /.flatpak-info mechanism for identifying a process as untrusted to a portal is perhaps not ideal. It was at the heart of a recent security vulnerability and it is used by WebKit’s UI process so it is treated as untrusted even when not running under Flatpak, which seems like a potential source of confusion/bugs. Perhaps an area for improvement if anyone has ideas?

 

I found this issue while working on Epiphany as part of the effort to improve the support for Progressive Web Apps in GNOME (which I just started this week!) and I’ll submit a patch shortly to make Epiphany’s sandbox detection Snap-friendly.

Despite having been a contributor to the GNOME project for almost 5 years now (first at Red Hat and now at Endless), I’ve never found the time to blog about my work. Fortunately in many cases collaborators have made posts or the work was otherwise announced. Now that Endless is a non-profit foundation and we are working hard at advocating for our solutions to technology access barriers in upstream projects, I think it’s an especially good time to make my first blog post announcing a recent feature in Flatpak, which I worked on with a lot of help from Alex Larsson.

On many low-end computers, persistent storage space is quite limited. Some Endless hardware for example has only 32 GB. And we want to fill much of it with useful content in the form of Flatpak apps so that the computers are useful even offline. So often in the past we have shipped computers that are already quite full before the user stores any files. Ideally we want that limited space to be used as efficiently as possible, and Flatpak and OSTree already have some neat mechanisms to that end, such as de-duplicating any identical files across all apps and their runtimes (and, in the case of Endless OS, including the OS files as well).

(For the uninitiated a runtime is basically a set of libraries that can be shared between Flatpak apps, and which the apps use at run-time.)

However, there’s room for improvement. In Flatpak versions prior to 1.9.1 (1.9.x is currently the unstable series), runtimes are, broadly speaking, not uninstalled when the last app using them is uninstalled or updated to use a newer runtime. In some special cases such as locale extensions runtimes are uninstalled, but the main runtimes such as the GNOME or KDE ones that take up the most space are left behind unless manually uninstalled. And those runtimes can take up a significant amount of disk space:

$ du -sh ~/.local/share/flatpak/runtime/org.gnome.Platform/x86_64/3.38
890M /home/mwleeds/.local/share/flatpak/runtime/org.gnome.Platform/x86_64/3.38


$ du -sh ~/.local/share/flatpak/runtime/org.kde.Platform/x86_64/5.14
969M /home/mwleeds/.local/share/flatpak/runtime/org.kde.Platform/x86_64/5.14

This does have a significant advantage: in case the runtime is needed again in the future it will not have to be re-downloaded. But ultimately it is not a good situation to have the user’s disk space increasingly taken up by unneeded Flatpak runtimes as their apps migrate to newer runtimes, with no way for non-technical users to remedy the situation.

For a while now Flatpak has had the ability to remove unused runtimes with the command flatpak uninstall –unused. But users should never need to use the command line to keep their computer running well. And users who choose to use the command line already run flatpak update regularly, so in the new implementation removing unused runtimes is integrated into the update command (in addition to happening behind-the-scenes in GNOME Software for GUI-only users).

A compromise was chosen between removing all unused runtimes and always leaving them installed, which is to remove unused runtimes which have been marked End Of Life on the server side, on the basis that such runtimes are unlikely to be needed again in the future. Of course for this to work properly, runtime publishers must properly set the EOL metadata when appropriate, as was recently fixed on Flathub. So please do so if you maintain any runtimes!

I’ve glossed over it so far but actually defining when a runtime is unused is not trivial: a runtime in the system installation may be used by an app in the current user’s per-user installation (which Flatpak can detect), a runtime in the system installation may be used by an app in another user’s per-user installation (which Flatpak cannot detect), and a runtime may be used for development purposes. For this latter case the current implementation offers two solutions: one can prevent a runtime from being automatically uninstalled by pinning it with the flatpak pin command. Additionally, runtimes that are manually installed (as opposed to being pulled in to satisfy a dependency requirement) are automatically pinned.

You can check if you have any pinned runtime patterns (the command accepts globs in addition to precise runtimes) by just executing flatpak pin without any arguments.

Long story short, with the upcoming releases of Flatpak 1.10 and GNOME Software 40, both will remove unused EOL runtimes during update operations and uninstall operations, freeing up disk space for users. If you maintain a software manager that supports Flatpak, you may consider using the new API to ensure unused runtimes are regularly cleaned up.

There is one improvement I’d like to make for this feature: we could take filesystem access time information into account when determining if a runtime is unused (perhaps removing a runtime that hasn’t been executed in a year?). But that is for another day…