Despite having been a contributor to the GNOME project for almost 5 years now (first at Red Hat and now at Endless), I’ve never found the time to blog about my work. Fortunately in many cases collaborators have made posts or the work was otherwise announced. Now that Endless is a non-profit foundation and we are working hard at advocating for our solutions to technology access barriers in upstream projects, I think it’s an especially good time to make my first blog post announcing a recent feature in Flatpak, which I worked on with a lot of help from Alex Larsson.
On many low-end computers, persistent storage space is quite limited. Some Endless hardware for example has only 32 GB. And we want to fill much of it with useful content in the form of Flatpak apps so that the computers are useful even offline. So often in the past we have shipped computers that are already quite full before the user stores any files. Ideally we want that limited space to be used as efficiently as possible, and Flatpak and OSTree already have some neat mechanisms to that end, such as de-duplicating any identical files across all apps and their runtimes (and, in the case of Endless OS, including the OS files as well).
(For the uninitiated a runtime is basically a set of libraries that can be shared between Flatpak apps, and which the apps use at run-time.)
However, there’s room for improvement. In Flatpak versions prior to 1.9.1 (1.9.x is currently the unstable series), runtimes are, broadly speaking, not uninstalled when the last app using them is uninstalled or updated to use a newer runtime. In some special cases such as locale extensions runtimes are uninstalled, but the main runtimes such as the GNOME or KDE ones that take up the most space are left behind unless manually uninstalled. And those runtimes can take up a significant amount of disk space:
$ du -sh ~/.local/share/flatpak/runtime/org.gnome.Platform/x86_64/3.38 890M /home/mwleeds/.local/share/flatpak/runtime/org.gnome.Platform/x86_64/3.38 $ du -sh ~/.local/share/flatpak/runtime/org.kde.Platform/x86_64/5.14 969M /home/mwleeds/.local/share/flatpak/runtime/org.kde.Platform/x86_64/5.14
This does have a significant advantage: in case the runtime is needed again in the future it will not have to be re-downloaded. But ultimately it is not a good situation to have the user’s disk space increasingly taken up by unneeded Flatpak runtimes as their apps migrate to newer runtimes, with no way for non-technical users to remedy the situation.
For a while now Flatpak has had the ability to remove unused runtimes with the command flatpak uninstall –unused. But users should never need to use the command line to keep their computer running well. And users who choose to use the command line already run flatpak update regularly, so in the new implementation removing unused runtimes is integrated into the update command (in addition to happening behind-the-scenes in GNOME Software for GUI-only users).
A compromise was chosen between removing all unused runtimes and always leaving them installed, which is to remove unused runtimes which have been marked End Of Life on the server side, on the basis that such runtimes are unlikely to be needed again in the future. Of course for this to work properly, runtime publishers must properly set the EOL metadata when appropriate, as was recently fixed on Flathub. So please do so if you maintain any runtimes!
I’ve glossed over it so far but actually defining when a runtime is unused is not trivial: a runtime in the system installation may be used by an app in the current user’s per-user installation (which Flatpak can detect), a runtime in the system installation may be used by an app in another user’s per-user installation (which Flatpak cannot detect), and a runtime may be used for development purposes. For this latter case the current implementation offers two solutions: one can prevent a runtime from being automatically uninstalled by pinning it with the flatpak pin command. Additionally, runtimes that are manually installed (as opposed to being pulled in to satisfy a dependency requirement) are automatically pinned.
You can check if you have any pinned runtime patterns (the command accepts globs in addition to precise runtimes) by just executing flatpak pin without any arguments.
Long story short, with the upcoming releases of Flatpak 1.10 and GNOME Software 40, both will remove unused EOL runtimes during update operations and uninstall operations, freeing up disk space for users. If you maintain a software manager that supports Flatpak, you may consider using the new API to ensure unused runtimes are regularly cleaned up.
There is one improvement I’d like to make for this feature: we could take filesystem access time information into account when determining if a runtime is unused (perhaps removing a runtime that hasn’t been executed in a year?). But that is for another day…
Thanks for that explanation, I was googling in how to do housekeeping with Linux in general and flatpaks too.
I just wish these housekeeping routines would be better implemented in Linix distros in general, for example journal logs which by default take up 10% or 4 GB on a desktop(!) OS.
Or old kernels .. they always pile up. Up until very recently not even apt autoremove worked under Ubuntu based distros, because kernels installed by update were falsely marked as manually installed.
The only distro that actually does autoremove kernels is Mint iirc.
It’s a mess often, even Windows cleans itself better these days.
/rant lol
Btw: You spoke about Flatpak 1.1 together with gnome, but I don’t NEED gnome 40 for the changes to take effect or? I have Kubuntu with Flatpaks installed too.
No, you don’t need GNOME 40. On Kubuntu the Flatpak 1.10+ command line would still remove unused runtimes, and KDE Discover also has some code for the same purpose.
Cool, thanks!
What about the other distros not using Gnome or KDE, do they all have to implement their own code for removal in their updater apps? That seems like a lot of work.
If that is the case, wouldnt it be better to let that be handled by the flatpak service itself, together with updates?
Yes, any graphical front-end to Flatpak would have to tell libflatpak to clean up unused runtimes in order for ones to be cleaned up that aren’t directly made unused by a transaction (those are cleaned up no matter what), as in https://gitlab.gnome.org/GNOME/gnome-software/-/merge_requests/525
There is no Flatpak service that runs indepedently of any front-end (graphical or CLI). If a distro wants automatic updates, something has to implement that functionality.
I have never found flatpak uninstall to flatpak uninstall –unused to do anything. Instead I’ve found that usually any app installed as a flatpak either requires some outdated runtime, or just clings to current version and prevents it from being removed when a newer update is installed. So in the end we end up with gigabytes worth of trash.