Fitting Endless OS images on small disks

Last week I read Jorge Castro’s article On “Wasting disk space” with interest, and not only because it cites one of my own articles 😉. Jorge encouraged me to write up the conversation we had off the back of it, so here we go!

People like to fixate on the disk space used by installing a calculator app as a Flatpak when you don’t have any other Flatpak apps installed. For example, on my system GNOME Calculator takes up 9.3 MB for itself, plus 803.1 MB for the GNOME 42 runtime it depends on. Regular readers will not be surprised when I say that that 803.1 MB figure looks rather different when you realise that Calculator is just one of 70 apps on my system that use that runtime; 11.5 MB of runtime per app feels a lot more reasonable.

But I do have one app installed which depends on the GNOME 3.34 runtime, which has been unsupported since August 2020, and the GNOME 3.34 runtime only shares 102 MB of its files with the GNOME 42 runtime, leaving 769 MB installed solely for this one 11 MB app. Not such a big deal on my laptop with a half-terabyte drive, but this gets to a point Jorge makes near the end of his article:

Yes, they take up more room, but it’s not by much, and unless you’re on an extremely size-restrained system (like say a 64GB Chromebook you are repurposing) then for most systems it’s a wash.

Part of the insight behind Endless OS is that it can be more practical and cost-effective to fill a large hard disk with apps and educational resources than to get access to high-bandwidth connectivity in remote or disadvantaged communities, and the devices we and our deployment partners use generally have plentiful storage. But inexpensive, lower-end devices with 64 GB storage are still quite common, and having runtimes installed for one or two apps consumes valuable space that could be used for more offline content. So this can be a real problem for us, if a small handful of apps are bloating an image to the point where it doesn’t fit, or leaves little space for documents & updates.

So, our OS image builder has a mode to tabulate the apps that will be preinstalled in a given image configuration, grouped by the runtime they use, with their approximate sizes. I added this mode when I was trying to make an Endless OS ISO fit on a 4.7 GB DVD a few years back, packing in as much content as possible. It’s a bit rough and ready, and tends to overestimate the disk space needed (because it does not take into account deduplication identical files between different apps and runtimes, or installing only a subset of translations for an app or runtime) but it is still useful to get a sense of where the space is going.

I ran this for the downloadable English configuration of Endless OS earlier today. I’ll spare you the full output here but if you are curious it’s in this gist. It shows that, like on my system, TurtleBlocks is the only reason the GNOME 3.34 runtime is preinstalled, and so removing that app or updating it to a more modern runtime would probably save somewhere between 1 GB and 2 GB in the image! Normally after seeing this I would go and see if I can take a few minutes to send an update to the app, but in this case someone has beaten me to it. (If you are a Python expert interested in block-based programming tools for kids, why not lend a hand getting this over the line?) It also shows that a couple of our unmaintained and sadly closed-source first-party apps, Resumé and My Budget, are stuck on an even more prehistoric runtime.

Running it on a deployment partner’s custom image that they reported is too big for the 64 GB target hardware showed that the Othman Quran browser is also using an older runtime; once more, someone else has already noticed in the last couple of days, and if you are an expert on GTK’s font selection your input would be welcome on that pull request.

This tool is how it came to pass that I updated the runtimes of gbrainy and Genius, two apps I have never used, last month, and Klavaro back in May, among others.

There tends to be a flurry of community activity every 6-12 months to update all the apps that depend on newly end-of-lifed runtimes, and many of these turn out not to be rocket science, though it is not (yet!) something that flatpak-external-data-checker can do automatically. Then we are left with a long tail of more lightly-maintained apps where the update is more cumbersome to do, as in the case of gbrainy, where it took a bit of staring at the unfamiliar error messages produced by the Mono C# compiler to figure out where the problem might lie. If you are good at debugging build failures, this kind of thing is a great way to contribute to Flathub; and if someone can remind me of the URL of the live-updating TODO list of apps on obsolete runtimes I once saw, I’ll add the link here.

While I’ve focused on one of the problems that apps depending on obsolete runtimes can cause, it’s not all bad news. If you really love, or need, an app that is abandoned or has not been updated in a while, the Flatpak model you can still install & use that app and its old runtime without your distribution having to keep around some obsolete version of the libraries, or you having to stay on an old version of the distro. As and when that app does get an update, the unused and end-of-lifed runtime on your system will be uninstalled automatically by modern versions of Flatpak.