GNOME 3.36 / Endless OS 3.8

Endless OS 3.8.0 has just been released, which brings GNOME 3.36 to our users. There’s a particularly big overlap between “improvements in Endless OS” and “improvements in GNOME” this cycle, so I wanted to take a minute to look back over what the Endless team worked on in GNOME 3.36.

Login & Unlock Screen

Allan Day has already written about the improvements to the login and unlock experience in GNOME 3.36, so I won’t retread this ground in too much detail. As he (and Nick Richards, in his trip report for Endless OS 3.8.0) mentioned, this change has been anticipated for a long time, so I’m particularly glad that Georges Stavracas and Umang Jain (together with Florian Müllner from Red Hat) could make this happen for this release. The first thing I interact with when I sit down at my computer is the login screen or the lock screen, and the refreshed design is a pleasure to use. (My daughter is sad that Granny’s cat is no longer visible on the lock screen, though.)

GNOME unlock dialog, with Will Thompson's name and face, and password “Tremendousdangerouslookingyak” visible

Peek Password

One improvement that’s perhaps most visible in the redesigned lock screen is the inline “eye” icon to reveal the text in the password field, which was implemented by Umang Jain independently of his work on the lock screen itself. The motivation for this change was actually another system dialogue: the Wi-Fi password dialogue.

During the development of the Hack product – a game-like platform for self-directed learning built atop Endless OS – the team ran many playtesting sessions. While the emphasis of these sessions was on Hack itself, the test users – typically younger teens – would often run through initial setup on a freshly-installed OS. Within a few clicks of turning on the computer, you select your Wi-Fi network and enter its password, which turned out to be a big stumbling block for many users. Wi-Fi passwords are long strings of randomly-generated characters, and on many occasions users simply couldn’t enter the password correctly. The entry has always had a Show Text option in the right-click menu, but right-clicking is itself an unfamiliar operation for younger users more familiar with mobile devices.

Parental Controls, redux

For a year or so, Endless OS has included a parental controls feature, which operates along a couple of axes:

  • Specific installed apps can be disabled for particular users. As a special case, all general-purpose web browsers are controlled by a separate toggle.
  • Not-yet-installed apps visible in GNOME Software — which we rebrand as App Center — can be filtered based on their OARS content rating metadata.
  • Users can be prevented from installing apps at all.

In past releases, this feature was hard to discover and use. At a superficial level, the UI to control it was buried in Settings → Details → Users → (select a non-administrator user) → (scroll down) → (frame within frame within frame). But the real issue was that many Endless OS systems have the child as the primary, administrator user, created through Initial Setup when the machine is unboxed. To meaningfully use parental controls, you’d need to create a separate parent user, then downgrade the child’s account, neither of which is a particularly discoverable operation.

In autumn last year, we met with Allan Day, Richard Hughes and Matthias Clasen from Red Hat to talk through this problem space. Following that, Robin Tafel, Philip Withnall and Matthew Leeds designed and implemented a new flow for parental controls. The key changes are:

  1. Parental controls can be enabled during initial setup. Check a box, choose some options, and specify a parent password.
  2. Once initial setup is complete, there is a dedicated Parental Controls app.

Screenshot of “About You” page from GNOME Initial Setup, showing “Set up parental controls for this user” checkbox (checked)

Screenshot of Parental Controls page of GNOME Initial Setup, showing options to restrict which applications can be installed or used

Screenshot of GNOME Initial Setup “Set a Parent Password” page, with two password fields and one password hint field

Screenshot of Parental Controls application, showing options to restrict which apps a user can install or run

There are a few downstream bits and bobs outstanding, such as a cross-reference from GNOME Settings’ Users panel, but the bulk of this feature is available upstream in GNOME Initial Setup, Software, and Shell 3.36. Parental controls needs close integration with the application management infrastructure, and Flatpak upstream has the necessary hooks. On Endless OS, supporting Flatpak apps — plus Chromium as a special case — is good enough, since that is the sole mechanism for installing applications. It would be great to see support in Malcontent for other package and app managers.

Special thanks to Jakub Steiner for creating a great icon at very short notice.

Malcontent icon: Silhoutte of parent and child holding hands

Renaming Folders

One of the biggest differences between vanilla GNOME and Endless OS is the app grid, which in Endless is on the desktop and fully under the user’s control. Georges Stavracas has been incrementally chipping away at this, and support for renaming folders landed in GNOME 3.36.

Screenshot of renaming a folder titled “Jeux”

The Long Tail

Besides highly-visible new features and redesigns, much (perhaps even most?) of the work of maintaining a desktop is in the parts you don’t see: improving libraries and plumbing, incremental tweaks to user interfaces, and dealing with the wide variety of hardware, software and users that interact with GNOME. Spelunking through the commit histories of various projects, I see many names of colleagues present and past, including André Moreira Magalhães and Philip Chimento respectively. Jian-Hong Pan from the Endless kernel team makes an appearance in GNOME Settings, as does a feature from erstwhile Endless kernel hacker Carlo Caione dating back to 2018.

Umang Jain, Philip Withnall and Matthew Leeds have put a lot of work into improving the robustness of GNOME Software and Flatpak, and there’s more landing as we speak. I’m particularly glad that Matthew has been tracking down missing Flatpak app updates in GNOME Software – bugs which hide information can be the trickiest ones to spot. And Philip is solving the latest Mystery of the Missing Progress Bar when installing Flatpak apps in GNOME Software.

I’m certain I’ve missed many great contributions. Please forgive me, fellow Endlessers.

A Broad Church

Perhaps my favourite part of being involved in GNOME is collaborating with great people from organisations who, in a different world, might be bitter rivals. All of the work I’ve described was a joint effort with others from the GNOME community; and, just as other distributors share the fruits of our labour, we and our users share the fruits of theirs. This is the latest in a long line of great GNOME releases – long may this trend continue.

Vanilla is a complex and delicious flavour

Last week, Tobias Bernard published a thought-provoking article, There is no “Linux” Platform (Part 1), based on a talk at LAS 2019. (Unfortunately I couldn’t make it to LAS, and I haven’t found the time to watch a recording of the talk, so I’m going solely from the blog post here.) The article makes some interesting observations, and I found a fair few things to agree with. But I want to offer a counterpoint to this paragraph of the final section, “The Wrong Incentives”:

The Endless OS shell is a great example of this. They started out with vanilla GNOME Shell, but then added ever more downstream patches in order to address issues found in in-house usability tests. This means that they end up having to do huge rebases every release, which is a lot of work. At the same time, the issues that prompted the changes do not get fixed upstream (Endless have recently changed their strategy and are working upstream much more now, so hopefully this will get better in the future).

If we’re looking at the code shipping in Endless OS today, then yes, our desktop is vanilla GNOME Shell with a few hundred patches on top, and yes, as a result, rebasing onto new GNOME releases is a lot of work. But the starting point for Endless OS was not “what’s wrong with GNOME?” but “what would the ideal desktop look like for a new category of users?”.

When Endless began, the goal was to create a new desktop computing product, targeting new computer users in communities which were under-served by existing platforms and products. The company conducted extensive field research, and designed a desktop user interface for those users. Prototypes were made using various different components, including Openbox, but ultimately the decision was made to base the desktop on GNOME, because GNOME provided a collection of components closest to the desired user experience. The key point here is that basing the Endless desktop on GNOME was an implementation detail, made because the GNOME stack is a robust, feature-rich and flexible base for a desktop.

Over time, the strategy shifted away from being based solely around first-party hardware, towards distributing our software to a broader set of users using standard desktop and laptop hardware. Around the same time, Endless made the switch from first- and third-party apps packaged as a combination of Debian packages and an in-house system towards using Flatpak for apps, and contributed towards the establishment of Flathub. Part of the motivation for this switch was to get Endless out of the business of packaging other people’s applications, and instead to enable app developers to directly target desktop Linux distributions including, but not limited to, Endless OS.

A side-effect of this change is that our user experience has become somewhat less consistent because we have chosen not to theme apps distributed through Flathub, with the exception of minimize/maximize window controls and a different UI font; and, of course, Flathub offers apps built with many different toolkits. This is still a net positive: our users have access to many more applications than they would have done if we had continued distributing everything ourselves.

As the prototypal Endless OS user moved closer to the prototypal GNOME user, we have focused more on finding ways to converge with the GNOME user experience. In some cases, we’ve simply removed functionality which we don’t think is necessary for our current crop of users. For example, Endless OS used to target users whose display was a pre-digital TV screen, with a 720×480 resolution. I think persuading the upstream maintainers of GNOME applications to support this resolution would have been a hard sell in 2014, let alone in 2019!

Some other changes we’ve made can and have been simply be proposed upstream as they are, but the bulk of our downstream functionality forms a different product to GNOME, which we feel is still valuable to our users. We are keen to both improve GNOME, and reduce the significant maintenance burden which Tobias rightly refers to, so we’re incrementally working out which functionality could make sense in both Endless and GNOME in some form, working out what that form could be, and implementing it. This is a big project because engaging constructively with the GNOME community involves more thought and nuance than opening a hundred code-dump merge requests and sailing away into the sunset.

If you are building a product whose starting point is “GNOME, but better”, then I encourage you to seriously consider whether you can work upstream first. I don’t think this is a groundbreaking idea in our community! However, that was not the starting point for Endless OS, and even today, we are aiming for a slightly different product to GNOME.

Back out to the big picture that is the subject of Tobias’ article: I agree that desktop fragmentation is a problem for app developers. Flatpak and Flathub are, in my opinion, a major improvement on the status quo: app developers can target a common environment, and have a reasonable expectation of their apps working on all manner of distributions, while we as distro maintainers need not pretend that we know best how to package a Java IDE. As the maintainer of a niche app written using esoteric tools, Flathub allowed me – for the first time since I wrote the first version in 2008 – to distribute a fully-functional, easy-to-install application directly to users without burdening distribution developers with the chore of packaging bleeding-edge versions of Haskell libraries. It gave me a big incentive to spend some of my (now very limited) free time on some improvements to the app that I had been putting off until I had a way to get them to users (including myself on Endless OS) in a timely manner.

On the other hand, we shouldn’t underestimate the value of GNOME – and distros like Debian – being a great base for products that look very different to GNOME itself: it enables experimentation, exploration, and reaching a broader base of users than GNOME alone could do, while pooling the bulk of our resources. (On a personal level, I owe pretty much my entire career in free software to products based on Debian and the GNOME stack!)

Some caveats: I joined Endless in mid-2016, midway through the story above, so I am relying on my past and current colleagues’ recollections of the early days of the company. Although today I am our Director of Platform, I am still just one person in the team! We’re not a hive mind, and I’m sure you’ll find different opinions on some of these points if you ask around.

Flatpak External Data Checker

(This post is a slightly longer version of a lightning talk I gave at GUADEC 2019.)

Many non-free applications’ binaries cannot be redistributed (particularly not in modified form), so they cannot be included directly in a Flatpak. To work around this, Flatpak supports the concept of “extra data”: files which will be downloaded and unpacked from a third-party URI when the app is installed. The URI is accompanied by a checksum and a size, to provide some hope that the data unpacked on the user’s system is the same as what the packager tested. This is used by, for example, the Dropbox Flatpak.

Of course, the Flatpak needs to be kept up to date when new versions of the app are released. At best, the old URL will still point to the same file, so at least the old version of the app will continue to be installed; in some cases, however, vendors publish new versions of the app at the same URL, which means the Flatpak cannot be installed until it is updated.

Some time ago, Joaquim Rocha started work on Flatpak External Data Checker to periodically check a Flatpak manifest and report when it needs updating. As well as just checking that a URL is reachable and has the expected size and checksum, it also knows how to follow a redirect to a stable URI for the latest version (a helpful pattern some apps use), or to find the latest package in an apt repository. I subsequently taught it how to determine the new app’s version, update the AppData file, commit the necessary changes to Git, and send a pull request (like this one).

I tried moderately hard to preserve YAML and XML comments and formatting. For JSON, I gave up trying to preserve formatting (let alone json-glib’s non-standard extensions); the output is at least deterministic, so once it’s reformatted the JSON, the diffs will be smaller in future.

At Endless, we run this for a short list of apps on Flathub (and a few on Endless’s Flatpak repo). If you want to get PRs for an app you maintain, add the necessary metadata to your Flathub application’s manifest, then send a pull request to update the list of repos we check. I hope that in the medium term we could move this over to Flathub’s build infrastructure and run it on every repo (with some way to opt out).

There are a fair few open issues – PRs, suggestions and bug reports all very welcome!

γυαδεκ? χκπτγεδ?

GUADEC in Thessaloniki was a great experience, as ever. Thank you once again to the GNOME Foundation for sponsoring my attendence!

Sponsored by GNOME Foundation

Some personal highlights, in no particular order:

  • A lot of useful and informative discussion at the GNOME Advisory Board meeting on Thursday – we ran out of time, which seems like a good sign.
  • After Benjamin Berg and Iain Lane’s great talk on Managing GNOME Sessions with Systemd, Benjamin and I discussed the special-case they had to make to run GNOME Initial Setup’s “copy worker” early in the user session, and whether we might be able to improve this and various other aspects by launching Initial Setup in a different way.
  • Via Matthias’ talk on Portals, I got thinking about the occasional requests for an “is this app installed?” portal, and I realised that you can actually fake it with existing machinery in some cases. If you care about a specific app, you probably want to be able to talk to it, so you specify --talk-name=org.example.Foo; at which point you can call org.freedesktop.DBus.ListActivatableNames() and check whether org.example.Foo is in the returned list.
  • The Intern Lightning Talks were inspiring: it’s great to see what has caught the interest of new contributors. This year, I was inspired by Srestha Srivastava’s work on Boxes to send a merge request to osinfo-db to generate the necessary XML for Endless OS. This in turn led to a great discussion with Fabiano and Felipe, and to some more issues and merge requests.
  • Alex Larsson was a tough act to follow at the lightning talks, but based on hallway discussion, my bit on Flatpak External Data Checker was of interest. (I taught it how to update appdata on the flight home. The person sitting next to me told me that writing code on flights is a young-person thing, which I took as a compliment.)
  • Not one, but two talks on user testing! One thing I took away is that while it’s possible to conduct remote usability testing, you’ll miss out on body language cues from the test subjects, and in the specific case of GNOME you’ll either bias the sample towards people who already use GNOME, or you’ll introduce the additional variable of whatever remote access tool the user uses. Not ideal!

On the Endless front, the launch of the Coding Education Challenge, and the various talks from my esteemed colleagues about varied activities, were all great to see.

There were lots of clashes for me, so I’m grateful to the AV team for their great work on recording all the talks. (Unfortunately, one of the talks I couldn’t make it to, on GDPR, was not recorded, to avoid distributing what could be construed as legal advice. Alas!) Many thanks to the local team and the GNOME Foundation staff and volunteers who made the event run so smoothly.

eMusic in Banshee

I’ve been an eMusic customer for many years, and I’m pretty happy with it. Banshee comes with a plugin—courtesy of Eitan—to help download entire albums from eMusic without using their own downloader application. But you have to go search in your browser, and then hope the necessary MIME type handlers are set up to pass the .emx file eMusic gives you to Banshee, and also have remembered to enable the plugin.

Having enjoyed the notorious Amazon store integration, I thought I’d try my hand at something similar for eMusic. Here’s a quick demo video of downloading a couple of albums: one free, one not. Not shown in the video: playing track previews inside Banshee, and downloading invididual tracks.

I’m pleased to say that I wrote approximately no code (which is good, because I don’t really speak C♯): it’s derived from the Amazon store plugin, with most of the code removed. What remains is in a branch on fd.o; I’ve updated bgo#623828 with a link if anyone fancies reviewing this. (I am very tempted to set up a personal cheese and wine fund.)


Today, in “ridiculous specification I’ve come across in the course of entirely legitimate work” …

In case just typing a status message wasn’t good enough, XEP-0108: User Activity defines an XMPP extension for telling your contacts precisely what you’re up to in a machine-readable form … as long as it’s one of a rigidly-defined taxonomy of activities. Buying groceries is a kind of chore, whereas shopping is supposedly a form of relaxation; I think some might disagree that you’re inactive while praying.

RFC 4480: Rich Presence Extensions to the Presence Information Data Format goes a step further and allows you to specify that you’re in a location that’s too dark for video, too noisy for audio, and inappropriate for IM. Your mood can be “invincible”, but—distressingly—neither “quixotic” nor “apathetic” are permitted. I think I’ll stick to LiveJournal.

MeeGo Conference: Telepathy 1.0 roadmap

The queue for registration

Along with something like 15 other Collaborans, I’m at the MeeGo Conference 2010 in Dublin. I’ve somehow managed never to visit Ireland before, so I’m hoping to find some time to explore a bit while I’m here.

I’m giving a talk titled The Road to Telepathy 1.0 tomorrow at 9am (yerk!) in the Vavasour Suite. I’ll be sketching out a rough roadmap, or rather collection of related goals and schedules; also, there’ll be an update on ongoing feature development. Right afterwards, at 9.30, the inimitable Mikhail Zabaluev is speaking on Developing Communication Protocol Implementations in Telepathy, based on Nokia’s experience developing SIP, cellular and Skype backends for Telepathy over the past years.

Also, at the Collabora stand we’re demoing DSP-accelerated video calls on an OMAP board using GStreamer, and integration with various services in MeeGo Netbook via Telepathy. Collabora folks working on all kinds of projects will be around; see you there!

Bustle 0.2.3: now with pairs of D-Bus traffic charts side-by-side

I just released Bustle 0.2.3. The notable improvement is that you can now view a pair of D-Bus traffic logs side-by-side on the same chart. So if you’ve taken a trace of the session bus and the system bus, and want to see how the bus traffic matches up on the two, this is the release you’ve been waiting for! (If not, well, I made the ugly pink lines a more tasteful grey, and fixed some bugs you never noticed.)

While I was refactoring to support the second log, I would have liked to have been able to run Bustle in a “batch” mode to render straight to a file, and then run some kind of visual diff tool to compare the output of the branch versus the last release. Coincidentally, when I opened my inbox, I found a mail requesting the same feature! I imagine that this could come in handy for producing automated reports: maybe you’d have a weekly cron job that produces some stats about the traffic using bustle-count and, if it goes significantly up or down, sends an email to tick off or congratulate the relevant team as appropriate with an attached diagram. ;-) If anyone fancied having a crack at this, it shouldn’t be too hard.

Sending SMSes with Empathy and Telepathy-Ring

Early last month, Lassi Syrjälä released Telepathy-Ring, Nokia’s Telepathy connection manager for GSM telephony, under the LGPL. The version used on the N900 talks to a proprietary daemon to drive the cellular hardware, but this new 2.x.y series has been ported to oFono, Intel and Nokia’s Free cellular modem daemon. I was trying out Ring using oFono’s phone simulator backend, until it was pointed out that oFono also supports my laptop’s built-in GSM chip. Oh really? Let’s see…

A few minutes of tweaking later, and I was looking at an apparently-unremarkable Empathy conversation window:

Screenshot of an SMS conversation in Empathy

Ring needed a few little hacks to get this going, mostly because laptops’ GSM chips don’t generally support making GSM calls, which Ring expects to be able to do. But I didn’t have to touch any other Telepathy components’ source: I installed my Ring branch, opened the Empathy accounts dialog, created a new “tel” account, and here we are. +447771██████ in that screenshot is my real actual phone, and this conversation looks just how you’d expect.

Of course, right now this is a proof-of-concept; it’s not really ready for non-developers. I’m planning to clean up my Ring patches for submission upstream over the next few weeks, and will try to trick someone into writing a custom account configuration UI for Empathy; hopefully we can get this working properly pretty soon! Thanks to Lassi, Pekka Pessi (Ring’s original author), and others at Maemo; the oFono team; and other Telepathy and Empathy hackers for making this so straightforward!

Foschart: a FOSDEM schedule app for the N900

Hello internet! I am at FOSDEM 2010 in Brussels. I tried the fosdem-maemo schedule application for my Nokia N900, and decided to write an alternative app which is easier to use with my fingers, and looks more like a Maemo application.

The result is foschart. It’s just something I knocked together in a few hours yesterday, but it’s pretty usable already. It supports showing talks grouped by track, by room, and just in chronological order, and a list of favourites. It’s all happily kinetic-scrollable, etc., and is very snappy once it’s started.

There’s no proper release or package yet; if you want to package it up properly, please do! But for now, apt-get install python-hildon, then copy and schedule.xml to /opt/foschart, and foschart.desktop to /usr/share/applications/hildon. Then it should show up in your application list, and away you go. As ever, patches welcome. Enjoy!


The illustrious Jonny Lamb has made a package!