Flatpak External Data Checker

(This post is a slightly longer version of a lightning talk I gave at GUADEC 2019.)

Many non-free applications’ binaries cannot be redistributed (particularly not in modified form), so they cannot be included directly in a Flatpak. To work around this, Flatpak supports the concept of “extra data”: files which will be downloaded and unpacked from a third-party URI when the app is installed. The URI is accompanied by a checksum and a size, to provide some hope that the data unpacked on the user’s system is the same as what the packager tested. This is used by, for example, the Dropbox Flatpak.

Of course, the Flatpak needs to be kept up to date when new versions of the app are released. At best, the old URL will still point to the same file, so at least the old version of the app will continue to be installed; in some cases, however, vendors publish new versions of the app at the same URL, which means the Flatpak cannot be installed until it is updated.

Some time ago, Joaquim Rocha started work on Flatpak External Data Checker to periodically check a Flatpak manifest and report when it needs updating. As well as just checking that a URL is reachable and has the expected size and checksum, it also knows how to follow a redirect to a stable URI for the latest version (a helpful pattern some apps use), or to find the latest package in an apt repository. I subsequently taught it how to determine the new app’s version, update the AppData file, commit the necessary changes to Git, and send a pull request (like this one).

I tried moderately hard to preserve YAML and XML comments and formatting. For JSON, I gave up trying to preserve formatting (let alone json-glib’s non-standard extensions); the output is at least deterministic, so once it’s reformatted the JSON, the diffs will be smaller in future.

At Endless, we run this for a short list of apps on Flathub (and a few on Endless’s Flatpak repo). If you want to get PRs for an app you maintain, add the necessary metadata to your Flathub application’s manifest, then send a pull request to update the list of repos we check. I hope that in the medium term we could move this over to Flathub’s build infrastructure and run it on every repo (with some way to opt out).

There are a fair few open issues – PRs, suggestions and bug reports all very welcome!

γυαδεκ? χκπτγεδ?

GUADEC in Thessaloniki was a great experience, as ever. Thank you once again to the GNOME Foundation for sponsoring my attendence!

Sponsored by GNOME Foundation

Some personal highlights, in no particular order:

  • A lot of useful and informative discussion at the GNOME Advisory Board meeting on Thursday – we ran out of time, which seems like a good sign.
  • After Benjamin Berg and Iain Lane’s great talk on Managing GNOME Sessions with Systemd, Benjamin and I discussed the special-case they had to make to run GNOME Initial Setup’s “copy worker” early in the user session, and whether we might be able to improve this and various other aspects by launching Initial Setup in a different way.
  • Via Matthias’ talk on Portals, I got thinking about the occasional requests for an “is this app installed?” portal, and I realised that you can actually fake it with existing machinery in some cases. If you care about a specific app, you probably want to be able to talk to it, so you specify --talk-name=org.example.Foo; at which point you can call org.freedesktop.DBus.ListActivatableNames() and check whether org.example.Foo is in the returned list.
  • The Intern Lightning Talks were inspiring: it’s great to see what has caught the interest of new contributors. This year, I was inspired by Srestha Srivastava’s work on Boxes to send a merge request to osinfo-db to generate the necessary XML for Endless OS. This in turn led to a great discussion with Fabiano and Felipe, and to some more issues and merge requests.
  • Alex Larsson was a tough act to follow at the lightning talks, but based on hallway discussion, my bit on Flatpak External Data Checker was of interest. (I taught it how to update appdata on the flight home. The person sitting next to me told me that writing code on flights is a young-person thing, which I took as a compliment.)
  • Not one, but two talks on user testing! One thing I took away is that while it’s possible to conduct remote usability testing, you’ll miss out on body language cues from the test subjects, and in the specific case of GNOME you’ll either bias the sample towards people who already use GNOME, or you’ll introduce the additional variable of whatever remote access tool the user uses. Not ideal!

On the Endless front, the launch of the Coding Education Challenge, and the various talks from my esteemed colleagues about varied activities, were all great to see.

There were lots of clashes for me, so I’m grateful to the AV team for their great work on recording all the talks. (Unfortunately, one of the talks I couldn’t make it to, on GDPR, was not recorded, to avoid distributing what could be construed as legal advice. Alas!) Many thanks to the local team and the GNOME Foundation staff and volunteers who made the event run so smoothly.

eMusic in Banshee

I’ve been an eMusic customer for many years, and I’m pretty happy with it. Banshee comes with a plugin—courtesy of Eitan—to help download entire albums from eMusic without using their own downloader application. But you have to go search in your browser, and then hope the necessary MIME type handlers are set up to pass the .emx file eMusic gives you to Banshee, and also have remembered to enable the plugin.

Having enjoyed the notorious Amazon store integration, I thought I’d try my hand at something similar for eMusic. Here’s a quick demo video of downloading a couple of albums: one free, one not. Not shown in the video: playing track previews inside Banshee, and downloading invididual tracks.

I’m pleased to say that I wrote approximately no code (which is good, because I don’t really speak C♯): it’s derived from the Amazon store plugin, with most of the code removed. What remains is in a branch on fd.o; I’ve updated bgo#623828 with a link if anyone fancies reviewing this. (I am very tempted to set up a personal cheese and wine fund.)

Moody.

Today, in “ridiculous specification I’ve come across in the course of entirely legitimate work” …

In case just typing a status message wasn’t good enough, XEP-0108: User Activity defines an XMPP extension for telling your contacts precisely what you’re up to in a machine-readable form … as long as it’s one of a rigidly-defined taxonomy of activities. Buying groceries is a kind of chore, whereas shopping is supposedly a form of relaxation; I think some might disagree that you’re inactive while praying.

RFC 4480: Rich Presence Extensions to the Presence Information Data Format goes a step further and allows you to specify that you’re in a location that’s too dark for video, too noisy for audio, and inappropriate for IM. Your mood can be “invincible”, but—distressingly—neither “quixotic” nor “apathetic” are permitted. I think I’ll stick to LiveJournal.

MeeGo Conference: Telepathy 1.0 roadmap


The queue for registration

Along with something like 15 other Collaborans, I’m at the MeeGo Conference 2010 in Dublin. I’ve somehow managed never to visit Ireland before, so I’m hoping to find some time to explore a bit while I’m here.

I’m giving a talk titled The Road to Telepathy 1.0 tomorrow at 9am (yerk!) in the Vavasour Suite. I’ll be sketching out a rough roadmap, or rather collection of related goals and schedules; also, there’ll be an update on ongoing feature development. Right afterwards, at 9.30, the inimitable Mikhail Zabaluev is speaking on Developing Communication Protocol Implementations in Telepathy, based on Nokia’s experience developing SIP, cellular and Skype backends for Telepathy over the past years.

Also, at the Collabora stand we’re demoing DSP-accelerated video calls on an OMAP board using GStreamer, and integration with various services in MeeGo Netbook via Telepathy. Collabora folks working on all kinds of projects will be around; see you there!

Bustle 0.2.3: now with pairs of D-Bus traffic charts side-by-side

I just released Bustle 0.2.3. The notable improvement is that you can now view a pair of D-Bus traffic logs side-by-side on the same chart. So if you’ve taken a trace of the session bus and the system bus, and want to see how the bus traffic matches up on the two, this is the release you’ve been waiting for! (If not, well, I made the ugly pink lines a more tasteful grey, and fixed some bugs you never noticed.)

While I was refactoring to support the second log, I would have liked to have been able to run Bustle in a “batch” mode to render straight to a file, and then run some kind of visual diff tool to compare the output of the branch versus the last release. Coincidentally, when I opened my inbox, I found a mail requesting the same feature! I imagine that this could come in handy for producing automated reports: maybe you’d have a weekly cron job that produces some stats about the traffic using bustle-count and, if it goes significantly up or down, sends an email to tick off or congratulate the relevant team as appropriate with an attached diagram. ;-) If anyone fancied having a crack at this, it shouldn’t be too hard.

Sending SMSes with Empathy and Telepathy-Ring

Early last month, Lassi Syrjälä released Telepathy-Ring, Nokia’s Telepathy connection manager for GSM telephony, under the LGPL. The version used on the N900 talks to a proprietary daemon to drive the cellular hardware, but this new 2.x.y series has been ported to oFono, Intel and Nokia’s Free cellular modem daemon. I was trying out Ring using oFono’s phone simulator backend, until it was pointed out that oFono also supports my laptop’s built-in GSM chip. Oh really? Let’s see…

A few minutes of tweaking later, and I was looking at an apparently-unremarkable Empathy conversation window:

Screenshot of an SMS conversation in Empathy

Ring needed a few little hacks to get this going, mostly because laptops’ GSM chips don’t generally support making GSM calls, which Ring expects to be able to do. But I didn’t have to touch any other Telepathy components’ source: I installed my Ring branch, opened the Empathy accounts dialog, created a new “tel” account, and here we are. +447771██████ in that screenshot is my real actual phone, and this conversation looks just how you’d expect.

Of course, right now this is a proof-of-concept; it’s not really ready for non-developers. I’m planning to clean up my Ring patches for submission upstream over the next few weeks, and will try to trick someone into writing a custom account configuration UI for Empathy; hopefully we can get this working properly pretty soon! Thanks to Lassi, Pekka Pessi (Ring’s original author), and others at Maemo; the oFono team; and other Telepathy and Empathy hackers for making this so straightforward!

Foschart: a FOSDEM schedule app for the N900

Hello internet! I am at FOSDEM 2010 in Brussels. I tried the fosdem-maemo schedule application for my Nokia N900, and decided to write an alternative app which is easier to use with my fingers, and looks more like a Maemo application.

The result is foschart. It’s just something I knocked together in a few hours yesterday, but it’s pretty usable already. It supports showing talks grouped by track, by room, and just in chronological order, and a list of favourites. It’s all happily kinetic-scrollable, etc., and is very snappy once it’s started.

There’s no proper release or package yet; if you want to package it up properly, please do! But for now, apt-get install python-hildon, then copy foschart.py and schedule.xml to /opt/foschart, and foschart.desktop to /usr/share/applications/hildon. Then it should show up in your application list, and away you go. As ever, patches welcome. Enjoy!

Update:

The illustrious Jonny Lamb has made a package!

Clearing unwanted recent tags from the N900 photo viewer

The camera/photo viewer on the N900 has a pretty nice tag cloud widget, which lets you quickly label your photos before you upload them to Flickr. (The novelty hasn’t yet worn off!) But an autocompletion accident left me with a tag in the widget that I’d really prefer not to be there when I’m showing off my nice new phone to people.

I spent a happy¹ few hours trying to figure out where it gets the set of tags from. The viewer asks Tracker for the most commonly-used tags, but this tag wasn’t used on any of my photos, so wasn’t coming from there. In fact, it didn’t appear in any of Tracker’s database files! After a bit of investigation, I discovered that the photo viewer keeps its own independent set of recently-used tags, not in Tracker, but in GConf, at /apps/osso/image-viewer/recent_tags. Lest you should find yourself in my position, a quick

gconftool --set --type list --list-type=string
/apps/osso/image-viewer/recent_tags '[]'

will expunged your undesired utterances from the cloud. Bug report time. Next stop: finding a tool that lets the user remove typos from the autocompletion database …

1. Grr.

Bustle 0.2.1

A couple of days ago, I released version 0.2.1 of Bustle, someone’s favourite D-Bus profiler. As the version number suggests, there aren’t really any big new features; most of the changes just make it a bit nicer to use, like showing you all the bus names a service owns, ellipsizing strings, a slightly less spartan UI, etc. Having finally gotten around to cutting a release, I’ve started wondering what to work on next. There are various small things I have in mind, such as searching, filtering, integrating the various statistic tools (bustle-time and friends) into the UI, and so on, but it’d be nice to have a larger goal to work towards.

One recurring feature request is the ability to see messages’ arguments. This isn’t currently possible because the simple plain-text logs produced by the monitor (which is a variation on the theme of dbus-monitor --profile) only includes the message header. I’ve thought for a while that the right thing to do would be to log the raw dbus messages, together with a timestamp, but wasn’t sure what the files would look like. (Maybe shove the timestamps into the message headers?) Rob had a nice idea: why not log to pcap files? This avoids inventing a new format—the UI would just use libpcap and feed each message through the dbus parser—and would also let you look at the logs in WireShark, if you’re into that kind of thing. I’m hoping to find some time to give this a shot soon. (Maybe on a cold Christmas evening, in front of a fire?)

In the meantime, have a peek at what your D-Bus-using applications are up to, and let me know what’s missing!