Technical Blog of Richard Hughes

The ODRS is the service that produces ratings and reviews for gnome-software. I built the service a few years ago, and it’s been dutifully trucking on ever since. There are over 25,000 reviews, 50k votes, and over 4k different applications reviewed. Over half a million clients get application reviews every single day.

Recently it’s been showing signs of needing work, and so I’ve spent a few days converting it to Python 3, then to SQLAlchemy, and then fixing all the broken stuff that we’ve lived with for a while (e.g. no emoji support because we were not using utf8mb4…). Part of the new work will be making it easier to flag and then moderate reviews, and that needs your help. Although any unauthenticated user can report a review for any reason, some reviews should be automatically marked at submission if they contain known bad words. There is almost no reason to write a review in locale en_GB and use the word fuck and so I think marking that review as needing moderation before it’s shown to thousands of people is a sensible thing to do.

To this to work, I can’t just use a blacklist of words as some words are only really vulgar in some regions, and some are perfectly valid words in other languages. For this reason I need the blacklist to be keyed to the submitted locale.

This is where I need your help. If you can spare 2 minutes, and know a lot of dirty words in your language can you please add them to this spreadsheet. Much appreciated.

Alex Diavatis is the person behind the WOGUE account on YouTube. For a while he’s been posting videos about GNOME. I think the latest idea is that he’s trying to “shame” developers into working harder. From the person who’s again on the other end of his rants it’s having the opposite effect.

We’re all doing our best, and I’m personally balancing about a dozen different plates trying to keep them all spinning. If any of the plates fall on the floor, perhaps helping with triaging bugs, fixing little niggles or just saying something positive might be a good idea. In fact, saying nothing would be better than the sarcasm and making silly videos.

When firmware is uploaded to the LVFS we perform online checks on it. For example, one of the tests is looking for known badness like embedded UTF-8/UTF-16 BEGIN RSA PRIVATE KEY strings. As part of this we use CHIPSEC (in the form of chipsec_util -n uefi decode) which searches the binary for a UEFI volume header which is a simple string of _FVH and then decompresses the volumes which we then read back as component shards. This works well on plain EDK2 firmware, and the packages uploaded by Lenovo and HP which use IBVs of AMI and Phoenix. The nice side effect is that we can show the user what binaries have changed, as the vendor might have accidentally forgotten to mention something in the release notes.

The elephants in the room were all the hundreds of archives from Dell which could not be loaded by chipsec with no volume header detected. I spent a few hours last night adding support for these archives, and the secret is here:

1. Decompress the firmware.cab archive into firmware.bin, disregarding the signing and metadata.
2. If CHIPSEC fails to analyse firmware.bin, look for a > 512kB decompress-able Zlib section somewhere after the capsule header, actually in the PE binary.
3. The decompressed blob is in PFS format, which seems to be some Dell-specific format that’s already been reverse engineered.
4. The PFS blob is not further compressed and is in one continuous block, and so the entire PFS volume can be passed to chipsec for analysis.

The Zlib start offset seems to jump around for each release, and I’ve not found any information in the original PE file that indicates the offset. If anyone wants to give me a hint to avoid searching the multimegabyte blob for two bytes (and then testing if it’s just chance, or indeed an Zlib stream…) I would be very happy, even if you have to remain anonymous.

So, to sum up:

CapsuleHeader
  PE Binary
    Zlib stream
      PFS
        FVH
          PE DXEs
          PE PEIMs
          …

I’ll see if chipsec upstream wants a patch to do this as it’s probably useful outside of the LVFS too.

First, thanks to everyone for the all the help with UEFI modules. There are a ton left, but also a lot done, so we’re getting there. If anyone is intimately familiar with ME firmware, I’d also really like some help with this Intel Management Engine document too. Thanks!

For about every 250 bug reports I recieve I get an email offering to help. Most of the time the person offering help isn’t capable of diving right in the trickiest parts of the code and just wanted to make my life easier. Now I have a task that almost anyone can help with…

For the next version of the LVFS we deploy we’re going to be showing what was changed between each firmware version. Rather than just stating the firmware has changed from SHA1:DEAD to SHA1:BEEF and some high level update description provided by the vendor, we can show the interested user the UEFI modules that changed. I’m still working on the feature and without more data it’s kinda, well, dull. Before I can make the feature actually useful to anyone except a BIOS engineer, I need some help finding out information about the various modules.

In most cases it’s simply googling the name of the module and writing 1-2 lines of a summary about the module. In some cases the module isn’t documented at all, and that’s fine — I can go back to the vendors and ask them for more details about the few we can’t do ourselves. What I can’t do is ask them to explain all 150 modules in a specific firmware release, and I don’t scale to ~2000 Google queries. With the help of EDK2 I’ve already done 213 myself but now I’ve run out of puff.

So, if you have a spare 5 minutes I’d really appreciate some help. The shared spreadsheet is here, and any input is useful. Thanks!

Yesterday Dell unveiled their new range of docking stations. I’ve had a WD19TB for a few months, and it seems to work fine; you can plug one thunderbolt cable into my XPS 13 and it turns it into a workstation, driving two screens, connecting me to wired Ethernet and connecting all my USB stuff. When I want to run away, I just unplug one thing and my workstation turns back into a portable laptop. The dock also randomly has a headphone out socket, although I like to drive my sound through a USB sound card and S/PDIF anyway.

The impressive bit for me is that Dell wanted firmware upgrading to work perfectly from day 1 – there has been testable firmware embargoed on the LVFS for many weeks. Although the dock is one physical thing, internally it’s made up of different components and subsystems, all with slightly different quirky flashing protocols. Behind the scenes and without great fan-fair, Dell added the “composite firmware” support to fwupd, so not only do the devices look logically correct when using fwupdmgr get-topology it means you can update all the different technology on the device with one .cab file.

This means, to update to the dock I don’t need to tell you lots of technical information about how to update to some super new version of something. Just click on the upgrade button in GNOME Software when the firmware update has been downloaded for you. This is exactly how firmware updates should work – you buy the hardware you need from the shop, plug it in, get the latest bug fixes and features automatically with no “googling” or using crazy commands on the command line.

I’ve also been working with another OEM on their next generation of docking stations. This work can of course piggy-back onto the composite feature when there is hardware ready for test. In my opinion they’re about 9 months behind Dell at this point, so I guess if you want a docking station that’s supported on the LVFS you know what to buy…

This is important if you’re the upstream maintainer of an application: If you change the desktop ID then it’s like breaking API. Changing a desktop ID should be done carefully and in a development branch only — and then you need to communicate it and give the distros a chance to adapt to the new name.

If you’re just changing the desktop ID and not forking development, you also need to add something like this in your metainfo.xml file:

  <provides>
    <id>old-name.desktop</id>
  </provides>

GNOME Software gets lots of bugs about showing “duplicate” search results, but there’s no reliable way it can know that calibre-gui.desktop is the same app as com.calibre_ebook.calibre without some help. If you’re a packager building an application for something like Flathub you only need to include the extra provides line if you’re adding a new metainfo.xml file rather than just using rename-appdata-file in the JSON file.

For a while, fwupd has been able to verify the PCR0 checksum for the system firmware. The attestation checksum can be used to verify that the installed firmware matches that supplied by the vendor and means the end user is confident the firmware has not been modified by a 3rd party. I think this is really an important and useful thing the LVFS can provide. The PCR0 value can easily be found using tpm2_pcrlist if the TPM is in v2.0 mode, or cat /sys/class/tpm/tpm0/pcrs if the TPM is still in v1.2 mode. It is also reported in the fwupdmgr get-devices output for versions of fwupd >= 1.2.2.

The device checksum as a PCR0 is slightly different than a device checksum for a typical firmware. For instance, a DFU device checksum can be created using sha256sum firmware.bin (assuming the image is 100% filling the device) and you don’t actually have to flash the image to the hardware to get the device checksum out. For a UEFI UpdateCapsule you need to schedule the update, reboot, then read back the PCR0 from the hardware. There must be an easier way…

Assuming you have a vendor account on the LVFS, first upload the client certificate for your user account to the LVFS:

Then, assuming you’re using fwupd >= 1.2.6 you can now do this:

fwupdmgr refresh
fwupdmgr update
…reboot…
fwupdmgr report-history --sign

Notice the –sign there? Looking back at the LVFS, there now exists a device checksum:

This means the firmware gets the magic extra green tick that makes everyone feel a lot happier:

In the next release of appstream-glib the appstream-util validate requirements got changed, which might make your life easier, or harder — depending if you already pass or fail the validation. The details are here but the rough jist is that we’ve relaxed a lot of the style rules (e.g. starts with a capital letter, ends with a full stop, less than a certain number of chars, etc), and made stricter some of the more important optional parts of the specification. For instance, requiring <content_rating> for any desktop or console application.

Even if you don’t care upstream, the new validation will soon be turned on for any apps built in Flathub, and downstream “packagers” will be pestering you for details as updates are now failing. Although only a few apps fail, some of the missing metadata tags are important enough to fail building. To test your app right now with the new validator:

$ flatpak remote-add --if-not-exists gnome-nightly https://sdk.gnome.org/gnome-nightly.flatpakrepo
$ flatpak install gnome-nightly org.gnome.Sdk
$ flatpak run --command=bash --filesystem=home:ro org.gnome.Sdk//master
# appstream-util validate /home/hughsie/Code/gnome-software/data/appdata/org.gnome.Software.appdata.xml.in
# exit

Of course, when the next tarball is released it’ll be available in your distribution as normal, but I wanted to get some early sanity checks in before I tag the release.

The LVFS is now an official Linux Foundation project! I did a mini-interview if you want some more details about where the project came from and where it’s heading. I’m hoping the move to the Linux Foundation gives the project a lot more credibility with existing LF members, and it certainly takes some of the load from me. I’ll continue to develop the lvfs-website codebase as before, and still be the friendly face when talking to OEMs and ODMs.

In the short term, not much changes, although you might start see some rebranding of the website itself. The server is also moving from a little VM in AMS to a fully scalable orchestrated thing maintained by people who actually understand how to be a sysadmin. If you’re interested in what’s happening on the LVFS, be sure to join the announcement mailing list. We’re averaging about 450,000 firmware downloads a month, and still growing steadily, with more and more vendors joining every month.

In related news, there’s lots of new firmware on the LVFS, much of it addressing serious CVEs on lots of different laptop models. If you’ve not updated recently, now is the time to fix that.