Technical Blog of Richard Hughes

Here’s an update for people waiting for news on the ColorHug+ spectrophotometer, and perhaps not the update that you were hoping for. Three things have recently happened, and each of them makes producing the ColorHug+ even harder than it was before:

1. A few weeks I became a father again. Producing the ColorHug and ColorHug2 devices takes a significant amount of time, brain, muscle and love, and I’m still struggling with dividing up my time between being a modern hands-on dad and also a full time job at Red Hat. ColorHug was (and still is) a hobby that got a little out of control, and not something that brings in any significant amount of money. A person spending £300 on a complex device is going to expect at least some level of support, even when I’ve had no sleep and only have half a brain on a Saturday morning.
2. Brexit has made the GBP currency plunge in value over the last 12 months, which in theory should be good as it will encourage exports. What’s slightly different for me is that 80% of the components for each device are purchased in USD and EUR, and the remaining ones in GBP have risen accordingly with the currency plunge. I have no idea what a post-Brexit Britain looks like, but I think it’s a prudent choice to not “risk” £20k in an investment I’d essentially hope to break even on long term, for fun.
3. The sensor for the ColorHug+ was going to be based on the bare chip SPARK from OceanOptics. I’ve spent a long time working out all the quirks of the sensor, making it work with a UV and wideband illuminant and working out all the packaging questions. The price of the sensor was always going to be expensive (it was greater than half of the RRP in one component alone, even buying a massive batch) but last month I got an email saying the sensor was going to be discontinued and would no longer be available. This is figuratively and also quite literally back to the drawing board.

I’ve included some photos above to show I’ve not been full of hot air for the last year or so, and to remind everyone that the PCB, 3D light guide model and client software are all in the various ColorHug git repos if you want to have a go at building one yourself (although, buy the sensor quickly…). I’ll still continue selling ColorHug2 devices, and supporting all the existing hardware but this might be the end of the line for ColorHug spectrometer. I’ll keep my eye on all the trade magazines for any new sensor that is inexpensive, reliable and accurate enough for ICC profiles, so all this might just be resurrected in the future, but for the short term this is all on ice. If you want a device right now the X-Rite i1Studio is probably the best of the bunch, although it is sold by Pantone with an RRP of £450. Fair warning: Pantone and free software are not exactly bedfellows, although it does work with ArgyllCMS using a reverse engineered userspace driver that might void your warranty.

I’ll update the website at some point this evening, I’m not sure whether to just post all this or remove the ColorHug+ page completely. Perhaps a sad announcement, but perhaps not one that’s too unexpected considering the lack of updates in the last few months. Sorry to disappoint everybody.

I’ve spent the last couple of evenings designing an OpenHardware USB 2.0 1-port hub tentatively called the ColorHub (although, better ideas certainly welcome). Back a bit: What’s the point in a 1-port hub?

The finished device is a Cypress 2 port hub internally, with a PIC microcontroller hard wired onto the other “fixed” port. The microcontroller can control the hub and the USB power of the other “removable” port, so you can simulate an unplug, replug or hub reset using a few simple commands. This also allows you forcefully reset hardware that’s not responding, and also allows you to add hardware tests to test enumeration and device removal. With the device it is trivial to write a script to replug a device 5000 times over one evening, or re-connect a USB device that’s not responding for whatever reason. The smart hub also reports when USB devices are connected to the downstream port, and even when they have not enumerated correctly. There could be commands to get the status of that and also optionally wait until those things have happened.

The other killer feature for me is that the microcontroller has lots of spare analog and digital IO, and with two included solid-state MOSFET relays you can wire up two physical switches so that no user interaction is required. This means you can test hardware that has these kind of requirements:

• Remove USB plug
• Press and hold buttons A&B
• Insert USB plug
• Release buttons A&B

It would be fairly trivial to wire up the microcontroller ADC to get a rough power consumption figure, or to set some custom hub descriptors; it would be completely open and “hackable” like the ColorHug.

I’ve made just one prototype and am using it quite nicely in the fwupd self tests, but talking to others yesterday this seems the kind of device that would be useful for other people doing similar QA activities. I need to build another 2 for the other devices requiring manual button-presses in the fwupd hardware cardboard-box-tests and it’s exactly the same price to order 50 tiny PCBs as 5.

The dangerous question: Would anyone else be interested in purchasing this kind of thing? The price would be in the £50-60 range, so certainly not cheap, but this is really the cost of ultra-small batches of moderately complicated electronics these days. If you’re interested, send me an email (richard_at_hughsie_dot_com) and depending on demand I’ll either design some nice custom PCBs or just hack together two more prototypes for my own use. Please also tell me if something like this already exists: if so I can save some time and just buy something that someone else has built. Comments welcome.

I wrote an article on the LVFS for OpenSource.com. If you’re interested in an overview of how firmware updates work in Linux, and a little history it might be an interesting read.

Usually near the end of the process of getting a vendor on the LVFS I normally ask them to send me hardware for the tests. Once we’ve got a pretty good idea that the hardware update process is going to work with fwupd (i.e. they’re not insisting on some static linked ELF to be run…) and when they’ve got legal approval to upload the firmware to the LVFS (without an eyewateringly long EULA) we start thinking about how to test the hardware. Once we say “Product Foo from Vendor Bar is supported in Linux” we better make damn sure it doesn’t regress when something in the kernel changes or when someone refactors a plugin to support a different variant of a protocol.

To make this task a little more manageable, we have a little python script that helps automate the devices that can be persuaded to enter DFU mode themselves. To avoid chaos, I also have a little cardboard tray under a little HP Microserver with two 10-port USB hubs with everything organised. Who knew paper-craft would be such an important skill at Red Hat…

As the astute might notice, much of the hardware is a bare PCB. I don’t actually need the complete device for testing, and much of the donated hardware is actually a user return or with a cosmetic defect, or even just a pre-release PCB without the actual hardware attached. This is fine, and actually preferable to the entire device – I only have a small office!

As much of the hardware needs special handling to put it in update mode we can’t 100% automate this task, and sometimes it really is just me sitting in front of the laptop pressing and holding buttons for 30 minutes before uploading a tarball, but it’s sure it comforting to know that firmware updates are tested like this. As usual, thanks should be directed to Red Hat for letting me work on this kind of stuff, they really are a marvelous company to work for.

In my previous blog post I hinted at you just have to add one line to a data file to add support for new AVR32 microcontrollers and this blog entry should give a few more details.

A few minutes ago I merged a PR that moves the database of supported and quirked devices out of the C code and into runtime loaded files. When fwupd is installed in long-term support distros it’s very hard to backport new versions as new hardware is released. The idea with this functionalty is that the end user can drop an additional (or replace an existing) file in a .d directory with a simple format and the hardware will magically start working. This assumes no new quirks are required, as this would obviously need code changes, but allows us to get most existing devices working in an easy way without the user compiling anything.

The quirk files themselves are simple key files and are documented in the fwupd gtk-doc documentation.

Over 10 years ago the dfu-programmer project was forked into dfu-utils as the former didn’t actually work at all well with generic devices supporting vanilla 1.0 and 1.1 specification-compliant DFU. It was then adapted to also support the STM variant of DFU (standards FTW). One feature that dfu-programmer did have, which dfu-util never seemed to acquire was support for the AVR variant of DFU (very different from STM DFU, but doing basically the same things). This meant if you wanted to program AVR parts you had to use the long-obsolete tool rather than the slightly less-unmaintained newer tool.

Today I merged a PR in fwupd that adds support for flashing AVR32 devices from Atmel. These are the same chips found in some Arduino protoype boards, and are also the core of many thousands of professional devices like the Nitrokey device. You can already program this kind of hardware in Linux, using clunky commands like:

# dfu-programmer at32uc3a3256s erase
# dfu-programmer at32uc3a3256s flash --suppress-bootloader-mem foo.ihx
# dfu-programmer at32uc3a3256s launch

The crazy long chip identifier is specified manually for each command, as the bootloader VID/PID isn’t always unique for each chip type. For fwupd we need to be able to program hardware without any user input, and without any chance of the wrong chip identifier bricking the hardware. This is possible to do as the chip itself knows its own device ID, but for some reason Atmel wants to make it super difficult to autodetect the hardware by publishing a table of all the processor types they have produced. I’ll cover in a future blog post how we do this mapping in fwupd, but at least for hardware like the Nitrokey you can now use the little dfu-tool helper executable shipped in fwupd to do:

# dfu-tool write foo.ihx

Or, for normal people, you can soon just click the Update button in GNOME Software which uses the DFU plugin in fwupd to apply the update. It’s so easy, and safe.

If you manufacture an AVR32 device that uses the Atmel bootloader (not the Arduino one), and you’re interested in making fwupd work with your hardware it’s likely you just have to add one line to a data file. If your dfu-tool list already specifies a Chip ID along with can-download|can-upload then there’s no excuse at all as it should just work. There is a lot of hardware using the AT32UC3, so I’m hopeful spending the time on the AVR support means more vendors can join the LVFS project.

Some great news: the Jabra Speak devices are now supported using fwupd, and firmware files have just been uploaded to the LVFS.

You can now update the firmware just by clicking on a button in GNOME Software when using fwupd >= 1.0.0. Working with Jabra to add the required DFU quirks to fwupd and to get legal clearance to upload the firmware has been a pleasure. Their hardware is well designed and works really well in Linux (with the latest firmware), and they’ve been really helpful providing all the specifications we needed to get the firmware upgrade working reliably. We’ll hopefully be adding some different Jabra devices in the coming months to the LVFS too.

More vendor announcements coming soon too.

tl;dr: If you feel like you want to donate to the LVFS, you can now do so here.

Nearly 100 million files are downloaded from the LVFS every month, the majority being metadata to know what updates are available. Although each metadata file is very small it still adds up to over 1TB in transfered bytes per month. Amazon has kindly given the LVFS a 2000 USD per year open source grant which more than covers the hosting costs and any test EC2 instances. I really appreciate the donation from Amazon as it allows us to continue to grow, both with the number of Linux clients connecting every hour, and with the number of firmware files hosted. Before the grant sometimes Red Hat would pay the bandwidth bill, and other times it was just paid out my own pocket, so the grant does mean a lot to me. Amazon seemed very friendly towards this kind of open source shared infrastructure, so kudos to them for that.

At the moment the secure part of the LVFS is hosted in a dedicated Scaleway instance, so any additional donations would be spent on paying this small bill and perhaps more importantly buying some (2nd hand?) hardware to include as part of our release-time QA checks.

I already test fwupd with about a dozen pieces of hardware, but I’d feel a lot more comfortable testing different classes of device with updates on the LVFS.

One thing I’ve found that also works well is taking a chance and buying a popular device we know is upgradable and adding support for the specific quirks it has to fwupd. This is an easy way to get karma from a previously Linux-unfriendly vendor before we start discussing uploading firmware updates to the LVFS. Hardware on my wanting-to-buy list includes a wireless network card, a fingerprint scanner and SSDs from a couple of different vendors.

If you’d like to donate towards hardware, please donate via LiberaPay or ask me for PayPal/BACS details. Even if you donate €0.01 per week it would make a difference. Thanks!

Today I released fwupd version 1.0.0, a version number most Open Source projects seldom reach. Unusually it bumps the soname so any applications that link against libfwupd will need to be rebuilt. The reason for bumping is that we removed a lot of the cruft we’ve picked up over the couple of years since we started the project, and also took the opportunity to rename some public interfaces that are now used differently to how they were envisaged. Since we started the project, we’ve basically re-architected the way the daemon works, re-imagined how the metadata is downloaded and managed, and changed core ways we’ve done the upgrades themselves. It’s no surprise that removing all that crufty code makes the core easier to understand and maintain. I’m intending to support the 0_9_X branch for a long time, as that’s what’s going to stay in Fedora 26 and the upcoming Fedora 27.

Since we’ve started we now support 72 different kinds of hardware, with support for another dozen-or-so currently being worked on. Lots of vendors are now either using the LVFS to distribute firmware, or are testing with one or two devices in secret. Although we have 10 (!) different ways of applying firmware already, vendors are slowly either switching to a more standard mechanism for new products (UpdateCapsule/DFU/Redfish) or building custom plugins for fwupd to update existing hardware.

Every month 165,000+ devices get updated using fwupd using the firmware on the LVFS; possibly more as people using corporate mirrors and caching servers don’t show up in the stats. Since we started this project there are now at least 600,000 items of hardware with new firmware. Many people have updated firmware, fixing bugs and solving security issues without having to understand all the horrible details involved.

I guess I should say thanks; to all the people both uploading firmware, and the people using, testing, and reporting bugs. Dell have been a huge supporter since the very early days, and now smaller companies and giants like Logitech are also supporting the project. Red Hat have given me the time and resources that I need to build something as complicated and political as shared infrastructure like this. There is literally no other company on the planet that I would rather work for.

So, go build fwupd 1.0.0 in your distro development branch and report any problems. 1.0.1 will follow soon with fixes I’m sure, and hopefully we can make some more vendor announcements in the near future. There are a few big vendors working on things in secret that I’m sure you’ll all know :)

Soon I’m going to merge a PR to fwupd that breaks API and ABI and bumps the soname. If you want to use the stable branch, please track 0_9_X. The API break removes all the deprecated API and cruft we’ve picked up in the months since we started the project, and with the upcoming 1.0.0 version coming up in a few weeks it seems a sensible time to have a clean out. If it helps, I’m going to put 0.9.x in Fedora 26 and F27, so master branch probably only for F28/rawhide and jhbuild at this point.

In other news, 4 days ago I became a father again, so expect emails to be delayed and full of confusion. All doing great, but it turns out sleep is for the weak. :)