The future of package management in Fedora

I spent a couple of days last week in Brno talking about the future of package (and application) management in Fedora. Things we discussed:

  • We currently don’t cater for desktop users by showing details about the packaging layer (GPG keys, packages, etc) and not being able to search while we install/remove
  • We need a centralized application store to stay competitive with other distros and OSs
  • Our metadata and package mirroring policies hurt end users not using the command line
  • I presented (and demoed) gnome-software with its plugin architecture that would allow us to switch to using packages and blobs like glick and listaller
  • I made the case for application metadata, so we can get things like localised application details, screenshots and ratings using a few different methods

Some interesting points came up, and this is approximately 30 hours of talking condensed into a few bullet points:

  • YUM upstream will soon be considered deprecated, and we will move into a DNF/hawkey/librepo-based future. This includes PackageKit. I’m going to be building a hawkey based backend with help from the maintainer, and he is is aware of what PK does and any unusual tasks that are performance critical.
  • We should keep old versions of metadata on the server to stop metadata refresh explosions happening where yesterdays primary gets updated because a transaction only has todays filelists installed. This will significantly reduce the amount of bandwidth used by the metadata updates.
  • We should keep old versions of packages on the mirrors, to avoid the case where we depsolve fine, get 404 on the package download and then have to re-download MD, depsolve, etc. YUM apparently has issues with multiple versions of a package being present in the metadata, so we should probably only reference the latest packages in the MD (which also keeps the MD to sane size).
  • We should ship the per-arch solv files in the repo MD. This avoids SAT solutions like libsolv from spending 20 seconds+ per repo rebuilding .solv files from sqlite or xml metadata, and allows us to kill the dnf cron job
  • We should teach rpm to update it’s own SAT database, which we can do with an RPM plugin.
  • We want a software center, and fedora-tagger can provide the ratings/comments information. We might need an OCS server for screenshots, or can tie in screenshots with automated QA somehow.
  • We are going to teach koji about appstream data, so a simple extract script (to be written by me) can produce a .tar file of icons and a .xml file of translated descriptions at the end of each koji build
  • We are going to teach the compose tools to xmlmerge all the appstream .xml files and ship as appstream.xml
  • We are going to teach the compose tools to join all the tar files and ship as appstream-data.tar.gz
  • We are going to investigate the use of meta-desktop files to install a super-set of applications, e.g. KDE, or “Python developer” which allows for screenshots, ratings and all that stuff.

It was an interesting couple of days, and quite a few people will be ping’ed over the next few days to make some of what we discussed a reality. I’m convinced the changes we can make here will give us a slick and featureful application installer, something that can really be an asset for Fedora and RHEL. Comments, as always, welcome.

Auto-EDID Results [updated]

A couple of weeks ago I asked people to run a command which uploaded all their auto-EDID display profiles to me. This was a massive success with 1858 profiles being added to a large dataset. These were scanned by the cd-find-broken tool, and results plotted on my G+ page. As there’s been so much new data I’m updating the graphs:

edid-vendors

I’m actually using this data to make sure we show something sane in the client UIs. Some interesting vendors are not included, e.g.:

  • “System manufacturer”,3
  • “To Be Filled By O.E.M.”,4

edid-vendors-broken

This is a chart of vendors Doing It Wrong™ by including random data (or implausible data) as the display primaries.

edid-cmf

This shows what program created the Auto-EDID ICC profile. Unknown is probably a mixture of oyranos and also early versions of gnome-setting-daemon which didn’t set the extra metadata.

edid-vendors-noserial

Last graph I promise. This shows a chart of all the vendors who do not populate the serial number in the EDID blob. I’ll explain why this is bad.

When we construct the device ID for colord, we use the vendor{-model}{-serial} as part of the key.  This allows you to use different ICC profiles even if you’ve got two “identical” external panels attached. Without the serial number, “lenovo-foo” looks the same as “lenovo-foo” and colord treats them as if they were the same panel. This sucks if the panels were not bought at the same time and have identical backlight burn time. Ohh, and we can’t use the connection name (e.g. DVI-1) as it would suck if you had to reassign all your profiles if you moved the connector to DVI-2…

This isn’t always a disaster: Laptops. We only need the make and model to ensure this is unique on the system as you can’t typically have two internal panels installed. This explains the Lenovo, Samsung, Dell and Apple entries I think, so don’t get out the pitchforks just yet. Unfortunately there’s nothing in the ICC profile that says “this is a laptop” so we can’t be more selective and hence this graph isn’t super useful. But, even on laptops, vendors should really be doing something semi-sane with the serial number, even if it’s just the batch number.

A new 0.1.34 colord was released this week. Thanks again to everyone that uploaded profiles.