The future of package management in Fedora

I spent a couple of days last week in Brno talking about the future of package (and application) management in Fedora. Things we discussed:

  • We currently don’t cater for desktop users by showing details about the packaging layer (GPG keys, packages, etc) and not being able to search while we install/remove
  • We need a centralized application store to stay competitive with other distros and OSs
  • Our metadata and package mirroring policies hurt end users not using the command line
  • I presented (and demoed) gnome-software with its plugin architecture that would allow us to switch to using packages and blobs like glick and listaller
  • I made the case for application metadata, so we can get things like localised application details, screenshots and ratings using a few different methods

Some interesting points came up, and this is approximately 30 hours of talking condensed into a few bullet points:

  • YUM upstream will soon be considered deprecated, and we will move into a DNF/hawkey/librepo-based future. This includes PackageKit. I’m going to be building a hawkey based backend with help from the maintainer, and he is is aware of what PK does and any unusual tasks that are performance critical.
  • We should keep old versions of metadata on the server to stop metadata refresh explosions happening where yesterdays primary gets updated because a transaction only has todays filelists installed. This will significantly reduce the amount of bandwidth used by the metadata updates.
  • We should keep old versions of packages on the mirrors, to avoid the case where we depsolve fine, get 404 on the package download and then have to re-download MD, depsolve, etc. YUM apparently has issues with multiple versions of a package being present in the metadata, so we should probably only reference the latest packages in the MD (which also keeps the MD to sane size).
  • We should ship the per-arch solv files in the repo MD. This avoids SAT solutions like libsolv from spending 20 seconds+ per repo rebuilding .solv files from sqlite or xml metadata, and allows us to kill the dnf cron job
  • We should teach rpm to update it’s own SAT database, which we can do with an RPM plugin.
  • We want a software center, and fedora-tagger can provide the ratings/comments information. We might need an OCS server for screenshots, or can tie in screenshots with automated QA somehow.
  • We are going to teach koji about appstream data, so a simple extract script (to be written by me) can produce a .tar file of icons and a .xml file of translated descriptions at the end of each koji build
  • We are going to teach the compose tools to xmlmerge all the appstream .xml files and ship as appstream.xml
  • We are going to teach the compose tools to join all the tar files and ship as appstream-data.tar.gz
  • We are going to investigate the use of meta-desktop files to install a super-set of applications, e.g. KDE, or “Python developer” which allows for screenshots, ratings and all that stuff.

It was an interesting couple of days, and quite a few people will be ping’ed over the next few days to make some of what we discussed a reality. I’m convinced the changes we can make here will give us a slick and featureful application installer, something that can really be an asset for Fedora and RHEL. Comments, as always, welcome.

40 responses to “The future of package management in Fedora”

  1. kparal

    I’m really looking forward to this. Thanks, Richard, for working on it.

    1. Pierre-Yves Luyten

      Yup, big stuff there! don’t forget to have a bugzilla product =)

  2. Stéphane

    What would be cool, would be also to ease the policy in order to accept some free software packages implementing patented (often only relevant for the US) technologies, this is hurting end users way more than the way you access the packages IMHO.

  3. Ondra

    could you please explain in more detail, how does this relate to AppStream initiative?

    http://www.freedesktop.org/wiki/Distributions/AppStream/

  4. sadig

    Hughsie,

    what does ‘YUM upstream will soon be considered deprecated, and we will move into a DNF/hawkey/librepo-based future. This includes PackageKit. I’m going to be building a hawkey based backend with help from the maintainer, and he is is aware of what PK does and any unusual tasks that are performance critical.’

    mean?

    Could you elaborate more?
    As Fedora is known as testbed for new RHEL technologies, how will that impact RHEL (7?, 8?) Admins?

    Will this change being incorporated in a repo admin system like Pulp?

    Would like to know about those plans.

    Thanks,

    \sh

  5. Matthias

    This is great news! Thank you for the update :) If you need any extra stuff for the AppStream part, I’ll help. Also, an AppStream-based software-center, which is not as slow as the USC, is currently being created (written in Vala).

  6. Israel Torres

    Noo! Yum don’t go! (Slow motion) :P

    Well. I undestand if it calls for better performance in Fedora, so thumbs up! Go for it! I’m already familiar with YUM though :(

    1. Craig

      dnf is just “faster yum” for most people. Users will barely notice the difference, except for the nice performance improvements.

  7. xxx

    Can you please solve the intel sandy bridge power regression problem on fedora (from the past 2 years till now) and then change the rpm?!

    1. Crag

      What has that got to do with packaging? Try the bug tracker.

  8. Matias

    Hi,
    Thanks!. I look forward these changes!!. =)

    A off topic.. In that state is zif?. I hoped that you try to use this, but you talk to replace yum with dnf..

  9. Dylan McCall

    Did you happen to see Ubuntu’s discussion about “Click Packages?” (https://launchpad.net/click-package, https://lists.ubuntu.com/archives/ubuntu-devel/2013-May/037074.html). If you haven’t, I suggest you take a look, because there might be some overlapping ideas. That project is in very very early stages, but I’ll be very excited if the major distros are aware of each other’s efforts on this front so we can maybe have some strong collaboration around application packaging in the future ;)

  10. George Machitidze

    Yep, that’s how it should be done. On the desktop. At least.
    DPKG vs RPM-related saga must come to the end for the desktop users, they must see same UI on all distros.
    Canonical made very bad thing by putting it’s own Ubuntu-only app manager. Working underground without OSS world is very wrong. There is a common middleware – packaging layer. Upper layer software must be able to work with everything. It’s not a kernel feature, which can run on Linux and won’t run on *BSD (sorry for that word folks).

    1. Craig

      That attitude in Ubuntu is not about to change any time soon. Mark Shuttleworth is too much of a Crony Capitalist.

  11. George Machitidze

    p.s. hughsie, “They’ve basically re-invented listaller in the click packages proposal”.
    Nope, it’s not their invention. Opensuse had same thing several years ago too, and it has click-installation support now, which is actively used with repos, not as separate library/app mess like in Mac OS X. And don’t forget about CNR too, there were several universal, distro-independent attempts too.

  12. cyan

    Ubuntu has a perfect app store which though cannot be compared to an Android one but its light years ahead of other distros. The mir part as long as they release the code in GPL and have good performance, what’s the big deal. I hope fedora solves all these installer problems, with F19 beta there are still basic problems with the installer and i hope fedora also shows some love to the desktop users which i feel Ubuntu and Unity are doing better than gnome

  13. Ade Malsasa Akbar

    It is amazing to see line here: “…hurt end users not using the command line…”. This shows your careness for your users.

    But maybe you should add a new line “..hurt end users don’t have internet…”. And I think you should add something like (or better than) Mintbackup on Linux Mint. Your package management system should able to backup what software you have installed. And user should able to choose what software, **not whole**. After backup, the software in one folder can be copied into another Fedora, and install it there. So, no internet again for another Fedora to install.

    This is the biggest thing missing (distribute the software installer/backup choosen installed software) in Linux I feel for years. Just my opinion.

  14. Brent McCray

    Awaiting changes and happy for the focus on features that would allow fedora to shine.

  15. Saleem Ansari

    This is great news! Looking forward to it. I use zif sometimes so I was curious to know, how would zif be related to this initiative ?

    1. Craig

      I guess the planned Hawkey PackageKit backend will be the new zif.

  16. Makaro

    why leaving yum instead of tweak that software ?
    I’ve use Fedora for 4 years and after this change I’ll leave Fedora for sure. And also about sofware center, did Fedora want to challenging Ubuntu?

    I’m just regular user who love using Fedora, but after too much changes in Fedora lately, I’ll gone and move to rolling release distro.

    thanks for take this out :D so I can leave earlier

    1. Craig

      “I’ve use Fedora for 4 years and after this change I’ll leave Fedora for sure.”

      Cool amateur dramatics, bro. dnf has almost exactly the same set of commands as yum. It just has different internals to make it faster, lighter on bandwidth, more correct etc. Just try it for 10 seconds and you’ll see that. Are you really prepared to change distro just because someone tried to make yum faster and better? I think it’s more a case that you’re too lazy to learn something with *very* little extra learning curve and too willing too whine like a baby.

  17. Wade Hampton

    Sounds very interesting. Any new install tools must support the following use cases:
    - simple end users with GUI, screen shots, etc. (think Android market)
    - servers using scripting or command line (update of YUM)
    - disconnected systems (systems not on the Internet, mirrored updates/packages)
    - near-seamless support of 3rd party and local repos
    — corporate or project-specific
    — isolated systems with local repos
    — cached repos
    — repos supporting patented software

  18. Craig

    Sounds like a great roadmap. The only thing that worries me is the insistence on everything being XML again. The amount of pointless serialization bloat and redundancy in the XML metadata is absolutely insane.

    The excuse that “XML compresses well, so it’s not a problem” is completely and utterly bunk too. It still has to be extracted, read, parsed and stored in memory. XML is a universal solution to a non-existent problem and is an absolutely horrible waste of resources, especially on low end hardware that doesn’t have the luxury of being wasteful.

Bad Behavior has blocked 2769 access attempts in the last 7 days.