Technical Blog of Richard Hughes

Two new features coded today:

For PackageKit we don't want to be refreshing caches or checking for updates when there is no network connection. Does anyone have a short guide on how to monitor and query the connected state of nm (session or system), just like evolution does? Any nm guys want to write me a quick patch or jot down some quick code? Cheers.

EDIT: Done, thanks guys.

git clone git://people.freedesktop.org/~hughsient/PackageKit

Patches accepted. Thanks.

edit: Now git clone git://anongit.freedesktop.org/git/packagekit

Work on PackageKit continues. The new manager application has lots of human-unfriendly options now, but allows people to experiment with all the features and more importantly allow people to look at the code and work out how to port their app to support PackageKit.

There's quite a few API additions now also, with different search modes and also error codes and descriptions. The API will grow some more soon also, to get the extended description of a package along with a URL and any further information not contained in the summary. Then I think it's complete.

I've now split the dummy and yum backends, so in theory the yum backend could be soon complete. I'm adding the yum scripts as skvidal is kindly writing them, and hopefully everything should soon be in place for the other backends.

So, answering some FAQ's (I've had hundreds of emails about PK in the last 2 weeks):

• Hopefully gnome-app-install will use the PackageKit API soon. Sebastian Heinlein is already volunteering to do the porting.
• A conary backend can be created really soon. I just want to finish a little more the yum backend and move some of the shared spawn code so the minimum code is duplicated. I'll also need to add a shitload of documentation.
• A trivial apt backend will be created by me doing screenscraping of apt-get. A much better backend can be created by someone who knows C++ and is familiar with libapt.
• You don't have to have a backend that supports percentage updates. If you don't know the progress, just emit NoPercentageUpdates and then the UI should just do the right thing and spin.
• Backends don't have to support all options of all methods. Just set an error and return false if not implemented.
• Error codes have to be standardized so they can be localized. The error detail field can just be the untranslatable output. If you are creating a backend and you need another error enum, yell, and we can add it to the supported list.
• Upgrading, installing or removing packages has to be silent. The user cannot be prompted for questions “set applet setuid?” as these will not be handled in PackageKit. This is not an issue for rpm but will be for apt. The backend should do the right thing, as these messages mean very little the the average user and cannot be translated.
• PackageKit does not replace up2date. PackageKit is a way for users to interact with the packaging system, not for an administrator to install software on remote machines.
• PackageKit is not yet another system daemon. It quits when not doing anything, and only starts when something wants information or a task completed.
• PackageKit does not do dependency resolution. This problem has already been solved by the backend systems and I don't really want to re-invent the wheel.
• PackageKit does not have the fine-grained API to do everything. For instance, synaptic should still use libapt as can do much more than can be provided by PK.
• PackageKit is not a replacement to red carpet. Red carpet was really great and years ahead of it's time, but tried to do everything package related on the system, and moved onto the enterprise centralized management model. Although cool, this latter point made things too political IMO.
• PackageKit is not vapourware. I'm just waiting for fd.o git space and if that doesn't come about soon I'll just dump the code on code.google.com for a temporary solution. Then I'll start taking patches. All the applications work, they are not mockups. There are F8 rpm's in my repo also if you want a sneak peak.
• PackageKit is fast. It takes about 1 second to search for installed and available packages with “power” in the description – no blocking of the UI happens at all as it's all async. All the applications start instantly with no root password needed at startup.
• PackageKit is designed from the ground up to work with fast user switching and logging in and out of sessions during upgrades. You can start a system upgrade, log out, log in as another user, and be notified when the upgrade is complete, all without risking your rpmdb.
• PackageKit does not stop you using the low level tools yourself. You can use rpm and yum or dpkg and apt when PK is inactive (99.999% of the time) when you really need some command line love. No more fighting over yum, yum-updatesd, pup and pirut.
• PackageKit is not sponsored by anyone. Whilst interning at RedHat I spent a few days on the API and some UI designs, but all the coding is done in my free time.

So, good things.

PackageKit now has an update applet and application installer. This demo-application will win no awards for UI design, but the idea is to show how easy it is to make an async frontend for the provided dbus interface.

Also, the trivial update icon runs once per active session, and communicates with a single system instance of PackageKit. When the service is needed it is activated using dbus system activation, and when it's doing nothing the system daemon quits.

This means we can queue multi-session updates and installs. For instance I can start an update in my user session, switch to my girlfriends session, and then search for pidgin, and queue the install. The install will start when my update has completed. Maybe a more useful situation would be for the gdm login to start doing the daily updates before a user even logs in. OpenOffice could install language packs in the background as soon as you start typing in a new language. Lots of fun stuff.

Of course very little of this actually works properly. The dummy backend does mostly the right thing, and the apt and yum backends can search, but I still need to do the rest of the actions. The PolicyKit bit is still to be done.

So, good things.

What I've done:

• Defined a DBUS API
• A system daemon that manages asynchronous jobs, and sends out notifications of job status changed
• A dummy backend doing all the actions but just time-wasting (acts as a nice self-test)
• A yum backend that can do async searching (nothing else yet)
• An APT backend that can do async searching (nothing else yet)
• A client gobject library for easy integration for GNOME stuff
• A command line client that can work async or sync.

What's left to do:

• Full API for yum and apt
• A better apt backend that can do percentage completion signals (requires using libapt-pkg and C++…)
• Other backends (portage, smart, urpmi etc)
• Better documentation
• Update applet proof-of-concept (should be easy using the gobject library)
• Simple application installer proof-of-concept

So basically the idea works. Or at least I've not come up against any insurmountable problems so far. Expect the code to hit fd.o in the next week or so when I've polished up the hacks and got rid of a few FIXME's.
I'm away from an Internet connection for the next few days, so if you send me mail it will take me a few days to reply.

PackageKit is the name of the system package abstraction. Why PackageKit? Well, I was getting jealous of all the other cool stuff happening with daemons ending in Kit and wanted my own. And, if I type the word “manager” once again I'm going to have to rip my hands from my arms. :-)

There is a wiki page with all the use cases so far and some UI designs. Feel free to add comments or suggestions at the end of the page.

I've spent most of today working out a asynchronous DBUS API that can work with all of the use-cases, and starting a system activated daemon that can manage the sequential jobs. The daemon now 80% works, and I'm just writing a “dummy” backend with a “do nothing but take time doing it” role for testing. The other backends (yum/apt) are in the source tree but are currently empty files.

I only know a little about APT, so now I need your help. I need a way of searching for installed and not-installed software. I think I need either:

• a C binding for APT (I'm guessing that doesn't exist)
• a way of APT letting me search for “power” and returning tab delimited output.

The output would ideally be one line per entry:
gnome-power-manager<tab>GNOME Power Manager<tab>GNOME Power Manager is a system daemon that eats lots of power
power-pacman<tab>Power Pacman<tab>Power Pacman is a cool game where a dot eats other dots

The same thing for YUM would also be great as well, although I'm focusing on the apt backend first as I think it's the easier target for now. Thanks!

How about these as UI designs? Click for a bigger image. I'm no UI design guru (and I was advised not to do art for GSCE…), so  please complain loudly if you don't agree or have better suggestions. When we've got a simple GUI fleshed out we can work towards user interactions and how to use an API. Thanks.

Before we talk about API and system requirements we should talk about use cases and system interactions. I think this is where update systems have gone wrong in the past, closely integrating with the existing package system rather than studying the complete ideal user interactions.

Feel free to disagree and correct the interactions – either email me or preferably leave a comment.

Boot Time Security Update

Toby logs into his GNOME desktop. A notification area icon with a critical icon appears in the top right and a libnotify popup tells him there are 3 three critical security updates. The libnotify popup has three buttons:

• Update now in the background
• Always do updates automatically
• Ignore for now

Toby clicks the first button and the update completes in the background. When completed, after a few minutes, another libnotify popup appears telling Toby that the update was completed and after a few seconds the status icon disappears.

Downloading an Unknown Application

Suzanne wants to open a word file. She opens the software finder tool and types “office file” into the search box. A list of software appears, with OpenOffice being the top entry. She clicks the OpenOffice entry to highlight it, and clicks “Install now”. Suzanne is not an administrator, but because she is locally logged in and the package is from the “distro core signed repository” the root password is not required. A notification area icon appears with a downloading icon and the package manager is closed. When OpenOffice is installed, a libnotify popup tells Suzanne that the software has been downloaded and is now ready to use.

Installing debuginfo files automatically

Simon wants to borrow the computer while Suzanne waits for OpenOffice to download. He uses fast-user switching to switch to a new login. He notices the same downloading icon in his session which indicates Suzannes' download is still in progress. He starts Pidgin which then crashes. The bug-buddy window appears which prompts him to install the debuginfo so a valid backtrace can be detected. He clicks yes, and a libnotify windows appears telling Simon that the request has been queued and that he will be notified when the debuginfo has been installed. When installed, the bug-buddy helper continues and submits a valid bug.

Installing new features

Suzanne switches back to her session and wants to add some clipart to the word file she has just opened. She clicks “Insert” and then “Clipart” and then a windows pops up telling her that clipart is not installed. She clicks “Install” and a progress bar appears and moves across as the clipart is downloaded and then installs. When finished, the dialog disappears and she chooses a picture of a cat.

Installing and updating software is a really crap experience in most of the Linux distributions that I've used. Problems:

• You run the software install as a user session as root, meaning if you start an update, and then log out, or X crashes you loose your rpmdb or equivalent installer state.
• If you start a software update and the GUI update tool hangs you loose your rpmdb.
• There are long “hangs” with no progress bars, at least on Fedora.
• Every time you log in to a new user a new update tool the entire STACK is loaded, once for each user.
• The update tools and install tools are distro specific and a PITA to maintain.
• Most of the distro tools (yes I'm looking at pirut and pup here) suck ass from a UI and HIG point of view.
• You can't run a update notifier, a software installer and a package query tool all at the same time.
• The fedora ones take AGES to load and actually let me search for software.
• All the tools ask me for a root (or user) password. I own the bloody machine, I shouldn't need to do this.

Don't get me wrong, apt is fantastic, yum is still pretty good, but the user experience is utter crap. Ubuntu are the clear winners so far, but still fail on many issues outlines above. The first thing I do when installing fedora is uninstall pup and yum-updatesd, as they are just too broken for me to use. So rant over. What should we do about it? Package management as a system service. Yes, per system. When you install packages you do it as a the root user, usually for all users so it makes sense to do this in the system context. Using the new system activation stuff also allows us to start the service only when it is actually needed, i.e. when someone is trying to install something, or there is a logged in user who wants updates. APT, yum, portage etc as backends Yes, so users can still drop to a root shell and use the full command set of the native tool, but common stuff is abstracted over nice abstract interfaces. DBUS API? Sure, then we can lock it down per user or per group on interface type, so for instance let all users in the admin group install and remove packages, but only let students query packages. Common tools? YES!! We choose best of breed. We have one GNOME upstream update watcher (the ubuntu one is very good) we have one GNOME software install tool for power users (synaptic is very, very good) and also a GNOME tool for end users. The ubuntu simple application chooser is also very good. Details? Remember this is just designed for common stuff, rather than abstract every little detail of the underlying backend. These are my ramblings so far:

 org.freedesktop.PackageManagement.Query as=FindPackages(s) throws NotFound() i=GetInstalledVersion (s) throws NotInstalled() as=GetOptionalDependencies(s) throws NotFound() as=GetSuggestedDependencies(s) throws NotFound() as=GetForcedDependencies(s) throws NotFound() org.freedesktop.PackageManagement.Manage b=RemovePackages (as) throws OtherPackagesDepend(as), NotFound() bas=RemovePackagesWithDependencies (as), NotFound(as) b=InstallPackages(as) throws NotFound(as), DependenciesNotSatisfied(as) bas=InstallPackagesWithDependencies(as) throws NotFound(as) ::RemovedPackages(as) ::InstalledPackages(as) org.freedesktop.PackageManagement.Update b=IsThereUpdates(i) a(sis)=GetUpdates() throws NoUpdates() b=UpdateSystem() throws NoUpdates() ::NewUpdates(a(sis)) 

Where to go from here? I don't know. One thing I don't know how to fix is the callbacks for percentage completeness. Do we have to make the API asynchronous? Is there interest in fixing this stuff? Am I the only person who thinks the current bodges are insufficient for the Linux Desktop of the future? Please, leave comments (or email me) with any suggestions, corrections, or insults. Thanks.