A new glib library for PackageKit

For the last couple of weeks I’ve been re-writing the existing PackageKit-glib bindings. The old library was beginning to limit what we could do with client applications. The biggest problems were:

  • A lot of the dbus calls are sync, which slowed down application startup and user input “snappyness”
  • A lot of the methods were not cancellable
  • It’s very hard to add details and update details about a set of packages, without caching all the data in the client
  • Having to “reset” a single heavyweight client instance before doing each operation (mitigated in some respects with PkClientPool)

As the days have progressed, I’ve slowly add more functionality to the packagekit-glib2 library, with the aim of removing the original glib library for 0.6.x. After two weeks of hacking, I’ve deprecated the glib1 library, with everything in the PackageKit git master tree is now using glib2. Overall, working with everything asynchronous works really well. The GUI clients benefit the most, although it makes the text clients much simpler too:

[hughsie@hughsie-laptop client]$ wc -l pk-console*.c
 2580 pk-console-glib1.c
 1625 pk-console-glib2.c

One of the main benefits is that the “dance” (the multiple transactions) can be done asynchronously, and completely hidden from the client. To do this, I’ve got the following classes:

  • PkControl: For getting properties on the main interface, and doing methods such as GetTid
  • PkClient: For scheduling a single transaction
  • PkResults: For storing the completed results of the transaction (packages, update-details, all the compound objects)
  • PkProgress: For storing current progress, for passing to async clients
  • PkTask: For managing the “dance”, for instance, requeuing transaction for simulate, gpg-keys, eulas and for trusted. It superclasses PkClient.

Now, PkTask on it’s own isn’t very interesting, as it fails at every step of the dance by default. PkTask does however have klass methods that can be overridden by classes implementing PkTask. There’s an example I made for the “make check” functionality called PkTaskWrapper which basically accepts each step of the dance without asking the user. More interestingly, there’s a class called PkTaskText that implements PkTask and handles all the callbacks using console commands like fgets. That’s what text stuff like pkcon and pk-debuginfo-install are now using, and why they are a lot smaller now.

This means that pkcon can create an instance of PkTaskText, and all the interactions are done for it in the PkTaskText object. The actual interaction is hidden from pkcon, and all it gets is the progress callback as the transactions are scheduled. And because PkTaskText can be used as a PkTask, it can also be used as a PkClient, so the methods that are not wrapped can be used with a single object. It also means that other clients can just use PkTaskText and get all the complex interaction stuff handled automatically.

Of course, in gnome-packagekit, it will make things much cleaner as each PkClient can be converted to use an async GpkTask, which will handle all the interactions. So, instead of having all the scary logic about what methods to send in response to different signals and return codes in multiple places, we can just derive from PkTask and do them all in one place. Of course, moving from a 80% sync library to a 99% async library is going to be hard for projects that make heavy use of libpackagekit-glib. The most obvious example is gnome-packagekit, and so far I’m already about 80% they way through. Diffstat reports “59 files changed, 6311 insertions(+), 8684 deletions(-)” with the majority of porting work just switching code to use the async variants. When it at least compiles I’ll push it to git master, but until then it lives in the glib2 git branch.

The new code is much cleaner, more debuggable and most of all supportable, so when 0.6.x is released (a few months away) we can remove a metric ton of code (~20000 LOC) from the server. In preparation we’ve written a porting guide, although it’ll get more love when the design for packagekit-glib2 is set in stone.

PackageKit and device rebinding

A few people mentioned on my last blog post that instead of nagging the user to unplug or restart, we should just rebind the device. I didn’t do this yesterday as it was quite hard to do the layering correctly as the session is running as a normal user.

But of course, doing things correctly is often harder than doing things quickly. I’ve merged an optional helper into PackageKit that just pokes the hardware in the right place to make it rebind and re-request firmware. This is all done using the pkexec functionality in PolicyKit1, and a custom policy rule.

Device has been virtually unplugged and plugged in
Device has been virtually unplugged and plugged in

If you want this new functionality you need to build contrib/device-rebind in the PackageKit project, and also have GUdev and PolicyKit1 installed. If you don’t have these things, the session will fall back to just asking the user to reboot.

Ohh, and if the rebind fails in any way, we just fall back to asking the user to re-plug or restart like we did before. Device rebind functionality is currently Linux specific, but patches for other operating systems welcome.

PackageKit and firmware

For a few months now, PackageKit has been able to install firmware for devices. Sometime in 0.5.x series the functionality broke, so I spent the morning fixing up the module properly.

So, when you start your session (or insert the device), you get greeted by:

Firmware request
Firmware request

Notice, we now display the device model, but in this case where a device is waiting for firmware, normally the results are not complete and thus not pretty. USB WLAN is the best we can do in this case. If the user clicks install, the install continues in the background, and then the user gets a few minutes later:

Replug please!
Replug please!

but if it’s not a removable device:

Restart please!
Restart please!

Of course, you’ll need a pretty new distribution to have GUdev installed, but if you don’t it’ll fall back to being unhelpful.

gnome-power-manager and blanking (removal of bodges)

I’ve been working with the xorg people upstream, trying to sort out all the remaining blanking problems properly, rather than just working around the problem. I’ll explain the key issues:

gnome-power-manager uses a counter inside Xorg called IDLETIME. This counter is incremented only when the user does not move the mouse, or click some keys. When the user clicks something, the IDLECOUNTER is reset. Unfortunately, the IDLETIME counter was also being reset (in two places!) when the DPMS level is set. Now, this doesn’t affect most users of IDLETIME, as the screen doesn’t blank that often. For the most part, IDLETIME was a welcome addition to the X server.

For gnome-power-manager, we set up a XSync alarm for IDLE counter being over a certain value, and then we set up a XSync alarm for the IDLE counter being reset. When the alarm goes off we wait the policy time for the “display sleep” and then turn off the panel using DPMS. Which then resets the idletime, which turns the panel back on. Urgh.

So, what we do is handle the reset event, and if the event is less than a few milliseconds since we did a DPMS action, we ignore the alarm. Of course, if we ignore the alarm, then we don’t get the reset event when the user moves the mouse and the IDLECOUNTER gets reset. So, in this case, gnome-power-manager sets up a 1ms timeout when we detect an alarm a small time since a DPMS event. This triggers almost immediately, and so we get the alarm fired practically straight away.

Except, due to another X bug, if we set an alarm value on the timer that’s already been passed, we don’t get the alarm fired. So, if you’ve got a high load value, or a slow system, you could miss the alarm. So, we had to raise the bodge alarm value to 50ms, rather than 1ms. Urgh.

But then, there’s a nice 50ms race between the two timers, and 50ms is a small amount of time in human terms, right? No. When the user is reading something, and the display blanks, most users move the mouse pretty much straight away. If you hit this 50ms race (which some people seem able to do, me included) then gnome-power-manager misses the reset event, and if configured to do so, gnome-power-manager will still think your idle, and then go on to suspend your system. Urgh.

So, the only way to fix gnome-power-manager and remove all these ugly kludges would be to fix the xserver. I’ve sent two patches to xorg-devel which remove the IDLECOUNTER reset when DPMS off is sent. The second upstream patch is here.

So, I’ll remove the kludges from gnome-power-manager git master today and will depend on a runtime version of the xserver that has these patches applied. If you are trying to run gnome-power-manager with a broken version of X, gnome-power-manager will warn you in the notification area. Distributors will just need to patch xserver with my previous patch and the current one to have all the issues resolved with git master.

edit: updated with links to the signed off patches in xserver!

Blanking in gnome-power-manager (fixed!)

So, Peter Hutterer is my new best friend. Peter has found and patched the bug in the X server that was causing the failure of negative transition triggers in XSync. For you and me, that means that XIDLETIME now works as expected, and gnome-session and gnome-power-manager do the right thing. This means no more random blanking when typing.

There’s a pending Fedora 11 update here and a build for rawhide here — it should be tagged for F12alpha, but no promises at this stage.

The important patch for the xserver is here and I urge all distros to cherry-pick this commit into their stable branch, as this affects a lot of users. Hopefully this is the end to the saga of accidental screen blanking.

GtkPlug, and browser plugins

Anyone familiar with GtkPlug? I’m having issues with the PackageKit XEmbed browser plugin — unless I set the sizes for the widgets using gtk_widget_set_size_request, or set the widgets to expand to all the space, I get widgets with close to zero width and height:

The line across the widget is meant to be a GtkButton! Ideas welcome, and experts rewarded by beer! If anyone has examples of how to do a browser plugin with GTK controls, that would be great too. Thanks!

Accidental blanking and gnome-power-manager

Okay, after having F11 g-p-m blank the screen on me whilst watching some short videos in totem yesterday, I got really angry. It seems lots of other people feel the same way.

I’ve audited all the IDLETIME code in gnome-power-manager multiple times, and when I’ve run it in a console and watched the output, it all seems to work 100% okay for days on end, and then fails when you’re actually trying to do something. And then I had an epiphany: It only seems to fall over gnome-session is involved with handling inhibits.

So, this works:

GPMIDLEDEBUG=1 ./gnome-power-manager
…wait 10 seconds…

IDLETIME fires the idle alarm expired (and g-p-m dims the screen) and the reset alarm is setup

…move the mouse…

IDLETIME fires the reset alarm expired

Then I issue an inhibit request to org.gnome.SessionManager with parameters (‘test-program’,0,’testing’,8) using d-feet and get back a cookie like normal. Session becomes inhibited.

…wait 10 seconds…

IDLETIME fires the idle alarm expired (but g-p-m doesn’t dim the screen, as the session is inhibited from totem) and the reset alarm is setup

…move the mouse…

NOTHING. No event from X.

…close d-feet…

The inhibit gets auto-revoked, session becomes non-inhibited, and g-p-m assumes that x has been idle for a long time, and also the session is not inhibited, and so switches off the screen. You can see this when using GPMIDLEDEBUG as the second icon is a box, not a computer icon.

Now g-p-m is confused, and has to be restarted before it will reset the new idletime counter. You can’t reproduce with the original idlecounter-demo program when using XNextEvent, but you can as soon as you hook into gdk with gdk_window_add_filter. It really looks like something is doing GDK_FILTER_REMOVE on the reset alarm at some point. There’s a test program here which you can see the bug without gnome-power-manager running, just to prove it’s not a silly bug or race in gnome-power-manager. It could also be an X bug, as the XSync stuff isn’t that widely used, although all gnome-session is doing is XSyncDestroyAlarm’ing an alarm, which shouldn’t affect gnome-power-manager’s alarms at all.

After looking in the forums, this problem looks like it’s triggered lots, and by many different users. I would appreciate any help here as I’m well and truly stuck. Thanks.

Richard.

p.s. if anyone knows how to debug gnome-session to see the debug output, I would be very grateful. Any attempts at replacing gnome-session process in my session lead me to a forced logout.

Firefox plugin woes

The PackageKit browser plugin hasn’t had a lot of love recently. It was written in C++ a few months ago by Owen, but started to bitrot over the last few xulrunner releases.

I’ve spruced up the code, and ported everything to C (using GObject where possible) and now it all seems to work with one exception: invalidaterect doesn’t seem to work where I think it’s supposed to.

I’m setting up the plugin with some default content, which gets shown to the screen. I’m then doing a PackageKit query, and a few ms later I get the results. I then update some internal state, and call the invalidaterect for the whole drawing area, expecting a GraphicsExpose event to render the new content. But alas, invalidaterect seems to do nothing.

I’ve even tried adding a forceredraw call after invalidaterect, but that also gets ignored. If I manually resize the epiphany/firefox window then the GraphicsExpose event gets called, and the plugin then shows the correct state. This is a windowless plugin and nsplugginwrapper has been removed, and so I think I’m doing everything by the book, so to speak. Help welcomed, and anyone pointing out the bug will be rewarded with a beverage of your choice. Code is here. Thanks.

Updating shared libraries

I want to add functionality to PackageKit to detect when a new version of a shared library is installed, and there are applications still using the old version that no longer exists. We can then inform the user if they need to restart the computer or log off and back on if the library was updated for a security vulnerability.

Does anyone have any example code (preferably written in C, although I’m not that bothered) that I could use? I think this would be quite nice functionality in future versions of PackageKit.

Thanks.