Application installing

In the Linux desktop we have a very big problem: We focus very much on packages. Packages are interesting to programmers, but users care about applications.

I’ll explain the difference:

  • Packages can contain none, one or multiple applications. Packages are called “openoffice-common”
  • Applications only belong to one package. Applications are called “OpenOffice.org Writer”

Now, when a user wants to install an application, they have to research on Google what the package name is called (which is different on each distro) or hope that the application name is mentioned in the description of the package in the distro metadata. Not ideal at all.

Now, I said as a desktop we have one big problem, well, we actually have two: We don’t all speak English well. Some of us speak no English at all.

We want to be able to display package descriptions to the user in all languages, but we don’t want to download 80x the metadata to do so. Most packaging systems only understand en_US anyway, and there certainly isn’t the resources to translate every spec file or emerge instruction for each distro. To add to the problems, each package needs an icon, in various sizes so we can show the application icon rather than a generic box.

So we’re sunk, right?

No. In each package, there are desktop files that contain all the applications, with nice translations gently massaged and QA’d by upstream. It would be nice if we could search on that data. At the moment, this is impossible, unless we want to download every package in the archive, and extract the data from it. This is sort of how Ubuntu does gnome-app-install, and it seems to work fairly well. It is Ubuntu specific, but maybe we could work on that.

So, we cache all the desktop files, and push this out to the repo metadata?

No. If you did that you make a lot of people very unhappy, as even compressed the metadata and icons make up over 80Mb.

So we’re sunk, right?

No. What we can do is create sub-packages for each repo (e.g. rpmfusion-appdata) which ships a tarball of icons and a few hundred Kb of SQL. Every time the repo maintainer can be bothered (once a month?) the new data is generated, and a new package pushed out to the mirrors. If the repo maintainer can’t be bothered to do that, then none of the new packages will show up in the application browser. It’s optional.

This data clogs up my system right?

Well, it’s only a few tens of Mb if you want all the icons in most of the sizes, if you only choose the 48×48 option then it’s much less. When you install the new $repo-appdata subpackage, it removes all the stale applications and re-adds the latest data. This happens in the vendor spec files as postinst scripts.

How do I query the data?

It’s a simple sqlite database in /var/lib/app-install/desktop.db — the icons are located in /usr/share/app-install/icons/$size/*.png — there’s no GUI installer yet using this, but expect a few before two long.

Great! Another Ubuntu v.s. Red Hat standards war!

No. Roderick Greening, Sebastian Heinlein and myself together drafted the specification together, and made it generic enough for all the distros to use. It’s totally expected that each distro will code a different tool to extract the metadata, but that’s because they are different in some important ways.

So the maintainers have to install everything just to get the desktop files?!

No. You can download and install a package to a prefix without the deps — we don’t need the binary to run, we just need the data. in this way we don’t need to download -data subpackages, only the one with the desktop file in.

Can I add some more features to the spec?

Yes, in a little while. We want to get version 1 of the spec finished, with it being used in a few distros. When we’re comfortable this works correctly, we’ll start working on version 2, and add stuff like popularity metrics and metadata about suggesting gnome-power-manager rather than kpowersave if you’re running GNOME. There are lots of things we need to add for this to work really well.

So, Comments?

Working at Red Hat

About two years ago, I was at a conference with a load of GNOME people. I mentioned over drinks to two friendly Red Hat hackers, that I had an idea about a packaging framework. It was just an idea, as I was working for a large defence company, and had precious little time to maintain gnome-power-manager, let alone start anything new.

Nearly a year ago, I was hired by Red Hat to work on power management stuff. Now, my boss is one of those cool bosses that gives you quite a bit of ‘space’ and with his blessing I started to hack on PackageKit. 18 months later PackageKit is feature-complete, and the defacto standard across a dozen or so distributions.

Every week or so, I put up a new screenshot on this blog of cool stuff I’m working on. Every week people critique my ideas, and I go away to fix them up so the next version is that little bit better. Every week a few people say thanks, and tell me I’m doing some cool stuff, which is nice.

I don’t want people to think that PK or DK-p are Red Hat projects, but the simple economics is that they pay me to hack on cool projects. Whilst working at Red Hat I’m working alongside the very best people in the industry, in an environment that rewards innovation and thinking a little bit different.

What I’m trying to say is, most things you see on this blog are possible because of Red Hat. Sometimes I feel Red Hat doesn’t get the credit it deserves. Red Hat Rocks.

GStreamer vs. Windows Media Player

I’ve always been quietly impressed with GStreamer — in a world of multimedia where everything is so complicated, applications using GStreamer pretty much “just work” without any problems.

This was until today. I subscribe to a Polish TV archive, that has archived versions of soaps my girlfriend enjoys (M jak miłość). Windows Media Player 11 plays the mms stream perfectly, with no visual artifects or sound skipping, but in totem the video is unwatchable. As a further point, mplayer seems to play this mms stream with no video glitches, and only an occasional audio blip.

On gstreamer-0.10.21-2.fc10.i386, using playbin and debug level 2, I get:

0:00:11.977812682 15562  0x8ab4e50 WARN              asfdemux asfpacket.c:104:asf_payload_find_previous_fragment: Previous fragment does not match continued fragment
...
0:00:19.046052292 15562  0x8ab4e50 WARN              asfdemux asfpacket.c:336:gst_asf_demux_parse_payload:<asfdemux0> Offset doesn't match previous data?!
0:00:19.046086304 15562  0x8ab4e50 WARN              asfdemux asfpacket.c:104:asf_payload_find_previous_fragment: Previous fragment does not match continued fragment


Which I guess means the payload timestamps are wrong, then then I get about a bazillion of:

0:00:31.263371321 15562  0x8e0b408 WARN              GST_PADS gstpad.c:2992:gst_pad_iterate_internal_links_default:<selector_video_src1:src> Making unsafe iterator
...

and then when it skips:

0:00:36.949382321 15644  0x9d0c538 WARN         baseaudiosink gstbaseaudiosink.c:1395:gst_base_audio_sink_render:<audiosink-actual-sink-pulse> warning: Compensating for audio synchronisation problems
0:00:36.949410537 15644  0x9d0c538 WARN         baseaudiosink gstbaseaudiosink.c:1395:gst_base_audio_sink_render:<audiosink-actual-sink-pulse> warning: Unexpected discontinuity in audio timestamps of more than half a second (0:00:00.649705215), resyncing
WARNING: from element /GstPlayBin:playbin0/GstBin:abin/GstAutoAudioSink:audiosink/GstPulseSink:audiosink-actual-sink-pulse: Compensating for audio synchronisation problems
Additional debug info:
gstbaseaudiosink.c(1395): gst_base_audio_sink_render (): /GstPlayBin:playbin0/GstBin:abin/GstAutoAudioSink:audiosink/GstPulseSink:audiosink-actual-sink-pulse:
Unexpected discontinuity in audio timestamps of more than half a second (0:00:00.649705215), resyncing
...

This makes the video unwatchable. I’m guessing the server is somehow broken, but it works in Windows Media Player perfectly and 99% okay on mplayer.
Any help on how to debug this or report a proper bug gratefully received. Thanks.

gnome-power-manager and DeviceKit-power

DeviceKit-power is a system activated service I’ll be writing about in detail in a blog post in the near future. It’s a bit like HAL, but makes common API calls easy for applications (is battery power low?) and also moves the battery profiling down to the system layer from the session so it works for all users and more importantly from the GDM login screen. It’s very lightweight and low on resource usage when it’s active.

Here are some screenshots from gnome-power-manager trunk (compile with –enable-devkit-power).

Statistics profile for my battery
Statistics profile for my battery
History for my Watts Up Pro
History for my Watts Up Pro

The monitor device is a Watts Up Pro device. As you plug it in, DeviceKit-power opens the device, and starts reading data. I’m using it here to monitor my power strip with my laptop and all the power-warts attached. I’ll be converting gnome-power-manager to use DeviceKit-power rather than HAL during the next release cycle. Hopefully, I’ll maintain both HAL and DeviceKit-power backends, although this isn’t making the number of #ifdefs any smaller, which sucks from a readability point of view.

The GUI’s are pretty raw, so suggestions and criticisms are welcome.

PackageKit self checks

PackageKit is quite a complicated code base. As with all my projects, there is a substantial self check framework that’s designed to catch bugs and regressions before we push releases. This was something enforced by my previous employer, but I’m sure it’s a good idea for any non-trivial code base, and it’s something I’ll continue to do.

Self check example
Self check example

The number of tests currently:

daemon: 216
libpackagekit: 202
gnome-packagekit: 75

The tests are all injecting valid and invalid input and then testing if the code does the right thing. This works really well for the daemon and the library, but does not work well for the GUI applications that need a full GUI framework.

I’ve tried dogtail, but I’m finding it hard to use, and really wanted something I could integrate with my existing system in C. Do any of you hackers recommend anything in particular or should I persist with dogtail? So far it’s looking best of the bunch.

PackageKit Collections

From PackageKit 0.3.3 onwards, a new type of package is supported called a collection.

 What gpk-application looks like with collection support

A collection is a metapackage that can represent a group, where the package mapping is done inside the backend. It is not like a catalog where an external file provides a meta-group. This enables the “group install” and “group remove” functionality people have been requesting since version 0.1.0, without get another abstract group mapping or extra API to support.

I’ve made a few changes in the daemon to properly support this new type, and in the yum backend Tim Lauridsen has added support using the comps group mapping. Mike Langlie has drawn us some great icons, and Anders F Björklund implemented collections for smart, and helped us with the design process. The extra type should be trivial to add to other backends too.

We added the feature in just a few hours of hacking, as everyone was working together. I’m amazed at how people work together so well all over the globe, in different timezones and with different work priorities – thanks guys.

Comments, as always, appreciated