Using Tailor to Convert a Gnome CVS Module

In my previous post, I mentioned using Tailor to import jhbuild into a Bazaar-NG branch. In case anyone else is interested in doing the same, here are the steps I used:

1. Install the tools

First create a working directory to perform the import, and set up tailor. I currently use the nightly snapshots of bzr, which did not work with Tailor, so I also grabbed bzr-0.7:

$ wget http://darcs.arstecnica.it/tailor-0.9.20.tar.gz
$ wget http://www.bazaar-ng.org/pkg/bzr-0.7.tar.gz
$ tar xzf tailor-0.9.20.tar.gz
$ tar xzf bzr-0.7.tar.gz
$ ln -s ../bzr-0.7/bzrlib tailor-0.9.20/bzrlib

2. Prepare a local CVS Repository to import from

The import will run a lot faster with a local CVS repository. If you have a shell account on window.gnome.org, this is trivial to set up:

$ mkdir cvsroot
$ cvs -d `pwd`/cvsroot init
$ rsync -azP window.gnome.org:/cvs/gnome/jhbuild/ cvsroot/jhbuild/

3. Check for history inconsistency

As I discovered, Tailor will bomb if time goes backwards at some point in your CVS history, and will probably bomb out part way through. The quick fix for this is to directly edit the RCS ,v files to correct the dates. Since you are working with a copy of the repository, there isn’t any danger of screwing things up.

I wrote a small program to check an RCS file for such discontinuities:

http://www.gnome.org/~jamesh/code/backward-time.py

When editing the dates in the RCS files, make sure that you change the dates in the different files in a consistent way. You want to make sure that revisions in different files that are part of the same changeset still have the same date after the edits.

4. Create a Tailor config file

Here is the Tailor config file I used to import jhbuild:

#!
"""
[DEFAULT]
verbose = True
projects = jhbuild
encoding = utf-8

[jhbuild]
target = bzr:target
start-revision = INITIAL
root-directory = basedir/jhbuild.cvs
state-file = tailor.state
source = cvs:source
subdir = .
before-commit = remap_author
patch-name-format =

[bzr:target]
encoding = utf-8

[cvs:source]
module = jhbuild
repository = basedir/cvsroot
encoding = utf-8
"""

def remap_author(context, changeset):
    if '@' not in changeset.author:
        changeset.author = '%s <%s@cvs.gnome.org>' % (changeset.author,
                                                      changeset.author)
    return True

The remap_author function at the bottom maps the CVS user names to something closer to what bzr normally uses.

5. Perform the conversion

Now it is possible to run the conversion:

$ python tailor-0.9.20/tailor -vv --configfile jhbuild.tailor

When the conversion is complete, you should be left with a bzr branch containing the history of the HEAD branch from CVS. Now is a good time to check that the converted bzr looks sane.

6. Use the new branch

Rather than using the converted branch directly, it is a good idea to branch off it and do the development there:

$ bzr branch jhbuild.cvs jhbuild.dev

The advantage of doing this is that you have the option of rsyncing in new changes to the CVS repository and running tailor again to incrementally import them. You can then merge those changes to your development branch.

Revision Control Migration and History Corruption

As most people probably know, the Gnome project is planning a migration to Subversion. In contrast, I’ve decided to move development of jhbuild over to bzr. This decision is a bit easier for me than for other Gnome modules because:

  • No need to coordinate with GDP or GTP, since I maintain the docs and there is no translations.
  • Outside of the moduleset definitions, the large majority of development and commits are done by me.
  • There aren’t really any interesting branches other than the mainline.

I plan to leave the Gnome module set definitions in CVS/Subversion though, since many people help in keeping them up to date, so leaving them there has some value.

I performed a test conversion using Tailor 0.9.20. My first attempt at performing the conversion failed part way through. Looking at what had been imported, it was apparent that the first few changesets created weren’t the first changesets I’d created in CVS. What was weirder still was the dates on those changesets: they were dated 1997, while I hadn’t started jhbuild til 2001.

It turns out that it was caused by clock skew on the CVS server back in September 2003, so the revision dates for a few files are not monotonic. I did the quick fix of directly editing the RCS files (I was working off a local copy of the repo), which allowed the conversion to run through to completion. The problem has been reported as bug #37 in Tailor’s bug tracker.

This made me a bit worried about whether the CVS to Subversion conversion script being used for the rest of the Gnome modules was also vulnerable to this sort of clock skew problem. Sure enough it was, and the first real changeset of jhbuild had been imported as revision 323.

I did a bit more checking of the CVS repository, and found that there were 98 other modules exhibiting clock skew in their revision history, spread over 1245 files (some files with multiple points of skew). I’ve only checked the SVN test conversions of some of these modules, but all the ones I checked exhibited the same type of corruption.

It is going to be a fair bit of work cleaning it all up before the final conversion.

Bugzilla to Malone Migration

The Bugzilla migration on Friday went quite well, so we’ve now got all the old Ubuntu bug reports in Launchpad. Before the migration, we were up to bug #6760. Now that the migration is complete, there are more than 28000 bugs in the system. Here are some quick points to help with the transition:

  • All bugzilla.ubuntu.com accounts were migrated to Launchpad accounts with a few caveats:
    1. If you already had a Launchpad account with your bugzilla email address associated with it, then the existing Launchpad account was used.
    2. No passwords were migrated from Bugzilla, due to differences in the method of storing them. You can set the password on the account at https://launchpad.net/+forgottenpassword.
    3. If you had a Launchpad account but used a different email to the one on your Bugzilla account, then you now have two Launchpad accounts. You can merge the two accounts at https://launchpad.net/people/+requestmerge.
  • If you have a bugzilla.ubuntu.com bug number, you can find the corresponding Launchpad bug number with the following URL:

    http://launchpad.net/malone/bugtrackers/ubuntu-bugzilla/$BUGZILLA_ID

    This will redirect to the Launchpad bug watching that bugzilla bug. This URL can easily be used to make a Firefox keyword bookmark.

  • You can file bugs on Ubuntu at https://launchpad.net/distros/ubuntu/+filebug. Note that the form expects a source package name rather than a binary package name. If you only have a binary package name, you can use the following command to find the source package name:

    apt-cache show $packagename | grep ^Source:

    We’ll make it easier to enter bugs when you only know the binary package name in the future.

  • The Launchpad data model for bugs differs from Bugzilla in that a single bug can be targetted at multiple packages or products (internally, we call these bug tasks). To change information about a bug task (source package name, assignee, status, priority, severity, etc), you must first click on the bug target in the “fix requested in” table at the top of the bug page.

There are still a few issues that need to be ironed out. The mailing lists subscribed to most Ubuntu bugs are not yet properly configured to accept mail from Launchpad, so result in “held for moderation” messages. These issues should get fixed shortly.

gnome-gpg improvement

The gnome-gpg utility makes PGP a bit nicer to use on Gnome with the following features:

  • Present a Gnome password entry dialog for passphrase entry.
  • Allow the user to store the passphrase in the session or permanent keyring, so it can be provided automatically next time.

Unfortunately there are a few usability issues:

  • The anonymous/authenticated user radio buttons are displayed in the password entry dialog, while they aren’t needed.
  • The passphrase is prompted for even if gpg does not require it to complete the operation.
  • If the passphrase is entered incorrectly, the user is not prompted for it again like they would be with plain gpg.
  • If an incorrect passphrase is provided by gnome-keyring-daemon, you need to remove the item using gnome-keyring-manager or use the --force-passphrase command line argument.

I put together a patch to fix these issues by using gpg‘s --status-fd/--command-fd interface. Since this provides status information to gnome-gpg, it means it knows when to prompt for and send the passphrase, and when it gave the wrong passphrase.

I also swiped the zenity_util_show_dialog() function from Zenity to make the password dialog a transient of the terminal that ran it, so the passphrase dialog stays on the same desktop and can’t be obscured by that terminal.

The changes can be found here:

http://www.gnome.org/~jamesh/arch/james@jamesh.id.au/gnome-gpg–devel–0

(a Bazaar 1.x branch, since Colin was using Arch).

There are still a few issues with handling non-password prompts from gpg, but it works quite well for the basics.

Drive Mount Applet (again)

Thomas: that behaviour looks like a bug. Are all of those volumes mountable by the user? The drive mount applet is only meant to show icons for the mount points the user can mount.

Note also that the applet is using the exact same information for the list of drives as Nautilus is. If the applet is confusing, then wouldn’t Nautilus’s “Computer” window also be confusing?

To help debug things, I wrote a little program to dump all the data provided by GnomeVFSVolumeMonitor:

http://www.gnome.org/~jamesh/code/gvfs-list-drives.c

Does the output look sane for you? In particular, are any drives or volumes marked “user visible” that should not be?

Preferences for the Drive Mount Applet

In my previous article, I outlined the thought process behind the redesign of the drive mount applet. Although it ended up without any preferences, I don’t necessarily think that it doesn’t need any preferences.

A number of people commented on the last entry requesting a particular preference: the ability to hide certain drives in the drive list. Some of the options include:

  1. Let the user select which individual drives to display
  2. Let the user select which classes of drive to display (floppy, cdrom, camera, music player, etc).
  3. Select whether to display drives only when they are mounted, or only when they are mountable (this applies to drives which contain removable media).

Of these choices, the first is probably the simplest to understand, so might be the best choice. It could be represented in the UI as a list of the available drives with a checkbox next to each. In order to not hide new drives by default, it would probably be best to maintain a list of drives to hide rather than drives to show.

It does bring up the question of how to identify what a “drive” is. On my Ubuntu system, the first USB mass storage device I plug in usually gets the same mount point. If we identify drives by their mount point, hiding that mount point will effectively hide all of those drives. Perhaps the HAL UDI would be appropriate here.

The third choice is also interesting: why display an icon for a removable media drive if there is no media in the drive? This sort of feature could probably be implemented independently of the previously discussed choice. It is also the sort of change that probably needs to be addressed in gnome-vfs and HAL though. Fixing it at that level would also provide the same benefit to other GnomeVFSVolumeMonitor using apps, such as Nautilus.

Features vs. Preferences

As most people know, there has been some flamewars accusing Gnome developers of removing options for the benefit of “idiot users”. I’ve definitely been responsible for removing preferences from some parts of the desktop in the past. Probably the most dramatic is the drive mount applet, which started off with a preferences dialog with the following options:

  • Mount point: which mount point should the icon watch the state of?
  • Update interval: at what frequency should the mount point be polled to check its status?
  • Icon: what icon should be used to represent this mount point. A selection of various drive type icons were provided for things like CDs, Floppys, Zip disks, etc.
  • Mounted Icon and Unmounted Icon: if “custom” was selected for the above, let the user pick custom image files to display the two states.
  • Eject disk when unmounted: whether to attempt to eject the disk when the unmount command is issued.
  • Use automount-friendly status test: whether to use a status check that wouldn’t cause an automounter to mount the volume in question.

These options (and the applet in general) survived pretty much intact from the Gnome 1.x days. However the rest of Gnome (and the way people use computers in general) had moved forward since then, so it seemed sensible to rethink the preferences provided by the applet:

  1. Nautilus’s volume handling has matured a lot since then, and been pushed down to the platform as the GnomeVFSVolumeMonitor API. This API makes it possible to enumerate mounted volumes and mount points on the system, so we can do a lot better than providing an entry box and file chooser to select a mount point.
  2. The GnomeVFSVolumeMonitor provides asynchronous notification of mount/unmount events, removing the need for the applet to poll the status. If the applet isn’t polling, then there is no reason for it to provide the update interval preference.
  3. The GnomeVFSVolumeMonitor API provides icon names for volumes depending on the drive type. If we can detect that a disk is a floppy or a cdrom or whatever, why ask them what sort of icon to use? This change also means that the icon can be picked from the user’s selected icon theme, providing better integration with the rest of the desktop (not to mention the accessibility benefits when the HighContrast icon theme is used).
  4. Certain types of volumes always make sense to eject on unmount. Other volumes don’t. Since we know the volume type, we should be able to just do the right thing.
  5. Since the applet is no longer directly checking the mount point status, the “Use automout-friendly status test” preference doesn’t make sense. But even if it was applicable, it is the sort of preference that only has one sane value: assuming both types of status check work, why wouldn’t you want to use the one that works with automounters?

The other major change I made was due to a change in the types of volumes people mount: USB devices. If you have a fixed number of mount points/devices you care about, then the old model works pretty well. If you have a large number of devices, and rarely plug them all in at once, you probably don’t want to create drive mount applets for all of them. My solution was to alter the drive mount applet to display a button for each user mountable volume on the system rather than one applet per mount point.

The result was an applet with no preferences. However, I’d contend that it has more features than before. It has been improved further since then, to provide media-type specific options (e.g. start the movie player if you insert a DVD Video disc).