Re: Lazy loading

  • Post author:
  • Post category:Uncategorized

Emmanuel: if you are using a language like Python, you can let the language keep track of your state machine for something like that:

def load_items(treeview, liststore, items):
    for obj in items:
        liststore.append((obj.get_foo(),
                          obj.get_bar(),
                          obj.get_baz()))
        yield True
    treeview.set_model(liststore)
    yield False

def lazy_load_items(treeview, liststore, items):
    gobject.idle_add(load_items(treeview, liststore, item).next)

Here, load_items() is a generator that will iterate over a sequence like [True, True, ..., True, False]. The next() method is used to get the next value from the iterator. When used as an idle function with this particular generator, it results in one item being added to the list store per idle call til we get to the end of the generator body where the “yield False” statement results in the idle function being removed.

For a lot of algorithms, this removes the need to design and debug a state machine equivalent. Of course, it is possible to do similar things in C but that’s even more obscure 🙂.

Gnome Logo on Slashdot

  • Post author:
  • Post category:Uncategorized

Recently, Jeff brought up the issue of the use of the old Gnome logo on Slashdot. The reasoning being that since we decided to switch to the new logo as our mark back in 2002, it would be nice if they used that mark to represent stories about us.

Unfortunately this request was shot down by Rob Malda, because the logo is “either ugly or B&W (read:Dull)”.

Not to be discouraged, I had a go at revamping the logo to meet Slashdot’s high standards. After all, if they were going to switch to the new logo, they would have done so when we first asked. The result is below:

This is based on an earlier design, but I think the drop shadow really completes the image. Iain managed to come up with a variant suitable for use on the games sub-site.

Gnome-gpg 0.4.0 Released

  • Post author:
  • Post category:Uncategorized

I put out a new release of gnome-gpg containing the fixes I mentioned previously.

The internal changes are fairly extensive, but the user interface remains pretty much the same. The main differences are:

  • If you enter an incorrect passphrase, the password prompt will be displayed again, the same as when gpg is invoked normally.
  • If an incorrect passphrase is stored in the keyring (e.g. if you changed your key’s passphrase), the passphrase prompt will be displayed. Previously you would need to use the --forget-passphrase option to tell gnome-gpg to ignore the passphrase in the keyring.
  • The passphrase dialog is now set as a transient for the terminal that spawned it, using the same algorithm as zenity. This means that the passphrase dialog pops up on the same workspace as the terminal, and can’t be obscured by the terminal.

Using Tailor to Convert a Gnome CVS Module

In my previous post, I mentioned using Tailor to import jhbuild into a Bazaar-NG branch. In case anyone else is interested in doing the same, here are the steps I used:

1. Install the tools

First create a working directory to perform the import, and set up tailor. I currently use the nightly snapshots of bzr, which did not work with Tailor, so I also grabbed bzr-0.7:

$ wget http://darcs.arstecnica.it/tailor-0.9.20.tar.gz
$ wget http://www.bazaar-ng.org/pkg/bzr-0.7.tar.gz
$ tar xzf tailor-0.9.20.tar.gz
$ tar xzf bzr-0.7.tar.gz
$ ln -s ../bzr-0.7/bzrlib tailor-0.9.20/bzrlib

2. Prepare a local CVS Repository to import from

The import will run a lot faster with a local CVS repository. If you have a shell account on window.gnome.org, this is trivial to set up:

$ mkdir cvsroot
$ cvs -d `pwd`/cvsroot init
$ rsync -azP window.gnome.org:/cvs/gnome/jhbuild/ cvsroot/jhbuild/

3. Check for history inconsistency

As I discovered, Tailor will bomb if time goes backwards at some point in your CVS history, and will probably bomb out part way through. The quick fix for this is to directly edit the RCS ,v files to correct the dates. Since you are working with a copy of the repository, there isn’t any danger of screwing things up.

I wrote a small program to check an RCS file for such discontinuities:

http://www.gnome.org/~jamesh/code/backward-time.py

When editing the dates in the RCS files, make sure that you change the dates in the different files in a consistent way. You want to make sure that revisions in different files that are part of the same changeset still have the same date after the edits.

4. Create a Tailor config file

Here is the Tailor config file I used to import jhbuild:

#!
"""
[DEFAULT]
verbose = True
projects = jhbuild
encoding = utf-8

[jhbuild]
target = bzr:target
start-revision = INITIAL
root-directory = basedir/jhbuild.cvs
state-file = tailor.state
source = cvs:source
subdir = .
before-commit = remap_author
patch-name-format =

[bzr:target]
encoding = utf-8

[cvs:source]
module = jhbuild
repository = basedir/cvsroot
encoding = utf-8
"""

def remap_author(context, changeset):
    if '@' not in changeset.author:
        changeset.author = '%s <%s@cvs.gnome.org>' % (changeset.author,
                                                      changeset.author)
    return True

The remap_author function at the bottom maps the CVS user names to something closer to what bzr normally uses.

5. Perform the conversion

Now it is possible to run the conversion:

$ python tailor-0.9.20/tailor -vv --configfile jhbuild.tailor

When the conversion is complete, you should be left with a bzr branch containing the history of the HEAD branch from CVS. Now is a good time to check that the converted bzr looks sane.

6. Use the new branch

Rather than using the converted branch directly, it is a good idea to branch off it and do the development there:

$ bzr branch jhbuild.cvs jhbuild.dev

The advantage of doing this is that you have the option of rsyncing in new changes to the CVS repository and running tailor again to incrementally import them. You can then merge those changes to your development branch.

Revision Control Migration and History Corruption

As most people probably know, the Gnome project is planning a migration to Subversion. In contrast, I’ve decided to move development of jhbuild over to bzr. This decision is a bit easier for me than for other Gnome modules because:

  • No need to coordinate with GDP or GTP, since I maintain the docs and there is no translations.
  • Outside of the moduleset definitions, the large majority of development and commits are done by me.
  • There aren’t really any interesting branches other than the mainline.

I plan to leave the Gnome module set definitions in CVS/Subversion though, since many people help in keeping them up to date, so leaving them there has some value.

I performed a test conversion using Tailor 0.9.20. My first attempt at performing the conversion failed part way through. Looking at what had been imported, it was apparent that the first few changesets created weren’t the first changesets I’d created in CVS. What was weirder still was the dates on those changesets: they were dated 1997, while I hadn’t started jhbuild til 2001.

It turns out that it was caused by clock skew on the CVS server back in September 2003, so the revision dates for a few files are not monotonic. I did the quick fix of directly editing the RCS files (I was working off a local copy of the repo), which allowed the conversion to run through to completion. The problem has been reported as bug #37 in Tailor’s bug tracker.

This made me a bit worried about whether the CVS to Subversion conversion script being used for the rest of the Gnome modules was also vulnerable to this sort of clock skew problem. Sure enough it was, and the first real changeset of jhbuild had been imported as revision 323.

I did a bit more checking of the CVS repository, and found that there were 98 other modules exhibiting clock skew in their revision history, spread over 1245 files (some files with multiple points of skew). I’ve only checked the SVN test conversions of some of these modules, but all the ones I checked exhibited the same type of corruption.

It is going to be a fair bit of work cleaning it all up before the final conversion.