JHBuild

ViewCVS

After the breakin at the gnome.org web server, the CVS server were moved over to the new server HP donated. However, the LXR and Bonsai tools weren’t considered as high a priority, so have not been restored yet.

Since it was easier to set up than either LXR or Bonsai, I set up ViewCVS (with jdub‘s help), so we now have online repository browsing again. It doesn’t provide all the features found in the other packages, so it’ll be good to get them set up again too though.

We made a few superficial changes to the templates to make them look a bit more gnomey, but they probably still need a bit more work.

jhbuild

Merged a patch to add an optional "make clean" phase to jhbuild, and added a "make check" phase (also off by default). These should be useful for the unattended “tinderbox” mode.

It does increase the complexity a bit, so I think I’ll have to look at how to simplify it a bit (probably pull a bit more of the logic out of the program code).

Made some changes to the way “jhbuild bootstrap” works. Whereas previously bootstrap would check to see if each required build tool was installed by the distro and only build the tools that were missing, it now builds all the tools.

If you wish to use the build tools supplied by your distro, it is now recommended that you don’t run bootstrap. To perform the “check that required tools are installed” job that bootstrap used to do, you can instead run the “jhbuild sanitycheck” command, which will do these checks and report any errors. The sanitycheck command also checks for other configuration problems as well, such as whether the all the different automake versions will be able to find the libtool macros.

The upside for me is that there are now only 2 combinations of distro packages vs. packages installed by jhbuild, as opposed to the 2n combinations previously (where n is the number of bootstrap packages). This greatly reduces the number of ways someone can screw up their system.

Since bootstrap is a lot simpler now, I also changed how it was implemented. Rather than using a separate code path, it now uses the same build framework as the main build. The bootstrap command is now functionally equivalent to building meta-bootstrap from the bootstrap module set. Changing the list of bootstrap packages can now be done as easily as modifying any other module set.

Removing the redundant code path should make things a little more robust, since it reduces the amount of infrequently used and/or untested code.

Weather

It has been really hot and humid here for the past few days. While it is not uncommon to have hot weather in Perth, high humidity is quite unusual. It seems to be due to the floods up in the north of the state (they had a report on the news about an 18 person town that had been without a pub for 3 days).

There was a big thunder storm last night, so hopefully things will get back to normal. Unfortunately, it is still quite hot (9:20am at the moment, and its 33°C with 62% relative humidity) and there has been an order preventing people from using air conditioners due to supply problems at the power company.

On another note, if you are using Gnome 2.5.x, the weather applet will now display forecasts for most of the Australian locations, downloaded from BOM. You can also get it to display a radar image by manually entering in the URL in the preferences (the one for Perth is here).

jhbuild

jhbuild now does syntax highlighting of cvs output. In particular, it will display conflicts in red. It also repeats the list of conflicts when the checkout is complete (a useful idea I stole from Mozilla’s client.mk), which is very useful for large modules when you have modified a few files.

To get CVS to output in line buffered mode while passing its output to me, I needed to use a pseudo terminal, at which point I found a few bugs and annoyances in the Python standard library pty.spawn function:

1. It leaks the pty master fd on each run.
2. It doesn’t return the exit status of the child.
3. It doesn’t provide a way to stick the pty connection into non-blocking mode, which is needed if you want to read lines from the child as they are produced.

I reported it as bug 897935, so hopefully it will be fixed in a future version. For now, I have my own private copy of the function.

jhbuild

Had a pretty good response to the jhbuild changes. There was a number of problems I didn’t catch during my testing (more that I would have liked). However, I think I caught the last few ones with pychecker.

I suppose the next thing to do is to help the fd.org guys set things up so they can manage their module sets from their own CVS tree. That will make it easier to recommend as a build tool.

It is also almost at a stage where it could be packaged as a tarball to be used for building Gnome developer snapshots. (Jeff has said that this might be easier than updating garnome).

Creative Commons

I was looking around on the creative commons website, and came accross this page. Essentially, O’Reilly is planning on licensing their current books under an effective 14 or 28 year copyright (pending author approval).

In addition to that, they say they are planning on releasing their out of print books under an attribution license straight away (again, pending author approval). I was looking over the list of books on the page, and the out of print list included the X series. I have a couple of books from the series, and they have been quite useful.

It looks like they started this a while back and haven’t gotten very far, but it would be very useful once the rest of the X series is available.

Checked in a fairly big set of modifications to jhbuild, designed to make it a bit more modular and the code less messy. I had been working on these changes for a while now, and had been keeping track of them on the jhbuild-ng branch.

Here are a few of the main changes:

Code reorganised into a package

The code has been reorganised into a Python package. Unfortunately this means that the old shell script used to start jhbuild won’t work. Rerunning “make install” will fix this though. This will make it easier to extend things in the future.

AMD64 support

If you are running on an AMD64 Linux machine, the libraries jhbuild builds will end up in ${prefix}/lib64, as they should. If you really want your 64-bit libraries to go in ${prefix}/lib still, you can add “use_lib64 = False” in your config file.

The code is set up to try and set use_lib64 correctly for the machine it is running on. If you find a case where use_lib64 is set incorrectly, please file a bug report.

Unattended build with reporting

Based on a suggestion from someone in the audience at Gnome.conf.au, I implemented a new non-interactive mode that redirects all output to a bunch of files in a directory, along with an HTML page giving a summary of how the build went.

An example of the output can be seen here (generated from a run on an Opteron system). This functionality is available through the jhbuild tinderbox command.

It would be nice to get some of the Gnome companies (eg. Sun) to set up some build boxes running something like this on uncommon hardware to help test Gnome builds.

Build environment sanity check

A jhbuild sanitycheck command. This is intended to be used as a way of checking that the build environment is sane. I plan on changing jhbuild bootstrap to be a script intended for installing all the build tools in the build prefix. This way, if you use the build tools provided by your distro, you can simply run sanitycheck to make sure everything is okay and omit bootstrap.

This command should become more useful as I extend it (most likely in response to bug reports).

Load module sets from web servers

Previously, all the build information in jhbuild was maintained with the code in Gnome CVS. This worked pretty well for building Gnome, since people who were able to check code in that would break the build could also fix the build instructions in jhbuild.

Since then jhbuild has become less Gnome specific, and includes rules for building a number of non-Gnome packages including a lot of stuff from freedesktop.org, such as the X server. Having the build rules stored in Gnome CVS isn’t anywhere near as convenient for them, which was the reason for this new feature.

Now it is possible to specify a full URL as the module set in the config file. Jhbuild will then download the module definition file from the web server and use that. There is code to check whether the modules file has changed if the cached copy is too old too (these checks use various HTTP features to keep the bandwidth usage to a minimum). Hopefully this feature will be useful for other large projects looking for a build tool.

I have tested the code a bit, but I wouldn’t be too surprised if one or two bugs have been introduced. If you find a regression, make sure you report it in bugzilla.

It should now be a lot easier to add new features, which is a good thing.