16.05.2008 Becoming a Gtk+ maintainer

Amongst many other things during the Gtk+ Hackfest 2008, it was brought to my attention that Gtk+ maintainership is sometimes perceived as some kind of leet circle, hard to join for newcomers. I can’t really imagine how “hard” it is for newcomers to join Gtk+ maintenance these days. The learning curve probably is steeper now than it was in the last decade, but then, we do have documentation now and didn’t have any back then… ;-)

In any case, I think that definitely none of the Gtk+/GLib core maintainers would want to bar someone else from contributions or helping out with maintenance tasks. So to aid that process, I’ve written down what I keep telling people who approach me about this in person. A lot of it might seem overly obvious to veterans, but for newcomers or contributors looking into extending their activities on the project I hope to provide helpful starting points.

Much of Gtk+ is still maintained as a huge monolithic whole. That is, a very few people are taking care of a wide variety of components. I think ultimately we would all be much better off, if we had more people being responsible for individual components like particular widgets (e.g. GtkNotebook) or widget families (e.g. GtkBox, GtkHBox, GtkVBox). So the following are steps and tasks useful for anyone wanting to get into Gtk+ component maintenance.

First of all, we have the GtkTasks wiki page, a task list with various ways in which people can contribute to the Gtk+ project and start getting involved. In particular for the following positions, no one has signed up so far to volunteer on a regular basis:

  • Patch/Bug Monitoring & Filing - collect bugs from mailing lists and IRC to make sure they are being filed.
  • FAQ Maintainer - this involves monitoring the mailing list(s), taking CC:-mails from other people on possible FAQ material and regularly updating the FAQ accordingly.
  • Build Monitoring - run and maintain Gtk+ tool-chain builds on various platforms.
  • Making Releases - we are looking for someone assisting in the release process and to take over release building during some periods in the future.
  • Implementing Test Programs - there’s much work needed in our unit test coverage, so we’re always looking for people willing to implement more unit tests.

For general information about the maintenance situation, the State of the Gtk+ Maintenance write-up is still fairly valid. However many good things have been suggested by the community since, such as the GtkTasks page and the Hackfest. At the end, the write-up addresses how occasional contributors or developers at different experience levels can help out the project. For instance with activities such as: Bug triage and verification, Review and clarify documentation, Revision hunting for regressions, Refactor code areas, Work on optimizations, and the list goes on.

If none of this sounds interesting to potential new maintainers, be warned that regular maintenance means having to put up with pretty much all of these at some point. ;-)
However, probably the most straight forward way to take on maintenance for a particular code portion (usually a particular widget) is to start becoming familiar with it and work on improving it right away:

  • Cleanup indentation - lots of contributions to Gtk+ in the past have weakened coding style in various areas. In general we use GNU coding style plus a few extra rules similar to the Gimp coding style.
  • Perform code cleanups - look for outstanding migrations/refactorings in the code or if the implementation could be simplified/streamlined in areas.
  • Check documentation and examples - contributing by improving the existing documentation, testing existing examples and providing new ones is a very straight forward way to learn about a new component. Also, documentation patches are usually easily and quickly approved.
  • Provide unit tests - writing new unit tests for existing component APIs is even better than providing documentation examples. You get immediate feedback and they should in general be easy to approve to go into upstream. Also, it is definitely an area where Gtk+ and GLib still need lots of work.
  • Review bug reports and patches - go through the Gtk+ bug load of a particular component, see what issues could be closed or need info. Find patches that could be applied or need work and provide/fix patches along the way. Also, feel free to provide review for existing patches where you feel confident to provide reasonable input. For existing maintainers, looking at other people’s review is one of the best ways to figure if another person is up to the task of taking over component maintenance.
  • Actively nag existing maintainers or people with SVN accounts for trivial patches (like Cody and Mathias who signed up for Patch testing & committing) to review, approve and apply your changes.
  • Participate in the project forums like gtk-devel-list and #gtk+ (needed to nag people from the core team and other developers), and read and reply in other related mailing lists.

The first few points actually mean working with the code by providing patches, and filing new bug reports with those patches attached. While this may at first increase our bug load, if someone shows sincere interest in taking over component maintenance, sticks to the coding style and provides useful patches, there’s nothing stopping us from granting new SVN accounts so contributors can commit their own patches after approval.

Finally, the project is welcoming every new contributor. But be reminded that no approval is needed from anyone to start your work. In actuality, asking for it will most probably get you a reserved or negative answer, because improving the project is all about working on it, not making or requesting promises. So, everyone is invited to read through the task lists/descriptions and get their hands dirty immediately. A fair bit of self incentive is needed to take on maintenance of a component anyway, so you’ll have to get yourself motivated on your own there won’t be anyone else doing it for you.

24.04.2008 Announcing Rapicorn 8.4.0

Rapicorn 8.4.0 has just been released to the world: Rapicorn v8.4.0 Announcement

Lots of things have happened since the last snapshot over a year ago, some of which kept me from making releases or snapshots earlier. ;-)
Others actually took place in the code base as interesting developments, summarized below.

There’s quite some more stuff in my development queue, but at some point one just has to draw a line and throw out what’s vaguely ready so far, so this is it for today:

  Overview of changes in Rapicorn 8.4.0:

  * Changed versioning scheme to YEAR.WEEK.REVISION.
  * License update to GNU LGPL 2.1.
  * Added a publically installed tool: rapidrun
  * Support println() and close() commands in GUI files.
  * Introduce simple Application and Window object APIs.
  * Merged libbirnet into Rapicorn as librapicorncore.
  * Implemented expose region merging/comprssion.
  * Reiimplemented rectangle gradient shader.
  * Switched to autogenerated ChangeLogs.
  * Improved feedback on parser errors.
  * Fixed Gtk+ version checks.
  * Added PNG saving support.
  * Removed PERL build dependency.
  * Rewrote asyncronous main loops.
  * Many improvements to text editing areas.
  * Speed up blitting logic for local displays.
  * Added SIMD optimized rendering functions for MMX CPUs.
  * Fixed some reference counting issues and child removal.
  * Improved vertical text ellipsization to line granularity.
  * Removed error prone default values from property mechanism.
  * Install tutorial under ${prefix}/doc/rapicornXXXX/tutorial/.
  * Misc compiler and threading fixes, depend on g++-3.4.6.
  * Lots of bug fixes, cleanups and improved test coverage.

07.04.2008 On Moving Gtk+ to Git

There have been several requests about hosting Gtk+ (and GLib) as a Git repository recently and since that topic has come up more and more often, I meant to write about this for quite some time.

Let’s first take a look at the actual motivation for such a move. There are a good number of advantages we would get out of using Git as a source code repository for Gtk+ and GLib:

  • We can work offline with the Gtk+ history and source code.
  • We can work much faster on Gtk+ even when online with tools such as git-log and git-blame.
  • It will be much easier for people to branch off and do development on their own local branches for a while, exchange their code easily with pulling branches between private repositories, etc. I.e. get all the advantages of a truly distributed versioning system.
  • With Git it’s much easier to carry along big Gtk+ changes including history by using cherry picking and (interactive) rebasing.
  • We can make proper public backups of the source code repositories. This ought to be possible already via svnsync, but isn’t for svn.gnome.org because we run an svn 1.3.x server instead of an svn 1.4.x server that is required by svnsync. (Yes, this issue has been raised with the sysadmins already.)

A quick poll on IRC has shown that all affected core maintainers are pretty much favoring Git over SVN as a source code repository for GLib/Gtk+.

However, here are the points to be considered for not moving the repositories to Git (just yet):

  • Complexity; Git is often perceived to be harder to use than SVN, there are several attempts to mitigate this problem though: Easy Git Giggle yyhelp.
    With some of the above, Git is as easy to use as SVN is, so Git complexity doesn’t need to be holding anyone off at this point.
  • Git may be stable and mature these days, but git-svn is not there yet. It is generally good enough to create local Git mirrors of SVN repositories and work from those to have most of the Git convenience on top of an SVN project.
    But git-svn has seen structural changes recently, quite some rewriting and bug fixing that indicate it’s still too much in flux for the one-and-only SVN->Git migration for projects at the scale of Gtk+. This is not meant as criticism on git-svn, fairly the opposite actually. It’s good to see such an important component be alive and vivid. All issues I’ve raised with the maintainer so far have been addressed, but it seems advisable to wait for some stabilization before trusting all the Gtk+ history to it.
  • Gitweb interfaces already exist for GLib/Gtk+ mirrors, for example on testbit: Testbit Git Browser.
    These can be used for cloning which is much faster than a full git-svn import. Alternatively, shallow git-svn imports can be created like this:
      git-svn clone -T trunk -b branches -t tags -r 19001 svn://svn.gnome.org/svn/gtk+
    

    This will create a repository with a mere ~1000 revisions, including all changes since we branched for 2.12. We’re using such a shallow repository for faster cloning of our GSEAL() development at Imendio: view gtk+.git.

  • In summer 2006, we’ve had the first test migration of all of GNOME CVS to SVN, in December 2006 we’ve had the final migration. During that period, the Beast project stayed migrated to SVN to work out the quirks from the CVS->SVN migration before we migrate all other GNOME modules and have to fix up everyones converted modules. There were quite some issues that needed fixing after the initial test migration and in the end we had to rebuild the Beast development history from pieces. Preventing the other GNOME modules from such hassle was the entire point in migrating Beast early on, so I’m not complaining.
    However, given the size and importance of GLib/Gtk+, the development history of those projects shouldn’t be put at a similar risk. That is, GLib/Gtk+ shouldn’t be pioneering our next source repository migration, let some other small project do this and work out the quirks first.
  • git-svn already provides a good part of the Git advantages to developers while Gtk+ stays hosted in SVN. Albeit due to mismatching hashes, syncing branches between distinct git-svn checkouts of different people is tedious. Setting up an “official” git-svn mirror on git.gtk.org could probably help here. Also, to ease integration, jhbuild could be extended to use git-svn instead of svn commandos to update SVN modules, if the current module has a .git/ subdirectory instead of a .svn/ subdirectory.

The bottom line is, there’s a good number of advantages that Git already can (or could) provide for our development even without migrating the repositories to Git right away. When exactly will be a good point for migrating GLib/Gtk+ and possibly other GNOME modules might not be an easy call, but right now is definitely too early.

05.02.2008 Thread-safe class initializers

I finally got around to fix a long-standing and tricky bug report: Bug 64764 - Class initialization isn’t thread safe.
Thread safety problems in class initializers and _get_type() functions caused nasty problems in other components that depend on parallel type creation, in particular GStreamer (Dependency Graph for bug 64764). With both being fixed now, testing feedback about GType/GObject threading problems using GLib trunk is appreciated.

30.10.2007 YummyYummySourceControl Version 0.9

A couple people have reported minor and major bugs in the last yyhelp version, particularly after yycommit got reimplemented to operate on top of git-commit(1) instead of cg-commit(1). Besides some others, this new release fixes all known yycommit issues and also (re-)introduces some new features: yyhelp (v0.9)

	Overview of Changes in YummyYummySourceControl-0.9:

	* use plain "git-commit" with temporary index file to stage
	  commit files, this works around git-commit-1.5.2.5 not
          handling deleted files as command line args correctly.
	* also list remote branches for yylsbranches.
	* fix leading dot getting stripped from modified files if $gitprefix=.
	* require and use gawk for time formatting, which mawk doesn't support.
	* properly honor the [FILES...] arguments to yycommit.
	* terminate sed command blocks with semicolon (needed on BSD).
	* resurrected yyhelp.auto-push-commits functionality of yycommit.

13.10.2007 Yummy-Yummy Porcelain Version 0.8

Here is a new release of YummyYummySourceControl, a shallow porcelain script around common git(7) command variants: yyhelp (v0.8)

This version supports a new command to grep and match an extended regular expression on the full project history by invoking git-grep(1) on all existing commits, displaying matches by commit hash and file name. Also, a CVS/SVN/Cogito style version of yycommit has now been implemented on top of git-commit(1). So YummyYummySourceControl-0.8 finally gets rid of the last remaining cogito dependency, making it a free standing Git Porcelain Suite. ;-)
Of course there have also been some other miscellaneous changes and docu updates.

	Overview of Changes in YummyYummySourceControl-0.8:

	* removed cogito(7) dependency.
	* made yyview start gitk in the background.
	* suppress yydiff outputs of non-checked-out files (ignored by yycommit).
	* implemented yycommit SVN/CVS/CG-alike on top of git-commit.
	* added '-s' option to yyChangeLog to skip SVN revisions.
	* implemented yyHistoryGrep.

I have been asked some of the purpose of YummyYummySourceControl, so i will extend a bit on that here:

- On some occasions i had to dig quite deeply into the nitty-gritty details of git (and cogito). However i can simply not be bothered to deal with the complexity of its command set for everyday use. I need my source control interface to be really simple when i concentrate on the various projects i’m involved in.

- I refuse to bother with the state (or existence) of git’s index file for anything but the correct implementation of yyhelp. SVN and CVS don’t force me to deal with an intermediate cache state between repository and working tree, YummyYummySourceControl keeps me from seeing it in git.

- I might be using yydiff and yyview up to a dozen or so times per hour while working. Passing options along here or piping the output to a pager or starting stuff in the background really becomes unaffordable at that rate. These commands do all the necessary stuff out of the box, all i have to type is “yyd”+Tab+Return and “yyv”+Tab+Return for each of them now.

- I’m too old and stupid to remember to pull after i’ve pushed and what the git-svn(1) command variants look like. yypull and yypushpull handle that for me.

- Personally, i find ChangeLog style logs much more parsable than git-log(1) output. yyChangeLog gives me that for the git commit history.

- To allow proper git merging, cherry picking, rebasing, etc, i’ve switched to commit my ChangeLog entries as commit message only (with introductionary one-liner instead of the date + email header) and for Rapicorn auto-generate the project ChangeLog for tarballs. Even if the ChangeLog is not auto generated as with Beast or Gtk+, i can still use this commit message scheme for git-svn checkouts, as long as i update the project’s SVN ChangeLog before yypushpull-ing the changes from my local tree to SVN. yyChangeLog -s will generate the remaining ChangeLog portion missing from SVN.

- To help my utter laziness, yystatus -c can preformat file name change lists for my ChangeLog-style commit messages, i’ve always missed that with CVS and SVN.

- To help against sudden confusion and lengthy URL searches, yyinfo tells me all i want to know about a repository (git URL + revision, SVN URL + revision, etc).

And i love to have grep-able access to all the data involved in my day to day activities, yyHistoryGrep now gives me that for my project histories.

Given git’s speed, the possibility for light-weight local repositories, and using yyhelp’s ease, revision control has become so available for me, that i could very comfortably apply it to portions of my home directory and /etc/ on a remote server (e.g. i’ve created a ~/.git/ a couple months ago where i checked in a handful selected files like TODO, my blog diary, .emacs and a few other frequently changing TODO-style files and i’m happily committing to it multiple times a day).

I’m not claiming that yyhelp solves anyone else’s problems, but it certainly has helped my SCM usage a great deal, and for me serves as a key to make a frequently used subset of the git machinery accessible at very few finger tips.

21.09.2007 Beasty Bits (final spurt)

We’ve been fairly busy recently with resolving the milestone bugs of the next Beast release. The good news is that pretty much all of the hard issues are sorted out by now, the bad news is that according to the release plan some essential release features are still missing ;)

It’s the large recent work that Stefan Westerfeld (who btw finally decided to start his own blog) has put into shaping up the remaining bits of bsewavetool (man page draft).
This is a very handy tool, that’ll finally be installed for public consumption in the upcoming release (it has been developed in SVN since 2004!). It can clip/normalize/ogg-encode/highpass/lowpass/upsample/downsample/etc chunks from the BSE multi sample files, and is normally used for shell-scripting the construction of sample kits from an unsorted pile of raw unprocessed sample data (note to self: blog some example use cases after the release).

Another addition are interesting new instruments by Krzysztof Foltman.

Oh - and before i forget, the synthesis link section on the website got some new updates as well: Beast Synthesis Links,

Also, Stefan pointed me at a YouTube video of Beast the other day:

The video is very nicely done, but it has the usual YouTubeish low quality artefacts.

19.09.2007 Yummy-Yummy git-svn

I’ve had to add another bit to yyhelp to make my everyday coding more convenient, so it got support for git-svn(1). The yypull and yypushpull commands will now detect a git(7) repository set up with git-svn, and unless the repository also has a regular git upstream, pulling and pushing will be carried out via git-svn rebase and git-svn dcommit. yyinfo shows the update method being used for a repository and also displays the SVN HEAD revision now if any. That, plus miscellaneous changes make version 0.6 of yyhelp:

	Overview of Changes in YummyYummySourceControl-0.6:

	* for git-svn repositories, use rebase and dcommit to pull and push
	* fixed argument parsing for yybranch and yybranchdel
	* support revision argument for yydiff
	* check revision argument to yywarp

08.09.2007 Yummy-Yummy Buglets

The YummyYummySourceControl script got a few buglet fixups during the last days, so I’ve put up version 0.5b: yyhelp.
Here’s the obligatory list of changes since the last version:

	Overview of Changes in YummyYummySourceControl-0.5b:

	* update stat info for yycommit to ignore touched but unmodified files
	* use full paths for yyrestore to fix spurious file resurrections in parent dirs
	* fixed bogus error messages on yycommit

29.08.2007 Yummy-Yummy Again

I’ve put up a new release of the slightly renamed YummyYummySourceControl script here: yyhelp (0.5).
The script still depends on cogito(7) for the implementation of yycommit, but other such dependencies have been removed at this point so it’ll eventually depend on Git(7) only. Following is a sum up of the changes between the last version and v0.5:

	Overview of Changes in YummyYummySourceControl-0.5:

	* Help improvements.
	* Support auto-push after commits.
	* Improved installation support: --install-aliases & --uninstall-aliases.
	* Automagically support less as pager.
	* Buglet fixes in yydiff.
	* Provided native implementation of yystatus with extra options.
	* New command yyinfo to list repository information.
	* New command yyblame with colorization support.
	* Allow tagging of specific revisions.

Here’s the synopsis of the new command set:

	SYNOPSIS
	    yyadd          [FILES…] - add files to git repository
	    yyblame        [FILES…] - annotate file source code
	    yybranch    [-f] <branch> - add branch (forcable)
	    yybranchdel [-f] <branch> - delete branch (forcable)
	    yyChangeLog               - show git log in ChangeLog style
	    yycommit       [FILES…] - commit current working tree
	    yydiff         [FILES…] - show committable differences in
	                                working tree
	    yygc                      - repack and prune repository
	    yyhelp                    - display yy* help information
	    yyinfo                    - display repository information
	    yylsbranches              - list branches
	    yylstags                  - list tags
	    yypull                    - pull upstream sources
	    yypushpull                - push & pull upstream sources
	    yyremove       [FILES…] - remove files from git repository
	    yyreset                   - reset (revert to HEAD) all files in the tree
	    yyrestore      [FILES…] - forcefully re-checkout specific files
	    yystatus       [-t|-u|-c] - display working tree status
	                                -t: trim unknown; -u: show unknown;
	                                -c: trim and use commit message style
	    yytag    <tag> [revision] - add tag
	    yytagdel <tag>            - delete tag
	    yyuncommit                - undo the last commit (must be unpushed)
	    yyview                    - browse & navigate the history
	    yywarp           <branch> - checkout new branch

Bad Behavior has blocked 155 access attempts in the last 7 days.