Guadec has been in interesting conference, particularly because it took place in Istanbul this year. I tried to keep a few notes throughout the days to wrap up the experience and discussions here.

Sunday

Headed off for Istanbul, partial Imendio meet-up at the airport in Vienna, gathered remaining Imendians at the airport in Istanbul. Like many others, we stayed at the Golden Long Hotel near the seaside.

Monday

Kris and I met up with the Gnome release team where we summarized the Gtk+-3.0 ideas that have been cooked up during and after the Berlin Gtk+ Hackfest.

Tuesday

I was planning to attend the Maemo BOF in the morning, but shortly after arriving at the conference venue, Federico literally dragged me into the DVCS BOF. Still, I only managed to attend the second half of it which was almost exclusively about Bazaar features. People told me the first hour had been quite the contrary and focused mostly on Git hyping. Clearly, there was no consensus after the meeting on what the future versioning system in Gnome will be.
I’m not surprised that is the case. As things currently stand, SVN has very active development and user communities, Git is very actively developed, Bazaar is as well, as are a couple other VCSes. Active developer and user communities are generally a good sign for a healthy project and also an indicator for future relevance. Thus, in any larger community such as the Gnome community, it’s easy to find lots of critics and lots of supporters for each of the bigger versioning systems and that’s unlikely to change much. Consequently, there’ll not be an easy consensus on switching to a single versioning system any time soon, so I think the most productive approach for Gnome to take is to prepare the hosting of multiple VCSes, certainly SVN, Git and Bazaar. Needless to say that cross-VCS integration will also become increasingly important in the future, so focus on maturing and extending git-svn, bzr-svn, bzr-git and the like makes a lot of sense.

Later that day, we had a Gtk+ Developers meeting in the medium sized presentation room. The place was a bit too large to have the planned face to face discussions so we’ve had to sacrifice some of the spontaneity to microphone resource sharing. Kris took minutes during the meeting and will probably post those to gtk-devel-list once he’s found a minute to process them. The meeting was quite productive nevertheless, we discussed the upcoming Gtk+ 2.14, features and schedule for 2.16 and 3.0 and a bit of the post-3.0 road map.
Right after that discussion, Kris and I attended the advisory board meeting where we briefly wrapped up the developers meeting, the Gtk+ Hackfest in Berlin and the improvements the Gtk+ project has seen since the State of the Gtk+ Maintenance email. In particular, we stressed that we now have the GtkTasks and Gtk+ 3.0 Tasks wiki pages which can serve as an entry point for contributors and assistants to the project at various experience levels, in particular for companies that want to sponsor developer resources. Also for people that have an interest in long term Gtk+ project involvement, feel free to read up on how to become a Gtk+ maintainer.

Pizza looks weird in Istanbul BTW:

Wednesday

Almost by accident (I was mostly looking for an air conditioned hall in the afternoon), I happened to be watching “Gnome Documentation: A year in review” by Don Scorgie where he described the new user documentation tool Mallard. For some time now, I’ve been working on a Wiki syntax parser for Doxer to unify the markup I have to use for my blog, inline documentation and CMS content markup at testbit.eu. (I have a blog entry in the queue on this for another day.) In this context, implementing a Doxer markup backend that generates Mallard’s XML input could be an attractive markup alternative for future Gnome documentation - it at least ended up on my ever growing TODO list. ;)

Lots of people approached me throughout this and the following days for a chat about Gtk+-3.0 and what comes after that. With the merging of the GSEAL branch into upstream trunk recently, there’s been a lot of focus on the technical preparative work we’re doing for the actual 3.0 release which is planned as an ABI break to Gtk+-2.16 without adding any new features (it’s basically just a re-release with all deprecated code removed and current “private” API really made private by moving it to non-installed source files).

What has been lacking emphasis in this course is that 3.0 is going to be the necessary enabler, needed to work on implementing future visions of Gtk+ and to refactor the code base back to a healthy state where it becomes maintainable again. Quite expectedly, the need for the sealing and accompanied ABI break has been questioned several times, so I’ll reiterate the reasoning here:

• Everybody falls the first time.
  Code development in open source projects is very evolutionary, especially for projects that don’t clone or reimplement existing specified APIs like Libc. A whole chapter is spent on prototypes in The Mythical Man-Month:
  “Plan to throw one away; you will, anyhow.”
  So newly added components and APIs are almost certain to need fixups or revamps in future iterations (likely more than once). If critical internals are being exposed and eternal ABI stability has been promised, this however becomes impossible. Given the development history of Gtk+ and the variety of interests in this project, it is vital for its future success to prepare for future changes and allow iterative improvements. After all, progressive improvements, appreciation of contributions and adaptions to changing circumstances is where the free software development model shows its strength. The waterfall design model is not it.
• Gtk+-2.x is essentially a dead end.
  Everybody agrees that Gtk+-2.x is pretty much dead in a few revisions because of the huge work involved in its maintenance and no relief in sight with its current ABI maintenance policy. This is at least true for all current and past core team members (i.e. everyone who actively tried maintaining 2.x over a significant period). The question is whether we move to an entirely new toolkit (Clutter, Rapicorn, Qt, HippoCanvas, etc) or whether that “new” toolkit is Gtk+-3.0 which may be largely API compatible with Gtk+-2.x. In either case, applications and libraries will need to migrate to a new toolkit base with a different ABI, the main difference is the involved porting effort.
• GLib and Gtk+ do have a means to deal with API changes:
  1) We provide new alternative interfaces (functions).
  2) We deprecate old interfaces (functions) or provide compatibility code in old interfaces.
  Notably, this does only work for API that is exported via function symbols. Structure fields that are directly accessed from application code can’t be deprecated and removed without breaking ABI, and there is no compatibility code upstream could provide for these kind of accesses either. That’s why we want to move away from exposing any structure internals in 3.0 and beyond.
• 3.0 will ABI-incompatibly remove all deprecated and private APIs.
  Of course, the above described deprecation scheme only scales well if deprecated APIs are really removed from the code base at some point. Technically, this is an ABI break which is why GLib/Gtk+ have not been doing this since 2.0. However, lots of other vendors do this to keep a healthy code base, e.g. Qt does break ABI between major releases, Python 3.0 will be incompatible with 2.5, Apple does remove long deprecated APIs in newer releases of Mac OS X, Symbian broke API and ABI in 9.x, Microsoft broke behavior from .NET 1.1 to 2.0, and the list goes on…
  By exposing only function symbols as future public interfaces, we’ll be able to provide arbitrary compatibility functionality for old interfaces on top of new components, add helpful runtime warnings for iterative migration and constrain future ABI breaks to removal of properly deprecated interfaces.
• User visible gains are post-3.0 features:
  Since GLib and Gtk+ are largely volunteer contribution based projects, it’s close to impossible to plan exact arrival of future features. However the following is a list of things that have (partially) been discussed as post-3.0 work during the Gtk+ Hackfest already:
  • Full support of alpha transparency for all widgets;
  • Support for (partial) stacking of widgets (needs transparency);
  • Offering easier layouting facilities;
  • Support for animated visible transitions between widget states;
  • Providing new UI metaphors based on simulation of physical effects like acceleration, 3D browsing of image collections, 3D skimming through notebook pages, and more;
  • Using IDL based type data generators and code generators to improve the way widgets are implemented;
  • Implementing a new theming system for the toolkit;
  • Moving towards exposing widget features only via interfaces that have their own handle (asymmetric query_interface).

This is how GSEAL, the Gtk+-3.0 release and a couple remaining outstanding tasks are going to enable development of exciting future user visible features. The next step for Gtk+ to get work in visionary areas off the ground is to start consideration of feature feasibility and implementations, required resources and tentative schedules.

At the end of the day, we had the Opening Cocktails Party, during which I managed to catch Hallski tattooing J5:

Thursday

Thursdays most interesting event was of course the keynote by Kris which got hijacked by the Gnome release team for the announcement of Gnome 3.0 which is essentially Gnome 2.30 cleaned up and based on Gtk+-3.x. Kris’ slides are available online:

	http://people.imendio.com/kris/gtk-state-of-the-union-2008.pdf

The slides provide a good overview of what Gtk+-2.14 will bring, prospects for 2.16 and visions/requests from the community for Gtk+’s future. As previously described, Gtk+-3.0 is about enabling refactorings and development of new features, and the plan is to do our best to make the transition away from old deprecated code as easy as possible. Other than properly porting an existing Gtk+-2.x application to work with the G_DISABLE_DEPRECATED, GTK_DISABLE_DEPRECATED, GSEAL_ENABLE switches, no additional changes will be required to build and run an application on Gtk+-3.0.

At the end of the day, there was the boat trip through the Bosporous which provided a beautiful sight along the coast line.

Friday

I managed to attend the latter half of the lightning talks which was as always quite interesting. I should probably ignore my laziness and actually prepare short lightning talks for next year about Rapicorn and possibly Doxer… ;)

I took particular interest in Transifex, an online translation platform that can work together with multiple VCSes and that we’d ideally move all Gnome translations to in the future. There are two things I’d like to see fixed in a future translation workflow from a developer perspective:

• The .po templates should really be generated by the developers of the upstream project by automatic means, e.g.:
  make update-po -C po/
  So the upstream version of intltool and po/Makefile.* are used instead of possibly broken or outdated intltool/gettext versions on the translators system.
• Developers should be able to determine merge points for translations, and also review related non-po file changes, rather than having translators wildly commit into upstream repositories (which may conflict with other VCS workflows like branch merges or commits around release phases).

Later on, Federico presented his ideas for timeline tabs for the desktop. This should make it rather easy to find documents or URLs from previous days or weeks, because out of natural necessity, humans generally have good chronological associations. So the new and nice part about this approach is that it can provide good visual access to the chronologic dimension, something a file system doesn’t usually reveal easily, and that’s not easily made accessible by the most prominent desktop metaphors either.

I feel very tempted to start an implementation of the desktop tabs with Rapicorn, however with a few refinements of Federico’s proposal:

• The view should provide a “chronological zoom” slider to switch the view between years/months/weeks/days/hours.
• To be most useful, we’ll need a crawler that tries to (re-)construct past file modification history without relying on programs pushing journal entries about file edits. This will be needed anyway unless every program on this planet provides file editing journal information.
• I think the journaling hooks need to be implemented via DBus and not rely on Nautilus, so they’re usable by all cross-desktop applications and non-GUI programs.
• Various filters by file extensions, magic and possibly more will also be needed in the tab view (this was partly raised during the discussion phase at the end of the presentation).

Saturday

About half of the Imendians headed home on Saturday, we had some early leaves on Friday already and left some others in Istanbul for additional vacations.

Oh, and since I’ve been asked about my nickname here and there, I decided to add tabs to my hackergotchi for clarification:

Aftermath

There have been quite some discussions on the Gtk+-3.0 plan after Kris’ keynote. One thing that was brought up is that releasing an ABI incompatible but featureless new version of Gtk+ and calling it 3.0 is rather unconventional. An alternative scheme could involve releasing the ABI incompatible cleaned up version as 2.99.0, make 2.99.x the new development branch and release 3.0 with cleaned up ABI and new features (would have been 3.2 in the original plan).

Every other week, someone asks how to use the new unit testing framework in GLib (released with GLib-2.16.0) and Gtk+ (to be released with the next stable). First, here is a write-up from last December that summarizes all important aspects: Test Framework Mini Tutorial.

For people building packages that use the new framework, the following new Makefile rules will be of interest:

• make test
  Run all tests recursively from $(TEST_PROGS), abort on first error.
• make test-report
  Run all tests recursively, continue on errors and generate test-report.xml.
• make perf-report
  Run all tests recursively, enable performance tests (this usually takes significantly longer), continue on errors, generate perf-report.xml.
• make full-report
  Run all tests recursively, enable performance tests, enable slow (thorough) tests, continue on errors, generate full-report.xml.
• make check
  Run make test in addition to automake checks.

After GUADEC, we will be looking into getting build machines setup that’ll regularly build GLib, Gtk+ and friends, run the unit testing suites and provide reports online.

For those pondering to write unit tests, but too lazy to look at the tutorial:

• Implementing a test program is very easy, the only things needed are:

    // initialize test program
    gtk_test_init (&argc, &argv);
    // hook up your test functions
    g_test_add_func ("/Simple Test Case", simple_test_case);
    // run tests from the suite
    return g_test_run();

• In most cases, a test function can be as simple as:

    static void
    simple_test_case (void)
    {
      // a suitable test
      g_assert (g_bit_storage (1) == 1);
      // a test with verbose error message
      g_assert_cmpint (g_bit_storage (1), ==, 1);
    }

  Tests that abort, e.g. via g_assert() or g_error(), are registered as failing tests with the framework. Also, the gtester utility used to implement the above Makefile rules will restart a test binary after a test function failed and continue to run remaining tests if g_test_add_func() has been used multiple times.

• Checks in tests can be written with if() and g_error() or exit(1), or simply by using variants of g_assert(). For unit tests in particular, an extended set of assertions has been added, the benefit of using these are the printouts of the involved values when an assertion doesn’t hold:

    g_assert_cmpstr   (stringa, cmpop, stringb);
    g_assert_cmpint   (int64a,  cmpop, int64b);
    g_assert_cmpuint  (uint64a, cmpop, uint64b);
    g_assert_cmphex   (uint64a, cmpop, uint64b);
    g_assert_cmpfloat (doublea, cmpop, doubleb);

  For instance:
  char *string = "foo"; g_assert_cmpstr (string, ==, "bar");
  yields:
  ERROR: assertion failed (string == "bar"): ("foo" == "bar")

• The framework makes it easy to test program output in unit tests:

    static void
    test_program_output (void)
    {
      if (g_test_trap_fork (0, G_TEST_TRAP_SILENCE_STDOUT |
                               G_TEST_TRAP_SILENCE_STDERR))
        {
          g_print ("some stdout text: somagic17\n");
          g_printerr ("some stderr text: semagic43\n");
          exit (0);
        }
      g_test_trap_assert_passed();
      g_test_trap_assert_stdout ("*somagic17*");
      g_test_trap_assert_stderr ("*semagic43*");
    }

• And it is similarly easy to test and verify intentional program abortion:

    static void
    test_fork_fail (void)
    {
      if (g_test_trap_fork (0, G_TEST_TRAP_SILENCE_STDERR))
        {
          g_assert_not_reached();
        }
      g_test_trap_assert_failed();
      g_test_trap_assert_stderr ("*ERROR*test_fork_fail*not*reached*");
    }

• The above and more tests are showcased in GLib: glib/tests/testing.c.

There’s of course lots of room left to improve GLib and Gtk+ unit tests, and also to improve the current framework. For a speculative, non-comprehensive list, here are some ideas from the unit testing section of my personal TODO:

• Introduce 2D marker recognition for graphical unit testing of Gtk+ layouts (prototyped in Rapicorn).
• Provide functionality to determine image similarities to allow for pixel image based unit tests (port this from Rapicorn).
• Implement state dumps to automate result specification and verification in unit tests. (This will allow us to avoid adding lots of abusable testing hooks to our API.)
• Integrate performance statistics (like #354457) and other related information into test reports.
• Publically install a variant of the Makefile.decl file used in Gtk+ to implement the test framework rules and Xvfb swallowing of test programs. This is needed by other projects to run unit tests the way Gtk+ does.
• Implement the unit test ideas that are outlined at the end of this email: Gtk+ unit tests (brainstorming).

The XML GUI definition files used in Rapicorn and also in Beast (described briefly in an earlier blog post) supported a simple $(function,arguments...) evaluation syntax, similar to GNU Make. I’ve never been very happy with this syntax, but it was fairly easy to implement at the start and followed naturally from early $VARIABLE expansion features. At some point last year I considered various alternative syntax variants and discussed the ideas with Stefan Westerfeld. Over the course of the last two months, I finally got around to implement them.

I’ve never grown familiar with reverse polish notation, and although Guile is the canonical scripting language for Beast, I’ve always found myself very inefficient with expressing my thoughts in lisp expressions. So the new syntax definitely had to support infix expressions - despite the more complex parsing logic required to parse them. Bison already ships with an example calculator that parses infix expressions, so that’s a quick start as far as the syntax rules go. Integration is quite a different story though, more on that later.

Since in Rapicorn the expressions are used to compute widget property values, they are likely to be executed very often, i.e. each time a widget is created from an XML file description. Consequently, I wanted to use a pre-parsed AST to carry out the evaluation and avoid mixing evaluation logic with parser logic, which would have forced reparsing expressions upon each evaluation. At first I quickly threw together some C++ classes for the arithmetic operators and used those as nodes for the AST, similar to:

  class ASTNodeNot : virtual public ASTNode {
    ASTNode &m_operand;
  public:
    explicit ASTNodeNot (ASTNode &operand) :
      m_operand (a)
    {}
    virtual Value
    eval (Env *env) const
    {
      Value a = m_operand.eval();
      return Value (!a.asbool());
    }
  };

The supported syntax is quickly summarized:

  Operators: ( + - * / ** < > <= >= == != or and not )
  Functions: name ( args... )
  Inputs:    FloatingPoint 'SingleQuotedString' "DoubleQuotedString"

In this notation, FloatingPoint includes hexadecimal numbers and of course integers and the quoted strings support C style escape sequences like octal numbers, ‘\n‘, ‘\t‘ and so on. The functions can be implemented by the parser API user, but a good set of standard arithmetic functions is already supported like ceil(), floor(), min(), max(), log(), etc. So it’s a very basic parser, but covers the vast majority of expressions needed to position widgets or configure packing properties.

One thing that turned out to be tricky is the binary operator semantics for strings. At the very least, I wanted support for "string" + "string" and "string" == "string". Since both operators are supported for numbers as well, the exact behavior of "string" + FloatingPoint and "string" == FloatingPoint had to be defined. I managed to find a few programming language precedents here in Perl, Python, and ECMAScript (Javascript). They of course all handle the cases differently. In the end I settled with ECMAScript semantics:

  Value1 == Value2  # does string comparisons if both values are strings
  Value1 + Value2   # does string conversion if either value is a string

Unit testing for the parser turned out to be particularly easy to implement. All that’s needed is a small utility that reads expressions and prints/verifies the evaluation results. Throwing in some additional shell code allowed a normal text file to drive unit testing. It simply contains expressions and expected results on alternating lines. Btw, libreadline can be really handy in cases like this. Using it takes a mere 5-10 lines of additional code to support a nice interactive command line interface including history for the evaluator test shell.

After some initial testing, the C++ AST node classes seemed like an awful lot of pointers, fragmentation and VTable calls for a supposedly straight forward expression evaluation. Also, adding the missing destruction semantics would have significantly increased the existing class logic. So I tried to come up with a leaner and foremost flat memory presentation. In the end, I settled with a single array that grows in 4 byte (integer) steps, embeds strings literally (padded to 4 byte alignment) and uses array offsets instead of pointers for references. A single multiplication operator is encoded with 3 integers that way: MUL_opcode factor1_index factor2_index. That’s essentially 12 bytes per binary operator, still significantly more than the parser input, but also significantly smaller than allocating an equivalent C++ class. Evaluation of the expression stored in the array doesn’t need any VTable calls, and a single straight forward evaluation function can be used, that implements the different operators as switch statement cases. Also release semantics are persuasively trivial for a single consecutive array.

What’s left was to figure a way to embed expression evaluation in XML values or text blocks. Previously, $(expression) was substituted everywhere, but with the new syntax, variables (or rather constants defined within the Rapicorn core or via the ‘<arg:ArgName default="5"/>‘ syntax supported by Rapicorn XML files) didn’t use a $-prefix anymore. So sticking with $() seemed to make little sense. As it’s implemented now, backticks are used to cause expression evaluation, with the empty expression evaluating to a single backtick:

  XML Value/Text         Parser Result
    Foo  5 + 5      ->     Foo  5 + 5
    Foo `5 + 5`     ->     Foo 10
    ``Foo``         ->     `Foo`

We will see how useful the current expression style turns out to be. I don’t consider every implementation bit outlined here solidly engraved into stone yet. So as always, I’m open to constructive feedback.

As forewarned, I have a few more words on integrating Flex and Bison with each other and into one’s own library. First, Flex and Bison turned out not to be exactly simple to configure, especially if none of the generated symbols should be exported from a library or clash with a possible second parser linked into the same library or program. Also some fiddling is required to pass on proper line count information from the lexer to the parser, getting character counts as well is non-trivial but lucky wasn’t strictly needed for Rapicorn expressions. The simplest setup I managed to come up with works as follows:

  sinfex.hh     # public API
  sinfeximpl.hh # internal structure definitions
  sinfex.cc     # evaluator implementation
  sinfex.l      # scanner rules for Flex
  sinfex.y      # parser rules for Bison
  sinfex.lgen   # generated by Flex
  sinfex.ygen   # generated by Bison

The only compiled file in this list is sinfex.cc which includes sinfex.lgen and sinfex.ygen as part of an anonymous C++ namespace. A linker script ldscript.map used when finally linking the library prevents anonymous symbols from being exported. The anonymous namespacing of everything declared in sinfex.lgen and sinfex.ygen is what prevents clashes with a possible second parser linked into the library. This isn’t as elegant as i was hoping for, but at least effective in a practical sense. There unfortunately is no way to configure Flex or Bison to generate only static functions and variables. And yes, I have also looked into variants like flex++, bison++, bisonc++, byacc, etc, but they usually show much of the same problems and also tend to make matters worse by generating more files or more complex code.

Just like LinuxTag last year, I went to Berlin the past week to help running the Gnome booth for LinuxTag 2008.

Due to a sports accident, our booth bunny Sven Herzberg unfortunately couldn’t make it, so on Tuesday I took over booth management and merchandise from him and hurried to Berlin in an ICE instead of a car as was originally planned. In Berlin, I met up with Mathias Hasselmann who brought the European Gnome event box and together we set up the booth until late in the night.

On Wednesday morning, Michael Köchling and Christian Dywan arrived, so we had enough people to properly man the booth. Michael seems to be an early riser, since he managed to show up at 09:00 for all days, so i passed the booth keys on to him.

Around lunch time, I was dragged away for an interview about business involvement in free software and Gnome in particular by Malgorzata Ciesielska, a business school student from Copenhagen. She also interviewed other people like Lennart Poettering who also sporadically hung around our booth.

Later that day, Lennart and i went over the new libcanberra API in a lengthy discussion. Libcanberra is a new library for playback or activation of sound events in response to desktop actions that Lennart currently works on. We talked about the needs of timing information for some usage cases, possibly also dispatching forced feedback controls via the library and implementation of a Gtk+ module to hook canberra functionality up with GUI events. What turned out to be a bit tricky is to derive actual semantic information from the low level X event notification that Gtk+ signals proxy, such as dialog-confirmed, dialog-cancelled, menu-item-selected, menu-item-cancelled, combobox-popup, combobox-selected, combobox-cancelled, etc. This extraction requires significantly more logic and special casing of event notification than Lennart apparently had originally hoped for.

On Thursday I attended the Linux Kernel - Quo vadis? talk by Thomas Gleixner which was quite interesting. I managed to catch him afterwards to talk about the prospects of having a memory pressure signal in the Linux kernel. For GLib and Gtk+, this’d be quite useful to voluntarily release pixmap or GSlice caches, particularly desired on embedded platforms.

Friday I sat down with Vincent Untz for a very productive discussion about Gnome/Gtk+ release prospects, autotools, intltool features and more. Later I had a chance to chat with Alexander Neundorf about KDE’s recent CMake migration process. Overall, they seem to be pretty happy with the results. Major benefits from migrating from autotools to CMake seem to be:

• Build process speedups due to getting rid of libtool.
• Simplicity; the build setup is back to a manageable level again. For KDE, the previous combinatoric mess of autotools was hardly fully understood by any single person.
• Unification/merging of build files for Unixes and Windows. (Duplication of build logic between auto* files, nmake and MS project files is currently a major annoyance for Gtk+’s Win32 maintainers.)

On Saturday the last day of the conference, I attended Anne Østergaard’s presentation about Gnome Foundation structures and achievements, the slides of which are available here: The GNOME Foundation (PDF).

After that, I went to Jono Bacon’s talk in which he explained how the free software community is a creative and productive community, which sets it apart from other common community types in our society (usually fan communities). He went on pointing out how this collaborative and open community as a whole (and thus every significant contribution to it) impacts and gradually changes the really big IT companies from within in an unprecedented manner. Come to think of it, this is an incredible achievement that we should rejoice in, especially because it is a morally correct change in that it strives towards openness.

All in all, it was a nice conference again. Personally, I particularly enjoy meeting up with other hackers for productive face to face sync ups. So I’d like to thank Christian and especially Michael for their great efforts in patiently answering bypasser’s Gnome questions at the booth, while I wandered off to talks or had technical discussions in its back.

Amongst many other things during the Gtk+ Hackfest 2008, it was brought to my attention that Gtk+ maintainership is sometimes perceived as some kind of leet circle, hard to join for newcomers. I can’t really imagine how “hard” it is for newcomers to join Gtk+ maintenance these days. The learning curve probably is steeper now than it was in the last decade, but then, we do have documentation now and didn’t have any back then… ;-)

In any case, I think that definitely none of the Gtk+/GLib core maintainers would want to bar someone else from contributions or helping out with maintenance tasks. So to aid that process, I’ve written down what I keep telling people who approach me about this in person. A lot of it might seem overly obvious to veterans, but for newcomers or contributors looking into extending their activities on the project I hope to provide helpful starting points.

Much of Gtk+ is still maintained as a huge monolithic whole. That is, a very few people are taking care of a wide variety of components. I think ultimately we would all be much better off, if we had more people being responsible for individual components like particular widgets (e.g. GtkNotebook) or widget families (e.g. GtkBox, GtkHBox, GtkVBox). So the following are steps and tasks useful for anyone wanting to get into Gtk+ component maintenance.

First of all, we have the GtkTasks wiki page, a task list with various ways in which people can contribute to the Gtk+ project and start getting involved. In particular for the following positions, no one has signed up so far to volunteer on a regular basis:

• Patch/Bug Monitoring & Filing - collect bugs from mailing lists and IRC to make sure they are being filed.
• FAQ Maintainer - this involves monitoring the mailing list(s), taking CC:-mails from other people on possible FAQ material and regularly updating the FAQ accordingly.
• Build Monitoring - run and maintain Gtk+ tool-chain builds on various platforms.
• Making Releases - we are looking for someone assisting in the release process and to take over release building during some periods in the future.
• Implementing Test Programs - there’s much work needed in our unit test coverage, so we’re always looking for people willing to implement more unit tests.

For general information about the maintenance situation, the State of the Gtk+ Maintenance write-up is still fairly valid. However many good things have been suggested by the community since, such as the GtkTasks page and the Hackfest. At the end, the write-up addresses how occasional contributors or developers at different experience levels can help out the project. For instance with activities such as: Bug triage and verification, Review and clarify documentation, Revision hunting for regressions, Refactor code areas, Work on optimizations, and the list goes on.

If none of this sounds interesting to potential new maintainers, be warned that regular maintenance means having to put up with pretty much all of these at some point. ;-)
However, probably the most straight forward way to take on maintenance for a particular code portion (usually a particular widget) is to start becoming familiar with it and work on improving it right away:

• Cleanup indentation - lots of contributions to Gtk+ in the past have weakened coding style in various areas. In general we use GNU coding style plus a few extra rules similar to the Gimp coding style.
• Perform code cleanups - look for outstanding migrations/refactorings in the code or if the implementation could be simplified/streamlined in areas.
• Check documentation and examples - contributing by improving the existing documentation, testing existing examples and providing new ones is a very straight forward way to learn about a new component. Also, documentation patches are usually easily and quickly approved.
• Provide unit tests - writing new unit tests for existing component APIs is even better than providing documentation examples. You get immediate feedback and they should in general be easy to approve to go into upstream. Also, it is definitely an area where Gtk+ and GLib still need lots of work.
• Review bug reports and patches - go through the Gtk+ bug load of a particular component, see what issues could be closed or need info. Find patches that could be applied or need work and provide/fix patches along the way. Also, feel free to provide review for existing patches where you feel confident to provide reasonable input. For existing maintainers, looking at other people’s review is one of the best ways to figure if another person is up to the task of taking over component maintenance.
• Actively nag existing maintainers or people with SVN accounts for trivial patches (like Cody and Mathias who signed up for Patch testing & committing) to review, approve and apply your changes.
• Participate in the project forums like gtk-devel-list and #gtk+ (needed to nag people from the core team and other developers), and read and reply in other related mailing lists.

The first few points actually mean working with the code by providing patches, and filing new bug reports with those patches attached. While this may at first increase our bug load, if someone shows sincere interest in taking over component maintenance, sticks to the coding style and provides useful patches, there’s nothing stopping us from granting new SVN accounts so contributors can commit their own patches after approval.

Finally, the project is welcoming every new contributor. But be reminded that no approval is needed from anyone to start your work. In actuality, asking for it will most probably get you a reserved or negative answer, because improving the project is all about working on it, not making or requesting promises. So, everyone is invited to read through the task lists/descriptions and get their hands dirty immediately. A fair bit of self incentive is needed to take on maintenance of a component anyway, so you’ll have to get yourself motivated on your own there won’t be anyone else doing it for you.

Rapicorn 8.4.0 has just been released to the world: Rapicorn v8.4.0 Announcement

Lots of things have happened since the last snapshot over a year ago, some of which kept me from making releases or snapshots earlier. ;-)
Others actually took place in the code base as interesting developments, summarized below.

There’s quite some more stuff in my development queue, but at some point one just has to draw a line and throw out what’s vaguely ready so far, so this is it for today:

  Overview of changes in Rapicorn 8.4.0:

  * Changed versioning scheme to YEAR.WEEK.REVISION.
  * License update to GNU LGPL 2.1.
  * Added a publically installed tool: rapidrun
  * Support println() and close() commands in GUI files.
  * Introduce simple Application and Window object APIs.
  * Merged libbirnet into Rapicorn as librapicorncore.
  * Implemented expose region merging/comprssion.
  * Reiimplemented rectangle gradient shader.
  * Switched to autogenerated ChangeLogs.
  * Improved feedback on parser errors.
  * Fixed Gtk+ version checks.
  * Added PNG saving support.
  * Removed PERL build dependency.
  * Rewrote asyncronous main loops.
  * Many improvements to text editing areas.
  * Speed up blitting logic for local displays.
  * Added SIMD optimized rendering functions for MMX CPUs.
  * Fixed some reference counting issues and child removal.
  * Improved vertical text ellipsization to line granularity.
  * Removed error prone default values from property mechanism.
  * Install tutorial under ${prefix}/doc/rapicornXXXX/tutorial/.
  * Misc compiler and threading fixes, depend on g++-3.4.6.
  * Lots of bug fixes, cleanups and improved test coverage.

There have been several requests about hosting Gtk+ (and GLib) as a Git repository recently and since that topic has come up more and more often, I meant to write about this for quite some time.

Let’s first take a look at the actual motivation for such a move. There are a good number of advantages we would get out of using Git as a source code repository for Gtk+ and GLib:

• We can work offline with the Gtk+ history and source code.
• We can work much faster on Gtk+ even when online with tools such as git-log and git-blame.
• It will be much easier for people to branch off and do development on their own local branches for a while, exchange their code easily with pulling branches between private repositories, etc. I.e. get all the advantages of a truly distributed versioning system.
• With Git it’s much easier to carry along big Gtk+ changes including history by using cherry picking and (interactive) rebasing.
• We can make proper public backups of the source code repositories. This ought to be possible already via svnsync, but isn’t for svn.gnome.org because we run an svn 1.3.x server instead of an svn 1.4.x server that is required by svnsync. (Yes, this issue has been raised with the sysadmins already.)

A quick poll on IRC has shown that all affected core maintainers are pretty much favoring Git over SVN as a source code repository for GLib/Gtk+.

However, here are the points to be considered for not moving the repositories to Git (just yet):

• Complexity; Git is often perceived to be harder to use than SVN, there are several attempts to mitigate this problem though: Easy Git Giggle yyhelp.
  With some of the above, Git is as easy to use as SVN is, so Git complexity doesn’t need to be holding anyone off at this point.
• Git may be stable and mature these days, but git-svn is not there yet. It is generally good enough to create local Git mirrors of SVN repositories and work from those to have most of the Git convenience on top of an SVN project.
  But git-svn has seen structural changes recently, quite some rewriting and bug fixing that indicate it’s still too much in flux for the one-and-only SVN->Git migration for projects at the scale of Gtk+. This is not meant as criticism on git-svn, fairly the opposite actually. It’s good to see such an important component be alive and vivid. All issues I’ve raised with the maintainer so far have been addressed, but it seems advisable to wait for some stabilization before trusting all the Gtk+ history to it.
• Gitweb interfaces already exist for GLib/Gtk+ mirrors, for example on testbit: Testbit Git Browser.
  These can be used for cloning which is much faster than a full git-svn import. Alternatively, shallow git-svn imports can be created like this:

    git-svn clone -T trunk -b branches -t tags -r 19001 svn://svn.gnome.org/svn/gtk+
  

  This will create a repository with a mere ~1000 revisions, including all changes since we branched for 2.12. We’re using such a shallow repository for faster cloning of our GSEAL() development at Imendio: view gtk+.git.

• In summer 2006, we’ve had the first test migration of all of GNOME CVS to SVN, in December 2006 we’ve had the final migration. During that period, the Beast project stayed migrated to SVN to work out the quirks from the CVS->SVN migration before we migrate all other GNOME modules and have to fix up everyones converted modules. There were quite some issues that needed fixing after the initial test migration and in the end we had to rebuild the Beast development history from pieces. Preventing the other GNOME modules from such hassle was the entire point in migrating Beast early on, so I’m not complaining.
  However, given the size and importance of GLib/Gtk+, the development history of those projects shouldn’t be put at a similar risk. That is, GLib/Gtk+ shouldn’t be pioneering our next source repository migration, let some other small project do this and work out the quirks first.
• git-svn already provides a good part of the Git advantages to developers while Gtk+ stays hosted in SVN. Albeit due to mismatching hashes, syncing branches between distinct git-svn checkouts of different people is tedious. Setting up an “official” git-svn mirror on git.gtk.org could probably help here. Also, to ease integration, jhbuild could be extended to use git-svn instead of svn commandos to update SVN modules, if the current module has a .git/ subdirectory instead of a .svn/ subdirectory.

The bottom line is, there’s a good number of advantages that Git already can (or could) provide for our development even without migrating the repositories to Git right away. When exactly will be a good point for migrating GLib/Gtk+ and possibly other GNOME modules might not be an easy call, but right now is definitely too early.

I finally got around to fix a long-standing and tricky bug report: Bug 64764 - Class initialization isn’t thread safe.
Thread safety problems in class initializers and _get_type() functions caused nasty problems in other components that depend on parallel type creation, in particular GStreamer (Dependency Graph for bug 64764). With both being fixed now, testing feedback about GType/GObject threading problems using GLib trunk is appreciated.

	Overview of Changes in YummyYummySourceControl-0.9:

	* use plain "git-commit" with temporary index file to stage
	  commit files, this works around git-commit-1.5.2.5 not
          handling deleted files as command line args correctly.
	* also list remote branches for yylsbranches.
	* fix leading dot getting stripped from modified files if $gitprefix=.
	* require and use gawk for time formatting, which mawk doesn't support.
	* properly honor the [FILES...] arguments to yycommit.
	* terminate sed command blocks with semicolon (needed on BSD).
	* resurrected yyhelp.auto-push-commits functionality of yycommit.

This version supports a new command to grep and match an extended regular expression on the full project history by invoking git-grep(1) on all existing commits, displaying matches by commit hash and file name. Also, a CVS/SVN/Cogito style version of yycommit has now been implemented on top of git-commit(1). So YummyYummySourceControl-0.8 finally gets rid of the last remaining cogito dependency, making it a free standing Git Porcelain Suite. ;-)
Of course there have also been some other miscellaneous changes and docu updates.

	Overview of Changes in YummyYummySourceControl-0.8:

	* removed cogito(7) dependency.
	* made yyview start gitk in the background.
	* suppress yydiff outputs of non-checked-out files (ignored by yycommit).
	* implemented yycommit SVN/CVS/CG-alike on top of git-commit.
	* added '-s' option to yyChangeLog to skip SVN revisions.
	* implemented yyHistoryGrep.

I have been asked some of the purpose of YummyYummySourceControl, so i will extend a bit on that here:

- On some occasions i had to dig quite deeply into the nitty-gritty details of git (and cogito). However i can simply not be bothered to deal with the complexity of its command set for everyday use. I need my source control interface to be really simple when i concentrate on the various projects i’m involved in.

- I refuse to bother with the state (or existence) of git’s index file for anything but the correct implementation of yyhelp. SVN and CVS don’t force me to deal with an intermediate cache state between repository and working tree, YummyYummySourceControl keeps me from seeing it in git.

- I might be using yydiff and yyview up to a dozen or so times per hour while working. Passing options along here or piping the output to a pager or starting stuff in the background really becomes unaffordable at that rate. These commands do all the necessary stuff out of the box, all i have to type is “yyd”+Tab+Return and “yyv”+Tab+Return for each of them now.

- I’m too old and stupid to remember to pull after i’ve pushed and what the git-svn(1) command variants look like. yypull and yypushpull handle that for me.

- Personally, i find ChangeLog style logs much more parsable than git-log(1) output. yyChangeLog gives me that for the git commit history.

- To allow proper git merging, cherry picking, rebasing, etc, i’ve switched to commit my ChangeLog entries as commit message only (with introductionary one-liner instead of the date + email header) and for Rapicorn auto-generate the project ChangeLog for tarballs. Even if the ChangeLog is not auto generated as with Beast or Gtk+, i can still use this commit message scheme for git-svn checkouts, as long as i update the project’s SVN ChangeLog before yypushpull-ing the changes from my local tree to SVN. yyChangeLog -s will generate the remaining ChangeLog portion missing from SVN.

- To help my utter laziness, yystatus -c can preformat file name change lists for my ChangeLog-style commit messages, i’ve always missed that with CVS and SVN.

- To help against sudden confusion and lengthy URL searches, yyinfo tells me all i want to know about a repository (git URL + revision, SVN URL + revision, etc).

And i love to have grep-able access to all the data involved in my day to day activities, yyHistoryGrep now gives me that for my project histories.

Given git’s speed, the possibility for light-weight local repositories, and using yyhelp’s ease, revision control has become so available for me, that i could very comfortably apply it to portions of my home directory and /etc/ on a remote server (e.g. i’ve created a ~/.git/ a couple months ago where i checked in a handful selected files like TODO, my blog diary, .emacs and a few other frequently changing TODO-style files and i’m happily committing to it multiple times a day).

I’m not claiming that yyhelp solves anyone else’s problems, but it certainly has helped my SCM usage a great deal, and for me serves as a key to make a frequently used subset of the git machinery accessible at very few finger tips.