The Lifecycle of a Patch (or: Working Upstream)

community, freesoftware, gimp, gnome, inkscape, maemo, meego 5 Comments

Reposted from Neary Consulting

Yesterday I looked into what it means to be a maintainer of a package. Today, I’m going to examine how to affect change in a distribution like MeeGo, and what it means to work upstream. To do so, we’re going to look at how code gets from a developer’s brain into the hands of a user.

So – how can you make a change in a Linux-based distribution? Here’s what happens when everything works as it should:

  1. You open a bug report for the feature against your distribution
  2. You identify the module or modules you need to change to implement the new feature
  3. You open bug reports for each of the modules concerned, detailing the feature and the changes needed in that module for the feature
  4. You write a patch to implement the feature, and propose it (appropriately cut up for ease of review) to the maintainers of those modules
  5. Once the code has gone through the appropriate review process, it will be committed to the source control of the module(s)
  6. Some time later, the maintainer of each module will include that code in a stable release of the module
  7. Some time after that, the new stable versions will be packaged and uploaded to MeeGo
  8. Your code will be included in the next release of the distribution following the upload.

When people talk about “working upstream” in MeeGo or Linaro, this is what they mean.

To simplify matters for our analysis, let’s consider that the feature we want to implement is self-contained in one module (or related modules which release together). There are two different scenarios we’ll consider:

  1. The module is maintained by people not associated with your distribution (for example, a GNU or GNOME project)
  2. The module is maintained by people closely related to your distribution (for example, Unity in Ubuntu, or oFono in MeeGo)

We will also look at a third situation, where you find and fix a bug in the software you are using – that is, a released version of a distribution (the proverbial “scratching an itch”).

For each case, I will try to pick a representative feature/patch and follow it from developer through to distribution to Real Users.

What if your code changes different projects?

If your code touches several modules (for example, if you are proposing some new API in GTK+ which you want to use in the GIMP) then things can get complicated – you will need a stable version of GTK+ to be released before you can ship a stable release of the GIMP which depends on it.

This issue of staggered releases is one that Andrew Cowie pointed out a few years ago for language bindings. To avoid making bindings on shifting sands, he preferred to package new APIs once they had been included in a stable GNOME release. In turn, Java GNOME developers rarely depend on development release bindings, and they would wait for the new API to be included in a stable bindings release. For example, the gtk_orientable_get_orientation, added to GTK+ at the end of September 2008, was released in GTK+ 2.16, in March 2009. The first version of Java-GNOME which depended on GTK+ 2.16 was version 4.0.13, released in August 2009. That was packaged in distributions in Autumn 2009, and so most users would not have access to the newer bindings for a few months after that – perhaps early 2010 – at which point, the API was written 18 months beforehand.

And that is when you have a regular release schedule you can rely on! Pity the developer who wants to release a GIMP plug-in which depends on some API included in GIMP 2.8 – the last stable GIMP release, 2.6, came out in October 2008, and over two years later, 2.8 still has not released. And when you combine unreliable release schedules for distributions and applications, the results are cumulative: users of the stable Debian distribution are still using GIMP 2.4 releases. The GIMP 2.4 released in October 2007. Features added to the GIMP in late 2007 are still not in the hands of users of stable Debian distributions.

Getting features to users

It is difficult to generalise when users upgrade their Linux distributions, or even to say what proportion of Linux users are new users at any given time. It would be over-simplifying to say that developers use bleeding-edge distributions, power users upgrade early to the latest and greatest, new users install the latest distributions available, but will only upgrade every 18 months or so afterwards, and conservative users stick with “Long term service” or stable distributions. Most developers I know use their computer for work (and thus want a stable distribution) and only install the latest versions of various dependencies they need to work on their project. But let’s generalise and say that this is roughly the case. So (guesstimating) about 10% of your users will be upgrading to the latest distribution very quickly after its release, a further 20% in the months after when the bugs are shaken out, and the rest will follow along in their own time, perhaps 12 or 18 months later.

To make this concrete, let’s follow the life of a single patch. This is complete anecdata, but in my defence, the patch has been chosen by random, from a project which I know has good community processes and release management in place. The patch we’re going to follow adds an extension to Inkscape to render objects along triangular paths.

  1. Bug #226001 opened on 2008-05-03 by inductiveload, with a description of the feature to be added, and proposed code to implement it. The code, as an extension, may have a lower bar for acceptance than code which is core to a project.
  2. Patch submission reviewed on 2008-05-03, minor comments, but patch is accepted (note: This was not the authors first submission to Inkscape)
  3. Patch corrected to respond to comments and committed on 2008-05-03 (did I mention these guys had good community processes!?!)
  4. Inkscape 0.47-pre0, containing the Triangle extension, released on 2009-07-02
  5. Inkscape 0.47-pre4 included in Ubuntu 9.10

So for a feature developed in mid 2008, most Inkscape users will still not have the feature by the end of 2009, 18 months later. This is both a typical and atypical example: in many projects, patch proposals lay unreviewed for days, weeks, sometimes months, but the 0.47 release cycle was a particularly long one for Inkscape. However, I think the lag from code written to presence on user’s hard drives of ~12 to 18 months is about correct.

Does it have to be this hard?

If this were the only way to get features into a distribution, trying to improve MeeGo by contributing upstream would be a very frustrating experience. Happily, there are ways to accelerate the process. Taking the MeeGo kernel as an example, where Greg Kroah-Hartman recently threw in the towel on persuading people to propose patches upstream; the process is supposed to work like this:

  1. Propose a patch for inclusion upstream. This patch will then ship in a future stable kernel release (let’s say 2.6.38).
  2. After peer review, when the code has been accepted for inclusion in the kernel upstream, propose a backport for inclusion in the MeeGo kernel. The back-ported patch will be maintained across the next MeeGo release, and will be dropped when the kernel version included in the MeeGo project catches up with 2.6.38.

The overhead here is reduced basically to the peer review process of the upstream project, and the cumulative cost of merging a patch over the course of 6 months.

As a distributor (or a developer working on a specific distribution), this allows you to get code to everyone, eventually, and have that code included in your distribution as soon as you are sure that it is up to the standard expected by the community. Currently in MeeGo, the trend seems to be more towards submitting patches concurrently upstream and to MeeGo kernel maintainers (or even submitting them upstream once they have been accepted into the MeeGo kernel). In the case that a patch requires substantial modifications, or is rejected outright, upstream, the kernel maintainers are then left carrying a patch indefinitely in the distribution. For one patch, this might not be a big deal, but for thousands of patches, the maintenance and integration burden of these patches adds up.

It is also not unusual for kernel developers to maintain their own git branches for a long time. Three examples that come to mind are inotify, which Robert Love maintained for over a year for both Novell and in the kernel before it was accepted into the mainline, ReiserFS, which was maintained for several years out-of-tree before being shipped with the Linux kernel in 2001, and the fast  desktop patchset which Con Kolivas maintained for almost five years on the -ck kernel branch. Distributions will occasionally ship a substantial diff to upstream if there is a maintainer committed to getting the code upstream eventually. Allocating someone to work over a long period to make everyone happy and comfortable with your code may enable you to ship a big patch to upstream, but this will not be sustainable long term.

To summarise: when working upstream, as a distribution, you should only ship with patches which have been accepted in a development version of upstream already, if you can help it.

Meetings in telephone boxes

Sometimes, however, when upstream and downstream coincide, you can simplify things considerably, while also adding a small measure of risk.

In MeeGo, to continue with that example, the distribution architects have a pretty good idea when they can expect emergency telephony to be ready for oFono and the MeeGo telephony stack, because they’re writing it. By co-ordinating the upstream release management with downstream packaging, you can make promises as a distribution which you can’t with community-developed software.

When upstream and downstream are co-ordinating each other, we cut out the middleman. The workflow becomes:

  1. Report a bug/feature request against a component of the distribution
  2. Develop a patch which implements the feature, and submit it directly to the distribution bug tracker
  3. Once it has been reviewed and accepted, you know that your patch will be included in the next version of the distribution.

This gives a distribution much more control, both over what gets done, and when, and explains both the Ayatana and MeeGo UX development projects. However, being able to plan around the release is no guarantee that the release will happen on time: GNOME has in the past been stung by planning during the 2.6 development cycle to depend on a new version of GTK+, only to find that the release was delayed. In the end, the GTK+ release shipped in time for the 2.6 release at the end of March.

Scratch scratch

The other patch lifecycle I’d like to mention, because it is so relevant to distributions, was pointed out to me by Federico Mena Quintero yesterday. What happens to a patch that someone makes and submits to a distribution when they find a bug in stable released software? This is one of the key advantages of free software – if you find a bug in the software you use, and you have the wherewithall, you can fix the bug and share that fix with everyone else.

However, as we have seen, there is typically a lag of several months from the time that software is released and the time it is being used by large numbers of users through distributions. With releases of Red Hat Enterprise Linux, Novell Suse Linux Desktop and Ubuntu LTS being supported for up to 5 years, it is possible that important bugs will be fixed in these stable versions for years after the original developers have moved on, and are no longer maintaining older stable versions.

Let’s say I find and fix a bug in Rhythmbox 0.12.5, which ships with Ubuntu 9.10. I open a bug report on Launchpad, attach a fix to the source .deb there, and I update my local copy. As a user, I’m happy – I have fixed my problem and shared the solution with others. If I’m particularly conscientious, I might open a bug on gnome.org against Rhythmbox and attach my patch there, but since the development version is now 0.13.2, the best you can hope for is that the patch applies cleanly to the master branch, and will be included in the next release. It is very unlikely that the upstream maintainers will release another update to the 0.12 series at this point.

Now imagine that you are a maintainer for Suse, and someone reports the same bug against a long-term service release.In practice, there are several different versions being maintained by different distributions, and no good way to know if the same bug has been reported and fixed by someone else. You end up searching for a fix in upstream bug trackers, and in the bug trackers of each of the other main distributions. According to Federico at the time:

Patches for old versions are traded in the black market. You have friends in another distro? You ask them first, “did you guys already fix this?” Those patches don’t ever manage to reach CVS, where everyone would be able to get them.

Ideally, you could collaborate ahead of time with other distributions to ensure that you are all using the same branch of upstream modules, and are committing patches upstream. The Linux kernel is moving to this model, and there are also discussions underway in GNOME to co-ordinate this type of activity. Mark Shuttleworth has also pushed for something similar by encouraging projects in the core Linux platform to have a regular cadence of releases, so that everyone can synchronise their longer term service offerings every couple of years.

But at the moment, the best you can hope for is that your patch will be included in an upcoming release for your distribution, and which point other users of the distro can avail of it, and that upstream will patch their development version and latest stable versions, and get your patch to everyone in a few months.

Working upstream

The goal of this article is to explain what working upstream actually means, and how to make that more palatable for a distribution that wants to get features written and included in their next release. Hopefully, by pointing out some of the shortcomings of the way patches circulate from developers to users, some of these issues can be addressed.

In any case, one thing is clear – if you are carrying a patch as a distribution without ever submitting it upstream, you are making a costly mistake. You will be carrying code that others won’t, and bearing all of the merge and maintenance burden for that code for years to come. The path to maximum happiness is to co-ordinate with other distributions and with upstream to ensure that everyone is working in the same place, and sharing work as much as possible.

What’s involved in maintaining a package?

community, freesoftware, gnome, maemo, meego 7 Comments

Reposted from Neary Consulting

An interesting question was asked on a MeeGo mailing list recently: What does it mean to be a maintainer of something? How much time does it take to maintain software? It resulted in a short discussion which went down a few back alleys, and I think has some useful general information for people working with projects like MeeGo, which are part software development, part distribution.

Are you maintaining software, or a package?

The first question is whether you are asking about maintaining something in the Debian sense, or the GNOME sense?

A Debian package maintainer:

  • Tracks upstream development, and ensures new releases of software are packaged and uploaded in a timely manner
  • Work with distribution users and other maintainers to identify bugs and integration issues
  • Ensure bugs and feature requests against upstream software are reported upstream, and bugs fixed upstream are propagated to the distribution packages
  • Fix any packaging related issues, and maintain any distribution-specific patches which have not (yet) been accepted or released upstream

A GNOME project maintainer:

  • Makes regular releases of the software they maintain (typically a .tar.gz with “./configure; make; make install” to build)
  • Are the primary guardians of the roadmap for the module, and sets the priorities for the project
  • Works with packagers, documenters, translators and other contributors to the software to ensure clear communication of release schedules and  priorities
  • Acts as a central point of contact for release planning, bug reports and patch review and integration
  • A typical maintainer is also the primary developer of the software in question, but this is not necessarily the case

Obviously, these two jobs are very different. One places a high priority on coding & communication, another on integration, testing, and communication.

So how much time does maintaining software take?

Well, how long is a piece of string?

To give opposite extremes as examples: Donald Knuth probably spends a median time of 0 hours per week maintaining TeX and Metafont. On the other hand, Linus Torvalds has worked full time maintaining the Linux kernel for at least the past 15 years, and has been increasingly delegating large chunks of maintenance to lieutenants. The maintenance of the Linux kernel is a full time job for perhaps dozens of people.

On a typical piece of GNOME software (let’s take Brasero as an example) much of the work is simplified by following the GNOME release schedule – the schedule codifies string freezes and interface freezes to simplify the co-ordination of translation and documentation. In addition, outside of translation commits, Brasero has had contributions from its maintainer, Philippe Rouquier, and 6 other developers in the last 3 months. Most of these changes are related to the upcoming GTK+ 3 API changes, and involve members of the GTK+ 3 team helping projects migrate.

In total since the 2.32.0 release, there have been 55 commits relating to translations, 50 commits from Philippe, 9 from Luis Medina, co-maintainer of the module, and there were 4 commits by other developers. Of Philippe’s 50 commits, 14 were related to release management or packaging (“Update NEWS file”), 5 were committing patches by other developers that had gone through a review process, and the remainder were features, bug fixes or related to the move to the new GTK+. Of Luis’s commits, 2 were packaging related, and 2 were committing patches by other developers.

This is a lot of detail, but the point I am making is that the “maintenance” part of the work is relatively small, and that the bigger part of maintenance is actually sending out the announcements, paying attention to bug reports and performing timely patch review. I would be interested to know how much time Philippe has spent working on Brasero over the past release cycle. I would guess that he has spent a few hours (somewhere between 5 and 10) a week.

On the other hand, the Debian maintainer for the Brasero package has a different job. There are 6 bugs currently forwarded upstream from the Debian bug tracker, and another 35 or so awaiting some final determination. A number of these look like packaging bugs (“you need version X of dependency Y installed”). The last release packaged and uploaded was 2.30.3-2, dating from November, and there have been 4 releases packaged in the past 8 months, none by the maintainer.

A typical Debian maintainer is a “Debian developer” for several packages. Pedro Fragoso, the Debian maintainer of Brasero, maintains 5 packages. I think it is fair to say that the amount of time a package maintainer spends maintaining an individual package is quite low, unless it is extremely popular. Perhaps a few hours a month.

The package maintainer has little or no say (beyond interacting with the project maintainer and forwarding on bug reports & feature requests) in what happens upstream, or which features have a high priority. His influence comes primarily from the fact that he is representing a larger user base and can indicate which bugs his distro’s users are running into and reporting regularly, or which feature requests are generating a lot of feedback.

What’s in a word?

It’s clear that a package maintainer is not the same thing as a project maintainer. So when Sivan asked on the MeeGo developer list how he could become a maintainer, he clarified later to say that what he was really asking was “How can I affect change in MeeGo?” To do that, you need to write some code that changes a module, or a number of modules, and then you need to get that code into MeeGo.

How that happens, in all its gory details, is the next instalment in this series of at least 2 articles: The Lifecycle of a Patch (or: Working Upstream).

Community Building Guide

community, freesoftware, gnome, maemo 4 Comments

I wrote another guest article for the VisionMobile blog last week, which just went live yesterday, titled “Open Source community building: a guide to getting it right”.

Exerpt:

Community software development can be a powerful accelerator of adoption and development for your products, and can be a hugely rewarding experience. Working with existing community projects can save you time and money, allowing you to get to market faster, with a better product, than is otherwise possible. The old dilemma of “build or buy” has definitively changed, to “build, buy or share”.

Whether you’re developing for Android, MeeGo , Linaro or Qt, understanding community development is important. After embracing open development practices, investing resources wisely, and growing your reputation over time, you can cultivate healthy give-and-take relationships, where everyone ends up a winner. The key to success is considering communities as partners in your product development.

By avoiding the common pitfalls, and making the appropriate investment of time and effort, you will reap the rewards. Like the gardener tending his plants, with the right raw materials, tools and resources, a thousand flowers will bloom.

After focusing recently on a lot of the things that people do wrong, I wanted to identify some of the positive things that companies can do to improve their community development experiences: try to fit in, be careful who you pick to work in the community, and ensure that your developers are engaging the project well. If you are trying to grow a community development project around a piece of software, then you should ensure that you lower the barriers to entry for new contributors, ensure that you create a fair and just environment where everyone is subject to the same rules, and don’t let the project starve for lack of attention to things like patch review, communication, public roadmapping and mentoring.

The original title of the article was “Here be dragons: Best practices for community development” – I’ll let you decide whether the VisionMobile editors made a good decision to change it or not.

Follow-up to “Shy Developer Syndrome”

community, freesoftware, maemo, work 4 Comments

Reposted from neary-consulting.com

My article on “Shy Developer Syndrome” a few weeks ago garnered quite a bit of interest, and useful feedback. Since a lot of it adds valuable perspectives to the problem, I thought I should share some of my favourite responses.

Here on gnome.org, Rodney Dawes argued that developers tend to stay away from mailing lists because the more public lists are very noisy:

For me, mailing lists are a huge risk vs. low return problem. They can become a time sink easily, and it’s quite often that pointless arguments get started on them, as offshoots of the original intent of the thread. Web Forums also have this problem. And, to really get much of anything out of a list, you must subscribe to it, as not everyone who replies, is going to put you specifically in the recipients headers. That means, you’re now suddenly going to get a lot more mail than you normally would for any highly active project. And for anyone trying to get involved in an open source community, 99% of the mail on that list is probably going to be totally irrelevant to them. It will just make tracking the conversation they are trying to have, much harder.

I agree with Rodney that dealing with a new level of volume of email is one of the trickiest things for new contributors. I still remember when I signed up to lkml for an afternoon in college, only to find 200 new emails 3 hours later. I panicked, unsubscribed, and gave up that day on being a Linux kernel hacker.

Since then, however, I have learned some email habits which are shared by other free software hackers I know. Everyone I know has their own tricks for working with medium or high volume mailing lists, and some combination of them may make things livable for you, allowing you to hear the signal without being drowned out by the noise. LifeHacker is a good source of tips.

Rob Staudinger says something similar, pointing the finger at bikeshed discussions as a big problem with many community lists:

Will the zealots go and suggest postgresql’s process model was poor, or samba’s memory allocator sucks? Unlikely, but they will tell you your GUI was bad or that you’re using a package format they don’t like, just because it’s so easy to engage on that superficial level.

Over at LWN, meanwhile, Ciaran O’Riordan makes a good point. Many developers working on free software want to separate their work and personal lives.

When I leave the office at 6pm, my work should have no more relevance until the following morning. Same when I quit a company. I might choose to tell people where I work/worked, but it should be a choice, and I should be able to choose how much I tell people about my work. Having mailing list posts and maybe even cvs commits might be too detailed. Maybe waaay too detailed.

Finally, over at neary-consulting.com, MJ Ray suggested that asking individuals to respond to a request can backfire:

Publicly referring to individuals on a mailing list is a double-edged sword. It might bolster the confidence of the named individual, but it also reduces the confidence of other people who might have answered the question. In general, I feel it’s best not to personalise comments on-list. Some e-democracy groups require all messages to be addressed to a (fictional or powerless) chair or editor, similar to the letters pages of The Times.

While I agree with MJ in situations where the answer is accessible to the wider community, but often only developers working for you, the manager, are in a position to reply – at that point, you have a choice: get the information off your developer and answer yourself, or ask him to answer the question. and I’ve found that asking on the list has the positive side-effects I mentioned.

Curing “Shy Developer Syndrome”

community, freesoftware, maemo, marketing, work 7 Comments

From the Neary Consulting blog:

One of the most common issues I have seen with experienced professional software developers who start to work on community software is a reluctance to engage with public communication channels like mailing lists. Understanding the reasons why, and helping your developers overcome their timidity, is key to creating a successful and fruitful relationship with the community you are working with.

In my experience, common reasons for this timidity are a lack of confidence in written English skills, or technical skills, nervousness related to public peer review, and seeing community interaction as “communication” or “marketing” (which are not part of their job), rather than just “getting stuff done” (which, of course, is part of their job).

Read more…

A modest proposal re. Unity

community, freesoftware, gnome 59 Comments

Having slept on it since writing my initial reactions yesterday I now have a proposal for Canonical & GNOME, which I hope the people concerned will consider.

Yesterday, I said “the best possible outcome I can see is that one of the two projects will become an obvious choice within a year or so”. So my proposal is this: let’s have a bake-off, Unity vs GNOME Shell, under the big tent of the GNOME project.

What needs to happen? Unity would have to agree to sync to the GNOME release schedule. Canonical will need to drop their copyright assignment requirement for Unity, and should ideally commit to using some of GNOME’s infrastructure. How much will need to be discussed. I’m sure that the Unity developers will want to continue to use Launchpad for bug tracking and bzr for source control, but perhaps the development mailing list could move to gnome.org, and the Unity website could be gnome.org/projects/unity or unity.gnome.org instead of unity.ubuntu.com?

GNOME will have to accept Launchpad as a platform for the development of GNOME software – there are potential integration issues, it is a headache using Launchpad & Bugzilla, co-ordinating Rosetta & upstream translation teams, and so on. But right now there is a general feeling that gnome.org is for “official” GNOME software, and Launchpad is for Ubuntu. We need to change that perception if we hope to be inclusive of Canonical and the greater Ubuntu developer community in GNOME. In fact, broadening the definition of what we call GNOME software was a key plank in the release team platform for 3.0 – resolving this question (and the equivalent question for projects hosted on Google Code and Sourceforge) will go a long way to growing the big tent. The GNOME project should also work to make it easy to switch from one shell to the other.

Developers who feel drawn to one philosophy or the other should work to make sure that their vision is the best it can be by September 2011. And at that point, presumably at the Desktop Summit in Berlin, GNOME, as a project, should choose one of the two, and put our full weight behind it.

This is potentially naïve on my part. Over the years, we have allowed a lot of animosity to build up between Canonical and Red Hat, among others. As a community, we’ve stood passively by while this has happened. Some will point to efforts to engage which were rebuffed. Others will point to a lack of real commitment to engage.

In situations like this, no-one is 100% right, no-one is 100% wrong. All we can do is look at the current situation, and ask ourselves: how do we get to where we want to go, from where we are? We have two choices – we can, like the Mayoman asked for directions to Galway by tourists, respond “If I was going to Galway I wouldn’t start from here at all”, or we can roll up our sleeves and try to make things a little better.

So – how about it? Is this a non-starter, or is it worth starting some conversations about it?

Oracle, OpenOffice.org, LibreOffice

community, freesoftware 22 Comments

There has been a lot of commentary in recent days about the OpenOffice.org community council decision to ask people who have aligned themselves with The Document Foundation (TDF) to resign their seats on the council. So, of course, what we need is a little bit more commentary.

First, when reading the minutes, it’s worth noting that this was not a voted decision. At 21:50, Louis Suarez-Potts proposed “that the TDF members of the CC consider the points those of us who have not joined TDF have made about conflict of interest and confusion [and] resign their offices, so as to remove the apparent conflict of interest their current representational roles produce”. He then proposed a deadline of Tuesday “to deal with this” – by emergency meeting of the council. So there was no decision to expel anyone, Louis made a proposal which did not obtain a consensus decision. That said, reading the minutes, there is clear alignment between supporters of TDF on one side and the rest of the council on the other side. And “the rest of the council” is Louis Suarez-Potts, Andreas Bartel, Eike Rathke, Juergen Schmidt, Matthias Huetsch and Martin Hollmichel on behalf of Stefan Taxhet – all Oracle employees.

Second, let’s put this in perspective. Louis is the OpenOffice.org community manager, and has been for several years. Andreas, Eike, Juergen, Matthias and Stefan are all former Sun employees, and as best as I can tell, also former StarDivision employees. So this is hardly an Oracle corporate decree. It is a group of long-time contributors to OpenOffice.org taking a position on the community council vis à vis LibreOffice. I have been telling people for years that corporations don’t get to be members of free software communities – although their employees might. Let’s apply that same standard when commenting on actions by individuals, and assume less cloak-and-dagger. “Oracle” have not decided anything here.

That said, looking at the council structure, it’s clear that it has been set up so as to make it very difficult for Oracle/Sun employees to ever find themselves in a minority on the council, if they do decide to act as a block.

Third, let’s look at the substance of the discussion itself. The Document Foundation defines itself as “an independent self-governing meritocratic Foundation, created by leading members of the OpenOffice.org Community”. Implicit in this is a criticism of OpenOffice.org: the project is not independent, self-governing or meritocratic. Calls for OpenOffice.org to be managed by an independent not-for-profit are not new – in fact, I was one of those calling for it over the past few years. So clearly, the creation of The Document Foundation is a reaction to perceived and actual dysfunction in the operation of OpenOffice.org. In fact, in its initial communications, The Document Foundation clearly stated that the name LibreOffice was provisional, and that they hoped that Oracle would allow the foundation to use the traditional OpenOffice.org name – in other words, TDF wants LibreOffice to supersede OpenOffice.org as a project.

How, then, can people who have aligned themselves with the new initiative keep a straight face and say they are not trying to undermine OpenOffice.org? Traditional contributors to OpenOffice.org are now faced with a choice: will I update my source code daily from OOo, or from LibreOffice? I have met project volunteers who had no prior knowledge of the split who are now torn between two groups. They feel like they have to decide. For the instigators of the revolution to pretend otherwise is disingenuous.

I’ve been told that I’m conflating “OpenOffice.org” the project, and the OpenOffice.org community council, which was created to represent “the OpenOffice.org community” – and that all of the non-Oracle community representatives feel it’s in the best interests of the OOo community to align with TDF, and as such it is Oracle representatives on the council who are in conflict of interest with what is best for the community they are representing. That may be the case, I’ll leave you to decide. It seems to me that supporting LibreOffice implies withdrawing support for OpenOffice.org, and thus forfeiting any mandate you might have to represent the OpenOffice.org community. Individual community members will surely make up their own mind.

A fork is like a schoolyard football game. Tommy brings a ball to school, and gets to be captain of a team every time, picks the best players for himself, because if he doesn’t, he threatens to take his ball away. After a few weeks of this, the other players get annoyed, and everyone chips in to buy a ball for the group. At this point, Tommy has lost all power – he still has his ball, but he’s not team captain any more. He gets left standing against the wall till the very end to teach him a lesson in humility. So he sits out the games for a while in protest, starts a competing game at the other end of the yard with his ball and his rules. All the best players go to the other game, though, and some of the kids make fun of “Tommy’s” game – invitations to give up his game & join the new one seem more like taunts than honest efforts to include him. After a while, he realises that playing football with others is more fun, and more important, than playing with his football.

I welcome the creation of The Document Foundation. I believe that it will be good for office software on the free desktop. I would prefer supporters of TDF to be dignified and forthright during the split, and to come straight out and say “we think TDF offers a better future for OOo than OOo does – so come join us”. Let the dust settle, and maybe in 6 months or a year, when the functionality gap between LO and OOo has widened a little, propose once more that Oracle and IBM engage with the foundation. At that point, such a proposal will seem like an honest effort at reconciliation. Right now, it feels more like rubbing Tommy’s nose in it.

Updated to reflect feedback, 11:30, 2010-10-21

GNOME Census report released

community, freesoftware, gnome, guadec, work 8 Comments

I was delighted to see that the GNOME Census presentation I gave yesterday at GUADEC has gotten a lot of attention. And I’m pleased to announce a change of plan from what I presented yesterday: The report is now available under a Creative Commons license.

Why the change of heart? My intention was never to make a fortune with the report, my main priority was covering my costs and time spent. And after 24 hours, I’ve achieved that. I have had several press requests for the full report, and requests from clients to be allowed to use the report both with press and with their clients.

This solution is the best for all involved, I think – I have covered my costs, the community (and everyone else) gets their hands on the report with analysis as soon as possible, and my clients are happy to have the report available under a license which allows them to use it freely.

You can download the full report now for free.

Rotten to the (Open) Core?

freesoftware, General 18 Comments

Open core, Open core,  more Open core… the debate goes on and on, with Monty the latest to weigh in.

When you get down to it this is a fight over branding – which is why the issue is so important to the OSI folks (who are all about the brand). I don’t actually care that much how SugarCRM, Jahia, Alfresco et al make the software they sell to their customers. As a customer I’m asking a whole different set of questions to “is this product open source?” I want to know how good the service and support is, how good the product is, and above all, does it solve the problem I have at a price point I’m comfortable with. The license doesn’t enter into consideration.

So if that’s the case (and I believe it is), why the fighting? Because of the Open Source brand, and all the warm-and-fuzzies that procures. “Open solutions” are the flavour of the decade, and as a small ISV building a global brand, being known as Open Source is a positive marketing attribute. The only problem is that the warm-and-fuzzies implied by Open source – freedom to change supplier or improve the software, freedom to try the software before purchasing, the existence of a diverse community of people with knowledge, skills and willingness to help a user in difficulty – don’s exist in the Open Core world. The problem is that for the most part, the Open Core which you can obtain under the OSI-approved license is not that useful.

Yesterday on Twitter, I said “Open Core is annoying because the “open core” bit is pretty much useless. It doesn’t do exactly what it says on the tin.”

Now, I wasn’t expecting this to be particularly controversial, but I got some push-back on this. Dan Fabulich replied “Ridiculous. Like the free version of MySQL is useless?” Which leads me to think of Inigo Montoya on the top of the Cliffs on MoherInsanity turning to Vizzini and saying “You keep using that word. I do not think it means what you think it means.”

With all this talk of Open Core, clearly some confusion has crept in. Perhaps it’s on my part. So allow me to elaborate what I understand by “Open Core”.

First, companies can’t be Open Core. Products are Open Core. So whereas Monty considers that from 2006 on, MySQL was not an “Open Source company”, I would contend that MySQL Server has always been, and continues to be, Free Software, and an Open Source product. That is, not Open Core.

Open Core for me means you provide a free software product, improve it, and don’t release the improvements under the free software licence. In my mind, Mac OS X is not “Open Core” just because it’s based on the NetBSD kernel, it is proprietary software.

Perhaps it would be useful to give some examples of what is Open Core:

  • Jahia is Open Core – significant features and stabilisation work are present in the Enterprise Edition are not available at all in the Community Edition
  • SugarCRM is obviously Open Core. Key features related to reporting, workflow, administration and more are only present in the commercial editions
  • JasperSoft BI Suite is Open Core. Lots of useful features are only available to people buying the product.

The key here is that support contracts and extra features are only available if you also pay licensing fees. To take the oft-cited example of InnoDB hot back-up tool for MySQL, you can purchase this and use it with the GPL licensed MySQL Server.

This is why I say that Open Core products “don’t do exactly what it says on the tin” – the features you see advertised on the project’s website are not available to you along with software freedom.

I have talked to companies who deliberately avoid adding “spit & polish” to the community edition to encourage people to trade up for things like better documentation, attractive templates and easy installation – and don’t provide an easy way for the community edition users to share their own work. Other products have an open source engine that doesn’t do much except sit there, and all useful functionality is available as paid modules. Yes, a persistent, skilled, patient developer can take the Open Source version of the product and make it do something useful. For the most part, however, if you want to actually use the software without becoming an expert in its internals, you’ll need some of the commercial upgrades.

There is another name for this which is even more pejorative, Crippleware. Deliberately hobbled software. And that’s what I think gets people riled up – if you’re releasing something as free software, then there should at least be the pretence that you are giving the community the opportunity to fend for itself – even if that is by providing an “unofficial” git tree where the community can code up GPL features competing with your commercial offering, or a nice forum for people to share templates, themes and extensions and fend for themselves. But what gets people riled is hearing a company call themselves “an Open Source company” when most of the users of their “open source” product do not have software freedom. It’s disingenuous, and it is indeed brand dilution.

That said, let me repeat – I have no problem with companies doing this. I have no problem with them advertising their GPL-licensed stuff as Open Source. I would just like to see more of these companies providing a little bit of independence and autonomy to their user community. But then, that’s potentially not in their long-term interest – even if it is difficult to imagine a situation where the community-maintained version outstrips the “Enterprise” edition in features and stability.

GNOME Training confirmed!

community, freesoftware, gnome, maemo, work 4 Comments

A few days ago, I took the risk of setting off alarm bells on the GNOME developer training sessions planned for GUADEC this year. It was a risk, and comments from the naysayers reminded me that it’s easier to do nothing than it is to take a risk. I’m happy to say that the risk paid off.

Thanks to all who spread the word, a couple of prospects I was aware of confirmed their presence on the course, and I received a new group booking. The training is now feasible, and we are confirming that it will happen. There is still room on the course, and I expect to sell a few more spots in the coming days.

I did get one interesting suggestion in a Twitter reply to the announcement, and I’ve adopted it. If you are interested in attending one or two of the modules (say, community processes and the GNOME platform overview, but not the practical session or Linux developer tools), you can do so for the much lower price of €400 per module and €750 for two modules, not including a GUADEC registration.

Anyone who would like to avail of this offer, please contact me, and we will take care of getting you signed up.

Thank you all for your help and support!

http://blogs.gnome.org/bolsh/2010/06/28/gnome-developer-training-in-danger/

« Previous Entries Next Entries »