Life is so good, it gets better every day

Proprietary Development Models Also Fall Flat

Okay, so I’ve switched the topic around, but computerworld’s latest article about how open source development models fall flat, is rather embarassingly wrong. In particular, Nancy [from Sun Microsystems] says that the geek creed ‘If you can’t install it, you don’t deserve to use it’ is still alive in many open source projects. However true that might be in some cases, we should look at a number of proprietary development models that have embarassingly failed in this respect – from our beloved Staroffice/OpenOffice.org to Netscape/Mozilla and many other examples. Those projects employ hordes of build engineers, with very technical, complicated build environments that the average geek has no chance of building. Perhaps it might be argued that she used ‘install’ from a user point of view, but if we expect people to contribute they need to be able to build first, then install. Fortunately there are OpenOffice.org folks who are improving this situation.

Nancy also goes on to say that the open source mantra that ‘everyone can contribute’ is actually misleading because adding to an open source project is basically limited to code, bugs and patches. I’m rather surprised with this, since Nancy has been involved in the GNOME project, where this is obviously not the case – with contributions of 70+ translations, heaps of online user and technical documentation and countless usability advice.

I think we, as corporates, could learn a lot more about open source development models. Despite their perceived lack of organization, they do seem to work very effectively, with many good practices that we could adopt.

So Who Reads These Things Anyway?

So I often wonder who reads my blog. I often wonder who reads planetgnome or who reads planetsun. So, when I read articles about Java and open source like this and this I wonder if the right people will notice and read them. James, I guess I’m looking at you right now.

Goodbye Lenin

We had a night at the cinema, and I was rather impressed with Goodbye Lenin. I didn’t know much about it before I went there, but I really liked the film. I’m not sure whether it was because the story was so intriguing or whether I could connect with the characters or what, but it just worked for me. Got back home and cooked a nice steak with some port gravy and drank a bottle of italian red wine. A real chill out day, that was so badly needed.

Do As The Kiwis Do

2 weeks ago we headed to a Crusaders game. They played The Bulls in the Super 12. Watching rugby over here is a completely different ballgame – they’ve livened it up a bunch, and really encouraged it as a family occasion. We’ve also organized some tickets to see England play the All Blacks down in Dunedin thanks to Carolyn for mid June.

I Want To Teach The World To Sing, In Perfect Harmony

On looking back over the past couple of months, it’s pretty clear that JDS has become a hugely successful product. We’ve broken previous records of mass deployments in the deal with the Chinese Government, we’ve started selling cheap PC’s with JDS pre-installed at Walmart, and everyone in Sun wants to have some part of our project. JDS is the new thing within Sun, and I have been hugely excited and motivated being a part of it.

However, despite the huge strides we’ve been making, both from a technical level, marketing and planning level, things have not run easy. One of the issues we’re facing is how we might effectively track a moving community product like GNOME. With continuing lack of resources [but who doesn’t, right?], this is one of the key things we need to learn, if only by continuing to make mistakes.

What should we feel the need to track? Arguably we could just take the GNOME sources, apply various levels of branding, bug fixing and feature development and ship it as our product. We’ve done this for CDE in the past, but without the feature development and innovation that’s needed on the desktop, that we can both leverage from the community and drive ourselves, people will soon start to turn their eyes to other more exciting projects.

GNOME is a rapidly evolving project, perhaps a little more rapidly evolving than JDS, and keeping pace with it is fraught with difficulties. With a defined, tested and proven 6 monthly release schedule we’re in a better situation than most open source projects, in that we, Sun, can plan and expect releases of the core desktop and platform.

The problem statement was the following –

” We maintain separate versions of community code based on a snapshot of a stable release. This means excessive forward porting to different community and internal branches ”

We can break this down into a number of potential development areas, some of which almost certainly overlap.

Bug Fixes

Bug fixes are generally, but not always, straight forward patches that usually go upstream as soon as they are created.

Despite the relative build up of bug fixes, most are likely to go upstream. So long as there is a mentality of pushing patches upstream as soon as possible these should be less of an issue. Red Hat by comparison only fix high priority bugs in their local stable branch. With very different QA rules to Sun, their developers have the opportunity to mark bugs fixed as ‘UPSTREAM’, meaning that the fix will go in the latest community sources and, only then, propagate down into their future releases. The priority, depending on the bug in question, is on fixing it in the community. This allows them to spend their precious resources on feature development. There are no religious reasons for doing this – it is purely practical, and it seems to work well. Could Sun be quite so practical – perhaps not, based on our current customer expectations, which may well be a good thing.

Branding

Branding of our product seems to include making trivial changes eg. menu re-ordering, default panel setup, theming, …

While these changes are relatively simplistic, with increase numbers we end up maintaining many patches that may never go upstream. If we’re not careful, these patches can create more patches. For example, moving away from the community default menu has resulted in moving away from the community documentation causing us not only shipping different versions of online user help but also having to provide patches to fix up the various help links around the desktop.

Even a simple string change, has huge consequences. GNOME has translations for almost 70 different languages, each of variable levels of support within the project. With the latest GNOME 2.6 release, there are 36 fully supported languages – coverage of 80%+. While Sun officially has support for it’s ’10 Big Language Rule’, changing just 1 string is going to break 70 – 10 languages. That’s a big hit, and we need to think carefully before making such changes.

Value Add / Features

While these could be arguably considered as ‘branding’, these are generally non-trivial changes with a large level of impact eg. Launch button, menu stripe, Nautilus Mount/Sharing.

While we certainly shouldn’t prevent innovating and adding new features to the desktop, we should carefully plan them, so that they get the right amount of project time. Too often have we created new features without any real thought into project planning and whether we have enough time in our schedule to properly finish off the implementation. As a result, we are stuck with many patches that are poorly written/planned [as a direct result of schedule], unpolished and far from ready to be taken upstream. As these numbers of patches increase we start to develop a fork of our product, and updating to new versions of the community product becomes a time consuming, cruel lesson in merging and rewriting. We have neither time nor resources for this, and if the level of patches of these type increase, we will be stuck in maintainer land rather than developer and innovation land.

Module Replacement

Module replacement is replacing a community shipped module and replacing it with another module of equal or better functionality eg. Glow and Evolution, Yelp and Java Help, GNOME Dictionary and jDictionary

This opens up another can of worms, and it’s sometimes hard to balance the argument between desktop integration and better functionality. Obviously we should strive to ship the best set of applications available to us, and that the user will benefit from. Ultimately we have to do what’s best for Sun but we such a decision should be taken very reluctantly. Not only does such a move result in a ‘fork’ but also results in more fragmentation in the Linux desktop. Is fragmentation of the desktop such an important issue? In my mind, yes, crucially. While it might be argued that as long as we keep conforming to open standards, every divergent move by Sun potentially results in a less cooperative and supportive community, with positive influence being harder to achieve. Having a community that trust you, is key to Sun’s continuing involvement.

What’s best for Sun often isn’t what is best for our users/customers and the support that we can leverage from the open source community. Achieving that balance is always going to be hard. Being stuck with some code that you are going to have to maintain, localize and document, let alone innovate, on your own is harder.

So, what are the conclusions and solutions can be drawn other than to be more careful with our planning, development and project management?

Ideally, the sensible approach is to work on community code and avoid diverging JDS as much as possible from the core desktop and platform. It is clear that we have a huge dependancy on the success of the GNOME project – without which we would not have a product [for which we owe many thanks for]. While we can continue to invest developer time into innovative, future projects like Looking Glass, we can not hope to bolster the resources for a complete, featureful and stable desktop environment like a typical open source community. ‘Innovation happens elsewhere’ has been a typical quote within Sun walls for many years. The belief is returning that Sun can indeed lead with innovative ideas, but as we start to fragment our applications and, to a lesser extent, our platform we lose out on many remarkable advances within the wider open source community. That’s a slightly harder pill to swallow, and can certainly appreciate the reluctance of such an approach.

Can we afford to see history repeat itself? We’re already seeing vendors all push for their own combinations of solutions, yet desperately wish they could all standardize to some extent. We’ve all seen and suffered greatly from the UNIX wars of the past. Have we forgotten what lessons we have learnt from this? Moreover, how often should we expect to see similar quotes such as this –

” And as long as Sun thinks they are going to beat Microsoft by replaying the Unix wars and fragmenting Linux, they are going to continue losing. Funny that all they worry about with Java is fragmentation, when GPL communities do a much better job of not fragmenting than they do. ” – Luis Villa

and more importantly, be able to come up with a honest reply to them?

How might we achieve this? With a predictable schedule for GNOME, it should easily be possible to time our development, and synchronize with the community. Like it, or lump it, the community unstable development phase is our security and our insurance for future JDS work. Unless we ensure its survival and take part in its direction, it will be hard for JDS, and GNOME, to be considered as a viable desktop alternative to Windows and, harder still, to prove that we’re innovating on the desktop. With a predictable schedule already in place, we need to synchronize the schedules at our end. Many resources are wasted by continuously changing plans mid way through a developmental cycle. We need to stick to our original plans where possible, develop the exact same fundamentals of time based releases, and pull together our product releases where it makes sense.

Why does this make sense? For a start, developing the features we want and working with community unstable development means that we have a positive influence and recognition within the community. We are developing both for our own benefit, and the benefit of the GNOME project. Both sides win, period.

With a change in development plan, we would not only be guaranteed that our features get upstream to community code, since by their nature they are developed there, but also that the maintenance of our feature patches becomes minimal. We could invest some degree of effort back porting some of these features with the knowledge that no porting will be required in the future. Not only do we align ourselves better with community development, and on fundamentally important projects like ‘Desktop Lockdown’, we benefit from exposure within the many sub groups of GNOME – everything from documentation and usability, to translation and QA. Any features that we develop in an unstable stream also benefit from ‘soak’ time, in which they are well tested, well thought out and implemented according to community coding standards.

We’re not the only ones with this issue. In fact many groups within Sun are facing the same problems on a daily basis. And not just Sun.

” I think the main reason much of XD2 didn’t make it upstream was that it was done behind closed doors without buy-in and much of it was done in a pretty hacky way (for understandable reasons). If we instead take the approach of first making the change in 2.8 in an upstream-acceptable way and then backporting to 2.6, then we know stuff won’t be lost. ” – Havoc Pennington

But is a development plan enough? It is not only about work practices. It is also about mindset. To work effectively and to get the most out of open source, you need to consider yourself a part of the community. Buy into the strategy, and if you don’t agree with that, work hard to change it. With proper rationale, there is no reason that any community member won’t feel the same way as you.

Fortunately there are people within Sun who *do* care about these types of problems and are willing to spend time to find solutions and create a new mindset, however hard that challenge might be. These aren’t exactly new and they’re not particularly earth shattering but they are the challenges that drive me and make me want to stay at Sun. Is Sun all doom and gloom right now? Hell no, far from it. We’re changing, and it is wonderful to see it.

Thanks to the bunch of people I bounced this off for review – it’s hugely been appreciated.

Mexican Night

We had a bunch of people around last night for a bit of Mexican [Paddy, Sam, Steve, Carolyn, Jo and Shane], which turned into a good night. I bought a nice box of Coronas and some lime, and with Sam bringing some Margherita mix and Sangria from the girls, it had the makings of a pretty alcoholic night. The food went down well, and we watched the Crusaders in the Super 12 match. Then headed into The Holy Grail for some more rounds, amongst the cattle market, before a final trip to KFC, walk up Manchester street and a stumble home with the 2 americans kicking every lamp post in sight in an effort to turn one off – 1 out of about 50 wasn’t a bad effort in hindsight.

Who Says It Wouldn’t Happen

I think I may have created the first vendor patch against the new file chooser. Fear me.

Numb Fingers

Our friend Joerg stopped by for a bit of time while waiting on some new parts for his bike. It’s been fun having another person in the house, and we took him out for a bit of climbing last weekend. I think he enjoyed it, but his panel beater hands weren’t quite up to crimping rock. I was having an off day, resting on two 22’s and flailing around miserably on a 25 [7a+] – it’s not that I couldn’t do the moves, I can, it’s just that I can’t seem to maintain my stamina through all the moves, especially having to clip. This week I had a cold, so that’s put me back a little, even though my body desperately needed a break from climbing for a while.

We’re having our first house party on Saturday since we moved in. It should be a fun night, although the numbers aren’t quite up to normal Irish style.

The Fine Line

So after the meeting with the GNOME Foundation and Mozilla, I got to thinking a little about Sun’s involvement with the Mozilla project. Alan pointed me to the differences between Sun’s version and the communities in this README. Not only do you get a quality product from Sun, but you also get 100+ bug fixes that aren’t in the community version. Fantastic! You get the feeling there’s a disconnect somewhere.

The Sun blogging alias is getting interesting these days, as we start talking about the infrastructure and polices for blogging throughout the company. It’s really hugely encouraging to see this type of discussion, although there seems to be a fine line forming between blogging for social interaction and blogging for a corporate strategy. I figure we need to focus on the first one.

Pain Is…

Pain is getting up for a meeting at 4am and then finding out you’re an hour early. I want to shout out some loud profanities, but I still can’t string two words together yet.