Community analysis as risk management
May 18, 2009 4:47 pm communityRecently I’ve had a number of conversations with potential clients which have reinforced someting which I have felt for some time. Companies don’t know how to evaluate the risk associated with free software projects.
First, a background assumption. Most software built in the world, by a large margin, is in-house software.
IT departments of big corporations have long procurement proceses where they evaluate the cost of adopting a piece of infrastructure or software, including a detailed risk analysis. They ask a long list of questions, including some of these.
- How much will the software cost over 5 years?
- What service package do we need?
- How much will it cost us to migrate to a competing solution?
- Is the company selling us this software going to go out of business?
- If it does, can we get the source code?
- How much will it cost us to maintain the software for the next 5 years, if we do?
- How much time & money will it cost to build an equivalent solution in-house?
There are others, but the nub of the issue is there: you want to know what the chances are that the worst will happen, and how much the scenario will cost you. Companies are very good at evaluating the risk associated with commercial software – I would not be surprised to learn that there are actuarial tables that you can apply, knowing how much a company makes, how old it is and how many employees it has which can tell you its probability of still being alive in 1, 3 and 5 years.
Companies built on free software projects are harder to gauge. Many “fauxpen source” companies have integrated “community” in their sales pitch as an argument for risk mitigation. The implicit message is: “You’re not just buying software from this small ISV, if you choose us you get this whole community too, so you’re covered if the worst happens”. At OSBC, I heard one panellist say “Open Source is the ultimate source escrow” – you don’t have to wait until the worst happens to get the code, you can get it right now, befor buying the product.
This is a nice argument indeed. But for many company-driven projects, it’s simply not the case that the community will fill the void. The risk involved in the free software solution is only slightly smaller than buying a commercial software solution.
And what of community-driven projects, like GNOME? How do you evaluate the risk there? There isn’t even a company involved.
There are a number of ways to evaluate legal risk of adopting free software – Black Duck and Fossology come to mind. But very little has been written about evaluating the community risks associated with free software adoption. This is closely associated to the community metrics work I have pointed to in the past – Pia Waugh and Randy Metcalfe’s paper is still a reference in this area, as is Siobhan O’Mahony and Joel West’s paper “The Role of Participation Architecture in Growing Sponsored Open Source Communities”.
This is a topic that I have been working on for a while now in fits and starts – but I think the time has come to get the basic ideas I use to evaluate risk associated with free software projects down on paper and out in the ether. In the meantime, what kinds of criteria do my 3 readers think I should be keeping in mind when thinking about this issue? Do you have references to related work that I might not have heard about yet?
May 18th, 2009 at 11:42 pm
I totally agree that companies are very concerned with measuring the risk of open source. I know one company that has done a study of how much it would cost them if they got sued for open source (win or lose) and keeps that much money around, just in case.
OpenLogic has an open source evaluation process that includes measuring community. They measure number of members, number of open bugs, how long bugs are open, # of mails on the mailing list, etc.
May 22nd, 2009 at 12:30 am
Looking at the worst case is the wrong approach, you should be looking at the expectation value. Sum( (Chance of thing happening)*(Cost of thing happening) ) across all things. Same technique you can use to figure out how much money you lose by buying a lottery ticket.
May 22nd, 2009 at 12:34 am
To add onto that, you can then decide how much risk you want to take by just dropping the lowest-probability events off your list till you’ve dropped the bottom 1%, 5%, or however much you want to assume won’t happen. Since low-probability events are often also high-cost, this is one way of deciding how much money to save.