A few weeks ago, Matt Griffiths wrote a response to my blog post Yahoo! Working on Open Source GFS Clone? entitled Why would Yahoo support an open source version of the Google File System? where he wrote

So why would Yahoo! do this? Why would they create open source versions of tools that could give them a short-term competitive advantage? I think Joel Spolsky said it best:

Smart companies try to commoditize their products' complements. [Joel Spolsky]

Yahoo! is not in the business of selling software. They sell advertising. Software is one of their biggest complements. The best way to commoditize software is to open source it.

I disagree that software is a complement to what Yahoo! sells which is advertising real estate on their online services such as search, email and news sites. Software is more of an input into the production process (i.e. a means to the end) than a complement of the finished product.

A complement is something that is purchased along with your product to make it useful. If you are a vendor of consumer operating systems, then desktop PCs are your product's complement and you want PCs to be as cheap as possible. If you are a manufacturer of gas guzzling SUVs vendor, then gasoline is your product's complement and you want gas prices to be as low as possible. If you are a software consultant, then shrink-wrap software is your product's complement and you want software licenses to be as cheap as possible. None of these examples is analogous to Yahoo! open sourcing a tool that makes it easier and cheaper for them to build online services and thus have more & better real estate to sell to advertisers.

I can see a couple of explanations for Yahoo! making this move. On the one hand, they could believe that distributed software development isn't their core competency and would like to outsource some of the harder bits by Open Sourcing it. The problem with that approach is that as the Mozilla project has shown, it may take years to get a critical mass of external developers working on the project and it would still need significant contributions from Yahoo! to stay afloat. Another justification, could be that Yahoo! realizes that even with technologies like GFS/Hadoop there is still a lot of hardware expenditure, operations expertise and infrastructure software needed to run a megascale service. These are often beyond the resources of most competitors and for the ones that do have those resources (e.g. Google) they already have similar technologies. Thus there is little to fear of a competitor using Hadoop against them. A third option could be that Hadoop may not be considered to be strategic by Yahoo! management which is why it has been allowed to be open sourced. In that case, management may either be underestimating the importance of technologies like Hadoop or may just truly think that their competitive advantage lies elsewhere.

Bah, I should probably get to work instead of engaging in idle speculation on a Thursday morning. :)