Why Yahoo should spin out Hadoop team – but not how you think
Recent commentary in the Wall Street Journal (left out of the link out of irritation at the WSJ paywall) and ReadWriteWeb (http://www.readwriteweb.com/cloud/2011/04/yahoo-weighs-spinning-out-hado.php) suggests that Yahoo is in discussions with Benchmark Capital to spin out their Hadoop development team and form a new company.
In my opinion, this is a great idea. Why?
- I think there’s a big market there. A quick look on indeed.com shows over 3000 jobs posted related to Hadoop. Big data analytics is a large and growing space, and Hadoop has established a clear leadership position there.
- I think there is zero chance of Yahoo capturing this value as an internal project. Yahoo has some great properties but it has enough challenges in managing its media business. Running two different businesses (media and software) is very hard, and Yahoo would be ill-advised to try.
- Spin it out on its own, with funding from a top-teir VC. Rumor has it that discussions with Benchmark are in progress.
- Join forces with Cloudera; figure out a fair value for the assets being spun out and work with the Cloudera team to share ownership based on that valuation.
There are fewer moving parts with the standalone spinout so its easier to do, but I’m not convinced its the right thing. I just don’t think two rival firms commercializing one open source project will work well, and I’m afraid both would have the critical mass (and cash) to last a while, which would create a rift in the Hadoop world just as it’s picking up momentum.
Here’s to hoping (however unlikely) that Yahoo, Cloudera, and their VC’s can find a way to create one great company that can help Hadoop realize its full potential.
Thoughts?
— Max
6/27 addendum: looks like this happened, gigaom is now reporting the spinout will go forward under the name HortonWorks (ugh).
Max Schireson's blog 
I read that article yesterday and thought it was darn interesting. Partially this is because I work at Oracle and stuff like this has a chance of landing here and adding to the list of organizations I have to support. Partially because I live near Red Hat and anything in their arena tickles my antenna.
I don’t quite understand the end game for a lot of these companies. Or maybe I do and the gamble is too bald. I’m old enough to remember a market for a half dozen *very* good text editors for programmers, competing desktop/groupware database tools, middleware, etc, etc. There was a steady convergence to a much much smaller group of *things* that power daily tech technology.
So now we have this explosion of tools and infrastructure bits, especially in the NoSQL and Cloud space. So my monkey mind thinks we’re going to have a brutal consolidation – some really big winners and a lot of loser and zombies littering the landscape.
If that’s true, what is the right move for Cloudera/Ya-Hadoop?
-XC
PS – I have an unparalleled record of bad M&A and stock picking, so take that into account.
PPS – My comments are my own, not Oracle’s, etc.
I agree, there are many more companies in the data management space right now than are sustainable; consolidation is coming. That’s part of why I think it should be spun out and combined with Cloudera rather than spun out separately.
Thanks as always for the thoughts,
— Max