Google and Darwinian Software Evolution

Darwinian Software. Its a fact. Google rocked the “Geek-eratti” last week announcing that they had scuttled MapReduce in favor of Dataflow. To any regular corporate Joe this does not really mean anything.  But if you happen to be talking to “that guy” at a cocktail party bla, bla bla-ing about Big Data here’s the real take-away.

Google torpedoed one of the original pillars of big data processing in favor of something better.  Countless companies have been sold on the merits of MapReduce. And to a certain extent it’s kind of like getting the rug pulled out from underneath you, that the future is no longer really the future.

What’s really important, however, is what it signifies. The software world has taken a pretty big evolutionary leap in the last decade and this is nothing short of a sea change.  When you can make really big fundamental changes to improve how you deliver software, and no user knows the difference, something has really changed.

If you work at a company there is a 99.9% chance you deal with some old software that makes you want to pull your hair out. So how come Google can hot swap out a key pillar of their software and your software vendor can’t even respond to simple requests without a fist full of dollars and months and months of anguish.

The answer isn’t, “It’s Google”.  The answer is, “Its the architectural design”.  New development methodologies and technology is out there and open to anyone willing to embrace it. In the two years we have had K3 on the market, we’ve overhauled our fundamentals twice to make it better. If your software is on the wrong side of software Darwinian evolution, it’s time to encourage your vendor to get on the right side of software evolution or go extinct.