Will 2015 be the 'year of re-platforming' as firms move wholesale to NoSQL and Hadoop?

Couchbase CEO Bob Wiederhold says the stars are aligned for big changes in enterprise data platforms this year

New technologies flower only when external conditions are favourable. Couchbase CEO Bob Wiederhold (pictured) believes that for NoSQL databases spring has arrived early, and that 2015 will be the year that NoSQL leaves the labs and test benches and emerges as a mainstream technology, capable of supporting heavyweight enterprise applications reliably and consistently on a large scale.

"We think that in the second half of 2015 we're going to see a significantly increasing number of enterprises start a broad re-platforming," he said.

"Our biggest customers are starting to think about how and when they are going to do that."

By "re-platforming" he means moving away from the relational systems that have for years supported enterprise applications, meaning SQL databases such as Oracle, Microsoft SQL Server, MySQL and the dedicated hardware on which they run, and onto the new solutions that in many cases are able to do the job as well or better on cheaper commodity hardware.

Customers, Wiederhold said, have spent the past couple of years experimenting with new platforms and have come back with a number of demands.

"They're telling us, 'you need to have these 12 or 14 features before we can [deploy NoSQL to support mission-critical applications], you need to be able to live up to this SLA on a more consistent basis, you need to have the right partnerships, perhaps with a systems integrator, in place'," he said.

Other important pre-requisites include better auditing and security capabilities and, importantly, improved ease of use from an administrative perspective, a factor that is sometimes overlooked when a new solution moves on from the realm of the enthusiastic amateur to to corporate mainstream.

"When you get the experts deploying new technology for the first time then you can deal with a node taking 10 minutes to come up after failure, but when you're deploying more broadly, supporting more applications and having a much larger number of people deal with it, it needs to be more like 30 seconds. You have to raise the bar. So - simpler to use, more features and handling all kinds of different failure scenarios gracefully."

Wiederhold is confident that these capabilities will be in place within the next 12 months, not just in Couchbase's solution but also for the other NoSQL "leaders", who he names as MongoDB and DataStax Cassandra. However, for the less prominent players he predicts a bleak new year as they struggle to raise the finance required to catch up with the leaders.

"What happens in all rapidly growing industries is what's happening in the NoSQL space. The leaders are separating themselves from the rest of the pack. It's going to be increasingly hard for any of the smaller players to catch up... many are going to go out of business or they'll be acquired for relatively low valuations."

So how is Couchbase gearing up for the big time? The first consideration is ensuring 24/7 support by setting up development and helpdesk centres in the US, UK and India. Second is forming relationships with the channel.

"We're rapidly moving from the early adopter to the early majority. The early majority is going to use the services and systems integrators much more heavily than the early adopters do, so it's a real focus of ours to begin signing partnerships with the systems integrators. The first one that we're ready to announce is Wipro."

NoSQL security features have always lagged those found in mature enterprise SQL databases such as Oracle, and this is something Wiederhold is looking to rectify, in part through a partnership with data security firm Vormetric, which was announced in October.

"Security is a very high priority for us - we're investing very heavily. It's become a very high priority for our customers, especially as they move beyond the first few mission-critical applications."

Then there is broadening the number of viable use cases, for example making the transition from SQL easier via the N1QL query language, and tweaking the architecture to provide automatically tunable memory so that the amount of data stored in memory is optimised according to the job at hand. These things will allow Couchbase to extend its reach from operational workloads into more analytical use cases, Wiederhold claims.

The broad changes Wiederhold predicts are not restricted to NoSQL. He believes a number of related technologies will take off at around the same time.

"The re-platforming is not just NoSQL it's all sorts of things. It's Hadoop, it's text-based search, its open source BI tools - it's a much bigger movement then just NoSQL by itself," he said.

Matt Aslett, research director data platforms and analytics at analyst firm 451 Research, agreed that the market is seeing some movement but said the change will be slower and more limited than the new players and their cheerleaders are predicting.

"We do see early adopters beginning to engage with emerging (NoSQL, NewSQL and Hadoop) vendors as they consider their options for creating next-generation data platforms. We expect these next-generation platforms to be based around tenets - multi-tenant, multi-model, multi-datacentre, hybrid, agile, elastic, distributed, automated and delivered as a service - that are far removed from those used to describe existing data platforms," he said.

"The key questions are how fast will the shift to these new platforms occur, and will the incumbent vendors have enough time to develop and acquire the functionality required to protect their dominant positions?"

The inbuilt conservatism of enterprise IT will see the majority muddling through with what they have now for the foreseeable future, leaving a minority making the sort of changes Wiederhold describes. However, as the landscape changes, the incumbents will find it harder to maintain the status quo.

"The multiple tenets driving those next-generation data platform concepts are so many and so varied that ‘good enough' will not be, well, good enough," Aslett said, predicting major consolidations in the near future.

"We are potentially talking, after all, about a fundamental refactoring of the data management estate. As such, it seems likely that none of the existing vendors, established or otherwise, has the wherewithal to address it. Expect consolidation among the emerging start-ups as vendors look to combine the capabilities required."