What's behind the ascendency of enterprise open source?

With open source filling mission-critical niches everywhere, we look at the drivers

Microsoft's abrupt volte face to embrace open source a few years ago met with a wave of cynicism from those who reflexively typed the company's name as ‘M$' and referred disparagingly to its best-known product as ‘Windoze'.

You don't see much of that ‘Great Satan' talk anymore, even in the most partisan of Linux forums, although there remains a deal of suspicion about the motives of Microsoft and other tech giants as they take up key positions in the Linux Foundation and other open source industry collaborations. Be that as it may, most now see Redmond's move as a canny one, a recognition of the way the wind was blowing and a determination not to be caught on the hop.

From its back-bedroom and academic roots, open source is now a serious enterprise option and is found everywhere from mobile phone operating systems to banking internals in the gleaming skyscrapers of the City. Of course, not all open source is born equal, and some source code is definitely more open than others (more about that later), but being able to adopt and adapt other people's work and to share the results has undoubtedly lit a fire under technological innovation.

Open is on the up

In a recent study from Red Hat, a total of 69 per cent of IT professionals and influencers rated open source software as ‘important' or ‘extremely important' (four or five on a five-point scale) to their organisations, with just one per cent saying it is not at all important.

A significant proportion of cloud, big data and web technologies (containers, Hadoop, streaming, etc) are open source, as are most of the newer databases. The range of applications for which there is a viable open-source option is growing all the time. Organisations don't necessarily select it because it's cheaper (although the fact that free versions are almost always available certainly helps with adoption at the early stages). Instead they choose it for its quality, the fact they can contribute to its development in order to meet their own needs, because it fills gaps in coverage by proprietary solutions - plus, of course, an active community means that patches, fixes and updates should be issued more promptly.

Facebook alone open-sourced 158 projects last year

Programs that would once have been guarded jealously behind closed doors are now routinely open sourced with the aim of attracting developers in order to cut their time to market (think Kubernetes which started life in Google, Apache Kafka which was an internal project at LinkedIn, and React from Facebook; Facebook alone open-sourced 158 projects last year).

What's behind the rise?

There are many trends underlying the ascendency of open source. One is the increased use of cloud, both as a way of working and as a distribution platform. Pre-GitHub, most projects were in-house efforts - maybe that's still the case, but the easy availability of code on cloud repositories has certainly changed the game. It has also enabled players from countries like China to start playing a much bigger role. Anyone attending a software summit in the last couple of years could not have failed to notice the vastly larger presence of Chinese sponsors, media and developers.

"It's a very positive thing," said Alan Clark, chairman of the OpenStack Foundation and CTO at Suse. "You get more contributors which means you get more done. But they also come in with different perspectives, different markets and different requirements, and that helps improve the software."

Another trend is the bringing together of multiple open-source components to build platforms that span the various deployment models. Suse is a case in point, creating modular versions of its Linux distribution to run everywhere from the data centre to edge devices to public cloud, and overlaying that with other components including Open Stack to create a software defined infrastructure layer.

"It's so you can manage the different types of virtualisation technologies like containerisation and bare metal and you don't have to do that is silos, you can manage them together, which gives you higher agility, lower risk and lower cost," Clark explained, adding that the next layer up caters for DevOps and delivery - another movement driving enterprise open source.

Turn to next page

What's behind the ascendency of enterprise open source?

With open source filling mission-critical niches everywhere, we look at the drivers

Many projects revolve around small utilities, component parts rather than enterprise suites, but the latter exist too and may be downloaded from GitHub or subscribed to on AWS, Azure, Alibaba and other cloud service providers (CSPs).

And there are specialist CSPs that host only open source projects for a specific set of use cases. One such is Aiven, which provides managed services around a stack of open source databases, streaming and monitoring tools to customers including the UK Government and Toyota. Supported projects include Postgres, Cassandra, Elasticsearch and Kafka. CEO Oskari Saarenmaa said demand is being driven by the exponentially rising number of connected devices and the need to manage "the firehose of data" they produce. Frequently, open source solutions can be found where no others exist, he said, but many lack polish and are hard to use.

"We wanted to build a platform that brings the best of these technologies in front of people who are building something new and don't want to take ownership of all the systems and have to learn all the quirks and gotchas," said Saarenmaa.

Two camps

Most open source projects fall into one of two camps, according to Saarenmaa. There are the collaborations between vendors, customers and academia to create software to "solve big problems for big companies". These may make money for a couple of the vendors involved but even those firms don't own the project or have a controlling stake in its overall development. In the database world, Postgres would fall into this category.

In the other camp are projects that are largely controlled by one company which may own the core IP, an example being MongoDB.

Somewhere between these two sit companies that are built on projects that are maintained by a foundation. Confluent was set up by some of the original creators of Kafka, but Kafka is an Apache Software Foundation project. Databricks is similarly placed with Apache Spark.

Saatchmaa believes that ultimately it's the collaborative efforts that will survive and thrive. Noting that MongoDB, Redis Labs and Confluent recently licensed some non-core modules under a restrictive non-open source license to prevent what they see as predatory actions by the big cloud companies, he said this move, while completely understandable, may backfire in the long term.

"We believe that the best thing is to stay open and to be backed by many contributors. There may be something behind the claim but the truly successful projects that will be around for a long time will not be controlled by a single company."

The accusation that Amazon in particular has taken a lot without giving much back is fair, he said, but open source projects have gained by the network effect of being hosted on AWS. "[It is] giving an opportunity for people to work with different kind of companies and users of open source software that they probably would not have bumped into otherwise," said Saatchmaa.

In a recent piece for Computing, EnterpriseDB senior sales engineer Jan Karremans said that open source projects should link together in a more customer-focused way than they have in the past . " A more pragmatic response would be to acknowledge it is a trend and discuss what it means for open source models moving forward without portraying it as a scene from Game of Thrones where good battles evil," he said.

However, for Confluent, VP product marketing Gaetan Castelein described the application of more restrictive terms to some of its modules and connectors as ‘uneventful'. Relations with Apache and the Kafka developer community have not changed, he said, nor have relations with CSPs soured, and for end-users things look the same.

"You can still have access to open source, you can still modify the code, so it looks and feels as if it was open source, the only change is if you're a cloud service provider. You cannot use these products to compete directly against Confluent."

So far this had not occurred, Castelein said, and the Confluent Community Licence was really a precautionary measure.

The past few years has seen a trend for more permissive licensing, with Apache 2 and MIT winning out over the older GPL models, mostly for reasons of reduced complexity: customers want to avoid the legal wrangles that can occur by mixing differently licensed software, and this filters down to developers. This is unlikely to change and so it could be that MongoDB, Redis Labs, Confluent and others find themselves on the wrong side of the fence - even if their changes are aimed at protection from CSPs. Only lawyers like complexity, after all.

An expanded version of this article with more information on open source trends, foundations and licenses is available on our market intelligence service Delta.

Delta is a new market intelligence service from Computing to help CIOs and other IT decision makers make smarter purchasing decisions - decisions informed by the knowledge and experience of other CIOs and IT decision makers.

Delta is free from vendor sponsorship or influence of any kind, and is guided by a steering committee of well-known CIOs, such as Charles Ewen, Christina Scott, Steve Capper and Laura Meyer.

Ten crucial technology areas are already covered at launch, with more data appearing and more areas being covered every week. Sign-up here for your free trial of the Computing Delta website.