Cloud IaaS and PaaS - how do the main vendors stack up?

Customers care most about uptime, security and costs according to Computing's research

At the start of this year, Computing carried out an extensive research programme on cloud Infrastructure and Platform-as-a-Service for our new Delta market analysis tool. This included a focus group, in-depth interviews with IT leaders, and an online survey that was completed by 230 IT decision makers with experience in selecting business IaaS/PaaS services.

Vendor recognition

Cloud is increasingly dominated by a handful of giants. Amazon was the first mover and is still the market leader by most measures (although direct comparisons are difficult) but Azure is catching up, according to many industry observers, thanks to Microsoft's substantial foothold in the enterprise.

Google is also a familiar face but it is substantially smaller in terms of its platform and infrastructure customers.

In terms of vendor recognition Microsoft topped the list, followed by AWS, Google and VMware.

How well do you know the following vendors' IaaS or PaaS solutions? VMware is best known for its virtualisation innovations. Its first venture into public cloud, vSphere Air, was not successful and its latest tie in with AWS is less than two years old, so it seems likely its brand recognition is in part based on its ubiquity in the data centre.

Similarly, Salesforce is much better known for its SaaS CRM offering than its PaaS, so we would ascribe a proportion of its visibility to this.

A hosting provider of long standing, Rackspace has had to change its business model to accommodate cloud and now seeks to straddle the main vendors as a trusted party in multi-cloud and hybrid cloud scenarios.

Meanwhile Oracle, IBM and SAP are veterans of on-premises enterprise software and infrastructure each with their own spin on the public cloud.

These figures translated quite closely into the number of respondents who had trialled IaaS/PaaS services by these vendors.

Which of these vendors IaaS/ PaaS offerings have you trialled and which did you adopt?

This finding is interesting for the fact that while Azure was well ahead in terms of trials and adoption, AWS is larger than Microsoft's IaaS/PaaS service by most measures. Previous Computing surveys have also put Microsoft ahead of AWS quite consistently, suggesting either that AWS has a smaller presence in the UK, that the audience polled was made up disproportionately of ‘Microsoft houses' or that O365 (SaaS) is frequently conflated with Azure (IaaS/PaaS), whereas in fact it is a separate line.

The above pecking order was preserved when survey respondents were asked about which vendors were leading the field, with Microsoft coming in ahead of AWS and then Google, VMware and IBM.

Asked about which vendors are losing ground, the most picked were IBM, Oracle, Fujitsu, Opentext and HPE, in that order. Clearly, the enterprise software and consultancy sections of the industry still have some work to do in proving their credentials in the cloud age.

Key factors in choosing a vendor

The most crucial factors when assessing a cloud vendor were uptime and availability figures, security credentials (most importantly the lack of any serious breaches) and running costs, present and projected.

In general how important are these factors when looking at a vendor for IaaS / PaaS solutions? (1 = not important; 7 = very important)

Uptime and availability

Achieving greater uptime is one of the main drivers for moving services to the cloud, and the fabled always-on, responsive digital business is something that's only become possible since the arrival of the as-a-service model. But availability is still not a given. Cloud services can and do go down, and one can not necessarily trust the uptime figures published by the vendor. Nor should customers expect compensation concomitant to their losses in most cases, no matter what the SLA might say.

Nevertheless, it's true to say that most organisations would be very hard-pressed to match the capabilities of the cloud providers in terms of uptime and availability. Indeed, satisfaction averaged 5.1/7.0 for uptime and availability and, a few complaints about outages aside, most of the nine vendors included in this part of the survey (Amazon, Google, IBM, Microsoft, Oracle, Rackspace, Salesforce, SAP, VMware) scored relatively highly on this factor.

Vendors registering fewer than 20 trials were excluded from this part of the research.

Cost savings

One of the original cloud drivers was its potential for reducing costs, and certainly some companies can save a great deal by rationalising their operations in the cloud. What's more, it's an easy story to sell to the board.

However, while some may genuinely be able to cut expenditure by moving data centre operations into the cloud, this is by no means guaranteed and any savings will depend very much on the use case. It's now generally accepted that savings are likely to be insignificant in most cases.

Most companies now realise that cloud is about efficiency and flexibility rather than cost savings, and while costs are important it's more about optimising expenditure rather than the bottom line. Respondents weren't exactly ecstatic about the price of cloud services but they weren't too dispondent either. Many cloud services come with a standard pricelist and so subscription costs are much easier to calculate when compared with on-premises licenses, and can often be negotiated downewards, especially for large customers. Nevertheless, there are a few ‘gotchas'.

Please rate [the vendor's] IaaS/PaaS solutions on the following factors (1 = not at all satisfied; 7 = very satisfied)

Average satisfaction levels across nine vendors Amazon, Google, IBM, Microsoft, Oracle, Rackspace, Salesforce, SAP, VMware

Asked about their experiences with the nine leading cloud vendors (as identified above; those with fewer than 20 trialers were not included) the cost of adding extras and technical support - services that all organisations will need sooner or later - were a consistent point of contention. Cloud companies roll out vanilla services at a low price and make their money on the extras. It pays to bear in mind the likely price rises from new services or increased use of existing ones when estimating RoI. Rackspace, Salesforce and SAP were all marked down for the cost of adding additional services with the latter two also called out on the cost of support.

Costs can be significantly reduced if the demand is predictable. Substantial discounts are available for those able to plan their usage in advance. For example, committing to EC2 Reserved-Instances (RI) on AWS for a year or more can mean a 75 per cent saving over the same virtual machine spun up on demand.

Security and compliance

All nine vendors performed satisfactorally on security credentials - once a show-stopper for cloud. While we would not necessarily expect respondents to make data breaches public in a survey, the almost complete absence of any commentary, either positive or negative, on this issue during the various stages of the research process suggests that security considerations are much less of an obstacle than they once were.

Concerns over security have been replaced, to some extent, by GDPR and other regulatory requirements. In the run-up to GDPR, global cloud providers busied themselves building new data centres in Europe. Most if not all of the big providers now offer geo-restricted regions to ensure the customer's data never leaves Europe.

Despite this, complaints that cloud vendors did not adequately respond to the regulatory requirements of Europe were quite common. There was also thought to be a general lack of sector-specific expertise within the public cloud providers. Many public cloud providers span the globe, but they have had to learn to be local too. An appropriate EU/UK focus was of high importance to our respondents who were mostly UK-based, and this was an important factor when choosing a cloud partner. It was also seen as an area of weakness in most of the vendors we studied, whose economies of scale are made possible by boilerplating services.

By definition, multinational companies span many jurisdictions, and complying with local data protection, security and privacy legislation across the piece can be a challenge. Some choose a series of local providers, but this can add management complexity. The global players have responded by building in granular controls for compliance within various territories, but some respondents felt they fell short. It's not enough to simply build in tickbox controls; providers need to make sure they stay ahead of any changes in the legislation.

"Where data resides is highly important, but also having an awareness of local law is equally important," said a head of IT in the public sector.

Cloud IaaS and PaaS vendors were rated by IT professionals across a number of other key criteria, including support, commercial flexibility and innovation. Sign up for a free trial to Delta to learn more.

Delta is a new market intelligence service from Computing to help CIOs and other IT decision makers make smarter purchasing decisions - decisions informed by the knowledge and experience of other CIOs and IT decision makers.

Delta is free from vendor sponsorship or influence of any kind, and is guided by a steering committee of well-known CIOs, such as Charles Ewen, Christina Scott, Steve Capper and Laura Meyer.

Ten crucial technology areas are already covered at launch, with more data appearing and more areas being covered every week. Sign-up here for your free trial of the Computing Delta website.