Beating physics - latency doesn't have to be an issue

One issue that needs to be addressed when implementing or procuring a hosted service is latency – the time it takes for a response to come back from the provider’s data centre after an action is taken out on an end user device. In the majority of cases, even a good connection between the access device and the hoster’s data centre will result in a round trip latency of around one third to one half second.

Geography also plays a part – the speed of light will always be a limiting factor, and as such, a data centre in America will have higher built-in latencies than one down the road in London for a UK user. While this does not sound like much, if you had to wait half a second for a response on every keystroke, you would soon give up and find a different way of doing things. Standard client/server systems involve a lot of “chattiness” between the client and the server – and deployments where the core application (server) is in a hosted data centre that purely replicate such a model will not work effectively.

OK – client/server computing is meant to be on its last legs as everyone moves to newer architectures and web-based applications. The trouble here is that the majority of web-based applications are not truly web-based – they use browser plug-ins or application accelerators embedded into the browser or other components that need to be installed at the client. This really is just another form of client/server – and when the server is hosted, can still lead to the issues outlined above.

However, having the right architecture can not only reduce the issue of latency, but can often give a better experience than would be the case if the application was hosted inside a private data centre.

The key here is to ensure that the main work is carried out in the hosted data centre – not at the client or between the client and the data centre. This involves the use of a three-tier architecture, using virtual desktops. Here, the main business logic still takes place on the server itself (whether this is virtualised or not). The client logic takes place on a virtual desktop that sits within the same datacentre, talking to the servers at data centre speed – far faster that would be obtained in a standard client/server model within an in-house environment. All that is sent back to the client (which could be a PC, tablet or smartphone) is the changes to the visual client – a small amount of data that even high latency, low bandwidth connections can generally deal with pretty easily.

This still leaves issues around areas such as audio and video. However, newer codecs brought through as voice over IP (VoIP) has increased in usage means that audio is now a very low bandwidth stream – a single VoIP channel will be less than 100kbs for a high quality call. Video is getting better too, but even within a corporate environment can be a bandwidth hog that can slow everything down across a LAN connection. However, using quality of service (QoS) tagging through 802.1p/q and multi-protocol tagging services (MPLS), specific audio and video streams can be given high priorities as required to ensure better performance between the data centre and the client.

In some cases, the data streams can also be manipulated to provide better performance. For example “packet shaping” can be used to pass data in fewer, larger packets; data compression can be utilised to decrease the amount of data that needs to move between the data centre and the client. In advanced cases, software can be applied to the client in a once-only manner that can talk to wide area network (WAN) acceleration equipment housed in the service provider’s data centre to provide data caching and deduplication as well as dealing with the data “chatter” inherent to the majority of modern applications, so that only a very small proportion of expected traffic actually needs to traverse the connection.

When just looking at an external data centre provided by an external co-location provider, it will be down to the customer to implement any of the above. However, when procuring services from a platform, infrastructure or software as a service (P/I/SaaS) vendor, it is down to them to ensure that they implement an architecture that minimises latency and data chatter outside of their managed environment. For the buyer, being aware of the areas that could cause issues is the main part of the battle – and being able to ask the right questions can ensure that the best provider is chosen. Many such providers will offer parts of an advanced low-latency service as basic and value-add services, and may be able to offer specific services – such as WAN acceleration – as a customer-specific option.

Latency is perceived as many as a killer blow to the use of hosted or external cloud systems. However, by ensuring that a suitable application architecture is in place, latency will not be an issue – and overall performance and end user experience will be much improved.

Clive Longbottom

Service Director, Business Process Analysis, Quocirca Ltd

(Originally posted at Lunacloud Compute & Storage Blog)