Artificial Intelligence: the answer to Big Data skills shortages?

Satya Ramaswamy, vice president and global head of TCS Digital Enterprise, discusses the shortage of data scientists, and argues that AI should step in to analyse unstructured data streams

Big Data - it's the enterprise term of the moment. Businesses of all sizes and from a wide variety of industries are coming to understand the benefits of utilising it to streamline operations and ultimately gain competitive advantage over competitors. The Emerging Big Returns on Big Data, a global trend study conducted by TCS earlier this year, found that over half of the businesses surveyed already had a Big Data programme in place and those that did not were aiming to implement one by 2015.

The benefits of Big Data are well known but I believe the current debate is too narrowly focussed on the short-term, and is only the beginning of what Big Data can achieve. The technology is currently being used to solve problems that enterprise has been experiencing for some time - speed, storage and cost - but the real power of the advances made go far beyond this. The future of Big Data is one of powerful Artificial Intelligence that will enable businesses to analyse beyond three-dimension and gain a deeper understanding of developing trends. Big Data will not only make business operations faster and cheaper, but smarter - beyond human capabilities.

The popularity of Big Data developed over the past two to three years as the economic climate encouraged businesses to optimise processes to reduce costs and increase revenue. This rapidly increased the Big Data analytics space and placed a high demand on the small pool of people with Big Data skills capable of helping businesses achieve the desired advantage. The result is that Big Data in the enterprise space has advanced faster than the development of "data scientists" to manage it, thereby creating a major skills shortage.

This has led to calls from within the industry to encourage school children to get in involved in STEM (science, technology, engineering and mathematics) disciplines to hopefully increase the numbers studying these subjects at university and choosing the sector as a career. However, I believe it is impractical to assume we are able create enough data scientists to manage Big Data, nor should we want to do so.

Increased machine-learning can offer great benefits and is critical to realising the true benefits of Big Data. As humans, we can only analyse data in one, two or three-dimensions, but advancing towards artificial intelligence driven by cognitive and computational intelligence will foster a deeper understanding of the raw data that all businesses produce. We are already on the path to realising this.

[Turn to next page]

Artificial Intelligence: the answer to Big Data skills shortages?

Satya Ramaswamy, vice president and global head of TCS Digital Enterprise, discusses the shortage of data scientists, and argues that AI should step in to analyse unstructured data streams

The technological advances already achieved by implementing Big Data initiatives could deliver human-like intelligence and interaction capabilities through neocortex-inspired algorithms such as Deep Learning and Sparse Data representation. In plain terms, businesses will be able to analyse greater amounts of data in greater depth to find trends that are not apparent through human analysis alone. Artificial Intelligence can help businesses see changes or patterns much, much faster which will in turn result in more efficient, cost effective businesses.

This does not mean that the education of a future generation of data scientists is not important. In fact, the opposite is true. TCS has partnerships with hundreds of schools across the world to encourage as many young people as possible to embrace the sciences and enter the IT sector. But the idea that we need an army of data scientists to process the huge amounts of information at our disposal is misguided. Rather, the key to leveraging Big Data to its full capacity is machines and data scientists to working in unison - machines to do the bulk of the automation and deeper analysis, and data scientists to translate what the smaller set of outcomes means in real business terms.

We've only just begun to break the surface of Big Data. So instead of fixating on the shortage of data scientists and bursting the Big Data bubble before it has begun, we as an industry should be looking forward and embracing machine-technology as part of the skills solution.