Data scientists are in-demand and well paid - so why is there a skills gap?

Meeting demand for the sexiest job of the 21st century

Our world is built on data. Each digital transaction, interaction and reaction generates and captures data, which holds within it insights on behaviours, preferences, trends and forecasts. Data scientists are responsible for unlocking these insights and extracting intelligence which influences our lives, in commerce and at home.

The Harvard Business Review called the data scientist ‘the sexiest job of the 21st century'. As problem solvers and analysts, data scientists are the professionals identifying patterns, noticing trends and making new discoveries, often working with real-time data, machine learning and AI. According to global recruitment consultants Robert Walters, data scientists "generally have a foundation in computer science, modeling, statistics, analytics, and math - coupled with a strong business sense. It's this merging of esoteric intelligence and practical knowledge that makes the data scientist so valuable to a company". These highly-skilled professionals interrogate and make sense of the information available, making a significant contribution to the company's performance.

A skills shortage

Data scientists are in high demand, with forecasts from IBM suggesting that the number of data scientists will reach 28 per cent by 2020. In the US alone, the number of roles for all US data professionals will reach 2.7 million. Yet these roles are harder to fill, remaining open for 45 days on average - one working week longer than the market average.

‘Machine learning, big data and data science skills are the most challenging to recruit for, and can potentially create the greatest disruption if not filled' says IBM's The Quant Crunch report. Demand is clearly outstripping supply for data scientists. Is there a reason for this? Could it be the case that the specialism is diluted as everyone knows a little about interpreting data - just enough to get by? And what is the danger if we don't fill this skills gap?

We have a large team of skilled data scientists at Pitney Bowes, who work hard to uncover insights for our clients and our business. Kristin Rahn, Director of Product Management for Data Science and Analytics, is one of them. Kristin has balanced her career between roles in analytics and the management of analytics teams, and roles at software companies in positions equivalent to product management.

"There is a shortage of candidates for all STEM roles," she says. "For data science in particular, the analytics classes are often the ‘I hated that in school' classes. Also, data science is performed for the benefit of others in the corporate world, which implies that the data scientist must have communication and consulting skills. You may have analysts with technical knowledge but who are lacking in the communication and consulting skills they need to add value in a commercial environment".

Data-driven businesses pushing up demand

Powerful software programmes give us access to deeper analytics than ever before. This analysis of data generated by people, places and things is a goldmine of invaluable insight. Studies have found quantifiable benefits in those businesses which make decisions based on data.

Even back in 2012, two researchers from MIT Sloane - Andrew McAfee and Erik Brynjolfsson - found that higher-performing businesses displayed more common practices of data-driven decision-making than their competitors. They said, "Companies in the top third of their industry in the use of data-driven decision making were, on average, five per cent more productive and six per cent more profitable than their competitors." This performance difference, they said, was "statistically significant and economically important and was reflected in measurable increases in stock market valuations".

As more data is generated, through mobile and web, via the Internet of Things and the Industrial Internet, the more experts we need to interpret the data and ‘translate' it, so it can be applied across different business functions. This requirement will only increase as more and more businesses are ‘data-driven', and look to generate new revenue streams from their data. Data scientists can unlock the value of data and apply it to the business.

What role should educators and recruiters play?

Educators need to reposition the role, and emphasise that it isn't about number crunching: data scientists are futurologists and digital influencers. They're also commercially-minded: training providers must ensure analytical skills are backed up with commercial acumen, and develop an understanding of the impact and application of data science. IBM's Quant Crunch report states, ‘To meet explosive demand, higher education needs to be nimble and responsive, and its bachelor's, graduate, certificate, and executive-level programs have to be responsive to workforce needs'.

"Data science and related analytics programmes need to help students understand that teamwork, communication and consulting skills are vital for their careers," says Kristin Rahn. "Students should be given a chance to work on problems as close to ‘real world' as possible, learn to interview a domain expert and present results."

If the data science skills shortage continues, businesses will be limited in the potential their data presents to the company. Productivity gains could be reduced and economic output decreased. Without deep understanding of their data, businesses will find it very hard to fulfil their potential and drive growth. For Kristin, the role is fulfilling and rewarding. "I love seeing the delight when people understand a positive outcome from an analytics project. I call it the ‘magic wand' moment," she says. "Analytics has helped them understand something, and given them the power to make a decision to help their business."

Andy Berry is VP EMEA of Software Solutions at Pitney Bowes.