Decision Intelligence is the next evolution of AI, says machine learning pioneer

"I wouldn't fly in an aeroplane if a pilot hadn't been through simulator training, says Dr Lorien Pratt. "I want him to crash his plane in the simulator 20 times before he gets in the cockpit, right?"

Pratt, who has been working with AI and machine learning since the 1980s and who invented the subfield of inductive transfer, is enthusiastic about the power of AI-driven simulation in decision-making. Why should it be restricted to pilots? She wants to see graphical representations brought to bear on big political and business decisions too, as a much more intuitive way to connect actions and outcomes than is common today. Data and 'insights' on their own are too abstract, she says; what's needed is more integration.

"I don't know how we got into this cul-de-sac. You've got some decision maker whose job it is to make a choice or a policy that's going to impact millions of people, and somebody gives them a stack of PowerPoint graphs and say, here's this trend and here's that trend. That's like serving a bowl of flour, a bowl of water and a stick of butter in a restaurant. It's not in a digestible form."

She offers a personal example. During the pandemic, she found herself perusing numerous graphs and charts as she tried to track the spread of the disease, but when her mother asked if she should go to the grocery store, she was unable to provide an answer. Decision intelligence (DI), a field co-pioneered by Pratt, would put her mother in the driving seat just like the pilot in a simulator. Rather than confronting her with messy data and graphs, it would allow her, via a slick UI, to try various scenarios as to when the best time to go to the grocery store might be.

While under the hood it uses machine learning models fed by masses of data, with DI the workings are hidden from the user. This is very important for decision makers, but the BI and decision-support arena is still very tool-oriented, especially at the AI end.

"People want to see actions and outcomes. They don't want to see the machinery of how those actions lead to the outcomes."

A new AI winter?

Having witnessed a good few years of positive progress, AI is now starting to see failures. AI is difficult to integrate into everyday systems and processes and expectations are not being met.

"We've reached a point of diminishing returns," Pratt says, adding ominously: "We may be heading for another AI winter".

With data its only source of intelligence, deep learning is limited in what it can achieve; data is messy and flawed and data preparation is laborious. Also, engineers have a tendency to try to design people out rather than keeping the human (the supposed beneficiary after all) in the loop.

Instead of trying to make everything autonomous and eliminating human intervention, AI engineers need to get to the next stage, which is actually adding the people and processes to the mix. This, Pratt claims, will be the next evolutionary stage of AI.

"Kitty Hawk, the first aeroplane, was just a technology. Then the airline industry grew up to incorporate people and processes around that tech. AI as it evolves to DI is also maturing. We've been very singularly technology focused, and what we're doing with DI is we're bringing in the people and the process side of it as well."

DI promises to reduce the time to take complex decisions, lessen the risk of unforeseen consequences, keeping people in the loop while eliminating bias: while business and political leaders claim to want to make data-driven decisions, more often than not it's the gut that wins out because those graphs and charts don't trigger the same emotional response.

Pratt has lived through AI winters before, hard times when funding dried up and AI dare not speak its name. But this time it's more serious, she says, as artificial intelligence is urgently needed to handle pressing 'wicked problems': "You know, climate, poverty, inequality, Covid, democracy".

Wicked problems are those with a huge number of variables; commonly they have sociological components too. They often involve actions at a distance in space or time (the results of an action to mitigate climate change will not be measurable for years, if at all), and they're frequently exponential, bumping along for a while then suddenly rocketing upwards. Another characteristic is the interdependence of critical factors: "To solve climate, you've got to get democratic state stability, you've got to get the regulatory stuff right, you've got to get carbon down and you've got to get human opinion onside."

To tackle such problems, therefore, you need to pull in data and models from numerous disciplines and domains and integrate it seamlessly, which is not happening right now.

"I believe that the reason we haven't solved them is because we live inside these individualised silos with our analyses," said Pratt.

Grand unification

A "voice in the wilderness" for many years, Pratt enthuses that DI is now taking off. It was picked up by Gartner last year as an emerging trend, with the analyst firm predicting a third of large organisations will be using some form of DI by next year. Meanwhile, Pratt's book Link is doing brisk business.

"Now finally, we're seeing a hockey stick," she said.

Maybe so, but it's early days. There is a standard reference architecture featuring visualisation and modelling layers plus an integration layer that can connect to other models, including "soft" information like behavioural economics. But there are no industry standards as yet, nor is there a market leader in terms of vendors.

Nevertheless, vendors are coming on board. The consultancy Quantellia which Pratt co-founded is joined by the likes of IBM Cloud, Oracle Cloud, Google Cloud, Pyramid Analytics, Busigence and others who are starting to offer services and solutions.

Commercial use cases are also starting to emerge.

A financial services firm is using DI to improve its credit risk analysis, using a single solution instead of multiple ones to generate and test hypotheses and put those into action, and an energy company is deploying it for forecasting demand as it integrates renewable sources. Pratt says she helped the Scottish government model the spread of Covid via aerosols, and also aided a bank in working work out its flying policy as it moves towards Net Zero targets.

"This is the grand unification of where we've been lost inside silos," said Pratt. "It is the opportunity to bring the silos back together. At the point of the decision at the point where a human is trying to achieve some outcome with a policy or something else, and they're making some choice. The fog between the choice and the and the outcome is starting to clear, and it's a really exciting time within the industry."