Big data in space: 'It's important to know where you are when you want to communicate with Earth'

Dr Redouane Boumghar, research fellow in data science for space operations at the European Space Agency, explains how big data and collaboration tools are revolutionising space travel

Space may be the final frontier, but it's seems to be getting ever closer and more familiar, as private firms like Elon Musk's SpaceX turn the extra-terrestrial missions that used to be the preserve of governments, into corporate business as usual.

The European Space Agency (ESA), a consortium of governmental organisations with 22 member states, is similarly busy turning rocket launches into the stuff of ordinary Wednesday mornings rather than major events.

And part of the revolution is the use of big data to manage everything from the smallest piece of equipment on a minor satellite, the full-blown plans to build a colony on Mars.

Speaking at Computing's Big Data & IoT Summit recently, Dr Redouane Boumghar, research fellow in data science for space operations at the ESA, discussed one of the satellite programmes he's currently working on.

"The goal of the space craft is to get pictures of stars to help redefine their positions, so we need lots of data around their magnitude, and lots of other points," said Boumghar. "The idea is to have a good catalogue of stars for satellites to navigate by. It's important to know where you are when you want to communicate with Earth," he added.

Each satellite sends around 1.3 Terabytes of uncompressed data down to Earth. And it's all kept. "We have an archiving culture," explains Boumghar, adding that there are 15 similar missions outputting roughly the same volumes of data.

But the data flows from other sources too, beyond the attempt to map the stars more accurately. Each satellite is equipped with many sensors, and it's Boumghar's job to understand what their readings say about the craft's status and behaviour.

"We get about 4-16 Gbs per day in telemetry data from each satellite, which we use for health monitoring. So if we know the temperature of each part of the craft, we have a better idea of how well each system might be working," he says.

This enables decision making such as turning different subsystems on or off, depending on how that might affect its heat, or how to affect its orbit, perhaps if the craft needs to avoid some space debris.

Ultimately it all comes down to how the data is used - and Boumghar explains that there is still plenty of gold to be mined there.

"We want to extract more knowledge. We're already using machine learning, and we would like to use AI with virtual assisatants, to help our understanding of what's happening with our space craft. If the virtual assistants have accurate definitions of what's happening, they can take accurate counter measures. So you observe, decide, take action, then observe again to see if the response has worked."

Collaboration is key to the performance of Boumghar's division at the ESA, and he describes one of the best tools for this as the head office's cafeteria. But there are collaborative technology tools too.

"I work in the advanced machine concept group. We're partly there to help people work in an agile way as well as to analyse data. We want people to have the right skills and information, once they have that they can deliver," he says.

Boumghar has taken the maps -called ‘graphs' - his team builds for spacecraft data flow, and adapted it to track and define the flow of information around his organisation.

"We're trying to build graphs for spacecraft, why not do the same with people, to help in managing the information flow? If we have that graph, that means everyone knows what everyone else is doing.

"We also have an internal Facebook style tool called ESA connect, but it doesn't just work by itself, you have to encourage people to use it.

"Collaboration avoids double-spending in big organisations, because your ideas are rarely new. You really don't want two people developing machine learning at same time," he adds.

He finishes with some advice for any firm looking to improve its own collaboration.

"Be open internally. Create your own teams, but make sure you advertise it, because doing something in isolation is rarely good.

"We have operations teamed up with engineers. There are infrastructure people, who are setting up servers and plugging in cables. And then we have people creating prototypes around analytics engines. We know what the operations people need, but the infrastructure guys don't, so they go through us. It's our job to connect them together."

However, he also sounded a final note of caution, explaining that some departments can operate more like fiefdoms, creating their own factions and being very unwilling to share or collaborate.

"Not everyone wants to share their projects," admits Boumghar. "You watch the Game of Thrones, I've been living it!"