How do you move petabytes of data out of Europe? With a van

IT leaders discuss strategies to move large volumes of data out of Europe following the Brexit vote

With Britain set to leave Europe, IT leaders across the country are considering how to move large volumes of data back into the UK in order to comply with data protection laws, including the upcoming GDPR.

During a discussion at a recent Computing event, 'Architecting your organisation for the future', panellists discussed the challenge of moving large and ever-expanding data volumes across pipes which are not growing at the same rate.

An audience member, who wished to remain anonymous, explained the scale of his problem. "Brexit stuck a spanner in the storage game. We may have to move petabytes of data out of Europe into the UK."

Nick Ioannou, Head of IT, at architects Ratcliffe Groves Partnership said he had experienced the same issue.

"When I tried to move our email archive, I realised it would take months to transmit across the network. So I had to copy it all to disk and courier it. When you move petabytes you need a van, it just takes too long otherwise."

Lukas Oberhuber, CTO of Simply Business explained that his organisation employs a different approach, with multiple copies of the same data always in separate locations.

"We have multiple servers that are constantly synching with one another. We use MongoDB which expects multiple servers that are in synch, so our data is always replicated."

The audience member responded, stating that networks are currently too slow for large-scale data transit.

"We keep stuff on Amazon but they don't have data centres in the UK yet. The biggest problem is that the pipes don't match up to the data sizes."

Oberhuber agreed, and added that the challenge is the same when attempting to write to, or change, large volumes of data.

"Changing petabytes of data if you need to encounters the same problem. Pipes and storage systems can't do it. No one expects to be able to access all this data at once."

Alex Chen, director at IBM said that writing to certain types of flash drive is one solution.

"One of the use cases for DeepFlash is that people use it as half a petabyte in 3U config. It's like a big thumb drive, so it's really fast at reading and writing. And it's low cost enough that you can move the flash somewhere, and then come back for more."

Earlier in the event, Phil Durbin, Head of IT Systems at the Salvation Army said that software-defined storage could be one solution to the problems of vendor lock-in.