New framework for supercomputers could enable performance to reach exascale level, claim researchers

'Key value systems' could improve HPC flexibility and performance

Researchers claim to have devised a new framework that could help supercomputing to reach exascale levels of performance.

'Key value systems' can be used to enable high performance computing (HPC) systems to operate with more flexibility, scalability and performance, accordin to a new study by researchers at Virginia Tech University.

HPC systems, or supercomputers, play a significant role in computational science, especially at such organisations as the Met Office.

Supercomputers can process such queries in very little time; modern supercomputers can perform nearly a quadrillion floating point operations per second (FLOPS), also known as a petaflop.

However, the storage platforms needed to build such HPC data systems have been limited by a framework that demands users choose between high availability or customisation of features.

Now, researchers at Virginia Tech claim to have developed a new framework called BespoKV, that might one day enable scientists to build supercomputers able to perform one billion billion calculations in one second (exascale computing).

BespoKV is based on the concept of key value (KV) systems, which store important data on fast memory-based storage rather than slower disks.

According to the researchers, BespoKV's unique selling point is its ability to create several KV stores with desirable features. It takes a ‘datalet' (single-server KV store) and then creates ready-to-use distributed KV stores immediately.

The researchers also claim that BespoKV eliminates the need to redesign a system from scratch to complete a specific task, saying: ‘A developer can drop a datalet into BespoKV and offload the ‘messy plumbing' of distributed systems to the framework'.

This study is relevant to industries that process large amounts of data, such as large credit card firms, film streaming websites and social media .

"Developers from large companies can really sink their teeth into designing innovative HPC storage systems with BespoKV," said Ali Butt, professor of computer science.

"Data-access performance is a major limitation in HPC storage systems and generally employs a mix of solutions to provide flexibility along with performance, which is cumbersome. We have created a way to significantly accelerate the system behaviour to comply with desired performance, consistency, and reliability levels."

Findings of the research were presented this week at the Association of Computing Machinery/IEEE Supercomputing Conference in Dallas, Texas.