Lossless Data Compression Points to a Fast Future.

A recent report estimates that some 2.5 quintillion bytes of data are currently generated every day – and by 2020 that’s likely to be 44 trillion gigabytes daily. OCC is currently working with start-up company SISP Technologies on a revolutionary new data compression algorithm that will significantly increase data transmission speeds, reduce storage costs and overcome current constraints on bandwidth.

The algorithm, which compresses data in segments, was devised by mathematician and financial expert Nicholas Stavrinou, now CEO of SISP Technologies. He shared his idea with software developer Stuart Marlow (now SISP’s CTO), who saw its potential application to lossless data compression. “We’re seeing an exponential growth in ‘big data’ from sources such as sensors, edge devices and other connected devices. We quickly recognised there’s a real need for intelligent, domain-specific lossless data compression,” says Stavrinou.

The algorithm works by dividing a file into different-sized segments that will compress well as independent units. Each segment is examined to identify a periodic pattern of groups of bits that can be encoded efficiently. Each group of bits is then replaced by an efficient variable-length output encoding. Some information necessary for decompression is stored as a header.

Aware of OCC’s reputation for transforming ideas and research into commercial applications, SISP approached the company with an early prototype of the algorithm written in Visual Basic. “OCC is well known for building innovative applications, so we engaged them to work closely alongside Stuart to develop production-ready code in C,” says Stavrinou.

“SISP wanted us to help them turn their algorithm into a commercial product that they could sell as a service or module,” explains Rachael Bartholomew, OCC’s Head of Client Services. “We’ve worked together to iteratively improve the algorithm and attain the key performance indicators they wanted in terms of computation speed and ratio. In some data transmission scenarios, even a tiny difference in the size of a file can make a big difference to your business, so we used OCC’s skills and innovation expertise to squeeze every last drop of compression out of the application.”

Stavrinou adds, “OCC has worked flexibly and collaboratively with us and we’ve felt that they are truly part of our R&D team.” An added benefit, he notes, was that OCC undertook the development work using practices and processes to provide traceability, ensuring both the correctness of the implementation and legal adherence to the patents SISP is filing.

SISP already has a patent granted in the UK, patents about to be granted in the USA and Europe and a number of pending applications in the UK and internationally. The company is now entering a phase of active negotiations for commercial partnerships and hopes to have products available in 2020.

“Our algorithm is highly configurable and allows real-time compression across existing networks, with complete retrieval of the material compressed,” says Stavrinou. “We believe it offers huge benefits to any markets which depend on smart-sensor derived data, such as satellite and LiDAR applications, as well as the financial services sector.”