The amount of digital information produced worldwide is growing exponentially, reaching higher values than expected. Simultaneously, with the emergence of cloud computing services, the information from different users stopped being stored solely in their personal computers and started being stored remotely by companies. It is therefore critical for firms to find efficient ways, at reduced costs, to accommodate this explosion of digital information.

Deduplication emerges as a key response to this challenge, as it makes it possible to eliminate duplicated information (videos, music, etc.) belonging to different users. Recent studies show that using deduplication, it is possible to reduce the amount of information stored in cloud computing infrastructures up to 80%. Repeated information is deleted automatically and in a transparent way to those who use the service, which means that each person continues to access their documents without realising there is only one shared copy.

The paper presented by the researchers at HASLab/INESC TEC describes the concept in detail, reviewing the systems and the existing deduplication algorithms, presenting a new classification. The paper also discusses new scenarios where this technique can be applied the challenges to overcome.

The INESC TEC researchers mentioned in this article are associated with the following partner institutions: U.Minho.

INESC TEC, September 2014