This project addresses the critical design problem of protecting user data efficiently across diverse environments. Sponsored by Huawei, our goal is to create a data deduplication solution that ensures data integrity and high performance.
The current deduplication algorithm for data protection is offloaded to the storage base file system. We aim for an application-layer deduplication algorithm that is independent of the storage file system.
Our design is driven by specific requirements: scalability, robust privacy measures, and reliable performance under high-frequency backup demands. Initial analyses identified two primary challenges: maintaining privacy in dynamic network environments and implementing effective data deduplication and compression without impacting system performance.
Upon completion, our solution will offer a high-performing, scalable, and secure data protection system, aligned with the growing data demands of modern organizations.
