This page was translated automatically using artificial intelligence (DeepL). The German version is binding. More information about automatic translation
English version below
In our research project, we have investigated decentralised storage concepts that combine erasure coding with proof-of-replication.
The rapid growth of sensitive data in the digital age calls for innovative storage solutions that overcome the limitations of conventional cloud services, such as single points of failure, high bandwidth costs and a lack of transparency regarding data processing and security. Decentralised storage systems address these limitations by utilising peer-to-peer (P2P) networks and blockchain technology to eliminate centralised control and improve data availability and fault tolerance. These systems achieve robustness by fragmenting data and enabling reconstruction even if some fragments are lost [1]. Redundancy mechanisms are crucial in decentralised networks, as nodes may frequently go offline. We use Reed–Solomon coding as the erasure coding scheme for redundancy, as it is highly computationally efficient. It is primarily described by two numbers, k and n, where n is the total number of fragments into which the original file is divided, and k (also known as the number of data fragments) indicates the minimum number of fragments required to reconstruct the original file. The system can tolerate the loss of up to m fragments, where m = n – k (so-called parity fragments) [2].
Cryptographic proofs such as Proof-of-SpaceTime (PoSt) [3], Proof-of-Retrievability (PoR) [4] and Proof-of-Replication (PoRep) [5] ensure the integrity and availability of the data. Existing decentralised storage systems such as Sia [6] or Storj [2] offer weaker protection against deduplication and Sybil attacks. Our work fills an important research gap by combining the robust cryptographic storage of PoRep with erasure coding to achieve strong security with reduced storage overhead. Erasure coding splits the data into smaller data and parity fragments and distributes them across the nodes, which significantly reduces storage overhead. PoRep uses cryptographic protocols to ensure that storage providers allocate unique storage space for data D, thereby preventing deduplication. The provider stores D in a sector where a computationally intensive sealing process converts D into a uniquely encoded replica, R.
The proof-of-replication process consists of three main phases:
- Encoding – The data is divided into nodes and structured into a layered Stacked-DRG graph. Each node is labelled sequentially and encrypted using a key derived from the graph labels, thereby creating a unique replica.
- Replication – A unique replica is generated using a ReplicaID that is bound to the provider and the sector, ensuring tamper-proof, verifiable storage.
- Merkle Tree & Proof Generation – A Merkle tree is constructed using the encrypted data to generate a proof. The final PoRep proof contains data, metadata and the provider’s identity, and is compressed before being transmitted to the blockchain.
We have implemented a Python-based system that combines erasure coding and PoRep to enable secure and efficient decentralised storage. When using a (k=10, n=14) erasure coding scheme, we found that the encoding time scales linearly with the file size, ensuring good scalability. We evaluated the storage efficiency of a (10,14) erasure coding scheme and compared it with the 5-fold replication method commonly used in systems such as Filecoin [3]. This was measured for various file sizes. Our erasure coding approach required only 40% additional storage (i.e. a total of 1.4 times the original data size), whilst 5-fold replication requires 400% additional storage. Despite this significant reduction in storage requirements, our method can still tolerate up to 28.5% data loss and thus exhibits high fault tolerance, as shown in Figure 1. Figure 1 illustrates the storage requirements of erasure coding and replication. We have also evaluated the sealing and verification performance of PoRep, as shown in Figure 2.
It is evident that the sealing time increases significantly with sector size, which underlines the computationally intensive nature of the sealing process. This supports PoRep’s security property: it ensures that the provider has already carried out the intensive sealing process and stored the unique sealed copy of the data. Attempting to generate this sealed copy on demand would be computationally infeasible, which proves the provider’s prior commitment to storage. In contrast, the verification time remains low and scales linearly, as it benefits from the logarithmic structure of Merkle trees. In summary, the integration of erasure coding with PoRep provides a secure, fault-tolerant and storage-efficient solution that is ideal for applications such as archiving and decentralised backup systems.
About the person
Shrigouri Navaratna is from India and came to Germany in 2021 to do her Master’s degree at Mittweida University of Applied Sciences. She studied Applied Mathematics for Networks and Data Science and wrote her Master’s thesis at the Blockchain Competence Centre Mittweida (BCCM) on the topic of decentralised storage systems under the supervision of Professor Andreas Ittner and Mario Oettler. In her thesis, she focused on researching and analysing erasure coding, encryption and storage proofs to improve the reliability and verifiability of decentralised storage systems. After completing her Master’s degree in 2024, she began working as a research assistant at BCCM and continued her research and development work on the Decentralised Storage System project. Outside of her work and studies, she enjoys playing badminton and table tennis.
Bibliography
[1] N, Racin (2023): Improving Data Availability in Decentralised Storage Systems, University of Stavanger, Norway. ISBN: 978-82-8439-158-8.
[2] Storj Labs (2016): "Storj: A Peer-to-Peer Cloud Storage Network," [Online] github.com/storj/whitepaper. [Accessed: 18 January 2025]
[3] Protocol Labs (2017): “Filecoin: A Decentralised Storage Network”, [Online] filecoin.io/filecoin.pdf. [Accessed: 16 January 2025]
[4] Shacham, H., Waters, B. (2013): Compact Proofs of Retrievability. J Cryptol 26, pp. 442–483
[5] B. Juan, D. David, G. Nicola (2017). “Proof of Replication”, [Online] filecoin.io/proofof- replication.pdf. [Accessed 6 August 2024]
[6] D. Vorick, L. Champine: “Sia: Simple Decentralised Storage”, [Online] sia.tech/sia.pdf. [Accessed 30 March 2025]
English version / english version
In our recent research, we investigated decentralised storage designs that combine erasure coding with Proof-of-Replication.
The rapid growth of sensitive data in the digital age necessitates innovative storage solutions that address the limitations of traditional cloud services, such as single points of failure, high bandwidth costs, and a lack of transparency in data handling and security. Decentralised storage systems address these limitations by utilising peer-to-peer (P2P) networks and blockchain technology to eliminate centralised control, improve data availability and fault tolerance. These systems achieve robustness by fragmenting data, enabling reconstruction even when some fragments are lost [1]. Redundancy mechanisms are critical in decentralised networks because nodes may go offline frequently. We use Reed-Solomon encoding as the erasure coding scheme for redundancy due to its computational efficiency. It is primarily characterised by two numbers, k and n, where n is the total number of fragments into which the original file is divided, and k (also known as the number of data fragments) is the minimum number of fragments required to reconstruct the original file. The system can tolerate the loss of up to m fragments, where m = n – k (known as parity fragments) [2].
Cryptographic proofs such as Proof-of-SpaceTime (PoSt) [3], Proof-of-Retrievability (PoR) [4], and Proof-of-Replication (PoRep) [5] ensure data integrity and availability. Existing decentralised storage systems such as Sia [6] or Storj [2] offer weaker security against deduplication and Sybil attacks. Our work addresses a key research gap by integrating PoRep’s robust cryptographic storage with erasure coding to combine strong security with reduced storage overhead. Erasure coding splits data into smaller fragments and distributes them across nodes, which significantly reduces storage overhead. PoRep uses cryptographic protocols to ensure that storage providers allocate unique storage for data D, preventing deduplication. The provider stores D in a sector, where a computationally intensive sealing process transforms D into a uniquely encoded replica, R.
The Proof-of-Replication process comprises three main phases:
- Encoding – Data is split into nodes and organised into a layered Stacked-DRG graph. Each node is sequentially labelled and encoded using a key derived from the graph labels, producing a unique replica.
- Replication – A unique replica is generated using a ReplicaID linked to the provider and sector, ensuring tamper-proof, verifiable storage.
- Merkle Tree & Proof Generation – A Merkle tree is constructed over the encoded data to create a commitment. The final PoRep proof includes data, metadata and the provider’s identity, and is compressed before being submitted to the blockchain.
We implemented a Python-based system combining erasure coding and PoRep to enable secure and efficient decentralised storage. Using a (k=10, n=14) erasure coding scheme, we observed that encoding time scales linearly with file size, ensuring good scalability. We evaluated the storage efficiency of a (10,14) erasure coding scheme and compared it with the 5x replication method commonly used in systems such as Filecoin [3], across various file sizes. Our erasure coding approach required only 40 per cent additional storage (i.e. a total of 1.4 times the original data size), whereas 5x replication results in 400 per cent additional storage. Despite this significant reduction in storage overhead, our scheme can still tolerate up to 28.5 per cent data loss, demonstrating strong fault tolerance as shown in Figure 1. We also evaluated PoRep’s sealing and verification performance, as shown in Figure 2, which indicates that sealing time increases significantly with sector size, highlighting the computationally intensive nature of the sealing process. This supports PoRep’s security property: it ensures that the provider must have already performed the computationally intensive sealing operation and stored the unique sealed replica of the data. Attempting to generate this sealed replica on demand would be computationally infeasible, thereby proving their prior commitment to storage. In contrast, verification time remains low and scales linearly, benefiting from the logarithmic structure of Merkle trees. In conclusion, our work on integrating erasure coding with PoRep offers a secure, fault-tolerant and storage-efficient solution, ideal for applications such as archival and decentralised backup systems.
The person
Shrigouri Navaratna is from India and came to Germany to study for her Master’s degree at Mittweida University of Applied Sciences in 2021. She studied Applied Mathematics for Network and Data Sciences and wrote her Master’s thesis with the Blockchain Competence Centre Mittweida (BCCM) on the topic of decentralised storage systems under the supervision of Professor Andreas Ittner and Mario Oettler. Her thesis focused on researching and analysing erasure coding, encryption and storage proofs to enhance the reliability and verifiability of decentralised storage systems. After completing her master’s degree in 2024, she began working as a research assistant at BCCM and continued her research and development within the ‘Decentralised Storage System’ project. Outside of her work and studies, she enjoys playing badminton and table tennis.
References
[1] N, Racin (2023): Improving Data Availability in Decentralised Storage Systems, University of Stavanger, Norway. ISBN: 978-82-8439-158-8.
[2] Storj Labs (2016): "Storj: A Peer-to-Peer Cloud Storage Network," [Online] github.com/storj/whitepaper. [Accessed: 18 January 2025]
[3] Protocol Labs (2017): “Filecoin: A Decentralised Storage Network”, [Online] filecoin.io/filecoin.pdf. [Accessed: 16 January 2025]
[4] Shacham, H., Waters, B. (2013): Compact Proofs of Retrievability. J Cryptol 26, pp. 442–483
[5] B. Juan, D. David, G. Nicola (2017). “Proof of Replication”, [Online] filecoin.io/proofof-replication.pdf. [Accessed 6 August 2024]
[6] D. Vorick, L. Champine: “Sia: Simple Decentralised Storage”, [Online] sia.tech/sia.pdf. [Accessed 30 March 2025]
