Decentralizing file storage with IPFS, Filecoin, and SIA

The Rise of Decentralized File Storage

File storage and sharing is a cornerstone of the digital world. As more and more data is created every day, there is an ever-increasing need for reliable and secure ways to store and access files. Centralized file storage services like Dropbox, Google Drive, Microsoft OneDrive, and iCloud meet many basic user needs, but they also come with drawbacks related to cost, privacy, security, ownership, and longevity.

An emerging solution is decentralized file storage networks based on blockchain and peer-to-peer technologies. By spreading data across many nodes on a network instead of storing it in centralized data centers, decentralized storage aims to deliver affordability, security, and redundancy. Three leading projects in this space are: the InterPlanetary File System (IPFS), Filecoin, and SIA. Each takes a unique approach to decentralization, but they share the common goal of giving users more control over their data.

What is Decentralized File Storage?

Decentralized file storage refers to solutions where users store files on peer-to-peer networks instead of servers owned by a single company. This approach spreads data across many nodes (i.e. computers) that participate in the network. Nodes connect directly to each other instead of through intermediaries.

With decentralized storage, no single entity controls user data. This contrasts sharply with centralized cloud storage provided by companies like Amazon, Microsoft, Google, Dropbox and Box. Users must trust these third parties to keep data secure and available. If any of them goes offline, gets hacked, loses data, or changes policies, users suffer the consequences.

In decentralized models, no individual node hosts all copies of files by default. If some nodes fall offline, the rest of the network continues functioning. This makes decentralized storage resistant to failures and outages. It also offers more privacy and ownership for users, since no company intermediates access to files.

Key Benefits of Decentralized File Storage

Decentralized storage solutions aim to provide:

• Affordability — By removing centralized intermediaries that charge ongoing fees, costs come down for end users. Host nodes can earn tokens for providing storage capacity and bandwidth to the network.
• Security — Spreading data across many nodes provides redundancy if some nodes fail. And without centralized servers, there is no single point of attack or failure. Data encryption protects files.
• Ownership & Control — Users maintain ownership rights over their own files. There are no third party terms, conditions or access controls to worry about.
• Reliability & Uptime — File availability and durability is maintained even if some nodes go offline by replicating file shards across different nodes.
• Transparency & Auditability — Transactions on decentralized storage networks are publicly verifiable through block explorers since they are based on blockchain technology. This creates accountability.
• Censorship Resistance — No single entity can restrict access to files by removing content deemed inappropriate or illegal. Decentralized models have no provision for censoring data.

As data volumes grow exponentially in the digital age, centralized stores have clear bandwidth, scaling and cost constraints. Decentralized methods are the only known models that can keep pace with humanity’s inexorable creation of data.

The IPFS, Filecoin & SIA Difference

Several projects are pioneering decentralized storage, but three stand out from the pack — IPFS, Filecoin, and SIA. Here is an overview of each network:

InterPlanetary File System (IPFS)

• Overview — IPFS is an open source protocol and peer-to-peer network for storing and accessing files. It pieces together multiple files systems to create a single global filesystem called the IPFS Cluster.
• Nodes — Instead of data living on huge servers, files are split into pieces and cryptographically hashed. Nodes store these hashed file fragments locally and demand is distributed across nodes. IPFS nodes can exit and enter seamlessly without impacting integrity of the network.
• Incentives — Participating in IPFS is largely volunteer driven and nodes are not incentivized monetarily. Users receive tokens but only for testing experimental features. The focus is on efficient routing via node composition rather than incentives.
• Use Cases — Distributing large files with version histories like wikis, websites, package systems and massively multiplayer online games. The New York Times, Wikipedia and many other major institutions use IPFS.

Filecoin

• Overview — Filecoin is blockchain based storage network that allows users to rent out spare hard drive space in exchange for tokens. It interfaces with IPFS but adds powerful economic incentives.
• Nodes — There are two kinds of nodes — Retrieval Miners that serve files and Storage Miners that offer storage capacity to store files. Miners earn tokens for providing bandwidth, storage space and computation resources.
• Incentives — Filecoin mints a fixed supply of tokens over time and pays Miners in this cryptocurrency for maintaining files on the network. Tokens incentivize Filecoin nodes to offer reliable storage.
• Use Cases — Filecoin excels at generalized commoditized storage. Use cases include archiving, data backups and distributed streaming. It is adopted by many media and energy companies in China.

SIA

• Overview — SIA is a blockchain powered storage platform secured by smart contracts. Renters pay Skyhour tokens to access storage capacity while Hosts earn tokens for providing infrastructure services.
• Nodes — Storage Hosts provide capacity by allowing partial or full access to their own hardware disks and bandwidth. Renters can upload, access and share files on SIA for lower costs than centralized alternatives.
• Incentives — Skyhour token payments incentivize Hosts on the network to offer reliable storage space, verify file availability and maximize uptime. Tokens are burned when files become inaccessible due to node failures.
• Use Cases — Offering commoditized storage space cheaper than cloud storage providers. SIA excels at infrastructure level contracts rather than end user apps. Companies like Herblink and MineBox build on SIA.

IPFS, Filecoin or SIA — Which One To Choose?

Here is a comparison of key pros and cons:

IPFS

Pros:

• Wide ecosystem support
• Interfaces easily with websites
• Resilient to attacks and failures
• No fees or transaction costs

Cons:

• Volunteer nodes limit scalability
• Unpredictable reliability
• Not incentivized for data durability
• Unencrypted by default

When to use: Distributing censorship-resistant content like news sites. Cases where data is rarely accessed after publishing. Applications with tech-savvy users.

Filecoin

Pros:

• Economic incentives improve durability
• Powerful data replication and redundancy
• Integrates smoothly with IPFS
• Usage-based pricing model

Cons:

• High token volatility impacts service fees
• Complex protocols have a learning curve
• Lower flexibility than raw IPFS

When to use: General purpose storage for enterprises and institutions. Use cases where strong privacy and encryption matter. Pay-per-use commercial applications.

SIA

Pros:

• Granular storage usage payments
• Usage based smart contracts
• Low transaction overhead
• Cheaper than cloud providers

Cons:

• No file sharding or replication
• Limited tools and client integrations
• Smaller ecosystem than IPFS and Filecoin

When to use: Building storage infrastructure solutions. Creating decentralized alternatives to the cloud. Long-term time capsule storage with less frequent access.

The Future of Decentralized Storage

Distributed storage combined with edge computing shortens data transfer distances. This greatly improves access speeds while slashing bandwidth costs. As internet-connected devices continue proliferating globally, decentralized storage allows secure harvesting of their unused local storage. The combined capacity across continents would be unimaginably vast.

Much work remains to move decentralized storage mainstream. Maturing cryptography, efficient data routing algorithms, and robust incentive designs are all challenges. Luckily an explosion of blockchain innovation is helping rapid progress. With monumental advances in distributed systems, decentralized storage finally makes economic and technological sense. Its advent could rank among the most significant internet revolutions yet.

Sources:

1. https://www.sciencedirect.com/topics/computer-science/decentralized-cloud-storage
2. https://dl.acm.org/doi/abs/10.1145/2619239.2626325
3. https://Protocol.ai/posts/ipfs-content-routing/
4. https://arxiv.org/abs/1902.01555
5. https://people.csail.mit.edu/rinard/paper/dsn06.pdf
6. https://ipfs.io/
7. https://docs.ipfs.io/concepts/dweb-protocols/
8. https://www.sia.tech/sia.pdf
9. https://youtu.be/asuA88D4PUQ