How would you design an online file-sharing system?

I once watched a 2GB upload fail at 98%. The user had to start over. No resume. No partial progress. Just gone.

That experience taught me: file sharing isn't about moving bytes. It's about making unreliable networks feel reliable.

What the System Needs to Do

Functional Requirements

• Users upload, download, update, and delete files from any device.
• Files sync automatically across all devices where the user is logged in.
• Version history with the ability to restore previous versions (snapshotting).

Non-Functional Requirements

• High availability — the service must be accessible at all times.
• Durability — any uploaded file must never be lost. Ever.
• Large file support — users will upload multi-gigabyte files. The system must handle them gracefully.

High-Level Design

The architecture splits into three core services: a block server for file storage, a metadata server for tracking file information, and a sync service for keeping devices up to date.

The Block Server

Files aren't stored as single blobs. They're chunked into blocks. Each block is stored independently. This enables three powerful things:

1. Resumable uploads — if a connection drops at 98%, you only re-upload the missing blocks, not the whole file.
2. Delta sync — when a file changes, only the modified blocks are re-uploaded.
3. Deduplication — identical blocks (even across different users) are stored once.

To build a solid block server:

• Use cloud object storage (S3, GCS, or Azure Blob Storage) as the backing store.
• Implement chunking on the client side. Split large files into fixed-size blocks (typically 4-8MB).
• Compute SHA-256 hashes per block. Use the hash as the block ID. This gives you content-addressable storage and free deduplication.
• Support resumable uploads. Track which blocks have been successfully stored. Resume from where you left off.

The trade-off: chunking adds client-side complexity. Every client (desktop, mobile, web) needs to implement the chunking and hashing logic correctly.

The Metadata Server

The metadata server tracks everything about files without storing the files themselves: file names, paths, sizes, owners, permissions, version history, and which blocks compose each file.

Key design decisions:

• Use a distributed database (Cassandra, DynamoDB, or sharded PostgreSQL) for horizontal scalability.
• Design the schema around access patterns. You'll query by user ID, by file path, and by block hash.
• Ensure strong consistency for writes (a file move or rename should be immediately visible).
• Scale reads with caching and read replicas.
• Implement access control at this layer — the metadata server decides who can see and modify which files.

The Sync Service

This is the hardest part. For every change on any device, the sync service must efficiently propagate that change to all other subscribed devices.

The key principle: transmit as little data as possible. Bandwidth is expensive and latency matters.

Here's how it works:

1. Client makes a change (edit, rename, delete).
2. Client computes block hashes for the changed file.
3. Client sends the hash list to the sync service.
4. Sync service compares hashes against the stored version. Only requests blocks with new or changed hashes.
5. Other devices are notified via push. They pull the updated metadata and download only the changed blocks.

A neat optimization: if the server already has a block with the same hash (even from a different user), it can skip the upload entirely. Same content, already stored. This is deduplication in action.

The Trade-Offs

Chunking vs. simplicity. Whole-file uploads are simple to implement. Chunked uploads are complex but enable resume, delta sync, and dedup. For any serious file-sharing system, chunking is worth the complexity.

Strong consistency vs. availability. Metadata writes need strong consistency (you can't lose a file rename). But file content can tolerate eventual consistency during sync. Use the right model for each layer.

Deduplication vs. encryption. If you encrypt each block with a user-specific key, you can't deduplicate across users (different keys produce different ciphertext for the same content). Convergent encryption (deriving the key from the content hash) solves this but introduces security trade-offs. Most systems choose one or the other.

The best file-sharing systems are invisible. Users don't think about blocks, hashes, or sync protocols. They just save a file on their laptop and find it on their phone. Making that feel effortless is the engineering challenge.