Requirements and Scale
Functional: upload and download files, sync files across devices, support sharing, version history. Non-functional: highly reliable (no data loss), eventual consistency for sync is acceptable, high availability.
Scale: 500M users, 100M daily active. If each user stores 200 files averaging 100 KB, total storage = 500M × 200 × 100KB = 10 PB. Read/write ratio ≈ 1:1 (every upload triggers downloads on other devices).
Block Server — The Core Innovation
Uploading entire files on every change is wasteful. Instead, the client's desktop agent splits each file into fixed-size blocks (4 MB each). Each block has a hash (SHA-256). On change, only modified blocks are uploaded — this is called delta sync. The server stores blocks as individual objects in cloud storage (S3). The block server deduplicates identical blocks across users (two users with the same file share the same block objects).
Benefits: (1) Reduced bandwidth — only changed blocks transferred. (2) Storage efficiency — deduplication. (3) Parallelism — blocks upload concurrently. On download, the client fetches only the blocks it doesn't already have and reassembles the file locally.
graph LR Client["Desktop Client"] --> BlockServer["Block Server"] BlockServer --> CloudStorage["S3 / Object Store (per-block objects)"] Client --> MetadataServer["Metadata Server"] MetadataServer --> MetadataDB[(Metadata DB MySQL + Cache)] Client -->|Long Poll| NotificationServer["Notification Server"]
Metadata Database
Stores file tree structure, not file content. Schema: Workspace (user_id, storage_limit), File (file_id, name, path, is_folder, latest_version, created_at, updated_at), FileVersion (version_id, file_id, file_size, created_at, created_by_device), Block (block_hash, block_path_in_S3, block_size).
Relational DB (MySQL) with caching layer (Memcached). Sharded by user_id. Synchronization conflict resolution: last-write-wins with timestamps, or surface conflict copies to the user (Dropbox creates a "conflicted copy" file).
Synchronization — Keeping Devices in Sync
When Device A updates a file: (1) Desktop agent detects change via OS filesystem events. (2) Chunks new/modified blocks, computes hashes, uploads changed blocks to Block Server. (3) Sends new metadata (file version + block list) to Metadata Server. (4) Metadata Server notifies all other online devices via Message Queue.
Device B receives notification: (1) Fetches updated metadata from Metadata Server. (2) Compares local block list with new block list. (3) Downloads only missing/changed blocks from S3 via Block Server. (4) Reassembles file.
For change notification, long-polling is used (client keeps an open connection; server responds when there's a change). This is more efficient than short polling since most connections are idle most of the time.
Version History and Recovery
Every edit creates a new FileVersion row pointing to the current block list. Old versions are retained for 30 days (or unlimited for paid plans). To restore: fetch the block list for the target version and reassemble. S3 block objects are never deleted as long as any version references them — a garbage collection background job periodically removes orphaned blocks (no version references them and they are older than N days).
Security and Deduplication
Blocks are encrypted client-side before upload (AES-256) so the block server never sees plaintext. Block hashes are computed on plaintext before encryption for deduplication purposes. Metadata is encrypted at rest in the DB. File sharing: generate a signed URL (time-limited token) that allows the recipient to download the file without an account. Team folders use access control lists (ACLs) at the folder level.