Command Palette

Search for a command to run...

PHASE 4Intermediate ~8 min· topic 14 of 18Level 3

Problem 4.14 — File Storage System (LLD)

In one line

Storage abstraction, naming, chunking metadata, and access control — the LLD core under Dropbox/Drive HLDs.

0/18 · 0%

Think of it like this

Google Drive or Dropbox. What you see as a single 'file' is really two things behind the scenes: the actual bytes stored somewhere cheap, and a small info card (name, owner, size) that points to those bytes.

Key ideas

  1. 01

    Ports: BlobStore (put/get/delete) implemented by LocalDisk, S3, MockStore — testability via the port.

  2. 02

    File meta: id, name, size, mime, ownerId, createdAt, checksum (MD5/SHA-256 for integrity).

  3. 03

    Chunking: large files split into fixed-size chunks with chunk indices — enables resume + parallel upload.

  4. 04

    Naming: never trust user filenames — generate UUID/sha-based object keys; keep display name in metadata.

  5. 05

    Access control: user → folder tree; permission model (owner, read, write, share) checked on every operation.

  6. 06

    Versioning/trash: pointer model — metadata row points at the current blob; older blobs retained (soft delete).

  7. 07

    Operations: upload (multipart), download, list, rename, move, share — each small but full of edge cases.

Java / Spring map

  • →

    StoragePort interface + S3Scanner adapter; FileMeta JPA entity; checksum util.

Code & diagrams

FileStorage.javajava

The metadata/blob split: the blob is content-addressed, and versioning is just pointing metadata at a new blob key.

public interface BlobStore {                  // PORT — testable with an in-memory fake
  String put(byte[] bytes) throws IOException;   // returns a content-addressed key
  byte[] get(String key) throws IOException;
  void delete(String key) throws IOException;
}

public final class Sha256BlobStore implements BlobStore {
  private final Path root;                       // swap for an S3Adapter without touching callers
  public Sha256BlobStore(Path root) { this.root = root; }

  public String put(byte[] bytes) throws IOException {
    String key = sha256Hex(bytes);                // never trust a user-supplied filename as the key
    Path dest = root.resolve(key.substring(0, 2)).resolve(key);   // fan out into subdirectories
    if (!Files.exists(dest)) { Files.createDirectories(dest.getParent()); Files.write(dest, bytes); }
    return key;                                   // identical content → identical key → free dedupe
  }
  public byte[] get(String key) throws IOException { return Files.readAllBytes(pathFor(key)); }
  public void delete(String key) throws IOException { Files.deleteIfExists(pathFor(key)); }
  private Path pathFor(String key) { return root.resolve(key.substring(0, 2)).resolve(key); }
  private String sha256Hex(byte[] b) { /* MessageDigest.getInstance("SHA-256") */ return "…"; }
}

// Metadata: a small row that POINTS AT a blob key. This is what makes versioning free.
public record FileMeta(String id, String displayName, String ownerId,
                        String currentBlobKey, long size, Instant createdAt) {
  public FileMeta withNewVersion(String newBlobKey, long newSize) {
    return new FileMeta(id, displayName, ownerId, newBlobKey, newSize, Instant.now());
    // the OLD blob is untouched — soft "undo" is just re-pointing currentBlobKey at the old key
  }
}

public final class FileService {
  private final BlobStore blobs;
  private final FileMetaRepository metas;

  public FileMeta upload(String ownerId, String displayName, byte[] content) throws IOException {
    String key = blobs.put(content);
    FileMeta meta = new FileMeta(UUID.randomUUID().toString(), displayName, ownerId, key, content.length, Instant.now());
    return metas.save(meta);
  }
}

Explain without notes

01

What does the metadata-blob split buy you, and how does versioning plug in for free?

Practice

01

Implement MultipartUpload with checksum verification and resume from the last committed chunk.

Trade-offs

  • ↔

    Chunking sizes: 5MB = fewer requests but coarser resume; 1MB = fine resume, more metadata rows. Pick with numbers.

Run it in production

You've designed it. Now build, operate, and break the same idea hands-on in the DevOps courses:

Completion checklist

  • I can name the storage port and the three invariants (unique id, checksum, access check) file ops must keep.

Back to phase