Problem 4.14 — File Storage System (LLD)
In one line
Storage abstraction, naming, chunking metadata, and access control — the LLD core under Dropbox/Drive HLDs.
Think of it like this
Google Drive or Dropbox. What you see as a single 'file' is really two things behind the scenes: the actual bytes stored somewhere cheap, and a small info card (name, owner, size) that points to those bytes.
Key ideas
- 01
Ports: BlobStore (put/get/delete) implemented by LocalDisk, S3, MockStore — testability via the port.
- 02
File meta: id, name, size, mime, ownerId, createdAt, checksum (MD5/SHA-256 for integrity).
- 03
Chunking: large files split into fixed-size chunks with chunk indices — enables resume + parallel upload.
- 04
Naming: never trust user filenames — generate UUID/sha-based object keys; keep display name in metadata.
- 05
Access control: user → folder tree; permission model (owner, read, write, share) checked on every operation.
- 06
Versioning/trash: pointer model — metadata row points at the current blob; older blobs retained (soft delete).
- 07
Operations: upload (multipart), download, list, rename, move, share — each small but full of edge cases.
Java / Spring map
- →
StoragePort interface + S3Scanner adapter; FileMeta JPA entity; checksum util.
Code & diagrams
The metadata/blob split: the blob is content-addressed, and versioning is just pointing metadata at a new blob key.
public interface BlobStore { // PORT — testable with an in-memory fake
String put(byte[] bytes) throws IOException; // returns a content-addressed key
byte[] get(String key) throws IOException;
void delete(String key) throws IOException;
}
public final class Sha256BlobStore implements BlobStore {
private final Path root; // swap for an S3Adapter without touching callers
public Sha256BlobStore(Path root) { this.root = root; }
public String put(byte[] bytes) throws IOException {
String key = sha256Hex(bytes); // never trust a user-supplied filename as the key
Path dest = root.resolve(key.substring(0, 2)).resolve(key); // fan out into subdirectories
if (!Files.exists(dest)) { Files.createDirectories(dest.getParent()); Files.write(dest, bytes); }
return key; // identical content → identical key → free dedupe
}
public byte[] get(String key) throws IOException { return Files.readAllBytes(pathFor(key)); }
public void delete(String key) throws IOException { Files.deleteIfExists(pathFor(key)); }
private Path pathFor(String key) { return root.resolve(key.substring(0, 2)).resolve(key); }
private String sha256Hex(byte[] b) { /* MessageDigest.getInstance("SHA-256") */ return "…"; }
}
// Metadata: a small row that POINTS AT a blob key. This is what makes versioning free.
public record FileMeta(String id, String displayName, String ownerId,
String currentBlobKey, long size, Instant createdAt) {
public FileMeta withNewVersion(String newBlobKey, long newSize) {
return new FileMeta(id, displayName, ownerId, newBlobKey, newSize, Instant.now());
// the OLD blob is untouched — soft "undo" is just re-pointing currentBlobKey at the old key
}
}
public final class FileService {
private final BlobStore blobs;
private final FileMetaRepository metas;
public FileMeta upload(String ownerId, String displayName, byte[] content) throws IOException {
String key = blobs.put(content);
FileMeta meta = new FileMeta(UUID.randomUUID().toString(), displayName, ownerId, key, content.length, Instant.now());
return metas.save(meta);
}
}Explain without notes
What does the metadata-blob split buy you, and how does versioning plug in for free?
Practice
Implement MultipartUpload with checksum verification and resume from the last committed chunk.
Trade-offs
- ↔
Chunking sizes: 5MB = fewer requests but coarser resume; 1MB = fine resume, more metadata rows. Pick with numbers.
Run it in production
You've designed it. Now build, operate, and break the same idea hands-on in the DevOps courses:
Completion checklist
I can name the storage port and the three invariants (unique id, checksum, access check) file ops must keep.