Capstone: Plan a Document Question-Answering Tool

New words in this topic, in plain English. Come back here whenever one feels fuzzy.

RAG

Retrieval-augmented generation: find relevant text first, then give it to a language model along with the question.

Chunk

A piece of a document, small enough to fit in a prompt and focused enough to match a question.

Embedding

A list of numbers that stands for the meaning of a piece of text, so similar texts have similar numbers.

Retriever

The part that finds the chunks most related to a question.

Protocol

A typed description of the methods an object must have, without requiring inheritance.

Frozen dataclass

A data class whose fields cannot be changed after it is created.

Pipeline

Stages in a row where each stage's output is the next stage's input.

Stub

A placeholder function with the right inputs and outputs that returns fixed fake data.

1. 1

   Add a requirement and a test for it

   Write a new requirement in one sentence, for example 'a question with no matching document says it does not know'. Then write, in words, the test that would check it using the skeleton pipeline. You will implement it in Topic 17.7.

2. 2

   Change a stage without touching the others

   In the skeleton example, make ask_model return an answer in uppercase. Run it. Only one function changed, and the rest of the program did not notice. That is what clean interfaces buy you.

Errors are the best teachers. Make each change, read the error, guess what went wrong, then reveal the answer.

from dataclasses import dataclass


@dataclass(frozen=True)
class Chunk:
    doc: str
    text: str


chunk = Chunk("a.txt", "hello")
chunk.text = "changed"

1. 1

   What docqa does. You point it at a folder of documents and ask a question in plain English. It finds the parts of the documents most related to the question, gives them to a language model together with the question, and prints an answer that cites the sources it used. This pattern is called retrieval-augmented generation (RAG). Topic 16.9 built it in miniature; here you build it as a proper, tested, shippable project.

2. 2

   Requirements first. A good list is short and testable. Docqa must: read .txt files from a folder; split them into overlapping chunks; find the top-k chunks for a question; build a prompt that contains those chunks as numbered sources; stream the model's answer; print which sources were used; run from the command line with a config; work offline with a fake model (so tests and demos need no API key); and fail with a clear message, never a stack trace, for ordinary mistakes such as a missing folder. Write down what it will not do yet: PDFs, a web interface, a real vector database, authentication.

3. 3

   Architecture: a pipeline of small parts. loader reads files into Document objects. chunker turns a document into Chunk objects. embedder turns text into vectors (lists of numbers that stand for meaning; Topic 16.8). retriever stores the chunk vectors and returns the top-k Hit objects for a question. model takes a prompt and streams text. cli wires the parts together. Each part knows only the data classes and interfaces around it, so you can replace any one (a real embedding model for the toy one, a real language model for the fake one) without touching the others.

4. 4

   **Interfaces with typing.Protocol** (Topic 10.6). An interface is the list of methods a part promises to offer. A Protocol describes the methods an object must have, without inheritance. Embedder needs embed(texts) -> array; ModelClient needs stream(prompt) -> Iterator[str]. Any class with those methods fits, including a test fake, so the pipeline code is written once against the Protocol. This is the same dependency injection idea from Topic 17.5, and it is what makes the whole tool testable without the network.

5. 5

   Data classes carry the data between stages (Topic 8.8). Use @dataclass(frozen=True) for Document, Chunk, Hit and Answer: they are immutable (so a stage cannot secretly change what another stage returned), have readable repr for debugging, and are easy to compare in tests. Keep plain dictionaries for configuration you read from files, and convert them into a typed Config object at the edge.

6. 6

   Build order. Start with the data classes and a skeleton pipeline of stubs that runs end to end and prints a fixed answer. Replace one stub at a time with the real thing, running the tests after each: loader, chunker, embedder and retriever, prompt builder and fake model, CLI, logging and retries, real model client. A program that always runs end to end is much easier to debug than five untested parts joined at the end.

• ↔

  Small modules versus fewer files. Many small modules give clean boundaries but more files to navigate. Group by responsibility, and merge modules that always change together.

• ↔

  Protocols versus abstract base classes. Protocols describe structure and need no inheritance, which suits fakes and third-party classes. Abstract base classes enforce implementation at class creation and can share code. Prefer Protocols for interfaces between parts.

• Done when you can write the requirements of docqa as short, testable sentences.

• Done when you can draw the pipeline and name the data that flows between stages.

• Done when you can write frozen dataclasses and Protocols for the stages.

• Done when you can explain why you build a stub pipeline first.

• Done when you can lay out the project folders in the src layout.