Command Palette

Search for a command to run...

Hectal
Phase 14Advanced15 of 15 in LLM Engineering

Interviews and Capstone

Answer the questions LLM interviewers ask, design two LLM systems on a whiteboard, solve the coding-round problems, then build a production-grade assistant.

This last phase turns everything you learned into the things a hiring panel and a real team look for. You will practise the concept questions that come up again and again (tokens, context, sampling, RAG against fine-tuning, evaluation, injection), each with a model answer that points back to the topic where you built the idea.

Then you will do the two kinds of whiteboard work an LLM engineer is given: system design (a customer-support assistant and a document question-answering service, with back-of-envelope estimates you can run) and coding-round problems (a rate limiter, retry with jitter, a streaming parser and a history trimmer, each with tests that print PASS or FAIL in three languages).

The phase ends with the capstone, a production-grade assistant that combines every layer of the course, with milestones and an acceptance checklist; a map of where to go next; and a one-page playbook that lists the recipe from every phase so you can find it again in a minute.

0/7 · 0%
7 topics ~3 h 16 code blocks & diagrams
Start with the first topic
1
14.1

Concept Questions Interviewers Ask

A dozen concept questions come up in almost every LLM engineering interview: tokens, context, sampling, hallucination, RAG against fine-tuning, evaluation, injection, structured output, cost and failures. This topic gives a model answer to each and points to the topic that taught it.

20 min 1 code practice

2
14.2

System Design: A Customer-Support Assistant

Design a support assistant that answers from help articles, looks up orders and hands over to a human, with numbers for traffic, tokens, cost and storage. You will practise the interview flow: requirements, estimates, architecture, safety, evaluation, operations.

29 min 3 code practice

3
14.3

System Design: Document Q&A at Scale

Design a service that answers questions from millions of private documents for many companies, with ingestion, hybrid retrieval, citations, freshness, deletes and strict separation between tenants. The estimator gives storage, ingestion cost and question cost.

26 min 1 code practice

4
14.4

LLM-Flavoured Coding Problems

Four small coding-round problems taken from real LLM plumbing: a token-bucket rate limiter, retry with full jitter, a streaming line parser that survives split chunks, and a history trimmer that keeps the system message. Each has tests that print PASS or FAIL, in three languages.

34 min 4 code practice

5
14.5

Capstone: Build a Production-Grade Assistant

Build one assistant that combines every layer of the course: retrieval with citations, a read-only tool, structured output, streaming, limits and retries, guards, evaluation in CI, observability and a service wrapper. This topic gives the milestones, an acceptance checklist and a runnable skeleton that wires fake versions of each layer.

22 min 1 code practice

6
14.6

Where to Go Next

You now have the foundation of LLM engineering. This topic shows how to find your weakest area, which courses on Hectal go deeper (two are still planned), how role roadmaps will build on this course, and what a portfolio should contain.

19 min 1 code practice

7
14.7

Playbook: The Whole Course on One Page

Every recipe from the course, one per phase, with the topic where you built it, a symptom lookup, and four runnable recipes that are used together most often: typed calls, resilient calls, a safe tool runner and an evaluation gate.

27 min 5 code practice

Practise this phase

Hands-on exercises: write the code, run it, and press Check to have it tested automatically.

1

Estimate monthly cost and calls in flight

Write monthly_cost(calls_per_day, tokens_in, tokens_out, price_in, price_out, days=30) that returns the cost in dollars. Prices are dollars per million tokens, and tokens_in and tokens_out are the average tokens of one call.

Intermediate
2

Parse a stream that arrives in pieces

Write a class SseParser with a method feed(chunk) that returns a list of the events finished by this chunk. A chunk can end anywhere, even in the middle of a line.

Advanced
3

Trim the history to a token budget

Write trim_history(messages, budget). The first message may be a system message; always keep it. Then keep the newest messages that fit in the remaining budget, stopping at the first message that does not fit (no gaps). Use the given count_tokens.

Intermediate
4

Retry with full jitter

Write retry_call(call, rng, sleep, max_attempts=5, base_ms=200, cap_ms=10_000). It calls call(attempt) (attempt starts at 0) and returns its result. When call raises Retryable, wait and try again. The wait is a random whole number of milliseconds from 0 up to min(cap_ms, base_ms * 2**attempt): take int(rng.next() * ceiling). Call sleep(wait).

Advanced
5

Score a prompt version and gate the change

Write score(answers, golden) that takes a dictionary of the model's answers by case id and a dictionary of expected labels by case id, and returns the integer percentage of cases answered correctly (right * 100 // total). Ignore leading and trailing spaces and letter case in the answers. A missing answer is wrong.

Intermediate