Topic 14.6
Where to Go Next
In one line
You now have the foundation of LLM engineering. This topic shows how to find your weakest area, which courses on Hectal go deeper (two are still planned), how role roadmaps will build on this course, and what a portfolio should contain.
Think of it like this
Finishing driving school. You know the controls and the rules of the road, and now you must choose where to drive. A map shows roads to different towns: a city for deep retrieval work, a harbour for agents, a workshop for platform and operations. You do not visit them all at once. You pick one, with a first trip planned.
Words you'll meet
New words in this topic, in plain English. Come back here whenever one feels fuzzy.
- Gap check
- A quick honest rating of your skills in each area, to find where to study next.
- Specialty
- The area you go deepest in, such as retrieval, agents or platform work.
- Portfolio
- A small set of finished projects and write-ups that show what you can do.
- Roadmap
- An ordered list of what to learn for a goal or a role.
- Changelog
- A provider's list of changes to models and APIs. Reading it is part of the job.
Step by step
01Step 1: run the gap check
Run the example, then change the scores to match your own honest ratings. The three lowest areas are your next study targets. If two areas tie, take the one closer to your goal first.
02Step 2: choose one track
Pick one of three tracks for the next three months: application depth (retrieval, then agents), platform depth (operations, deployment, models) or interview depth (system design and coding). The diagram shows how they start from the same base.
03Step 3: pick one project that fits the track
Application track: extend the capstone with a better retriever and measure the gain. Platform track: put the capstone behind a container, a CI eval gate and dashboards. Interview track: redo the two designs in this phase with your own numbers and time yourself. One project, finished, with measurements.
04Step 4: collect evidence as you go
After each work session, add one line to a log: what you changed, the number before and after, what surprised you. These lines become the portfolio write-up with almost no extra work.
05Step 5: stay current without chasing news
Once a month: read your provider's changelog, re-run the eval set on the current model, check your cost per conversation, and list one thing to try. Skip anything that does not change a number you track.
Try it yourself
- 1
Be honest
Replace the scores with your real ratings, and set each portfolio item to what is true today. Run it. Is the first area on the list the one you expected?
- 2
Plan the first week
Pick the lowest area and write three tasks for the next seven days, each ending in a measurable result, such as "recall at 5 on 20 questions".
- 3
Write the one-page summary
Write a README summary for your capstone in five lines: what it does, the three numbers that matter, the biggest failure you hit, how you fixed it, and what is not covered.
Code & diagrams
Replace the scores with your own. The order is lowest score first, and the original order breaks ties.
Expected output
1. operations (score 0): Phase 11, and the Observability course on this site
2. retrieval and RAG (score 1): Phase 8, then the coming RAG and Vector Search course
3. evaluation (score 1): Phase 9
portfolio: 2 of 4 pieces ready
still to write: an eval report with numbers
still to write: a short design doc with trade-offsBreak it on purpose
Errors are the best teachers. Make each change, read the error, guess what went wrong, then reveal the answer.
Break #1
Sort the wrong way
In the sort key, negate the score: key=lambda i: (-AREAS[i][1], i) (Python), so the highest scores come first.
Myth vs fact
Myth
I should learn every framework before I build.
Fact
Frameworks change quickly and wrap the same ideas you built by hand. Depth in the ideas and one finished project transfers to any framework.
Myth
A certificate matters more than a project.
Fact
For engineering roles, a working project with measured results and an honest failure story is stronger evidence.
Myth
Models will keep changing, so skills do not last.
Fact
Models change, but evaluation, retrieval design, reliability and safety habits stay useful across versions.
Pro corner
Extra depth for experienced readers. New to this? Skip it for now and come back later.
- ▸
Keep your own small benchmark of five tasks you care about and run it on every new model. It teaches you more about the real differences than any public leaderboard (Topic 13.7).
- ▸
Specialise on a problem, not a tool: "answers from messy documents with proof" outlasts any single vector database.
- ▸
Read incident write-ups from teams running LLM products. They show failure modes that no tutorial covers, and they give you stories for interviews.
Remember this
- 1
Why this exists. Without a direction, learners finish a course and then re-read it, or jump to the newest framework each week. A short gap check and one chosen project turn a broad foundation into depth, and depth is what gets you hired or promoted.
- 2
Run a gap check first. Rate yourself from 0 (never done it) to 3 (shipped it and measured it) in seven areas: calling and parsing, streaming and cost, tools, retrieval, evaluation, safety, operations. The example below sorts the areas and prints the three lowest with the place to go. Be honest, because only a shipped and measured piece of work deserves a 3.
- 3
Two courses on Hectal are planned and not live yet. The coming RAG and Vector Search course will go deeper than Phase 8: chunking strategies, hybrid search, reranking, measuring retrieval quality and running a vector database. The coming AI Agents course will go deeper than Phase 7: planning loops, memory, multi-step tool use, and how to test and secure agents. Until they are published, Phases 7 and 8 and the capstone are your base for both.
- 4
Courses that are live and pair well. The Python course (
/python) has an AI-era phase and strengthens your core language. System Design (/system-design) covers queues, caches, databases and scale, which every LLM service uses. DSA (/dsa) prepares the other half of an interview. Docker (/docker) and CI/CD (/cicd) take your service and your eval gate to production. Observability (/observability) deepens Phase 11. Core Java (/java) and JavaScript (/javascript) help if your services are written in those languages. - 5
Role roadmaps are planned. Separate roadmaps are planned for an AI engineer, a forward-deployed engineer (who builds with customers) and an AI scientist (who works on models themselves). They will list which courses and projects fit each role. For now, an application-focused LLM engineer follows this course, then RAG and Agents. A platform-leaning engineer adds Phases 11 to 13 in depth plus Docker, CI/CD and Observability. A research-leaning path needs machine learning and deep learning, which are outside this course.
- 6
Portfolio: evidence, not claims. Three pieces are enough: the capstone with a README, diagram and eval report; a short write-up with numbers about one cost, latency or reliability improvement you measured; and a small tool or contribution that others can run. Reviewers trust numbers and failure stories more than a list of buzzwords. Keep the eval set and the model-change habit alive: when a provider updates a model, re-run your set and write down what moved.
Explain it without notes
How would you decide what to study after finishing this course?
What should be in an LLM engineering portfolio, and why?
How do you keep your skills current when models change monthly?
Practice
Write next_steps(weak_areas) that maps the weak areas retrieval, tools and operations to the coming RAG and Vector Search course, the coming AI Agents course and the Observability course, and ignores other areas. Show it on three areas.
Write readiness(scores) that turns a list of 0 to 3 ratings into an integer percentage of the maximum. Show it on the seven ratings 3, 2, 2, 1, 1, 2, 0.
Trade-offs
- ↔
Depth in one area gets you hired for that area; breadth helps you lead a whole system. Early on, build depth after you have the breadth from this course.
- ↔
Waiting for the planned courses delays you; building a project now teaches you what to look for in them.
- ↔
A public portfolio shows your work but needs care about data and keys; publish only what you may share and keep secrets out of the history.
Done when you can
Done when you have rated yourself honestly in all seven areas.
Done when you can name your chosen track and your first project.
Done when you can say which Hectal courses fit each track and which are still planned.
Done when your portfolio plan has three pieces, each with a number.
Done when you have a monthly habit for re-running your evals on the current model.