Command Palette

Search for a command to run...

Hectal
Phase 16Advanced17 of 18 in Python

Python in the AI Era

How Python powers AI work: the library map, calling models, tokens, streaming, structured output, RAG, agents, evals, ML basics, PyTorch and AI assistants.

Almost every AI product you use is Python on the inside: a thin layer of ordinary code around a model. A model is a program that has already learned from a huge amount of example data. This phase teaches the ordinary code around it from scratch, with the Python you already know. Every runnable example uses a fake model written in the example itself: a plain function that returns scripted text. We do this because a real model needs a paid account, a secret key and an internet connection, and none of those work inside this lesson page. The fake has the same shape as the real thing, so you see the real mechanics (messages, tokens, retries, tool calls, retrieval) with no API key and no cost. It does not show how smart a real model is.

Before you start. This phase leans on earlier topics. If you skipped ahead, go back for these first: functions (Topics 5.1 to 5.3), dictionaries (Topic 4.4) and nested data (Topic 4.8), JSON (Topic 7.7), dataclasses (Topic 8.8), generators (Topic 9.2), type hints (Topic 10.1), pytest and fakes (Topics 11.2 and 11.4), async and await (Topics 12.6 to 12.8), NumPy arrays and vectors (Topics 14.1 and 14.5), HTTP clients and retries (Topics 15.1 and 15.2) and Pydantic (Topic 15.4). Each topic below repeats the key idea in one line when it needs it, but those lines are reminders, not lessons.

You will meet the whole stack as a map, then build the pieces by hand: a client, a token budget, a stream, a validator, a prompt template, a vector search, a RAG pipeline, an agent loop and an eval. Real SDK code is shown in separate blocks you can copy later when you have a key. Doing it by hand first is what lets you debug any framework.

This is an honest introduction, not the whole subject. Each topic ends by naming the Hectal course that will go deeper: LLM engineering, RAG and agents, machine learning and deep learning are each planned as a full course. Security content here is defensive only: what prompt injection is, why retrieved and tool text is untrusted, and how to reduce the risk.

0/14 · 0%
14 topics ~6 h 45 code blocks & diagrams
Start with the first topic
1
16.1

The AI Python Landscape: Which Library Does What

AI work in Python is a stack of layers: NumPy and pandas for data, scikit-learn for classic machine learning, PyTorch for neural networks, Hugging Face for trained models, SDKs for hosted LLMs, frameworks and FastAPI on top. Knowing what each layer is for tells you what to learn first.

27 min 1 diagram 2 code practice

2
16.2

Jupyter Notebooks and the Experiment Workflow

A notebook is a document of code cells that share one running Python process, which makes it excellent for exploring data and risky for anything you need to repeat. The workflow is to explore in a notebook, then move the working code into modules with tests, seeds and locked dependencies.

25 min 1 diagram 2 code practice

3
16.3

Calling an LLM API from Python

Calling a language model is sending a list of messages (system, user, assistant) as JSON over HTTP and reading text back. Different providers use slightly different shapes, so you hide the call behind a small function and test everything else with a fake client.

27 min 3 code practice

4
16.4

Tokens, Context Windows, Cost and Rate Limits

Models read and write tokens, not characters or words. A context window caps how many tokens fit in one call, cost is a simple formula of input and output tokens times rates, and rate limits mean you must trim history and retry with backoff.

28 min 4 code practice

5
16.5

Streaming Responses with Generators and Async

Streaming sends a model's answer to you in small pieces as it is generated, so the user sees text appear instead of waiting for the whole reply. In Python this is a generator (or an async generator) that yields chunks, which you print, forward, collect and can cancel.

26 min 3 code practice

6
16.6

Structured Output: JSON, Schemas and Pydantic

Programs need data, not prose, so you ask a model for JSON and then treat its answer as untrusted input: parse it, validate it against a schema with Pydantic, and if it is invalid send the error back and retry a limited number of times.

27 min 4 code practice

7
16.7

Prompts as Code: Templates, Few-Shot and Versioning

A prompt is a program written in words, so treat it like code: build it from templates, keep few-shot examples as data, version it in Git, and keep untrusted text clearly separated from your instructions. Prompt injection is the risk when that separation fails.

27 min 3 code practice

8
16.8

Embeddings and Similarity Search with NumPy

An embedding turns text into a vector so that texts with similar meaning end up close together. With NumPy you can score a query against every stored vector with one matrix multiplication and take the top results, which is the heart of semantic search.

27 min 3 code practice

9
16.9

Retrieval-Augmented Generation from Scratch

RAG answers questions from your own documents: split them into chunks, embed them, retrieve the chunks closest to the question, put them in the prompt as numbered sources, and have the model answer and cite them. It is a few dozen lines of Python, and most failures come from bad chunks and missing context.

28 min 3 code practice

10
16.10

Tool Calling and an Agent Loop from Scratch

Tool calling lets a model ask your code to run a function: you describe tools with JSON schemas, the model replies with a tool name and arguments, your code validates and runs it, and the result goes back. An agent is that exchange in a loop with a step limit, an allow-list and human approval for risky actions.

28 min 3 code practice

11
16.11

Testing and Evaluating AI Code

Model output is not deterministic, but the code around it must be testable. Use fakes and recorded replies for unit tests, an eval set with scoring functions to measure quality, regression checks to catch drops, and logs of prompts and replies to debug what happened.

26 min 3 code practice

12
16.12

Classic Machine Learning in One Sitting: scikit-learn

Machine learning fits a rule to example data. You split the data into train and test sets, fit a model on the training part, predict and score on the test part, and watch for overfitting. You can build a nearest-neighbour classifier and a polynomial fit from scratch with NumPy before using scikit-learn.

26 min 4 code practice

13
16.13

Tensors and PyTorch: A First Look

A tensor is a NumPy-like array that can run on a GPU and remember how it was computed, so PyTorch can find gradients automatically. Training a neural network is five repeated steps: forward pass, loss, gradients, update, repeat. You can run the whole idea with plain NumPy first.

27 min 3 code practice

14
16.14

Writing Python with AI Assistants

AI coding assistants such as Copilot, Cursor, Claude Code and Codex write plausible Python quickly, and plausible is not the same as correct. Give them context, ask for tests, review what they produce, and verify with tests, type checks and Ruff before you trust it.

26 min 3 code practice

Practise this phase

Hands-on exercises: write the code, run it, and press Check to have it tested automatically.

1

Trim a chat history to a token budget

Write trim_to_budget(messages, budget). The first message is the system message and is always kept. Then keep the newest messages that fit, so the total token count of the kept messages is at most budget.

Intermediate
2

Parse and validate model JSON, with retries

Write parse_with_retry(model, prompt, max_attempts=3). model is a function that takes a prompt string and returns the model's reply text.

Advanced
3

Retrieve the top-k most similar vectors

Write top_k(query, vectors, k). query is a list of numbers, vectors is a list of lists. Return the indices of the k vectors with the highest cosine similarity to the query, best first.

Intermediate
4

Build a RAG prompt with numbered sources

Write build_rag_prompt(question, sources) that returns one string with these lines, in order: an instruction line, then one line per source in the form [1] text, [2] text and so on, then the last line Question: <question>.

Beginner
5

Run one tool call safely

Write run_tool_call(call, tools). call is a dict with "name" and "arguments" (a JSON string, as in the OpenAI-compatible API). tools maps a name to a pair (function, required_argument_names), where the second part is a list of argument names. The model asked for this call, and your code decides how to run it (Topic 16.10).

Advanced
6

Score a model on an eval set

Write score_model(model, cases). Each case is a dict with "prompt" and "must_contain" (a list of words). A case passes if the model's reply contains every word, ignoring case. model is a function that takes a prompt string and returns the reply text. Return the share of passing cases as a number from 0.0 to 1.0. An empty list of cases gives 0.0. (Evals: Topic 16.11.)

Intermediate