Command Palette

Search for a command to run...

Hectal
Phase 2Beginner3 of 15 in LLM Engineering

Calling a Model

A model call is an HTTPS request with JSON. Build it by hand, read the reply, keep keys safe, classify errors, set timeouts and hide providers behind one interface.

Every SDK, framework and agent product ends in the same thing: an HTTPS POST with a JSON body, and a JSON reply. This phase builds that request and parses that reply by hand, against a small fake server that runs in your browser, before any library is shown. Once you can read the raw request and response, SDK code stops being magic.

Then it covers what turns a demo call into a dependable one: how messages and roles make a conversation (the model remembers nothing), how to read finish_reason and usage defensively, where API keys may and may not live, what each HTTP error status means and which are worth retrying, why every call needs a timeout, and how to hide the provider behind a small interface so tests are cheap and switching providers is a config change.

Real-model labs let you send requests to a free hosted model and read the raw JSON. Real HTTP and SDK code (curl, Python, Java, JavaScript, the OpenAI-compatible SDKs, Anthropic, Vercel AI SDK, Spring AI, LangChain4j) is shown but not run, because it needs a key and a network. The phase ends with a production-shaped model client in all three languages.

0/9 · 0%
9 topics ~3 h 22 code blocks & diagrams
Start with the first topic
1
2.1

The HTTP API Under Every SDK

Calling a model means sending an HTTPS POST with a JSON body (model name, messages, limits) and reading a JSON reply. SDKs wrap this request; knowing the raw shape lets you debug, switch providers and call any model from any language.

20 min 3 code practice

2
2.2

Messages and Roles

A chat is a list of messages, each with a role: system (rules), user (the person) and assistant (the model). The model remembers nothing between calls, so your program stores the list and sends all of it every time, trimming it when it grows too long.

21 min 2 code practice

3
2.3

Reading the Response

A reply is more than text: finish_reason says why the model stopped (finished, cut off, wants a tool, blocked) and usage says how many tokens you paid for. Read every field defensively, because any of them can be missing, null or empty.

19 min 2 code practice

4
2.4

API Keys and Secrets

An API key is a password that spends your money. Keep it in an environment variable on a server, never in browser code, the repository or logs, route browsers through your own endpoint, and rotate it the moment it leaks.

19 min 2 code practice

5
2.5

SDKs: What They Add

An SDK is a library that wraps the HTTP request for you: it builds the body, retries safe failures, parses the reply into typed objects and adds streaming helpers. Most providers copy the OpenAI shape, Anthropic uses its own, and frameworks add a common layer on top.

22 min 4 code practice

6
2.6

Errors and Status Codes

When a call fails, the HTTP status says whose problem it is: 4xx usually means your request needs fixing, 429 means slow down, 5xx means the provider is struggling. Classify every error so code retries only what can succeed and fails loudly on the rest.

21 min 2 code practice

7
2.7

Timeouts and Cancellation

A model call can take seconds or hang for minutes. Set a connect timeout, a read timeout and a total deadline on every call, pass the remaining time down, and cancel calls whose answer nobody needs any more.

21 min 3 code practice

8
2.8

One Interface, Many Providers

Give your code one tiny ChatModel interface and put every real client, fake and recorder behind it. Features then never learn which provider they use, tests need no key, and switching providers becomes a configuration change.

19 min 1 code practice

9
2.9

Playbook: A Production Model Client

Two recipes to copy: a model client with a key from the environment, a timeout on every call, error classification and retries with jitter, and a wrapper that writes one safe log line per call with tokens, latency and cost. Both sit behind the same complete interface.

23 min 3 code practice

Practise this phase

Hands-on exercises: write the code, run it, and press Check to have it tested automatically.

1

Build the request, read the reply

Write build_body(model, system, user, max_tokens) that returns the chat-completions request as a dictionary with keys model, messages (a system message first, then the user message) and max_tokens.

Beginner
2

Trim a conversation to a token budget

estimate_tokens(m) and total(messages) are given. Write trim(messages, budget) that returns a new list: it keeps every system message, then drops the oldest other messages until total is at most budget.

Beginner
3

Read any reply without crashing

Write summarise_reply(text) that parses a chat-completions reply and returns a dictionary with keys answer, finish and tokens.

Beginner
4

Retry only what can succeed

send(attempt) returns (status, headers, text) for the given attempt number (starting at 0). Write call_with_retries(send, max_attempts=3) that returns (text, waits) on status 200.

Intermediate
5

Redact API keys from log lines

Keys look like sk- followed by 8 or more letters or digits. Write mask(key) returning the first 3 characters, ... and the last 4 characters, and redact(text) that replaces every key in text with its masked form.

Beginner
6

Wrap any model in a recorder

FakeModel is given. Write a class Recorder whose constructor takes an inner model. Its complete(messages, max_tokens=100) calls inner.complete(messages, max_tokens) and returns the reply unchanged.

Intermediate