Command Palette

Search for a command to run...

Hectal
Back to the lesson: Topic 11.2 — Logging Prompts and Responses Safely
LLM Engineering · Example 2 of 2

Sampling and retention

Seeded generator: state = state * 48271 % 2147483647. Errors and timeouts are always kept, healthy calls are kept 1 time in 4.

Prompts and replies are the most useful thing to log and the most dangerous: they contain names, emails, secrets and private documents. Safe logging uses structured lines, a redactor before the write, sampling, and a retention limit.

Change the code and press Run (Ctrl+Enter). Predict what will happen first, then break it on purpose and read what changes. Your edits are saved and match the lesson page.

Ready for more? This phase has 5 hands-on exercises with automatic checks.

Practice questions

Write the code in the editor, run it, then open the model answer to compare.

01

Write mask_email(text) that replaces each email with its first letter, *** and the domain, so ana@example.com becomes a***@example.com. Use re.sub with a function.

02

Write is_expired(age_days, kept_days) and purge(logs, kept_days) where logs is a list of (request_id, age_days); purge returns the ids of logs to delete (older than kept_days).

Explain it without notes

01

Why must redaction happen before the log line is written, not when someone reads the log?

02

Why do you always keep failed calls but only sample healthy ones?

03

What are the limits of pattern-based redaction, and what do you do about them?

Sampling and retention
Sign in to run this example in your browser.

Expected output

r1 ok: keep (draw 0.001)
r2 ok: drop (draw 0.571)
r3 error: keep (error: always keep)
r4 ok: drop (draw 0.448)
r5 ok: drop (draw 0.654)
r6 timeout: keep (error: always keep)
r7 ok: drop (draw 0.629)
r8 ok: drop (draw 0.716)
kept 3 of 8 full-text logs
deleted by the retention rule: r6