Command Palette

Search for a command to run...

Hectal
Phase 6: Cost, Latency and Limits
LLM Engineering · Exercise 2 of 6 PreviousNext
Beginner

Cache answers with a key and an expiry

Write make_key(model, prompt, temperature) that returns "<model>|<temperature to one decimal>|<prompt with runs of spaces collapsed and the ends trimmed>". For example make_key("m", " Hi there ", 0) is "m|0.0|Hi there".

Write the class TtlCache(ttl_ms) with put(key, value, now) and get(key, now). A value put at time t is valid while now < t + ttl_ms. get returns None for a missing or expired key.

What Check tests

  1. The key joins model, temperature and the normalised prompt
  2. A different temperature gives a different key
  3. A value is found before it expires
  4. A value is gone at exactly t + ttl
  5. A missing key gives None

Stuck? One hint at a time

The solution unlocks after your first Check. Try first: that is where the learning happens.

main.py
Sign in to run your code.

Output

Press Run to see what your program prints.