Command Palette

Search for a command to run...

Hectal
Phase 4: Structured Output
LLM Engineering · Exercise 5 of 6 PreviousNext
Intermediate

Classify with unknown and a confidence threshold

Write decide(reply, labels, threshold). reply is the model's text, labels is the list of allowed labels and threshold is an integer.

Parse the reply as JSON with the keys label and confidence. Return the label only if the reply is valid JSON, the label is in labels, the confidence is an integer (not a boolean) and it is at least threshold. In every other case return "unknown".

What Check tests

  1. A good reply returns its label
  2. Low confidence gives unknown
  3. Exactly at the threshold is accepted
  4. A label outside the list gives unknown
  5. Bad JSON or a bad confidence gives unknown

Stuck? One hint at a time

The solution unlocks after your first Check. Try first: that is where the learning happens.

main.py
Sign in to run your code.

Output

Press Run to see what your program prints.