Command Palette

Search for a command to run...

Hectal
Phase 9: Evaluation and Testing
LLM Engineering · Exercise 6 of 6 Previous
Advanced

A judge that survives swapping

Write fair_verdict(judge, answer_a, answer_b). judge(first, second) returns "first" or "second": the position it prefers.

Call the judge twice: once with A first and once with B first. Return "A" or "B" when both calls pick the same answer, and "tie" when they disagree.

What Check tests

  1. a judge that really prefers one answer gets a verdict
  2. a judge that always prefers the first position gives a tie
  3. a judge that always prefers the second position gives a tie
  4. the judge is called exactly twice

Stuck? One hint at a time

The solution unlocks after your first Check. Try first: that is where the learning happens.

main.py
Sign in to run your code.

Output

Press Run to see what your program prints.