Day 11 of 14 · Your First AI App · 14 min

Evaluating AI Output

If you cannot measure whether it got better, you are only guessing

Every change to a , a model or a retrieval setting can make one thing better and another thing worse. Without measurement, you work blind. (evals) is for AI engineering what automated tests are for normal software.

What this lesson covers

  1. 01Build a golden set
  2. 02Three ways to score
  3. 03A tiny eval loop
  4. 04Make evals part of the routine
  5. 05A warning

Included with this lesson

  • 8narrated slides
  • 3storyboard panels
  • 3quiz questions

2 prompt templates

  • Golden Set Row

    A standard format for each evaluation example

    Members
  • Judge Rubric

    Criteria for a model-as-judge, to be validated against humans

    Members

Take the full lesson free

Enrol to unlock the complete lesson, the narrated deck, the templates and the quiz — and work towards your certificate.

Start the course — free