A11Advanced 60 minutes reading and codeLocked

Eval-Driven Agent Development

Anthropic's guidance: "start with simple prompts, optimize them with comprehensive evaluation, and add multi-step agentic systems only when needed." Evaluation is not the last step. It's the outer loop that drives every other choice in Phase 14.

In this lesson

  1. 01Learning Objectives
  2. 02The Problem
  3. 03The Concept
  4. 04Build it
  5. 05Use it
  6. 06Exercises
  7. 07Key Terms
  8. 08Further Reading

About 1000 words of reading, with code, 7 quiz questions and the tutor.

The full lesson opens with an access code

You can see what this lesson covers and the milestone it sets. The full text, code, quizzes, the tutor and the AI reviews come with the K850 practice, along with weekly build sessions and code review.

Codes come with the K850 practice. Book a place

Book the K850 practiceRead an open lesson

Milestone forAn observable back-office agent

Grade trajectories, not just answers

Label 100 statement lines with the correct match or escalation. Score whole trajectories: right tools, right order, right stop, right approval request. Every change to prompts or tools runs this suite.

Done when

  • 100 labelled statement lines with expected trajectories (use the data pack)
  • Trajectory scoring: tools, order, stop and approval, not only final answers
  • The suite runs on every change and blocks merges below your threshold

Stretch: Track trajectory score over ten commits and plot it

Reviews open with an access code.

Interview practice

Questions on this topic that AI engineering interviews ask, with the companies reported to ask them. Answer the way you would out loud; Jev scores it and DeepSeek tells you what to add.

Scoring opens with an access code. You can still read the questions and prepare.

A10Prompt Injection and the PVE DefenseNext · A12Production Runtimes: Queue, Event, Cron

Adapted from open-source work: AI Engineering from Scratch by Rohit Ghumare (MIT, lesson text, code and quizzes); Awesome LLM Apps by Shubham Saboo (Apache-2.0, starter code); AI Engineering Interview Questions by Outcome School (Apache-2.0, interview questions). Capstones, milestones, data packs and Zambian context by Zambrite.

Chat on WhatsApp