Lesson 10 · Domain 4 — Prompt Engineering & Structured Output (20% of exam)

Explicit Criteria & Few-Shot Prompting

New domain. Domain 4 is about getting reliable, precise output from Claude — starting with the most common failure mode: instructions that sound reasonable but don't actually constrain behavior.

Vague instructions don't reduce false positives

"Be conservative." "Only report high-confidence findings." These sound like precision controls. They aren't — a model has no stable, calibrated internal notion of "conservative" to dial up or down. What actually works is explicit, categorical criteria: not "check that comments are accurate," but "flag a comment only when the claimed behavior contradicts the code's actual behavior." One is a vibe. The other is a test the model can apply consistently.

This matters more than it sounds like it should, because false positives are contagious to trust: once a reviewer catches a category producing noise, they start discounting every category's findings, including the accurate ones. If one category is the clear offender, the proportionate response is to disable that specific category temporarily while you sharpen its criteria — not to loosen the whole system's confidence threshold, and not to ignore it either.

Few-shot: for consistency, and for judgment

Detailed prose instructions alone often produce inconsistent formatting or inconsistent handling of edge cases. Few-shot examples fix both — but their real value is the second one: 2–4 well-chosen examples showing reasoning for ambiguous cases teach the model to generalize that judgment to genuinely novel situations, not just to match the examples verbatim.

const fewShotExamples = [
  {
    role: "user" as const,
    content: "Comment: 'Cache TTL is 5 minutes.' Code: TTL = 300 // seconds",
  },
  {
    role: "assistant" as const,
    content: "No issue — 300 seconds equals 5 minutes; the comment is accurate.",
  },
  {
    role: "user" as const,
    content: "Comment: 'Cache TTL is 5 minutes.' Code: TTL = 30 // seconds",
  },
  {
    role: "assistant" as const,
    content: "Issue: the comment claims 5 minutes, but the code sets 30 seconds.",
  },
];

const messages = [...fewShotExamples, { role: "user" as const, content: newComment }];

The same technique is what makes structured extraction more reliable on messy real-world documents — showing correct handling of varied structures (inline citations vs. a bibliography, informal measurements vs. precise ones) reduces hallucination when the model hits a document shape it hasn't seen described in words.

Why this matters beyond the exam

Primary source: Anthropic's Claude API docs cover prompt engineering technique directly, including few-shot design — this lesson compresses Task Statements 4.1 and 4.2.

Check your understanding

Want help turning a vague instruction you're currently using into explicit, testable criteria? Share it and we'll rework it together.