Tasks / Product Craft

Customer research call guide

Can the model write a guide that uncovers behaviour rather than opinions?

Task type v1.0 · 2 tasksLast changed 6 Oct 2026 · ChangelogDifficulty

What AI gets right here, and what you’ll still have to catch

From 18 graded outputs by 9 models. 33% were usable with at most a quick edit.

Reliably right

  1. Asks what people did, not what they think96% pass
    Core questions ask for specific recent actions and events ('What happened next?', 'Who did you call?'), not opinions or predictions.
    Sonnet 5.5 · API · Why are warehouses leaving? Two execs, two theories
  2. Tests both theories fairly92% pass
    Both theories get questions that could disprove them (e.g., no action on robotics, happiness after slow go-live), and open-ended timeline questions leave room for a third cause.
    Sonnet 5.5 · API · Why are warehouses leaving? Two execs, two theories
  3. Different questions for user and signer92% pass
    The guide has distinct tracks: daily user questions focus on day-to-day experience and workarounds, signer questions on the decision process and alternatives.
    Sonnet 5.5 · API · Why are warehouses leaving? Two execs, two theories

Where it slips

  1. Marks what to cut if the call runs over28% pass
    The guide includes timed sections that add up to 45 minutes but does not say what to cut if time runs short, nor does it mark must-ask questions.
    GPT-6.1 Sol · API · Why are warehouses leaving? Two execs, two theories
  2. Doesn't trust the default reason47% pass
    The plan names the pre-selected default as a bias but does not use the behaviour columns to show that most 'too expensive' cancellers had stopped cooking; it only picks two contrasting individuals without the aggregate pattern.
    GPT-6 Astra · ChatGPT · Too expensive, by default
  3. Uses the supplied evidence correctly61% pass
    The claim 'Customers will say what helps their negotiation' is an unsupported behavioural statement not derived from the context.
    Haiku 5.5 · API · Why are warehouses leaving? Two execs, two theories

How it’s graded

Each check is built on how world-class product and growth leaders do this work, as they’ve described it on Lenny’s Podcast, and names the leader it comes from.

  1. Designed to falsify

    For each learning goal, does the guide include a question whose honest answer could show the team's hypothesis is wrong?

    Passes when Every learning goal has at least one disconfirming, non-leading question, such as asking about the last time they had the problem and what they did instead.

  2. Asks what people did, not what they think

    Are the core questions about specific things the person actually did recently, rather than opinions, predictions or hypotheticals?

    Passes when The core of the guide asks for recent, specific instances (the last time, what happened, what they did) and follows up on them; opinions and predictions are absent or clearly secondary.

  3. Doesn't lead the witness

    Are the questions free of leading wording, of pitching the idea, and of hints at the answer the team hopes for?

    Passes when Questions are neutral and open, and the idea is not described (or only at the very end, after the problem questions).

Plus our standard checks

Uses the supplied evidence correctly · Addresses the actual decision · Respects explicit constraints · Identifies material uncertainty · Avoids unsupported claims · Produces the required deliverable · Marks what to cut if the call runs over · and 2 written for each task, which you’ll see in the tasks below.

The tasks

Read the brief, then compare up to three outputs, each with the LLM judge’s verdict on every check. Highlights mark what a PM had to fix.

The brief

You're the PM for retention at Sprig. The execs each have a theory about why subscribers cancel (below). Plan next week's six churn interviews: who to recruit from the cancellation list and why, and the guide the interviewers will use, in no more than 900 words. Two of the interviewers are new to it. The files are attached.

What the model was given5 items: About Sprig, exec_hypotheses.md, cancel_form.md, cancellations_last_quarter.csv, research_constraints.md
About SprigA meal-kit subscription: recipes and ingredients delivered weekly. Subscribers can skip a week or cancel any time.
exec_hypotheses.md7 lines · Download
# What the execs think (for the research)

- CEO: "It's price. Competitors discount harder."
- Head of Food: "Our recipes are too repetitive after a few months."
- Head of Ops: "Late deliveries."

Retained subscribers (for comparison): 81% cooked at least 3 recipes a week in their first month; 34% of cancellers did.
cancel_form.md3 lines · Download
# The cancel form

A single-choice list, in this order: "Too expensive" (selected by default), "Recipes got repetitive", "Moving / travelling", "Delivery problems", "Other". Free text is optional.
cancellations_last_quarter.csv300 rows · Download
cancellation,cancel_reason_selected,weeks_subscribed,deliveries_skipped_last_6_weeks,recipes_cooked_last_4_weeks,free_text
S-8000,Recipes got repetitive,47,3,0,kept piling up in the fridge
S-8001,Recipes got repetitive,16,3,2,too much food for two
S-8002,Moving / travelling,33,2,5,cheaper at the supermarket
S-8003,Too expensive (pre-selected),29,3,2,kept piling up in the fridge
S-8004,Too expensive (pre-selected),11,5,0,
S-8005,Too expensive (pre-selected),20,5,0,too much food for two
…
research_constraints.md4 lines · Download
# Research constraints

- Six calls, 30 minutes each, next week. The researcher can recruit from the cancellation list (consent to contact is on file for all of them).
- Two of the PMs running calls are new to interviewing.
What a strong answer doesThe answer key the graders mark against

Reads the cancellation data before trusting it: 'Too expensive' is the top reason (156 of 300), but it's the pre-selected first option, and 118 of those 156 had skipped two or more deliveries and cooked two or fewer recipes in the last four weeks: they'd stopped using the product before they left. Overall 204 of 300 skipped two or more deliveries. So the guide investigates what happened to the cooking habit, and price as one hypothesis to test rather than the answer. Recruits deliberately from the list across the patterns: people who stopped cooking and picked 'too expensive', people who kept cooking and still picked 'too expensive' (the real price cases), and people who said recipes got repetitive. The guide asks about specific recent weeks (the last box they skipped, what happened to the food, what they ate instead), each exec hypothesis has a question whose honest answer could disprove it, questions don't lead or mention price first, the guide fits 30 minutes with timings and marks what to cut, and gives the new interviewers prompts for digging deeper. Notes that the default on the cancel form should be removed.

Critical failures (cap the score at 40)
  • Builds the guide around confirming that price is the reason
Task

Staff-level · v1.2

The outputs

9 setups answered this brief.

Needs more than a quick edit8 of 12 checks passed

The 79% statistic is incorrect and would corrupt the analysis; a PM would need to correct it before using the output.

Output

Sprig Churn Research Plan & Interview Guide

Why Our Current Data is Misleading

Before recruiting, we must look at how our exit survey distorts reality: 1. Default Bias: "Too expensive" is pre-selected. Half our cancellers leave it checked, but their free-text responses and behavioral data tell a different story: 79% of cancellers cooked 3 or fewer recipes in their final 4 weeks, and most skipped multiple deliveries. 2. The Value-Perception Trap: When food goes uncooked ("kept piling up in the fridge", "didn't have time to cook", "too much food for two"), the effective cost-per-meal skyrockets. Subscribers aren't necessarily leaving because of sticker price or competitor discounts; they are leaving because paying for food you end up throwing away feels expensive.

Our 6 interviews will test the execs' hypotheses (Price, Recipe Fatigue, Delivery) against this underlying Utilization/Lifestyle Fit problem.

---

Recruitment Plan (6 Participants)

Note: Invite 2–3 backups per slot using the backup IDs provided.

1. The "Default Price" Low-Utilizer (Tests CEO Hypothesis vs. Waste) * Recruit: S-8016 (Backup: S-8029) — 14 weeks, "Too expensive (pre-selected)", skipped 4/6, cooked 0, free text: "kept piling up in the fridge". * Why: Represents our largest churn segment. Left the default reason checked, but stopped cooking entirely and threw food away. Tests if "price" actually means "wasted food."

2. The High-Usage Price Chiller (Tests True Price Sensitivity) * Recruit: S-8056 (Backup: S-8096) — 29 weeks, "Too expensive (pre-selected)", skipped 1/6, cooked 12, free text: "cheaper at the supermarket". * Why: Cooked 3 meals/week right up to cancellation (matching our retained user benchmark) with zero waste, yet still felt it was too expensive. Tests the CEO's hypothesis on a user who actually got full utility.

3. The Long-Tenure "Bored" Subscriber (Tests Head of Food Hypothesis) * Recruit: S-8057 (Backup: S-8246) — 53 weeks, "Recipes got repetitive", skipped 4/6, cooked 3, no free text. * Why: Actively changed the dropdown to "Repetitive" after a year of loyalty and recent skipping. Tests genuine menu fatigue vs. seasonal routine changes.

4. The Time-Poor "Repetitive" Subscriber (Tests Menu Fatigue vs. Cooking Effort) * Recruit: S-8022 (Backup: S-8013) — 20 weeks, "Recipes got repetitive", skipped 4/6, cooked 3, free text: "didn't have time to cook". * Why: Selected "repetitive" at 5 months, but wrote they had no time. Did the flavors get boring, or did the 45-minute prep work become a chore?

5. The "Too Much Food" Down-Sizer (Tests Plan Flexibility) * Recruit: S-8006 (Backup: S-8240) — 29 weeks, "Too expensive (pre-selected)", skipped 0/6, cooked 8, free text: "too much food for two". * Why: Consistently cooked 2 meals/week without skipping, but felt overwhelmed by volume. Tests if inflexible portion/box sizes are driving "price" churn.

6. The Early-Tenure Ops Failure (Tests Head of Ops Hypothesis) * Recruit: S-8097 (Backup: S-8152) — 2 weeks, "Delivery problems", skipped 2/6, cooked 10, no free text. * Why: High intent (cooked 10 meals in a very short window) but churned immediately citing delivery. Isolates pure operational failure during the critical first-month habit-building window.

---

30-Minute Interview Guide

Golden Rules for New Interviewers

  • Don't ask about the future or hypotheticals: Never ask "Would you have stayed if it were 20% cheaper?" (People always say yes). Ask what actually happened.
  • Ask for stories, not opinions: Instead of "How were the recipes?", ask "Walk me through the last box you unpacked."
  • Don't lead the witness: Do not mention their survey selection ("I see you cancelled because of price"). Let them bring it up.
  • Embrace silence: Count to 5 in your head after they finish speaking. They will often fill the silence with the real truth.

1. Intro & Warm-up (3 mins)

  • "Hi [Name], thanks for joining. I'm [Name] on the product team at Sprig. We're talking to former subscribers to understand how Sprig fit into your life and where it fell short. There are no right or wrong answers—criticism won't hurt my feelings, it actually helps us improve. Do you mind if I record this just for internal notes?"

2. The Origin & Early Habit (5 mins)

  • Goal: Understand the job they hired Sprig to do and their first-month habit.
  • "Take me back to when you first signed up for Sprig. What was going on in your life that made you want to try it?"
  • "In that first month, what did a typical week with a Sprig box look like?"

3. The Last Box (10 mins) — CRITICAL SECTION

  • Goal: Uncover the behavioral reality of food waste, prep time, and portion sizes.
  • "Think about the very last Sprig box that arrived at your door. Walk me through what happened to that box from the moment you brought it inside."
  • Follow-up: "How many of those meals did you end up cooking?"
  • If meals went uncooked: "What got in the way of cooking the others? What ended up happening to those ingredients?" (Listen for: guilt of throwing away food, fridge clutter, unexpectedly busy evenings).
  • "Looking back at the last month or two before you cancelled, I noticed you skipped a few weeks. How did you decide whether to get a box or skip?"

4. The Trigger & Executive Probes (9 mins)

  • Goal: Pinpoint the exact moment of cancellation and test exec hypotheses without leading.
  • "What was the exact moment you decided, 'Okay, I'm going to cancel my subscription today'?"
  • Use the relevant probe below based on what they bring up (or their profile):
  • If they mention Price / Value: "When you felt it wasn't worth the cost, how were you doing that math? Did you switch to another meal kit, or what are you doing for dinners now?"
  • If they mention Recipes / Food: "When you logged in to pick recipes toward the end, what was going through your head? Was it the flavors, the ingredients, or the time/effort to cook them?"
  • If they mention Volume / Leftovers: "How did the amount of food in the box match up with what your household actually eats?"
  • If they mention Delivery: "Tell me specifically what happened with the delivery. How did that impact your plans for the week?"

5. Wrap-up (3 mins)

  • "If a friend asked you today whether they should try Sprig, what would you tell them?"
  • "Is there anything else about your experience we didn't cover? Thank you so much for your time!"

Claims the judge couldn’t find in the brief

  • 79% of cancellers cooked 3 or fewer recipes in their final 4 weeks

Grades and run details

Decision model 67 · LLM judge 62
Decision model checks
  • failUses the supplied evidence correctly18%
  • partialAddresses the actual decision41%
  • passRespects explicit constraints18%
  • partialIdentifies material uncertainty31%
  • failAvoids unsupported claims19%
  • partialProduces the required deliverable25%
  • passDoesn't trust the default reason83%
  • passRecruits from the data100%
  • passDesigned to falsify52%
  • passAsks what people did, not what they think98%
  • passDoesn't lead the witness49%
  • partialMarks what to cut if the call runs over34%
Run
Run
#1
API response time
21 s
Submitted
13 Oct 2026

Needs more than a quick edit7 of 12 checks passed

The output invents a 'discount cliff' not present in the data and lacks must-ask/cut markings, so a PM would need to verify evidence and add cutting instructions.

Output

Strategic Recruitment Plan

The cancellation form pre-selects "Too expensive", severely skewing quantitative data toward price. Furthermore, while 81% of retained users cook ≥3 recipes/week in month 1, only 34% of churned users do—suggesting retention is fundamentally won or lost on habit formation, pace, and food waste guilt (evident in recurring free-text notes like "kept piling up").

To evaluate exec hypotheses (Price, Recipe Repetition, Delivery) against these underlying behavioral dynamics, we will recruit six subscribers across distinct behavioral archetypes:

IDWeeks SubscribedSkips (Last 6 Wks)Cooked (Last 4 Wks)Form ReasonFree TextRationale & Hypothesis Tested
:---:---:---:---:---:---:---
S-805629112Too expensiveCheaper at supermarketCEO (Price): High cooking engagement, low skips. Tests whether high-value cooks leave for discounted competitors vs. grocery shopping.
S-81965721Repetitive[None]Head of Food (Repetition): Long tenure; actively changed radio button to "Repetitive"; cooking volume collapsed. Tests recipe fatigue threshold.
S-80722318Delivery problems[None]Head of Ops (Delivery): High cook volume; deliberately chose "Delivery problems". Tests if logistical failures break established habits.
S-80161440Too expensiveKept piling up in fridgeThe "Food Guilt" Trap: High skips, zero cooking. Tests if "Too expensive" is actually guilt over uneaten food and subscription velocity.
S-80062908Too expensiveToo much food for twoPortion/Household Mismatch: Highly engaged, zero skips, but churned. Tests unit economics and portion sizing vs. true price sensitivity.
S-81204010Too expensiveCheaper at supermarketEarly Onboarding Drop-off: Active cook (10 recipes) who dropped at Week 4 discount cliff. Tests initial value proposition and promo-hopper behavior.

Backup recruits: S-8044 (Repetitive), S-8115 (Delivery), S-8038 (Price/High engagement).

---

Interviewer Guide (30 Minutes)

> Interviewer Note (for new PMs): > 1. Never mention their cancel form response. It was pre-selected and biases answers. > 2. Focus on stories, not opinions. Don't ask "Was it too expensive?" Ask "Walk me through the last box you unpacked." > 3. Embrace 3 seconds of silence. Let the participant fill pauses.

#### 1. Welcome & Rapport (3 mins) * "Thanks for taking the time! I’m [Name], a product manager at Sprig. We’re working to make our boxes better, and we learn the most from people who decided Sprig wasn't right for them." * "There are no right or wrong answers, and you won’t hurt my feelings. I’m recording this for notes—is that okay?"

#### 2. The Early Days & Habit Formation (7 mins) Goal: Understand the baseline expectation and initial activation (testing month-1 cooking habit). * "Think back to when you first signed up for Sprig. What was going on in your life that made you want meal kits?" * "Walk me through your first couple of weeks. How often were you cooking the meals?" * "Did you cook with someone else? How did the portion sizes fit your household?"

#### 3. The Turning Point & Kitchen Reality (12 mins) Goal: Neutral deep dive to evaluate price, delivery, repetition, and food waste without leading. * "Think about the last box you received before canceling. Walk me through what happened from the moment it arrived to when you decided to cancel." * If Ops signal: "Did it arrive when and how you expected? How did that affect your week?" * If Food signal: "How did picking your meals feel toward the end compared to the beginning? Did you feel excited by the menu?" * If Fridge/Time signal: "Did you ever have ingredients left over when the next box arrived? How did you handle that?" * "I noticed you skipped a few deliveries [or never skipped]. How did skipping fit into your routine?" * "Tell me about the specific moment you went to the website to hit 'Cancel'. What tipped you over the edge that day?"

#### 4. The Alternative & True Cost (5 mins) Goal: Test CEO hypothesis (competitors vs. grocery vs. takeout). * "Now that you aren't using Sprig, what are you doing for dinner on busy weeknights?" * "Did you try or consider any other meal kits or services? Why or why not?" * "When you think about the total cost of Sprig, what felt like the biggest driver of whether it was 'worth it'?"

#### 5. Wrap-Up (3 mins) * "If you had a magic wand and could change one thing about Sprig so that it fit your life perfectly, what would it be?" * "Thank you so much for your candor. We’ll send your $30 gift card within 24 hours."

---

Post-Interview Debrief Checklist (Immediately after call)

Capture within 10 minutes in Slack/Notion: 1. Primary root cause: (e.g., Recipe boredom, Food waste guilt, Box delivery timing, Absolute price, Household mismatch). 2. True competitor: (e.g., Trader Joe’s, HelloFresh, Takeout, Cooking from pantry). 3. Exec hypothesis validation: Did this support CEO, Food, or Ops? (Yes/No/Partial + 1-sentence quote).

Claims the judge couldn’t find in the brief

  • S-8120 cooked 10 recipes in 4 weeks and dropped at Week 4 discount cliff.

Grades and run details

Decision model 63 · LLM judge 54
Decision model checks
  • failUses the supplied evidence correctly16%
  • partialAddresses the actual decision41%
  • partialRespects explicit constraints11%
  • partialIdentifies material uncertainty53%
  • partialAvoids unsupported claims25%
  • partialProduces the required deliverable23%
  • partialDoesn't trust the default reason69%
  • passRecruits from the data100%
  • passDesigned to falsify45%
  • passAsks what people did, not what they think79%
  • passDoesn't lead the witness45%
  • partialMarks what to cut if the call runs over58%
Run
Run
#1
API response time
34 s
Submitted
6 Oct 2026

Check by check

Where the outputs differ comes first. Open a check to read the judge’s reasons. “Mixed” means the two graders disagreed.

Where they differ 1

Addresses the actual decisionRightWrong
Gemini 4 Argon · API

Commits clearly to a six-participant recruitment plan and a complete interview guide.

Gemini 3.8 Flash · API

The output does not state what result or condition would change the recruitment choice.

All got wrong 4

Uses the supplied evidence correctlyWrongWrong
Gemini 4 Argon · API

The output claims 79% of cancellers cooked ≤3 recipes, but the actual figure is 73%.

Gemini 3.8 Flash · API

The claim about a 'Week 4 discount cliff' for S-8120 is invented; no discount information exists in the provided data.

Identifies material uncertaintyWrongWrong
Gemini 4 Argon · API

Does not name specific unknowns or say what would change the call if results contradict assumptions.

Gemini 3.8 Flash · API

The output does not name specific unknowns that could change the decision or say how they would be resolved.

Avoids unsupported claimsWrongWrong
Gemini 4 Argon · API

The 79% figure is an unsupported claim not backed by the cancellations list.

Gemini 3.8 Flash · API

The 'Week 4 discount cliff' is an unsupported interpretation presented as fact, and no caveat is given.

Marks what to cut if the call runs overWrongWrong
Gemini 4 Argon · API

Timings sum to 30 minutes but the guide does not mark which sections to cut if time runs short.

Gemini 3.8 Flash · API

Timings add up to 30 minutes but the guide does not mark which questions are must-ask or what to cut if time runs over.

All got right 7

Respects explicit constraintsRightRight
Gemini 4 Argon · API

Stays within 900 words, addresses the reader, and includes advice for new interviewers.

Gemini 3.8 Flash · API

The output respects the 900-word limit, provides a recruitment plan and interview guide, and includes a note for new interviewers.

Produces the required deliverableRightRight
Gemini 4 Argon · API

Delivers a complete recruitment list and timed interview guide usable by the PM.

Gemini 3.8 Flash · API

It delivers a ready-to-use recruitment table and a timed interview guide for the retention PM.

Doesn't trust the default reasonRightRight
Gemini 4 Argon · API

Acknowledges the pre-selected default bias and shows that many who selected it had stopped cooking.

Gemini 3.8 Flash · API

It explicitly names the pre-selected bias and uses behavioural data to show many 'too expensive' cancellers had stopped cooking.

Recruits from the dataRightRight
Gemini 4 Argon · API

Defines six behavioural segments with specific IDs, testing each hypothesis.

Gemini 3.8 Flash · API

Six distinct behavioural archetypes are drawn from the data, each linked to an exec hypothesis.

Designed to falsifyRightRight
Gemini 4 Argon · API

Each hypothesis has a probe that could yield a disconfirming honest answer (e.g., on time, waste, delivery).

Gemini 3.8 Flash · API

Each hypothesis has a question whose honest answer could disprove it, e.g. asking about what they do for dinner now instead.

Asks what people did, not what they thinkRightRight
Gemini 4 Argon · API

Core questions ask about the last box, specific weeks, and actions rather than opinions or predictions.

Gemini 3.8 Flash · API

Core questions ask about specific recent events like the last box received and the moment of cancellation.

Doesn't lead the witnessRightRight
Gemini 4 Argon · API

Questions are open and neutral, price is not mentioned first, and interviewers are warned not to lead.

Gemini 3.8 Flash · API

The guide instructs interviewers not to mention the cancel form and questions are neutrally worded.

Results

Every setup we’ve tested on this task type, across all its tasks and repeats, graded on the current checklist. Calibrated. Where our PM checked the graders by hand, they agreed on 95% of 42 checks.

#Model · HarnessTask scoreDecision modelLLM judgeRunsCritical failures
1Sonnet 5.5withAPI88.088.52None
2GPT-6.1 SolwithAPI83.784.92None
3Opus 5.5withClaude92.074.72None
4GPT-6 AstrawithChatGPT89.774.52None
5Haiku 5.5withAPI89.770.12None
6GPT-6 LunawithAPI83.773.92None
7Gemini 4 ArgonwithAPI75.670.12None
8Gemini 3.8 FlashwithAPI67.851.92None
9Gemini 3.5 Flash-LitewithGemini63.926.121 capped

What you get for the cost

GPT-6.1 Sol · API scores within 4 points of Sonnet 5.5 · API at a third of the cost.

Task score against relative cost for each setup. The table below has the same figures.304050607080901001×3×10×30×Cost per output, relative to the cheapest setup (log scale) →↑ Task scoreSonnet 5.5 · API: task score 87, 43× the cheapestGPT-6.1 Sol · API: task score 83, 13× the cheapestOpus 5.5 · Claude: task score 82, 71× the cheapestGPT-6 Astra · ChatGPT: task score 81, 66× the cheapestHaiku 5.5 · API: task score 78, 2.3× the cheapestGPT-6 Luna · API: task score 78, the cheapestGemini 4 Argon · API: task score 70, 13× the cheapestGemini 3.8 Flash · API: task score 57, 8.3× the cheapestGemini 3.5 Flash-Lite · Gemini: task score 42, 3.0× the cheapest, 1 critical failureSonnet 5.5 · APIGPT-6.1 Sol · APIOpus 5.5 · ClaudeGPT-6 Astra · ChatGPTHaiku 5.5 · APIGPT-6 Luna · APIGemini 4 Argon · APIGemini 3.8 Flash · APIGemini 3.5 Flash-Lite · Gemini
Best valueNobody beats these on both score and costAt least one critical failure
Task score and relative cost by setup
SetupTask scoreRelative cost (cheapest = 1)
GPT-6 Luna · API781.0×
Haiku 5.5 · API782.3×
Gemini 3.5 Flash-Lite · Gemini423.0×
Gemini 3.8 Flash · API578.3×
GPT-6.1 Sol · API8313×
Gemini 4 Argon · API7013×
Sonnet 5.5 · API8743×
GPT-6 Astra · ChatGPT8166×
Opus 5.5 · Claude8271×

Cost is each output’s tokens at the model’s list price, shown relative to the cheapest setup. Runs done in an app have no usage meter, so theirs are estimated from the same setup’s measured runs. How we compare cost

About the task

The PM job

Preparing for customer calls.

Why it matters

Leading questions produce the answers you wanted. A good guide is designed to prove you wrong.

What good looks like

  • Past-behaviour questions
  • No leading prompts
  • Clear learning goals
  • Could disprove the team's hypothesis

Deliberately not measured

    Capability tested

    Question design

    The failure we’re looking for

    Hypothetical, leading questions

    Grading

    Decision model and LLM judge, calibrated against a blind PM review