Tasks / Growth

Activation & onboarding review

Can the model find the friction that matters most and prioritise the fixes?

Task type v1.1 · 2 tasksLast changed 6 Oct 2026 · ChangelogDifficulty

What AI gets right here, and what you’ll still have to catch

From 16 graded outputs by 8 models. 69% were usable with at most a quick edit.

Reliably right

  1. Respects explicit constraints100% pass
    The output is a review addressed to Omar, well under 800 words, and respects the requested form.
    GPT-6 Luna · API · The checklist isn't the moment
  2. Produces the required deliverable100% pass
    The review covers activation definition, evidence, changes, design proposal, and goal, is addressed to Omar, within length, and is actionable.
    GPT-6 Luna · API · The checklist isn't the moment
  3. Activation defined by what predicts retention100% pass
    Activation is defined by the behaviour that predicts retention (bank+cat 10+), and fixes are judged by their effect on that event.
    GPT-6 Luna · API · The checklist isn't the moment

Where it slips

  1. Uses the supplied evidence correctly44% pass
    The output states 'we have no step-level data for screens 2–8', 'a blank builder is hard for non-analysts', and 'people invite colleagues to something useful, not to a setup screen' as facts, none of which appear in the brief or follow from arithmetic.
    Opus 5.5 · Claude · Analytics tool losing users at setup
  2. Avoids unsupported claims55% pass
    It presents 'a blank builder is hard for non-analysts' and 'people invite colleagues to something useful, not to a setup screen' as established facts without labelling them as hypotheses.
    Opus 5.5 · Claude · Analytics tool losing users at setup
  3. Addresses the actual decision56% pass
    The output commits to a ranked order but does not state what result or condition would change that order, as required.
    GPT-6 Astra · ChatGPT · Analytics tool losing users at setup

How it’s graded

Each check is built on how world-class product and growth leaders do this work, as they’ve described it on Lenny’s Podcast, and names the leader it comes from.

  1. Activation defined by what predicts retention

    Does the review define (or question) the activation event by whether it predicts retention, and judge fixes by their effect on that event rather than on step completion?

    Passes when Names the activation behaviour and its link to retention (from the supplied data, or as an explicit assumption to verify), and ranks fixes by their effect on it.

  2. Gets people to value sooner

    Do the recommended fixes shorten the path to the first moment of value, by removing or deferring steps before it or starting people with templates or sample data, rather than adding tours, tooltips or extra steps?

    Passes when The main fixes cut or defer steps before first value, or give a warm start, and say which steps go.

Plus our standard checks

Uses the supplied evidence correctly · Addresses the actual decision · Respects explicit constraints · Identifies material uncertainty · Avoids unsupported claims · Produces the required deliverable · Gets the base of every number right · and 2 written for each task, which you’ll see in the tasks below.

The tasks

Read the brief, then compare up to three outputs, each with the LLM judge’s verdict on every check. Highlights mark what a PM had to fix.

The brief

You're the PM for onboarding at Tidepool. Omar, our Head of Product, has asked for a review of onboarding (his note is below). Write it in no more than 800 words: what activation should mean for us, the evidence, what to change in onboarding, and what to make of the design team's proposal and next quarter's goal. The files are attached.

What the model was given4 items: About Tidepool, omar_note.md, onboarding_today.md, first_week_behaviour.csv (August sign-ups: how many who did each thing in their first week were still active at day 60)
About TidepoolA personal finance app: connect your bank, see where your money goes, and budget.
omar_note.md1 lines · Download
From Omar (Head of Product): "Activation is flat at 61%. Review our onboarding and tell me what to fix. The design team has proposed a celebration screen and a progress bar for the checklist."
onboarding_today.md8 lines · Download
# Onboarding today (for 4,000 sign-ups in August)

1. Sign up (100%)
2. Setup checklist: name your account, pick a theme, set a savings goal, enable notifications (61% complete it)
3. Connect a bank (50%)
4. Categorise transactions (34% reach 10)

…
first_week_behaviour.csv (August sign-ups: how many who did each thing in their first week were still active at day 60)first_week_behaviour,retained_at_day_60,churned_by_day_60 All sign-ups,1120,2880 Completed the setup checklist,742,1696 Connected a bank account,964,1036 Categorised 10+ transactions,891,489 Connected a bank AND categorised 10+ transactions,842,238 Created a budget,410,820 Opened the app 4+ days,870,1310
What a strong answer doesThe answer key the graders mark against

Works out retention for each first-week behaviour from the file: all sign-ups 28% (1,120 of 4,000); completed the checklist about 30% (barely better than average); connected a bank about 48%; categorised 10+ transactions about 65%; connected a bank and categorised 10+ about 78%, and that group is about three-quarters of everyone retained (842 of 1,120). So the checklist isn't activation and raising its completion to 75% would do little; activation should be connecting a bank and categorising ten transactions in week one. Recommends cutting or deferring checklist steps that sit before that (theme, notifications, savings goal) and getting people to connect a bank and categorise sooner (for example suggested categories to confirm), and resets the goal to the new activation rate (about 27% today, 1,080 of 4,000). Treats the celebration screen and progress bar as polishing the wrong step. Notes that the data shows correlation and proposes a test to confirm that driving the behaviour improves retention.

Critical failures (cap the score at 40)
  • Recommends raising checklist completion as the main goal
Task

Staff-level · v1.0

The outputs

8 setups answered this brief.

Usable with a quick edit9 of 11 checks passed

The review is clear, evidence-based, and covers all required points; a PM could act on it with only minor wording tweaks.

Output

Onboarding review: activate on financial value, not setup

Recommendation: Replace checklist completion as our activation metric with connecting a bank and categorising at least 10 transactions within seven days of signup. Reorient onboarding around that outcome. The celebration screen and progress bar are secondary; making the current checklist more engaging risks improving the wrong metric.

What activation should mean

For Tidepool, activation should mean a user has taken the first meaningful steps toward understanding their spending—not finished configuring the app.

Bank connection brings their financial data into Tidepool; categorisation makes that data useful. Together, these are our strongest available proxy for experiencing core value. This is a working definition, not proof that exactly 10 categorisations creates a lasting habit. We should validate the threshold and whether users actually understand something useful about their money.

Budget creation can follow that first insight. The evidence does not support making it a mandatory activation requirement today.

What the evidence says

Among August’s 4,000 signups, overall day-60 retention was 28%.

First-week behaviourUsersDay-60 retention
------:---:
Completed setup checklist2,43830.4%
Connected a bank2,00048.2%
Categorised 10+ transactions1,38064.6%
Connected a bank and categorised 10+1,08078.0%
Created a budget1,23033.3%
Opened the app on 4+ days2,18039.9%

The checklist has weak separation: completers retained at 30.4%, versus 24.2% for non-completers. Its 61% completion rate is therefore a poor headline measure of delivered value.

By contrast, only 27% of signups connected a bank and categorised 10+ transactions, but they accounted for 842 of 1,120 retained users—75% of everyone retained. Users who did not achieve both retained at 9.5%.

These are observational associations. More motivated users may both complete these actions and remain active; we cannot promise 78% retention simply by pushing people through them. Behaviour groups also overlap, so the reported steps should not be treated as a strictly sequential funnel. Before testing, reconcile the rounded onboarding figures with event counts and standardise the day-60 activity definition.

What to change

  1. Remove configuration as a gate to value. Defer theme selection and account naming; make savings goals optional. Request notification permission when there is a clear benefit, such as a relevant budget alert—not as checklist work.
  1. Bring bank connection forward. Explain why access is needed, what data is used, and how users retain control. Instrument connection starts, provider errors, unsupported banks, abandonment and successful imports. Preserve a skip/retry path rather than forcing consent.
  1. Guide users from imported transactions to spending insight. Offer a short, low-effort review of suggested categories. Let users confirm or correct them, then show an understandable spending summary. Do not require ten arbitrary manual edits merely to satisfy the metric.
  1. Offer a budget as the next useful action. Use the spending summary to suggest a realistic starting budget, rather than asking users to invent one before seeing their finances.

Test the redesigned flow against today’s onboarding. Measure seven-day activation first, then downstream retention; monitor connection failures, abandonment and user-reported trust and usefulness.

Design proposal and next quarter’s goal

A progress bar is worth testing if it tracks meaningful steps toward understanding spending. It should not encourage completion of low-value setup tasks. A celebration screen belongs after a genuine milestone and should reinforce the insight gained, not interrupt access to it. Neither should take priority over bank-connection and categorisation friction.

Replace the 61% → 75% checklist goal with a goal for the new activation definition. I propose 27% → 35% within seven days, provisionally, subject to instrumentation validation and early experiment results. That is a planning target—not a forecast supported by this dataset.

Keep checklist completion as a diagnostic. Judge success by increased meaningful activation without sacrificing trust or retention, and confirm the retention effect once next quarter’s cohorts mature.

Grades and run details

Decision model 82 · LLM judge 100
Decision model checks
  • failUses the supplied evidence correctly90%
  • passAddresses the actual decision86%
  • passRespects explicit constraints30%
  • passIdentifies material uncertainty83%
  • passAvoids unsupported claims43%
  • passProduces the required deliverable42%
  • passComputes retention by behaviour32%
  • passTests before trusting the correlation97%
  • passActivation defined by what predicts retention99%
  • failGets the base of every number right82%
  • passGets people to value sooner78%
Run
Run
#1
Time to output
30 s
Submitted
5 Oct 2026

Usable with a quick edit8 of 11 checks passed

The memo is structured, evidence-based, and a PM could act on it with a quick edit to soften the unsupported causation claim about app opens.

Output

To: Omar From: PM, Onboarding Re: Onboarding review

Summary

Our activation metric is measuring the wrong thing. Completing the setup checklist is 61% of sign-ups, but it barely separates users who stay from those who leave: 30% of completers are retained at day 60, against 24% of non-completers. The behaviours that matter are connecting a bank account and categorising transactions. Users who do both are retained at 78%. Users who do neither are retained at 6%. I recommend redefining activation, putting bank connection at the centre of onboarding, and reshaping the design proposal around it.

What activation should mean

Activation = connected a bank account and categorised 10+ transactions in the first 7 days.

Baseline: 27% of August sign-ups (1,080 of 4,000). This definition reflects the core value (seeing where money goes), is the only combination with a large retention gap, and is measurable from event data.

Evidence

First-week behaviourShare of sign-upsRetained at day 60
Completed checklist61%30%
Did not complete checklist39%24%
Connected a bank50%48%
Did not connect a bank50%8%
Categorised 10+35%65%
Did not categorise 10+65%9%
Bank and categorised27%78%
Bank only23%13%
Categorised only8%16%
Neither43%6%
Opened app on 4+ days55%40%
Created a budget31%33%

Four things stand out:

  • Bank connection and categorising are complementary. Each alone yields 13–16% retention; together they yield 78%. The value appears only when both happen.
  • The biggest fixable leak is after bank connection. 46% of users who connect a bank never categorise 10 transactions.
  • Budget creation is weak in week one. Its retention (33%) is barely above the 28% baseline, so it shouldn't define activation yet.
  • App-open frequency is an outcome, not a cause. It's a useful engagement metric, but users who are already activated open the app more.

These are correlations. Users who do more are probably more motivated to begin with, so the 78% overstates the causal effect. We need experiments to confirm the lift, but the gap is large enough to act on now.

What to change in onboarding

  1. Make bank connection the first real step. Only half of sign-ups connect, and the cosmetic steps (name, theme) come first. Move bank linking up, explain the payoff ("see where last month went"), and reduce friction: clear read-only reassurance, and recovery paths for failed connections.
  2. Help users categorise 10 transactions quickly. Pre-suggest categories, allow one-tap confirmation, and show progress ("3 of 10: your spending is taking shape"). Target the 46% who stop short.
  3. Demote checklist items that don't carry the value. Theme, name, and notifications should be optional. The savings goal is untested in our data, so test it before cutting it.
  4. Move budget creation later. Prompt it after the user has categorised 10 transactions, and measure whether that lifts budget creation and retention in weeks 2–4.

The design team's proposal

The celebration screen and progress bar are good craft aimed at the wrong target. A progress bar on the current checklist will push users through theme and notifications faster. That raises the 61% number without touching the steps that predict retention, and may do nothing for bank connection.

The celebration is valuable, but only at a milestone that matters. I'd ask the design team to rebuild the progress bar around the two activation steps (bank connected, 10 transactions categorised), with the celebration triggered when both are done. We test it against the current flow and ship only if activation moves.

Next quarter's goal

Replace "checklist completion 61% → 75%" with "activation (bank connected and 10+ categorised in week one) 27% → 35%."

Sizing: in a 4,000-user cohort, 27% → 35% is about 320 more activated users. At 78% retention, that's roughly 250 more retained at day 60. A checklist point, by contrast, is worth about 10 retained users at current completer retention. Marginal users will retain worse than average, so treat 250 as an upper bound.

35% is ambitious but reachable: it requires bank connection to rise from 50% to about 60%, and categorising conversion from 54% to about 58%. The checklist stays as a leading indicator, not the goal.

Next steps

  1. Adopt the new activation definition this week and add it to the onboarding dashboard.
  2. Run an A/B test of bank-first onboarding against the current flow.
  3. Ask design to rebuild the progress bar and celebration around the activation steps, and test before shipping.
  4. Confirm the retention lift with a holdout or experiment, since current figures are correlational.

Grades and run details

Decision model 82 · LLM judge 92
Decision model checks
  • failUses the supplied evidence correctly19%
  • passAddresses the actual decision94%
  • passRespects explicit constraints58%
  • passIdentifies material uncertainty70%
  • passAvoids unsupported claims34%
  • passProduces the required deliverable92%
  • passComputes retention by behaviour67%
  • passTests before trusting the correlation100%
  • passActivation defined by what predicts retention100%
  • failGets the base of every number right19%
  • passGets people to value sooner82%
Run
Run
#1
API response time
29 s
Submitted
8 Oct 2026

Usable with a quick edit11 of 11 checks passed

The output is clear, evidence-based, and actionable; a PM could use it with minor edits without rechecking evidence or redoing reasoning.

Output

To Omar — onboarding review

What activation should mean

Activation should mean a new user has reached Tidepool’s core value: connected a bank account and categorized at least 10 transactions within their first seven days. Completing setup tasks is not the same as understanding where your money goes.

What the evidence says

Of 4,000 August sign-ups, 61% completed the checklist, 50% connected a bank, and 34% categorized 10+ transactions. The behavioral data points to which events are most associated with retention at day 60:

  • All sign-ups: 28% retained (1,120/4,000)
  • Completed the checklist: 30% (742/2,438)
  • Connected a bank: 48% (964/2,000)
  • Categorized 10+ transactions: 65% (891/1,380)
  • Connected a bank and categorized 10+: 78% (842/1,080)

The combined behavior is the strongest signal and represents 27% of all sign-ups today. By contrast, checklist completion is barely above the overall retention rate. Creating a budget is associated with 33% retention, so it is not a better activation milestone on this evidence.

These are associations, not proof that onboarding changes will cause retention to rise: users who connect accounts or categorize transactions may already be more motivated. We should treat the numbers as a strong direction for what value to help users reach, then test changes and track outcomes.

What to change

  • Move bank connection earlier. Defer account naming, theme choice, savings goals, and notifications until after users see value. Explain why connection helps, address security concerns, and make the flow as clear and low-friction as possible.
  • Guide users to their first insight. After connection, help them review and categorize transactions with sensible suggestions. Measure progress toward 10 categorized transactions, not just completion of onboarding screens.
  • Offer budgeting in context. Prompt users to create a budget after they have seen their spending, rather than treating it as a prerequisite.

Design proposal and next-quarter goal

A progress bar could help, but only if it tracks the path to value—connect a bank, review transactions, categorize 10—not the existing checklist. A celebration screen after checklist completion would celebrate the wrong milestone; consider celebrating the first meaningful spending insight instead. Test both against a control, measuring bank connection, 10+ categorization, and retention—not checklist completion alone.

Replace the goal of raising checklist completion from 61% to 75% with a provisional target: raise seven-day activation from 27% to 35% of sign-ups next quarter. Track day-60 retention as a lagging outcome and guardrail. This keeps the goal tied to Tidepool’s promise while making clear that improved activation is a hypothesis to validate, not a guaranteed retention lift.

Grades and run details

Decision model 100 · LLM judge 100
Decision model checks
  • passUses the supplied evidence correctly53%
  • passAddresses the actual decision86%
  • passRespects explicit constraints78%
  • passIdentifies material uncertainty39%
  • passAvoids unsupported claims68%
  • passProduces the required deliverable93%
  • passComputes retention by behaviour82%
  • passTests before trusting the correlation88%
  • passActivation defined by what predicts retention100%
  • passGets the base of every number right44%
  • passGets people to value sooner79%
Run
Run
#1
API response time
20 s
Submitted
6 Oct 2026

Check by check

Where the outputs differ comes first. Open a check to read the judge’s reasons. “Mixed” means the two graders disagreed.

Where they differ 3

Uses the supplied evidence correctlyMixedMixedRight
GPT-6 Astra · ChatGPT

All factual claims about the current situation are directly supported by the supplied context or correct arithmetic.

Haiku 5.5 · API

Every factual statement about the current situation is taken from the supplied context or follows from correct arithmetic.

GPT-6 Luna · API

All factual claims about the current situation are directly from the supplied context or derived by correct arithmetic.

Avoids unsupported claimsRightMixedRight
GPT-6 Astra · ChatGPT

Hypotheses and forecasts are clearly labelled as such, and confident claims are supported by the evidence.

Haiku 5.5 · API

States 'App-open frequency is an outcome, not a cause' as an established fact, but only correlation is shown in the data; this causal direction is an unsupported interpretation.

GPT-6 Luna · API

Interpretations are clearly labelled as associations and hypotheses, not established facts.

Gets the base of every number rightMixedMixedRight
GPT-6 Astra · ChatGPT

All percentages are computed from the correct denominators, and bases are stated or clear from context.

Haiku 5.5 · API

All derived percentages (24%, 30%, 78%, 6%, 48%, 13%, 16%, etc.) use the correct denominators, with no ambiguous bases.

GPT-6 Luna · API

All percentages are computed from the correct denominators, and bases are stated where needed (e.g., 1,120/4,000, 842/1,080).

All got right 8

Addresses the actual decisionRightRightRight
GPT-6 Astra · ChatGPT

Commits early to a clear activation definition and changes, and states that validation and experiment results would change the call.

Haiku 5.5 · API

Commits to a clear activation definition, specific onboarding changes, and a verdict on the design proposal and next quarter's goal, with conditions for change.

GPT-6 Luna · API

The output commits unambiguously to a new activation definition, specific onboarding changes, rejecting the design proposal for the checklist, and a new goal, and says to test and track retention to validate.

Respects explicit constraintsRightRightRight
GPT-6 Astra · ChatGPT

The review is under 800 words, addresses Omar, and covers all requested elements.

Haiku 5.5 · API

Output is a memo to Omar, addresses all parts of the brief, and is well within the 800-word limit.

GPT-6 Luna · API

The output is a review addressed to Omar, well under 800 words, and respects the requested form.

Identifies material uncertaintyRightRightRight
GPT-6 Astra · ChatGPT

Names the observational nature, threshold uncertainty, and need for validation; says how to resolve them.

Haiku 5.5 · API

Explicitly notes that retention figures are correlations and proposes A/B tests and a holdout to resolve the causal uncertainty.

GPT-6 Luna · API

It explicitly names the correlation-vs-causation risk and proposes a controlled test measuring retention to resolve it.

Produces the required deliverableRightRightRight
GPT-6 Astra · ChatGPT

The output is a complete review with recommendation, evidence, changes, and view on proposal and goal, usable as is.

Haiku 5.5 · API

Memo format, addressed to Omar, includes activation definition, evidence, changes, design proposal view, and next quarter goal, all under the word limit.

GPT-6 Luna · API

The review covers activation definition, evidence, changes, design proposal, and goal, is addressed to Omar, within length, and is actionable.

Computes retention by behaviourRightRightRight
GPT-6 Astra · ChatGPT

Computes day-60 retention for each first-week behaviour and compares with the 28% average.

Haiku 5.5 · API

Computes retention for each first-week behaviour including checklist (≈30%), bank+cat (78%), and baseline (28%), and compares them.

GPT-6 Luna · API

It computes day-60 retention for each first-week behaviour (checklist ~30%, bank+cat ~78%, average 28%) and compares them.

Tests before trusting the correlationRightRightRight
GPT-6 Astra · ChatGPT

Notes the correlation risk and proposes a specific test (redesigned flow vs. today, measuring retention) to check causation.

Haiku 5.5 · API

Notes correlation risk and proposes a specific A/B test of bank-first onboarding and a holdout experiment with retention as the measure.

GPT-6 Luna · API

It notes the correlation risk and proposes a specific test (control group, measuring bank connection, 10+ categorization, and retention).

Activation defined by what predicts retentionRightRightRight
GPT-6 Astra · ChatGPT

Defines activation by retention (bank + categorise 10+) and ranks fixes by their effect on that event.

Haiku 5.5 · API

Defines activation by retention (bank+cat 78%), not checklist; ranks fixes by their effect on that activation behaviour.

GPT-6 Luna · API

Activation is defined by the behaviour that predicts retention (bank+cat 10+), and fixes are judged by their effect on that event.

Gets people to value soonerRightRightRight
GPT-6 Astra · ChatGPT

Main fixes cut or defer configuration steps before value and bring bank connection and categorisation forward.

Haiku 5.5 · API

Recommended fixes move bank connection earlier, pre-suggest categories, and demote or defer theme/name/notifications, cutting steps before value.

GPT-6 Luna · API

The main fixes cut or defer steps before first value (defer theme, savings goal, notifications; move bank connection earlier) and guide users to categorize quickly.

Results

Every setup we’ve tested on this task type, across all its tasks and repeats, graded on the current checklist. Calibrated. Where our PM checked the graders by hand, they agreed on 79% of 53 checks.

#Model · HarnessTask scoreDecision modelLLM judgeRunsCritical failures
1GPT-6 AstrawithChatGPT85.995.52None
2Sonnet 5.5withAPI78.4100.02None
3GPT-6.1 SolwithAPI85.990.92None
4GPT-6 LunawithAPI90.081.82None
5Haiku 5.5withAPI75.986.72None
6Opus 5.5withClaude88.478.42None
7Gemini 3.8 FlashwithAPI66.651.92None
8Gemini 3.5 Flash-LitewithGemini52.539.42None

What you get for the cost

GPT-6 Luna · API scores within 5 points of GPT-6 Astra · ChatGPT at about a hundredth of the cost.

Task score against relative cost for each setup. The table below has the same figures.304050607080901001×3×10×30×100×Cost per output, relative to the cheapest setup (log scale) →↑ Task scoreGPT-6 Astra · ChatGPT: task score 91, 122× the cheapest (1 of 2 outputs estimated)Sonnet 5.5 · API: task score 90, 32× the cheapestGPT-6.1 Sol · API: task score 89, 15× the cheapestGPT-6 Luna · API: task score 86, the cheapestHaiku 5.5 · API: task score 82, 1.8× the cheapestOpus 5.5 · Claude: task score 82, 101× the cheapest (1 of 2 outputs estimated)Gemini 3.8 Flash · API: task score 56, 13× the cheapestGemini 3.5 Flash-Lite · Gemini: task score 44, 3.4× the cheapest (1 of 2 outputs estimated)GPT-6 Astra · ChatGPTSonnet 5.5 · APIGPT-6.1 Sol · APIGPT-6 Luna · APIHaiku 5.5 · APIOpus 5.5 · ClaudeGemini 3.8 Flash · APIGemini 3.5 Flash-Lite · Gemini
Best valueNobody beats these on both score and costCost partly estimated (app runs with no usage meter)
Task score and relative cost by setup
SetupTask scoreRelative cost (cheapest = 1)
GPT-6 Luna · API861.0×
Haiku 5.5 · API821.8×
Gemini 3.5 Flash-Lite · Gemini443.4× (partly estimated)
Gemini 3.8 Flash · API5613×
GPT-6.1 Sol · API8915×
Sonnet 5.5 · API9032×
Opus 5.5 · Claude82101× (partly estimated)
GPT-6 Astra · ChatGPT91122× (partly estimated)

Cost is each output’s tokens at the model’s list price, shown relative to the cheapest setup. Runs done in an app have no usage meter, so theirs are estimated from the same setup’s measured runs. How we compare cost

About the task

The PM job

Reviewing a signup and onboarding flow that is losing users.

Why it matters

Anyone can list fifty UX nits. The job is finding the two that explain the drop-off, backed by the funnel data supplied.

What good looks like

  • Ties each issue to the funnel data
  • Prioritises by likely impact
  • Defines activation as the behaviour that predicts retention, not finishing onboarding

Deliberately not measured

  • Accessibility audit completeness
  • Visual redesign
Capability tested

Consequential critique

The failure we’re looking for

A generic UX checklist

Grading

Decision model and LLM judge, calibrated against a blind PM review