Tasks / Growth

Diagnose a retention problem

Can the model work out from raw data what's really happening to retention, separating churn from measurement and seasonality, and find the cause?

Task type v1.1 · 2 tasksLast changed 6 Oct 2026 · ChangelogDifficulty

What AI gets right here, and what you’ll still have to catch

From 16 graded outputs by 8 models. 81% were usable with at most a quick edit.

Reliably right

  1. Produces the required deliverable100% pass
    The diagnosis is complete, actionable, and in the right format for Leo; a PM could act on it with light edits.
    GPT-6 Astra · ChatGPT · Flat, while sign-ups grow
  2. Understands before it justifies100% pass
    It tests the summer explanation against the data, finds the break aligns with the onboarding change not the paid spend increase, and reaches its own diagnosis from the evidence.
    GPT-6 Astra · ChatGPT · Flat, while sign-ups grow
  3. Splits growth into new, lost and returning100% pass
    It splits growth into new, retained, churned, and returning users, and uses the net growth equation to show which flow changed.
    GPT-6 Astra · ChatGPT · Flat, while sign-ups grow

Where it slips

  1. Gets the base of every number right61% pass
    The SMB adoption churn calculation is incorrect; 39% of ~$7.4k/month churn is ~$2.9k/month, not $34k MRR, and the annualized figure is ~$34.8k ARR, not $0.4M.
    Sonnet 5.5 · API · The collapse that wasn't
  2. Avoids unsupported claims67% pass
    The output presents causal interpretations (e.g., onboarding changes broke retention, low-intent users) as established fact without labelling them as hypotheses.
    Gemini 3.5 Flash-Lite · Gemini · Flat, while sign-ups grow
  3. Identifies material uncertainty77% pass
    The output does not name specific unknowns that could change the decision or say how they would be resolved.
    Gemini 3.5 Flash-Lite · Gemini · Flat, while sign-ups grow

How it’s graded

Each check is built on how world-class product and growth leaders do this work, as they’ve described it on Lenny’s Podcast, and names the leader it comes from.

  1. Understands before it justifies

    Does the diagnosis start from what the data shows (which users, when, at what point in their life with the product) before accepting or proposing an explanation, including the one it was handed?

    Passes when Tests the explanation it was given against the data, and reaches its own diagnosis from the evidence.

  2. Splits growth into new, lost and returning

    Does the diagnosis separate new users or revenue from what was lost and what came back, rather than reading a single total?

    Passes when Uses growth accounting (new, retained, churned, returning, or new, expansion, contraction, churn) to show which flow changed.

  3. Reads the shape of the curves

    Does the diagnosis read how retention curves behave over time (flattening, still falling, or dipping and recovering) and what each shape means?

    Passes when Describes the curve shapes in the data and draws the right conclusion from each, such as a dip that recovers being seasonal.

Plus our standard checks

Uses the supplied evidence correctly · Addresses the actual decision · Respects explicit constraints · Identifies material uncertainty · Avoids unsupported claims · Produces the required deliverable · Gets the base of every number right · and 2 written for each task, which you’ll see in the tasks below.

The tasks

Read the brief, then compare up to three outputs, each with the LLM judge’s verdict on every check. Highlights mark what a PM had to fix.

The brief

You're the PM for activation at Stride. Leo, our Head of Growth, thinks weekly actives are flat because it's summer, and wants to double paid acquisition (his note is below). Before he does, write him a diagnosis of no more than 800 words: what's actually happening to retention, the evidence, and what we should do instead of (or before) spending more on acquisition. The files are attached.

What the model was given6 items: About Stride, leo_note.md, weekly_growth_accounting.csv (weeks run Monday to Sunday), weekly_signup_cohorts.csv (share of each week's sign-ups active in each following week), release_and_campaign_log.md, winback_campaign_results.md
About StrideA fitness habit app: short guided workouts and a weekly plan. Free, with Premium at $9.99 a month.
leo_note.md1 lines · Download
From Leo (Head of Growth): "Weekly actives have been flat for three months even though sign-ups are up 30%. It's summer: people are outdoors and on holiday. I want to double paid acquisition to push through it."
weekly_growth_accounting.csv (weeks run Monday to Sunday)week,week_starting,new_users,returning_users,retained_users,churned_users,weekly_active_users 1,2026-07-06,6034,973,44884,7116,51891 2,2026-07-13,6142,989,44529,7362,51660 3,2026-07-20,6184,1040,45074,6586,52298 4,2026-07-27,6385,991,44843,7455,52219 5,2026-08-03,6597,975,44085,8134,51657 6,2026-08-10,6689,2541,42516,9141,51746 7,2026-08-17,6779,2666,42506,9240,51951 8,2026-08-24,6955,2552,42723,9228,52230 9,2026-08-31,7156,2618,42134,10096,51908 10,2026-09-07,7230,2675,42393,9515,52298 11,2026-09-14,7564,2586,41984,10314,52134 12,2026-09-21,7550,2557,41808,10326,51915 13,2026-09-28,7847,2635,41636,10279,52118
weekly_signup_cohorts.csv (share of each week's sign-ups active in each following week)signup_week,signups,week_1_retained_pct,week_2_retained_pct,week_3_retained_pct,week_4_retained_pct,week_5_retained_pct,week_6_retained_pct,week_7_retained_pct,week_8_retained_pct 1,6000,55.4,43.2,36.4,34.2,31.9,30.9,29.8,30.1 2,6150,55.3,43.2,37.0,34.2,32.4,30.5,29.9,30.2 3,6300,55.2,43.0,37.7,33.4,32.1,30.6,29.8,30.0 4,6450,54.3,42.3,37.0,33.4,32.4,31.4,30.0,29.5 5,6600,58.4,39.4,29.6,24.5,22.1,21.1,19.8,19.9 6,6750,58.6,39.9,30.1,24.4,21.8,21.2,19.9, 7,6900,58.4,40.5,29.4,24.9,22.2,21.2,, 8,7050,58.1,40.1,29.9,25.0,21.3,,, 9,7200,57.5,40.5,29.7,25.0,,,, 10,7350,58.3,40.5,29.4,,,,, 11,7500,57.4,40.6,,,,,, 12,7650,57.5,,,,,,,
release_and_campaign_log.md6 lines · Download
# Release and campaign log

- Week 3: New workout library (40 new routines).
- Week 5: Shorter sign-up. Removed the goal-setting step ("What do you want to achieve, and how many days a week?") to cut sign-up time from 3 minutes to 1. Sign-up completion rose from 61% to 74%.
- Week 6: Win-back emails start: anyone inactive for 21 days gets "We miss you: here's a free week of Premium".
- Week 9: Paid social budget up 25%.
winback_campaign_results.md3 lines · Download
# Win-back campaign, weeks 6 to 13 (from the lifecycle team)

About 1,600 extra returning users a week compared with before the campaign. Of users who came back through a win-back email, 12% were still active two weeks later. Of users who came back on their own before the campaign, 41% were.
What a strong answer doesThe answer key the graders mark against

Shows that flat weekly actives aren't the summer: new users rose about 30% (about 6,000 to 7,800 a week) but churned users rose faster (about 7,100 to 10,300), and returning users jumped from about 1,000 to 2,600 a week only because of the win-back emails from week 6, of whom 12% stay two weeks against 41% of organic returners, so actives are being propped up. The cohort table shows why churn rose: cohorts from week 5, when the goal-setting step was removed from sign-up, retain far worse (week 4 about 25% against 34%, week 8 about 20% against 30%) even though more people finish sign-up. So more paid acquisition would pour more users into a leaky first month. Recommends restoring goal-setting (or testing it back against the short sign-up, judged on week-4 retention, not sign-up completion), measuring returning users separately so the campaign doesn't hide churn, and holding paid acquisition until new-user retention recovers.

Critical failures (cap the score at 40)
  • Recommends doubling paid acquisition now
Task

Mid-level · v1.0

The outputs

8 setups answered this brief.

Usable with a quick edit10 of 12 checks passed

The output is well-structured, evidence-based, and actionable. A PM could soften the causal claim and add a caveat about uncertainty with a quick edit, without redoing the analysis.

Output

To: Leo, Head of Growth From: PM for Activation Subject: Diagnosis: Why WAU is flat and why we shouldn’t double paid acquisition yet

*

Executive Summary

WAU is not flat because of summer seasonality. It is flat because of a structural collapse in early habit retention starting in Week 5, which is currently masked by two temporary factors: the Week 6 win-back email campaign and higher sign-up volume.

Our core retained user base has dropped by over 3,200 users, and weekly churn has surged by 44% (from ~7,100 to over 10,200). Doubling paid acquisition today would pour money into an onboarding funnel that currently retains a third fewer users long-term than it did two months ago.

Here is what the data shows and what we should do instead.

---

1. The Evidence: The Week 5 Onboarding Change Broke Habit Retention

If summer outdoor seasonality were the culprit, we would expect a gradual, platform-wide decay across all cohorts. Instead, the cohort data reveals an immediate step-change break between Cohort 4 and Cohort 5:

  • Cohorts 1–4 (Before Week 5): Week 4 retention was steady at 33.4%–34.2%, flattening to a healthy 30% at Week 8.
  • Cohorts 5+ (After Week 5): Week 4 retention plummeted to 24.5%–25.0%, and Week 8 dropped to 19.9%.

This represents a ~33% relative drop in long-term cohort retention.

Why did this happen? In Week 5, we removed the goal-setting step ("What do you want to achieve, and how many days a week?") to cut sign-up time from 3 minutes to 1 minute. While sign-up completion jumped from 61% to 74% and bumped Week 1 retention slightly (55% to 58%), we stripped away the user’s commitment and personal investment.

Stride is a habit product. Without clear goals and a tailored weekly schedule, users drop off rapidly after Week 1 because they lack direction and accountability.

---

2. The Illusion: Why Topline WAU Looks Flat

If retention collapsed by a third, why hasn't WAU cratered? Two factors created an artificial floor:

  1. Top-of-Funnel Inflow: Sign-ups grew ~30% (boosted by higher sign-up completion and the Week 9 paid budget increase), adding ~1,800 gross new sign-ups per week.
  2. The Week 6 Win-Back "Sugar Rush": The win-back campaign added ~1,600 returning users per week (jumping from ~1,000 to ~2,600 in growth accounting).

However, this resurrected volume is extremely low-quality: * Historically, 41% of organic returning users were still active two weeks later. * For win-back users, only 12% remain active after two weeks.

We are cycling low-intent, discount-incentivized users who churn almost immediately. Meanwhile, underlying retained users dropped from 45,074 (Week 3) to 41,636 (Week 13), while churn surged from 7,116 to 10,279 per week.

---

3. What Happens If We Double Paid Acquisition Now?

Doubling paid acquisition right now will accelerate cash burn without growing the business:

  • Impaired Unit Economics: With Week 8 retention down from 30% to 20%, LTV has dropped sharply. Premium conversion will suffer because fewer users build the 3-week habit required to see the value of paid features.
  • Higher CAC for Diminishing Returns: Scaling paid spend when the conversion engine is degraded will cause CAC to rise while payback periods stretch unsustainably.
  • A Leaky Bucket: In Week 13, we added 7,847 new users and churned 10,279. Adding more sign-ups does not fix a machine that is churning users faster than it can onboard them.

---

4. Recommended Action Plan

Before increasing ad spend, we need to plug the leak and fix activation.

1. Reinstate and Optimize Goal-Setting (Immediate): * Re-introduce the goal-setting and schedule-commitment step in onboarding, framed as building the user's "Personalized Weekly Plan." * Test progressive onboarding: let users quickly select goals without friction, but ensure their first workout calendar is anchored before Day 1. * Target: Return Week 4 retention to ≥33%. 2. Re-evaluate Win-Back & Lifecycle Spend: * Stop relying on the 21-day "free Premium week" email to prop up WAU. Shift CRM focus to Days 1–7 activation, reinforcing the weekly plan set during onboarding. 3. Hold Paid Social at Current Levels: * Keep paid spend flat (or pull back the Week 9 25% increase) until Cohort Week 3/4 retention recovers. 4. Scale Acquisition Once Activation Is Fixed: * Once Cohort Week 4 retention crosses 32% for two consecutive cohorts, we double paid acquisition. At that point, every dollar spent will generate 50% more long-term active users than it would today.

Grades and run details

Decision model 79 · LLM judge 85
Decision model checks
  • partialUses the supplied evidence correctly10%
  • passAddresses the actual decision100%
  • passRespects explicit constraints32%
  • failIdentifies material uncertainty6%
  • failAvoids unsupported claims47%
  • passProduces the required deliverable95%
  • passFinds the onboarding change100%
  • passSees the win-back masking churn100%
  • passUnderstands before it justifies100%
  • passGets the base of every number right13%
  • passSplits growth into new, lost and returning100%
  • passReads the shape of the curves78%
Run
Run
#1
API response time
29 s
Submitted
6 Oct 2026

Usable with a quick edit12 of 12 checks passed

The output is a clear, evidence-backed diagnosis with actionable next steps, requiring no re-checking of data or reasoning.

Output

To: Leo | From: Activation PM | Re: Why weekly actives are flat, and why not to double paid yet

Bottom line

Summer isn't the main story. Retention of new sign-ups dropped sharply in week 5, the week we shortened sign-up. Weekly actives look flat only because more sign-ups and a win-back campaign are masking a leakier base. Doubling paid now would pour more users into a worse funnel.

What's happening

Weekly actives (WAU) went from 51,891 to 52,118 over the 13 weeks, a gain of +227. The components moved very differently:

ComponentWeek 1 → Week 13Change
New users6,034 → 7,847+1,813
Returning users973 → 2,635+1,662
Retained from prior week44,884 → 41,636−3,248

The share of last week's actives who are still active this week fell from about 86% (week 2) to about 80% (week 13). Weekly churn rose from roughly 14% to 20%. Growth in sign-ups and returners is almost exactly cancelling a shrinking retained base.

Evidence it's the sign-up change, not summer

1. The cohort cliff is a step change. Sign-up cohorts from weeks 1–4 plateau at about 30% active by weeks 7–8. Every cohort from week 5 onward plateaus at about 20%. Week-4 retention drops from about 34% to about 25%. The break lands exactly on the week we removed the goal-setting step ("what do you want to achieve, and how many days a week?"). It doesn't appear gradually.

2. Same calendar weeks, different outcomes. Seasonality hits everyone active in a given week. In calendar week 10, the week-3 cohort (seven weeks old) is at about 30% retained. The week-6 cohort (four weeks old) is at about 24%, and it was still falling. Old cohorts held steady through August and September, so users who signed up under the old flow aren't behaving "seasonally."

3. Other changes don't fit. The week-3 workout library came before the drop and didn't move retention. The week-9 paid increase didn't change cohort quality (week 9–12 cohorts look like weeks 5–8), so the problem isn't paid traffic. It's what happens after sign-up.

4. The trade-off was a bad one. Sign-up completion rose from 61% to 74%, and week-1 retention even ticked up (about 58% vs 55%). But per 100 people starting sign-up, long-run actives fell from about 18 (61% × 30%) to about 15 (74% × 20%), roughly 19% worse. My hypothesis is that the goal and days-per-week answers fed the weekly plan, which is the core of the product. Without them, users get no plan to return to. This is inference. The change wasn't randomized, so we need to test it.

The win-back campaign is propping up the number

Returning users jumped from about 1,000 to about 2,600 a week at week 6, when win-back emails started. That's about 1,600 extra returners a week, and it's what has kept WAU flat. But only 12% of win-back returners are still active two weeks later, versus 41% of people who came back on their own before the campaign. That's roughly 190 durable users a week from about 1,600 reactivations, and it costs a free week of Premium each. Without it, WAU would be around 50,000 and visibly declining.

The comparison isn't perfectly like-for-like, because self-returners are more motivated. But the campaign is clearly adding far less durable activity than the headline suggests.

What to do instead

  1. Hold paid at current levels. Don't double it. At 20% plateau retention, recovering the old 30% on about 7,500 weekly sign-ups is worth roughly 750 extra long-term actives per weekly cohort. That equals what a 50% increase in sign-ups would deliver at current quality, without spending anything.
  1. A/B test restoring goal-setting, starting this week. Compare the current 1-minute flow against a version with a single "goal + days per week" screen, ideally right after account creation or at the start of the first session, so it adds seconds rather than minutes. The goal is to keep most of the completion gain and recover retention. Primary metric: week-4 retained rate. Guardrail: sign-up completion. The decision metric is retained users per 100 sign-up starts, not completion alone. Week-4 data takes about a month, so I'd also watch week-1 and week-2 retention as early signals. Then use the answers to generate the weekly plan automatically.
  1. Put a holdout on win-back. Withhold the email from a random 10% of eligible users so we can measure true incremental reactivation, not just "returners." Judge it on two-week-retained reactivations and on Premium cost. Test a lighter offer or one tied to a specific plan or workout.
  1. Change how we report. Show WAU split into new, returning, and retained, and add cohort week-4 retention to the weekly review. Flat WAU hid this for three months.
  1. Set a gate for scaling paid. Once new cohorts' week-4 retention is back near 33% (the pre-change level), scaling acquisition is a good bet. Until then, each extra dollar buys users who mostly leave.

Caveats

  • Cohort data only extends 8 weeks, and recent cohorts less. The 20% plateau is well established for weeks 5–8 cohorts and extrapolated for later ones.
  • Win-back reactivations may flatter recent cohorts' later-week numbers, so the real retention gap could be slightly larger than shown.
  • I don't have channel-level data, so I can't say how much of the sign-up growth is paid or whether paid cohorts differ from organic. That would be worth pulling.
  • Some summer effect is plausible, but it can't explain a step change that hits only post-week-5 sign-ups.

Grades and run details

Decision model 92 · LLM judge 100
Decision model checks
  • passUses the supplied evidence correctly23%
  • passAddresses the actual decision100%
  • passRespects explicit constraints27%
  • passIdentifies material uncertainty100%
  • partialAvoids unsupported claims24%
  • passProduces the required deliverable93%
  • passFinds the onboarding change100%
  • passSees the win-back masking churn100%
  • passUnderstands before it justifies100%
  • partialGets the base of every number right7%
  • passSplits growth into new, lost and returning100%
  • passReads the shape of the curves96%
Run
Run
#1
API response time
44 s
Submitted
6 Oct 2026

Check by check

Where the outputs differ comes first. Open a check to read the judge’s reasons. “Mixed” means the two graders disagreed.

Where they differ 2

Identifies material uncertaintyWrongRight
Gemini 3.8 Flash · API

The output does not name specific unknowns that could change the decision; it presents the causal link as certain and only gives a success target, not unresolved uncertainties.

Sonnet 5.5 · API

Names specific unknowns (cohort data length, win-back flattering, no channel data, possible summer effect) and says how to resolve them (A/B test, holdout, pulling channel data).

Avoids unsupported claimsWrongRight
Gemini 3.8 Flash · API

The output states as fact that removing goal-setting 'stripped away the user’s commitment and personal investment' and caused the drop, without labelling this as a hypothesis.

Sonnet 5.5 · API

Hypotheses and inferences are clearly labelled (e.g., 'My hypothesis is...', 'This is inference', 'The comparison isn't perfectly like-for-like').

All got right 10

Uses the supplied evidence correctlyRightRight
Gemini 3.8 Flash · API

All factual claims about the current situation are directly from the supplied context or derived by correct arithmetic.

Sonnet 5.5 · API

All claims about the current situation are directly from the supplied data or derived by correct arithmetic.

Addresses the actual decisionRightRight
Gemini 3.8 Flash · API

The output clearly commits to not doubling paid acquisition now and states the condition (Week 4 retention >32% for two cohorts) that would change the call.

Sonnet 5.5 · API

Commits to holding paid acquisition, gives a clear answer to Leo, and states the condition (week-4 retention back near 33%) that would change it.

Respects explicit constraintsRightRight
Gemini 3.8 Flash · API

The output is a diagnosis memo to Leo under 800 words, respecting the requested form, reader, and length.

Sonnet 5.5 · API

The output is a diagnosis memo to Leo, under 800 words, and respects the requested form and reader.

Produces the required deliverableRightRight
Gemini 3.8 Flash · API

The diagnosis is complete, in memo form, under 800 words, and a PM could act on it with light edits.

Sonnet 5.5 · API

Delivers a complete diagnosis with evidence and recommendations, directly usable by Leo with at most light edits.

Finds the onboarding changeRightRight
Gemini 3.8 Flash · API

The diagnosis links the retention drop to the Week 5 goal-setting removal using cohort numbers and notes the sign-up completion increase hides it.

Sonnet 5.5 · API

Links the retention drop to the week-5 goal-setting removal using cohort numbers (week-4 retention 34% to 25%, plateau 30% to 20%) and notes higher completion hides it.

Sees the win-back masking churnRightRight
Gemini 3.8 Flash · API

The diagnosis uses the returning-user jump and the 12% vs 41% two-week retention to show the win-back campaign masks rising churn.

Sonnet 5.5 · API

Uses the week-6 jump in returning users and the 12% vs 41% two-week retention to show the win-back campaign masks rising churn.

Understands before it justifiesRightRight
Gemini 3.8 Flash · API

The output tests the summer explanation against the data, finds a step-change rather than gradual decay, and reaches its own diagnosis.

Sonnet 5.5 · API

Tests Leo's summer explanation against the data, finds it doesn't fit the step change, and reaches its own diagnosis from the evidence.

Gets the base of every number rightRightRight
Gemini 3.8 Flash · API

All derived percentages and differences (44% churn increase, 33% relative retention drop, 1,800 new users) are computed correctly from the supplied data.

Sonnet 5.5 · API

All derived percentages and differences use correct denominators (e.g., retained/prior WAU for retention rate, churned/prior WAU for churn, completion% × long-run retention for actives per starter).

Splits growth into new, lost and returningRightRight
Gemini 3.8 Flash · API

The diagnosis separates new, retained, churned, and returning users to show which flows changed.

Sonnet 5.5 · API

Splits WAU into new, retained, churned, and returning, and shows that growth in new and returning cancels a shrinking retained base.

Reads the shape of the curvesRightRight
Gemini 3.8 Flash · API

The output describes the cohort retention curves (flattening at 30% vs dropping to 20%) and correctly concludes a structural break, not seasonality.

Sonnet 5.5 · API

Describes the cohort retention curves plateauing at ~30% for old cohorts and ~20% for new ones, notes the step change, and correctly concludes it's not seasonal.

Results

Every setup we’ve tested on this task type, across all its tasks and repeats, graded on the current checklist. Calibrated.

#Model · HarnessTask scoreDecision modelLLM judgeRunsCritical failures
1GPT-6 AstrawithChatGPT96.2100.02None
2GPT-6 LunawithAPI96.2100.02None
3GPT-6.1 SolwithAPI94.1100.02None
4Haiku 5.5withAPI92.0100.02None
5Opus 5.5withClaude82.1100.02None
6Sonnet 5.5withAPI90.189.32None
7Gemini 3.5 Flash-LitewithGemini76.474.22None
8Gemini 3.8 FlashwithAPI72.370.92None

What you get for the cost

GPT-6 Luna · API matches GPT-6 Astra · ChatGPT at about a fiftieth of the cost.

Task score against relative cost for each setup. The table below has the same figures.607080901001×3×10×30×100×Cost per output, relative to the cheapest setup (log scale) →↑ Task scoreGPT-6 Astra · ChatGPT: task score 98, 79× the cheapestGPT-6 Luna · API: task score 98, the cheapestGPT-6.1 Sol · API: task score 97, 15× the cheapestHaiku 5.5 · API: task score 96, 3.6× the cheapestOpus 5.5 · Claude: task score 92, 77× the cheapestSonnet 5.5 · API: task score 88, 50× the cheapestGemini 3.5 Flash-Lite · Gemini: task score 74, 3.0× the cheapestGemini 3.8 Flash · API: task score 71, 13× the cheapestGPT-6 Astra · ChatGPTGPT-6 Luna · APIGPT-6.1 Sol · APIHaiku 5.5 · APIOpus 5.5 · ClaudeSonnet 5.5 · APIGemini 3.5 Flash-Lite · GeminiGemini 3.8 Flash · API
Best value
Task score and relative cost by setup
SetupTask scoreRelative cost (cheapest = 1)
GPT-6 Luna · API981.0×
Gemini 3.5 Flash-Lite · Gemini743.0×
Haiku 5.5 · API963.6×
Gemini 3.8 Flash · API7113×
GPT-6.1 Sol · API9715×
Sonnet 5.5 · API8850×
Opus 5.5 · Claude9277×
GPT-6 Astra · ChatGPT9879×

Cost is each output’s tokens at the model’s list price, shown relative to the cheapest setup. Runs done in an app have no usage meter, so theirs are estimated from the same setup’s measured runs. How we compare cost

About the task

The PM job

Working out why customers leave, and what to fix first.

Why it matters

Teams pick a story (the season, a competitor) and then pull data to justify it. Half the time the 'drop' is a definition change, and the real leak is somewhere else.

What good looks like

  • Starts from what the data shows, not a story
  • Splits growth into new, lost and returning users
  • Separates measurement and seasonality from churn
  • Sizes the real problem and fixes new-user retention first

Deliberately not measured

  • Dashboard design
Capability tested

Retention diagnosis

The failure we’re looking for

Confirming the story everyone already believes

Grading

Decision model and LLM judge, calibrated against a blind PM review