Usable with a quick edit12 of 12 checks passed
The output is a clear, evidence-backed diagnosis with actionable next steps, requiring no re-checking of data or reasoning.
Output
To: Leo | From: Activation PM | Re: Why weekly actives are flat, and why not to double paid yet
Bottom line
Summer isn't the main story. Retention of new sign-ups dropped sharply in week 5, the week we shortened sign-up. Weekly actives look flat only because more sign-ups and a win-back campaign are masking a leakier base. Doubling paid now would pour more users into a worse funnel.
What's happening
Weekly actives (WAU) went from 51,891 to 52,118 over the 13 weeks, a gain of +227. The components moved very differently:
| Component | Week 1 → Week 13 | Change |
|---|---|---|
| New users | 6,034 → 7,847 | +1,813 |
| Returning users | 973 → 2,635 | +1,662 |
| Retained from prior week | 44,884 → 41,636 | −3,248 |
The share of last week's actives who are still active this week fell from about 86% (week 2) to about 80% (week 13). Weekly churn rose from roughly 14% to 20%. Growth in sign-ups and returners is almost exactly cancelling a shrinking retained base.
Evidence it's the sign-up change, not summer
1. The cohort cliff is a step change. Sign-up cohorts from weeks 1–4 plateau at about 30% active by weeks 7–8. Every cohort from week 5 onward plateaus at about 20%. Week-4 retention drops from about 34% to about 25%. The break lands exactly on the week we removed the goal-setting step ("what do you want to achieve, and how many days a week?"). It doesn't appear gradually.
2. Same calendar weeks, different outcomes. Seasonality hits everyone active in a given week. In calendar week 10, the week-3 cohort (seven weeks old) is at about 30% retained. The week-6 cohort (four weeks old) is at about 24%, and it was still falling. Old cohorts held steady through August and September, so users who signed up under the old flow aren't behaving "seasonally."
3. Other changes don't fit. The week-3 workout library came before the drop and didn't move retention. The week-9 paid increase didn't change cohort quality (week 9–12 cohorts look like weeks 5–8), so the problem isn't paid traffic. It's what happens after sign-up.
4. The trade-off was a bad one. Sign-up completion rose from 61% to 74%, and week-1 retention even ticked up (about 58% vs 55%). But per 100 people starting sign-up, long-run actives fell from about 18 (61% × 30%) to about 15 (74% × 20%), roughly 19% worse. My hypothesis is that the goal and days-per-week answers fed the weekly plan, which is the core of the product. Without them, users get no plan to return to. This is inference. The change wasn't randomized, so we need to test it.
The win-back campaign is propping up the number
Returning users jumped from about 1,000 to about 2,600 a week at week 6, when win-back emails started. That's about 1,600 extra returners a week, and it's what has kept WAU flat. But only 12% of win-back returners are still active two weeks later, versus 41% of people who came back on their own before the campaign. That's roughly 190 durable users a week from about 1,600 reactivations, and it costs a free week of Premium each. Without it, WAU would be around 50,000 and visibly declining.
The comparison isn't perfectly like-for-like, because self-returners are more motivated. But the campaign is clearly adding far less durable activity than the headline suggests.
What to do instead
- Hold paid at current levels. Don't double it. At 20% plateau retention, recovering the old 30% on about 7,500 weekly sign-ups is worth roughly 750 extra long-term actives per weekly cohort. That equals what a 50% increase in sign-ups would deliver at current quality, without spending anything.
- A/B test restoring goal-setting, starting this week. Compare the current 1-minute flow against a version with a single "goal + days per week" screen, ideally right after account creation or at the start of the first session, so it adds seconds rather than minutes. The goal is to keep most of the completion gain and recover retention. Primary metric: week-4 retained rate. Guardrail: sign-up completion. The decision metric is retained users per 100 sign-up starts, not completion alone. Week-4 data takes about a month, so I'd also watch week-1 and week-2 retention as early signals. Then use the answers to generate the weekly plan automatically.
- Put a holdout on win-back. Withhold the email from a random 10% of eligible users so we can measure true incremental reactivation, not just "returners." Judge it on two-week-retained reactivations and on Premium cost. Test a lighter offer or one tied to a specific plan or workout.
- Change how we report. Show WAU split into new, returning, and retained, and add cohort week-4 retention to the weekly review. Flat WAU hid this for three months.
- Set a gate for scaling paid. Once new cohorts' week-4 retention is back near 33% (the pre-change level), scaling acquisition is a good bet. Until then, each extra dollar buys users who mostly leave.
Caveats
- Cohort data only extends 8 weeks, and recent cohorts less. The 20% plateau is well established for weeks 5–8 cohorts and extrapolated for later ones.
- Win-back reactivations may flatter recent cohorts' later-week numbers, so the real retention gap could be slightly larger than shown.
- I don't have channel-level data, so I can't say how much of the sign-up growth is paid or whether paid cohorts differ from organic. That would be worth pulling.
- Some summer effect is plausible, but it can't explain a step change that hits only post-week-5 sign-ups.
Grades and run details
Decision model 92 · LLM judge 100
Decision model checks
- passUses the supplied evidence correctly23%
- passAddresses the actual decision100%
- passRespects explicit constraints27%
- passIdentifies material uncertainty100%
- partialAvoids unsupported claims24%
- passProduces the required deliverable93%
- passFinds the onboarding change100%
- passSees the win-back masking churn100%
- passUnderstands before it justifies100%
- partialGets the base of every number right7%
- passSplits growth into new, lost and returning100%
- passReads the shape of the curves96%
Run
- Run
- #1
- API response time
- 44 s
- Submitted
- 6 Oct 2026