Diagnose a retention problem

To: Leo | From: Activation PM | Re: Why weekly actives are flat, and why not to double paid yet

Bottom line

Summer isn't the main story. Retention of new sign-ups dropped sharply in week 5, the week we shortened sign-up. Weekly actives look flat only because more sign-ups and a win-back campaign are masking a leakier base. Doubling paid now would pour more users into a worse funnel.

What's happening

Weekly actives (WAU) went from 51,891 to 52,118 over the 13 weeks, a gain of +227. The components moved very differently:

The share of last week's actives who are still active this week fell from about 86% (week 2) to about 80% (week 13). Weekly churn rose from roughly 14% to 20%. Growth in sign-ups and returners is almost exactly cancelling a shrinking retained base.

Evidence it's the sign-up change, not summer

1. The cohort cliff is a step change. Sign-up cohorts from weeks 1–4 plateau at about 30% active by weeks 7–8. Every cohort from week 5 onward plateaus at about 20%. Week-4 retention drops from about 34% to about 25%. The break lands exactly on the week we removed the goal-setting step ("what do you want to achieve, and how many days a week?"). It doesn't appear gradually.

2. Same calendar weeks, different outcomes. Seasonality hits everyone active in a given week. In calendar week 10, the week-3 cohort (seven weeks old) is at about 30% retained. The week-6 cohort (four weeks old) is at about 24%, and it was still falling. Old cohorts held steady through August and September, so users who signed up under the old flow aren't behaving "seasonally."

3. Other changes don't fit. The week-3 workout library came before the drop and didn't move retention. The week-9 paid increase didn't change cohort quality (week 9–12 cohorts look like weeks 5–8), so the problem isn't paid traffic. It's what happens after sign-up.

4. The trade-off was a bad one. Sign-up completion rose from 61% to 74%, and week-1 retention even ticked up (about 58% vs 55%). But per 100 people starting sign-up, long-run actives fell from about 18 (61% × 30%) to about 15 (74% × 20%), roughly 19% worse. My hypothesis is that the goal and days-per-week answers fed the weekly plan, which is the core of the product. Without them, users get no plan to return to. This is inference. The change wasn't randomized, so we need to test it.

The win-back campaign is propping up the number

Returning users jumped from about 1,000 to about 2,600 a week at week 6, when win-back emails started. That's about 1,600 extra returners a week, and it's what has kept WAU flat. But only 12% of win-back returners are still active two weeks later, versus 41% of people who came back on their own before the campaign. That's roughly 190 durable users a week from about 1,600 reactivations, and it costs a free week of Premium each. Without it, WAU would be around 50,000 and visibly declining.

The comparison isn't perfectly like-for-like, because self-returners are more motivated. But the campaign is clearly adding far less durable activity than the headline suggests.

What to do instead

1. Hold paid at current levels. Don't double it. At 20% plateau retention, recovering the old 30% on about 7,500 weekly sign-ups is worth roughly 750 extra long-term actives per weekly cohort. That equals what a 50% increase in sign-ups would deliver at current quality, without spending anything.

1. A/B test restoring goal-setting, starting this week. Compare the current 1-minute flow against a version with a single "goal + days per week" screen, ideally right after account creation or at the start of the first session, so it adds seconds rather than minutes. The goal is to keep most of the completion gain and recover retention. Primary metric: week-4 retained rate. Guardrail: sign-up completion. The decision metric is retained users per 100 sign-up starts, not completion alone. Week-4 data takes about a month, so I'd also watch week-1 and week-2 retention as early signals. Then use the answers to generate the weekly plan automatically.

1. Put a holdout on win-back. Withhold the email from a random 10% of eligible users so we can measure true incremental reactivation, not just "returners." Judge it on two-week-retained reactivations and on Premium cost. Test a lighter offer or one tied to a specific plan or workout.

1. Change how we report. Show WAU split into new, returning, and retained, and add cohort week-4 retention to the weekly review. Flat WAU hid this for three months.

1. Set a gate for scaling paid. Once new cohorts' week-4 retention is back near 33% (the pre-change level), scaling acquisition is a good bet. Until then, each extra dollar buys users who mostly leave.

Caveats

• Cohort data only extends 8 weeks, and recent cohorts less. The 20% plateau is well established for weeks 5–8 cohorts and extrapolated for later ones.
• Win-back reactivations may flatter recent cohorts' later-week numbers, so the real retention gap could be slightly larger than shown.
• I don't have channel-level data, so I can't say how much of the sign-up growth is paid or whether paid cohorts differ from organic. That would be worth pulling.
• Some summer effect is plausible, but it can't explain a step change that hits only post-week-5 sign-ups.

To: Leo, Head of Growth From: PM, Activation Subject: Diagnosis: Why WAU is flat (and why doubling paid acquisition right now will burn cash)

Leo, you suggested doubling paid acquisition to push through flat summer WAU. I’ve dug into the numbers, and the data tells a different story.

Weekly actives are flat (averaging ~52,000 for 13 weeks), but sign-ups are up 30% (from ~6,000 to ~7,800). The issue isn't seasonal churn—it's that our onboarding changes broke cohort retention, and our win-back campaigns are masking the leak.

Doubling down on paid acquisition right now will just pour more water into a bucket with a widening hole. Here is the diagnosis, the evidence, and what we should do instead.

---

1. What’s actually happening?

#### A. Onboarding optimization created low-intent users (Week 5) In Week 5, we removed the goal-setting step to cut sign-up time from 3 minutes to 1. Completion jumped from 61% to 74%, and sign-ups grew.

However, look at the cohort data (`weeklysignupcohorts.csv`). For Cohorts 1–4 (pre-change), Week 1 retention hovered around 55%, and Week 4 retention was ~34%. From Cohort 5 onward, Week 1 retention jumped to ~58% (thanks to frictionless sign-up), but Week 4 retention cratered from ~34% down to ~25%.

By removing goal-setting, we optimized for top-of-funnel conversion at the expense of user intent. We are acquiring users who don't know why they are here or what plan fits them, leading them to drop off heavily by week 4.

#### B. Core retention is quietly declining Look at the growth accounting data (`weeklygrowthaccounting.csv`). Total `retained_users` (users active week-over-week) dropped steadily from ~44,800 in Week 1 to ~41,600 in Week 13, a loss of over 3,200 loyal weekly actives.

#### C. Win-backs are masking the bleeding So why is WAU flat if retained users dropped by 3,200? Look at `returning_users`. From Weeks 1–5, returning users averaged ~980 per week. Starting in Week 6 (when the win-back email launched), returning users spiked to ~2,600 per week.

The win-back campaign is successfully dragging churned users back, propping up our headline WAU. But it’s a leaky bucket: lifecycle data shows that only 12% of win-back users are active two weeks later (compared to 41% of organic returning users). They churn right back out.

---

2. Why "Summer" isn't to blame

Your theory about summer (outdoors, holidays) would manifest as broad seasonal churn across all cohorts. But our data shows: 1. Older cohorts (1–4) maintained stable long-term retention. 2. The drop in retention is strictly tied to Cohort 5 onwards—the exact week we shipped the shorter sign-up flow.

It is product-led churn, not seasonal.

---

3. What we should do instead of doubling paid acquisition

Before we scale ad spend, we need to fix activation and stop relying on expensive band-aids. Let’s execute a 3-step pivot:

1. Re-introduce intent-matching to onboarding (without hurting conversion): We need to qualify users so they receive the right weekly plan. Instead of a heavy 3-minute form, let’s test a lightweight, single-tap goal selector or defer personalization to the end of the first workout. We must restore Week 4 retention from 25% back toward 34%.
2. Refine the win-back targeting: Pumping win-back emails to all churned users after 21 days is bringing back low-intent users who churn in 14 days. Let’s restrict win-backs to users who reached a milestone or completed at least 3 workouts before churning, matching the profile of our 41% organic returners.
3. Optimize the Week 3–4 activation loop: We shipped 40 new routines in Week 3, but post-onboarding drop-off is neutralizing it. We need to focus activation efforts on guiding new users to these routines before they hit the Week 3 cliff.

Bottom line: Let's hold off on doubling paid acquisition for 4 weeks. If we fix the onboarding leak and improve post-signup intent, organic growth + recovered retention will naturally push us past our WAU plateau without wasting ad dollars on leaky traffic. Let's sync on this today.