Diagnose a retention problem

To: Leo (Head of Growth) From: PM, Activation Subject: Hold on doubling paid spend — our flat WAU is an activation and win-back issue, not summer seasonality

Leo,

Before we double paid acquisition, we need to look closely at what is happening beneath our top-line WAU. While it looks like a summer slump, the data shows our flat WAU (~52,000) is actually caused by two product and lifecycle changes from August: the Week 5 onboarding change broke new-user habit formation, and the Week 6 win-back campaign is masking that core retention drop with temporary, low-intent resurrected users.

Doubling paid spend right now will pour expensive acquisition into a leaky bucket. Here is the diagnosis and what we should do before scaling spend.

---

1. Why it’s not summer seasonality

If summer weather and holidays were driving our flat WAU, we would see a broad dip across all users in July and August, followed by a recovery in September. Instead: * Pre-Week 5 cohorts are rock solid: Users who signed up in Weeks 1–4 retained at ~30% by Week 8 (hitting late August and September) with zero seasonal decay. * Retention is still worsening in late September: Our weekly churned users rose from ~7,100 in early July to 10,279 in Week 13 (Sept 28). Meanwhile, our core `retained_users` (active both this week and last week) have steadily eroded from ~45,000 in Weeks 1–4 down to 41,636 in Week 13.

2. Root Cause #1: The Week 5 shorter sign-up broke long-term activation

In Week 5 (Aug 3), we removed the goal-setting step ("What do you want to achieve, and how many days a week?") to cut sign-up time from 3 minutes to 1. Sign-up completion jumped from 61% to 74%, and Week 1 retention ticked up ~3 percentage points (from ~55% to ~58%).

However, long-term cohort retention collapsed immediately starting with the Week 5 cohort: * Week 4 Retention: Dropped from ~33.8% (Weeks 1–4 cohorts) to ~24.7% (Weeks 5–9 cohorts)—a ~9 percentage point drop. * Week 8 Retention: Dropped from ~30.0% (Weeks 1–4 cohorts) to 19.9% (Week 5 cohort)—a one-third relative loss in long-term retained users.

Why this happened: Stride’s core value is a weekly workout plan. By removing the step where users state their goals and target workout days, we likely broke the mechanism that personalizes their weekly plan and anchors their habit commitment. Furthermore, cutting friction let through lower-intent users who curious-click in Week 1, skewing Week 1 retention up while failing to build a multi-week habit.

The Math: Even with a 21% boost in sign-up completion (61% $\rightarrow$ 74%), a 33% relative drop in Week 8 retention (30% $\rightarrow$ 19.9%) means every 1,000 visitors to our sign-up flow now yields ~19% fewer long-term active users than before Week 5.

3. Root Cause #2: The Week 6 win-back campaign is inflating WAU and churn

Why hasn’t WAU fallen if core retention is down? Because in Week 6 (Aug 10), lifecycle launched the 21-day inactive win-back email offering a free week of Premium: * Resurrected users spiked: `returningusers` jumped from ~990/week (Weeks 1–5) to ~2,600/week (Weeks 6–13)—adding ~1,600 temporary users per week. * They immediately re-churn: Lifecycle data shows only 12% of win-back users remain active two weeks later (compared to 41% of organic returning users). Once their free week of Premium ends, nearly 90% (~1,400 users/week) churn right back out. * This explains why weekly `churnedusers` spiked from ~7,400 in Week 4 to >9,100 in Week 6 and >10,300 by Week 11. We are propping up headline WAU by cycling ~1,600 promo-chasers in and out of the app every week while our true core base (`retained_users`) shrinks by ~3,200 users.

(Note: When we increased paid social by 25% in Week 9, new users grew, W2–W4 cohort percentages held steady at the post-Week 5 levels, and `retained_users` still dropped by another ~500 users over the next four weeks.)

---

What we should do before scaling acquisition

1. Hold off on doubling paid acquisition (and revert the 25% Week 9 increase if CAC is inefficient): Right now, 80% of new sign-ups churn by Week 7. Doubling spend at current retention rates will burn budget for a fraction of the LTV we were getting in July. 2. Restore goal-setting to the activation flow (this week): We need to A/B test bringing back the goal and frequency questions immediately: * Variant A (Control): Current 1-minute sign-up. * Variant B (Revert): Original 3-minute sign-up with goal-setting. * Variant C (Deferred Onboarding): Keep account creation fast, but require the goal-setting & weekly schedule commitment immediately before the user starts their first workout or generates their first weekly plan. * Success metric: Net retained users at Week 4 per 1,000 top-of-funnel visitors (not sign-up completion or W1 retention). 3. Fix the win-back mechanic: Giving a free week of Premium to unactivated users doesn't build a habit. We should shift the win-back hook from a temporary paywall unlock to re-establishing a weekly plan (e.g., "Let's reset your weekly schedule—pick 2 days to work out this week"), and track `retainedusers` separately from win-back `returningusers` in our growth reporting.

Once we see Week 4 cohort retention recover back above 32%, we should turn the paid acquisition tap all the way up.

To: Leo | From: Activation PM | Re: Why WAU is flat, and why I'd hold off on doubling paid

Bottom line: This isn't a summer dip. Retention has been deteriorating since early August, and new sign-ups and win-back emails are hiding it. Doubling paid now would mostly buy users who leave faster.

1. What's happening

WAU = new + returning + retained users, which reconciles exactly every week. Across weeks 1–13, WAU moved only +227 (51,891 → 52,118). That flatness is three movements netting out:

• New users: +1,813 (up 30%, as you said)
• Returning users: +1,662, nearly all from win-back
• Retained users (active last week and this week): −3,248

Retained users held near 44.5–45k through week 4, then stepped down from week 5 and kept sliding to 41.6k. Churn (last week's actives who didn't return) rose from about 14% of prior-week actives in weeks 2–4 to about 20% in weeks 11–13.

2. Why it isn't summer

Seasonality would hit all cohorts alike. It doesn't. Cohorts 1–4 (signed up July to late July) had week-4 retention of 33–34%. Cohorts 5 onward (from the week of 3 August, the first week of the shorter sign-up) drop to 24–25% and stay there, even though cohorts 2–4 went through the same weather. The break lines up with the week-5 release, which removed the goal-setting step.

The funnel math shows the change is a net loss. Per 100 visitors:

We're converting more visitors into accounts, but by week 8 each signup is about 20% less likely to still be active. We're getting more users who don't build the habit.

3. Win-back is flattering the numbers

Win-back adds about 1,600 returning users a week. Of users who came back through a win-back email, 12% were still active two weeks later. Of users who came back on their own, 41% were. At 1,600 a week, that's roughly 190 durable returners instead of about 650 if they'd come back organically. We're also giving away a free week of Premium. "Returning users" now looks like a health signal, but most of it is low-quality.

The week-9 paid social increase (+25%) isn't visible as a break in the new-user trend, so I can't yet say what paid is adding.

What I'd do instead

1. Hold the paid increase. Keep current spend. Revisit when cohort week-4 retention is back above 30%.

1. Restore the goal-setting step, or move it into the app after the first workout. Run it as a holdout (50/50, old flow vs new), and judge on week-4 and week-8 retention per signup, not completion. Using the cohort tables, bringing cohort retention back to about 30% at week 4 is worth roughly 3–4k weekly actives at steady state. That's about the size of the whole retained-user decline. This is my rough estimate and needs confirming with the holdout.

1. Find the churn in tenured users. Most of the 41k retained base isn't new. Segment weeks 4–13 churn by tenure, plan usage, workout type, and platform. Also check whether win-back returners churn quickly the following week, since they add to churn in the same weeks.

1. Rework win-back. Test a smaller, conditional offer, or a guided re-entry workout instead of a free Premium week. Measure two-week retention against a holdout. Keep it only if the incremental retained users justify the cost.

1. Change what we report. Show the WAU decomposition (new, returning, retained) and cohort retention by signup week every week. Stop counting returners as growth.

On the summer theory: it's testable. If we have last year's data for the same weeks, compare churn rates. If summer were the cause, we'd expect last year's churn to rise over these weeks too, and we'd expect older cohorts to show the same pattern as new ones. The data so far points to the week-5 change.

Ask: Can we set up the sign-up holdout this week and pull the tenure-level churn cut before any paid budget decision?

To: Leo | From: Activation PM | Re: Why weekly actives are flat, and why not to double paid yet

Bottom line

Summer isn't the main story. Retention of new sign-ups dropped sharply in week 5, the week we shortened sign-up. Weekly actives look flat only because more sign-ups and a win-back campaign are masking a leakier base. Doubling paid now would pour more users into a worse funnel.

What's happening

Weekly actives (WAU) went from 51,891 to 52,118 over the 13 weeks, a gain of +227. The components moved very differently:

The share of last week's actives who are still active this week fell from about 86% (week 2) to about 80% (week 13). Weekly churn rose from roughly 14% to 20%. Growth in sign-ups and returners is almost exactly cancelling a shrinking retained base.

Evidence it's the sign-up change, not summer

1. The cohort cliff is a step change. Sign-up cohorts from weeks 1–4 plateau at about 30% active by weeks 7–8. Every cohort from week 5 onward plateaus at about 20%. Week-4 retention drops from about 34% to about 25%. The break lands exactly on the week we removed the goal-setting step ("what do you want to achieve, and how many days a week?"). It doesn't appear gradually.

2. Same calendar weeks, different outcomes. Seasonality hits everyone active in a given week. In calendar week 10, the week-3 cohort (seven weeks old) is at about 30% retained. The week-6 cohort (four weeks old) is at about 24%, and it was still falling. Old cohorts held steady through August and September, so users who signed up under the old flow aren't behaving "seasonally."

3. Other changes don't fit. The week-3 workout library came before the drop and didn't move retention. The week-9 paid increase didn't change cohort quality (week 9–12 cohorts look like weeks 5–8), so the problem isn't paid traffic. It's what happens after sign-up.

4. The trade-off was a bad one. Sign-up completion rose from 61% to 74%, and week-1 retention even ticked up (about 58% vs 55%). But per 100 people starting sign-up, long-run actives fell from about 18 (61% × 30%) to about 15 (74% × 20%), roughly 19% worse. My hypothesis is that the goal and days-per-week answers fed the weekly plan, which is the core of the product. Without them, users get no plan to return to. This is inference. The change wasn't randomized, so we need to test it.

The win-back campaign is propping up the number

Returning users jumped from about 1,000 to about 2,600 a week at week 6, when win-back emails started. That's about 1,600 extra returners a week, and it's what has kept WAU flat. But only 12% of win-back returners are still active two weeks later, versus 41% of people who came back on their own before the campaign. That's roughly 190 durable users a week from about 1,600 reactivations, and it costs a free week of Premium each. Without it, WAU would be around 50,000 and visibly declining.

The comparison isn't perfectly like-for-like, because self-returners are more motivated. But the campaign is clearly adding far less durable activity than the headline suggests.

What to do instead

1. Hold paid at current levels. Don't double it. At 20% plateau retention, recovering the old 30% on about 7,500 weekly sign-ups is worth roughly 750 extra long-term actives per weekly cohort. That equals what a 50% increase in sign-ups would deliver at current quality, without spending anything.

1. A/B test restoring goal-setting, starting this week. Compare the current 1-minute flow against a version with a single "goal + days per week" screen, ideally right after account creation or at the start of the first session, so it adds seconds rather than minutes. The goal is to keep most of the completion gain and recover retention. Primary metric: week-4 retained rate. Guardrail: sign-up completion. The decision metric is retained users per 100 sign-up starts, not completion alone. Week-4 data takes about a month, so I'd also watch week-1 and week-2 retention as early signals. Then use the answers to generate the weekly plan automatically.

1. Put a holdout on win-back. Withhold the email from a random 10% of eligible users so we can measure true incremental reactivation, not just "returners." Judge it on two-week-retained reactivations and on Premium cost. Test a lighter offer or one tied to a specific plan or workout.

1. Change how we report. Show WAU split into new, returning, and retained, and add cohort week-4 retention to the weekly review. Flat WAU hid this for three months.

1. Set a gate for scaling paid. Once new cohorts' week-4 retention is back near 33% (the pre-change level), scaling acquisition is a good bet. Until then, each extra dollar buys users who mostly leave.

Caveats

• Cohort data only extends 8 weeks, and recent cohorts less. The 20% plateau is well established for weeks 5–8 cohorts and extrapolated for later ones.
• Win-back reactivations may flatter recent cohorts' later-week numbers, so the real retention gap could be slightly larger than shown.
• I don't have channel-level data, so I can't say how much of the sign-up growth is paid or whether paid cohorts differ from organic. That would be worth pulling.
• Some summer effect is plausible, but it can't explain a step change that hits only post-week-5 sign-ups.