Diagnose a retention problem

To: Leo | From: Activation PM | Re: Why weekly actives are flat, and why not to double paid yet

Bottom line

Summer isn't the main story. Retention of new sign-ups dropped sharply in week 5, the week we shortened sign-up. Weekly actives look flat only because more sign-ups and a win-back campaign are masking a leakier base. Doubling paid now would pour more users into a worse funnel.

What's happening

Weekly actives (WAU) went from 51,891 to 52,118 over the 13 weeks, a gain of +227. The components moved very differently:

The share of last week's actives who are still active this week fell from about 86% (week 2) to about 80% (week 13). Weekly churn rose from roughly 14% to 20%. Growth in sign-ups and returners is almost exactly cancelling a shrinking retained base.

Evidence it's the sign-up change, not summer

1. The cohort cliff is a step change. Sign-up cohorts from weeks 1–4 plateau at about 30% active by weeks 7–8. Every cohort from week 5 onward plateaus at about 20%. Week-4 retention drops from about 34% to about 25%. The break lands exactly on the week we removed the goal-setting step ("what do you want to achieve, and how many days a week?"). It doesn't appear gradually.

2. Same calendar weeks, different outcomes. Seasonality hits everyone active in a given week. In calendar week 10, the week-3 cohort (seven weeks old) is at about 30% retained. The week-6 cohort (four weeks old) is at about 24%, and it was still falling. Old cohorts held steady through August and September, so users who signed up under the old flow aren't behaving "seasonally."

3. Other changes don't fit. The week-3 workout library came before the drop and didn't move retention. The week-9 paid increase didn't change cohort quality (week 9–12 cohorts look like weeks 5–8), so the problem isn't paid traffic. It's what happens after sign-up.

4. The trade-off was a bad one. Sign-up completion rose from 61% to 74%, and week-1 retention even ticked up (about 58% vs 55%). But per 100 people starting sign-up, long-run actives fell from about 18 (61% × 30%) to about 15 (74% × 20%), roughly 19% worse. My hypothesis is that the goal and days-per-week answers fed the weekly plan, which is the core of the product. Without them, users get no plan to return to. This is inference. The change wasn't randomized, so we need to test it.

The win-back campaign is propping up the number

Returning users jumped from about 1,000 to about 2,600 a week at week 6, when win-back emails started. That's about 1,600 extra returners a week, and it's what has kept WAU flat. But only 12% of win-back returners are still active two weeks later, versus 41% of people who came back on their own before the campaign. That's roughly 190 durable users a week from about 1,600 reactivations, and it costs a free week of Premium each. Without it, WAU would be around 50,000 and visibly declining.

The comparison isn't perfectly like-for-like, because self-returners are more motivated. But the campaign is clearly adding far less durable activity than the headline suggests.

What to do instead

1. Hold paid at current levels. Don't double it. At 20% plateau retention, recovering the old 30% on about 7,500 weekly sign-ups is worth roughly 750 extra long-term actives per weekly cohort. That equals what a 50% increase in sign-ups would deliver at current quality, without spending anything.

1. A/B test restoring goal-setting, starting this week. Compare the current 1-minute flow against a version with a single "goal + days per week" screen, ideally right after account creation or at the start of the first session, so it adds seconds rather than minutes. The goal is to keep most of the completion gain and recover retention. Primary metric: week-4 retained rate. Guardrail: sign-up completion. The decision metric is retained users per 100 sign-up starts, not completion alone. Week-4 data takes about a month, so I'd also watch week-1 and week-2 retention as early signals. Then use the answers to generate the weekly plan automatically.

1. Put a holdout on win-back. Withhold the email from a random 10% of eligible users so we can measure true incremental reactivation, not just "returners." Judge it on two-week-retained reactivations and on Premium cost. Test a lighter offer or one tied to a specific plan or workout.

1. Change how we report. Show WAU split into new, returning, and retained, and add cohort week-4 retention to the weekly review. Flat WAU hid this for three months.

1. Set a gate for scaling paid. Once new cohorts' week-4 retention is back near 33% (the pre-change level), scaling acquisition is a good bet. Until then, each extra dollar buys users who mostly leave.

Caveats

• Cohort data only extends 8 weeks, and recent cohorts less. The 20% plateau is well established for weeks 5–8 cohorts and extrapolated for later ones.
• Win-back reactivations may flatter recent cohorts' later-week numbers, so the real retention gap could be slightly larger than shown.
• I don't have channel-level data, so I can't say how much of the sign-up growth is paid or whether paid cohorts differ from organic. That would be worth pulling.
• Some summer effect is plausible, but it can't explain a step change that hits only post-week-5 sign-ups.

To: Leo — Fix the retention leak before doubling acquisition

Weekly actives are flat because rising acquisition and reactivation are offsetting worsening retention. Summer could contribute, but the clearest break in the data follows the week-5 onboarding change—not the later increase in paid spend.

What’s happening

1. We are replacing more users, not building a larger active base.

Weekly active users equal retained users + new users + returning users. Equivalently, weekly net growth equals new + returning − churned.

From week 1 to week 13: - New users rose 30%, from 6,034 to 7,847. - Returning users rose from 973 to 2,635. - Weekly churn rose 44%, from 7,116 to 10,279. - Retained users fell from 44,884 to 41,636.

The result: weekly actives moved just 0.4%, from 51,891 to 52,118. The share of the previous week’s actives retained fell from 86.3% to 80.2%. This is deterioration beneath a flat headline.

2. New users look better after one week, but substantially worse afterward.

Compare signup cohorts before and after the week-5 release:

Only the week-5 post-change cohort has an observed week-8 result. Blank cells are unobserved, not zero.

The week-4 decline is roughly nine percentage points, or 27% relative, and repeats across multiple mature post-change cohorts. Earlier cohorts flatten near 30% by weeks 7–8; the first mature post-change cohort flattens near 20%. That suggests a smaller durable habit-forming base, not merely delayed engagement.

3. Win-back is masking the problem, not fixing it.

Returning users jumped immediately after win-back emails began in week 6. Lifecycle attributes about 1,600 extra weekly returns to the campaign, but only 12% of email-returned users were active two weeks later, versus 41% of earlier organic returners.

Those populations are not comparable enough to establish a causal campaign effect. Nevertheless, most campaign returners are not sustaining activity. Repeated returns and lapses can support WAU without creating lasting growth.

Likely diagnosis—and what remains uncertain

In week 5, we removed goal-setting and weekly-frequency selection. Signup completion improved from 61% to 74%, but subsequent cohorts became much less durable.

The leading hypothesis is that we removed a commitment and planning step central to a fitness habit. Alternatively—or additionally—the easier funnel admitted more low-intent users, lowering average retention. Both could be happening.

Timing alone does not prove causality. We lack randomized evidence, acquisition-source breakdowns, and a prior-year seasonal comparison. Still, the abrupt cohort break starts before the week-9 paid-social increase and persists afterward. “It’s summer” is not a sufficient diagnosis. Also, new-cohort deterioration may not explain all aggregate churn; we should examine established users separately.

What we should do before spending more

1. Hold paid spend at its current level. Don’t double distribution into a deteriorating retention curve.

1. Test lightweight goal-setting immediately. Randomize new visitors between current onboarding and a short goal/frequency step that produces a concrete weekly plan. Measure week-4 retained users per onboarding entrant, not just signup completion or retention among completed signups. This captures the conversion–retention trade-off. Track first-workout completion and second-week adherence as early signals; Premium conversion and revenue as business guardrails.

1. Localize the leak. Segment equal-age cohorts by paid/organic source, campaign, onboarding exposure, and initial workout/plan behavior. Check existing-user retention around week 5. Reconcile the differing signup totals across the two exports before joining them.

1. Evaluate win-back with a randomized holdout. Optimize for incremental sustained activity and eventual revenue—not email-driven return visits alone.

Increase acquisition only after we demonstrate improved durable retention and acceptable channel economics. Right now, more acquisition would buy more arrivals; it has not demonstrated that it will buy lasting growth.

Leo — I wouldn’t double paid acquisition yet. We have a retention leak, not just a summer growth slowdown. More new users and campaign-driven returns are masking deteriorating retention. The clearest break coincides with the week-5 onboarding change, though we need an experiment to establish causality.

What’s actually happening

Weekly actives remain around 52,000 because inflows are replacing increasingly large losses:

New users rose 30%, but weekly churn rose 44%. The share of the prior week’s actives retained fell from approximately 86.3% to 80.2%. In week 13, 10,482 new and returning users barely exceeded 10,279 churned users: net growth was just 203.

Flat actives are therefore concealing a less durable user base.

The evidence points to an activation problem

1. New cohorts look better initially, then retain substantially worse. For signup cohorts 1–4, first-following-week retention averaged 55.1%; for cohorts 5–12, it averaged 58.0%. But by the fourth following week, retention falls from 33.8% for cohorts 1–4 to 24.8% for cohorts 5–9. At eight weeks, the earlier cohorts retain roughly 30%, versus 19.9% for cohort 5.

That is not simply fewer people trying Stride. More are getting through the front door, but fewer form a lasting habit.

2. The break aligns with removing goal-setting in week 5. Shorter signup raised completion from 61% to 74%, but removed the step asking users what they wanted to achieve and how often they would exercise. For a habit app built around a weekly plan, that step may have helped users commit to a relevant routine.

The timing makes this our leading hypothesis—not proven causality. We should also check acquisition mix and other changes. However, the paid-budget increase occurred in week 9, after the cohort deterioration began. “Summer” alone does not explain this sharp onboarding-aligned break.

3. Win-back is propping up the headline metric, not repairing retention. Returning users jumped from about 1,000 to 2,600 a week when emails launched in week 6. Yet only 12% of email-driven returners remained active two weeks later, versus 41% of earlier organic returners. These are different populations, so this is not a causal campaign comparison, but it clearly warns against treating every return as durable recovery.

What we should do before spending more

1. Hold acquisition spend steady while testing onboarding. Randomize the current short signup against a lightweight goal-setting step that produces a personalized weekly plan. Preserve the speed improvement where possible.

1. Optimize for durable activation, not signup completion. Make four-week retention a primary outcome, alongside first workout, plan adoption and repeat workout behavior. Measure retained users per signup and per signup starter so a longer flow cannot “win” merely by filtering people out. Track Premium conversion as a guardrail.

1. Locate the failure point. Segment cohorts by acquisition channel, campaign and onboarding exposure. Compare where users stop: before their first workout, after one workout, or when beginning the next weekly plan. Reconcile the signup and new-active counts before calculating acquisition efficiency.

1. Judge win-back on incremental lasting activity. Use a randomized holdout and measure two- and four-week retention, not email-driven returns alone. Test messages that restore a realistic plan rather than relying only on free Premium.

Decision: scale acquisition once we demonstrate improved multiweek retention and acceptable acquisition economics. Doubling spend now risks buying more users into a leaking habit loop—and making flat actives look healthier without fixing the underlying problem.