Diagnose a retention problem

Leo — I wouldn’t double paid acquisition yet. We have a retention leak, not just a summer growth slowdown. More new users and campaign-driven returns are masking deteriorating retention. The clearest break coincides with the week-5 onboarding change, though we need an experiment to establish causality.

What’s actually happening

Weekly actives remain around 52,000 because inflows are replacing increasingly large losses:

New users rose 30%, but weekly churn rose 44%. The share of the prior week’s actives retained fell from approximately 86.3% to 80.2%. In week 13, 10,482 new and returning users barely exceeded 10,279 churned users: net growth was just 203.

Flat actives are therefore concealing a less durable user base.

The evidence points to an activation problem

1. New cohorts look better initially, then retain substantially worse. For signup cohorts 1–4, first-following-week retention averaged 55.1%; for cohorts 5–12, it averaged 58.0%. But by the fourth following week, retention falls from 33.8% for cohorts 1–4 to 24.8% for cohorts 5–9. At eight weeks, the earlier cohorts retain roughly 30%, versus 19.9% for cohort 5.

That is not simply fewer people trying Stride. More are getting through the front door, but fewer form a lasting habit.

2. The break aligns with removing goal-setting in week 5. Shorter signup raised completion from 61% to 74%, but removed the step asking users what they wanted to achieve and how often they would exercise. For a habit app built around a weekly plan, that step may have helped users commit to a relevant routine.

The timing makes this our leading hypothesis—not proven causality. We should also check acquisition mix and other changes. However, the paid-budget increase occurred in week 9, after the cohort deterioration began. “Summer” alone does not explain this sharp onboarding-aligned break.

3. Win-back is propping up the headline metric, not repairing retention. Returning users jumped from about 1,000 to 2,600 a week when emails launched in week 6. Yet only 12% of email-driven returners remained active two weeks later, versus 41% of earlier organic returners. These are different populations, so this is not a causal campaign comparison, but it clearly warns against treating every return as durable recovery.

What we should do before spending more

1. Hold acquisition spend steady while testing onboarding. Randomize the current short signup against a lightweight goal-setting step that produces a personalized weekly plan. Preserve the speed improvement where possible.

1. Optimize for durable activation, not signup completion. Make four-week retention a primary outcome, alongside first workout, plan adoption and repeat workout behavior. Measure retained users per signup and per signup starter so a longer flow cannot “win” merely by filtering people out. Track Premium conversion as a guardrail.

1. Locate the failure point. Segment cohorts by acquisition channel, campaign and onboarding exposure. Compare where users stop: before their first workout, after one workout, or when beginning the next weekly plan. Reconcile the signup and new-active counts before calculating acquisition efficiency.

1. Judge win-back on incremental lasting activity. Use a randomized holdout and measure two- and four-week retention, not email-driven returns alone. Test messages that restore a realistic plan rather than relying only on free Premium.

Decision: scale acquisition once we demonstrate improved multiweek retention and acceptable acquisition economics. Doubling spend now risks buying more users into a leaking habit loop—and making flat actives look healthier without fixing the underlying problem.

Leo — I wouldn’t double paid acquisition yet. The flat WAU is masking a retention problem, not showing that summer is the whole story.

• The active base is eroding while inflows hide it. From week 1 to week 13, new users rose from 6,034 to 7,847 (+30%), but retained users fell from 44,884 to 41,636 (−7%). Weekly churn rose from 7,116 to 10,279. WAU stayed near 52,000 because more new users—and, from week 6, more returning users—offset that loss.
• New-user retention deteriorates sharply after week 4. For the first four signup cohorts, week-4 retention was 33–34%, and week-8 retention about 30%. For cohorts in weeks 5–8, week-4 retention was 24–25%; week-5 retention was about 22%, versus about 32% for the earlier cohorts at the same age. Week-1 retention actually improved (roughly 54–55% to 58%), then users fell away faster. So sign-ups are easier to get, but fewer people build a lasting workout habit.
• The timing points to a product change worth investigating. In week 5, we removed the goal-setting step. Sign-up completion rose from 61% to 74%, while the cohort retention pattern worsened immediately. That’s strong evidence to investigate whether the change brought in less-committed users or removed useful personalization—not proof that it caused the decline. Seasonality may contribute, but it doesn’t explain away the sharp cohort break.
• Win-back lifts WAU more than durable retention. Returning users rose from about 1,000 to about 2,600 a week after the week-6 campaign. But only 12% of email-returners were active two weeks later, compared with 41% of people who returned on their own. The campaign may be adding short-lived activity, not repairing the underlying habit problem.

What I’d do before scaling spend:

1. Test the onboarding change now. A/B test restoring goal-setting (or a shorter version) against the current flow. Measure signup completion alongside week-1 activation, week-4/8 retention, and workouts completed—not just registrations.
2. Improve the first-week habit loop. Give users a clear, personalized weekly plan and a timely next-workout prompt. Track results by signup week, onboarding version, and acquisition channel.
3. Put win-back on a holdout. Measure incremental, sustained activity and Premium conversion against users who don’t receive the offer; don’t count the initial return as success.
4. Hold off on doubling acquisition. The week-9 paid increase came after the retention decline had already begun. Scale only when newer cohorts retain better and paid users show acceptable downstream value.

Fixing the leak first gives us a better chance of turning additional acquisition into durable actives rather than more short-lived sign-ups.

To: Leo — Fix the retention leak before doubling acquisition

Weekly actives are flat because rising acquisition and reactivation are offsetting worsening retention. Summer could contribute, but the clearest break in the data follows the week-5 onboarding change—not the later increase in paid spend.

What’s happening

1. We are replacing more users, not building a larger active base.

Weekly active users equal retained users + new users + returning users. Equivalently, weekly net growth equals new + returning − churned.

From week 1 to week 13: - New users rose 30%, from 6,034 to 7,847. - Returning users rose from 973 to 2,635. - Weekly churn rose 44%, from 7,116 to 10,279. - Retained users fell from 44,884 to 41,636.

The result: weekly actives moved just 0.4%, from 51,891 to 52,118. The share of the previous week’s actives retained fell from 86.3% to 80.2%. This is deterioration beneath a flat headline.

2. New users look better after one week, but substantially worse afterward.

Compare signup cohorts before and after the week-5 release:

Only the week-5 post-change cohort has an observed week-8 result. Blank cells are unobserved, not zero.

The week-4 decline is roughly nine percentage points, or 27% relative, and repeats across multiple mature post-change cohorts. Earlier cohorts flatten near 30% by weeks 7–8; the first mature post-change cohort flattens near 20%. That suggests a smaller durable habit-forming base, not merely delayed engagement.

3. Win-back is masking the problem, not fixing it.

Returning users jumped immediately after win-back emails began in week 6. Lifecycle attributes about 1,600 extra weekly returns to the campaign, but only 12% of email-returned users were active two weeks later, versus 41% of earlier organic returners.

Those populations are not comparable enough to establish a causal campaign effect. Nevertheless, most campaign returners are not sustaining activity. Repeated returns and lapses can support WAU without creating lasting growth.

Likely diagnosis—and what remains uncertain

In week 5, we removed goal-setting and weekly-frequency selection. Signup completion improved from 61% to 74%, but subsequent cohorts became much less durable.

The leading hypothesis is that we removed a commitment and planning step central to a fitness habit. Alternatively—or additionally—the easier funnel admitted more low-intent users, lowering average retention. Both could be happening.

Timing alone does not prove causality. We lack randomized evidence, acquisition-source breakdowns, and a prior-year seasonal comparison. Still, the abrupt cohort break starts before the week-9 paid-social increase and persists afterward. “It’s summer” is not a sufficient diagnosis. Also, new-cohort deterioration may not explain all aggregate churn; we should examine established users separately.

What we should do before spending more

1. Hold paid spend at its current level. Don’t double distribution into a deteriorating retention curve.

1. Test lightweight goal-setting immediately. Randomize new visitors between current onboarding and a short goal/frequency step that produces a concrete weekly plan. Measure week-4 retained users per onboarding entrant, not just signup completion or retention among completed signups. This captures the conversion–retention trade-off. Track first-workout completion and second-week adherence as early signals; Premium conversion and revenue as business guardrails.

1. Localize the leak. Segment equal-age cohorts by paid/organic source, campaign, onboarding exposure, and initial workout/plan behavior. Check existing-user retention around week 5. Reconcile the differing signup totals across the two exports before joining them.

1. Evaluate win-back with a randomized holdout. Optimize for incremental sustained activity and eventual revenue—not email-driven return visits alone.

Increase acquisition only after we demonstrate improved durable retention and acceptable channel economics. Right now, more acquisition would buy more arrivals; it has not demonstrated that it will buy lasting growth.