Usable with a quick edit9 of 11 checks passed
The output is a complete, data-driven review with clear recommendations and next steps, requiring no re-checking of evidence or reasoning.
Output
To: Omar | Re: Onboarding review
Recommendation
Activation should mean connecting a bank and categorising 10+ transactions in the first week. Our current definition, completing the setup checklist, barely predicts retention, so the 61% is flat for a reason: we're measuring the wrong thing. I'd fix the path to the real activation event, use the design team's ideas only on that path, and replace next quarter's goal.
Evidence (4,000 August sign-ups, day-60 retention)
| First-week behaviour | Users | Retained at day 60 |
|---|---|---|
| All sign-ups | 4,000 | 28% |
| Completed checklist | 2,438 | 30% |
| Did not complete checklist | 1,562 | 24% |
| Connected bank | 2,000 | 48% |
| Did not connect bank | 2,000 | 8% |
| Categorised 10+ | 1,380 | 65% |
| Budget created | 1,230 | 33% |
| Opened app 4+ days | 2,180 | 40% |
| Bank AND 10+ categorised | 1,080 | 78% |
Three things stand out:
- The checklist is a weak signal. Checklist completers retain at 30%, against 28% for everyone. The checklist is theme, notifications, account name and goal, so it's mostly cosmetic setup.
- The combination matters, not either step alone. Backing out the overlaps from the CSV:
- Bank only (no categorising): about 920 users, 13% retained.
- Categorised only (no bank): about 300 users, 16% retained.
- Neither: about 1,700 users, 6% retained.
- Both: 78% retained.
Users who connect but never see their spending sorted do worse than average. The value moment is seeing where the money goes, which needs both steps. - Budgets and app opens aren't the answer. Budget creators retain at 33%, close to baseline. Opening the app 4+ days (40%) is engagement, not a lever we can pull in onboarding.
Caveat: this is observational. Motivated users probably do everything, so these numbers show who retains, not what causes it. The recommendations below are therefore tests, not guarantees.
What to change
The funnel is 100% → 50% connect → 27% connect and categorise. There are two leaks:
- Half of users never connect a bank, and they retain at 8%. Put bank connection first, straight after sign-up. Move theme, notifications and account name out of the critical path (defer or drop them). Look at where the connect flow fails and whether trust copy (read-only access, security) is needed. I can't tell from this data whether the checklist is causing bank drop-off, so I'd A/B test bank-first against the current order.
- 46% of those who connect (920 of 2,000) never reach 10 categorised transactions. This is the biggest fixable gap, since 78% of those who finish retain. Auto-categorise on connect, pre-fill best guesses, and walk users through their first 10 with one-tap confirmations rather than leaving them to find the feature.
I'd also instrument the funnel by step and by time, so we can see where in the first week people stall.
The design team's proposal
A celebration screen and a progress bar for the checklist would probably raise checklist completion, but that metric barely relates to retention, so we'd likely see a "win" with no change in retained users. I wouldn't ship them as proposed. Both ideas are cheap and sound, though, so I'd redirect them:
- Make the progress bar track the real path (connect bank → categorise 10).
- Put the celebration at the moment of value, such as the 10th categorised transaction, ideally with a first insight ("here's where your money went").
This is a good way to involve design in the work above rather than rejecting their proposal.
Next quarter's goal
Drop "checklist completion 61% → 75%". Pushing it up could even pull effort toward low-value steps.
Replace it with: percentage of sign-ups who connect a bank and categorise 10+ transactions within 7 days. Baseline is 27%. I'd propose a target around 35%, but that is a judgment call until we've sized the effect of the first experiments, so let's set it after a few weeks of data. Guardrail: day-60 retention among those users should hold near its current 78% (if we widen the funnel with less-motivated users, it may dip, and we should watch that). Track the real outcome, overall day-60 retention (28% today), alongside it.
Next steps
- Run the bank-first vs current-order A/B.
- Prototype auto-categorisation and a guided first-10 flow.
- Work with design on the redirected progress bar and celebration.
- Agree the new activation definition and dashboard before the quarter starts.
I'm happy to walk through the numbers if useful.
Grades and run details
Decision model 82 · LLM judge 100
Decision model checks
- failUses the supplied evidence correctly77%
- passAddresses the actual decision89%
- passRespects explicit constraints51%
- passIdentifies material uncertainty91%
- passAvoids unsupported claims35%
- passProduces the required deliverable65%
- passComputes retention by behaviour46%
- passTests before trusting the correlation95%
- passActivation defined by what predicts retention100%
- failGets the base of every number right56%
- passGets people to value sooner71%
Run
- Run
- #1
- API response time
- 26 s
- Submitted
- 6 Oct 2026