Usable with a quick edit7 of 10 checks passed
The invite placement error is a minor factual slip that can be corrected quickly; the rest is structured and actionable.
Output
MEMO
To: Onboarding squad Re: Where we lose people, and what to fix first
Where we lose people
- Signup to connect (100% to 41%): The largest loss. Most of our users are PMs and marketers who don't have database credentials, and the flow gives them no way forward without them.
- Connect to first chart (41% to 33%): About a fifth of people who connect never build a chart. Sync and the empty canvas are the likely causes.
- First chart to invite (33% to 9%): Expected, since invites come after value. Lower priority for now.
Retention is the reason to care about the first chart: 62% of users who reach one are active in week 2, versus 11% of those who don't. This is correlational, since engaged users may be the ones who build charts, but it is the strongest signal we have.
What to fix, in order
- Add sample data and a skip at step 5. Users without credentials currently hit a dead end. Sample data gets them to a chart in minutes, which is the milestone that predicts retention. This also targets the top support ticket (41%). Reason: highest expected impact on the number that matters.
- Let people in before email verification, and drop the role and team screens from the critical path. Step 2 blocks all progress, and steps 3–4 ask questions before the user has seen any value. Move role and team to after the first chart, where they can personalize templates. Reason: removes the earliest friction, when drop-off is highest.
- Rebuild the credentials path. Replace the 14-page guide with short, per-warehouse steps for Postgres, BigQuery and Snowflake, and add a "send these instructions to your data engineer" link that delivers a connect-only invite. The person who has the credentials is often not the person signing up. Reason: addresses the remaining credential tickets and the step 6 drop-off.
- Fix sync. Let leaving the page not cancel the sync. Show partial results as tables finish. Pre-select likely tables and add search to the step 7 list. Make CSV upload the fastest route to a first chart. Reason: sync failures are the second-largest ticket category (18%), and cancellation on exit loses users mid-setup.
- Make the first chart appear without effort. Templates (step 9) should render a live chart immediately. Replace the empty canvas at step 10 with a suggested chart the user can edit. Reason: a blank canvas is the last hurdle before the retention milestone.
- Move team invites after the first chart, and prompt for them there. Keep the skip link. Reason: invites are useful but come after value; test this once the upstream fixes land.
Measurement
Track signup-to-first-chart as the primary metric, with week-2 retention as the outcome. Run fixes 1 and 2 as separate experiments so we can see which one moves the funnel.
Claims the judge couldn’t find in the brief
- Most of our users are PMs and marketers who don't have database credentials.
- Sync and the empty canvas are the likely causes (of drop-off from connect to first chart).
- The person who has the credentials is often not the person signing up.
- There is a 'step 6 drop-off' that the credentials path fix addresses.
- Invite teammates is currently before the first chart (implied by 'Move team invites after the first chart').
Grades and run details
Decision model 70 · LLM judge 82
Decision model checks
- failUses the supplied evidence correctly33%
- failAddresses the actual decision46%
- passRespects explicit constraints38%
- partialIdentifies material uncertainty30%
- partialAvoids unsupported claims38%
- passProduces the required deliverable97%
- passPrioritises by impact73%
- passActivation defined by what predicts retention100%
- passGets the base of every number right45%
- passGets people to value sooner100%
Run
- Run
- #1
- API response time
- 11 s
- Submitted
- 8 Oct 2026