Usable with a quick edit6 of 9 checks passed
The evidence and priorities are usable, but it needs a sentence stating what would change the order and softer labelling of causal claims; no re-checking or redoing of the reasoning is needed.
Output
To: Onboarding Squad Subject: Where onboarding is leaking—and what to fix first
The biggest loss is before a data source is connected: only 41% of signups get that far. Another 8 percentage points drop before a first chart. Just 9% of signups invite a teammate. The strongest activation signal is the chart: week-2 retention is 62% for users who reach one, versus 11% for those who don’t. Prioritize getting people to a useful chart; treat invites as a later step.
Fixes, in priority order
- Offer a sample-data path that reaches a real first chart. Let users skip connecting their own data and explore a preloaded dataset, with a clear route to connect later. Many signups are PMs and marketers, and the most common onboarding ticket is missing database credentials (41%). A required technical connection is an especially poor first step for them. This also gives users value while they wait for access.
- Make connecting data simpler and recoverable. Replace or supplement the 14-page guide with short, connector-specific instructions and troubleshooting. Validate credentials early, explain the required access, and offer a clear retry path. Don’t cancel a sync when someone leaves the page; send progress and failure notices and let users resume. The credential and sync tickets point to concrete blockers, and successful connection is the gateway to the high-retention chart experience.
- Get users to a chart with fewer decisions. Replace the database-wide table list with suggested tables and plain-language descriptions; cut the 12 dashboard templates to a few recommended starting points. Provide a prebuilt chart or guided first-chart flow rather than an empty canvas. The 8-point drop between connection and first chart, plus the sync burden, suggests too much work remains after connection.
- Remove early friction that doesn’t create value. Test deferring email verification until after users enter the product, and move role/team questions until they can personalize something useful. These steps currently delay access before users see value.
- Make inviting a teammate contextual, not a hurdle. Keep it optional; prompt after a user has made or shared a chart, and explain the benefit. Only 9% of signups invite someone, but chart reach—not inviting—is the clearest retention signal.
Track step conversion and time-to-first-chart by path, especially sample-data versus live connection, before expanding the changes.
Check by check
Got wrong · 2
- Identifies material uncertaintyIt does not name specific unknowns that could change the priorities or say how resolving them would alter the call.
- Avoids unsupported claimsIt presents causal or evaluative claims such as 'successful connection is the gateway' and 'a required technical connection is an especially poor first step' as established fact rather than hypothesis.
Mixed · 1
- Addresses the actual decisionThe memo commits to a clear ranked order but never states what result or condition would change that order; the closing sentence is only about expanding changes.The two graders disagreed on this one.
Got right · 6
- Uses the supplied evidence correctlyAll factual statements about the current situation are taken correctly from the brief and supplied context, with no invented facts.
- Respects explicit constraintsIt is a memo to the onboarding squad, well under 500 words, and respects the requested form and reader.
- Produces the required deliverableThe required memo is complete, actionable, and usable by the onboarding squad as written or with light edits.
- Prioritises by impactFixes are ranked by likely impact on reaching a first chart, tied to the retention signal.
- Activation defined by what predicts retentionIt names first chart as the activation behaviour based on 62% vs 11% retention and ranks fixes by their effect on it.
- Gets the base of every number rightAll percentages use the correct denominators, and the 41% ticket figure is understood as a share of onboarding tickets rather than users.
Grades and run details
Decision model 89 · LLM judge 7 of 10 checks
Decision model checks
- passUses the supplied evidence correctly33%
- passAddresses the actual decision55%
- passRespects explicit constraints74%
- partialIdentifies material uncertainty63%
- partialAvoids unsupported claims52%
- passProduces the required deliverable97%
- passPrioritises by impact98%
- passActivation defined by what predicts retention100%
- passGets the base of every number right80%
Run
- Run
- #1
- API response time
- 12 s
- Submitted
- 29 Sept 2026