Tasks / Design

Activation & onboarding review

Can the model find the friction that matters most and prioritise the fixes?

Measures the modelTask v1.0 · 2 casesDifficulty

The PM job

Reviewing a signup and onboarding flow that is losing users.

Why it matters

Anyone can list fifty UX nits. The job is finding the two that explain the drop-off, backed by the funnel data supplied.

What good looks like

Ties each issue to the funnel data
Prioritises by likely impact
Distinguishes activation from mere completion

Deliberately not measured

Accessibility audit completeness
Visual redesign

Capability tested

Consequential critique

The failure we’re looking for

A generic UX checklist

Grading

Decision model, LLM judge and blind PM review

Results

Every setup we’ve tested on this task, across all cases and repeats.

#	Model · Harness	Task score	Decision model	LLM judge	PM review	Runs	Critical failures	Cost / run	Latency

Case viewer

Read the brief, then put up to three outputs side by side. The outputs are the point; the scores just tell you where to look.

The brief

Review the first-week experience of our fitness app, described below, and recommend the changes you would make, in priority order, as a memo to the growth PM. Keep it under 500 words.

ScenarioStride is a home-workout app. It costs £9.99 a month after a seven-day free trial, and the trial needs a card up front.

First week, step by stepDay 0: nine onboarding questions (goal, age, height, weight, fitness level, equipment, injuries, preferred days, preferred time), then the paywall to start the trial, then a plan with the first workout scheduled for the next morning. Every workout is 30–45 minutes. Day 1: a 7am push notification, “Your workout is ready.” Days 2–7: one push a day at the preferred time. Missing a workout moves the plan on to the next one; there is no shorter or catch-up option. The streak counter first appears after seven days.

FunnelInstall → finished onboarding 71% → started trial 52% → first workout 48% → three workouts in week one 19%. Users who complete three workouts in week one are four times as likely to still be subscribed at day 60.

Team planThe design team wants to spend this quarter redesigning the onboarding questionnaire and the paywall screen.

What a strong answer does

Focus on getting new users to three workouts in week one (shorter first workouts, a same-day first session, catch-up options, earlier progress feedback) rather than onboarding and paywall polish.

Case

v1.2 · synthetic · consumer, mobile