Tasks / Growth

Write a GTM plan

Can the model pick the customers who value the difference most, position against what they do today, and build a go-to-market that lasts beyond launch day?

Task type v1.1 · 2 tasksLast changed 6 Oct 2026 · ChangelogDifficulty

What AI gets right here, and what you’ll still have to catch

From 16 graded outputs by 8 models. 50% were usable with at most a quick edit.

Reliably right

Positioned against the real alternative100% pass
It names the real alternatives (accountant-run payroll with mistakes, owner spreadsheet with risk) and ties the message to confidence and accuracy that those alternatives lack.
GPT-6 Astra · ChatGPT · Payroll, for the people who fear it
Protects the self-serve engine100% pass
Uses the 70% free-to-paid stat and product-qualified deal data to argue against capping the free plan, and keeps self-serve open.
GPT-6 Astra · ChatGPT · Enterprise, without killing the engine
Does the capacity sums100% pass
Works out deals needed (69-80), qualified opportunities (134), and AE-years (8) with checkable arithmetic.
GPT-6 Astra · ChatGPT · Enterprise, without killing the engine

Where it slips

Avoids unsupported claims47% pass
Presents the incorrect 'five of six no decision' figure as established fact, which is an unsupported claim about the current situation.
Opus 5.5 · Claude · Enterprise, without killing the engine
Uses the supplied evidence correctly52% pass
The claim 'Five of our six losses on existing-user deals ended in no decision' is not supported by the data (4 no-decision, 2 Miro), so a statement about the current situation is invented.
Opus 5.5 · Claude · Enterprise, without killing the engine
Identifies material uncertainty73% pass
The plan does not name specific unknowns that could change the decision, nor does it say how they would be resolved or what would trigger a pivot.
Gemini 3.8 Flash · API · Payroll, for the people who fear it

How it’s graded

Each check is built on how world-class product and growth leaders do this work, as they’ve described it on Lenny’s Podcast, and names the leader it comes from.

Positioned against the real alternative
Does the plan name what the target customer does today without this product (including the status quo or doing nothing), and tie the message to the value that alternative lacks?
Passes when Names the real alternative and the differentiated value against it, and picks the segment that cares most about that value.
April DunfordLenny’s Podcast
Sells against a pain the buyer feels
Is the core message built on a pain or risk the target buyer already feels, with evidence from the material, rather than on generic upside like saving time?
Passes when Leads with the pain the evidence shows buyers care about, and uses that evidence to choose the message.
Jeanne DeWitt GrosserLenny’s Podcast
Diagnoses message or channel first
Does the plan work out whether the problem is the message (people reach it but don't convert) or the channels (the right people never see it), and spend accordingly?
Passes when Uses conversion and reach data to say which it is, and puts the effort there.
Emily KramerLenny’s Podcast
Plans beyond launch week
Does the plan say how demand keeps coming after launch, with the ongoing engine and what's measured over months, rather than treating launch day as the whole plan?
Passes when Gives the ongoing motion after launch and the measures that show whether it's working over time.
Arielle JacksonLenny’s Podcast

Plus our standard checks

Uses the supplied evidence correctly · Addresses the actual decision · Respects explicit constraints · Identifies material uncertainty · Avoids unsupported claims · Produces the required deliverable · and 2 written for each task, which you’ll see in the tasks below.

The tasks

Read the brief, then compare up to three outputs, each with the LLM judge’s verdict on every check. Highlights mark what a PM had to fix.

The brief

You're the PM for Ledgerly Payroll, a new add-on for our accounting product, launching to existing customers on 3 November. Marketing has drafted the launch plan below. Write the go-to-market plan you'd run instead (or the changes you'd make to theirs), in no more than 900 words: who we go after first, what we say to them, through which channels, and how we'll know it's working over the following three months. The files are attached.

What the model was given5 items: About Ledgerly, marketing_launch_draft.md, customers_by_size_and_payroll_method.csv (penalty notices: share who told us they got a payroll penalty notice last year), beta_headline_test.csv (two versions of the in-app payroll page, shown at random to beta customers), customer_calls.md

About LedgerlyAccounting software for small businesses: invoicing, bank feeds, bookkeeping. 8,000 paying customers. Payroll is a new add-on at $6 per employee per month.

marketing_launch_draft.md7 lines · Download

# Draft launch plan (Head of Marketing)

- Launch day 3 November: press release announcing Ledgerly as "the first Financial Operating System for small business".
- Paid social and search ads to new prospects: $60,000 in November.
- In-app banner to all 8,000 customers: "Run payroll in minutes".
- Webinar in launch week.
- Success measure: press mentions and sign-ups in launch week.

customers_by_size_and_payroll_method.csv (penalty notices: share who told us they got a payroll penalty notice last year)employees,current_payroll,customers,penalty_notices_last_year_pct 1-4,Owner does it in a spreadsheet,3140,4 1-4,Accountant runs it,1020,6 5-25,Owner does it in a spreadsheet,690,19 5-25,Accountant runs it,1480,23 5-25,Another payroll tool,1210,7 26-50,Another payroll tool,460,5

beta_headline_test.csv (two versions of the in-app payroll page, shown at random to beta customers)headline,landing_page_visits,started_setup,ran_first_payroll "Run payroll in minutes, right where you do your books",4210,92,38 Never get a payroll penalty again,4160,268,117

customer_calls.md10 lines · Download

# Customer calls about payroll (excerpts, September)

1. Bakery, 12 staff: "My accountant does payroll. Last spring they filed late and I got a $1,400 penalty. I don't know what I'm paying them for."
2. Plumbing firm, 18 staff: "Time isn't the issue. I'm scared of getting it wrong. The tax office letter was the worst week of my year."
3. Design studio, 3 staff: "It's me and two freelancers. A spreadsheet's fine."
4. Café group, 22 staff: "We use another tool, it's OK. Switching sounds like a nightmare mid-year."
5. Physio clinic, 9 staff: "I'd move tomorrow if it meant one less thing that can go wrong with the tax office."
…

What a strong answer doesThe answer key the graders mark against

Picks customers with 5 to 25 employees whose accountant runs payroll (1,480 customers, 23% had a penalty notice) and those doing it in a spreadsheet (690, 19%), not the smallest businesses (happy with a spreadsheet) or those on another payroll tool (switching mid-year is the barrier). Builds the message on fear of penalties and errors, not time: in the beta, 'Never get a payroll penalty again' got about three times as many setups as 'Run payroll in minutes' (268 against 92 from similar traffic, 6.4% against 2.2%) and more first payrolls (117 against 38), and the calls say the same. Reads this as a message problem, not a channel problem: the existing customer base already sees the page, so fix the message and reach existing customers in-app and through their accountants before spending $60,000 on ads to strangers. Drops the 'Financial Operating System' category claim. Plans beyond launch week: prompts around payroll and tax deadlines, accountant referrals, and monthly measures such as payroll attach in the target segment, first payrolls run, and penalties avoided.

Critical failures (cap the score at 40)

Keeps 'Run payroll in minutes' as the lead message despite the headline test

Task

Mid-level · v1.0

The outputs

8 setups answered this brief.

GPT-6 AstraChatGPT100Usable with a quick edit GPT-6 LunaAPI100Usable with a quick edit Opus 5.5Claude90Usable with a quick edit GPT-6.1 SolAPI87Usable with a quick edit Sonnet 5.5API85Needs more work Haiku 5.5API85Needs more work Gemini 3.8 FlashAPI83Usable with a quick edit Gemini 3.5 Flash-LiteGemini79Usable with a quick edit

Haiku 5.5API

Needs more than a quick edit10 of 12 checks passed

The plan assumes, without evidence, that individual penalty recipients can be pinpointed and that accountants are the decision-makers. A PM would need to verify these data and assumptions before acting.

Output

Ledgerly Payroll: Go-to-Market Plan

Core change: Sell payroll to existing customers who have the most to lose from getting it wrong, lead with accuracy and penalty protection, and hold most of the $60k until we have conversion data.

What I'd change in the draft

The headline. In the beta, "Never get a payroll penalty again" roughly tripled setup starts (6.4% vs 2.2%) and first payroll runs (2.8% vs 0.9%). But "never" is a promise we can't keep. I'd use claims the product can support, such as "Filing reminders and pre-payroll checks, so fewer penalties," and get legal sign-off before launch.
"First Financial Operating System." It's unverifiable and means nothing to a bakery owner. Drop it.
$60k on paid ads to new prospects. We haven't proven the product yet, and our 8,000 customers are cheaper to reach and already trust us. Paid spend should follow proven conversion.
Success measured by press mentions and launch-week sign-ups. Neither tells us whether customers pay, use payroll, or avoid penalties.

Who we go after, in order

Penalty exposure is concentrated in businesses with 5–25 staff:

Accountant-run at 5–25 staff: 23% got a penalty notice last year.
Spreadsheet-run at 5–25 staff: 19%.
Owner-run at 1–4 staff: 4%.
Other-tool users: 5–7%.

About 2,170 customers sit in the two high-penalty 5–25 groups, and roughly 470 of them got a notice last year.

Wave 1 (launch, 3 Nov): 5–25 staff, spreadsheet or accountant-run (~2,170 customers). This is where the penalty message earns its keep. Start with the customers who got a notice, using our survey data, then the rest of the segment.

Wave 2 (mid-November): owner-run at 1–4 staff (3,140 customers). This is the largest pool but has low penalty pain, so the message is time saved. Their Sunday-spreadsheet routine is the pitch.

Wave 3 (January): other-tool users (1,670 customers). Don't ask anyone to switch mid-year (customer 4 calls it "a nightmare"). Pitch migration at the start of the new year, and offer to do the import for them.

Later: accountant-run at 1–4 staff (1,020 customers). The accountant makes this decision, so reach them through the accountant channel below.

Skip payroll messaging for businesses with no payroll pain, such as the three-person design studio (customer 3).

What we say

Accountant-run, 5–25 staff: "Payroll filings checked and reminded before they're due. Keep your accountant and see their work in the same place." This addresses customer 2's fear of getting it wrong and customer 1's question about what they're paying for. The "keep your accountant" line is a hypothesis to test.
Spreadsheet-run, 5–25 staff: "Stop checking payroll on Sunday night," with penalty protection as the second line.
Owner-run, 1–4 staff: "Run payroll in minutes, right where you do your books." This is beta headline A, which is fine for a segment that cares about time, not penalties.
Other-tool users: "We'll move your payroll over for you at the start of the new year."

Accuracy claims have to match what the product does. Customer 8 says "If you can promise it's right, I'm in," so the proof needs to be concrete: which checks run, which reminders fire, and what happens when a figure looks wrong.

Channels

In-app, segmented (week 1). Show headline B to the 5–25 segments and headline A to the 1–4 segment. It's free and reaches everyone who logs in.
Targeted email and account-manager calls (weeks 1–2). Email wave 1 from named account managers. Have support call the highest-risk accounts, those who got a notice last year, first.
Accountant pilot (from launch, ~25 firms). Accountants serve many of our customers, and an accountant-run customer decides with their accountant. Invite the firms that manage the most of our customers to run payroll for clients inside Ledgerly. This means we need to decide up front how partners are paid.
Webinar (launch week). "A payroll penalty checklist for businesses with 5–25 staff," invited to wave 1 only. Narrow beats generic.
Paid (hold). Spend $10k on retargeting and search from week 3, but only if the wave 1 landing page converts at least as well as beta headline B. Hold the remaining ~$50k until the 60-day numbers justify it.
Press (low-key). A short release with a real customer quote and no "first" claim. It's cheap and supports the story without carrying the plan.

Measuring the next three months

Primary metric: payroll customers, meaning customers who have run at least one payroll, by segment, and the revenue from them ($6 per employee per month).

Targets (reset after the 30-day baseline): - Wave 1: 10% of the ~2,170 customers have run a payroll by day 90 (~220 customers, roughly $18k MRR at about 12 employees each). - Wave 2: 4% by day 90. - Wave 3: 10% of other-tool users migrated by end of January.

Decision rules: - If wave 1's first-payroll rate is under 2% after 30 days, pause paid spend and rewrite the messaging. - If it's over 5%, release the held budget to the segments that convert best.

Guardrails (any breach stops the rollout): - Zero filing errors or penalties traced to Ledgerly. - Support tickets per payroll run stay within agreed levels. - No rise in churn or refunds among existing customers.

Penalty outcomes won't show in three months. As a proxy, track on-time filing rates for adopters against matched non-adopters, and report it to the board in February.

Claims the judge couldn’t find in the brief

We have survey data identifying which specific customers received a penalty notice last year, allowing us to start with those individuals.
For accountant-run customers, the accountant makes the buying decision.

Grades and run details

Decision model 100 · LLM judge 77

Decision model checks

passUses the supplied evidence correctly10%
passAddresses the actual decision98%
passRespects explicit constraints30%
passIdentifies material uncertainty97%
passAvoids unsupported claims33%
passProduces the required deliverable69%
passPicks the segment with the pain100%
passUses the headline test97%
passPositioned against the real alternative100%
passSells against a pain the buyer feels100%
passDiagnoses message or channel first100%
passPlans beyond launch week100%

Run

Run: #1
API response time: 33 s
Submitted: 8 Oct 2026

Sonnet 5.5API

Needs more than a quick edit10 of 12 checks passed

The output incorrectly states that about half the base is a poor fit, which would require re-checking the data to correct; otherwise the plan is solid but that error undermines trust.

Output

Ledgerly Payroll: go-to-market plan

What I'd change in Marketing's draft

Draft	Problem	Change
$60k paid ads to new prospects	Payroll is an add-on, so a prospect would first have to adopt our accounting product. We have 8,000 customers and haven't yet proven the message with them.	Hold most of the budget until the message converts in-base (see Budget).
Banner to all 8,000	About half the base is a poor fit (details below).	Target about 2,170 customers first.
"Financial Operating System" press release	Customers never use this language. They talk about fear of penalties and the tax office.	Keep the press small and plain.
Success = press mentions and launch-week sign-ups	Neither measures payroll actually being run.	Measure first payroll run and repeat runs.

Who we go after first

Wave 1 (3 Nov): customers with 5–25 employees who run payroll via spreadsheet (690) or an accountant (1,480). That is 2,170 customers, 27% of the base.

They have the pain. 19% (spreadsheet) and 23% (accountant) got a payroll penalty notice last year, versus 4–6% for businesses with 1–4 staff and 5–7% for those already on a payroll tool.
They have the money. At $6 per employee, a 12-person business pays about $72 a month, against the $180 a month one landscaper pays their accountant.
The calls match the data. The bakery, plumber, physio, landscaper and dental practice all describe fear of mistakes, not lack of time.

Not now: - 1–4 employees (4,160 customers): The spreadsheet is "fine" and "free," and only 4–6% were penalized. Leave payroll available in the product but don't push it. - Other-tool users (1,670 customers): Penalty rates are low (5–7%), and switching mid-year "sounds like a nightmare." Revisit at the start of the next tax year with a migration offer.

What we say

The beta test was randomized. "Never get a payroll penalty again" beat "Run payroll in minutes" on setup starts (6.4% vs 2.2% of visits) and on first payroll runs (2.8% vs 0.9%), about 3x on both. Once people started, completion was similar (44% vs 41%), so the headline drives the decision to begin and the product handles the rest.

Core message: "Payroll filed right and on time, so the tax office letter never arrives."

Spreadsheet owners: "A spreadsheet won't tell you when you've got it wrong."
Accountant-run: "Stop paying $180 a month to worry about it." Use this to speak to cost and control without attacking accountants. Offer a way to invite their accountant in as a reviewer.

Accuracy promise: The dental practice said, "If you can promise it's right, I'm in." I'd like to offer a penalty-cover guarantee for errors caused by our software. Legal and Finance must approve it by 24 Oct. If they won't, the copy becomes "filed right and on time, with support to fix problems." We should not promise "never" without backing it. The beta headline did well, but a claim we can't stand behind creates risk.

Channels

Targeted in-app card and email to the Wave 1 list only, using the winning headline. This is the main channel, and it costs almost nothing.
Personal outreach from Customer Success to Wave 1 customers who told us they got a penalty notice (about 470 people by my estimate from the survey rates). Offer a 20-minute guided setup.
Webinar, reframed: "Penalty-proof your payroll" with a live setup walkthrough. Run it in launch week and again in week 4, since the second session catches people who weren't ready the first time.
Press: one plain release ("Ledgerly adds payroll, $6 per employee"), with no category claim. Follow with customer stories once we have them.
Paid: none to new prospects in November (see Budget).

Budget (the $60k)

~$15k: guided setup and migration help (contractor or CS time).
~$10k: webinars, email production, and case-study content.
~$35k: held until the January gate. If Wave 1 works, spend it on search ads for "payroll penalty" intent, aimed at businesses that would adopt accounting and payroll together.

How we'll know it's working (3 months)

Primary metric: paying accounts that have run at least two payrolls. This replaces sign-ups, press mentions and launch-week anything.

Targets (to be recalibrated after week 2, since the beta is my only baseline): - Month 1: at least 5% of Wave 1 has started setup. At least 45% of those run a first payroll, matching beta's 44%. - End of January: about 250 Wave 1 accounts running payroll (~11% of the segment). I'm assuming a 10-person average, which implies roughly $15k MRR. I'll replace that assumption with real headcount data. - Retention: at least 85% of those accounts run payroll in three consecutive cycles.

Guardrails (any breach pauses expansion): - Payroll errors or penalties caused by us: zero tolerance, reviewed weekly. - Support tickets per payroll run. - Churn of the core accounting subscription, which must not rise among adopters.

Measuring real impact: Hold out a random 10% of Wave 1 from all promotion. If adopters in the promoted group don't clearly exceed the holdout, we're only capturing people who would have found payroll anyway.

Decision gates: - Mid-December: If setup-to-first-payroll is below 35%, fix onboarding before pushing further. - Mid-January: If Wave 1 is at or above target and no guardrail is breached, release the held budget and start Wave 2 (other-tool users, ahead of the tax-year switch window). If it's below target, interview non-adopters before spending anything.

Caveats

Penalty rates are self-reported, and the eight calls are anecdotes.
Beta customers may be more enthusiastic than the base.
I haven't seen average headcount per segment, so the revenue estimates are rough.

Claims the judge couldn’t find in the brief

About half the base is a poor fit

Grades and run details

Decision model 100 · LLM judge 77

Decision model checks

passUses the supplied evidence correctly13%
passAddresses the actual decision99%
passRespects explicit constraints32%
passIdentifies material uncertainty99%
passAvoids unsupported claims38%
passProduces the required deliverable75%
passPicks the segment with the pain100%
passUses the headline test100%
passPositioned against the real alternative100%
passSells against a pain the buyer feels100%
passDiagnoses message or channel first100%
passPlans beyond launch week100%

Run

Run: #1
API response time: 37 s
Submitted: 6 Oct 2026

GPT-6.1 SolAPI

Usable with a quick edit10 of 12 checks passed

The plan is detailed and actionable; the unsupported claims are minor cautions that a PM could remove or rephrase without redoing the reasoning or re-checking evidence.

Output

Ledgerly Payroll: existing-customer launch plan

3 November–2 February

1. Start with customers who fear mistakes—not everyone who dislikes admin

Prioritise the 2,170 existing customers with 5–25 employees whose payroll is run in spreadsheets or by an accountant. They represent 27% of our customer base:

690 spreadsheet users: 19% reported a payroll penalty notice last year. Start here: they control the buying decision and have no incumbent software to replace.
1,480 accountant-run customers: 23% reported a penalty notice. Approach alongside the first group, but qualify who controls payroll and whether the accountant should participate.

These penalty rates are much higher than the 4–7% reported by the other segments. Customer interviews reinforce the problem: owners want confidence that payroll is right, not merely faster.

Deprioritise the 4,160 customers with 1–4 employees: several see spreadsheets as sufficient, and willingness to pay looks weaker. Also defer broad outreach to the 1,670 customers using another payroll tool: their reported penalty rates are lower, and switching creates friction.

Within the priority segments, invite customers with known concerns or upcoming payroll runs first. Don’t assume a segment-level penalty rate means any particular customer received a notice.

2. Sell confidence and control, without promising immunity

Replace “the first Financial Operating System” and the generic speed headline with:

> Feel more confident about payroll, right alongside your books. > Ledgerly Payroll costs $6 per employee per month. See whether it fits your business and get help setting up your first payroll.

Use separate supporting messages:

Spreadsheet users: “Move beyond manual payroll spreadsheets. See how Ledgerly Payroll works before your next pay run.”
Accountant-run customers: “Get a clearer view of payroll—and decide how you and your accountant will manage it.”

The beta provides strong directional evidence for this positioning. The penalty headline generated first-payroll completions equal to 2.8% of page visits versus 0.9% for the speed headline—roughly three times higher. Setup starts also increased, while setup-to-first-payroll conversion remained similar, around 42–44%. The message attracts interest; onboarding still needs work.

However, do not ship “Never get a payroll penalty again.” It is an absolute guarantee the evidence cannot support. Test truthful reassurance against the speed message, with legal review.

Before launch, confirm exactly which calculations, checks, deadlines and filing responsibilities Ledgerly supports. Demonstrate only verified capabilities and clearly explain customer responsibilities. For price context, a 10-person business pays $60/month; don’t claim universal savings versus accountants.

3. Use owned channels and assisted onboarding before acquisition spend

Replace the $60,000 November prospecting campaign with a focused existing-customer rollout. Hold that spend until activation, retention and support economics justify expansion.

Before 3 November - Prepare segment-specific emails, in-app pages, eligibility guidance and a clear setup checklist. - Equip support with payroll-specific escalation procedures and capacity for first-run assistance. - Instrument the journey from invitation through paid, successful payroll. - Validate readiness. If a critical workflow is unsafe, narrow launch eligibility rather than market around it.

November: activate the first cohort - Send targeted email and in-app invitations to the priority customers, paced to support capacity—not one banner to all 8,000. - Offer a practical demonstration and bookable setup sessions. Schedule follow-ups around each customer’s next payroll date. - Use the launch-week webinar to show setup, responsibilities and a complete payroll workflow, not deliver a corporate announcement. - Where customers agree, involve their accountants. Position them as collaborators, not the cause of every payroll problem.

December: remove friction - Contact customers who started but stalled; diagnose migration, employee-data, confidence and accountant-approval barriers. - Turn recurring questions into onboarding improvements and help content. - Publish customer proof only with permission and substantiated outcomes.

January–2 February: expand selectively - Broaden outreach once repeat usage and operational quality are sound. - Test customers considering a switch from another tool, with explicit migration guidance. - Consider a small paid acquisition experiment only after establishing acceptable acquisition cost and payback.

4. Measure paid adoption, repeat use and trust—not launch noise

Set these as initial planning targets, not forecasts; recalibrate after November’s first cohort:

By 2 February: 217 priority customers—10% of the target segment—have completed a successful payroll and become paying add-on customers.
Repeat use: at least 80% of activated customers complete their next scheduled payroll; separately track third-run retention where enough time has elapsed.
Commercial performance: report actual payroll MRR, paid employees, cancellations and contribution after onboarding/support costs. Customer count alone does not establish revenue.

Review weekly by segment and channel: invitations delivered, qualified visits, setup starts, setup completion, successful first payroll, paid conversion, time to activation and support effort. Use unique-customer denominators, not just page visits.

Run a controlled headline test and, where feasible, stagger outreach with a holdout to measure incremental adoption.

Track failed runs, product-attributable calculation or filing incidents, and customer confidence after first and repeat runs. Investigate every serious incident; pause the affected rollout if reliability is compromised.

The launch succeeds when customers repeatedly pay staff correctly and keep paying for Ledgerly—not when they read a press release.

Claims the judge couldn’t find in the brief

The beta provides strong directional evidence for this positioning
It is an absolute guarantee the evidence cannot support

Grades and run details

Decision model 88 · LLM judge 85

Decision model checks

failUses the supplied evidence correctly55%
passAddresses the actual decision91%
passRespects explicit constraints46%
passIdentifies material uncertainty83%
partialAvoids unsupported claims33%
passProduces the required deliverable78%
passPicks the segment with the pain100%
passUses the headline test37%
passPositioned against the real alternative96%
passSells against a pain the buyer feels83%
passDiagnoses message or channel first100%
passPlans beyond launch week100%

Run

Run: #1
API response time: 40 s
Submitted: 5 Oct 2026

Check by check

Where the outputs differ comes first. Open a check to read the judge’s reasons. “Mixed” means the two graders disagreed.

Where they differ 2

Uses the supplied evidence correctlyMixedMixedWrong

Haiku 5.5 · API

The plan asserts two facts about the current situation not supported by the brief: that individual penalty recipients can be identified from survey data, and that accountants make the payroll purchasing decision.

Sonnet 5.5 · API

The claim 'About half the base is a poor fit' is not supported by the data; the target segment is 27%, not half.

GPT-6.1 Sol · API

The output claims the beta provides strong directional evidence and that the evidence cannot support an absolute guarantee; neither is a fact from the brief or arithmetic, so not every statement about the current situation is supported.

Avoids unsupported claimsMixedMixedWrong

Haiku 5.5 · API

The plan states that the accountant makes the payroll decision as a fact, and implies precise individual penalty data exists, without labelling these as assumptions or hypotheses.

Sonnet 5.5 · API

Presents 'About half the base is a poor fit' as fact when the data shows 73% are not in the target, not half.

GPT-6.1 Sol · API

Presents the claim that the evidence cannot support an absolute guarantee as a fact, and the beta as strong directional evidence, without labelling them as interpretations.

All got right 10

Addresses the actual decisionRightRightRight

Haiku 5.5 · API

The output commits to a clear wave plan early and states decision rules (conversion thresholds for budget release and messaging changes).

Sonnet 5.5 · API

Commits to Wave 1 segment, message, channels, and decision gates with clear conditions for changing course.

GPT-6.1 Sol · API

Commits early to a primary segment (5-25 employees, spreadsheet/accountant-run) and says what would change the call (reliability issues, activation economics).

Respects explicit constraintsRightRightRight

Haiku 5.5 · API

The output is a go-to-market plan under 900 words, addresses the requested elements (segments, messaging, channels, measures), and respects the launch date and audience.

Sonnet 5.5 · API

Delivers a go-to-market plan under 900 words, addressing the required elements, and respects the reader and form.

GPT-6.1 Sol · API

The output is a go-to-market plan within 900 words, respecting the requested form and length.

Identifies material uncertaintyRightRightRight

Haiku 5.5 · API

The plan names unknowns (conversion rates, messaging effectiveness) and sets explicit thresholds and dates to resolve them.

Sonnet 5.5 · API

Names self-reported penalty data, anecdotal calls, beta enthusiasm bias, missing headcount data, and legal approval for guarantee, with resolution steps.

GPT-6.1 Sol · API

Names unknowns (exact payroll capabilities, legal review of guarantee, activation economics) and says how they would be resolved or would change the plan.

Produces the required deliverableRightRightRight

Haiku 5.5 · API

The output is a complete, actionable GTM plan in the required format, usable by a PM to execute the launch with light edits.

Sonnet 5.5 · API

Complete plan with changes, segment, message, channels, measures, budget, and caveats; usable as-is with light edits.

GPT-6.1 Sol · API

Delivers a complete, actionable go-to-market plan for the PM, within the word limit.

Picks the segment with the painRightRightRight

Haiku 5.5 · API

The plan correctly prioritizes 5-25 employee spreadsheet and accountant-run segments (high penalty rates), explains why 1-4 and other-tool users come later.

Sonnet 5.5 · API

Targets 5-25 employee customers with accountant or spreadsheet payroll, cites penalty rates, and explains why 1-4 and other-tool users are not first.

GPT-6.1 Sol · API

Targets 5-25 employees with spreadsheet or accountant-run payroll, uses the numbers and penalty rates, and explains why smaller businesses and tool users are deprioritised.

Uses the headline testRightRightRight

Haiku 5.5 · API

The plan uses the beta headline test results to base the core message on penalty protection for the high-pain segments and time-saving only for the low-pain segment.

Sonnet 5.5 · API

Uses the beta headline test results (3x setup lift) to build the message on penalties and accuracy.

GPT-6.1 Sol · API

Cites the beta test's 2.8% vs 0.9% first-payroll rates and builds the message on penalties and accuracy.

Positioned against the real alternativeRightRightRight

Haiku 5.5 · API

The plan names current alternatives (accountant, spreadsheet, another tool) and differentiates against the value they lack (accuracy, penalty protection) for each segment.

Sonnet 5.5 · API

Names the real alternatives (spreadsheet, accountant, other tool) and ties message to penalty avoidance and accuracy, targeting the segment that cares most.

GPT-6.1 Sol · API

Names the real alternatives (spreadsheet, accountant) and ties the message to confidence and accuracy they lack.

Sells against a pain the buyer feelsRightRightRight

Haiku 5.5 · API

The lead message for the primary segment is built on penalty fear and accuracy, directly supported by the customer calls and headline test.

Sonnet 5.5 · API

Leads with fear of penalties and tax office letters, supported by call quotes and headline test, not generic time-saving.

GPT-6.1 Sol · API

Leads with fear of mistakes and penalties, using the customer calls and penalty data, not generic time-saving.

Diagnoses message or channel firstRightRightRight

Haiku 5.5 · API

The plan diagnoses message as the bottleneck (uses in-app test data and existing reach) and defers paid spend until conversion is proven, focusing effort on fixing the message first.

Sonnet 5.5 · API

Diagnoses message as the problem (beta conversion data) and focuses on in-app/email to existing base, holding paid ads.

GPT-6.1 Sol · API

Uses the beta conversion data to diagnose a message problem and shifts spend to owned channels and message improvement before paid acquisition.

Plans beyond launch weekRightRightRight

Haiku 5.5 · API

The plan includes a multi-wave rollout, ongoing channels (email, accountant pilot, retargeting), and 90-day metrics with decision rules beyond launch week.

Sonnet 5.5 · API

Plans ongoing motion with second webinar, monthly measures, retention, guardrails, holdout group, and Wave 2 decision gate.

GPT-6.1 Sol · API

Plans December friction removal, January expansion, and measures repeat use, retention, and MRR over three months.

Results

Every setup we’ve tested on this task type, across all its tasks and repeats, graded on the current checklist. Calibrated.

#	Model · Harness	Decision model	LLM judge	Runs	Critical failures
1	GPT-6 AstrawithChatGPT	100.0	100.0	2	None
2	Opus 5.5withClaude	96.0	85.2	2	None
3	GPT-6 LunawithAPI	94.2	89.3	2	None
4	GPT-6.1 SolwithAPI	93.8	78.0	2	None
5	Sonnet 5.5withAPI	98.1	77.7	2	None
6	Haiku 5.5withAPI	94.2	77.7	2	None
7	Gemini 3.8 FlashwithAPI	73.9	81.9	2	None
8	Gemini 3.5 Flash-LitewithGemini	74.2	63.5	2	None

What you get for the cost

No setup scores within 5 points of the leader at half its cost or less here, so the leader is also the best value.

Nobody beats these on both score and cost

Task score and relative cost by setup
Setup	Task score	Relative cost (cheapest = 1)
GPT-6 Luna · API	90	1.0×
Haiku 5.5 · API	82	3.3×
Gemini 3.5 Flash-Lite · Gemini	68	3.5×
Gemini 3.8 Flash · API	77	11×
GPT-6.1 Sol · API	84	16×
Sonnet 5.5 · API	84	46×
Opus 5.5 · Claude	91	75×
GPT-6 Astra · ChatGPT	100	81×

Cost is each output’s tokens at the model’s list price, shown relative to the cheapest setup. Runs done in an app have no usage meter, so theirs are estimated from the same setup’s measured runs. How we compare cost

The PM job

Taking a product or add-on to market with marketing and sales.

Why it matters

Launch plans fail by trying to reach everyone, by selling upside nobody feels, and by treating launch day as the whole plan.

What good looks like

One primary segment: those who value the difference most
Positions against what customers do today
Sells against a pain the buyer feels, with evidence
Diagnoses whether the problem is the message or the channel
Plans beyond launch week

Deliberately not measured

Creative copywriting quality

Capability tested

Go-to-market judgement

The failure we’re looking for

Every channel, every segment, and nothing after launch day

Grading

Decision model and LLM judge, calibrated against a blind PM review

What AI gets right here, and what you’ll still have to catch

Reliably right

Where it slips

How it’s graded

Positioned against the real alternative

Sells against a pain the buyer feels

Diagnoses message or channel first

Plans beyond launch week

Plus our standard checks

The tasks

The outputs

Output

Ledgerly Payroll: Go-to-Market Plan

What I'd change in the draft

Who we go after, in order

What we say

Channels

Measuring the next three months

Claims the judge couldn’t find in the brief

Grades and run details

Decision model checks

Run

Output

Ledgerly Payroll: go-to-market plan

What I'd change in Marketing's draft

Who we go after first

What we say

Channels

Budget (the $60k)

How we'll know it's working (3 months)

Caveats

Claims the judge couldn’t find in the brief

Grades and run details

Decision model checks

Run

Output

Ledgerly Payroll: existing-customer launch plan

1. Start with customers who fear mistakes—not everyone who dislikes admin

2. Sell confidence and control, without promising immunity

3. Use owned channels and assisted onboarding before acquisition spend

4. Measure paid adoption, repeat use and trust—not launch noise

Claims the judge couldn’t find in the brief

Grades and run details

Decision model checks

Run

Check by check

Where they differ 2

All got right 10

Results

What you get for the cost

About the task

The PM job

Why it matters

What good looks like

Deliberately not measured