Tasks / Growth

Write a GTM plan

Can the model pick the customers who value the difference most, position against what they do today, and build a go-to-market that lasts beyond launch day?

Task type v1.1 · 2 tasksLast changed 6 Oct 2026 · ChangelogDifficulty

What AI gets right here, and what you’ll still have to catch

From 16 graded outputs by 8 models. 50% were usable with at most a quick edit.

Reliably right

  1. Positioned against the real alternative100% pass
    It names the real alternatives (accountant-run payroll with mistakes, owner spreadsheet with risk) and ties the message to confidence and accuracy that those alternatives lack.
    GPT-6 Astra · ChatGPT · Payroll, for the people who fear it
  2. Protects the self-serve engine100% pass
    Uses the 70% free-to-paid stat and product-qualified deal data to argue against capping the free plan, and keeps self-serve open.
    GPT-6 Astra · ChatGPT · Enterprise, without killing the engine
  3. Does the capacity sums100% pass
    Works out deals needed (69-80), qualified opportunities (134), and AE-years (8) with checkable arithmetic.
    GPT-6 Astra · ChatGPT · Enterprise, without killing the engine

Where it slips

  1. Avoids unsupported claims47% pass
    Presents the incorrect 'five of six no decision' figure as established fact, which is an unsupported claim about the current situation.
    Opus 5.5 · Claude · Enterprise, without killing the engine
  2. Uses the supplied evidence correctly52% pass
    The claim 'Five of our six losses on existing-user deals ended in no decision' is not supported by the data (4 no-decision, 2 Miro), so a statement about the current situation is invented.
    Opus 5.5 · Claude · Enterprise, without killing the engine
  3. Identifies material uncertainty73% pass
    The plan does not name specific unknowns that could change the decision, nor does it say how they would be resolved or what would trigger a pivot.
    Gemini 3.8 Flash · API · Payroll, for the people who fear it

How it’s graded

Each check is built on how world-class product and growth leaders do this work, as they’ve described it on Lenny’s Podcast, and names the leader it comes from.

  1. Positioned against the real alternative

    Does the plan name what the target customer does today without this product (including the status quo or doing nothing), and tie the message to the value that alternative lacks?

    Passes when Names the real alternative and the differentiated value against it, and picks the segment that cares most about that value.

  2. Sells against a pain the buyer feels

    Is the core message built on a pain or risk the target buyer already feels, with evidence from the material, rather than on generic upside like saving time?

    Passes when Leads with the pain the evidence shows buyers care about, and uses that evidence to choose the message.

  3. Diagnoses message or channel first

    Does the plan work out whether the problem is the message (people reach it but don't convert) or the channels (the right people never see it), and spend accordingly?

    Passes when Uses conversion and reach data to say which it is, and puts the effort there.

  4. Plans beyond launch week

    Does the plan say how demand keeps coming after launch, with the ongoing engine and what's measured over months, rather than treating launch day as the whole plan?

    Passes when Gives the ongoing motion after launch and the measures that show whether it's working over time.

Plus our standard checks

Uses the supplied evidence correctly · Addresses the actual decision · Respects explicit constraints · Identifies material uncertainty · Avoids unsupported claims · Produces the required deliverable · and 2 written for each task, which you’ll see in the tasks below.

The tasks

Read the brief, then compare up to three outputs, each with the LLM judge’s verdict on every check. Highlights mark what a PM had to fix.

The brief

You're the PM for Ledgerly Payroll, a new add-on for our accounting product, launching to existing customers on 3 November. Marketing has drafted the launch plan below. Write the go-to-market plan you'd run instead (or the changes you'd make to theirs), in no more than 900 words: who we go after first, what we say to them, through which channels, and how we'll know it's working over the following three months. The files are attached.

What the model was given5 items: About Ledgerly, marketing_launch_draft.md, customers_by_size_and_payroll_method.csv (penalty notices: share who told us they got a payroll penalty notice last year), beta_headline_test.csv (two versions of the in-app payroll page, shown at random to beta customers), customer_calls.md
About LedgerlyAccounting software for small businesses: invoicing, bank feeds, bookkeeping. 8,000 paying customers. Payroll is a new add-on at $6 per employee per month.
marketing_launch_draft.md7 lines · Download
# Draft launch plan (Head of Marketing)

- Launch day 3 November: press release announcing Ledgerly as "the first Financial Operating System for small business".
- Paid social and search ads to new prospects: $60,000 in November.
- In-app banner to all 8,000 customers: "Run payroll in minutes".
- Webinar in launch week.
- Success measure: press mentions and sign-ups in launch week.
customers_by_size_and_payroll_method.csv (penalty notices: share who told us they got a payroll penalty notice last year)employees,current_payroll,customers,penalty_notices_last_year_pct 1-4,Owner does it in a spreadsheet,3140,4 1-4,Accountant runs it,1020,6 5-25,Owner does it in a spreadsheet,690,19 5-25,Accountant runs it,1480,23 5-25,Another payroll tool,1210,7 26-50,Another payroll tool,460,5
beta_headline_test.csv (two versions of the in-app payroll page, shown at random to beta customers)headline,landing_page_visits,started_setup,ran_first_payroll "Run payroll in minutes, right where you do your books",4210,92,38 Never get a payroll penalty again,4160,268,117
customer_calls.md10 lines · Download
# Customer calls about payroll (excerpts, September)

1. Bakery, 12 staff: "My accountant does payroll. Last spring they filed late and I got a $1,400 penalty. I don't know what I'm paying them for."
2. Plumbing firm, 18 staff: "Time isn't the issue. I'm scared of getting it wrong. The tax office letter was the worst week of my year."
3. Design studio, 3 staff: "It's me and two freelancers. A spreadsheet's fine."
4. Café group, 22 staff: "We use another tool, it's OK. Switching sounds like a nightmare mid-year."
5. Physio clinic, 9 staff: "I'd move tomorrow if it meant one less thing that can go wrong with the tax office."
…
What a strong answer doesThe answer key the graders mark against

Picks customers with 5 to 25 employees whose accountant runs payroll (1,480 customers, 23% had a penalty notice) and those doing it in a spreadsheet (690, 19%), not the smallest businesses (happy with a spreadsheet) or those on another payroll tool (switching mid-year is the barrier). Builds the message on fear of penalties and errors, not time: in the beta, 'Never get a payroll penalty again' got about three times as many setups as 'Run payroll in minutes' (268 against 92 from similar traffic, 6.4% against 2.2%) and more first payrolls (117 against 38), and the calls say the same. Reads this as a message problem, not a channel problem: the existing customer base already sees the page, so fix the message and reach existing customers in-app and through their accountants before spending $60,000 on ads to strangers. Drops the 'Financial Operating System' category claim. Plans beyond launch week: prompts around payroll and tax deadlines, accountant referrals, and monthly measures such as payroll attach in the target segment, first payrolls run, and penalties avoided.

Critical failures (cap the score at 40)
  • Keeps 'Run payroll in minutes' as the lead message despite the headline test
Task

Mid-level · v1.0

The outputs

8 setups answered this brief.

Usable with a quick edit12 of 12 checks passed

The output is evidence-based, actionable, and covers all required elements; a PM could use it with minor wording tweaks without re-checking data or redoing reasoning.

Output

Ledgerly Payroll: a focused, trust-led launch

Launch on 3 November to existing customers with a pressing payroll problem—not to the whole market with a new category claim. Our goal is sustained, paid payroll usage over the next three months, not launch-week attention.

1. Who we pursue first

Prioritise businesses with 5–25 employees using spreadsheets or an accountant: 2,170 customers.

SegmentCustomersReported penalty notices last yearApproach
------:---:---
Accountant runs payroll1,48023%Largest opportunity; offer an owner-and-accountant walkthrough and a supported transition
Owner uses spreadsheets69019%Direct owner outreach focused on confidence, accuracy and setup help

These customers combine meaningful payroll complexity with substantially higher reported penalty incidence than other segments. The calls reinforce the pattern: fear of mistakes matters more than saving minutes.

The survey does not establish that accountants cause penalties. We should help owners gain confidence and involve their accountants where appropriate, not run an anti-accountant campaign.

Deprioritise: - 1–4 employees: low reported penalty incidence and evidence that free spreadsheets feel adequate. - Customers using another payroll tool: lower reported penalty incidence and explicit switching resistance. - 26–50 employees: revisit after proving onboarding and reliability with smaller teams.

Within the priority segments, recruit customers approaching a suitable payroll-cycle boundary who fit the product’s supported requirements. Don’t push a risky mid-cycle migration.

2. What we say—and must prove

Positioning: “More confidence in payroll, alongside your Ledgerly books.”

Suggested launch message:

> Worried about getting payroll wrong? > Meet Ledgerly Payroll, the new payroll add-on for your Ledgerly account. Check whether it fits your business and get help planning your first run. > $6 per employee per month, additional to your accounting subscription. > CTA: Check your fit and book setup help.

A 12-employee business pays $72 per month. Make pricing concrete without claiming savings unless we know what its current service includes.

The beta strongly favours the risk-reduction motivation: the penalty headline produced 117 first payrolls from 4,160 visits (2.8%), versus 38 from 4,210 (0.9%) for the speed headline—roughly three times the conversion.

But “Never get a payroll penalty again” is an unacceptable guarantee, and the test does not prove a safer alternative will perform equally well. Test qualified confidence-led copy against the speed message, using first payroll—not clicks—as the outcome.

Before publication, verify every supporting claim against the shipped product: supported payroll requirements, calculations, checks, filing responsibilities and support coverage. Demonstrate verified capabilities; don’t substitute reassuring language for evidence.

3. Channels and launch sequence

Before 3 November: establish readiness

  • PM, engineering and payroll specialists sign off supported customer types, critical workflows, known limitations and escalation procedures.
  • Prepare a migration checklist, pricing page, product demonstration and clear explanation of customer versus Ledgerly responsibilities.
  • Recruit a small assisted cohort across both priority segments; expand only when onboarding works reliably.

3 November–2 December: targeted launch

Replace the universal banner with segmented in-app messages and targeted email to the 2,170 eligible accounts. Use different examples for spreadsheet owners and accountant-managed businesses.

Run the webinar as a practical clinic: “Preparing your first Ledgerly payroll.” Show the workflow, explain limitations and answer switching questions. Invite accountants alongside owners.

Offer scheduled onboarding sessions and follow up with customers who start setup but stall. Sequence invitations to match support capacity.

Drop the “first Financial Operating System” press release. It is unsupported and disconnected from the buying need. Publish a straightforward customer announcement instead.

3 December–2 January: remove friction

Review setup abandonment, migration objections and support tickets weekly. Improve the biggest bottleneck before increasing invitation volume.

Build a customer proof point only with consent and verified results. Adapt clinic and support coverage around customers’ payroll schedules and holiday availability.

3 January–2 February: scale selectively

Expand outreach within the priority cohort if reliability and repeat usage hold. Test a small adjacent segment only after reviewing its requirements and switching barriers.

Pause the proposed $60,000 new-prospect advertising spend. Fund onboarding, measurement and targeted customer communications first; release broader acquisition budget only after proving retained adoption and service economics.

4. How we know it works

North star: paying businesses running repeat payroll successfully.

Instrument the account-level funnel by segment and channel:

Eligible → reached → setup started → setup completed → first successful payroll → next scheduled payroll → retained paid account.

Keep a random 10% holdout from proactive promotion, without blocking access, to estimate incremental adoption. Track assisted and self-serve onboarding separately.

Provisional three-month goals—not forecasts: - 200 accounts complete a first successful paid payroll, about 9% of the priority cohort. - At least 80% run their next scheduled payroll when due. - At least 90% paid retention at 60 days among accounts old enough to measure.

Track actual employee counts and payroll MRR, onboarding cost, support load, critical errors and filing exceptions where applicable. Pause expansion for unresolved critical failures.

Review weekly; make monthly scale-or-fix decisions. Monitor penalty complaints, but don’t claim penalty reduction from a three-month window. Press mentions and sign-ups are diagnostics, not success.

Grades and run details

Decision model 100 · LLM judge 100
Decision model checks
  • passUses the supplied evidence correctly9%
  • passAddresses the actual decision96%
  • passRespects explicit constraints46%
  • passIdentifies material uncertainty97%
  • passAvoids unsupported claims58%
  • passProduces the required deliverable82%
  • passPicks the segment with the pain100%
  • passUses the headline test57%
  • passPositioned against the real alternative85%
  • passSells against a pain the buyer feels100%
  • passDiagnoses message or channel first99%
  • passPlans beyond launch week100%
Run
Run
#1
Time to output
37 s
Submitted
5 Oct 2026

Needs more than a quick edit10 of 12 checks passed

The output incorrectly states that about half the base is a poor fit, which would require re-checking the data to correct; otherwise the plan is solid but that error undermines trust.

Output

Ledgerly Payroll: go-to-market plan

What I'd change in Marketing's draft

DraftProblemChange
$60k paid ads to new prospectsPayroll is an add-on, so a prospect would first have to adopt our accounting product. We have 8,000 customers and haven't yet proven the message with them.Hold most of the budget until the message converts in-base (see Budget).
Banner to all 8,000About half the base is a poor fit (details below).Target about 2,170 customers first.
"Financial Operating System" press releaseCustomers never use this language. They talk about fear of penalties and the tax office.Keep the press small and plain.
Success = press mentions and launch-week sign-upsNeither measures payroll actually being run.Measure first payroll run and repeat runs.

Who we go after first

Wave 1 (3 Nov): customers with 5–25 employees who run payroll via spreadsheet (690) or an accountant (1,480). That is 2,170 customers, 27% of the base.

  • They have the pain. 19% (spreadsheet) and 23% (accountant) got a payroll penalty notice last year, versus 4–6% for businesses with 1–4 staff and 5–7% for those already on a payroll tool.
  • They have the money. At $6 per employee, a 12-person business pays about $72 a month, against the $180 a month one landscaper pays their accountant.
  • The calls match the data. The bakery, plumber, physio, landscaper and dental practice all describe fear of mistakes, not lack of time.

Not now: - 1–4 employees (4,160 customers): The spreadsheet is "fine" and "free," and only 4–6% were penalized. Leave payroll available in the product but don't push it. - Other-tool users (1,670 customers): Penalty rates are low (5–7%), and switching mid-year "sounds like a nightmare." Revisit at the start of the next tax year with a migration offer.

What we say

The beta test was randomized. "Never get a payroll penalty again" beat "Run payroll in minutes" on setup starts (6.4% vs 2.2% of visits) and on first payroll runs (2.8% vs 0.9%), about 3x on both. Once people started, completion was similar (44% vs 41%), so the headline drives the decision to begin and the product handles the rest.

Core message: "Payroll filed right and on time, so the tax office letter never arrives."

  • Spreadsheet owners: "A spreadsheet won't tell you when you've got it wrong."
  • Accountant-run: "Stop paying $180 a month to worry about it." Use this to speak to cost and control without attacking accountants. Offer a way to invite their accountant in as a reviewer.

Accuracy promise: The dental practice said, "If you can promise it's right, I'm in." I'd like to offer a penalty-cover guarantee for errors caused by our software. Legal and Finance must approve it by 24 Oct. If they won't, the copy becomes "filed right and on time, with support to fix problems." We should not promise "never" without backing it. The beta headline did well, but a claim we can't stand behind creates risk.

Channels

  1. Targeted in-app card and email to the Wave 1 list only, using the winning headline. This is the main channel, and it costs almost nothing.
  2. Personal outreach from Customer Success to Wave 1 customers who told us they got a penalty notice (about 470 people by my estimate from the survey rates). Offer a 20-minute guided setup.
  3. Webinar, reframed: "Penalty-proof your payroll" with a live setup walkthrough. Run it in launch week and again in week 4, since the second session catches people who weren't ready the first time.
  4. Press: one plain release ("Ledgerly adds payroll, $6 per employee"), with no category claim. Follow with customer stories once we have them.
  5. Paid: none to new prospects in November (see Budget).

Budget (the $60k)

  • ~$15k: guided setup and migration help (contractor or CS time).
  • ~$10k: webinars, email production, and case-study content.
  • ~$35k: held until the January gate. If Wave 1 works, spend it on search ads for "payroll penalty" intent, aimed at businesses that would adopt accounting and payroll together.

How we'll know it's working (3 months)

Primary metric: paying accounts that have run at least two payrolls. This replaces sign-ups, press mentions and launch-week anything.

Targets (to be recalibrated after week 2, since the beta is my only baseline): - Month 1: at least 5% of Wave 1 has started setup. At least 45% of those run a first payroll, matching beta's 44%. - End of January: about 250 Wave 1 accounts running payroll (~11% of the segment). I'm assuming a 10-person average, which implies roughly $15k MRR. I'll replace that assumption with real headcount data. - Retention: at least 85% of those accounts run payroll in three consecutive cycles.

Guardrails (any breach pauses expansion): - Payroll errors or penalties caused by us: zero tolerance, reviewed weekly. - Support tickets per payroll run. - Churn of the core accounting subscription, which must not rise among adopters.

Measuring real impact: Hold out a random 10% of Wave 1 from all promotion. If adopters in the promoted group don't clearly exceed the holdout, we're only capturing people who would have found payroll anyway.

Decision gates: - Mid-December: If setup-to-first-payroll is below 35%, fix onboarding before pushing further. - Mid-January: If Wave 1 is at or above target and no guardrail is breached, release the held budget and start Wave 2 (other-tool users, ahead of the tax-year switch window). If it's below target, interview non-adopters before spending anything.

Caveats

  • Penalty rates are self-reported, and the eight calls are anecdotes.
  • Beta customers may be more enthusiastic than the base.
  • I haven't seen average headcount per segment, so the revenue estimates are rough.

Claims the judge couldn’t find in the brief

  • About half the base is a poor fit

Grades and run details

Decision model 100 · LLM judge 77
Decision model checks
  • passUses the supplied evidence correctly13%
  • passAddresses the actual decision99%
  • passRespects explicit constraints32%
  • passIdentifies material uncertainty99%
  • passAvoids unsupported claims38%
  • passProduces the required deliverable75%
  • passPicks the segment with the pain100%
  • passUses the headline test100%
  • passPositioned against the real alternative100%
  • passSells against a pain the buyer feels100%
  • passDiagnoses message or channel first100%
  • passPlans beyond launch week100%
Run
Run
#1
API response time
37 s
Submitted
6 Oct 2026

Usable with a quick edit12 of 12 checks passed

The plan is evidence-based, actionable, and covers all required elements; a PM could execute it with only minor wording adjustments.

Output

Ledgerly Payroll go-to-market plan | 3 November–31 January

1. Start with customers who have the clearest problem

First priority: existing customers with 5–25 employees who currently run payroll themselves or rely on an accountant. That’s 2,170 customers: 690 spreadsheet users and 1,480 whose accountant runs payroll. In these groups, 19% and 23%, respectively, said they received a payroll penalty notice last year. Treat that as a signal of pain—not proof that Ledgerly would prevent penalties.

Lead with accountant-run customers who want more visibility and control, while making clear Ledgerly isn’t accusing their accountant of poor work. Follow with spreadsheet users in the same size band, for whom a simpler alternative to manual payroll may appeal.

Defer broader prospecting. Among customers with 1–4 employees, reported penalty notices are lower (4–6%); several calls also suggest some are content with free spreadsheets. Customers already using another payroll tool reported fewer notices (7% for 5–25 employees), and one call highlighted switching friction. Test those groups later, after validating onboarding and migration. Don’t prioritize 26–50-employee customers until we know the product and support experience fit their needs.

2. Say what we can support—not what we can’t promise

The beta headline test is a useful directional signal:

  • “Never get a payroll penalty again”: 268/4,160 visitors started setup (6.4%); 117 ran first payroll (2.8%).
  • “Run payroll in minutes…”: 92/4,210 started (2.2%); 38 ran first payroll (0.9%).

The penalty-focused version performed better, but don’t use “Never get a payroll penalty again” unless Ledgerly can substantiate that guarantee. Instead test responsible variants such as:

  • “More confidence in payroll, alongside your books.”
  • “Take control of payroll and reduce the risk of costly mistakes.”

Be explicit that payroll is $6 per employee per month. Emphasize control, confidence and working in the same accounting product—not guaranteed outcomes or unverified features. Tailor the message: “Keep a closer eye on payroll” for accountant-run customers; “Move beyond the spreadsheet” for owner-run customers.

3. Use customer channels before buying reach

  • Before launch: Segment the 2,170 priority customers and invite a small cohort to try setup. Interview users who start but don’t finish; check readiness, support capacity and any limitations before broad promotion.
  • Launch, 3 November: Send segmented email to the priority cohort and show a targeted in-app message when those customers use Ledgerly. Link directly to a clear product and pricing page, not a generic announcement. Offer a live, practical webinar on setup and common payroll concerns, with time for questions.
  • During November–January: Follow up based on behavior: a setup reminder for visitors who don’t start, and personal onboarding or support for customers who start but don’t run payroll. Invite accountant-run customers to involve their accountant where appropriate. Expand in-app promotion to other segments only after reviewing early results.
  • Earned media: A factual product announcement is fine, but don’t make press mentions the launch objective. Skip the $60,000 broad paid campaign to new prospects until existing-customer activation and retention are proven.

4. Measure adoption and outcomes for three months

Track weekly by customer segment and by funnel step: eligible → reached → product-page visit → setup started → setup completed → first payroll run → repeat payroll run. Report both counts and conversion rates; compare messages with a randomized test where possible.

Use the beta as a benchmark, not a forecast. A reasonable initial goal is to beat its stronger variant’s 6.4% visit-to-setup-start and 2.8% visit-to-first-payroll rates in the priority cohort. Also set a three-month target for repeat payroll runs among first-time users, and review setup abandonment to find where customers get stuck.

Pair adoption metrics with guardrails: payroll-related support contacts, failed or corrected runs, cancellations/refunds, and customer feedback about confidence and accuracy. Review weekly with Product, Support and Marketing. If activation is weak, fix onboarding or clarify the offer before adding spend. If customers run payroll but don’t repeat, investigate reliability, fit and switching friction before expanding the campaign.

Grades and run details

Decision model 100 · LLM judge 100
Decision model checks
  • passUses the supplied evidence correctly77%
  • passAddresses the actual decision93%
  • passRespects explicit constraints66%
  • passIdentifies material uncertainty83%
  • passAvoids unsupported claims63%
  • passProduces the required deliverable82%
  • passPicks the segment with the pain100%
  • passUses the headline test77%
  • passPositioned against the real alternative98%
  • passSells against a pain the buyer feels96%
  • passDiagnoses message or channel first99%
  • passPlans beyond launch week100%
Run
Run
#1
API response time
17 s
Submitted
5 Oct 2026

Check by check

Where the outputs differ comes first. Open a check to read the judge’s reasons. “Mixed” means the two graders disagreed.

Where they differ 2

Uses the supplied evidence correctlyRightMixedRight
GPT-6 Astra · ChatGPT

All facts about the current situation (segment numbers, penalty rates, beta test results, call themes) are taken directly from the supplied context or derived by arithmetic, with no invented data.

Sonnet 5.5 · API

The claim 'About half the base is a poor fit' is not supported by the data; the target segment is 27%, not half.

GPT-6 Luna · API

All factual claims about the current situation are directly supported by the supplied data or arithmetic.

Avoids unsupported claimsRightMixedRight
GPT-6 Astra · ChatGPT

Interpretations like 'fear of mistakes matters more' are clearly tied to the call evidence, and no causal claims are presented as established fact without qualification.

Sonnet 5.5 · API

Presents 'About half the base is a poor fit' as fact when the data shows 73% are not in the target, not half.

GPT-6 Luna · API

Interpretations like 'want more visibility' are tied to call evidence and not presented as established fact; hypotheses are hedged.

All got right 10

Addresses the actual decisionRightRightRight
GPT-6 Astra · ChatGPT

The output commits early to targeting 5–25 employee businesses using spreadsheets or an accountant, with a trust-led message, and states that expansion pauses for unresolved critical failures.

Sonnet 5.5 · API

Commits to Wave 1 segment, message, channels, and decision gates with clear conditions for changing course.

GPT-6 Luna · API

The output commits to a clear primary segment, message, channels, and measures, and states conditions for expanding later.

Respects explicit constraintsRightRightRight
GPT-6 Astra · ChatGPT

The output is a go-to-market plan under 900 words, addresses the PM as reader, and respects the launch date and existing customer base.

Sonnet 5.5 · API

Delivers a go-to-market plan under 900 words, addressing the required elements, and respects the reader and form.

GPT-6 Luna · API

The plan stays under 900 words, does not lead with 'save time', and avoids paid ads to new prospects as the main channel.

Identifies material uncertaintyRightRightRight
GPT-6 Astra · ChatGPT

It names unknowns (product readiness, message performance, segment expansion) and says how to resolve them through testing, assisted cohorts, and holdout measurement.

Sonnet 5.5 · API

Names self-reported penalty data, anecdotal calls, beta enthusiasm bias, missing headcount data, and legal approval for guarantee, with resolution steps.

GPT-6 Luna · API

It identifies that the penalty message needs substantiation, treats the beta as directional, and specifies how weak activation or repeat usage would change the approach.

Produces the required deliverableRightRightRight
GPT-6 Astra · ChatGPT

The plan is complete, in the requested form, within the word limit, and a PM could act on it with light edits.

Sonnet 5.5 · API

Complete plan with changes, segment, message, channels, measures, budget, and caveats; usable as-is with light edits.

GPT-6 Luna · API

The output is a complete go-to-market plan with segment, message, channels, and three-month measures, within the word limit.

Picks the segment with the painRightRightRight
GPT-6 Astra · ChatGPT

It targets 5–25 employee businesses using spreadsheets or an accountant with the exact numbers and penalty rates, and explains why 1–4, other-tool, and 26–50 segments are deprioritised.

Sonnet 5.5 · API

Targets 5-25 employee customers with accountant or spreadsheet payroll, cites penalty rates, and explains why 1-4 and other-tool users are not first.

GPT-6 Luna · API

It targets 5–25 employee customers with accountant-run or spreadsheet payroll, cites penalty rates, and explains why smaller and tool-using segments are deferred.

Uses the headline testRightRightRight
GPT-6 Astra · ChatGPT

It cites the beta test results (penalty headline ~3x setups) and builds the message on penalties and accuracy, while noting the exact guarantee is unacceptable and should be tested further.

Sonnet 5.5 · API

Uses the beta headline test results (3x setup lift) to build the message on penalties and accuracy.

GPT-6 Luna · API

It cites the beta test results and builds the message on penalty and accuracy concerns, even while avoiding the unsubstantiated guarantee.

Positioned against the real alternativeRightRightRight
GPT-6 Astra · ChatGPT

It names the real alternatives (accountant-run payroll with mistakes, owner spreadsheet with risk) and ties the message to confidence and accuracy that those alternatives lack.

Sonnet 5.5 · API

Names the real alternatives (spreadsheet, accountant, other tool) and ties message to penalty avoidance and accuracy, targeting the segment that cares most.

GPT-6 Luna · API

It names the real alternatives (accountant, spreadsheet, another tool) and ties the message to confidence and reduced risk that those alternatives lack.

Sells against a pain the buyer feelsRightRightRight
GPT-6 Astra · ChatGPT

The core message leads with 'Worried about getting payroll wrong?' based on penalty fear from the calls and beta test, not on saving time.

Sonnet 5.5 · API

Leads with fear of penalties and tax office letters, supported by call quotes and headline test, not generic time-saving.

GPT-6 Luna · API

The core message leads with penalty fear and accuracy, supported by call quotes and the beta test, not generic time-saving.

Diagnoses message or channel firstRightRightRight
GPT-6 Astra · ChatGPT

It uses the beta conversion data to diagnose a message problem and shifts spend from new-prospect ads to in-app and email for existing customers, fixing the message first.

Sonnet 5.5 · API

Diagnoses message as the problem (beta conversion data) and focuses on in-app/email to existing base, holding paid ads.

GPT-6 Luna · API

It uses conversion data to diagnose that message matters more than reach, and focuses on in-app and email to existing customers before buying ads.

Plans beyond launch weekRightRightRight
GPT-6 Astra · ChatGPT

The plan spans three months with phased actions (readiness, targeted launch, friction removal, selective scaling) and ongoing measures like repeat payroll and retention.

Sonnet 5.5 · API

Plans ongoing motion with second webinar, monthly measures, retention, guardrails, holdout group, and Wave 2 decision gate.

GPT-6 Luna · API

The plan includes ongoing follow-up, behavior-based nudges, and three-month funnel and guardrail metrics beyond launch week.

Results

Every setup we’ve tested on this task type, across all its tasks and repeats, graded on the current checklist. Calibrated.

#Model · HarnessTask scoreDecision modelLLM judgeRunsCritical failures
1GPT-6 AstrawithChatGPT100.0100.02None
2Opus 5.5withClaude96.085.22None
3GPT-6 LunawithAPI94.289.32None
4GPT-6.1 SolwithAPI93.878.02None
5Sonnet 5.5withAPI98.177.72None
6Haiku 5.5withAPI94.277.72None
7Gemini 3.8 FlashwithAPI73.981.92None
8Gemini 3.5 Flash-LitewithGemini74.263.52None

What you get for the cost

No setup scores within 5 points of the leader at half its cost or less here, so the leader is also the best value.

Task score against relative cost for each setup. The table below has the same figures.607080901001×3×10×30×100×Cost per output, relative to the cheapest setup (log scale) →↑ Task scoreGPT-6 Astra · ChatGPT: task score 100, 81× the cheapestOpus 5.5 · Claude: task score 91, 75× the cheapestGPT-6 Luna · API: task score 90, the cheapestGPT-6.1 Sol · API: task score 84, 16× the cheapestSonnet 5.5 · API: task score 84, 46× the cheapestHaiku 5.5 · API: task score 82, 3.3× the cheapestGemini 3.8 Flash · API: task score 77, 11× the cheapestGemini 3.5 Flash-Lite · Gemini: task score 68, 3.5× the cheapestGPT-6 Astra · ChatGPTOpus 5.5 · ClaudeGPT-6 Luna · APIGPT-6.1 Sol · APISonnet 5.5 · APIHaiku 5.5 · APIGemini 3.8 Flash · APIGemini 3.5 Flash-Lite · Gemini
Nobody beats these on both score and cost
Task score and relative cost by setup
SetupTask scoreRelative cost (cheapest = 1)
GPT-6 Luna · API901.0×
Haiku 5.5 · API823.3×
Gemini 3.5 Flash-Lite · Gemini683.5×
Gemini 3.8 Flash · API7711×
GPT-6.1 Sol · API8416×
Sonnet 5.5 · API8446×
Opus 5.5 · Claude9175×
GPT-6 Astra · ChatGPT10081×

Cost is each output’s tokens at the model’s list price, shown relative to the cheapest setup. Runs done in an app have no usage meter, so theirs are estimated from the same setup’s measured runs. How we compare cost

About the task

The PM job

Taking a product or add-on to market with marketing and sales.

Why it matters

Launch plans fail by trying to reach everyone, by selling upside nobody feels, and by treating launch day as the whole plan.

What good looks like

  • One primary segment: those who value the difference most
  • Positions against what customers do today
  • Sells against a pain the buyer feels, with evidence
  • Diagnoses whether the problem is the message or the channel
  • Plans beyond launch week

Deliberately not measured

  • Creative copywriting quality
Capability tested

Go-to-market judgement

The failure we’re looking for

Every channel, every segment, and nothing after launch day

Grading

Decision model and LLM judge, calibrated against a blind PM review