Tasks / Growth

Write a GTM plan

Can the model pick the customers who value the difference most, position against what they do today, and build a go-to-market that lasts beyond launch day?

Task type v1.1 · 2 tasksLast changed 6 Oct 2026 · ChangelogDifficulty

What AI gets right here, and what you’ll still have to catch

From 16 graded outputs by 8 models. 50% were usable with at most a quick edit.

Reliably right

  1. Positioned against the real alternative100% pass
    It names the real alternatives (accountant-run payroll with mistakes, owner spreadsheet with risk) and ties the message to confidence and accuracy that those alternatives lack.
    GPT-6 Astra · ChatGPT · Payroll, for the people who fear it
  2. Protects the self-serve engine100% pass
    Uses the 70% free-to-paid stat and product-qualified deal data to argue against capping the free plan, and keeps self-serve open.
    GPT-6 Astra · ChatGPT · Enterprise, without killing the engine
  3. Does the capacity sums100% pass
    Works out deals needed (69-80), qualified opportunities (134), and AE-years (8) with checkable arithmetic.
    GPT-6 Astra · ChatGPT · Enterprise, without killing the engine

Where it slips

  1. Avoids unsupported claims47% pass
    Presents the incorrect 'five of six no decision' figure as established fact, which is an unsupported claim about the current situation.
    Opus 5.5 · Claude · Enterprise, without killing the engine
  2. Uses the supplied evidence correctly52% pass
    The claim 'Five of our six losses on existing-user deals ended in no decision' is not supported by the data (4 no-decision, 2 Miro), so a statement about the current situation is invented.
    Opus 5.5 · Claude · Enterprise, without killing the engine
  3. Identifies material uncertainty73% pass
    The plan does not name specific unknowns that could change the decision, nor does it say how they would be resolved or what would trigger a pivot.
    Gemini 3.8 Flash · API · Payroll, for the people who fear it

How it’s graded

Each check is built on how world-class product and growth leaders do this work, as they’ve described it on Lenny’s Podcast, and names the leader it comes from.

  1. Positioned against the real alternative

    Does the plan name what the target customer does today without this product (including the status quo or doing nothing), and tie the message to the value that alternative lacks?

    Passes when Names the real alternative and the differentiated value against it, and picks the segment that cares most about that value.

  2. Sells against a pain the buyer feels

    Is the core message built on a pain or risk the target buyer already feels, with evidence from the material, rather than on generic upside like saving time?

    Passes when Leads with the pain the evidence shows buyers care about, and uses that evidence to choose the message.

  3. Diagnoses message or channel first

    Does the plan work out whether the problem is the message (people reach it but don't convert) or the channels (the right people never see it), and spend accordingly?

    Passes when Uses conversion and reach data to say which it is, and puts the effort there.

  4. Plans beyond launch week

    Does the plan say how demand keeps coming after launch, with the ongoing engine and what's measured over months, rather than treating launch day as the whole plan?

    Passes when Gives the ongoing motion after launch and the measures that show whether it's working over time.

Plus our standard checks

Uses the supplied evidence correctly · Addresses the actual decision · Respects explicit constraints · Identifies material uncertainty · Avoids unsupported claims · Produces the required deliverable · and 2 written for each task, which you’ll see in the tasks below.

The tasks

Read the brief, then compare up to three outputs, each with the LLM judge’s verdict on every check. Highlights mark what a PM had to fix.

The brief

You're the PM for Ledgerly Payroll, a new add-on for our accounting product, launching to existing customers on 3 November. Marketing has drafted the launch plan below. Write the go-to-market plan you'd run instead (or the changes you'd make to theirs), in no more than 900 words: who we go after first, what we say to them, through which channels, and how we'll know it's working over the following three months. The files are attached.

What the model was given5 items: About Ledgerly, marketing_launch_draft.md, customers_by_size_and_payroll_method.csv (penalty notices: share who told us they got a payroll penalty notice last year), beta_headline_test.csv (two versions of the in-app payroll page, shown at random to beta customers), customer_calls.md
About LedgerlyAccounting software for small businesses: invoicing, bank feeds, bookkeeping. 8,000 paying customers. Payroll is a new add-on at $6 per employee per month.
marketing_launch_draft.md7 lines · Download
# Draft launch plan (Head of Marketing)

- Launch day 3 November: press release announcing Ledgerly as "the first Financial Operating System for small business".
- Paid social and search ads to new prospects: $60,000 in November.
- In-app banner to all 8,000 customers: "Run payroll in minutes".
- Webinar in launch week.
- Success measure: press mentions and sign-ups in launch week.
customers_by_size_and_payroll_method.csv (penalty notices: share who told us they got a payroll penalty notice last year)employees,current_payroll,customers,penalty_notices_last_year_pct 1-4,Owner does it in a spreadsheet,3140,4 1-4,Accountant runs it,1020,6 5-25,Owner does it in a spreadsheet,690,19 5-25,Accountant runs it,1480,23 5-25,Another payroll tool,1210,7 26-50,Another payroll tool,460,5
beta_headline_test.csv (two versions of the in-app payroll page, shown at random to beta customers)headline,landing_page_visits,started_setup,ran_first_payroll "Run payroll in minutes, right where you do your books",4210,92,38 Never get a payroll penalty again,4160,268,117
customer_calls.md10 lines · Download
# Customer calls about payroll (excerpts, September)

1. Bakery, 12 staff: "My accountant does payroll. Last spring they filed late and I got a $1,400 penalty. I don't know what I'm paying them for."
2. Plumbing firm, 18 staff: "Time isn't the issue. I'm scared of getting it wrong. The tax office letter was the worst week of my year."
3. Design studio, 3 staff: "It's me and two freelancers. A spreadsheet's fine."
4. Café group, 22 staff: "We use another tool, it's OK. Switching sounds like a nightmare mid-year."
5. Physio clinic, 9 staff: "I'd move tomorrow if it meant one less thing that can go wrong with the tax office."
…
What a strong answer doesThe answer key the graders mark against

Picks customers with 5 to 25 employees whose accountant runs payroll (1,480 customers, 23% had a penalty notice) and those doing it in a spreadsheet (690, 19%), not the smallest businesses (happy with a spreadsheet) or those on another payroll tool (switching mid-year is the barrier). Builds the message on fear of penalties and errors, not time: in the beta, 'Never get a payroll penalty again' got about three times as many setups as 'Run payroll in minutes' (268 against 92 from similar traffic, 6.4% against 2.2%) and more first payrolls (117 against 38), and the calls say the same. Reads this as a message problem, not a channel problem: the existing customer base already sees the page, so fix the message and reach existing customers in-app and through their accountants before spending $60,000 on ads to strangers. Drops the 'Financial Operating System' category claim. Plans beyond launch week: prompts around payroll and tax deadlines, accountant referrals, and monthly measures such as payroll attach in the target segment, first payrolls run, and penalties avoided.

Critical failures (cap the score at 40)
  • Keeps 'Run payroll in minutes' as the lead message despite the headline test
Task

Mid-level · v1.0

The outputs

8 setups answered this brief.

Usable with a quick edit10 of 12 checks passed

The plan is sound and evidence-based; a PM could trim ~50 words to meet the length limit without re-checking facts or redoing reasoning.

Output

Ledgerly Payroll: Go-to-Market Plan (launch 3 November)

The short version

Payroll is an add-on for people who already use Ledgerly, so the launch should be aimed at our existing customers. The draft plan spends most of its effort on new prospects and press, and it uses the headline that lost our own test. I'd make three changes:

  • Focus on the customers with the most painful problem.
  • Lead with the message that tripled conversion in beta.
  • Measure customers actually running payroll and still running it, not launch-week buzz.

1. Who we go after first

Primary: customers with 5–25 employees who run payroll in a spreadsheet or through their accountant. That's 2,170 customers.

  • They have the highest penalty rates: 19% for spreadsheet users and 23% for accountant-run payroll, against 4–7% everywhere else.
  • They have enough employees to matter. At roughly 12 staff, a customer is worth about $72 a month.
  • The calls describe this group exactly: the bakery's $1,400 penalty, the plumber's "scared of getting it wrong", the physio who'd "move tomorrow", and the landscaper paying $180 a month for mistakes.

Second wave: customers with 5–25 employees on another payroll tool (1,210), plus the 460 with 26–50 employees. They're fairly satisfied (5–7% penalties), and switching mid-year "sounds like a nightmare." We should approach them ahead of the next tax-year start, with a migration offer. Until then we just let them know payroll exists.

Not a focus: customers with 1–4 employees (4,160). They have low pain ("a spreadsheet's fine", "annoying, but free") and low revenue per customer. They can find payroll in the product without us pushing it.

2. What we say

The lead message is penalties and accuracy, not speed. In the beta, "Never get a payroll penalty again" won clearly over the speed headline:

Speed headlinePenalty headline
Visits that started setup2.2%6.4%
Visits that ran a first payroll0.9%2.8%

That's about three times as many customers running payroll, on comparable traffic of around 4,200 visits each. "Run payroll in minutes" is the losing message, and the calls explain why: "Time isn't the issue."

One important caveat: Legal must approve the penalty claim before we use it. "Never get a penalty again" is a promise. Penalties can still happen if a customer enters bad data or doesn't fund payroll on time. We have two options:

  • Back the promise: for example, "If a filing we're responsible for is late or wrong, we pay the penalty." This would be the strongest version, if Finance and Legal agree.
  • Soften the wording: for example, "Payroll filed correctly and on time, automatically." We would then re-test this against the winning headline in the first two weeks, because softening may reduce its effect.

Supporting points: - For accountant-run customers: "$6 per employee. For 6 staff, that's $36 a month, not $180." - For everyone: it already lives inside your books, so there's no re-entering data. This is a supporting point, not the headline.

Drop "the first Financial Operating System for small business." No customer has asked for this, it's hard to substantiate, and it says nothing about payroll.

3. Channels

ChannelWhat we do
In-app (targeted)Show the penalty-led payroll page to the 2,170 primary customers only, not a banner to all 8,000.
EmailA three-email sequence to the primary segment: the penalty story, the price comparison, and a booked setup slot.
Assisted setupOffer a free 30-minute call to get the first payroll running. In beta, more than half of customers who started setup never ran payroll, so this is our biggest point of loss.
WebinarMake it "How to avoid payroll penalties" with a live setup clinic. Invite the primary segment, and repeat it monthly rather than once in launch week.
AccountantsThis is a real risk. Some accountants will lose payroll fees, and they influence whether customers stay with Ledgerly at all. We should give accountants a read-only payroll view and position them as reviewers, so they aren't pushed out. Customer Success should monitor whether the core product loses customers in accountant-managed accounts.
PressKeep a modest announcement focused on payroll for small businesses, ideally with a beta customer's story. Press is not a success measure.

Budget: move the $60,000 away from paid ads to new prospects. New prospects would have to buy accounting before they can buy payroll, so that spend works against our focus. I'd use roughly: - $25,000 for assisted setup staffing and onboarding. - $5,000 for content and the webinar. - The rest held back until December, then released only if setup completion proves to be the bottleneck.

4. How we'll know it's working (November to January)

The main goal is customers who have run at least two payroll cycles. Signing up isn't enough; running payroll repeatedly is what matters.

MetricTarget by end of January
Primary-segment customers running payroll (2+ cycles)220, about 10% of the segment. That's roughly 2,600 employees and about $16,000 in monthly recurring revenue.
Setup started ÷ payroll page visits, primary segment6% or higher (at least matching the beta)
First payroll run ÷ setup startedUp from 44% to 60% or higher, through assisted setup
Time from setup start to first payrollMedian under 7 days
Penalties caused by LedgerlyZero. Every incident gets a root-cause review.
Customers who stop using payroll after their first runUnder 10%
Accounting product cancellations in the primary segmentNo increase compared with before launch

Check-ins: - Weekly: track the funnel by segment. - Week 2: read the headline re-test, if Legal changes the claim. - End of December: go/no-go on the held-back budget and on starting the switching campaign for customers on other payroll tools.

Caveats

  • The beta participants opted in, so real launch conversion will probably be lower than the beta numbers.
  • Penalty rates are self-reported.
  • The 220 target is a starting estimate, not a forecast. We should revise it after the first month of real data.

Grades and run details

Decision model 96 · LLM judge 85
Decision model checks
  • passUses the supplied evidence correctly24%
  • passAddresses the actual decision91%
  • passRespects explicit constraints42%
  • passIdentifies material uncertainty96%
  • partialAvoids unsupported claims38%
  • passProduces the required deliverable91%
  • passPicks the segment with the pain100%
  • passUses the headline test100%
  • passPositioned against the real alternative100%
  • passSells against a pain the buyer feels100%
  • passDiagnoses message or channel first100%
  • passPlans beyond launch week100%
Run
Run
#1
Time to output
39 s
Submitted
5 Oct 2026

Needs more than a quick edit10 of 12 checks passed

The output incorrectly states that about half the base is a poor fit, which would require re-checking the data to correct; otherwise the plan is solid but that error undermines trust.

Output

Ledgerly Payroll: go-to-market plan

What I'd change in Marketing's draft

DraftProblemChange
$60k paid ads to new prospectsPayroll is an add-on, so a prospect would first have to adopt our accounting product. We have 8,000 customers and haven't yet proven the message with them.Hold most of the budget until the message converts in-base (see Budget).
Banner to all 8,000About half the base is a poor fit (details below).Target about 2,170 customers first.
"Financial Operating System" press releaseCustomers never use this language. They talk about fear of penalties and the tax office.Keep the press small and plain.
Success = press mentions and launch-week sign-upsNeither measures payroll actually being run.Measure first payroll run and repeat runs.

Who we go after first

Wave 1 (3 Nov): customers with 5–25 employees who run payroll via spreadsheet (690) or an accountant (1,480). That is 2,170 customers, 27% of the base.

  • They have the pain. 19% (spreadsheet) and 23% (accountant) got a payroll penalty notice last year, versus 4–6% for businesses with 1–4 staff and 5–7% for those already on a payroll tool.
  • They have the money. At $6 per employee, a 12-person business pays about $72 a month, against the $180 a month one landscaper pays their accountant.
  • The calls match the data. The bakery, plumber, physio, landscaper and dental practice all describe fear of mistakes, not lack of time.

Not now: - 1–4 employees (4,160 customers): The spreadsheet is "fine" and "free," and only 4–6% were penalized. Leave payroll available in the product but don't push it. - Other-tool users (1,670 customers): Penalty rates are low (5–7%), and switching mid-year "sounds like a nightmare." Revisit at the start of the next tax year with a migration offer.

What we say

The beta test was randomized. "Never get a payroll penalty again" beat "Run payroll in minutes" on setup starts (6.4% vs 2.2% of visits) and on first payroll runs (2.8% vs 0.9%), about 3x on both. Once people started, completion was similar (44% vs 41%), so the headline drives the decision to begin and the product handles the rest.

Core message: "Payroll filed right and on time, so the tax office letter never arrives."

  • Spreadsheet owners: "A spreadsheet won't tell you when you've got it wrong."
  • Accountant-run: "Stop paying $180 a month to worry about it." Use this to speak to cost and control without attacking accountants. Offer a way to invite their accountant in as a reviewer.

Accuracy promise: The dental practice said, "If you can promise it's right, I'm in." I'd like to offer a penalty-cover guarantee for errors caused by our software. Legal and Finance must approve it by 24 Oct. If they won't, the copy becomes "filed right and on time, with support to fix problems." We should not promise "never" without backing it. The beta headline did well, but a claim we can't stand behind creates risk.

Channels

  1. Targeted in-app card and email to the Wave 1 list only, using the winning headline. This is the main channel, and it costs almost nothing.
  2. Personal outreach from Customer Success to Wave 1 customers who told us they got a penalty notice (about 470 people by my estimate from the survey rates). Offer a 20-minute guided setup.
  3. Webinar, reframed: "Penalty-proof your payroll" with a live setup walkthrough. Run it in launch week and again in week 4, since the second session catches people who weren't ready the first time.
  4. Press: one plain release ("Ledgerly adds payroll, $6 per employee"), with no category claim. Follow with customer stories once we have them.
  5. Paid: none to new prospects in November (see Budget).

Budget (the $60k)

  • ~$15k: guided setup and migration help (contractor or CS time).
  • ~$10k: webinars, email production, and case-study content.
  • ~$35k: held until the January gate. If Wave 1 works, spend it on search ads for "payroll penalty" intent, aimed at businesses that would adopt accounting and payroll together.

How we'll know it's working (3 months)

Primary metric: paying accounts that have run at least two payrolls. This replaces sign-ups, press mentions and launch-week anything.

Targets (to be recalibrated after week 2, since the beta is my only baseline): - Month 1: at least 5% of Wave 1 has started setup. At least 45% of those run a first payroll, matching beta's 44%. - End of January: about 250 Wave 1 accounts running payroll (~11% of the segment). I'm assuming a 10-person average, which implies roughly $15k MRR. I'll replace that assumption with real headcount data. - Retention: at least 85% of those accounts run payroll in three consecutive cycles.

Guardrails (any breach pauses expansion): - Payroll errors or penalties caused by us: zero tolerance, reviewed weekly. - Support tickets per payroll run. - Churn of the core accounting subscription, which must not rise among adopters.

Measuring real impact: Hold out a random 10% of Wave 1 from all promotion. If adopters in the promoted group don't clearly exceed the holdout, we're only capturing people who would have found payroll anyway.

Decision gates: - Mid-December: If setup-to-first-payroll is below 35%, fix onboarding before pushing further. - Mid-January: If Wave 1 is at or above target and no guardrail is breached, release the held budget and start Wave 2 (other-tool users, ahead of the tax-year switch window). If it's below target, interview non-adopters before spending anything.

Caveats

  • Penalty rates are self-reported, and the eight calls are anecdotes.
  • Beta customers may be more enthusiastic than the base.
  • I haven't seen average headcount per segment, so the revenue estimates are rough.

Claims the judge couldn’t find in the brief

  • About half the base is a poor fit

Grades and run details

Decision model 100 · LLM judge 77
Decision model checks
  • passUses the supplied evidence correctly13%
  • passAddresses the actual decision99%
  • passRespects explicit constraints32%
  • passIdentifies material uncertainty99%
  • passAvoids unsupported claims38%
  • passProduces the required deliverable75%
  • passPicks the segment with the pain100%
  • passUses the headline test100%
  • passPositioned against the real alternative100%
  • passSells against a pain the buyer feels100%
  • passDiagnoses message or channel first100%
  • passPlans beyond launch week100%
Run
Run
#1
API response time
37 s
Submitted
6 Oct 2026

Usable with a quick edit9 of 12 checks passed

The plan is clear, evidence-based, and actionable; a PM could use it with minor wording tweaks and without rechecking the data or redoing the reasoning.

Output

Critique of the Marketing Draft

The marketing team’s launch plan has three fatal flaws: 1. Wrong audience for the budget: Spending $60,000 on paid acquisition to cold prospects for a new add-on is wasteful. We have 8,000 existing, engaged accounting customers. Our CAC here should be close to zero. 2. Ignoring empirical data on messaging: The draft uses "Run payroll in minutes" and PR buzzwords ("Financial Operating System"). Our beta test proved this fails: "Never get a payroll penalty again" drove 3.1x more completed payrolls (2.81% conversion vs. 0.90%). Customers don't care about speed—they fear tax penalties and accountant errors. 3. Vanity metrics: Press mentions and launch-week signups tell us nothing about adoption, product usage, or recurring revenue.

Here is the revised go-to-market plan.

---

1. Target Audience: Who We Go After First

We must prioritize based on revenue potential ($6/employee/month) and pain intensity (penalty notices):

``` Target Tier Segment (Size & Method) Base Penalty % Priority Tier 1 (Core) 5–25 emp: Accountant runs it 1,480 23% Immediate (Day 1) 5–25 emp: Spreadsheet 690 19% Immediate (Day 1) Tier 2 (Secondary)5–25 emp: Another payroll tool 1,210 7% Prep for Jan 1 Switch 1–4 emp: Accountant runs it 1,020 6% Nurture Deprioritize 1–4 emp: Spreadsheet (3,140) 3,140 4% Self-serve banner only 26–50 emp: Another tool (460) 460 5% Later phase ```

  • Primary Beachhead (Tier 1: 2,170 customers): Businesses with 5–25 employees doing payroll via spreadsheets or accountants. They suffer a massive 19–23% penalty rate, generate $30–$150/month each in add-on revenue, and are actively frustrated by accountant fees and errors (e.g., customer paying $180/month for late filings).
  • Secondary Opportunity (Tier 2): Those using other payroll tools (1,210 customers) are hesitant to switch mid-year. We target them in late November/December for a clean transition on January 1 (start of the new tax year).
  • Deprioritized: 1–4 employee spreadsheet users (3,140 customers) are complacent ("spreadsheet is fine/free") with low penalty exposure. We will expose them to in-app banners, but spend zero outbound effort on them.

---

2. Positioning & Messaging: What We Say

  • The Core Hook: "Never get a payroll penalty again. Guaranteed accuracy, right inside Ledgerly."
  • The Value Proposition: Ledgerly automates tax calculations, filings, and direct deposits directly from your books—eliminating accountant errors, late tax notices, and Sunday-night spreadsheet anxiety.
  • Segment Angles:
  • For Accountant-run payroll: "Fire the $180/month bill and the filing mistakes. Automate accurate payroll in Ledgerly for a fraction of the cost."
  • For Spreadsheet users: "One tax office notice costs more than three years of Ledgerly Payroll. Get automated tax filings and sleep easy."
  • For Competitor switchers: "Make January 1 your fresh start: switch to unified bookkeeping + payroll in 15 minutes."

---

3. Channel Strategy & Execution

Reallocate the $60,000 paid budget into onboarding support and switching incentives.

#### A. In-App Contextual Triggers (November 3) * Deploy the winning headline ("Never get a payroll penalty again") across dashboard real estate. * Trigger contextual prompts when users categorize payroll-related expenses (e.g., bank feed lines showing accountant payroll fees or tax payments): "Paying an accountant for payroll? Automate it error-free in Ledgerly for $6/employee."

#### B. Direct Outbound & Email Drips (Segmented) * Tier 1 (Accountant & Spreadsheet, 5–25 emp): Send a 3-part sequence focused on penalty protection, automatic tax filing, and reconciliation. Include a penalty-risk calculator. * Concierge Migration: Offer dedicated, white-glove onboarding for 5–25 employee accounts to import employee tax files and YTD data.

#### C. Education, Not a Feature Webinar * Replace the generic product demo with an expert session: "Avoiding Payroll Penalties in Q4 and Prepping for the 2025 Tax Year." Feature a customer testimonial sharing the pain of a tax notice.

#### D. The "January 1 Fresh Start" Campaign (Mid-November – December) * Target competitor users (1,210 customers) with an early-bird onboarding offer: "Set up your company before December 15; your first payroll run on January 1 is free."

---

4. 3-Month Success Metrics

We will track activation and usage, not press mentions.

#### Leading Indicators (Operational) * Setup-to-Run Conversion: >45% of users who start setup complete their first payroll run within 30 days (up from beta's 43.6%). * Tier 1 Activation: >12% of the 5–25 employee accountant/spreadsheet cohort (260 accounts) activate by Day 90.

#### 3-Month Milestones

  • Month 1 (November): Initial Adoption
  • 150 businesses run their first payroll.
  • Concierge migration processes 50+ businesses with >5 employees.
  • Target MRR added: ~$9,000.
  • Month 2 (December): Pipeline & Retention
  • 100% of Month 1 users run their second payroll cycle (0% churn).
  • 200 additional competitor-switch businesses pre-configure payroll to go live on January 1.
  • Target cumulative MRR: ~$18,000.
  • Month 3 (January): The Tax Year Switch Surge
  • Over 450 total active customers running payroll by Jan 31.
  • Blended average of 8 employees/account = 3,600 employees on platform.
  • Run-Rate Target: $21,600 MRR (~$260,000 ARR) achieved within 90 days of launch.
  • Zero filing penalties incurred by customers using Ledgerly Payroll.

Grades and run details

Decision model 71 · LLM judge 92
Decision model checks
  • failUses the supplied evidence correctly79%
  • passAddresses the actual decision55%
  • partialRespects explicit constraints14%
  • failIdentifies material uncertainty90%
  • failAvoids unsupported claims49%
  • passProduces the required deliverable61%
  • passPicks the segment with the pain100%
  • passUses the headline test97%
  • passPositioned against the real alternative100%
  • passSells against a pain the buyer feels100%
  • passDiagnoses message or channel first94%
  • passPlans beyond launch week100%
Run
Run
#1
API response time
27 s
Submitted
6 Oct 2026

Check by check

Where the outputs differ comes first. Open a check to read the judge’s reasons. “Mixed” means the two graders disagreed.

Where they differ 5

Uses the supplied evidence correctlyRightMixedMixed
Opus 5.5 · Claude

All statements about the current situation are directly supported by the brief or supplied context.

Sonnet 5.5 · API

The claim 'About half the base is a poor fit' is not supported by the data; the target segment is 27%, not half.

Gemini 3.8 Flash · API

All factual claims about the current situation are taken directly from the supplied context or derived by correct arithmetic; the 43.6% conversion rate is the winning headline's setup-to-run rate, not an invention.

Respects explicit constraintsMixedRightRight
Opus 5.5 · Claude

The output is approximately 952 words, exceeding the 900-word limit.

Sonnet 5.5 · API

Delivers a go-to-market plan under 900 words, addressing the required elements, and respects the reader and form.

Gemini 3.8 Flash · API

The output is a go-to-market plan under 900 words, addresses the required elements, and respects the brief's form and reader.

Identifies material uncertaintyRightRightWrong
Opus 5.5 · Claude

Names specific unknowns (legal risk, beta opt-in bias, self-reported data) and how they would be resolved or change the plan.

Sonnet 5.5 · API

Names self-reported penalty data, anecdotal calls, beta enthusiasm bias, missing headcount data, and legal approval for guarantee, with resolution steps.

Gemini 3.8 Flash · API

The plan does not name specific unknowns that could change the decision, nor does it say how they would be resolved or what would trigger a pivot.

Avoids unsupported claimsRightMixedMixed
Opus 5.5 · Claude

Hypotheses and forecasts are labelled as such (e.g., 'probably be lower', 'starting estimate, not a forecast').

Sonnet 5.5 · API

Presents 'About half the base is a poor fit' as fact when the data shows 73% are not in the target, not half.

Gemini 3.8 Flash · API

Interpretations like 'customers fear penalties' are backed by the beta test and customer calls, and the output does not present unsupported causes or forecasts as fact.

Produces the required deliverableMixedRightRight
Opus 5.5 · Claude

The plan is present and complete but exceeds the 900-word limit, so it is not fully within the requested length.

Sonnet 5.5 · API

Complete plan with changes, segment, message, channels, measures, budget, and caveats; usable as-is with light edits.

Gemini 3.8 Flash · API

The output is a complete, actionable go-to-market plan within the word limit, with target audience, messaging, channels, and three-month metrics.

All got right 7

Addresses the actual decisionRightRightRight
Opus 5.5 · Claude

Commits early to a primary segment, message, and channels, and states conditions that would change the call (legal approval, re-test, go/no-go).

Sonnet 5.5 · API

Commits to Wave 1 segment, message, channels, and decision gates with clear conditions for changing course.

Gemini 3.8 Flash · API

The output commits early and unambiguously to a revised go-to-market plan with a primary segment, message, channels, and metrics, framed for the product manager.

Picks the segment with the painRightRightRight
Opus 5.5 · Claude

Targets 5–25 employee customers using spreadsheet or accountant, with numbers and penalty rates, and explains why smaller and tool-using segments come later.

Sonnet 5.5 · API

Targets 5-25 employee customers with accountant or spreadsheet payroll, cites penalty rates, and explains why 1-4 and other-tool users are not first.

Gemini 3.8 Flash · API

The plan targets 5–25 employee businesses using accountant-run or spreadsheet payroll, cites the penalty rates and customer counts, and explains why smaller businesses and tool-switchers are deprioritized.

Uses the headline testRightRightRight
Opus 5.5 · Claude

Cites the beta test results and builds the message on penalties and accuracy.

Sonnet 5.5 · API

Uses the beta headline test results (3x setup lift) to build the message on penalties and accuracy.

Gemini 3.8 Flash · API

The plan explicitly cites the beta headline test results (3.1x more completed payrolls) and builds the core message on penalties and accuracy.

Positioned against the real alternativeRightRightRight
Opus 5.5 · Claude

Names the real alternatives (spreadsheet, accountant, another tool) and ties the message to penalty risk and cost savings.

Sonnet 5.5 · API

Names the real alternatives (spreadsheet, accountant, other tool) and ties message to penalty avoidance and accuracy, targeting the segment that cares most.

Gemini 3.8 Flash · API

The plan names the real alternatives (accountant, spreadsheet, another tool) and ties the message to eliminating penalties and errors, targeting the segment that cares most.

Sells against a pain the buyer feelsRightRightRight
Opus 5.5 · Claude

Leads with the pain of penalties and errors, supported by calls and beta data, not generic time-saving.

Sonnet 5.5 · API

Leads with fear of penalties and tax office letters, supported by call quotes and headline test, not generic time-saving.

Gemini 3.8 Flash · API

The core message leads with the pain of tax penalties and errors, supported by the beta test and customer calls, not generic time-saving.

Diagnoses message or channel firstRightRightRight
Opus 5.5 · Claude

Diagnoses a message problem (losing headline, existing customers already see the page) and shifts budget from new-prospect ads to in-app, email, and assisted setup.

Sonnet 5.5 · API

Diagnoses message as the problem (beta conversion data) and focuses on in-app/email to existing base, holding paid ads.

Gemini 3.8 Flash · API

The plan uses conversion data to diagnose that the problem is the message, not reach, and reallocates budget from paid ads to in-app and existing-customer channels.

Plans beyond launch weekRightRightRight
Opus 5.5 · Claude

Plans ongoing monthly webinars, three-month metrics, weekly check-ins, and a December go/no-go, not just launch week.

Sonnet 5.5 · API

Plans ongoing motion with second webinar, monthly measures, retention, guardrails, holdout group, and Wave 2 decision gate.

Gemini 3.8 Flash · API

The plan includes ongoing in-app triggers, email drips, a January 1 campaign, and monthly metrics over three months, not just launch week.

Results

Every setup we’ve tested on this task type, across all its tasks and repeats, graded on the current checklist. Calibrated.

#Model · HarnessTask scoreDecision modelLLM judgeRunsCritical failures
1GPT-6 AstrawithChatGPT100.0100.02None
2Opus 5.5withClaude96.085.22None
3GPT-6 LunawithAPI94.289.32None
4GPT-6.1 SolwithAPI93.878.02None
5Sonnet 5.5withAPI98.177.72None
6Haiku 5.5withAPI94.277.72None
7Gemini 3.8 FlashwithAPI73.981.92None
8Gemini 3.5 Flash-LitewithGemini74.263.52None

What you get for the cost

No setup scores within 5 points of the leader at half its cost or less here, so the leader is also the best value.

Task score against relative cost for each setup. The table below has the same figures.607080901001×3×10×30×100×Cost per output, relative to the cheapest setup (log scale) →↑ Task scoreGPT-6 Astra · ChatGPT: task score 100, 81× the cheapestOpus 5.5 · Claude: task score 91, 75× the cheapestGPT-6 Luna · API: task score 90, the cheapestGPT-6.1 Sol · API: task score 84, 16× the cheapestSonnet 5.5 · API: task score 84, 46× the cheapestHaiku 5.5 · API: task score 82, 3.3× the cheapestGemini 3.8 Flash · API: task score 77, 11× the cheapestGemini 3.5 Flash-Lite · Gemini: task score 68, 3.5× the cheapestGPT-6 Astra · ChatGPTOpus 5.5 · ClaudeGPT-6 Luna · APIGPT-6.1 Sol · APISonnet 5.5 · APIHaiku 5.5 · APIGemini 3.8 Flash · APIGemini 3.5 Flash-Lite · Gemini
Nobody beats these on both score and cost
Task score and relative cost by setup
SetupTask scoreRelative cost (cheapest = 1)
GPT-6 Luna · API901.0×
Haiku 5.5 · API823.3×
Gemini 3.5 Flash-Lite · Gemini683.5×
Gemini 3.8 Flash · API7711×
GPT-6.1 Sol · API8416×
Sonnet 5.5 · API8446×
Opus 5.5 · Claude9175×
GPT-6 Astra · ChatGPT10081×

Cost is each output’s tokens at the model’s list price, shown relative to the cheapest setup. Runs done in an app have no usage meter, so theirs are estimated from the same setup’s measured runs. How we compare cost

About the task

The PM job

Taking a product or add-on to market with marketing and sales.

Why it matters

Launch plans fail by trying to reach everyone, by selling upside nobody feels, and by treating launch day as the whole plan.

What good looks like

  • One primary segment: those who value the difference most
  • Positions against what customers do today
  • Sells against a pain the buyer feels, with evidence
  • Diagnoses whether the problem is the message or the channel
  • Plans beyond launch week

Deliberately not measured

  • Creative copywriting quality
Capability tested

Go-to-market judgement

The failure we’re looking for

Every channel, every segment, and nothing after launch day

Grading

Decision model and LLM judge, calibrated against a blind PM review