Tasks / Growth

Write a GTM plan

Can the model pick the customers who value the difference most, position against what they do today, and build a go-to-market that lasts beyond launch day?

Task type v1.1 · 2 tasksLast changed 6 Oct 2026 · ChangelogDifficulty

What AI gets right here, and what you’ll still have to catch

From 16 graded outputs by 8 models. 50% were usable with at most a quick edit.

Reliably right

  1. Positioned against the real alternative100% pass
    It names the real alternatives (accountant-run payroll with mistakes, owner spreadsheet with risk) and ties the message to confidence and accuracy that those alternatives lack.
    GPT-6 Astra · ChatGPT · Payroll, for the people who fear it
  2. Protects the self-serve engine100% pass
    Uses the 70% free-to-paid stat and product-qualified deal data to argue against capping the free plan, and keeps self-serve open.
    GPT-6 Astra · ChatGPT · Enterprise, without killing the engine
  3. Does the capacity sums100% pass
    Works out deals needed (69-80), qualified opportunities (134), and AE-years (8) with checkable arithmetic.
    GPT-6 Astra · ChatGPT · Enterprise, without killing the engine

Where it slips

  1. Avoids unsupported claims47% pass
    Presents the incorrect 'five of six no decision' figure as established fact, which is an unsupported claim about the current situation.
    Opus 5.5 · Claude · Enterprise, without killing the engine
  2. Uses the supplied evidence correctly52% pass
    The claim 'Five of our six losses on existing-user deals ended in no decision' is not supported by the data (4 no-decision, 2 Miro), so a statement about the current situation is invented.
    Opus 5.5 · Claude · Enterprise, without killing the engine
  3. Identifies material uncertainty73% pass
    The plan does not name specific unknowns that could change the decision, nor does it say how they would be resolved or what would trigger a pivot.
    Gemini 3.8 Flash · API · Payroll, for the people who fear it

How it’s graded

Each check is built on how world-class product and growth leaders do this work, as they’ve described it on Lenny’s Podcast, and names the leader it comes from.

  1. Positioned against the real alternative

    Does the plan name what the target customer does today without this product (including the status quo or doing nothing), and tie the message to the value that alternative lacks?

    Passes when Names the real alternative and the differentiated value against it, and picks the segment that cares most about that value.

  2. Sells against a pain the buyer feels

    Is the core message built on a pain or risk the target buyer already feels, with evidence from the material, rather than on generic upside like saving time?

    Passes when Leads with the pain the evidence shows buyers care about, and uses that evidence to choose the message.

  3. Diagnoses message or channel first

    Does the plan work out whether the problem is the message (people reach it but don't convert) or the channels (the right people never see it), and spend accordingly?

    Passes when Uses conversion and reach data to say which it is, and puts the effort there.

  4. Plans beyond launch week

    Does the plan say how demand keeps coming after launch, with the ongoing engine and what's measured over months, rather than treating launch day as the whole plan?

    Passes when Gives the ongoing motion after launch and the measures that show whether it's working over time.

Plus our standard checks

Uses the supplied evidence correctly · Addresses the actual decision · Respects explicit constraints · Identifies material uncertainty · Avoids unsupported claims · Produces the required deliverable · and 2 written for each task, which you’ll see in the tasks below.

The tasks

Read the brief, then compare up to three outputs, each with the LLM judge’s verdict on every check. Highlights mark what a PM had to fix.

The brief

You're the PM for Ledgerly Payroll, a new add-on for our accounting product, launching to existing customers on 3 November. Marketing has drafted the launch plan below. Write the go-to-market plan you'd run instead (or the changes you'd make to theirs), in no more than 900 words: who we go after first, what we say to them, through which channels, and how we'll know it's working over the following three months. The files are attached.

What the model was given5 items: About Ledgerly, marketing_launch_draft.md, customers_by_size_and_payroll_method.csv (penalty notices: share who told us they got a payroll penalty notice last year), beta_headline_test.csv (two versions of the in-app payroll page, shown at random to beta customers), customer_calls.md
About LedgerlyAccounting software for small businesses: invoicing, bank feeds, bookkeeping. 8,000 paying customers. Payroll is a new add-on at $6 per employee per month.
marketing_launch_draft.md7 lines · Download
# Draft launch plan (Head of Marketing)

- Launch day 3 November: press release announcing Ledgerly as "the first Financial Operating System for small business".
- Paid social and search ads to new prospects: $60,000 in November.
- In-app banner to all 8,000 customers: "Run payroll in minutes".
- Webinar in launch week.
- Success measure: press mentions and sign-ups in launch week.
customers_by_size_and_payroll_method.csv (penalty notices: share who told us they got a payroll penalty notice last year)employees,current_payroll,customers,penalty_notices_last_year_pct 1-4,Owner does it in a spreadsheet,3140,4 1-4,Accountant runs it,1020,6 5-25,Owner does it in a spreadsheet,690,19 5-25,Accountant runs it,1480,23 5-25,Another payroll tool,1210,7 26-50,Another payroll tool,460,5
beta_headline_test.csv (two versions of the in-app payroll page, shown at random to beta customers)headline,landing_page_visits,started_setup,ran_first_payroll "Run payroll in minutes, right where you do your books",4210,92,38 Never get a payroll penalty again,4160,268,117
customer_calls.md10 lines · Download
# Customer calls about payroll (excerpts, September)

1. Bakery, 12 staff: "My accountant does payroll. Last spring they filed late and I got a $1,400 penalty. I don't know what I'm paying them for."
2. Plumbing firm, 18 staff: "Time isn't the issue. I'm scared of getting it wrong. The tax office letter was the worst week of my year."
3. Design studio, 3 staff: "It's me and two freelancers. A spreadsheet's fine."
4. Café group, 22 staff: "We use another tool, it's OK. Switching sounds like a nightmare mid-year."
5. Physio clinic, 9 staff: "I'd move tomorrow if it meant one less thing that can go wrong with the tax office."
…
What a strong answer doesThe answer key the graders mark against

Picks customers with 5 to 25 employees whose accountant runs payroll (1,480 customers, 23% had a penalty notice) and those doing it in a spreadsheet (690, 19%), not the smallest businesses (happy with a spreadsheet) or those on another payroll tool (switching mid-year is the barrier). Builds the message on fear of penalties and errors, not time: in the beta, 'Never get a payroll penalty again' got about three times as many setups as 'Run payroll in minutes' (268 against 92 from similar traffic, 6.4% against 2.2%) and more first payrolls (117 against 38), and the calls say the same. Reads this as a message problem, not a channel problem: the existing customer base already sees the page, so fix the message and reach existing customers in-app and through their accountants before spending $60,000 on ads to strangers. Drops the 'Financial Operating System' category claim. Plans beyond launch week: prompts around payroll and tax deadlines, accountant referrals, and monthly measures such as payroll attach in the target segment, first payrolls run, and penalties avoided.

Critical failures (cap the score at 40)
  • Keeps 'Run payroll in minutes' as the lead message despite the headline test
Task

Mid-level · v1.0

The outputs

8 setups answered this brief.

Usable with a quick edit9 of 12 checks passed

The plan is clear and actionable, with only a minor unsupported claim about acquisition cost that can be edited out; no re-checking of evidence is needed.

Output

To: Executive Team & Head of Marketing From: PM, Ledgerly Payroll Date: October 18 Subject: Revised GTM Plan: Ledgerly Payroll Launch (3 November)

The draft marketing plan targets a generic market with high-level branding. That’s a mistake. We are launching an add-on to an existing base of 8,000 accounting customers. Our unfair advantage isn't that we're a "Financial Operating System"—it's that we already hold their financial data.

We need to scrap the broad acquisition play for November, refocus entirely on our existing base, and lead with our strongest asset: penalty anxiety and accuracy, not speed.

Here are the critical changes I am making to our launch plan.

---

1. Who We Go After First (The Wedge)

The data and customer calls tell a clear story.

  • Ignore: The 1-4 employee segment doing payroll in spreadsheets (Retail shop: "Annoying, but free"). They don't feel enough pain to switch or pay $6/mo.
  • Ignore: Mid-year switchers currently using dedicated payroll tools (Café group: "Switching sounds like a nightmare").
  • Target (Our Wedge): Small businesses with 5–25 employees who currently rely on an accountant or a spreadsheet, and are bleeding money/stress on penalties.

Look at the data: 5–25 employee businesses using accountants or spreadsheets suffer 19% to 23% penalty rates annually. Furthermore, customer calls highlight deep resentment toward accountant-driven payroll failures ($1,400 fines, $180/mo fees for mistakes).

We will segment our 8,000 customers on Day 1 and suppress micro-businesses (1-4) from our primary payroll marketing to protect email reputation and conversion relevance.

2. What We Say to Them (The Message)

Marketing’s draft leads with: "Run payroll in minutes."

Our beta test proved this is the wrong hook. Look at the `betaheadlinetest.csv` data: * "Run payroll in minutes..." generated only 38 completed payrolls. * "Never get a payroll penalty again" generated 117 completed payrolls (more than 3x the activation rate).

Time isn't the primary enemy; fear is (Plumbing firm: "I'm scared of getting it wrong. The tax office letter was the worst week of my year").

Our Core Value Proposition: “Keep payroll inside Ledgerly. Zero manual data entry from your books, guaranteed accurate tax calculations, or we cover the penalty.” (Note: We should seriously explore a penalty-reimbursement guarantee as a killer differentiator).

3. Through Which Channels

We are shifting $50,000 of the proposed November budget away from broad paid social/search targeting cold prospects. Acquiring cold payroll customers is expensive and slow; cross-selling our 8,000 warm customers is high-margin.

  • Channel 1: Targeted In-App Real Estate (Nov 3): Replace the generic banner with a dynamic state. If the user has 5–25 employees and uses an accountant/spreadsheet, show the "Never get a payroll penalty again" headline.
  • Channel 2: Founder/PM Email Campaign (Nov 4–10): A direct, non-marketing-sounding email sent from me (PM) and the CEO to the 5–25 segment. Focus on the hidden costs of accountant errors and spreadsheet slip-ups. Include a 1-click "Import my data" preview.
  • Channel 3: The "Fire Your Accountant's Payroll Bill" Playbook: A targeted retargeting campaign (using the remaining $10k of paid budget) aimed strictly at custom matched lists of our 5-25 employee segment, highlighting how much they can save by bringing payroll into Ledgerly alongside their bookkeeping.

4. How We’ll Measure Success (Months 1–3)

Marketing’s draft relies on vanity metrics: press mentions and launch-week sign-ups. We care about retention and activation. Ledgerly Payroll is priced at $6/employee/month; a sign-up that doesn’t run payroll is worthless.

Over the next three months, we will track four strict OKRs:

  • Month 1 (Launch & Activation):
  • Metric: 500 active payroll accounts set up from the 5–25 employee segment.
  • Guardrail Metric: Zero critical calculation support tickets in the first 14 days. Accuracy is our brand promise; a buggy launch kills trust permanently.
  • Month 2 (Expansion & Accountant Displacement):
  • Metric: 30% of activated users migrating away from their external accountant/spreadsheet for payroll.
  • Metric: Average team size of adopting accounts sits solidly in the 8–15 employee range (validating our wedge).
  • Month 3 (Monetization & Retention):
  • Metric: $15,000 in recurring monthly add-on revenue (MRR).
  • Metric: 95% month-two retention for payroll runners (proving they didn't just test it for a month, but integrated it into their operations).

Summary of Changes

We are abandoning the "Financial Operating System" fluff and the broad acquisition spend. On November 3, we laser-focus on our 5–25 employee segment with a message built on fear-relief and accuracy, driving them straight into a high-converting in-app flow.

Grades and run details

Decision model 79 · LLM judge 77
Decision model checks
  • partialUses the supplied evidence correctly3%
  • passAddresses the actual decision61%
  • partialRespects explicit constraints29%
  • failIdentifies material uncertainty35%
  • partialAvoids unsupported claims49%
  • passProduces the required deliverable62%
  • passPicks the segment with the pain93%
  • passUses the headline test100%
  • passPositioned against the real alternative100%
  • passSells against a pain the buyer feels100%
  • passDiagnoses message or channel first96%
  • passPlans beyond launch week95%
Run
Run
#1
Time to output
6 s
Submitted
5 Oct 2026

Usable with a quick edit10 of 12 checks passed

The plan is sound and evidence-based; a PM could trim ~50 words to meet the length limit without re-checking facts or redoing reasoning.

Output

Ledgerly Payroll: Go-to-Market Plan (launch 3 November)

The short version

Payroll is an add-on for people who already use Ledgerly, so the launch should be aimed at our existing customers. The draft plan spends most of its effort on new prospects and press, and it uses the headline that lost our own test. I'd make three changes:

  • Focus on the customers with the most painful problem.
  • Lead with the message that tripled conversion in beta.
  • Measure customers actually running payroll and still running it, not launch-week buzz.

1. Who we go after first

Primary: customers with 5–25 employees who run payroll in a spreadsheet or through their accountant. That's 2,170 customers.

  • They have the highest penalty rates: 19% for spreadsheet users and 23% for accountant-run payroll, against 4–7% everywhere else.
  • They have enough employees to matter. At roughly 12 staff, a customer is worth about $72 a month.
  • The calls describe this group exactly: the bakery's $1,400 penalty, the plumber's "scared of getting it wrong", the physio who'd "move tomorrow", and the landscaper paying $180 a month for mistakes.

Second wave: customers with 5–25 employees on another payroll tool (1,210), plus the 460 with 26–50 employees. They're fairly satisfied (5–7% penalties), and switching mid-year "sounds like a nightmare." We should approach them ahead of the next tax-year start, with a migration offer. Until then we just let them know payroll exists.

Not a focus: customers with 1–4 employees (4,160). They have low pain ("a spreadsheet's fine", "annoying, but free") and low revenue per customer. They can find payroll in the product without us pushing it.

2. What we say

The lead message is penalties and accuracy, not speed. In the beta, "Never get a payroll penalty again" won clearly over the speed headline:

Speed headlinePenalty headline
Visits that started setup2.2%6.4%
Visits that ran a first payroll0.9%2.8%

That's about three times as many customers running payroll, on comparable traffic of around 4,200 visits each. "Run payroll in minutes" is the losing message, and the calls explain why: "Time isn't the issue."

One important caveat: Legal must approve the penalty claim before we use it. "Never get a penalty again" is a promise. Penalties can still happen if a customer enters bad data or doesn't fund payroll on time. We have two options:

  • Back the promise: for example, "If a filing we're responsible for is late or wrong, we pay the penalty." This would be the strongest version, if Finance and Legal agree.
  • Soften the wording: for example, "Payroll filed correctly and on time, automatically." We would then re-test this against the winning headline in the first two weeks, because softening may reduce its effect.

Supporting points: - For accountant-run customers: "$6 per employee. For 6 staff, that's $36 a month, not $180." - For everyone: it already lives inside your books, so there's no re-entering data. This is a supporting point, not the headline.

Drop "the first Financial Operating System for small business." No customer has asked for this, it's hard to substantiate, and it says nothing about payroll.

3. Channels

ChannelWhat we do
In-app (targeted)Show the penalty-led payroll page to the 2,170 primary customers only, not a banner to all 8,000.
EmailA three-email sequence to the primary segment: the penalty story, the price comparison, and a booked setup slot.
Assisted setupOffer a free 30-minute call to get the first payroll running. In beta, more than half of customers who started setup never ran payroll, so this is our biggest point of loss.
WebinarMake it "How to avoid payroll penalties" with a live setup clinic. Invite the primary segment, and repeat it monthly rather than once in launch week.
AccountantsThis is a real risk. Some accountants will lose payroll fees, and they influence whether customers stay with Ledgerly at all. We should give accountants a read-only payroll view and position them as reviewers, so they aren't pushed out. Customer Success should monitor whether the core product loses customers in accountant-managed accounts.
PressKeep a modest announcement focused on payroll for small businesses, ideally with a beta customer's story. Press is not a success measure.

Budget: move the $60,000 away from paid ads to new prospects. New prospects would have to buy accounting before they can buy payroll, so that spend works against our focus. I'd use roughly: - $25,000 for assisted setup staffing and onboarding. - $5,000 for content and the webinar. - The rest held back until December, then released only if setup completion proves to be the bottleneck.

4. How we'll know it's working (November to January)

The main goal is customers who have run at least two payroll cycles. Signing up isn't enough; running payroll repeatedly is what matters.

MetricTarget by end of January
Primary-segment customers running payroll (2+ cycles)220, about 10% of the segment. That's roughly 2,600 employees and about $16,000 in monthly recurring revenue.
Setup started ÷ payroll page visits, primary segment6% or higher (at least matching the beta)
First payroll run ÷ setup startedUp from 44% to 60% or higher, through assisted setup
Time from setup start to first payrollMedian under 7 days
Penalties caused by LedgerlyZero. Every incident gets a root-cause review.
Customers who stop using payroll after their first runUnder 10%
Accounting product cancellations in the primary segmentNo increase compared with before launch

Check-ins: - Weekly: track the funnel by segment. - Week 2: read the headline re-test, if Legal changes the claim. - End of December: go/no-go on the held-back budget and on starting the switching campaign for customers on other payroll tools.

Caveats

  • The beta participants opted in, so real launch conversion will probably be lower than the beta numbers.
  • Penalty rates are self-reported.
  • The 220 target is a starting estimate, not a forecast. We should revise it after the first month of real data.

Grades and run details

Decision model 96 · LLM judge 85
Decision model checks
  • passUses the supplied evidence correctly24%
  • passAddresses the actual decision91%
  • passRespects explicit constraints42%
  • passIdentifies material uncertainty96%
  • partialAvoids unsupported claims38%
  • passProduces the required deliverable91%
  • passPicks the segment with the pain100%
  • passUses the headline test100%
  • passPositioned against the real alternative100%
  • passSells against a pain the buyer feels100%
  • passDiagnoses message or channel first100%
  • passPlans beyond launch week100%
Run
Run
#1
Time to output
39 s
Submitted
5 Oct 2026

Needs more than a quick edit10 of 12 checks passed

The plan assumes, without evidence, that individual penalty recipients can be pinpointed and that accountants are the decision-makers. A PM would need to verify these data and assumptions before acting.

Output

Ledgerly Payroll: Go-to-Market Plan

Core change: Sell payroll to existing customers who have the most to lose from getting it wrong, lead with accuracy and penalty protection, and hold most of the $60k until we have conversion data.

What I'd change in the draft

  • The headline. In the beta, "Never get a payroll penalty again" roughly tripled setup starts (6.4% vs 2.2%) and first payroll runs (2.8% vs 0.9%). But "never" is a promise we can't keep. I'd use claims the product can support, such as "Filing reminders and pre-payroll checks, so fewer penalties," and get legal sign-off before launch.
  • "First Financial Operating System." It's unverifiable and means nothing to a bakery owner. Drop it.
  • $60k on paid ads to new prospects. We haven't proven the product yet, and our 8,000 customers are cheaper to reach and already trust us. Paid spend should follow proven conversion.
  • Success measured by press mentions and launch-week sign-ups. Neither tells us whether customers pay, use payroll, or avoid penalties.

Who we go after, in order

Penalty exposure is concentrated in businesses with 5–25 staff:

  • Accountant-run at 5–25 staff: 23% got a penalty notice last year.
  • Spreadsheet-run at 5–25 staff: 19%.
  • Owner-run at 1–4 staff: 4%.
  • Other-tool users: 5–7%.

About 2,170 customers sit in the two high-penalty 5–25 groups, and roughly 470 of them got a notice last year.

Wave 1 (launch, 3 Nov): 5–25 staff, spreadsheet or accountant-run (~2,170 customers). This is where the penalty message earns its keep. Start with the customers who got a notice, using our survey data, then the rest of the segment.

Wave 2 (mid-November): owner-run at 1–4 staff (3,140 customers). This is the largest pool but has low penalty pain, so the message is time saved. Their Sunday-spreadsheet routine is the pitch.

Wave 3 (January): other-tool users (1,670 customers). Don't ask anyone to switch mid-year (customer 4 calls it "a nightmare"). Pitch migration at the start of the new year, and offer to do the import for them.

Later: accountant-run at 1–4 staff (1,020 customers). The accountant makes this decision, so reach them through the accountant channel below.

Skip payroll messaging for businesses with no payroll pain, such as the three-person design studio (customer 3).

What we say

  • Accountant-run, 5–25 staff: "Payroll filings checked and reminded before they're due. Keep your accountant and see their work in the same place." This addresses customer 2's fear of getting it wrong and customer 1's question about what they're paying for. The "keep your accountant" line is a hypothesis to test.
  • Spreadsheet-run, 5–25 staff: "Stop checking payroll on Sunday night," with penalty protection as the second line.
  • Owner-run, 1–4 staff: "Run payroll in minutes, right where you do your books." This is beta headline A, which is fine for a segment that cares about time, not penalties.
  • Other-tool users: "We'll move your payroll over for you at the start of the new year."

Accuracy claims have to match what the product does. Customer 8 says "If you can promise it's right, I'm in," so the proof needs to be concrete: which checks run, which reminders fire, and what happens when a figure looks wrong.

Channels

  1. In-app, segmented (week 1). Show headline B to the 5–25 segments and headline A to the 1–4 segment. It's free and reaches everyone who logs in.
  2. Targeted email and account-manager calls (weeks 1–2). Email wave 1 from named account managers. Have support call the highest-risk accounts, those who got a notice last year, first.
  3. Accountant pilot (from launch, ~25 firms). Accountants serve many of our customers, and an accountant-run customer decides with their accountant. Invite the firms that manage the most of our customers to run payroll for clients inside Ledgerly. This means we need to decide up front how partners are paid.
  4. Webinar (launch week). "A payroll penalty checklist for businesses with 5–25 staff," invited to wave 1 only. Narrow beats generic.
  5. Paid (hold). Spend $10k on retargeting and search from week 3, but only if the wave 1 landing page converts at least as well as beta headline B. Hold the remaining ~$50k until the 60-day numbers justify it.
  6. Press (low-key). A short release with a real customer quote and no "first" claim. It's cheap and supports the story without carrying the plan.

Measuring the next three months

Primary metric: payroll customers, meaning customers who have run at least one payroll, by segment, and the revenue from them ($6 per employee per month).

Targets (reset after the 30-day baseline): - Wave 1: 10% of the ~2,170 customers have run a payroll by day 90 (~220 customers, roughly $18k MRR at about 12 employees each). - Wave 2: 4% by day 90. - Wave 3: 10% of other-tool users migrated by end of January.

Decision rules: - If wave 1's first-payroll rate is under 2% after 30 days, pause paid spend and rewrite the messaging. - If it's over 5%, release the held budget to the segments that convert best.

Guardrails (any breach stops the rollout): - Zero filing errors or penalties traced to Ledgerly. - Support tickets per payroll run stay within agreed levels. - No rise in churn or refunds among existing customers.

Penalty outcomes won't show in three months. As a proxy, track on-time filing rates for adopters against matched non-adopters, and report it to the board in February.

Claims the judge couldn’t find in the brief

  • We have survey data identifying which specific customers received a penalty notice last year, allowing us to start with those individuals.
  • For accountant-run customers, the accountant makes the buying decision.

Grades and run details

Decision model 100 · LLM judge 77
Decision model checks
  • passUses the supplied evidence correctly10%
  • passAddresses the actual decision98%
  • passRespects explicit constraints30%
  • passIdentifies material uncertainty97%
  • passAvoids unsupported claims33%
  • passProduces the required deliverable69%
  • passPicks the segment with the pain100%
  • passUses the headline test97%
  • passPositioned against the real alternative100%
  • passSells against a pain the buyer feels100%
  • passDiagnoses message or channel first100%
  • passPlans beyond launch week100%
Run
Run
#1
API response time
33 s
Submitted
8 Oct 2026

Check by check

Where the outputs differ comes first. Open a check to read the judge’s reasons. “Mixed” means the two graders disagreed.

Where they differ 6

Uses the supplied evidence correctlyRightRightMixed
Gemini 3.5 Flash-Lite · Gemini

All factual claims about the current situation are directly supported by the brief or supplied context.

Opus 5.5 · Claude

All statements about the current situation are directly supported by the brief or supplied context.

Haiku 5.5 · API

The plan asserts two facts about the current situation not supported by the brief: that individual penalty recipients can be identified from survey data, and that accountants make the payroll purchasing decision.

Addresses the actual decisionMixedRightRight
Gemini 3.5 Flash-Lite · Gemini

The output commits to a clear plan but does not state what result or condition would change that decision.

Opus 5.5 · Claude

Commits early to a primary segment, message, and channels, and states conditions that would change the call (legal approval, re-test, go/no-go).

Haiku 5.5 · API

The output commits to a clear wave plan early and states decision rules (conversion thresholds for budget release and messaging changes).

Respects explicit constraintsRightMixedRight
Gemini 3.5 Flash-Lite · Gemini

The plan respects the 900-word limit, addresses the required elements, and does not lead with 'save time' or make paid ads the main channel.

Opus 5.5 · Claude

The output is approximately 952 words, exceeding the 900-word limit.

Haiku 5.5 · API

The output is a go-to-market plan under 900 words, addresses the requested elements (segments, messaging, channels, measures), and respects the launch date and audience.

Identifies material uncertaintyWrongRightRight
Gemini 3.5 Flash-Lite · Gemini

The output does not name any unknowns that could change the decision or how they would be resolved.

Opus 5.5 · Claude

Names specific unknowns (legal risk, beta opt-in bias, self-reported data) and how they would be resolved or change the plan.

Haiku 5.5 · API

The plan names unknowns (conversion rates, messaging effectiveness) and sets explicit thresholds and dates to resolve them.

Avoids unsupported claimsWrongRightMixed
Gemini 3.5 Flash-Lite · Gemini

It presents 'Acquiring cold payroll customers is expensive and slow; cross-selling our 8,000 warm customers is high-margin' as fact without supporting evidence.

Opus 5.5 · Claude

Hypotheses and forecasts are labelled as such (e.g., 'probably be lower', 'starting estimate, not a forecast').

Haiku 5.5 · API

The plan states that the accountant makes the payroll decision as a fact, and implies precise individual penalty data exists, without labelling these as assumptions or hypotheses.

Produces the required deliverableRightMixedRight
Gemini 3.5 Flash-Lite · Gemini

The output is a complete go-to-market plan in memo form, under 900 words, covering who, what, channels, and measures.

Opus 5.5 · Claude

The plan is present and complete but exceeds the 900-word limit, so it is not fully within the requested length.

Haiku 5.5 · API

The output is a complete, actionable GTM plan in the required format, usable by a PM to execute the launch with light edits.

All got right 6

Picks the segment with the painRightRightRight
Gemini 3.5 Flash-Lite · Gemini

It targets 5-25 employee businesses using accountant/spreadsheet, cites penalty rates, and explains why smaller and tool-using segments are not first.

Opus 5.5 · Claude

Targets 5–25 employee customers using spreadsheet or accountant, with numbers and penalty rates, and explains why smaller and tool-using segments come later.

Haiku 5.5 · API

The plan correctly prioritizes 5-25 employee spreadsheet and accountant-run segments (high penalty rates), explains why 1-4 and other-tool users come later.

Uses the headline testRightRightRight
Gemini 3.5 Flash-Lite · Gemini

It uses the beta test results (38 vs 117 completed payrolls) to choose the penalty-focused message.

Opus 5.5 · Claude

Cites the beta test results and builds the message on penalties and accuracy.

Haiku 5.5 · API

The plan uses the beta headline test results to base the core message on penalty protection for the high-pain segments and time-saving only for the low-pain segment.

Positioned against the real alternativeRightRightRight
Gemini 3.5 Flash-Lite · Gemini

It names the real alternatives (accountant, spreadsheet, other tools) and ties the message to the accuracy and penalty avoidance they lack.

Opus 5.5 · Claude

Names the real alternatives (spreadsheet, accountant, another tool) and ties the message to penalty risk and cost savings.

Haiku 5.5 · API

The plan names current alternatives (accountant, spreadsheet, another tool) and differentiates against the value they lack (accuracy, penalty protection) for each segment.

Sells against a pain the buyer feelsRightRightRight
Gemini 3.5 Flash-Lite · Gemini

The core message is built on fear of penalties and errors, supported by customer calls and the headline test.

Opus 5.5 · Claude

Leads with the pain of penalties and errors, supported by calls and beta data, not generic time-saving.

Haiku 5.5 · API

The lead message for the primary segment is built on penalty fear and accuracy, directly supported by the customer calls and headline test.

Diagnoses message or channel firstRightRightRight
Gemini 3.5 Flash-Lite · Gemini

It diagnoses the problem as message (low conversion on the time headline) and shifts effort to fixing the message and reaching existing customers.

Opus 5.5 · Claude

Diagnoses a message problem (losing headline, existing customers already see the page) and shifts budget from new-prospect ads to in-app, email, and assisted setup.

Haiku 5.5 · API

The plan diagnoses message as the bottleneck (uses in-app test data and existing reach) and defers paid spend until conversion is proven, focusing effort on fixing the message first.

Plans beyond launch weekRightRightRight
Gemini 3.5 Flash-Lite · Gemini

It provides an ongoing motion with in-app, email, and retargeting, and measures success over three months with specific metrics.

Opus 5.5 · Claude

Plans ongoing monthly webinars, three-month metrics, weekly check-ins, and a December go/no-go, not just launch week.

Haiku 5.5 · API

The plan includes a multi-wave rollout, ongoing channels (email, accountant pilot, retargeting), and 90-day metrics with decision rules beyond launch week.

Results

Every setup we’ve tested on this task type, across all its tasks and repeats, graded on the current checklist. Calibrated.

#Model · HarnessTask scoreDecision modelLLM judgeRunsCritical failures
1GPT-6 AstrawithChatGPT100.0100.02None
2Opus 5.5withClaude96.085.22None
3GPT-6 LunawithAPI94.289.32None
4GPT-6.1 SolwithAPI93.878.02None
5Sonnet 5.5withAPI98.177.72None
6Haiku 5.5withAPI94.277.72None
7Gemini 3.8 FlashwithAPI73.981.92None
8Gemini 3.5 Flash-LitewithGemini74.263.52None

What you get for the cost

No setup scores within 5 points of the leader at half its cost or less here, so the leader is also the best value.

Task score against relative cost for each setup. The table below has the same figures.607080901001×3×10×30×100×Cost per output, relative to the cheapest setup (log scale) →↑ Task scoreGPT-6 Astra · ChatGPT: task score 100, 81× the cheapestOpus 5.5 · Claude: task score 91, 75× the cheapestGPT-6 Luna · API: task score 90, the cheapestGPT-6.1 Sol · API: task score 84, 16× the cheapestSonnet 5.5 · API: task score 84, 46× the cheapestHaiku 5.5 · API: task score 82, 3.3× the cheapestGemini 3.8 Flash · API: task score 77, 11× the cheapestGemini 3.5 Flash-Lite · Gemini: task score 68, 3.5× the cheapestGPT-6 Astra · ChatGPTOpus 5.5 · ClaudeGPT-6 Luna · APIGPT-6.1 Sol · APISonnet 5.5 · APIHaiku 5.5 · APIGemini 3.8 Flash · APIGemini 3.5 Flash-Lite · Gemini
Nobody beats these on both score and cost
Task score and relative cost by setup
SetupTask scoreRelative cost (cheapest = 1)
GPT-6 Luna · API901.0×
Haiku 5.5 · API823.3×
Gemini 3.5 Flash-Lite · Gemini683.5×
Gemini 3.8 Flash · API7711×
GPT-6.1 Sol · API8416×
Sonnet 5.5 · API8446×
Opus 5.5 · Claude9175×
GPT-6 Astra · ChatGPT10081×

Cost is each output’s tokens at the model’s list price, shown relative to the cheapest setup. Runs done in an app have no usage meter, so theirs are estimated from the same setup’s measured runs. How we compare cost

About the task

The PM job

Taking a product or add-on to market with marketing and sales.

Why it matters

Launch plans fail by trying to reach everyone, by selling upside nobody feels, and by treating launch day as the whole plan.

What good looks like

  • One primary segment: those who value the difference most
  • Positions against what customers do today
  • Sells against a pain the buyer feels, with evidence
  • Diagnoses whether the problem is the message or the channel
  • Plans beyond launch week

Deliberately not measured

  • Creative copywriting quality
Capability tested

Go-to-market judgement

The failure we’re looking for

Every channel, every segment, and nothing after launch day

Grading

Decision model and LLM judge, calibrated against a blind PM review