Tasks / Growth

Write a GTM plan

Can the model pick the customers who value the difference most, position against what they do today, and build a go-to-market that lasts beyond launch day?

Task type v1.1 · 2 tasksLast changed 6 Oct 2026 · ChangelogDifficulty

What AI gets right here, and what you’ll still have to catch

From 18 graded outputs by 9 models. 44% were usable with at most a quick edit.

Reliably right

  1. Positioned against the real alternative100% pass
    Names Miro, photos/slides, and doing nothing as alternatives, and ties the message to real-time tidy diagramming and starting without IT.
    GPT-6 Astra · ChatGPT · Enterprise, without killing the engine
  2. Sells against a pain the buyer feels100% pass
    The message addresses the governance and control pain that IT buyers feel when teams adopt tools without oversight, supported by SSO requests and security-page visits in the data.
    GPT-6 Astra · ChatGPT · Enterprise, without killing the engine
  3. Protects the self-serve engine100% pass
    Uses the 70% free-to-paid stat and product-qualified deal data to argue against capping the free plan, and keeps self-serve open.
    GPT-6 Astra · ChatGPT · Enterprise, without killing the engine

Where it slips

  1. Uses the supplied evidence correctly28% pass
    The claim 'Five of our six losses on existing-user deals ended in no decision' is not supported by the data (4 no-decision, 2 Miro), so a statement about the current situation is invented.
    Opus 5.5 · Claude · Enterprise, without killing the engine
  2. Avoids unsupported claims39% pass
    Presents the incorrect 'five of six no decision' figure as established fact, which is an unsupported claim about the current situation.
    Opus 5.5 · Claude · Enterprise, without killing the engine
  3. Follows the deals data67% pass
    Although it compares by source, the computed product-qualified ACV and sales cycle are wrong, so it does not correctly follow the deals data.
    GPT-6.1 Sol · API · Enterprise, without killing the engine

How it’s graded

Each check is built on how world-class product and growth leaders do this work, as they’ve described it on Lenny’s Podcast, and names the leader it comes from.

  1. Positioned against the real alternative

    Does the plan name what the target customer does today without this product (including the status quo or doing nothing), and tie the message to the value that alternative lacks?

    Passes when Names the real alternative and the differentiated value against it, and picks the segment that cares most about that value.

  2. Sells against a pain the buyer feels

    Is the core message built on a pain or risk the target buyer already feels, with evidence from the material, rather than on generic upside like saving time?

    Passes when Leads with the pain the evidence shows buyers care about, and uses that evidence to choose the message.

  3. Diagnoses message or channel first

    Does the plan work out whether the problem is the message (people reach it but don't convert) or the channels (the right people never see it), and spend accordingly?

    Passes when Uses conversion and reach data to say which it is, and puts the effort there.

  4. Plans beyond launch week

    Does the plan say how demand keeps coming after launch, with the ongoing engine and what's measured over months, rather than treating launch day as the whole plan?

    Passes when Gives the ongoing motion after launch and the measures that show whether it's working over time.

Plus our standard checks

Uses the supplied evidence correctly · Addresses the actual decision · Respects explicit constraints · Identifies material uncertainty · Avoids unsupported claims · Produces the required deliverable · and 2 written for each task, which you’ll see in the tasks below.

The tasks

Read the brief, then compare up to three outputs, each with the LLM judge’s verdict on every check. Highlights mark what a PM had to fix.

The brief

You're the PM for Ledgerly Payroll, a new add-on for our accounting product, launching to existing customers on 3 November. Marketing has drafted the launch plan below. Write the go-to-market plan you'd run instead (or the changes you'd make to theirs), in no more than 900 words: who we go after first, what we say to them, through which channels, and how we'll know it's working over the following three months. The files are attached.

What the model was given5 items: About Ledgerly, marketing_launch_draft.md, customers_by_size_and_payroll_method.csv (penalty notices: share who told us they got a payroll penalty notice last year), beta_headline_test.csv (two versions of the in-app payroll page, shown at random to beta customers), customer_calls.md
About LedgerlyAccounting software for small businesses: invoicing, bank feeds, bookkeeping. 8,000 paying customers. Payroll is a new add-on at $6 per employee per month.
marketing_launch_draft.md7 lines · Download
# Draft launch plan (Head of Marketing)

- Launch day 3 November: press release announcing Ledgerly as "the first Financial Operating System for small business".
- Paid social and search ads to new prospects: $60,000 in November.
- In-app banner to all 8,000 customers: "Run payroll in minutes".
- Webinar in launch week.
- Success measure: press mentions and sign-ups in launch week.
customers_by_size_and_payroll_method.csv (penalty notices: share who told us they got a payroll penalty notice last year)employees,current_payroll,customers,penalty_notices_last_year_pct 1-4,Owner does it in a spreadsheet,3140,4 1-4,Accountant runs it,1020,6 5-25,Owner does it in a spreadsheet,690,19 5-25,Accountant runs it,1480,23 5-25,Another payroll tool,1210,7 26-50,Another payroll tool,460,5
beta_headline_test.csv (two versions of the in-app payroll page, shown at random to beta customers)headline,landing_page_visits,started_setup,ran_first_payroll "Run payroll in minutes, right where you do your books",4210,92,38 Never get a payroll penalty again,4160,268,117
customer_calls.md10 lines · Download
# Customer calls about payroll (excerpts, September)

1. Bakery, 12 staff: "My accountant does payroll. Last spring they filed late and I got a $1,400 penalty. I don't know what I'm paying them for."
2. Plumbing firm, 18 staff: "Time isn't the issue. I'm scared of getting it wrong. The tax office letter was the worst week of my year."
3. Design studio, 3 staff: "It's me and two freelancers. A spreadsheet's fine."
4. Café group, 22 staff: "We use another tool, it's OK. Switching sounds like a nightmare mid-year."
5. Physio clinic, 9 staff: "I'd move tomorrow if it meant one less thing that can go wrong with the tax office."
…
What a strong answer doesThe answer key the graders mark against

Picks customers with 5 to 25 employees whose accountant runs payroll (1,480 customers, 23% had a penalty notice) and those doing it in a spreadsheet (690, 19%), not the smallest businesses (happy with a spreadsheet) or those on another payroll tool (switching mid-year is the barrier). Builds the message on fear of penalties and errors, not time: in the beta, 'Never get a payroll penalty again' got about three times as many setups as 'Run payroll in minutes' (268 against 92 from similar traffic, 6.4% against 2.2%) and more first payrolls (117 against 38), and the calls say the same. Reads this as a message problem, not a channel problem: the existing customer base already sees the page, so fix the message and reach existing customers in-app and through their accountants before spending $60,000 on ads to strangers. Drops the 'Financial Operating System' category claim. Plans beyond launch week: prompts around payroll and tax deadlines, accountant referrals, and monthly measures such as payroll attach in the target segment, first payrolls run, and penalties avoided.

Critical failures (cap the score at 40)
  • Keeps 'Run payroll in minutes' as the lead message despite the headline test
Task

Mid-level · v1.1

The outputs

9 setups answered this brief.

Needs more than a quick edit9 of 12 checks passed

The unsupported claim about January 1 switching ease would require the PM to verify tax-year rules before acting on the timing plan.

Output

Ledgerly Payroll: Revised GTM Plan (Nov 2023 – Jan 2024)

Why We Need to Pivot from the Draft Plan

Our mandate is launching an add-on to existing customers, yet the draft plan treats this like a net-new customer acquisition campaign. Based on our customer research and beta data, we need to make four critical changes:

  1. Cut the $60k ad spend and press release: Spending $60k on cold prospects when we have 8,000 warm customers is inefficient, and "Financial Operating System" is industry jargon that doesn't resonate with SMBs. Let’s reallocate that budget to customer onboarding and switching incentives.
  2. Stop selling speed; start selling accuracy: Our beta test proved that "Run payroll in minutes" converts poorly (0.9% visit-to-payroll). Customers aren’t worried about time; they are terrified of tax office penalties. "Never get a payroll penalty again" tripled our conversion rate to 2.8%.
  3. Segment the audience: Half of our customers have 1–4 employees, rarely get penalties, and are happy using free spreadsheets. Blasting all 8,000 customers equally will create noise and yield low-value accounts.
  4. Align timing with the tax calendar: Launching in November hits the "mid-year switching nightmare" objection. However, January 1 is the Clean-Slate Window—the easiest time of year to switch payroll providers because year-to-date tax balances reset. November should be about selling and onboarding for a January 1 first payroll run, alongside immediate switching for those in high pain.

---

1. Who We Go After First (Target Segments)

At $6/employee/month, our revenue concentrates in the 5–25+ employee range. We will prioritize based on pain (penalty rates) and LTV:

  • Tier 1 (Primary Target): 5–25 Employees on Accountants (1,480 customers) or Spreadsheets (690 customers).
  • Why: Highest pain and highest willingness to switch. 23% of the accountant group and 19% of the spreadsheet group got hit with a penalty last year. Furthermore, accountants charge ~$150–$200/mo for this; at $6/employee, a 12-person firm pays us just $72/mo. We are cheaper and more reliable.
  • Tier 2 (The January 1 Play): 5–50 Employees on Another Payroll Tool (1,670 customers).
  • Why: High revenue potential ($30–$300/mo per account), low current penalty pain (5–7%), and high friction to switch mid-year. We won't ask them to switch in November; we will book them now to switch for Q1 (Jan 1).
  • Tier 3 (Deprioritized / Self-Serve): 1–4 Employees (4,160 customers).
  • Why: Low pain (4–6% penalty rate), high price sensitivity ("spreadsheets are free"), and low ARPU ($6–$24/mo). They will see passive in-app navigation, not active campaigns.

---

2. What We Say (Messaging & Positioning)

Core Value Proposition: Never get a payroll penalty again. Accurate, automated payroll built directly into the books you already trust.

  • To Tier 1 (Accountant Users): "Stop paying your accountant $150+/month to file your taxes late." Highlight automatic tax filing, error-checking against their existing Ledgerly books, and cost savings.
  • To Tier 1 (Spreadsheet Users): "Take the tax-office anxiety out of Sunday nights." Emphasize peace of mind, automatic tax table updates, and eliminating manual entry errors.
  • To Tier 2 (Competitor Tool Users): "Start the new year with your books and payroll in one place—we’ll handle the January 1 switch for free." Overcome switching inertia by offering concierge migration for the new tax year.

---

3. Channels & Phased Rollout

Instead of a single "launch day" splash, we will run a 3-month campaign leveraging our unfair advantage: product data from our existing bookkeeping software.

Phase 1: High-Pain Capture (Nov 3 – Nov 30)

  • Contextual In-App Triggers (Tier 1): Using bank feed and bookkeeping data, trigger targeted prompts when users categorize transactions to "Payroll," "Tax Penalty," or "Accountant Fees."
  • Direct Email & Outbound Sales: Reallocate a portion of the $60k marketing budget to temporary onboarding specialists. Call and email the 2,170 Tier 1 customers offering a "Penalty-Free Guarantee" and free setup.
  • Customer Workshop (Replacing the generic webinar): Host "How to Fire Your Payroll Accountant (Without Making the Tax Office Mad)"—focused on compliance and how easy it is to transition.

Phase 2: The "New Year, Clean Books" Campaign (Dec 1 – Jan 15)

  • Concierge Migration Offer (Tier 1 & Tier 2): Launch a campaign throughout December: "Lock in your Jan 1 payroll switch before Dec 20, and our team will migrate your employee data from your old tool/accountant for free."
  • In-App Banner (All 5+ employee accounts): "Getting ready for year-end? Move payroll to Ledgerly for January and never get a penalty notice again."

---

4. How We’ll Know It’s Working (3-Month Metrics)

Press mentions and launch-week sign-ups are vanity metrics; our beta showed that 56% of users who start setup drop off before running payroll. In payroll, a customer isn't real until they run their first pay cycle.

We will track success from Nov 3 to Feb 3 across three funnel stages:

1. Activation (Leading Indicators - Tracked Weekly): * Landing Page to Setup Started: Target >6.0% (matching our winning beta test of 6.4%). * Setup Started to First Payroll Run: Target >50% (improving on beta's 43.6% via concierge onboarding). * "Jan 1 Switch" Reservations: Target 150+ accounts booked in Nov/Dec for a January start. 2. Adoption & Revenue (Lagging Indicators - Tracked Monthly): * Active Payroll Customers (ran ≥1 payroll by Jan 31): Target 250 customers (~11.5% of Tier 1, or ~6.5% of Tiers 1 & 2 combined). * Covered Employees & MRR: Target 3,000 employees paid through Ledgerly by Jan 31 (avg. 12 employees/account), generating $18,000 in new MRR ($216k ARR). 3. Product Quality & Trust (The Ultimate Goal): * Zero tax calculation/filing errors during the critical January year-end/Q1 filing period. * Payroll Retention: >95% of customers who run a first payroll run their second payroll.

Claims the judge couldn’t find in the brief

  • January 1 is the Clean-Slate Window—the easiest time of year to switch payroll providers because year-to-date tax balances reset.

Grades and run details

Decision model 75 · LLM judge 69
Decision model checks
  • failUses the supplied evidence correctly95%
  • passAddresses the actual decision52%
  • passRespects explicit constraints30%
  • failIdentifies material uncertainty67%
  • failAvoids unsupported claims80%
  • passProduces the required deliverable76%
  • passPicks the segment with the pain100%
  • passUses the headline test95%
  • passPositioned against the real alternative100%
  • passSells against a pain the buyer feels100%
  • passDiagnoses message or channel first95%
  • passPlans beyond launch week100%
Run
Run
#1
API response time
17 s
Submitted
13 Oct 2026

Usable with a quick edit8 of 12 checks passed

The plan is clear and actionable, with only a minor unsupported claim about acquisition cost that can be edited out; no re-checking of evidence is needed.

Output

To: Executive Team & Head of Marketing From: PM, Ledgerly Payroll Date: October 18 Subject: Revised GTM Plan: Ledgerly Payroll Launch (3 November)

The draft marketing plan targets a generic market with high-level branding. That’s a mistake. We are launching an add-on to an existing base of 8,000 accounting customers. Our unfair advantage isn't that we're a "Financial Operating System"—it's that we already hold their financial data.

We need to scrap the broad acquisition play for November, refocus entirely on our existing base, and lead with our strongest asset: penalty anxiety and accuracy, not speed.

Here are the critical changes I am making to our launch plan.

---

1. Who We Go After First (The Wedge)

The data and customer calls tell a clear story.

  • Ignore: The 1-4 employee segment doing payroll in spreadsheets (Retail shop: "Annoying, but free"). They don't feel enough pain to switch or pay $6/mo.
  • Ignore: Mid-year switchers currently using dedicated payroll tools (Café group: "Switching sounds like a nightmare").
  • Target (Our Wedge): Small businesses with 5–25 employees who currently rely on an accountant or a spreadsheet, and are bleeding money/stress on penalties.

Look at the data: 5–25 employee businesses using accountants or spreadsheets suffer 19% to 23% penalty rates annually. Furthermore, customer calls highlight deep resentment toward accountant-driven payroll failures ($1,400 fines, $180/mo fees for mistakes).

We will segment our 8,000 customers on Day 1 and suppress micro-businesses (1-4) from our primary payroll marketing to protect email reputation and conversion relevance.

2. What We Say to Them (The Message)

Marketing’s draft leads with: "Run payroll in minutes."

Our beta test proved this is the wrong hook. Look at the `betaheadlinetest.csv` data: * "Run payroll in minutes..." generated only 38 completed payrolls. * "Never get a payroll penalty again" generated 117 completed payrolls (more than 3x the activation rate).

Time isn't the primary enemy; fear is (Plumbing firm: "I'm scared of getting it wrong. The tax office letter was the worst week of my year").

Our Core Value Proposition: “Keep payroll inside Ledgerly. Zero manual data entry from your books, guaranteed accurate tax calculations, or we cover the penalty.” (Note: We should seriously explore a penalty-reimbursement guarantee as a killer differentiator).

3. Through Which Channels

We are shifting $50,000 of the proposed November budget away from broad paid social/search targeting cold prospects. Acquiring cold payroll customers is expensive and slow; cross-selling our 8,000 warm customers is high-margin.

  • Channel 1: Targeted In-App Real Estate (Nov 3): Replace the generic banner with a dynamic state. If the user has 5–25 employees and uses an accountant/spreadsheet, show the "Never get a payroll penalty again" headline.
  • Channel 2: Founder/PM Email Campaign (Nov 4–10): A direct, non-marketing-sounding email sent from me (PM) and the CEO to the 5–25 segment. Focus on the hidden costs of accountant errors and spreadsheet slip-ups. Include a 1-click "Import my data" preview.
  • Channel 3: The "Fire Your Accountant's Payroll Bill" Playbook: A targeted retargeting campaign (using the remaining $10k of paid budget) aimed strictly at custom matched lists of our 5-25 employee segment, highlighting how much they can save by bringing payroll into Ledgerly alongside their bookkeeping.

4. How We’ll Measure Success (Months 1–3)

Marketing’s draft relies on vanity metrics: press mentions and launch-week sign-ups. We care about retention and activation. Ledgerly Payroll is priced at $6/employee/month; a sign-up that doesn’t run payroll is worthless.

Over the next three months, we will track four strict OKRs:

  • Month 1 (Launch & Activation):
  • Metric: 500 active payroll accounts set up from the 5–25 employee segment.
  • Guardrail Metric: Zero critical calculation support tickets in the first 14 days. Accuracy is our brand promise; a buggy launch kills trust permanently.
  • Month 2 (Expansion & Accountant Displacement):
  • Metric: 30% of activated users migrating away from their external accountant/spreadsheet for payroll.
  • Metric: Average team size of adopting accounts sits solidly in the 8–15 employee range (validating our wedge).
  • Month 3 (Monetization & Retention):
  • Metric: $15,000 in recurring monthly add-on revenue (MRR).
  • Metric: 95% month-two retention for payroll runners (proving they didn't just test it for a month, but integrated it into their operations).

Summary of Changes

We are abandoning the "Financial Operating System" fluff and the broad acquisition spend. On November 3, we laser-focus on our 5–25 employee segment with a message built on fear-relief and accuracy, driving them straight into a high-converting in-app flow.

Grades and run details

Decision model 71 · LLM judge 77
Decision model checks
  • failUses the supplied evidence correctly23%
  • passAddresses the actual decision59%
  • partialRespects explicit constraints19%
  • failIdentifies material uncertainty38%
  • failAvoids unsupported claims51%
  • passProduces the required deliverable61%
  • passPicks the segment with the pain94%
  • passUses the headline test100%
  • passPositioned against the real alternative100%
  • passSells against a pain the buyer feels100%
  • passDiagnoses message or channel first97%
  • passPlans beyond launch week94%
Run
Run
#1
Time to output
6 s
Submitted
5 Oct 2026

Needs more than a quick edit10 of 12 checks passed

The output incorrectly states that about half the base is a poor fit, which would require re-checking the data to correct; otherwise the plan is solid but that error undermines trust.

Output

Ledgerly Payroll: go-to-market plan

What I'd change in Marketing's draft

DraftProblemChange
$60k paid ads to new prospectsPayroll is an add-on, so a prospect would first have to adopt our accounting product. We have 8,000 customers and haven't yet proven the message with them.Hold most of the budget until the message converts in-base (see Budget).
Banner to all 8,000About half the base is a poor fit (details below).Target about 2,170 customers first.
"Financial Operating System" press releaseCustomers never use this language. They talk about fear of penalties and the tax office.Keep the press small and plain.
Success = press mentions and launch-week sign-upsNeither measures payroll actually being run.Measure first payroll run and repeat runs.

Who we go after first

Wave 1 (3 Nov): customers with 5–25 employees who run payroll via spreadsheet (690) or an accountant (1,480). That is 2,170 customers, 27% of the base.

  • They have the pain. 19% (spreadsheet) and 23% (accountant) got a payroll penalty notice last year, versus 4–6% for businesses with 1–4 staff and 5–7% for those already on a payroll tool.
  • They have the money. At $6 per employee, a 12-person business pays about $72 a month, against the $180 a month one landscaper pays their accountant.
  • The calls match the data. The bakery, plumber, physio, landscaper and dental practice all describe fear of mistakes, not lack of time.

Not now: - 1–4 employees (4,160 customers): The spreadsheet is "fine" and "free," and only 4–6% were penalized. Leave payroll available in the product but don't push it. - Other-tool users (1,670 customers): Penalty rates are low (5–7%), and switching mid-year "sounds like a nightmare." Revisit at the start of the next tax year with a migration offer.

What we say

The beta test was randomized. "Never get a payroll penalty again" beat "Run payroll in minutes" on setup starts (6.4% vs 2.2% of visits) and on first payroll runs (2.8% vs 0.9%), about 3x on both. Once people started, completion was similar (44% vs 41%), so the headline drives the decision to begin and the product handles the rest.

Core message: "Payroll filed right and on time, so the tax office letter never arrives."

  • Spreadsheet owners: "A spreadsheet won't tell you when you've got it wrong."
  • Accountant-run: "Stop paying $180 a month to worry about it." Use this to speak to cost and control without attacking accountants. Offer a way to invite their accountant in as a reviewer.

Accuracy promise: The dental practice said, "If you can promise it's right, I'm in." I'd like to offer a penalty-cover guarantee for errors caused by our software. Legal and Finance must approve it by 24 Oct. If they won't, the copy becomes "filed right and on time, with support to fix problems." We should not promise "never" without backing it. The beta headline did well, but a claim we can't stand behind creates risk.

Channels

  1. Targeted in-app card and email to the Wave 1 list only, using the winning headline. This is the main channel, and it costs almost nothing.
  2. Personal outreach from Customer Success to Wave 1 customers who told us they got a penalty notice (about 470 people by my estimate from the survey rates). Offer a 20-minute guided setup.
  3. Webinar, reframed: "Penalty-proof your payroll" with a live setup walkthrough. Run it in launch week and again in week 4, since the second session catches people who weren't ready the first time.
  4. Press: one plain release ("Ledgerly adds payroll, $6 per employee"), with no category claim. Follow with customer stories once we have them.
  5. Paid: none to new prospects in November (see Budget).

Budget (the $60k)

  • ~$15k: guided setup and migration help (contractor or CS time).
  • ~$10k: webinars, email production, and case-study content.
  • ~$35k: held until the January gate. If Wave 1 works, spend it on search ads for "payroll penalty" intent, aimed at businesses that would adopt accounting and payroll together.

How we'll know it's working (3 months)

Primary metric: paying accounts that have run at least two payrolls. This replaces sign-ups, press mentions and launch-week anything.

Targets (to be recalibrated after week 2, since the beta is my only baseline): - Month 1: at least 5% of Wave 1 has started setup. At least 45% of those run a first payroll, matching beta's 44%. - End of January: about 250 Wave 1 accounts running payroll (~11% of the segment). I'm assuming a 10-person average, which implies roughly $15k MRR. I'll replace that assumption with real headcount data. - Retention: at least 85% of those accounts run payroll in three consecutive cycles.

Guardrails (any breach pauses expansion): - Payroll errors or penalties caused by us: zero tolerance, reviewed weekly. - Support tickets per payroll run. - Churn of the core accounting subscription, which must not rise among adopters.

Measuring real impact: Hold out a random 10% of Wave 1 from all promotion. If adopters in the promoted group don't clearly exceed the holdout, we're only capturing people who would have found payroll anyway.

Decision gates: - Mid-December: If setup-to-first-payroll is below 35%, fix onboarding before pushing further. - Mid-January: If Wave 1 is at or above target and no guardrail is breached, release the held budget and start Wave 2 (other-tool users, ahead of the tax-year switch window). If it's below target, interview non-adopters before spending anything.

Caveats

  • Penalty rates are self-reported, and the eight calls are anecdotes.
  • Beta customers may be more enthusiastic than the base.
  • I haven't seen average headcount per segment, so the revenue estimates are rough.

Claims the judge couldn’t find in the brief

  • About half the base is a poor fit

Grades and run details

Decision model 88 · LLM judge 77
Decision model checks
  • failUses the supplied evidence correctly31%
  • passAddresses the actual decision98%
  • passRespects explicit constraints46%
  • passIdentifies material uncertainty98%
  • partialAvoids unsupported claims25%
  • passProduces the required deliverable73%
  • passPicks the segment with the pain100%
  • passUses the headline test100%
  • passPositioned against the real alternative100%
  • passSells against a pain the buyer feels100%
  • passDiagnoses message or channel first100%
  • passPlans beyond launch week100%
Run
Run
#1
API response time
37 s
Submitted
6 Oct 2026

Check by check

Where the outputs differ comes first. Open a check to read the judge’s reasons. “Mixed” means the two graders disagreed.

Where they differ 3

Uses the supplied evidence correctlyWrongMixedWrong
Gemini 4 Argon · API

The claim that January 1 is the easiest time to switch because tax balances reset is not in the supplied context and is presented as fact about the situation.

Gemini 3.5 Flash-Lite · Gemini

All factual claims about the current situation are directly supported by the brief or supplied context.

Sonnet 5.5 · API

The claim 'About half the base is a poor fit' is not supported by the data; the target segment is 27%, not half.

Addresses the actual decisionRightMixedRight
Gemini 4 Argon · API

The output unambiguously commits to a ranked target segment, message, channels and measures, framed as a GTM plan for the reader.

Gemini 3.5 Flash-Lite · Gemini

The output commits to a clear plan but does not state what result or condition would change that decision.

Sonnet 5.5 · API

Commits to Wave 1 segment, message, channels, and decision gates with clear conditions for changing course.

Identifies material uncertaintyWrongWrongRight
Gemini 4 Argon · API

The output does not name specific unknowns that could change the plan or say how they would be resolved.

Gemini 3.5 Flash-Lite · Gemini

The output does not name any unknowns that could change the decision or how they would be resolved.

Sonnet 5.5 · API

Names self-reported penalty data, anecdotal calls, beta enthusiasm bias, missing headcount data, and legal approval for guarantee, with resolution steps.

All got wrong 1

Avoids unsupported claimsWrongWrongWrong
Gemini 4 Argon · API

It presents the January 1 clean-slate tax reset claim as an established fact with no source, rather than as a hypothesis.

Gemini 3.5 Flash-Lite · Gemini

It presents 'Acquiring cold payroll customers is expensive and slow; cross-selling our 8,000 warm customers is high-margin' as fact without supporting evidence.

Sonnet 5.5 · API

Presents 'About half the base is a poor fit' as fact when the data shows 73% are not in the target, not half.

All got right 8

Respects explicit constraintsRightRightRight
Gemini 4 Argon · API

The output stays within the word limit, addresses all requested parts, and respects the focus on existing customers.

Gemini 3.5 Flash-Lite · Gemini

The plan respects the 900-word limit, addresses the required elements, and does not lead with 'save time' or make paid ads the main channel.

Sonnet 5.5 · API

Delivers a go-to-market plan under 900 words, addressing the required elements, and respects the reader and form.

Produces the required deliverableRightRightRight
Gemini 4 Argon · API

The GTM plan is complete, under 900 words, with segments, message, channels, and 3-month metrics, usable with light edits.

Gemini 3.5 Flash-Lite · Gemini

The output is a complete go-to-market plan in memo form, under 900 words, covering who, what, channels, and measures.

Sonnet 5.5 · API

Complete plan with changes, segment, message, channels, measures, budget, and caveats; usable as-is with light edits.

Picks the segment with the painRightRightRight
Gemini 4 Argon · API

It targets 5-25 employees on accountant or spreadsheet payroll using the penalty data, and explains why smaller businesses and tool users are not first.

Gemini 3.5 Flash-Lite · Gemini

It targets 5-25 employee businesses using accountant/spreadsheet, cites penalty rates, and explains why smaller and tool-using segments are not first.

Sonnet 5.5 · API

Targets 5-25 employee customers with accountant or spreadsheet payroll, cites penalty rates, and explains why 1-4 and other-tool users are not first.

Uses the headline testRightRightRight
Gemini 4 Argon · API

It cites the beta headline test results and builds the core message on 'Never get a payroll penalty again'.

Gemini 3.5 Flash-Lite · Gemini

It uses the beta test results (38 vs 117 completed payrolls) to choose the penalty-focused message.

Sonnet 5.5 · API

Uses the beta headline test results (3x setup lift) to build the message on penalties and accuracy.

Positioned against the real alternativeRightRightRight
Gemini 4 Argon · API

The plan names each current alternative (spreadsheet, accountant, another tool) and ties the value to reliability and penalty avoidance.

Gemini 3.5 Flash-Lite · Gemini

It names the real alternatives (accountant, spreadsheet, other tools) and ties the message to the accuracy and penalty avoidance they lack.

Sonnet 5.5 · API

Names the real alternatives (spreadsheet, accountant, other tool) and ties message to penalty avoidance and accuracy, targeting the segment that cares most.

Sells against a pain the buyer feelsRightRightRight
Gemini 4 Argon · API

The core message leads with penalty and error fear, supported by customer calls and the headline test.

Gemini 3.5 Flash-Lite · Gemini

The core message is built on fear of penalties and errors, supported by customer calls and the headline test.

Sonnet 5.5 · API

Leads with fear of penalties and tax office letters, supported by call quotes and headline test, not generic time-saving.

Diagnoses message or channel firstRightRightRight
Gemini 4 Argon · API

It identifies the headline test as proving a message problem, not a channel problem, and reallocates budget from paid ads to in-app and email.

Gemini 3.5 Flash-Lite · Gemini

It diagnoses the problem as message (low conversion on the time headline) and shifts effort to fixing the message and reaching existing customers.

Sonnet 5.5 · API

Diagnoses message as the problem (beta conversion data) and focuses on in-app/email to existing base, holding paid ads.

Plans beyond launch weekRightRightRight
Gemini 4 Argon · API

The plan includes a phased December-January campaign and monthly metrics like MRR, retention, and penalty-free filings.

Gemini 3.5 Flash-Lite · Gemini

It provides an ongoing motion with in-app, email, and retargeting, and measures success over three months with specific metrics.

Sonnet 5.5 · API

Plans ongoing motion with second webinar, monthly measures, retention, guardrails, holdout group, and Wave 2 decision gate.

Results

Every setup we’ve tested on this task type, across all its tasks and repeats, graded on the current checklist. Calibrated.

#Model · HarnessTask scoreDecision modelLLM judgeRunsCritical failures
1GPT-6 AstrawithChatGPT95.8100.02None
2Opus 5.5withClaude94.185.22None
3GPT-6 LunawithAPI88.589.32None
4GPT-6.1 SolwithAPI93.878.02None
5Sonnet 5.5withAPI89.977.72None
6Haiku 5.5withAPI88.077.72None
7Gemini 3.8 FlashwithAPI72.181.92None
8Gemini 4 ArgonwithAPI74.070.32None
9Gemini 3.5 Flash-LitewithGemini70.063.52None

What you get for the cost

No setup scores within 5 points of the leader at half its cost or less here, so the leader is also the best value.

Task score against relative cost for each setup. The table below has the same figures.607080901001×3×10×30×100×Cost per output, relative to the cheapest setup (log scale) →↑ Task scoreGPT-6 Astra · ChatGPT: task score 98, 81× the cheapestOpus 5.5 · Claude: task score 90, 75× the cheapestGPT-6 Luna · API: task score 88, the cheapestGPT-6.1 Sol · API: task score 84, 16× the cheapestSonnet 5.5 · API: task score 80, 46× the cheapestHaiku 5.5 · API: task score 80, 3.3× the cheapestGemini 3.8 Flash · API: task score 76, 11× the cheapestGemini 4 Argon · API: task score 69, 18× the cheapestGemini 3.5 Flash-Lite · Gemini: task score 66, 3.5× the cheapestGPT-6 Astra · ChatGPTOpus 5.5 · ClaudeGPT-6 Luna · APIGPT-6.1 Sol · APISonnet 5.5 · APIHaiku 5.5 · APIGemini 3.8 Flash · APIGemini 4 Argon · APIGemini 3.5 Flash-Lite · Gemini
Nobody beats these on both score and cost
Task score and relative cost by setup
SetupTask scoreRelative cost (cheapest = 1)
GPT-6 Luna · API881.0×
Haiku 5.5 · API803.3×
Gemini 3.5 Flash-Lite · Gemini663.5×
Gemini 3.8 Flash · API7611×
GPT-6.1 Sol · API8416×
Gemini 4 Argon · API6918×
Sonnet 5.5 · API8046×
Opus 5.5 · Claude9075×
GPT-6 Astra · ChatGPT9881×

Cost is each output’s tokens at the model’s list price, shown relative to the cheapest setup. Runs done in an app have no usage meter, so theirs are estimated from the same setup’s measured runs. How we compare cost

About the task

The PM job

Taking a product or add-on to market with marketing and sales.

Why it matters

Launch plans fail by trying to reach everyone, by selling upside nobody feels, and by treating launch day as the whole plan.

What good looks like

  • One primary segment: those who value the difference most
  • Positions against what customers do today
  • Sells against a pain the buyer feels, with evidence
  • Diagnoses whether the problem is the message or the channel
  • Plans beyond launch week

Deliberately not measured

  • Creative copywriting quality
Capability tested

Go-to-market judgement

The failure we’re looking for

Every channel, every segment, and nothing after launch day

Grading

Decision model and LLM judge, calibrated against a blind PM review