Tasks / Growth

Write a GTM plan

Can the model pick the customers who value the difference most, position against what they do today, and build a go-to-market that lasts beyond launch day?

Task type v1.1 · 2 tasksLast changed 6 Oct 2026 · ChangelogDifficulty

What AI gets right here, and what you’ll still have to catch

From 16 graded outputs by 8 models. 50% were usable with at most a quick edit.

Reliably right

  1. Positioned against the real alternative100% pass
    It names the real alternatives (accountant-run payroll with mistakes, owner spreadsheet with risk) and ties the message to confidence and accuracy that those alternatives lack.
    GPT-6 Astra · ChatGPT · Payroll, for the people who fear it
  2. Protects the self-serve engine100% pass
    Uses the 70% free-to-paid stat and product-qualified deal data to argue against capping the free plan, and keeps self-serve open.
    GPT-6 Astra · ChatGPT · Enterprise, without killing the engine
  3. Does the capacity sums100% pass
    Works out deals needed (69-80), qualified opportunities (134), and AE-years (8) with checkable arithmetic.
    GPT-6 Astra · ChatGPT · Enterprise, without killing the engine

Where it slips

  1. Avoids unsupported claims47% pass
    Presents the incorrect 'five of six no decision' figure as established fact, which is an unsupported claim about the current situation.
    Opus 5.5 · Claude · Enterprise, without killing the engine
  2. Uses the supplied evidence correctly52% pass
    The claim 'Five of our six losses on existing-user deals ended in no decision' is not supported by the data (4 no-decision, 2 Miro), so a statement about the current situation is invented.
    Opus 5.5 · Claude · Enterprise, without killing the engine
  3. Identifies material uncertainty73% pass
    The plan does not name specific unknowns that could change the decision, nor does it say how they would be resolved or what would trigger a pivot.
    Gemini 3.8 Flash · API · Payroll, for the people who fear it

How it’s graded

Each check is built on how world-class product and growth leaders do this work, as they’ve described it on Lenny’s Podcast, and names the leader it comes from.

  1. Positioned against the real alternative

    Does the plan name what the target customer does today without this product (including the status quo or doing nothing), and tie the message to the value that alternative lacks?

    Passes when Names the real alternative and the differentiated value against it, and picks the segment that cares most about that value.

  2. Sells against a pain the buyer feels

    Is the core message built on a pain or risk the target buyer already feels, with evidence from the material, rather than on generic upside like saving time?

    Passes when Leads with the pain the evidence shows buyers care about, and uses that evidence to choose the message.

  3. Diagnoses message or channel first

    Does the plan work out whether the problem is the message (people reach it but don't convert) or the channels (the right people never see it), and spend accordingly?

    Passes when Uses conversion and reach data to say which it is, and puts the effort there.

  4. Plans beyond launch week

    Does the plan say how demand keeps coming after launch, with the ongoing engine and what's measured over months, rather than treating launch day as the whole plan?

    Passes when Gives the ongoing motion after launch and the measures that show whether it's working over time.

Plus our standard checks

Uses the supplied evidence correctly · Addresses the actual decision · Respects explicit constraints · Identifies material uncertainty · Avoids unsupported claims · Produces the required deliverable · and 2 written for each task, which you’ll see in the tasks below.

The tasks

Read the brief, then compare up to three outputs, each with the LLM judge’s verdict on every check. Highlights mark what a PM had to fix.

The brief

You're the PM for Ledgerly Payroll, a new add-on for our accounting product, launching to existing customers on 3 November. Marketing has drafted the launch plan below. Write the go-to-market plan you'd run instead (or the changes you'd make to theirs), in no more than 900 words: who we go after first, what we say to them, through which channels, and how we'll know it's working over the following three months. The files are attached.

What the model was given5 items: About Ledgerly, marketing_launch_draft.md, customers_by_size_and_payroll_method.csv (penalty notices: share who told us they got a payroll penalty notice last year), beta_headline_test.csv (two versions of the in-app payroll page, shown at random to beta customers), customer_calls.md
About LedgerlyAccounting software for small businesses: invoicing, bank feeds, bookkeeping. 8,000 paying customers. Payroll is a new add-on at $6 per employee per month.
marketing_launch_draft.md7 lines · Download
# Draft launch plan (Head of Marketing)

- Launch day 3 November: press release announcing Ledgerly as "the first Financial Operating System for small business".
- Paid social and search ads to new prospects: $60,000 in November.
- In-app banner to all 8,000 customers: "Run payroll in minutes".
- Webinar in launch week.
- Success measure: press mentions and sign-ups in launch week.
customers_by_size_and_payroll_method.csv (penalty notices: share who told us they got a payroll penalty notice last year)employees,current_payroll,customers,penalty_notices_last_year_pct 1-4,Owner does it in a spreadsheet,3140,4 1-4,Accountant runs it,1020,6 5-25,Owner does it in a spreadsheet,690,19 5-25,Accountant runs it,1480,23 5-25,Another payroll tool,1210,7 26-50,Another payroll tool,460,5
beta_headline_test.csv (two versions of the in-app payroll page, shown at random to beta customers)headline,landing_page_visits,started_setup,ran_first_payroll "Run payroll in minutes, right where you do your books",4210,92,38 Never get a payroll penalty again,4160,268,117
customer_calls.md10 lines · Download
# Customer calls about payroll (excerpts, September)

1. Bakery, 12 staff: "My accountant does payroll. Last spring they filed late and I got a $1,400 penalty. I don't know what I'm paying them for."
2. Plumbing firm, 18 staff: "Time isn't the issue. I'm scared of getting it wrong. The tax office letter was the worst week of my year."
3. Design studio, 3 staff: "It's me and two freelancers. A spreadsheet's fine."
4. Café group, 22 staff: "We use another tool, it's OK. Switching sounds like a nightmare mid-year."
5. Physio clinic, 9 staff: "I'd move tomorrow if it meant one less thing that can go wrong with the tax office."
…
What a strong answer doesThe answer key the graders mark against

Picks customers with 5 to 25 employees whose accountant runs payroll (1,480 customers, 23% had a penalty notice) and those doing it in a spreadsheet (690, 19%), not the smallest businesses (happy with a spreadsheet) or those on another payroll tool (switching mid-year is the barrier). Builds the message on fear of penalties and errors, not time: in the beta, 'Never get a payroll penalty again' got about three times as many setups as 'Run payroll in minutes' (268 against 92 from similar traffic, 6.4% against 2.2%) and more first payrolls (117 against 38), and the calls say the same. Reads this as a message problem, not a channel problem: the existing customer base already sees the page, so fix the message and reach existing customers in-app and through their accountants before spending $60,000 on ads to strangers. Drops the 'Financial Operating System' category claim. Plans beyond launch week: prompts around payroll and tax deadlines, accountant referrals, and monthly measures such as payroll attach in the target segment, first payrolls run, and penalties avoided.

Critical failures (cap the score at 40)
  • Keeps 'Run payroll in minutes' as the lead message despite the headline test
Task

Mid-level · v1.0

The outputs

8 setups answered this brief.

Usable with a quick edit10 of 12 checks passed

The plan is detailed and actionable; the unsupported claims are minor cautions that a PM could remove or rephrase without redoing the reasoning or re-checking evidence.

Output

Ledgerly Payroll: existing-customer launch plan

3 November–2 February

1. Start with customers who fear mistakes—not everyone who dislikes admin

Prioritise the 2,170 existing customers with 5–25 employees whose payroll is run in spreadsheets or by an accountant. They represent 27% of our customer base:

  • 690 spreadsheet users: 19% reported a payroll penalty notice last year. Start here: they control the buying decision and have no incumbent software to replace.
  • 1,480 accountant-run customers: 23% reported a penalty notice. Approach alongside the first group, but qualify who controls payroll and whether the accountant should participate.

These penalty rates are much higher than the 4–7% reported by the other segments. Customer interviews reinforce the problem: owners want confidence that payroll is right, not merely faster.

Deprioritise the 4,160 customers with 1–4 employees: several see spreadsheets as sufficient, and willingness to pay looks weaker. Also defer broad outreach to the 1,670 customers using another payroll tool: their reported penalty rates are lower, and switching creates friction.

Within the priority segments, invite customers with known concerns or upcoming payroll runs first. Don’t assume a segment-level penalty rate means any particular customer received a notice.

2. Sell confidence and control, without promising immunity

Replace “the first Financial Operating System” and the generic speed headline with:

> Feel more confident about payroll, right alongside your books. > Ledgerly Payroll costs $6 per employee per month. See whether it fits your business and get help setting up your first payroll.

Use separate supporting messages:

  • Spreadsheet users: “Move beyond manual payroll spreadsheets. See how Ledgerly Payroll works before your next pay run.”
  • Accountant-run customers: “Get a clearer view of payroll—and decide how you and your accountant will manage it.”

The beta provides strong directional evidence for this positioning. The penalty headline generated first-payroll completions equal to 2.8% of page visits versus 0.9% for the speed headline—roughly three times higher. Setup starts also increased, while setup-to-first-payroll conversion remained similar, around 42–44%. The message attracts interest; onboarding still needs work.

However, do not ship “Never get a payroll penalty again.” It is an absolute guarantee the evidence cannot support. Test truthful reassurance against the speed message, with legal review.

Before launch, confirm exactly which calculations, checks, deadlines and filing responsibilities Ledgerly supports. Demonstrate only verified capabilities and clearly explain customer responsibilities. For price context, a 10-person business pays $60/month; don’t claim universal savings versus accountants.

3. Use owned channels and assisted onboarding before acquisition spend

Replace the $60,000 November prospecting campaign with a focused existing-customer rollout. Hold that spend until activation, retention and support economics justify expansion.

Before 3 November - Prepare segment-specific emails, in-app pages, eligibility guidance and a clear setup checklist. - Equip support with payroll-specific escalation procedures and capacity for first-run assistance. - Instrument the journey from invitation through paid, successful payroll. - Validate readiness. If a critical workflow is unsafe, narrow launch eligibility rather than market around it.

November: activate the first cohort - Send targeted email and in-app invitations to the priority customers, paced to support capacity—not one banner to all 8,000. - Offer a practical demonstration and bookable setup sessions. Schedule follow-ups around each customer’s next payroll date. - Use the launch-week webinar to show setup, responsibilities and a complete payroll workflow, not deliver a corporate announcement. - Where customers agree, involve their accountants. Position them as collaborators, not the cause of every payroll problem.

December: remove friction - Contact customers who started but stalled; diagnose migration, employee-data, confidence and accountant-approval barriers. - Turn recurring questions into onboarding improvements and help content. - Publish customer proof only with permission and substantiated outcomes.

January–2 February: expand selectively - Broaden outreach once repeat usage and operational quality are sound. - Test customers considering a switch from another tool, with explicit migration guidance. - Consider a small paid acquisition experiment only after establishing acceptable acquisition cost and payback.

4. Measure paid adoption, repeat use and trust—not launch noise

Set these as initial planning targets, not forecasts; recalibrate after November’s first cohort:

  • By 2 February: 217 priority customers—10% of the target segment—have completed a successful payroll and become paying add-on customers.
  • Repeat use: at least 80% of activated customers complete their next scheduled payroll; separately track third-run retention where enough time has elapsed.
  • Commercial performance: report actual payroll MRR, paid employees, cancellations and contribution after onboarding/support costs. Customer count alone does not establish revenue.

Review weekly by segment and channel: invitations delivered, qualified visits, setup starts, setup completion, successful first payroll, paid conversion, time to activation and support effort. Use unique-customer denominators, not just page visits.

Run a controlled headline test and, where feasible, stagger outreach with a holdout to measure incremental adoption.

Track failed runs, product-attributable calculation or filing incidents, and customer confidence after first and repeat runs. Investigate every serious incident; pause the affected rollout if reliability is compromised.

The launch succeeds when customers repeatedly pay staff correctly and keep paying for Ledgerly—not when they read a press release.

Claims the judge couldn’t find in the brief

  • The beta provides strong directional evidence for this positioning
  • It is an absolute guarantee the evidence cannot support

Grades and run details

Decision model 88 · LLM judge 85
Decision model checks
  • failUses the supplied evidence correctly55%
  • passAddresses the actual decision91%
  • passRespects explicit constraints46%
  • passIdentifies material uncertainty83%
  • partialAvoids unsupported claims33%
  • passProduces the required deliverable78%
  • passPicks the segment with the pain100%
  • passUses the headline test37%
  • passPositioned against the real alternative96%
  • passSells against a pain the buyer feels83%
  • passDiagnoses message or channel first100%
  • passPlans beyond launch week100%
Run
Run
#1
API response time
40 s
Submitted
5 Oct 2026

Usable with a quick edit9 of 12 checks passed

The plan is clear, evidence-based, and actionable; a PM could use it with minor wording tweaks and without rechecking the data or redoing the reasoning.

Output

Critique of the Marketing Draft

The marketing team’s launch plan has three fatal flaws: 1. Wrong audience for the budget: Spending $60,000 on paid acquisition to cold prospects for a new add-on is wasteful. We have 8,000 existing, engaged accounting customers. Our CAC here should be close to zero. 2. Ignoring empirical data on messaging: The draft uses "Run payroll in minutes" and PR buzzwords ("Financial Operating System"). Our beta test proved this fails: "Never get a payroll penalty again" drove 3.1x more completed payrolls (2.81% conversion vs. 0.90%). Customers don't care about speed—they fear tax penalties and accountant errors. 3. Vanity metrics: Press mentions and launch-week signups tell us nothing about adoption, product usage, or recurring revenue.

Here is the revised go-to-market plan.

---

1. Target Audience: Who We Go After First

We must prioritize based on revenue potential ($6/employee/month) and pain intensity (penalty notices):

``` Target Tier Segment (Size & Method) Base Penalty % Priority Tier 1 (Core) 5–25 emp: Accountant runs it 1,480 23% Immediate (Day 1) 5–25 emp: Spreadsheet 690 19% Immediate (Day 1) Tier 2 (Secondary)5–25 emp: Another payroll tool 1,210 7% Prep for Jan 1 Switch 1–4 emp: Accountant runs it 1,020 6% Nurture Deprioritize 1–4 emp: Spreadsheet (3,140) 3,140 4% Self-serve banner only 26–50 emp: Another tool (460) 460 5% Later phase ```

  • Primary Beachhead (Tier 1: 2,170 customers): Businesses with 5–25 employees doing payroll via spreadsheets or accountants. They suffer a massive 19–23% penalty rate, generate $30–$150/month each in add-on revenue, and are actively frustrated by accountant fees and errors (e.g., customer paying $180/month for late filings).
  • Secondary Opportunity (Tier 2): Those using other payroll tools (1,210 customers) are hesitant to switch mid-year. We target them in late November/December for a clean transition on January 1 (start of the new tax year).
  • Deprioritized: 1–4 employee spreadsheet users (3,140 customers) are complacent ("spreadsheet is fine/free") with low penalty exposure. We will expose them to in-app banners, but spend zero outbound effort on them.

---

2. Positioning & Messaging: What We Say

  • The Core Hook: "Never get a payroll penalty again. Guaranteed accuracy, right inside Ledgerly."
  • The Value Proposition: Ledgerly automates tax calculations, filings, and direct deposits directly from your books—eliminating accountant errors, late tax notices, and Sunday-night spreadsheet anxiety.
  • Segment Angles:
  • For Accountant-run payroll: "Fire the $180/month bill and the filing mistakes. Automate accurate payroll in Ledgerly for a fraction of the cost."
  • For Spreadsheet users: "One tax office notice costs more than three years of Ledgerly Payroll. Get automated tax filings and sleep easy."
  • For Competitor switchers: "Make January 1 your fresh start: switch to unified bookkeeping + payroll in 15 minutes."

---

3. Channel Strategy & Execution

Reallocate the $60,000 paid budget into onboarding support and switching incentives.

#### A. In-App Contextual Triggers (November 3) * Deploy the winning headline ("Never get a payroll penalty again") across dashboard real estate. * Trigger contextual prompts when users categorize payroll-related expenses (e.g., bank feed lines showing accountant payroll fees or tax payments): "Paying an accountant for payroll? Automate it error-free in Ledgerly for $6/employee."

#### B. Direct Outbound & Email Drips (Segmented) * Tier 1 (Accountant & Spreadsheet, 5–25 emp): Send a 3-part sequence focused on penalty protection, automatic tax filing, and reconciliation. Include a penalty-risk calculator. * Concierge Migration: Offer dedicated, white-glove onboarding for 5–25 employee accounts to import employee tax files and YTD data.

#### C. Education, Not a Feature Webinar * Replace the generic product demo with an expert session: "Avoiding Payroll Penalties in Q4 and Prepping for the 2025 Tax Year." Feature a customer testimonial sharing the pain of a tax notice.

#### D. The "January 1 Fresh Start" Campaign (Mid-November – December) * Target competitor users (1,210 customers) with an early-bird onboarding offer: "Set up your company before December 15; your first payroll run on January 1 is free."

---

4. 3-Month Success Metrics

We will track activation and usage, not press mentions.

#### Leading Indicators (Operational) * Setup-to-Run Conversion: >45% of users who start setup complete their first payroll run within 30 days (up from beta's 43.6%). * Tier 1 Activation: >12% of the 5–25 employee accountant/spreadsheet cohort (260 accounts) activate by Day 90.

#### 3-Month Milestones

  • Month 1 (November): Initial Adoption
  • 150 businesses run their first payroll.
  • Concierge migration processes 50+ businesses with >5 employees.
  • Target MRR added: ~$9,000.
  • Month 2 (December): Pipeline & Retention
  • 100% of Month 1 users run their second payroll cycle (0% churn).
  • 200 additional competitor-switch businesses pre-configure payroll to go live on January 1.
  • Target cumulative MRR: ~$18,000.
  • Month 3 (January): The Tax Year Switch Surge
  • Over 450 total active customers running payroll by Jan 31.
  • Blended average of 8 employees/account = 3,600 employees on platform.
  • Run-Rate Target: $21,600 MRR (~$260,000 ARR) achieved within 90 days of launch.
  • Zero filing penalties incurred by customers using Ledgerly Payroll.

Grades and run details

Decision model 71 · LLM judge 92
Decision model checks
  • failUses the supplied evidence correctly79%
  • passAddresses the actual decision55%
  • partialRespects explicit constraints14%
  • failIdentifies material uncertainty90%
  • failAvoids unsupported claims49%
  • passProduces the required deliverable61%
  • passPicks the segment with the pain100%
  • passUses the headline test97%
  • passPositioned against the real alternative100%
  • passSells against a pain the buyer feels100%
  • passDiagnoses message or channel first94%
  • passPlans beyond launch week100%
Run
Run
#1
API response time
27 s
Submitted
6 Oct 2026

Usable with a quick edit9 of 12 checks passed

The plan is clear and actionable, with only a minor unsupported claim about acquisition cost that can be edited out; no re-checking of evidence is needed.

Output

To: Executive Team & Head of Marketing From: PM, Ledgerly Payroll Date: October 18 Subject: Revised GTM Plan: Ledgerly Payroll Launch (3 November)

The draft marketing plan targets a generic market with high-level branding. That’s a mistake. We are launching an add-on to an existing base of 8,000 accounting customers. Our unfair advantage isn't that we're a "Financial Operating System"—it's that we already hold their financial data.

We need to scrap the broad acquisition play for November, refocus entirely on our existing base, and lead with our strongest asset: penalty anxiety and accuracy, not speed.

Here are the critical changes I am making to our launch plan.

---

1. Who We Go After First (The Wedge)

The data and customer calls tell a clear story.

  • Ignore: The 1-4 employee segment doing payroll in spreadsheets (Retail shop: "Annoying, but free"). They don't feel enough pain to switch or pay $6/mo.
  • Ignore: Mid-year switchers currently using dedicated payroll tools (Café group: "Switching sounds like a nightmare").
  • Target (Our Wedge): Small businesses with 5–25 employees who currently rely on an accountant or a spreadsheet, and are bleeding money/stress on penalties.

Look at the data: 5–25 employee businesses using accountants or spreadsheets suffer 19% to 23% penalty rates annually. Furthermore, customer calls highlight deep resentment toward accountant-driven payroll failures ($1,400 fines, $180/mo fees for mistakes).

We will segment our 8,000 customers on Day 1 and suppress micro-businesses (1-4) from our primary payroll marketing to protect email reputation and conversion relevance.

2. What We Say to Them (The Message)

Marketing’s draft leads with: "Run payroll in minutes."

Our beta test proved this is the wrong hook. Look at the `betaheadlinetest.csv` data: * "Run payroll in minutes..." generated only 38 completed payrolls. * "Never get a payroll penalty again" generated 117 completed payrolls (more than 3x the activation rate).

Time isn't the primary enemy; fear is (Plumbing firm: "I'm scared of getting it wrong. The tax office letter was the worst week of my year").

Our Core Value Proposition: “Keep payroll inside Ledgerly. Zero manual data entry from your books, guaranteed accurate tax calculations, or we cover the penalty.” (Note: We should seriously explore a penalty-reimbursement guarantee as a killer differentiator).

3. Through Which Channels

We are shifting $50,000 of the proposed November budget away from broad paid social/search targeting cold prospects. Acquiring cold payroll customers is expensive and slow; cross-selling our 8,000 warm customers is high-margin.

  • Channel 1: Targeted In-App Real Estate (Nov 3): Replace the generic banner with a dynamic state. If the user has 5–25 employees and uses an accountant/spreadsheet, show the "Never get a payroll penalty again" headline.
  • Channel 2: Founder/PM Email Campaign (Nov 4–10): A direct, non-marketing-sounding email sent from me (PM) and the CEO to the 5–25 segment. Focus on the hidden costs of accountant errors and spreadsheet slip-ups. Include a 1-click "Import my data" preview.
  • Channel 3: The "Fire Your Accountant's Payroll Bill" Playbook: A targeted retargeting campaign (using the remaining $10k of paid budget) aimed strictly at custom matched lists of our 5-25 employee segment, highlighting how much they can save by bringing payroll into Ledgerly alongside their bookkeeping.

4. How We’ll Measure Success (Months 1–3)

Marketing’s draft relies on vanity metrics: press mentions and launch-week sign-ups. We care about retention and activation. Ledgerly Payroll is priced at $6/employee/month; a sign-up that doesn’t run payroll is worthless.

Over the next three months, we will track four strict OKRs:

  • Month 1 (Launch & Activation):
  • Metric: 500 active payroll accounts set up from the 5–25 employee segment.
  • Guardrail Metric: Zero critical calculation support tickets in the first 14 days. Accuracy is our brand promise; a buggy launch kills trust permanently.
  • Month 2 (Expansion & Accountant Displacement):
  • Metric: 30% of activated users migrating away from their external accountant/spreadsheet for payroll.
  • Metric: Average team size of adopting accounts sits solidly in the 8–15 employee range (validating our wedge).
  • Month 3 (Monetization & Retention):
  • Metric: $15,000 in recurring monthly add-on revenue (MRR).
  • Metric: 95% month-two retention for payroll runners (proving they didn't just test it for a month, but integrated it into their operations).

Summary of Changes

We are abandoning the "Financial Operating System" fluff and the broad acquisition spend. On November 3, we laser-focus on our 5–25 employee segment with a message built on fear-relief and accuracy, driving them straight into a high-converting in-app flow.

Grades and run details

Decision model 79 · LLM judge 77
Decision model checks
  • partialUses the supplied evidence correctly3%
  • passAddresses the actual decision61%
  • partialRespects explicit constraints29%
  • failIdentifies material uncertainty35%
  • partialAvoids unsupported claims49%
  • passProduces the required deliverable62%
  • passPicks the segment with the pain93%
  • passUses the headline test100%
  • passPositioned against the real alternative100%
  • passSells against a pain the buyer feels100%
  • passDiagnoses message or channel first96%
  • passPlans beyond launch week95%
Run
Run
#1
Time to output
6 s
Submitted
5 Oct 2026

Check by check

Where the outputs differ comes first. Open a check to read the judge’s reasons. “Mixed” means the two graders disagreed.

Where they differ 4

Uses the supplied evidence correctlyWrongMixedRight
GPT-6.1 Sol · API

The output claims the beta provides strong directional evidence and that the evidence cannot support an absolute guarantee; neither is a fact from the brief or arithmetic, so not every statement about the current situation is supported.

Gemini 3.8 Flash · API

All factual claims about the current situation are taken directly from the supplied context or derived by correct arithmetic; the 43.6% conversion rate is the winning headline's setup-to-run rate, not an invention.

Gemini 3.5 Flash-Lite · Gemini

All factual claims about the current situation are directly supported by the brief or supplied context.

Addresses the actual decisionRightRightMixed
GPT-6.1 Sol · API

Commits early to a primary segment (5-25 employees, spreadsheet/accountant-run) and says what would change the call (reliability issues, activation economics).

Gemini 3.8 Flash · API

The output commits early and unambiguously to a revised go-to-market plan with a primary segment, message, channels, and metrics, framed for the product manager.

Gemini 3.5 Flash-Lite · Gemini

The output commits to a clear plan but does not state what result or condition would change that decision.

Identifies material uncertaintyRightWrongWrong
GPT-6.1 Sol · API

Names unknowns (exact payroll capabilities, legal review of guarantee, activation economics) and says how they would be resolved or would change the plan.

Gemini 3.8 Flash · API

The plan does not name specific unknowns that could change the decision, nor does it say how they would be resolved or what would trigger a pivot.

Gemini 3.5 Flash-Lite · Gemini

The output does not name any unknowns that could change the decision or how they would be resolved.

Avoids unsupported claimsWrongMixedWrong
GPT-6.1 Sol · API

Presents the claim that the evidence cannot support an absolute guarantee as a fact, and the beta as strong directional evidence, without labelling them as interpretations.

Gemini 3.8 Flash · API

Interpretations like 'customers fear penalties' are backed by the beta test and customer calls, and the output does not present unsupported causes or forecasts as fact.

Gemini 3.5 Flash-Lite · Gemini

It presents 'Acquiring cold payroll customers is expensive and slow; cross-selling our 8,000 warm customers is high-margin' as fact without supporting evidence.

All got right 8

Respects explicit constraintsRightRightRight
GPT-6.1 Sol · API

The output is a go-to-market plan within 900 words, respecting the requested form and length.

Gemini 3.8 Flash · API

The output is a go-to-market plan under 900 words, addresses the required elements, and respects the brief's form and reader.

Gemini 3.5 Flash-Lite · Gemini

The plan respects the 900-word limit, addresses the required elements, and does not lead with 'save time' or make paid ads the main channel.

Produces the required deliverableRightRightRight
GPT-6.1 Sol · API

Delivers a complete, actionable go-to-market plan for the PM, within the word limit.

Gemini 3.8 Flash · API

The output is a complete, actionable go-to-market plan within the word limit, with target audience, messaging, channels, and three-month metrics.

Gemini 3.5 Flash-Lite · Gemini

The output is a complete go-to-market plan in memo form, under 900 words, covering who, what, channels, and measures.

Picks the segment with the painRightRightRight
GPT-6.1 Sol · API

Targets 5-25 employees with spreadsheet or accountant-run payroll, uses the numbers and penalty rates, and explains why smaller businesses and tool users are deprioritised.

Gemini 3.8 Flash · API

The plan targets 5–25 employee businesses using accountant-run or spreadsheet payroll, cites the penalty rates and customer counts, and explains why smaller businesses and tool-switchers are deprioritized.

Gemini 3.5 Flash-Lite · Gemini

It targets 5-25 employee businesses using accountant/spreadsheet, cites penalty rates, and explains why smaller and tool-using segments are not first.

Uses the headline testRightRightRight
GPT-6.1 Sol · API

Cites the beta test's 2.8% vs 0.9% first-payroll rates and builds the message on penalties and accuracy.

Gemini 3.8 Flash · API

The plan explicitly cites the beta headline test results (3.1x more completed payrolls) and builds the core message on penalties and accuracy.

Gemini 3.5 Flash-Lite · Gemini

It uses the beta test results (38 vs 117 completed payrolls) to choose the penalty-focused message.

Positioned against the real alternativeRightRightRight
GPT-6.1 Sol · API

Names the real alternatives (spreadsheet, accountant) and ties the message to confidence and accuracy they lack.

Gemini 3.8 Flash · API

The plan names the real alternatives (accountant, spreadsheet, another tool) and ties the message to eliminating penalties and errors, targeting the segment that cares most.

Gemini 3.5 Flash-Lite · Gemini

It names the real alternatives (accountant, spreadsheet, other tools) and ties the message to the accuracy and penalty avoidance they lack.

Sells against a pain the buyer feelsRightRightRight
GPT-6.1 Sol · API

Leads with fear of mistakes and penalties, using the customer calls and penalty data, not generic time-saving.

Gemini 3.8 Flash · API

The core message leads with the pain of tax penalties and errors, supported by the beta test and customer calls, not generic time-saving.

Gemini 3.5 Flash-Lite · Gemini

The core message is built on fear of penalties and errors, supported by customer calls and the headline test.

Diagnoses message or channel firstRightRightRight
GPT-6.1 Sol · API

Uses the beta conversion data to diagnose a message problem and shifts spend to owned channels and message improvement before paid acquisition.

Gemini 3.8 Flash · API

The plan uses conversion data to diagnose that the problem is the message, not reach, and reallocates budget from paid ads to in-app and existing-customer channels.

Gemini 3.5 Flash-Lite · Gemini

It diagnoses the problem as message (low conversion on the time headline) and shifts effort to fixing the message and reaching existing customers.

Plans beyond launch weekRightRightRight
GPT-6.1 Sol · API

Plans December friction removal, January expansion, and measures repeat use, retention, and MRR over three months.

Gemini 3.8 Flash · API

The plan includes ongoing in-app triggers, email drips, a January 1 campaign, and monthly metrics over three months, not just launch week.

Gemini 3.5 Flash-Lite · Gemini

It provides an ongoing motion with in-app, email, and retargeting, and measures success over three months with specific metrics.

Results

Every setup we’ve tested on this task type, across all its tasks and repeats, graded on the current checklist. Calibrated.

#Model · HarnessTask scoreDecision modelLLM judgeRunsCritical failures
1GPT-6 AstrawithChatGPT100.0100.02None
2Opus 5.5withClaude96.085.22None
3GPT-6 LunawithAPI94.289.32None
4GPT-6.1 SolwithAPI93.878.02None
5Sonnet 5.5withAPI98.177.72None
6Haiku 5.5withAPI94.277.72None
7Gemini 3.8 FlashwithAPI73.981.92None
8Gemini 3.5 Flash-LitewithGemini74.263.52None

What you get for the cost

No setup scores within 5 points of the leader at half its cost or less here, so the leader is also the best value.

Task score against relative cost for each setup. The table below has the same figures.607080901001×3×10×30×100×Cost per output, relative to the cheapest setup (log scale) →↑ Task scoreGPT-6 Astra · ChatGPT: task score 100, 81× the cheapestOpus 5.5 · Claude: task score 91, 75× the cheapestGPT-6 Luna · API: task score 90, the cheapestGPT-6.1 Sol · API: task score 84, 16× the cheapestSonnet 5.5 · API: task score 84, 46× the cheapestHaiku 5.5 · API: task score 82, 3.3× the cheapestGemini 3.8 Flash · API: task score 77, 11× the cheapestGemini 3.5 Flash-Lite · Gemini: task score 68, 3.5× the cheapestGPT-6 Astra · ChatGPTOpus 5.5 · ClaudeGPT-6 Luna · APIGPT-6.1 Sol · APISonnet 5.5 · APIHaiku 5.5 · APIGemini 3.8 Flash · APIGemini 3.5 Flash-Lite · Gemini
Nobody beats these on both score and cost
Task score and relative cost by setup
SetupTask scoreRelative cost (cheapest = 1)
GPT-6 Luna · API901.0×
Haiku 5.5 · API823.3×
Gemini 3.5 Flash-Lite · Gemini683.5×
Gemini 3.8 Flash · API7711×
GPT-6.1 Sol · API8416×
Sonnet 5.5 · API8446×
Opus 5.5 · Claude9175×
GPT-6 Astra · ChatGPT10081×

Cost is each output’s tokens at the model’s list price, shown relative to the cheapest setup. Runs done in an app have no usage meter, so theirs are estimated from the same setup’s measured runs. How we compare cost

About the task

The PM job

Taking a product or add-on to market with marketing and sales.

Why it matters

Launch plans fail by trying to reach everyone, by selling upside nobody feels, and by treating launch day as the whole plan.

What good looks like

  • One primary segment: those who value the difference most
  • Positions against what customers do today
  • Sells against a pain the buyer feels, with evidence
  • Diagnoses whether the problem is the message or the channel
  • Plans beyond launch week

Deliberately not measured

  • Creative copywriting quality
Capability tested

Go-to-market judgement

The failure we’re looking for

Every channel, every segment, and nothing after launch day

Grading

Decision model and LLM judge, calibrated against a blind PM review