Tasks / Strategy

Price and package a product

Can the model price on the value each kind of customer gets, using real usage and willingness-to-pay data, and get existing customers there?

Task type v1.1 · 2 tasksLast changed 6 Oct 2026 · ChangelogDifficulty

What AI gets right here, and what you’ll still have to catch

From 16 graded outputs by 8 models. 56% were usable with at most a quick edit.

Reliably right

  1. Doesn't trust the price test100% pass
    Names the 59/41 split mismatch, holiday timing, and small sample; does not rest the decision on the test.
    Opus 5.5 · Claude · Unlimited pages, falling margins
  2. Plans the renewals100% pass
    Addresses each enterprise renewal with its date and note, including Copperfield in January, Northwind's tripled volume, and Arcadia's competitor evaluation.
    Opus 5.5 · Claude · Unlimited pages, falling margins
  3. Each tier for a different buyer94% pass
    Each tier names its buyer (startups, mid-market, enterprise) and is priced on distinct value: startups protected, mid-market overage on heavy users, enterprise per-page with unlimited seats.
    Opus 5.5 · Claude · Unlimited pages, falling margins

Where it slips

  1. Uses the supplied evidence correctly34% pass
    Total MRR is stated as $11,670 (actual $11,820), and median 'getting expensive' for small agencies is given as about $37 (actual $33) and for agencies as about $55 (actual $51).
    Haiku 5.5 · API · Fifteen dollars for everyone
  2. Gets the base of every number right41% pass
    MRR total and two segment‑level median getting‑expensive figures are incorrectly computed from the supplied data.
    Haiku 5.5 · API · Fifteen dollars for everyone
  3. Avoids unsupported claims52% pass
    Median getting‑expensive values and total MRR are presented as fact but are not supported by the data; these are facts about the current situation, not labelled as assumptions.
    Haiku 5.5 · API · Fifteen dollars for everyone

How it’s graded

Each check is built on how world-class product and growth leaders do this work, as they’ve described it on Lenny’s Podcast, and names the leader it comes from.

  1. Each tier for a different buyer

    Is each plan built for a distinct kind of buyer, with what that buyer values, rather than more-of-the-same at a higher price?

    Passes when Every tier names its buyer and the value it's priced on, and includes what that buyer needs and the lower tier's buyer doesn't.

  2. Priced on willingness to pay

    Are the proposed prices grounded in evidence of what each kind of customer would pay and the value they get, rather than in cost-plus, competitors alone or a round-number rise?

    Passes when Uses the willingness-to-pay and value evidence in the material to set each price, and says where the evidence is thin.

  3. Moves existing customers fairly

    Does the proposal say how existing customers move to the new pricing (who changes first, with what notice, and the value that justifies it)?

    Passes when A transition plan: new customers first or a set notice period for existing ones, tied to value delivered, with how churn will be watched.

Plus our standard checks

Uses the supplied evidence correctly · Addresses the actual decision · Respects explicit constraints · Identifies material uncertainty · Avoids unsupported claims · Produces the required deliverable · Gets the base of every number right · and 2 written for each task, which you’ll see in the tasks below.

The tasks

Read the brief, then compare up to three outputs, each with the LLM judge’s verdict on every check. Highlights mark what a PM had to fix.

The brief

You're the PM for pricing at Inkwell. Our CEO, Priya Nair, wants to move everyone to $19 a user from January (her note is below). Before she announces it, she's asked you for a pricing and packaging proposal: what we charge, for what, to whom, and how we move existing customers onto it. No more than 900 words, with a table of the plans. The files are attached. The account export is a random sample of 150 paying accounts.

What the model was given6 items: About Inkwell, ceo_note.md, accounts_sample.csv, willingness_to_pay_survey.csv (price per user per month at which it's too cheap to trust, a bargain, getting expensive, too expensive), win_loss_last_two_quarters.csv, trial_funnel_q3.md
About InkwellProposal and contract software: write a proposal from templates, send it, get it e-signed. Sold to freelancers and agencies. One plan today: $15 per user per month after a 14-day free trial, with everything included.
ceo_note.md1 lines · Download
From Priya (CEO): "We haven't touched price since launch. Let's go to $19 a user from January for everyone, new and existing. Simple, and a 27% revenue bump."
accounts_sample.csv150 rows · Download
account_id,segment,seats,proposals_sent_90d,proposals_signed_90d,signed_value_90d_usd,months_as_customer,mrr_usd
INK-3001,Small agency (2-10),3,29,14,108444,18,45
INK-3002,Freelancer,1,12,5,5350,7,15
INK-3003,Small agency (2-10),3,62,34,200396,28,45
INK-3004,Agency (11-50),16,139,74,1587596,29,240
INK-3005,Small agency (2-10),6,18,9,42498,30,90
INK-3006,Agency (11-50),29,236,89,1192333,18,435
…
willingness_to_pay_survey.csv (price per user per month at which it's too cheap to trust, a bargain, getting expensive, too expensive)response_id,segment,too_cheap_usd,a_bargain_usd,getting_expensive_usd,too_expensive_usd W400,Freelancer,4,7,14,25 W401,Freelancer,4,9,16,30 W402,Freelancer,5,7,19,20 W403,Freelancer,3,9,14,23 W404,Small agency (2-10),9,12,29,56 W405,Small agency (2-10),10,13,28,47 W406,Small agency (2-10),6,14,31,47 W407,Freelancer,4,10,15,29 W408,Small agency (2-10),6,15,40,56 W409,Small agency (2-10),8,14,40,46 W410,Freelancer,4,8,15,25 W411,Freelancer,4,8,15,19 W412,Freelancer,5,9,18,18 W413,Small agency (2-10),9,12,38,49 W414,Agency (11-50),14,22,49,60 W415,Freelancer,3,6,18,18 W416,Freelancer,4,8,15,28 W417,Small agency (2-10),10,15,37,46 W418,Agency (11-50),13,30,55,73 W419,Freelancer,4,9,15,29 W420,Freelancer,3,7,13,29 W421,Freelancer,4,9,19,21 W422,Agency (11-50),11,29,61,67 W423,Small agency (2-10),8,13,26,62 W424,Freelancer,4,9,20,21 W425,Agency (11-50),10,30,51,73 W426,Small agency (2-10),9,13,38,49 W427,Small agency (2-10),6,17,31,49 W428,Freelancer,4,6,20,28 W429,Small agency (2-10),9,12,41,52 W430,Agency (11-50),9,26,50,69 W431,Freelancer,5,9,15,24 W432,Freelancer,3,8,19,28 W433,Agency (11-50),14,20,66,94 W434,Freelancer,3,10,15,27 W435,Small agency (2-10),7,16,28,46 W436,Freelancer,4,9,17,28 W437,Small agency (2-10),6,18,31,56 W438,Agency (11-50),14,27,59,86 W439,Freelancer,5,10,19,28 W440,Freelancer,5,7,12,25 W441,Agency (11-50),14,23,42,66 W442,Freelancer,4,6,13,26 W443,Small agency (2-10),9,13,34,47 W444,Freelancer,4,10,18,29 W445,Freelancer,4,9,19,29 W446,Small agency (2-10),10,18,30,55 W447,Freelancer,4,10,15,24 W448,Small agency (2-10),10,19,26,40 W449,Small agency (2-10),8,15,41,42 W450,Freelancer,5,8,14,26 W451,Freelancer,4,6,14,30 W452,Small agency (2-10),7,18,37,43 W453,Small agency (2-10),7,19,27,42 W454,Freelancer,3,8,13,29 W455,Agency (11-50),10,24,57,99 W456,Freelancer,4,9,17,24 W457,Small agency (2-10),9,17,32,55 W458,Freelancer,4,9,14,23 W459,Small agency (2-10),9,15,30,38 W460,Freelancer,4,7,18,28 W461,Freelancer,4,8,14,23 W462,Small agency (2-10),9,17,27,53 W463,Freelancer,3,9,14,21 W464,Agency (11-50),13,31,58,63 W465,Agency (11-50),15,26,42,80 W466,Freelancer,5,9,18,19 W467,Small agency (2-10),9,15,27,49 W468,Freelancer,4,8,14,23 W469,Small agency (2-10),9,12,29,51 W470,Agency (11-50),14,26,51,92 W471,Freelancer,5,7,14,26 W472,Freelancer,4,9,18,23 W473,Freelancer,3,10,12,22 W474,Freelancer,5,8,16,29 W475,Agency (11-50),11,20,67,62 W476,Agency (11-50),13,28,49,79 W477,Freelancer,5,10,14,22 W478,Agency (11-50),11,21,50,91 W479,Freelancer,4,9,13,29 W480,Freelancer,3,6,15,25 W481,Small agency (2-10),9,16,29,53 W482,Freelancer,4,10,18,27 W483,Small agency (2-10),10,15,40,54 W484,Agency (11-50),11,20,42,80 W485,Small agency (2-10),8,14,42,52 W486,Freelancer,5,8,16,30 W487,Small agency (2-10),8,15,27,49 W488,Agency (11-50),15,26,58,70 W489,Small agency (2-10),8,19,26,51 W490,Small agency (2-10),6,14,26,42 W491,Small agency (2-10),7,16,37,45 W492,Freelancer,4,6,13,21 W493,Small agency (2-10),8,18,42,44 W494,Small agency (2-10),8,17,38,52 W495,Small agency (2-10),7,13,26,45 W496,Small agency (2-10),9,18,42,56 W497,Freelancer,5,8,17,28 W498,Agency (11-50),10,24,51,92 W499,Freelancer,4,6,19,26 W500,Agency (11-50),10,21,58,93 W501,Freelancer,3,6,15,20 W502,Freelancer,5,7,19,20 W503,Small agency (2-10),9,15,40,59 W504,Small agency (2-10),7,19,29,58 W505,Small agency (2-10),8,18,27,62 W506,Freelancer,5,7,14,26 W507,Small agency (2-10),8,14,40,46 W508,Freelancer,5,9,13,28 W509,Freelancer,4,10,12,25 W510,Freelancer,4,10,18,28 W511,Freelancer,4,10,14,20 W512,Agency (11-50),10,20,42,73 W513,Freelancer,5,9,14,30 W514,Freelancer,4,7,15,26 W515,Freelancer,3,9,15,27 W516,Freelancer,4,9,16,19 W517,Small agency (2-10),8,13,36,54 W518,Freelancer,5,10,12,19 W519,Freelancer,3,8,19,24
win_loss_last_two_quarters.csv160 rows · Download
deal_id,quarter,segment,outcome,price_objection,lost_to
D-7001,2026-Q3,Small agency (2-10),won,no,
D-7002,2026-Q2,Freelancer,no decision,no,stayed with current process
D-7003,2026-Q3,Small agency (2-10),won,no,
D-7004,2026-Q2,Small agency (2-10),lost,no,PandaDoc
D-7005,2026-Q3,Agency (11-50),no decision,no,stayed with current process
D-7006,2026-Q2,Small agency (2-10),no decision,no,stayed with current process
…
trial_funnel_q3.md9 lines · Download
# Trial funnel, Q3 2026 (14-day free trial, then $15 per user per month)

| Segment | Trials started | Became paying | Paying accounts that cancelled in their second month |
|---|---|---|---|
| Freelancer | 1,900 | 171 | 38% |
| Small agency (2-10) | 420 | 130 | 6% |
| Agency (11-50) | 60 | 22 | 0% |
…
What a strong answer doesThe answer key the graders mark against

Proposes tiers built around three different buyers, not one price. Agencies close about $20,000 a seat each month through Inkwell (small agencies about $8,000, freelancers under $2,000), use the shared library, approvals and CRM sync, and put 'getting expensive' at about $51 a user (small agencies about $31), so they are badly underpriced at $15; win rates are high (125 of 160 deals) and price objections rare (6 of 160), mostly from freelancers. Freelancers put 'getting expensive' at about $15 and 'too expensive' at about $26, convert at 9% and cancel at 38% in month two, so $19 would push them past their limit and lose the few who stay. So: a freelancer plan at or below today's price (or a free plan with a usage limit, such as a few proposals a month, rather than a time-limited trial), a team plan for small agencies around the low $20s to $30, and an agency plan in the $40s or with the features agencies value (library, approvals, CRM sync), named for the buyer. Rolls it out to new customers first and gives existing customers notice and a grace period tied to new value, and says how it will measure the change (conversion, churn, revenue per account).

Critical failures (cap the score at 40)
  • Recommends a single flat price for every customer
Task

Mid-level · v1.1

The outputs

8 setups answered this brief.

Usable with a quick edit10 of 12 checks passed

The proposal is data-driven, complete, and directly usable by a product manager with only minor wording adjustments, without needing to re-verify evidence or redo analysis.

Output

Inkwell Pricing and Packaging Proposal

Recommendation

Don't move everyone to a flat $19. Our customers aren't one market.

  • Freelancers are 53% of accounts but only 10% of revenue, and they already find $15 expensive.
  • Agencies are 13% of accounts and 56% of revenue, and they consider $15 cheap.

A flat $19 overcharges the first group and still undercharges the second. I propose three plans priced by segment, with a staged move for existing customers. Before churn, this delivers about +44% MRR in year one and +95% once fully migrated, compared with +27% for a flat $19.

What the data says

Revenue concentration (150-account sample, $11,805 MRR)

SegmentAccountsSeatsShare of MRR
Freelancers808010%
Small agencies5026434%
Agencies2044356%

Freelancers are price-sensitive and short-lived. - 50 of 60 freelancer survey respondents put "getting expensive" below $19. For 35 of 60, $15 is already at or past that point. - 5 of the 6 price objections in win/loss came from freelancers. - Freelancers lose to "Google Docs and a free e-signature tool" (12 of 70 deals), not to PandaDoc. - Their trial conversion is 9%, and 38% cancel in month two. - No freelancer in the sample has been a customer longer than 9 months. - They mostly use one template and e-signature.

Agencies are underpriced. - Every agency respondent's "bargain" price is $20 or higher. - 7 of 20 put "too cheap to trust" at $14–15. - Agencies never raised price in win/loss. Their losses to PandaDoc were about product. - They had 0% early churn and use the features freelancers don't: shared library, approvals and CRM sync. - At $35, every sampled agency would pay under 0.5% of the contract value it signs through Inkwell.

Small agencies have room. - Every respondent's "getting expensive" point is $26 or higher. - Trial conversion is 31%, with 6% early churn.

Plans

PlanForPrice (per user/month)Includes
SoloFreelancers$12, 1 user onlyUnlimited proposals, templates, e-signature
Team2–10 users$25Solo + shared content library, approval workflows, CRM sync
AgencyRequired at 11+ users$35Team + multi-step approvals, roles and permissions, admin controls, priority support

The 14-day free trial stays on all plans.

Why these prices: - $12 Solo sits below "getting expensive" for 56 of 60 freelancers. It costs about 2% of total MRR to cut existing freelancers from $15. - $25 Team is below every small-agency respondent's "getting expensive" point. - $35 Agency is below every agency respondent's "getting expensive" point (lowest is $42) and well under "too expensive" (lowest is $60).

Plan fences: - The seat cap (Solo = 1, Team ≤ 10) is the main fence. It maps onto the segments the data already uses. - Agency features are deliberately the ones a larger team needs to manage people, not new product promises.

Moving existing customers

TodayYear 1From year 2 (next anniversary)
Freelancer, $15Solo $12 immediately (a price cut)$12
Small agency, $15Team at $19$25
Agency, $15Agency at $25$35

How to run it: - Notice. Give at least 60 days' notice to anyone whose price rises. If January is closer than that, launch new prices for new customers in January and move existing customers in February or March. - Lead with what's included. Existing teams keep every feature they use today. The message is "you're on the plan built for teams," not "the price went up." - Annual lock. Offer annual prepay at the year-1 rate, locked for 12 months. This converts the increase into commitment. - At-risk agencies. Have someone call the top agencies by MRR personally before the email goes out. - Your $19 still appears. It becomes the year-1 rate for existing small agencies.

Revenue impact (sample MRR, before churn)

ScenarioMRRChange
Today$11,805—
Flat $19$14,953+27%
Proposal, year 1$17,051+44%
Proposal, full list$23,065+95%

What this means: - The flat $19 figure assumes zero churn and no drag on new sales. Most of the churn risk sits with freelancers, the segment that pays the least. - Year 1 beats flat $19 unless we lose about 12% of MRR. That's roughly 1 in 5 agency seats, in a segment with 0% early churn and no price objections. - The real risk is concentrated. Agency seats carry most of the upside, so agency retention through migration is the thing to watch. Freelancer churn barely moves the total.

Risks and checks before announcing

1. The survey is stated willingness to pay, not behaviour. We don't know whether respondents are customers or prospects. To limit the risk: - Launch new-customer prices first and watch trial conversion by segment for 4–6 weeks before existing customers move. - Keep the agency year-1 rate at $25. 2. Check CRM sync usage before finalising Team. CRM sync is in Team so small agencies lose nothing. If heavy CRM-sync users are mostly 11+ seat accounts, it could move to Agency instead. I need feature usage by account size to confirm. 3. Agencies near the 10/11 seat boundary may resist the jump. Allow a 12-seat grace period on Team for existing customers in year 1. 4. Freelancer losses to free tools aren't solved by a cheaper Solo plan alone. A free tier (e.g., 2 sends a month) is worth testing separately on new trials. It isn't part of this launch because it could cannibalise paying freelancers, who send about 3 proposals a month. 5. Metrics to track monthly: - Trial-to-paid conversion by plan - Month-two cancellations for Solo - Agency logo and seat churn through each migration step - Win rate against PandaDoc

Decision needed from Priya: approve the three plans and the staged migration, and confirm the existing-customer timing (January if 60 days' notice is possible, otherwise February/March).

Grades and run details

Decision model 75 · LLM judge 100
Decision model checks
  • failUses the supplied evidence correctly44%
  • passAddresses the actual decision89%
  • partialRespects explicit constraints32%
  • passIdentifies material uncertainty97%
  • partialAvoids unsupported claims26%
  • passProduces the required deliverable55%
  • passSees who's underpriced76%
  • passAnswers the CEO's flat rise100%
  • passEach tier for a different buyer87%
  • failGets the base of every number right79%
  • passPriced on willingness to pay72%
  • passMoves existing customers fairly100%
Run
Run
#1
Time to output
1.8 min
Submitted
5 Oct 2026

Needs more than a quick edit3 of 12 checks passed

Contains a factual error in the median too-expensive claim and omits churn monitoring and value-gap analysis from account data, requiring rework.

Output

MEMORANDUM

TO: Priya Nair, CEO FROM: Pricing PM, Inkwell DATE: October 24, 2026 SUBJECT: Q1 2027 Pricing & Packaging Proposal

---

Executive Summary

Your instinct to capture more value is spot on—we haven’t touched price since launch, and our growth gives us pricing power. However, moving everyone to a flat $19/user in January carries hidden risks.

Based on our Van Westendorp pricing survey, win/loss data, and trial cohort metrics, a flat per-user hike across all segments will cause churn spikes among freelancers (whose median "too expensive" threshold is $25/mo) and leave money on the table for larger agencies (who derive immense value from collaboration features and are willing to pay significantly more).

Instead of a blunt instrument, I propose a packaged tier strategy starting January 1st that protects our low-end acquisition funnel, extracts enterprise-grade value from larger agencies, and grandfathering existing customers gracefully.

---

Data Insights & Rationale

1. Willingness to Pay Varies Widely by Segment: * Freelancers ($15 current): Median "too expensive" point is $25–$28. A jump to $19 is survivable, but pushing past $20 risks driving them back to free e-signature tools and Google Docs (our #1 lost-to competitor). * Small Agencies (2–10 seats): Median "too expensive" point is $46–$55 ($15–$18/user effective). They heavily utilize team features (shared libraries, approval workflows). * Agencies (11–50 seats): Median "too expensive" point is $70–$93 ($6–$8/user effective at scale). They view Inkwell as core operations infrastructure and integrate it with their CRM. They are severely under-monitored on a flat per-user model.

2. The Freelancer Churn Trap: Our Q3 trial funnel shows that 38% of freelancers cancel in their second month. Raising their price immediately in January without added value will accelerate this churn.

---

Proposed Pricing & Packaging Structure (Effective January 1)

We will transition from a single monolithic plan to a 3-tier product-led packaging model.

Plan NameTarget SegmentPriceCore Packaging & Features
:---:---:---:---
StarterFreelancers (1 user)$19 / mo (Flat)1 user, unlimited proposals, basic templates, e-sign, standard branding.
GrowthSmall Agencies (2–10 seats)$25 / user / moStarter features + Shared content library, team approval workflows, custom branding, priority support.
ScaleAgencies (11–50 seats)$39 / user / moGrowth features + CRM integrations (HubSpot/Salesforce), advanced audit logs, dedicated account review, SSO.

---

Revenue Impact Projections

  • Starter Tier ($19 flat): Retains freelancers at your target price point while securing a 26% ARPU lift on single-user accounts.
  • Growth & Scale Tiers: By packaging high-value features (CRM sync, approvals) into higher tiers, we capture enterprise willingness-to-pay. For an average 20-seat agency currently paying $300/mo ($15/seat), moving to the Scale tier at $39/seat shifts their MRR to $780—a 160% expansion, rather than a modest 27% bump.

---

Migration & Rollout Plan for Existing Customers

To avoid churn and backlash while hitting our January revenue targets, we will execute the following transition:

1. Grandfathering Window (Peace of Mind): Existing customers will be grandfathered on their current $15/user rate for 90 days (through March 31, 2027), provided they do not change plans. This prevents a sudden holiday-season churn spike. 2. Value-Based Migration (April 1): * Freelancers automatically transition to the Starter plan ($19/mo flat) on April 1, with a 30-day advance notice emphasizing new template updates. * Agencies will be mapped to the Growth or Scale tiers based on their current feature usage (e.g., accounts currently utilizing CRM sync will be slotted into Scale, with a personalized outreach from success reps highlighting the added utility). 3. Opt-Out & Retention Safety Net: Any existing customer who receives the April price change notice will be offered an annual prepayment discount (2 months free) to lock in a intermediate rate, preserving cash flow while securing retention.

Next Steps

With your approval on this structure, product and marketing will finalize the feature gating by November 15, allowing us to build billing migrations and draft customer communications well ahead of the January launch.

Claims the judge couldn’t find in the brief

  • Freelancers' median 'too expensive' threshold is $25/mo.

Grades and run details

Decision model 42 · LLM judge 38
Decision model checks
  • failUses the supplied evidence correctly70%
  • partialAddresses the actual decision34%
  • failRespects explicit constraints26%
  • failIdentifies material uncertainty63%
  • failAvoids unsupported claims59%
  • partialProduces the required deliverable35%
  • partialSees who's underpriced66%
  • passAnswers the CEO's flat rise22%
  • passEach tier for a different buyer80%
  • failGets the base of every number right69%
  • partialPriced on willingness to pay88%
  • passMoves existing customers fairly61%
Run
Run
#1
Time to output
5 s
Submitted
5 Oct 2026

Needs more than a quick edit9 of 12 checks passed

The proposal misses the signed value per seat data needed to fully show the value gap and justify higher agency pricing; a quick edit adding that analysis would make it ready.

Output

Inkwell pricing and packaging proposal

To: Priya Nair | Decision: segmented pricing, not $19 for everyone

Recommendation

Launch three feature-based plans for new customers in January. Keep the freelancer entry price at $15; charge more for agency collaboration and integration. Move existing agency customers gradually, preserving their current functionality.

A blanket $19 increase produces 26.7% more revenue only if paid seats remain unchanged. Losing 21.1% of paid seats would erase that uplift. It also raises prices for our most fragile segment while leaving agency willingness to pay underused.

What the evidence says

  • Revenue is concentrated in agencies. The random sample contains 80 freelancers, 50 small agencies and 20 larger agencies. Its 787 seats generate $11,805 MRR; agencies contribute 89.8% of MRR, despite representing only 46.7% of accounts.
  • Freelancer acquisition and retention are weak. Trial conversion is 9%, versus 31% for small agencies and 36.7% for larger agencies. Second-month cancellation is 38%, 6% and 0%, respectively. The last figure comes from only 22 conversions.
  • Price tolerance differs substantially. Survey “getting expensive” thresholds range from $12–20 for freelancers, $26–42 for small agencies and $42–67 for larger agencies. These are stated preferences, not proven demand curves; one larger-agency response has inconsistent thresholds.
  • Value and usage differ. Freelancers sent 3–12 proposals over 90 days, versus 18–63 for small agencies and 95–287 for larger agencies. Agency usage emphasizes the shared library, approvals and CRM sync; freelancers mostly use one template and e-signature.
  • Cheap substitutes matter. All 12 freelancer losses went to Google Docs plus free e-signature; four explicitly cited price. Agency losses went to PandaDoc, without recorded price objections. That does not establish why they lost—or prove agencies will accept any increase.

Proposed plans

Prices are USD, billed monthly. Retain the 14-day trial. Defer annual discounts until monthly pricing is validated.

PlanPriceIncludedIntended customer
------:------
Solo$15/month; one userTemplates, proposal creation and sending, e-signatureFreelancers needing the core proposal-to-signature workflow
Team$29/user/monthSolo capabilities plus shared content library and approval workflowsAgencies collaborating on proposals
Business$39/user/monthTeam capabilities plus CRM syncAgencies needing an integrated sales workflow

Team and Business have no minimum seat commitment or segment-based eligibility. A two-person agency can buy Business; a larger agency can buy Team. Charge for users who create, edit or approve proposals—not external recipients signing them.

Do not introduce proposal limits, signature fees or charges tied to contract value. They would discourage the activity that demonstrates value. Signed contract value is context, not revenue attributable to Inkwell.

These prices are testable starting points, not survey-derived optima. Business’s incremental value must justify its premium; validate CRM demand before treating larger agencies as automatic Business buyers.

Moving existing customers

  1. Give at least 90 days’ notice, stating the customer’s current bill, transition bill, eventual price and effective dates. If notice cannot be delivered in time, January is new-customer launch only.
  2. Existing freelancers remain at $15. Preserve all functionality already included; offer Team or Business upgrades voluntarily.
  3. Existing agencies move first to $19/user, at the first eligible renewal after notice, for six months. Then move to Team at $29, or Business at $39 if affirmatively selected.
  4. Preserve existing feature entitlements, including CRM sync, on migrated Team accounts. Do not remove functionality to manufacture an upgrade. Business packaging applies cleanly to new accounts; legacy exceptions are explicit.
  5. Honor contractual commitments. Offer cancellation and data export before increases, plus a documented, time-limited hardship extension. Avoid permanent discretionary discounts.

Revenue expectations and validation

At unchanged seats, the sample would generate:

  • Blanket $19: $14,953 MRR, +26.7%.
  • Agency transition at $19; freelancers unchanged: $14,633, +24.0%.
  • All agency seats eventually on Team: $21,703, +83.8%.
  • Small agencies on Team; larger agencies on Business: $26,133, +121.4%.

The last two are allocation scenarios, not forecasts: feature needs, downgrades, seat reductions and churn are unknown. Do not multiply sample totals into company revenue without the full account count and portfolio validation.

Before announcing final prices, interview customers across segments and test the proposed packages against today’s offer with new trials. Track conversion, second-month retention, revenue per trial, paid-seat contraction and support complaints by segment. Use a staged existing-customer rollout with a comparable holdout; pause expansion if incremental revenue after churn and seat contraction is non-positive.

CEO message: “We’re keeping an accessible solo plan and introducing agency plans priced around collaboration and integration—with advance notice and a gradual transition for existing customers.”

Grades and run details

Decision model 75 · LLM judge 85
Decision model checks
  • failUses the supplied evidence correctly30%
  • passAddresses the actual decision79%
  • partialRespects explicit constraints25%
  • passIdentifies material uncertainty96%
  • partialAvoids unsupported claims23%
  • passProduces the required deliverable50%
  • passSees who's underpriced50%
  • passAnswers the CEO's flat rise97%
  • passEach tier for a different buyer91%
  • failGets the base of every number right35%
  • passPriced on willingness to pay68%
  • passMoves existing customers fairly100%
Run
Run
#1
API response time
57 s
Submitted
5 Oct 2026

Check by check

Where the outputs differ comes first. Open a check to read the judge’s reasons. “Mixed” means the two graders disagreed.

Where they differ 8

Uses the supplied evidence correctlyMixedWrongMixed
Opus 5.5 · Claude

All factual claims about the current situation are directly supported by the supplied data or arithmetic from it.

Gemini 3.5 Flash-Lite · Gemini

States freelancers' median too-expensive as $25/mo, but the survey data gives a median of $26.

GPT-6.1 Sol · API

Every statement about the current situation is supported by the supplied data or arithmetic, with no inventions.

Addresses the actual decisionRightWrongRight
Opus 5.5 · Claude

The output clearly commits to a three-plan structure instead of a flat $19, and states what Priya needs to approve and what timing condition would change the rollout.

Gemini 3.5 Flash-Lite · Gemini

Does not say what result or condition would change the recommendation.

GPT-6.1 Sol · API

Commits to a clear three-tier segmented pricing answer early, addresses the CEO, and says what metric would change the course (non-positive incremental revenue).

Respects explicit constraintsRightMixedRight
Opus 5.5 · Claude

The proposal includes a plan table, stays within 900 words, and addresses what to charge, to whom, and how to migrate existing customers.

Gemini 3.5 Flash-Lite · Gemini

Includes a plan table, stays under 900 words, and addresses the CEO as requested.

GPT-6.1 Sol · API

Proposal is within 900 words, includes a plan table, and avoids the prohibited flat $19 for everyone.

Identifies material uncertaintyRightWrongRight
Opus 5.5 · Claude

It names the survey's stated-preference limitation, CRM sync usage uncertainty, and freelancer free-tool risk, and says how to resolve them (watch conversion, check usage, test free tier).

Gemini 3.5 Flash-Lite · Gemini

Does not name unknowns that could change the decision or how they would be resolved.

GPT-6.1 Sol · API

Names small agency conversion base, inconsistent survey response, testability of prices, and unknown churn/seat contraction; says how to validate.

Avoids unsupported claimsRightWrongRight
Opus 5.5 · Claude

Interpretations like 'price-sensitive' and 'underpriced' are backed by data, and forecasts are presented as risks or scenarios, not established facts.

Gemini 3.5 Flash-Lite · Gemini

Presents forecasts like 'a flat hike will cause churn spikes' as fact without labelling them as hypotheses.

GPT-6.1 Sol · API

Labels survey as stated preferences, acknowledges the limits of loss data, and does not present hypotheses as facts.

Sees who's underpricedRightWrongMixed
Opus 5.5 · Claude

It uses signed value per seat and survey price points to show agencies are severely underpriced while freelancers are near their limit.

Gemini 3.5 Flash-Lite · Gemini

Does not use signed value per seat from the account sample to show agencies are underpriced relative to value delivered.

GPT-6.1 Sol · API

Does not use the signed value per seat data to show agencies extract far more value and are underpriced; mentions revenue concentration but omits the large per-seat signed values.

Answers the CEO's flat riseRightMixedRight
Opus 5.5 · Claude

It explains that $19 exceeds most freelancers' 'getting expensive' point and still sits far below agencies' willingness to pay, with churn and revenue effects.

Gemini 3.5 Flash-Lite · Gemini

Does not show that $19 is above freelancers' 'getting expensive' point ($15 median), only references 'too expensive'.

GPT-6.1 Sol · API

Shows $19 exceeds freelancer 'getting expensive' range and falls below agency thresholds, explaining why a flat rise would overcharge one and undercharge the other.

Moves existing customers fairlyRightMixedRight
Opus 5.5 · Claude

It proposes new customers first, 60 days' notice for existing ones, year-1 discounted rates tied to value, and metrics to watch churn through migration.

Gemini 3.5 Flash-Lite · Gemini

Does not specify how churn will be monitored during the transition, only offers a retention safety net.

GPT-6.1 Sol · API

Provides a clear transition plan: 90 days' notice, freelancers stay at $15, agencies move to $19 then to Team/Business, with churn monitoring.

All mixed 1

Gets the base of every number rightMixedMixedMixed
Opus 5.5 · Claude

All percentages, revenue shares, and MRR calculations check out against the sample data, and the bases (accounts, seats, MRR) are stated.

Gemini 3.5 Flash-Lite · Gemini

Derived percentages (38% churn, 9% conversion, 160% expansion, 26% lift) are correctly calculated from the data.

GPT-6.1 Sol · API

All derived percentages and rates (26.7% uplift, 21.1% cancellation threshold, conversion rates) check out against the supplied data and are computed from correct denominators.

All got right 3

Produces the required deliverableRightRightRight
Opus 5.5 · Claude

The output is a complete pricing and packaging proposal with a table, addressed to the CEO, within the word limit, and actionable with light edits.

Gemini 3.5 Flash-Lite · Gemini

Delivers a complete pricing and packaging proposal with a table, migration plan, and is usable by the CEO.

GPT-6.1 Sol · API

Contains the required plan table with prices and features, stays under 900 words, and is actionable for the CEO with light edits.

Each tier for a different buyerRightRightRight
Opus 5.5 · Claude

Each plan (Solo, Team, Agency) names its buyer and includes features that buyer values (e-signature for freelancers, shared library/approvals/CRM for teams, admin/roles for agencies).

Gemini 3.5 Flash-Lite · Gemini

Each tier names a distinct buyer (freelancer, small agency, agency) and includes features that segment values.

GPT-6.1 Sol · API

Solo, Team, and Business plans are each built for a distinct buyer (freelancer, collaborating small agency, integrated larger agency) with features that buyer values.

Priced on willingness to payRightRightRight
Opus 5.5 · Claude

Prices are set relative to survey 'getting expensive' thresholds and value metrics, with the survey's limitations acknowledged.

Gemini 3.5 Flash-Lite · Gemini

Prices are set below the relevant 'too expensive' or 'getting expensive' medians from the survey for each segment.

GPT-6.1 Sol · API

Prices are set against the survey's 'getting expensive' ranges for each segment and the output notes where evidence is thin (stated preferences, small agency conversion base).

Results

Every setup we’ve tested on this task type, across all its tasks and repeats, graded on the current checklist. Calibrated. Where our PM checked the graders by hand, they agreed on 81% of 68 checks.

#Model · HarnessTask scoreDecision modelLLM judgeRunsCritical failures
1Opus 5.5withClaude76.0100.02None
2GPT-6 AstrawithChatGPT74.096.42None
3Sonnet 5.5withAPI76.089.32None
4GPT-6.1 SolwithAPI76.088.72None
5Haiku 5.5withAPI78.076.92None
6GPT-6 LunawithAPI74.064.02None
7Gemini 3.8 FlashwithAPI62.044.22None
8Gemini 3.5 Flash-LitewithGemini45.840.72None

What you get for the cost

Sonnet 5.5 · API scores within 5 points of Opus 5.5 · Claude at half the cost.

Task score against relative cost for each setup. The table below has the same figures.304050607080901001×3×10×30×100×Cost per output, relative to the cheapest setup (log scale) →↑ Task scoreOpus 5.5 · Claude: task score 89, 122× the cheapestGPT-6 Astra · ChatGPT: task score 86, 108× the cheapestSonnet 5.5 · API: task score 84, 61× the cheapestGPT-6.1 Sol · API: task score 81, 20× the cheapestHaiku 5.5 · API: task score 76, 2.9× the cheapestGPT-6 Luna · API: task score 68, the cheapestGemini 3.8 Flash · API: task score 51, 9.1× the cheapestGemini 3.5 Flash-Lite · Gemini: task score 42, 2.3× the cheapestOpus 5.5 · ClaudeGPT-6 Astra · ChatGPTSonnet 5.5 · APIGPT-6.1 Sol · APIHaiku 5.5 · APIGPT-6 Luna · APIGemini 3.8 Flash · APIGemini 3.5 Flash-Lite · Gemini
Best valueNobody beats these on both score and cost
Task score and relative cost by setup
SetupTask scoreRelative cost (cheapest = 1)
GPT-6 Luna · API681.0×
Gemini 3.5 Flash-Lite · Gemini422.3×
Haiku 5.5 · API762.9×
Gemini 3.8 Flash · API519.1×
GPT-6.1 Sol · API8120×
Sonnet 5.5 · API8461×
GPT-6 Astra · ChatGPT86108×
Opus 5.5 · Claude89122×

Cost is each output’s tokens at the model’s list price, shown relative to the cheapest setup. Runs done in an app have no usage meter, so theirs are estimated from the same setup’s measured runs. How we compare cost

About the task

The PM job

Setting what you charge, for what, and to whom.

Why it matters

Most products are priced once and never revisited, on a metric that doesn't track value. The fix is rarely a flat rise.

What good looks like

  • Tiers built for distinct buyers
  • A value metric that tracks what customers get
  • Grounded in willingness to pay and cost to serve
  • A plan for existing customers

Deliberately not measured

  • Billing implementation
  • Legal terms
Capability tested

Pricing judgement

The failure we’re looking for

A flat price rise on the wrong value metric

Grading

Decision model and LLM judge, calibrated against a blind PM review