Humanity’s Last PRD
Latest drop: Haiku 5.5 · 8 Oct
Tasks / Product Craft

Write a PRD

Can the model turn a brief into a spec engineers could build from, including how an AI feature behaves when it is wrong?

Task type v1.2 · 2 tasksLast changed 6 Oct 2026 · ChangelogDifficulty

What AI gets right here, and what you’ll still have to catch

From 16 graded outputs by 8 models. 25% were usable with at most a quick edit.

Reliably right

  1. Specifies AI failure behaviour100% pass
    It specifies what happens when uncertain or wrong: block ownership, show retry states, disable assignments on violation, and never fall back to calendar data.
    GPT-6 Astra · ChatGPT · Meeting summaries with action items
  2. Builds the regulator's rules in100% pass
    All three rules (2-business-day acknowledge, 15-business-day resolve or explain, no harder than paying) appear as testable requirements R8, R10, R13.
    Sonnet 5.5 · API · Disputes that aren't disputes
  3. Produces the required deliverable94% pass
    The PRD is complete, written for engineering and design leads, within the 1,000–1,500 word range, and directly actionable.
    GPT-6 Astra · ChatGPT · Meeting summaries with action items

Where it slips

  1. Proposes tests that could fail36% pass
    Launch gates have numeric thresholds and actions, but lack explicit measurement windows (e.g., pilot duration, number of meetings) needed to read out results.
    GPT-6.1 Sol · API · Meeting summaries with action items
  2. Avoids unsupported claims42% pass
    Presents the join log and identity tiers as established facts without labelling them as assumptions, when the context provides no support.
    Opus 5.5 · Claude · Meeting summaries with action items
  3. Uses the supplied evidence correctly44% pass
    The output invents a join log and identity tiers (Verified, Guest, Room) that are not in the supplied context, making unsupported claims about the current system.
    Opus 5.5 · Claude · Meeting summaries with action items

How it’s graded

Each check is built on how world-class product and growth leaders do this work, as they’ve described it on Lenny’s Podcast, and names the leader it comes from.

  1. Success settles trade-offs

    Does the PRD define success as a measurable outcome with a target and time frame, and say which goal wins when two of them conflict?

    Passes when Names a success metric with a target and time frame, and gives at least one explicit trade-off rule (for example, accepting lower coverage to keep precision above a stated level).

    Product and growth leadersOn Lenny’s Podcast

  2. Says who it's for, and who it isn't

    Does the PRD name the specific users it is for, say explicitly who it is not for, and make at least one scope decision that follows from that choice?

    Passes when Names the target users and at least one group it deliberately leaves out, and a scope decision visibly follows from the exclusion.

    Matt LeMayLenny’s Podcast

  3. Plans how people will find it

    Does the PRD say how the feature reaches people: how it rolls out, how users discover it, and who needs to be ready for it (support, sales), rather than stopping at what to build?

    Passes when Gives a staged rollout, a way users will discover the feature, and the teams that need to be ready, all fitted to this feature.

    Ami VoraLenny’s PodcastKarri SaarinenLenny’s Podcast

Plus our standard checks

Uses the supplied evidence correctly · Addresses the actual decision · Respects explicit constraints · Identifies material uncertainty · Avoids unsupported claims · Produces the required deliverable · Proposes tests that could fail · and 2 written for each task, which you’ll see in the tasks below.

The tasks

Read the brief, then compare up to three outputs, each with the LLM judge’s verdict on every check. Highlights mark what a PM had to fix.

The brief

You're the PM for card servicing at Clearbank. Write the PRD for self-serve disputes in the app, as Imogen asks below. It goes to the squad (engineering, design, support operations and compliance) who'll build it next quarter. 1,000 to 1,400 words. The dispute export and the background are attached.

What the model was given6 items: About Clearbank, imogen_note.md, how_disputes_work_today.md, regulator_rules.md, disputes_sample_last_quarter.csv (a random sample of 400 of last quarter's 2,400 disputes), merchant_data_option.md
About ClearbankA digital bank for individuals: current accounts and debit cards, run through a mobile app.
imogen_note.md1 lines · Download
From Imogen (Head of Product): "Write the PRD for self-serve disputes in the app. Support says it's their biggest cost after card freezes. The PRD goes to the squad (engineering, design, support ops and compliance) who will build it next quarter. 1,000 to 1,400 words."
how_disputes_work_today.md5 lines · Download
# How disputes work today

Customers raise a dispute by chatting to support. An agent collects details, then either explains the payment or opens a chargeback. Agents spend about 11 minutes per dispute. 2,400 disputes last quarter; 14% of them were resolved later than the regulator's deadline.

Card transactions show the merchant's billing name (for example "SQ *BRKLYN BAGEL 0412"), not its trading name or logo.
regulator_rules.md5 lines · Download
# Regulator's rules (summary from Legal)

- Acknowledge a dispute within 2 business days.
- Resolve genuine disputes within 15 business days, or tell the customer why it's taking longer.
- Don't make it harder to raise a dispute than to make a payment.
disputes_sample_last_quarter.csv (a random sample of 400 of last quarter's 2,400 disputes)dispute,customer_said,days_to_resolve,outcome CB-40000,I don't recognise this payment,8,Customer recognised it once shown the merchant's details CB-40006,I didn't receive what I paid for,17,Chargeback lost CB-40012,I don't recognise this payment,6,Customer recognised it once shown the merchant's details CB-40018,I don't recognise this payment,1,Customer recognised it once shown the merchant's details CB-40024,I don't recognise this payment,7,Customer recognised it once shown the merchant's details CB-40030,I didn't receive what I paid for,11,Chargeback lost CB-40036,I don't recognise this payment,5,Customer recognised it once shown the merchant's details CB-40042,I didn't receive what I paid for,16,Chargeback lost CB-40048,I was charged twice,5,Not a duplicate: two separate purchases CB-40054,I don't recognise this payment,8,Customer recognised it once shown the merchant's details CB-40060,I was charged twice,10,Not a duplicate: two separate purchases CB-40066,I was charged twice,7,Not a duplicate: two separate purchases CB-40072,I don't recognise this payment,8,Customer recognised it once shown the merchant's details CB-40078,I don't recognise this payment,5,Fraud confirmed CB-40084,I don't recognise this payment,7,Fraud confirmed CB-40090,I don't recognise this payment,8,Customer recognised it once shown the merchant's details CB-40096,I was charged twice,6,Not a duplicate: two separate purchases CB-40102,I didn't receive what I paid for,7,Chargeback won CB-40108,I don't recognise this payment,6,Fraud confirmed CB-40114,I was charged twice,9,Duplicate refunded CB-40120,I was charged twice,7,Not a duplicate: two separate purchases CB-40126,I don't recognise this payment,5,Customer recognised it once shown the merchant's details CB-40132,I was charged twice,9,Duplicate refunded CB-40138,Subscription I cancelled,18,Chargeback won CB-40144,I didn't receive what I paid for,21,Chargeback won CB-40150,I don't recognise this payment,6,Customer recognised it once shown the merchant's details CB-40156,I don't recognise this payment,5,Customer recognised it once shown the merchant's details CB-40162,I don't recognise this payment,6,Fraud confirmed CB-40168,I was charged twice,4,Not a duplicate: two separate purchases CB-40174,I don't recognise this payment,6,Customer recognised it once shown the merchant's details CB-40180,Fraud: I didn't make this payment,10,"Fraud confirmed, refunded" CB-40186,I don't recognise this payment,1,Customer recognised it once shown the merchant's details CB-40192,I don't recognise this payment,3,Customer recognised it once shown the merchant's details CB-40198,I don't recognise this payment,1,Fraud confirmed CB-40204,I don't recognise this payment,5,Customer recognised it once shown the merchant's details CB-40210,I don't recognise this payment,2,Customer recognised it once shown the merchant's details CB-40216,I don't recognise this payment,5,Customer recognised it once shown the merchant's details CB-40222,I don't recognise this payment,7,Customer recognised it once shown the merchant's details CB-40228,I didn't receive what I paid for,29,Chargeback won CB-40234,I don't recognise this payment,8,Fraud confirmed CB-40240,I don't recognise this payment,6,Customer recognised it once shown the merchant's details CB-40246,I didn't receive what I paid for,10,Merchant refunded CB-40252,I don't recognise this payment,4,Fraud confirmed CB-40258,I don't recognise this payment,1,Customer recognised it once shown the merchant's details CB-40264,I don't recognise this payment,1,Customer recognised it once shown the merchant's details CB-40270,I didn't receive what I paid for,21,Chargeback won CB-40276,I don't recognise this payment,4,Customer recognised it once shown the merchant's details CB-40282,I didn't receive what I paid for,17,Merchant refunded CB-40288,I don't recognise this payment,2,Customer recognised it once shown the merchant's details CB-40294,I don't recognise this payment,1,Fraud confirmed CB-40300,I don't recognise this payment,7,Fraud confirmed CB-40306,I don't recognise this payment,8,Fraud confirmed CB-40312,I didn't receive what I paid for,20,Merchant refunded CB-40318,I didn't receive what I paid for,26,Chargeback lost CB-40324,Subscription I cancelled,10,Merchant refunded CB-40330,I don't recognise this payment,1,Fraud confirmed CB-40336,Fraud: I didn't make this payment,10,"Fraud confirmed, refunded" CB-40342,I don't recognise this payment,7,Customer recognised it once shown the merchant's details CB-40348,Fraud: I didn't make this payment,9,"Fraud confirmed, refunded" CB-40354,I don't recognise this payment,4,Customer recognised it once shown the merchant's details CB-40360,I was charged twice,9,Not a duplicate: two separate purchases CB-40366,I don't recognise this payment,1,Customer recognised it once shown the merchant's details CB-40372,I don't recognise this payment,2,Customer recognised it once shown the merchant's details CB-40378,I don't recognise this payment,8,Fraud confirmed CB-40384,I didn't receive what I paid for,14,Merchant refunded CB-40390,I don't recognise this payment,7,Fraud confirmed CB-40396,I don't recognise this payment,8,Fraud confirmed CB-40402,I don't recognise this payment,1,Customer recognised it once shown the merchant's details CB-40408,I was charged twice,5,Not a duplicate: two separate purchases CB-40414,I didn't receive what I paid for,13,Chargeback lost CB-40420,Subscription I cancelled,11,Chargeback won CB-40426,Fraud: I didn't make this payment,10,"Fraud confirmed, refunded" CB-40432,I didn't receive what I paid for,13,Chargeback won CB-40438,Fraud: I didn't make this payment,4,"Fraud confirmed, refunded" CB-40444,I was charged twice,9,Duplicate refunded CB-40450,I didn't receive what I paid for,18,Chargeback won CB-40456,I don't recognise this payment,1,Fraud confirmed CB-40462,I don't recognise this payment,4,Fraud confirmed CB-40468,I don't recognise this payment,2,Customer recognised it once shown the merchant's details CB-40474,I don't recognise this payment,5,Customer recognised it once shown the merchant's details CB-40480,Fraud: I didn't make this payment,8,"Fraud confirmed, refunded" CB-40486,I didn't receive what I paid for,27,Chargeback lost CB-40492,I don't recognise this payment,4,Customer recognised it once shown the merchant's details CB-40498,I don't recognise this payment,5,Customer recognised it once shown the merchant's details CB-40504,I don't recognise this payment,7,Customer recognised it once shown the merchant's details CB-40510,I don't recognise this payment,5,Customer recognised it once shown the merchant's details CB-40516,I don't recognise this payment,5,Fraud confirmed CB-40522,I don't recognise this payment,5,Customer recognised it once shown the merchant's details CB-40528,Fraud: I didn't make this payment,8,"Fraud confirmed, refunded" CB-40534,I don't recognise this payment,2,Customer recognised it once shown the merchant's details CB-40540,I don't recognise this payment,4,Customer recognised it once shown the merchant's details CB-40546,I don't recognise this payment,2,Fraud confirmed CB-40552,I don't recognise this payment,7,Customer recognised it once shown the merchant's details CB-40558,I was charged twice,5,Duplicate refunded CB-40564,I don't recognise this payment,6,Customer recognised it once shown the merchant's details CB-40570,I don't recognise this payment,2,Customer recognised it once shown the merchant's details CB-40576,I don't recognise this payment,6,Customer recognised it once shown the merchant's details CB-40582,I didn't receive what I paid for,14,Chargeback lost CB-40588,I didn't receive what I paid for,18,Chargeback lost CB-40594,I don't recognise this payment,1,Customer recognised it once shown the merchant's details CB-40600,Fraud: I didn't make this payment,2,"Fraud confirmed, refunded" CB-40606,I don't recognise this payment,5,Customer recognised it once shown the merchant's details CB-40612,I don't recognise this payment,3,Fraud confirmed CB-40618,Fraud: I didn't make this payment,5,"Fraud confirmed, refunded" CB-40624,I don't recognise this payment,4,Customer recognised it once shown the merchant's details CB-40630,I don't recognise this payment,2,Customer recognised it once shown the merchant's details CB-40636,I don't recognise this payment,5,Customer recognised it once shown the merchant's details CB-40642,Fraud: I didn't make this payment,11,"Fraud confirmed, refunded" CB-40648,I don't recognise this payment,7,Customer recognised it once shown the merchant's details CB-40654,I don't recognise this payment,5,Fraud confirmed CB-40660,I don't recognise this payment,8,Customer recognised it once shown the merchant's details CB-40666,I don't recognise this payment,6,Fraud confirmed CB-40672,I was charged twice,5,Not a duplicate: two separate purchases CB-40678,Subscription I cancelled,10,Chargeback won CB-40684,Fraud: I didn't make this payment,9,"Fraud confirmed, refunded" CB-40690,Subscription I cancelled,7,Chargeback won CB-40696,I don't recognise this payment,6,Customer recognised it once shown the merchant's details CB-40702,I don't recognise this payment,5,Customer recognised it once shown the merchant's details CB-40708,I don't recognise this payment,8,Customer recognised it once shown the merchant's details CB-40714,I don't recognise this payment,7,Customer recognised it once shown the merchant's details CB-40720,I was charged twice,13,Not a duplicate: two separate purchases CB-40726,Subscription I cancelled,13,Chargeback won CB-40732,I don't recognise this payment,3,Fraud confirmed CB-40738,I didn't receive what I paid for,18,Chargeback lost CB-40744,Fraud: I didn't make this payment,8,"Fraud confirmed, refunded" CB-40750,I was charged twice,11,Not a duplicate: two separate purchases CB-40756,Subscription I cancelled,10,Merchant refunded CB-40762,I was charged twice,7,Duplicate refunded CB-40768,Subscription I cancelled,18,Chargeback won CB-40774,I didn't receive what I paid for,23,Chargeback won CB-40780,I didn't receive what I paid for,7,Chargeback won CB-40786,Subscription I cancelled,5,Chargeback won CB-40792,I didn't receive what I paid for,10,Merchant refunded CB-40798,I was charged twice,9,Duplicate refunded CB-40804,I didn't receive what I paid for,24,Merchant refunded CB-40810,I don't recognise this payment,2,Fraud confirmed CB-40816,Fraud: I didn't make this payment,11,"Fraud confirmed, refunded" CB-40822,I don't recognise this payment,4,Customer recognised it once shown the merchant's details CB-40828,Subscription I cancelled,13,Chargeback won CB-40834,I don't recognise this payment,2,Customer recognised it once shown the merchant's details CB-40840,I don't recognise this payment,7,Fraud confirmed CB-40846,Subscription I cancelled,7,Merchant refunded CB-40852,I don't recognise this payment,3,Fraud confirmed CB-40858,I don't recognise this payment,6,Customer recognised it once shown the merchant's details CB-40864,I was charged twice,4,Not a duplicate: two separate purchases CB-40870,I don't recognise this payment,8,Fraud confirmed CB-40876,I was charged twice,6,Not a duplicate: two separate purchases CB-40882,I don't recognise this payment,3,Customer recognised it once shown the merchant's details CB-40888,I don't recognise this payment,8,Fraud confirmed CB-40894,I don't recognise this payment,6,Fraud confirmed CB-40900,I don't recognise this payment,8,Customer recognised it once shown the merchant's details CB-40906,I don't recognise this payment,7,Customer recognised it once shown the merchant's details CB-40912,I don't recognise this payment,8,Fraud confirmed CB-40918,I didn't receive what I paid for,21,Chargeback won CB-40924,Fraud: I didn't make this payment,10,"Fraud confirmed, refunded" CB-40930,Fraud: I didn't make this payment,8,"Fraud confirmed, refunded" CB-40936,I don't recognise this payment,8,Customer recognised it once shown the merchant's details CB-40942,I was charged twice,6,Duplicate refunded CB-40948,I don't recognise this payment,6,Customer recognised it once shown the merchant's details CB-40954,I didn't receive what I paid for,14,Chargeback won CB-40960,I don't recognise this payment,7,Customer recognised it once shown the merchant's details CB-40966,I was charged twice,4,Not a duplicate: two separate purchases CB-40972,I don't recognise this payment,7,Customer recognised it once shown the merchant's details CB-40978,I don't recognise this payment,4,Customer recognised it once shown the merchant's details CB-40984,I didn't receive what I paid for,9,Merchant refunded CB-40990,I don't recognise this payment,1,Customer recognised it once shown the merchant's details CB-40996,I don't recognise this payment,8,Customer recognised it once shown the merchant's details CB-41002,I don't recognise this payment,1,Customer recognised it once shown the merchant's details CB-41008,I don't recognise this payment,1,Customer recognised it once shown the merchant's details CB-41014,Fraud: I didn't make this payment,2,"Fraud confirmed, refunded" CB-41020,I didn't receive what I paid for,27,Chargeback won CB-41026,I didn't receive what I paid for,28,Chargeback won CB-41032,I was charged twice,7,Duplicate refunded CB-41038,I don't recognise this payment,2,Fraud confirmed CB-41044,I don't recognise this payment,2,Customer recognised it once shown the merchant's details CB-41050,I don't recognise this payment,4,Fraud confirmed CB-41056,Fraud: I didn't make this payment,11,"Fraud confirmed, refunded" CB-41062,I was charged twice,3,Duplicate refunded CB-41068,I don't recognise this payment,1,Fraud confirmed CB-41074,I don't recognise this payment,1,Fraud confirmed CB-41080,I don't recognise this payment,1,Customer recognised it once shown the merchant's details CB-41086,I don't recognise this payment,6,Customer recognised it once shown the merchant's details CB-41092,I don't recognise this payment,1,Customer recognised it once shown the merchant's details CB-41098,I don't recognise this payment,8,Customer recognised it once shown the merchant's details CB-41104,I didn't receive what I paid for,29,Merchant refunded CB-41110,Subscription I cancelled,16,Chargeback won CB-41116,I didn't receive what I paid for,22,Chargeback won CB-41122,I didn't receive what I paid for,13,Chargeback lost CB-41128,I don't recognise this payment,8,Customer recognised it once shown the merchant's details CB-41134,I was charged twice,10,Not a duplicate: two separate purchases CB-41140,I don't recognise this payment,2,Customer recognised it once shown the merchant's details CB-41146,I don't recognise this payment,7,Customer recognised it once shown the merchant's details CB-41152,I was charged twice,5,Duplicate refunded CB-41158,I don't recognise this payment,4,Customer recognised it once shown the merchant's details CB-41164,I didn't receive what I paid for,12,Chargeback lost CB-41170,I don't recognise this payment,1,Customer recognised it once shown the merchant's details CB-41176,Fraud: I didn't make this payment,2,"Fraud confirmed, refunded" CB-41182,I don't recognise this payment,1,Fraud confirmed CB-41188,I don't recognise this payment,3,Fraud confirmed CB-41194,I don't recognise this payment,6,Customer recognised it once shown the merchant's details CB-41200,I don't recognise this payment,6,Customer recognised it once shown the merchant's details CB-41206,Fraud: I didn't make this payment,5,"Fraud confirmed, refunded" CB-41212,I don't recognise this payment,6,Customer recognised it once shown the merchant's details CB-41218,Fraud: I didn't make this payment,6,"Fraud confirmed, refunded" CB-41224,I don't recognise this payment,4,Customer recognised it once shown the merchant's details CB-41230,I don't recognise this payment,1,Customer recognised it once shown the merchant's details CB-41236,I don't recognise this payment,3,Fraud confirmed CB-41242,Subscription I cancelled,9,Merchant refunded CB-41248,I don't recognise this payment,6,Customer recognised it once shown the merchant's details CB-41254,I don't recognise this payment,7,Customer recognised it once shown the merchant's details CB-41260,I don't recognise this payment,5,Customer recognised it once shown the merchant's details CB-41266,I don't recognise this payment,8,Customer recognised it once shown the merchant's details CB-41272,I don't recognise this payment,2,Customer recognised it once shown the merchant's details CB-41278,I didn't receive what I paid for,11,Chargeback lost CB-41284,I don't recognise this payment,1,Customer recognised it once shown the merchant's details CB-41290,I don't recognise this payment,4,Customer recognised it once shown the merchant's details CB-41296,I don't recognise this payment,3,Customer recognised it once shown the merchant's details CB-41302,I don't recognise this payment,5,Customer recognised it once shown the merchant's details CB-41308,I was charged twice,11,Duplicate refunded CB-41314,I didn't receive what I paid for,27,Chargeback won CB-41320,I don't recognise this payment,8,Customer recognised it once shown the merchant's details CB-41326,Fraud: I didn't make this payment,5,"Fraud confirmed, refunded" CB-41332,I was charged twice,12,Not a duplicate: two separate purchases CB-41338,I don't recognise this payment,3,Customer recognised it once shown the merchant's details CB-41344,I didn't receive what I paid for,17,Merchant refunded CB-41350,I didn't receive what I paid for,21,Chargeback won CB-41356,I don't recognise this payment,4,Customer recognised it once shown the merchant's details CB-41362,Subscription I cancelled,14,Merchant refunded CB-41368,I don't recognise this payment,1,Customer recognised it once shown the merchant's details CB-41374,I don't recognise this payment,4,Customer recognised it once shown the merchant's details CB-41380,I don't recognise this payment,5,Customer recognised it once shown the merchant's details CB-41386,I don't recognise this payment,2,Customer recognised it once shown the merchant's details CB-41392,I don't recognise this payment,2,Customer recognised it once shown the merchant's details CB-41398,I don't recognise this payment,2,Fraud confirmed CB-41404,I don't recognise this payment,5,Customer recognised it once shown the merchant's details CB-41410,I don't recognise this payment,3,Fraud confirmed CB-41416,I don't recognise this payment,5,Fraud confirmed CB-41422,I was charged twice,12,Duplicate refunded CB-41428,Subscription I cancelled,9,Merchant refunded CB-41434,I was charged twice,6,Duplicate refunded CB-41440,Subscription I cancelled,15,Merchant refunded CB-41446,I don't recognise this payment,5,Fraud confirmed CB-41452,I don't recognise this payment,7,Fraud confirmed CB-41458,I don't recognise this payment,4,Customer recognised it once shown the merchant's details CB-41464,I don't recognise this payment,1,Customer recognised it once shown the merchant's details CB-41470,I was charged twice,6,Not a duplicate: two separate purchases CB-41476,I don't recognise this payment,5,Fraud confirmed CB-41482,I don't recognise this payment,4,Customer recognised it once shown the merchant's details CB-41488,I don't recognise this payment,4,Customer recognised it once shown the merchant's details CB-41494,I didn't receive what I paid for,26,Merchant refunded CB-41500,I didn't receive what I paid for,24,Chargeback won CB-41506,Subscription I cancelled,20,Chargeback won CB-41512,I didn't receive what I paid for,29,Merchant refunded CB-41518,Subscription I cancelled,20,Chargeback won CB-41524,I don't recognise this payment,8,Fraud confirmed CB-41530,I was charged twice,3,Not a duplicate: two separate purchases CB-41536,I didn't receive what I paid for,8,Chargeback won CB-41542,I don't recognise this payment,2,Fraud confirmed CB-41548,I don't recognise this payment,2,Fraud confirmed CB-41554,I was charged twice,12,Not a duplicate: two separate purchases CB-41560,Subscription I cancelled,6,Chargeback won CB-41566,I was charged twice,3,Not a duplicate: two separate purchases CB-41572,I don't recognise this payment,1,Fraud confirmed CB-41578,Fraud: I didn't make this payment,7,"Fraud confirmed, refunded" CB-41584,I don't recognise this payment,7,Customer recognised it once shown the merchant's details CB-41590,I don't recognise this payment,3,Customer recognised it once shown the merchant's details CB-41596,Subscription I cancelled,16,Merchant refunded CB-41602,Subscription I cancelled,15,Merchant refunded CB-41608,I don't recognise this payment,2,Customer recognised it once shown the merchant's details CB-41614,I don't recognise this payment,4,Customer recognised it once shown the merchant's details CB-41620,I was charged twice,5,Duplicate refunded CB-41626,I don't recognise this payment,1,Customer recognised it once shown the merchant's details CB-41632,I don't recognise this payment,8,Customer recognised it once shown the merchant's details CB-41638,Fraud: I didn't make this payment,3,"Fraud confirmed, refunded" CB-41644,I don't recognise this payment,3,Fraud confirmed CB-41650,I don't recognise this payment,2,Customer recognised it once shown the merchant's details CB-41656,I don't recognise this payment,6,Fraud confirmed CB-41662,I don't recognise this payment,1,Customer recognised it once shown the merchant's details CB-41668,I didn't receive what I paid for,12,Chargeback won CB-41674,I don't recognise this payment,8,Customer recognised it once shown the merchant's details CB-41680,I don't recognise this payment,7,Customer recognised it once shown the merchant's details CB-41686,I don't recognise this payment,3,Customer recognised it once shown the merchant's details CB-41692,I don't recognise this payment,2,Customer recognised it once shown the merchant's details CB-41698,I don't recognise this payment,2,Customer recognised it once shown the merchant's details CB-41704,I don't recognise this payment,7,Customer recognised it once shown the merchant's details CB-41710,I don't recognise this payment,2,Customer recognised it once shown the merchant's details CB-41716,Fraud: I didn't make this payment,8,"Fraud confirmed, refunded" CB-41722,I don't recognise this payment,3,Fraud confirmed CB-41728,I was charged twice,12,Duplicate refunded CB-41734,Fraud: I didn't make this payment,5,"Fraud confirmed, refunded" CB-41740,I don't recognise this payment,6,Fraud confirmed CB-41746,I don't recognise this payment,4,Customer recognised it once shown the merchant's details CB-41752,I don't recognise this payment,7,Customer recognised it once shown the merchant's details CB-41758,I was charged twice,13,Duplicate refunded CB-41764,I don't recognise this payment,1,Customer recognised it once shown the merchant's details CB-41770,I didn't receive what I paid for,23,Chargeback won CB-41776,I don't recognise this payment,4,Customer recognised it once shown the merchant's details CB-41782,I didn't receive what I paid for,27,Chargeback won CB-41788,I didn't receive what I paid for,23,Chargeback lost CB-41794,I don't recognise this payment,6,Customer recognised it once shown the merchant's details CB-41800,I don't recognise this payment,3,Fraud confirmed CB-41806,I don't recognise this payment,1,Fraud confirmed CB-41812,I was charged twice,4,Not a duplicate: two separate purchases CB-41818,I don't recognise this payment,8,Fraud confirmed CB-41824,I didn't receive what I paid for,18,Chargeback won CB-41830,Subscription I cancelled,13,Chargeback won CB-41836,I don't recognise this payment,7,Customer recognised it once shown the merchant's details CB-41842,I don't recognise this payment,7,Fraud confirmed CB-41848,I don't recognise this payment,5,Customer recognised it once shown the merchant's details CB-41854,I don't recognise this payment,6,Customer recognised it once shown the merchant's details CB-41860,I was charged twice,5,Duplicate refunded CB-41866,I don't recognise this payment,5,Customer recognised it once shown the merchant's details CB-41872,I don't recognise this payment,8,Fraud confirmed CB-41878,I don't recognise this payment,2,Fraud confirmed CB-41884,I don't recognise this payment,4,Customer recognised it once shown the merchant's details CB-41890,I was charged twice,7,Not a duplicate: two separate purchases CB-41896,I don't recognise this payment,5,Customer recognised it once shown the merchant's details CB-41902,Fraud: I didn't make this payment,9,"Fraud confirmed, refunded" CB-41908,I don't recognise this payment,4,Fraud confirmed CB-41914,I don't recognise this payment,2,Customer recognised it once shown the merchant's details CB-41920,I didn't receive what I paid for,24,Merchant refunded CB-41926,I don't recognise this payment,4,Customer recognised it once shown the merchant's details CB-41932,Fraud: I didn't make this payment,10,"Fraud confirmed, refunded" CB-41938,Subscription I cancelled,15,Chargeback won CB-41944,I didn't receive what I paid for,13,Chargeback won CB-41950,I don't recognise this payment,2,Fraud confirmed CB-41956,I didn't receive what I paid for,24,Chargeback lost CB-41962,I don't recognise this payment,4,Fraud confirmed CB-41968,I don't recognise this payment,8,Customer recognised it once shown the merchant's details CB-41974,Subscription I cancelled,15,Chargeback won CB-41980,I don't recognise this payment,1,Customer recognised it once shown the merchant's details CB-41986,Fraud: I didn't make this payment,2,"Fraud confirmed, refunded" CB-41992,Fraud: I didn't make this payment,6,"Fraud confirmed, refunded" CB-41998,I don't recognise this payment,8,Customer recognised it once shown the merchant's details CB-42004,I didn't receive what I paid for,12,Merchant refunded CB-42010,I don't recognise this payment,5,Customer recognised it once shown the merchant's details CB-42016,I don't recognise this payment,5,Customer recognised it once shown the merchant's details CB-42022,I was charged twice,11,Duplicate refunded CB-42028,I didn't receive what I paid for,25,Merchant refunded CB-42034,I didn't receive what I paid for,10,Merchant refunded CB-42040,I don't recognise this payment,5,Customer recognised it once shown the merchant's details CB-42046,I was charged twice,8,Duplicate refunded CB-42052,I don't recognise this payment,6,Customer recognised it once shown the merchant's details CB-42058,I don't recognise this payment,7,Fraud confirmed CB-42064,I don't recognise this payment,7,Customer recognised it once shown the merchant's details CB-42070,I don't recognise this payment,5,Customer recognised it once shown the merchant's details CB-42076,Fraud: I didn't make this payment,10,"Fraud confirmed, refunded" CB-42082,I don't recognise this payment,2,Customer recognised it once shown the merchant's details CB-42088,I don't recognise this payment,7,Customer recognised it once shown the merchant's details CB-42094,Subscription I cancelled,12,Chargeback won CB-42100,I don't recognise this payment,3,Customer recognised it once shown the merchant's details CB-42106,I don't recognise this payment,6,Customer recognised it once shown the merchant's details CB-42112,Subscription I cancelled,18,Chargeback won CB-42118,Fraud: I didn't make this payment,6,"Fraud confirmed, refunded" CB-42124,I don't recognise this payment,5,Fraud confirmed CB-42130,I didn't receive what I paid for,14,Chargeback lost CB-42136,I don't recognise this payment,8,Customer recognised it once shown the merchant's details CB-42142,Fraud: I didn't make this payment,2,"Fraud confirmed, refunded" CB-42148,I was charged twice,12,Not a duplicate: two separate purchases CB-42154,I don't recognise this payment,2,Customer recognised it once shown the merchant's details CB-42160,I don't recognise this payment,7,Customer recognised it once shown the merchant's details CB-42166,I don't recognise this payment,5,Customer recognised it once shown the merchant's details CB-42172,Fraud: I didn't make this payment,9,"Fraud confirmed, refunded" CB-42178,I don't recognise this payment,3,Fraud confirmed CB-42184,I don't recognise this payment,3,Customer recognised it once shown the merchant's details CB-42190,I was charged twice,7,Duplicate refunded CB-42196,I don't recognise this payment,1,Customer recognised it once shown the merchant's details CB-42202,I don't recognise this payment,2,Customer recognised it once shown the merchant's details CB-42208,I didn't receive what I paid for,21,Chargeback won CB-42214,I don't recognise this payment,6,Customer recognised it once shown the merchant's details CB-42220,I don't recognise this payment,3,Customer recognised it once shown the merchant's details CB-42226,I don't recognise this payment,6,Customer recognised it once shown the merchant's details CB-42232,I don't recognise this payment,1,Customer recognised it once shown the merchant's details CB-42238,I was charged twice,10,Duplicate refunded CB-42244,I don't recognise this payment,3,Customer recognised it once shown the merchant's details CB-42250,Subscription I cancelled,11,Chargeback won CB-42256,Fraud: I didn't make this payment,3,"Fraud confirmed, refunded" CB-42262,I don't recognise this payment,4,Customer recognised it once shown the merchant's details CB-42268,I didn't receive what I paid for,16,Merchant refunded CB-42274,I was charged twice,3,Duplicate refunded CB-42280,Fraud: I didn't make this payment,9,"Fraud confirmed, refunded" CB-42286,I was charged twice,11,Duplicate refunded CB-42292,I was charged twice,9,Not a duplicate: two separate purchases CB-42298,I don't recognise this payment,6,Customer recognised it once shown the merchant's details CB-42304,I don't recognise this payment,2,Customer recognised it once shown the merchant's details CB-42310,Subscription I cancelled,16,Merchant refunded CB-42316,I don't recognise this payment,4,Customer recognised it once shown the merchant's details CB-42322,I don't recognise this payment,4,Customer recognised it once shown the merchant's details CB-42328,Fraud: I didn't make this payment,3,"Fraud confirmed, refunded" CB-42334,I don't recognise this payment,4,Customer recognised it once shown the merchant's details CB-42340,I don't recognise this payment,1,Customer recognised it once shown the merchant's details CB-42346,I didn't receive what I paid for,19,Merchant refunded CB-42352,I was charged twice,12,Duplicate refunded CB-42358,I don't recognise this payment,3,Customer recognised it once shown the merchant's details CB-42364,I don't recognise this payment,7,Customer recognised it once shown the merchant's details CB-42370,I don't recognise this payment,3,Fraud confirmed CB-42376,I didn't receive what I paid for,17,Chargeback won CB-42382,I don't recognise this payment,4,Customer recognised it once shown the merchant's details CB-42388,I don't recognise this payment,5,Customer recognised it once shown the merchant's details CB-42394,I don't recognise this payment,5,Customer recognised it once shown the merchant's details
merchant_data_option.md3 lines · Download
# Merchant data we could use

Our card processor offers an enriched merchant feed (trading name, logo, location, website) for 94% of transactions, at $0.0008 per transaction. We process about 9 million card transactions a month.
What a strong answer doesThe answer key the graders mark against

Starts from the data: in the sample, 232 of 400 disputes (58%, so about 1,400 of the quarter's 2,400) were 'I don't recognise this payment', and 169 of those (about three in four) were resolved simply by showing the merchant's details; so the first thing to build is recognisable transactions (trading name, logo, location from the enriched feed, about $7,200 a month for 9 million transactions), shown before someone raises a dispute, with self-serve disputes for the genuine cases. Defines success with targets (disputes per 1,000 transactions, share resolved without an agent, regulator deadlines met against 14% late today) and says which wins when they conflict (never make disputing harder than paying, per the regulator). Covers the flows for genuine disputes (charged twice, not received, cancelled subscription, fraud) with acknowledgement within 2 business days and resolution within 15, what happens when the merchant data is missing (6% of transactions), and how the rollout reaches people, with support ready. Says who it's not for (for example business accounts, or disputes over a set amount that still go to an agent) and what's out of scope.

Critical failures (cap the score at 40)
  • Adds steps that make raising a dispute harder than making a payment
Task

Staff-level · v1.0

The outputs

8 setups answered this brief.

Sonnet 5.5API83Usable with a quick editGPT-6 AstraChatGPT73Needs more workGPT-6 LunaAPI69Needs more workHaiku 5.5API69Needs more workOpus 5.5Claude60Needs more workGPT-6.1 SolAPI58Needs more workGemini 3.5 Flash-LiteGemini40Needs more workGemini 3.8 FlashAPI37Needs more work
Gemini 3.8 FlashAPI

Needs more than a quick edit3 of 12 checks passed

The PRD contains factual errors in the sample analysis, omits key scope exclusions, lacks a rollout plan, and does not address trade-offs—requiring more than light edits before a squad could act on it.

Output

Product Requirements Document (PRD): Self-Serve Card Disputes

Author: PM, Card Servicing Audience: Squad (Engineering, Product Design, Support Operations, Compliance) Target Delivery: Q3 Status: Ready for Review

---

1. Executive Summary & Problem Context

At Clearbank, disputing a card transaction currently requires initiating an in-app chat with Support. Agents spend an average of 11 minutes per dispute manually gathering transaction details, explaining billing descriptors, or submitting chargebacks. Last quarter, Support handled 2,400 disputes (~440 agent hours), making disputes the single largest operating cost after card freezes. Crucially, 14% of disputes breached the regulator’s 15-business-day resolution window, exposing Clearbank to regulatory enforcement.

An analysis of 400 sampled disputes from last quarter reveals four distinct problem drivers: 1. Unrecognised Merchant Descriptors (48.5% of sample): Customers dispute cryptic billing names (e.g., `SQ *BRKLYN BAGEL 0412`). In nearly every case, the customer abandoned the dispute once the agent clarified the merchant’s trading name and location. 2. Duplicate Charges (11.5% of sample): Split between genuine duplicate settlements and two separate purchases made in quick succession. 3. Merchant & Subscription Issues (19.8% of sample): Goods not received, defective services, or cancelled subscriptions. These drive the longest resolution cycles (often 18–29 days), causing the bulk of regulatory SLA breaches because customers were not notified when investigations ran past 15 business days. 4. Confirmed Fraud (20.2% of sample): Unauthorised card use requiring immediate card compromise handling (freeze/cancel) and immediate provisional credit or refund.

Strategic Objectives

  • Deflect ~45% of dispute volume before submission by surfacing clear merchant data and transaction context.
  • Eliminate manual intake time by replacing chat conversations with a structured, automated in-app submission flow.
  • Achieve 100% regulatory compliance across all three mandates: sub-2-day acknowledgement, automated interim updates before day 15, and an intake friction level lower than making a payment.

---

2. Regulatory Compliance Framework

Compliance is a non-negotiable architectural constraint. Every user journey and automated trigger must map directly to regulatory rules:

Regulatory RuleProduct ImplementationAcceptance Criteria
:---:---:---
1. Acknowledge within 2 business daysAutomated push notification, in-app inbox message, and email confirmation triggered immediately upon submission.Sent within < 60 seconds of dispute submission (Target: 100%).
2. Resolve within 15 business days or explain delayAutomated tracking engine monitors business days. At Day 10, if uncompleted, an interim update is automatically dispatched explaining why the merchant review is pending.0% of disputes cross Day 15 without a formal written status update.
3. Friction parity (Not harder to dispute than to pay)1-tap entry directly from the transaction detail screen; pre-filled details; maximum 3 guided screens to submit.Submission flow completable in < 90 seconds without entering chat.

---

3. Merchant Data Feed Integration: Decision & Business Case

Clearbank’s card processor offers an enriched merchant feed (clean trading name, consumer-facing logo, geolocation, and website) covering 94% of card transactions at $0.0008 per transaction.

  • Monthly Transaction Volume: ~9,000,000
  • Monthly Feed Cost: $7,200 ($21,600/quarter; $86,400/year)
  • Dispute Deflection Value: ~1,160 disputes/quarter avoided (48.5% of 2,400). At 11 minutes per dispute, this saves 213 agent hours quarterly. Beyond direct support savings, this reduces card reissuance costs from panicked users, improves customer trust, and lifts app engagement across all 9 million monthly transactions.
  • Recommendation: Adopt the feed platform-wide. Integrate enriched data into the core transaction feed as well as the dispute intake screens.

---

4. User Journeys & Functional Requirements

``` [Transaction Detail Screen] │ ▼ ["Dispute this payment"] │ ▼ ┌────────────────────────────────────────┐ │ Step 1: Enriched Context & Deflection │ │ - Clean name, logo, map, website │ │ - "Does this look familiar?" │ └───────────────────┬────────────────────┘ │ Still unrecognised / issue ▼ ┌────────────────────────────────────────┐ │ Step 2: Dynamic Dispute Categorisation│ │ 1. I don't recognise this (Fraud) │ │ 2. Charged twice / wrong amount │ │ 3. Issue with merchant/order │ │ 4. Cancelled subscription │ └───────────────────┬────────────────────┘ │ ▼ ┌────────────────────────────────────────┐ │ Step 3: Tailored Evidence & Actions │ │ - Fraud: Auto-freeze card toggle │ │ - Duplicate: Select other charge │ │ - Merchant: Date cancelled / receipts │ └───────────────────┬────────────────────┘ │ ▼ ┌────────────────────────────────────────┐ │ Step 4: Instant Confirmation & Tracker│ │ - Immediate legal acknowledgment │ │ - Live in-app SLA status timeline │ └────────────────────────────────────────┘ ```

4.1 Entry Point: Transaction Details

  • Located on every settled and pending card transaction detail view via a clear secondary action button: "Report a problem / Dispute".
  • Available only to the primary cardholder. Disabled for internal account transfers.

4.2 Step 1: Pre-Dispute Deflection Screen

  • Displays enriched merchant trading name, Google Maps pin of physical terminal or merchant domain, verified logo, and official contact phone/web details.
  • For transactions where the descriptor differs significantly from the trading name, display a helper badge: "Billing descriptor on file: [Raw String]".
  • Action buttons: "Yes, I recognise this" (exits flow) and "Continue dispute".

4.3 Step 2: Categorisation & Guided Dynamic Intake

If the customer continues, the flow asks: "What went wrong?"

#### Category A: "I suspect fraud / I didn't authorise this" * Requirements: * System prompts customer to confirm whether they possess the physical debit card. * Auto-recommends immediate card freeze via a pre-checked toggle: "Freeze this card now to prevent further unauthorized charges". * Collects date/time customer noticed the charge and any travel context. * System flags case as `FRAUDHIGHPRIORITY`.

#### Category B: "I was charged twice" * Requirements: * System queries transaction history for identical or near-identical amounts at the same merchant within ±7 days. * Presents potential matching duplicates. Customer selects the corresponding charge. * If the system detects two authorizations with distinct timestamps and receipt identifiers, it displays an educational alert: "We found two distinct purchases on this date. Did you intend to buy both?"

#### Category C: "Problem with merchant or order" (Goods not received / damaged) * Requirements: * Form requires: Expected delivery date, summary of merchant contact attempts, and merchant response. * Document upload capability (photos, delivery tracking screenshots, cancellation receipts). Accepted formats: PDF, JPG, PNG (up to 10MB).

#### Category D: "Cancelled subscription" * Requirements: * Form collects: Date subscription was cancelled, cancellation confirmation number or email screenshot, and merchant terms agreed to.

4.4 Step 3: Submission & Automated Regulatory Acknowledgement

  • Customer reviews structured summary and taps "Submit Dispute".
  • Engine Action:
  • Generates unique Dispute ID (e.g., `CB-DISP-XXXXX`).
  • Dispatches instant Push Notification, In-App Inbox confirmation, and transactional Email stating: receipt confirmation, regulatory 15-day timeline, and next steps.
  • Sets status to `ACKNOWLEDGED`.

4.5 Step 4: Self-Service Dispute Tracker

* A dedicated "Disputes & Claims" hub in the app profile and linked directly from the disputed transaction. * Features a visual 4-stage stepper: 1. Submitted & Acknowledged 2. Under Review by Clearbank 3. Awaiting Merchant Response (Chargeback Filed) 4. Resolved * Displays estimated resolution deadline (target: 15 business days).

---

5. Automated Regulatory SLA & Delay Notification Engine

To eradicate the 14% SLA breach rate, the backend service will run an automated business-day tracker against every active case:

``` Day 0: Dispute Submitted ────────> Instant Legal Acknowledgment Sent │ Day 10 (Business Days): Case Open? ───┴──> Automated Trigger: - Push Notification + Email - Explains ongoing merchant inquiry - Gives updated completion estimate │ Day 15 (Business Days): Case Open? ──────> SLA Escalation to Ops Lead + Final Legal Extension Notice ```

* Day 10 Trigger: If a case status is not `RESOLVED` by 10 business days, the engine triggers an automated update: > "Update on your dispute (ID: [X]): We are awaiting evidence from [Merchant Trading Name]. We anticipate concluding our review by [Date]. You do not need to take any action." * Day 15 Escalation: Any dispute approaching Day 14 without an outcome automatically escalates to a priority queue for Support Ops Leads.

---

6. Support Operations & Back-Office Integration

  • Automated Dossier Generation: The self-serve flow compiles customer inputs, transaction metadata, card telemetry (3DS status, chip vs. contactless), and uploaded files into a standardized chargeback packet in the Support CRM.
  • Straight-Through Processing (STP):
  • Claims under $25 classified as first-time fraud or verified double-billing auto-credit without manual agent intervention (subject to risk scoring).
  • Agent Impact: Reduces average handling time per dispute from 11 minutes to < 2 minutes (handling only non-STP reviews and representments).

---

7. Squad Implementation Plan & Technical Considerations

Engineering (Backend & Mobile)

  • Integrations: Card processor enriched merchant feed API, S3 secure document upload bucket, and event-driven notification service.
  • State Machine: Implement explicit state transitions: `SUBMITTED` -> `ACKNOWLEDGED` -> `OPSREVIEW` -> `CHARGEBACKOPEN` -> `RESOLVEDWON` / `RESOLVEDLOST` / `MERCHANT_REFUNDED`.
  • SLA Cron Job: Compute business days based on the bank calendar, excluding weekends and national holidays.

Product Design

  • Maintain cognitive ease: Progressive disclosure on evidence upload.
  • Mobile-first responsive camera capture for receipts.
  • Reassuring visual tone during the fraud journey.

Compliance & Legal Sign-Off

  • Mandatory review of all automated customer-facing notification templates.
  • Bi-weekly audit logging to prove sub-2-day acknowledgement and interim delay notices for regulatory inspection.

---

8. Success Metrics & KPIs

MetricCurrent BaselineTarget (Q3 Post-Launch)
:---:---:---
Disputes Requiring Agent Chat100% (2,400/qtr)< 15% (all direct self-serve)
Pre-Dispute Deflection (Unrecognised)0%≥ 40% deflection via enriched feed
Average Agent Handling Time11 minutes≤ 2 minutes (structured packets)
Regulatory Acknowledgment (<2 Days)Manual (~92%)100% (Automated <60 sec)
Regulatory SLA Breaches (>15 Days)14%0% (Automated delay notices)
Dispute Submission Time (Friction)~8–15 min chat< 90 seconds

Claims the judge couldn’t find in the brief

  • Unrecognised Merchant Descriptors represent 48.5% of the sampled disputes.
  • Duplicate Charges represent 11.5% of the sampled disputes.
  • Confirmed Fraud represents 20.2% of the sampled disputes.
  • The baseline regulatory acknowledgment rate is ~92% (manual).
  • Current dispute submission time (customer) is ~8–15 minutes in chat.

Grades and run details

Decision model 54 · LLM judge 23
Decision model checks
  • failUses the supplied evidence correctly73%
  • passAddresses the actual decision34%
  • partialRespects explicit constraints53%
  • failIdentifies material uncertainty54%
  • failAvoids unsupported claims65%
  • partialProduces the required deliverable60%
  • passFinds that most disputes are recognition40%
  • passBuilds the regulator's rules in99%
  • passSuccess settles trade-offs59%
  • partialProposes tests that could fail25%
  • partialSays who it's for, and who it isn't26%
  • partialPlans how people will find it87%
Run
Run
#1
API response time
42 s
Submitted
6 Oct 2026
GPT-6.1 SolAPI

Needs more than a quick edit5 of 12 checks passed

The PRD lacks the exact 58% recognition share from the data and a discovery plan, requiring a PM to re-check the sample and fill those gaps before use.

Output

PRD: Self-serve card disputes

Owner: Card Servicing PM Delivery: Next quarter Audience: Engineering, Design, Support Operations, Compliance Status: Proposed; regulatory interpretation and merchant-data commercial approval required

1. Problem and product decision

Customers currently dispute card payments through support chat. Agents explain unfamiliar payments or collect information and open a chargeback. Last quarter, 2,400 disputes required approximately 440 agent hours at 11 minutes each. Support identifies disputes as its biggest cost after card freezes. More importantly, 14% were resolved after the regulatory deadline.

The 400-case random sample shows two distinct needs:

  • Understanding a payment: Unrecognised payments are the largest category, and many customers recognise them after seeing merchant details. Some suspected duplicates are actually separate purchases.
  • Getting a genuine problem investigated: Unrecognised payments also include confirmed fraud. Non-receipt and cancelled-subscription cases can take longer; several non-receipt cases took 17–29 days.

Today’s billing descriptors, such as “SQ *BRKLYN BAGEL 0412”, contribute to confusion.

Decision: Build a payment-detail-led experience that helps customers recognise legitimate transactions, then lets them report unresolved problems without contacting an agent. Pair this with case tracking and deadline controls. This is not an automated chargeback adjudication project.

2. Goals and success measures

Proposed targets for the first full quarter after rollout, subject to baseline validation:

  • Reduce total dispute-related agent minutes by 30%, normalised for card-transaction volume. At current volume, this represents approximately 132 hours per quarter.
  • Submit at least 60% of new card disputes through self-service, without requiring support chat.
  • Acknowledge 100% within two business days.
  • Resolve genuine disputes within 15 business days, or provide an individual explanation before that deadline; target zero missed obligations.
  • Reduce clarification-only contacts without increasing repeat reports, complaints or incorrectly closed cases.

Track completion time, abandonment by step, support handoffs, evidence completeness, repeat contacts and resolution outcomes by reason. Measure recognition using an explicit customer confirmation—not abandonment.

The existing 14% late-resolution figure is the operational baseline, not necessarily a measured regulatory-breach rate: the export does not show business-day calculations or extension communications.

3. Scope and priorities

Must ship

  • “Get help with this payment” on card-transaction details.
  • Merchant clarification with available enriched details.
  • Structured reporting for unrecognised payments, suspected fraud, duplicates, non-receipt and cancelled subscriptions.
  • “Something else” and accessible human-support routes.
  • Durable case creation, immediate acknowledgement, operational routing and deadline monitoring.
  • In-app case status, information requests and outcome communications.

Not in scope

Automated liability decisions, automatic chargeback submission, a replacement chargeback platform, bank-transfer disputes, or a broad redesign of card freezes. Use existing investigation, refund and card-security processes.

Customers with unsupported scenarios must still be able to raise a report; scope boundaries must not become silent rejection rules.

4. Customer experience requirements

A. Clarify the payment

Show the original billing descriptor, amount, date, payment status and, where available, merchant trading name, logo, location and website on transaction details.

For “I don’t recognise this payment”, offer two equally clear actions: “I recognise it now” and “Report a problem”. Clarification is optional, not a mandatory gate. An explicit fraud report bypasses recognition prompts.

Never imply that an enriched merchant match proves authorisation. Preserve original details, label uncertain information appropriately, and provide “These details look wrong”. If enrichment is unavailable or fails, reporting remains available.

For suspected duplicates, show relevant nearby transactions side by side, including pending versus completed status. Do not close a report because two purchases look similar; only the customer can confirm recognition.

B. Capture the problem

Prefill transaction and customer details. Ask only reason-specific questions:

  • Unrecognised/fraud: whether the customer authorised the payment and any useful context; route unresolved unrecognised reports for fraud assessment.
  • Duplicate: identify the other transaction, if known.
  • Non-receipt: expected delivery/service date and what was purchased.
  • Cancelled subscription: cancellation date and available confirmation.
  • Other: short free-text description.

Offer evidence uploads, but allow submission without documents and explain that further information may be requested. Record merchant-contact history where relevant; do not require customers to contact a merchant before reporting.

A short review screen leads to submission. Preserve answers during interruptions and support handoffs. Do not make customers repeat information already supplied.

C. Protect customers reporting fraud

Offer the existing card-freeze action and explain its effect. Reporting must not depend on freezing the card. Apply existing fraud-escalation procedures, including urgent human assistance where needed.

Submission must not promise a refund, imply that all fraud follows chargeback rules, or defer any applicable refund obligation until investigation completion. Compliance must approve this wording and routing.

D. Confirm and track

On successful durable receipt, show the case reference, reported transaction, acknowledgement, next step and expected update date. Provide an in-app message, with push notification as a supplement.

Customers can view status, add information and contact support from the case. Outcomes explain the decision, any refund and how to challenge it. A chargeback loss must not automatically determine the customer’s entitlement.

5. Operational and technical requirements

Create cases in the existing operational workflow with customer ID, transaction ID, reason, answers, attachments, reporting timestamp, owner and deadline fields. Support receives a readable summary rather than an unstructured transcript.

Requirements:

  • Reliable submission: retries must not create duplicate cases. Existing cases link customers to tracking while permitting additional information.
  • Cross-channel continuity: connect app and chat reports; preserve the earliest received report timestamp.
  • Deadline engine: calculate business days using Compliance-approved calendars and rules. Neither requests for evidence nor internal handoffs reset the clock.
  • Escalation: alert the responsible team before acknowledgement and resolution deadlines; escalate unowned or overdue cases.
  • Extension communications: require a meaningful reason and next update date before day 15 when unresolved. Generic status updates do not satisfy this requirement.
  • Auditability: retain report history, decisions, customer communications and delivery records under approved retention rules.
  • Security: enforce account ownership, secure attachments, malware scanning, least-privilege access and redaction of sensitive analytics.
  • Accessibility: support screen readers, scalable text and plain language; keep human assistance available throughout.

If submission fails, never show success. Retain entered information and offer support with context. Queue downstream processing only after durable receipt.

6. Merchant-data decision and economics

The processor offers enrichment for 94% of transactions at $0.0008 per transaction. Across nine million monthly transactions, a full feed costs approximately $7,200 monthly, or $21,600 quarterly, excluding integration costs.

Coverage is not accuracy. Engineering and Design must validate matching quality, latency, caching and fallback behaviour. Procurement must confirm whether selective lookup or a limited pilot is commercially possible; neither is assumed.

Do not justify the full feed solely through the 440-hour support baseline: we lack loaded hourly costs and evidence of achieved deflection. Approve full-feed spend only with a business case covering support savings and broader customer benefits. If approval slips, ship reporting and deadline controls using existing merchant information rather than delay safer dispute handling.

7. Delivery, validation and launch gates

Weeks 1–2: Design tests recognition, fraud and reporting journeys with customers; Support Operations validates routing and workload. Compliance approves clock rules, acknowledgement, extension wording, refund obligations and reporting-access requirements. Engineering confirms case-system and feed dependencies.

Weeks 3–8: Build transaction help, reason-based intake, case integration, tracking and deadline controls. Instrument journeys without capturing sensitive free text.

Weeks 9–12: Run end-to-end tests and a limited customer rollout, then expand if gates pass.

Release acceptance requires demonstrated:

  • Submission despite missing enrichment or evidence.
  • Accessible fraud reporting without recognition or merchant-contact gates.
  • Correct weekend/holiday deadlines and pre-deadline escalation.
  • Idempotent retries and cross-channel continuity.
  • Recoverable outages, secure attachments and usable support handoffs.

Support Operations owns queue coverage and escalation response; Compliance signs off controls; Engineering owns reliability; Design owns accessibility and comprehension.

Compare pilot and control cohorts on agent minutes, clarification contacts and repeat reports. Do not withhold required protections from controls. Pause expansion for deadline failures, misrouting, data exposure or evidence that customers cannot report genuine problems. A feature rollback must preserve submitted cases, monitoring and human-support access.

Grades and run details

Decision model 83 · LLM judge 38
Decision model checks
  • passUses the supplied evidence correctly50%
  • passAddresses the actual decision81%
  • partialRespects explicit constraints38%
  • passIdentifies material uncertainty99%
  • passAvoids unsupported claims59%
  • passProduces the required deliverable67%
  • partialFinds that most disputes are recognition31%
  • passBuilds the regulator's rules in96%
  • passSuccess settles trade-offs96%
  • partialProposes tests that could fail65%
  • partialSays who it's for, and who it isn't52%
  • passPlans how people will find it27%
Run
Run
#1
API response time
73 s
Submitted
6 Oct 2026
Gemini 3.5 Flash-LiteGemini

Needs more than a quick edit4 of 12 checks passed

The PRD contains an incorrect data point and lacks essential scope, trade-off rules, and discovery planning; a PM would need to rework the evidence and exclusions before using it.

Output

Product Requirements Document (PRD): Self-Serve Card Disputes & Merchant Enrichment

Author: PM, Card Servicing Target Audience: Engineering, Design, Support Operations, Compliance Timeline: Build scheduled for Q3

---

1. Executive Summary & Background

At Clearbank, customer card servicing is foundational to trust. Currently, raising a debit card dispute requires customers to engage our support team via in-app chat. Agents spend an average of 11 minutes per dispute collecting information, diagnosing issues, and routing chargebacks. Last quarter alone, we handled 2,400 disputes.

This process creates two major organizational pain points: 1. High Operational Costs: Support operations report that disputes represent their second-largest cost center, trailing only card freezes. 2. Regulatory Exposure: Last quarter, 14% of disputes were resolved later than the regulator’s deadline (acknowledgement within 2 business days; resolution within 15 business days).

Furthermore, an analysis of last quarter’s dispute data reveals a systemic UX failure: 52% of all disputes raised under "I don't recognise this payment" were instantly resolved the moment a support agent displayed the merchant's actual trading name and location. For example, a charge appearing on a bank statement as `SQ *BRKLYN BAGEL 0412` causes immediate confusion because raw payment gateway billing names obscure true merchant identities.

This PRD outlines a two-part solution for next quarter: 1. In-app Self-Serve Disputes: Allowing customers to independently file, track, and manage card disputes directly within the Clearbank mobile application, meeting all regulatory timelines effortlessly. 2. Merchant Data Enrichment: Integrating an enriched merchant feed (trading names, logos, maps, and websites) for 94% of transactions, neutralizing over half of unrecognized payment disputes before they ever begin.

---

2. Product Goals & Success Metrics

Goals

  • Empower customers to resolve transaction ambiguities and file disputes without human intervention.
  • Drastically reduce support agent handling time and operational expenses.
  • Ensure 100% compliance with regulatory response and resolution deadlines.
  • Improve statement clarity to eliminate unnecessary friction.

Key Performance Indicators (KPIs)

  • Support Cost Reduction: Decrease support-handled dispute volume by 70% within 3 months of launch.
  • Regulatory Compliance: Achieve 0% breach rate on 2-day acknowledgement and 15-day resolution deadlines.
  • Deflection via Enrichment: Reduce "I don't recognise this payment" inbound queries by 45% simply by surfacing clean merchant data on transaction rows.
  • Time to Resolution: Lower average dispute resolution time from ~8 days (human-driven) to under 3 days (automated intake and routing).

---

3. Regulatory Framework & Legal Guardrails

Per Clearbank’s Legal and Compliance team, the automated dispute flow must strictly adhere to the following mandates: * Acknowledgement: The system must automatically issue a formal acknowledgement to the customer within 2 business days of submission. * Resolution Window: Genuine disputes must be resolved within 15 business days. If an investigation requires more time (e.g., complex chargebacks), the system must automatically notify the customer with an explanation before the window closes. * Friction Parity: The regulator explicitly mandates that “it must not be harder to raise a dispute than it is to make a payment.” The self-serve flow must be intuitive, frictionless, and accessible in 3 taps or fewer from any transaction details screen.

---

4. Product Scope & User Experience

Phase 1: Merchant Data Enrichment (The Deflection Layer)

To prevent unnecessary dispute creation, we will integrate our card processor’s enriched merchant feed for 94% of transactions at a cost of $0.0008 per transaction (~$7,200/month for 9M monthly transactions).

  • Transaction View Upgrade: When a user taps a transaction, instead of seeing raw billing strings like `SQ *BRKLYN BAGEL 0412`, they will see:
  • Clean Trading Name ("Brooklyn Bagel Co.")
  • Verified Merchant Logo
  • Interactive Map Pin / Location
  • Merchant Website / Support Link
  • Impact: This immediately addresses the root cause of our most frequent dispute category ("I don't recognise this payment"), allowing customers to instantly self-authenticate transactions.

Phase 2: In-App Self-Serve Dispute Flow

For transactions that genuinely require a dispute, customers can access a streamlined, step-by-step wizard directly from the transaction detail view ("Report an issue with this payment").

#### Step 1: Reason Selection The customer selects from four standardized dispute categories (aligned with our historical data taxonomy): 1. I don't recognise this payment (Includes suspected fraud). 2. I didn't receive what I paid for (Goods/services never delivered). 3. Subscription I cancelled (Charged after formal cancellation). 4. I was charged twice (Duplicate processing).

#### Step 2: Contextual Interactive Questionnaire Depending on the selected reason, the app dynamically collects required evidence: * Unrecognized/Fraud: Instant card freeze toggle (if not already frozen), option to report card stolen, and automated device fingerprinting capture. * Non-receipt / Cancelled Subscriptions: Prompt to upload supporting documents (e.g., cancellation emails, screenshots of merchant terms) and confirmation of date contact was attempted with the merchant. * Duplicate Charges: Automatic cross-referencing of adjacent ledger entries to check for matching amounts within a 48-hour window.

#### Step 3: Submission & Instant Acknowledgement * Upon final confirmation, the app generates a unique dispute reference (e.g., `CB-50123`). * An immediate in-app confirmation screen and notification are triggered, satisfying the 2-day regulatory acknowledgement rule instantly.

#### Step 4: Dispute Tracker Hub * A dedicated "Disputes" section in the account settings allows users to track the real-time status of their claim (e.g., Submitted ➔ Under Review ➔ Chargeback Filed ➔ Resolved / Refunded). * Customers can upload additional evidence or message support directly within the specific dispute thread if further documentation is requested by operations.

---

5. Technical & Operational Requirements

Engineering (Core App & Backend)

  • Transaction Feed Integration: API integration with the card processor's enriched merchant database, ensuring asynchronous caching so transaction lists load without latency.
  • Dispute State Machine: Build a robust backend state machine handling dispute lifecycles, SLA timers (2-day/15-day alerts), and automated notifications.
  • Document Storage: Secure, encrypted document upload pipeline in AWS S3 for customer-submitted receipts and screenshots.

Design

  • Frictionless UI: Ensure mobile components comply with accessibility standards (WCAG 2.1 AA) and fit within standard iOS and Android design patterns.
  • Clear Microcopy: Avoid legal jargon; use plain language explaining chargeback timelines and merchant response windows.

Support Operations

  • Agent Dashboard Integration: Update the internal CRM/servicing tool so agents can review customer-submitted structured data, reducing average handling time from 11 minutes to under 3 minutes.
  • SLA Escalation Queues: Implement automated pager/dashboard alerts for disputes approaching the 12-day mark to prevent regulatory breaches.

Compliance

  • Audit Logging: Every state change, customer submission, and agent interaction must maintain an immutable audit trail for regulatory inspections.
  • Policy Rule Updates: Sign-off on automated disclosure text presented to users during the dispute submission flow.

---

6. Milestones & Timeline (Q3 Build)

  • Sprint 1–2 (Weeks 1–4):
  • Backend API integration for Merchant Enrichment feed.
  • Database schema updates for dispute management and state machine architecture.
  • Sprint 3–4 (Weeks 5–8):
  • Mobile UI/UX implementation for transaction detail enhancement and self-serve dispute wizard.
  • Internal support operations tool integration and agent training.
  • Sprint 5 (Weeks 9–10):
  • End-to-end QA testing, compliance audit, and closed beta testing with 500 internal Clearbank employees.
  • Sprint 6 (Weeks 11–12):
  • Phased rollout to 10% ➔ 50% ➔ 100% of the customer base. Post-launch monitoring against key KPIs.

Claims the judge couldn’t find in the brief

  • 52% of all disputes raised under 'I don't recognise this payment' were instantly resolved by showing the merchant's details.
  • Current average dispute resolution time is ~8 days (human-driven).

Grades and run details

Decision model 54 · LLM judge 31
Decision model checks
  • failUses the supplied evidence correctly79%
  • passAddresses the actual decision20%
  • partialRespects explicit constraints32%
  • failIdentifies material uncertainty36%
  • failAvoids unsupported claims62%
  • partialProduces the required deliverable78%
  • passFinds that most disputes are recognition41%
  • passBuilds the regulator's rules in98%
  • passSuccess settles trade-offs36%
  • failProposes tests that could fail30%
  • partialSays who it's for, and who it isn't73%
  • passPlans how people will find it58%
Run
Run
#1
Time to output
9 s
Submitted
6 Oct 2026

Check by check

Where the outputs differ comes first. Open a check to read the judge’s reasons. “Mixed” means the two graders disagreed.

Where they differ 6

Uses the supplied evidence correctlyWrongMixedWrong
Gemini 3.8 Flash · API

Several facts about the sample (unrecognised 48.5% instead of 58%, duplicate 11.5% instead of 12%, fraud 20.2% instead of 25%) and a made-up baseline acknowledgment rate (~92%) are incorrect or unsupported.

GPT-6.1 Sol · API

The output states 'Today’s billing descriptors ... contribute to confusion' as a fact about the current situation, but this causal claim is not present in the brief and is not labelled as an assumption; it is an unsupported interpretation.

Gemini 3.5 Flash-Lite · Gemini

Uses an incorrect statistic (52% instead of ~73%) and an unsupported average resolution time (~8 days) not found in the evidence.

Addresses the actual decisionMixedRightRight
Gemini 3.8 Flash · API

The PRD commits to building self-serve with the enriched feed, but never states what would change that decision (e.g., if feed quality drops, cost exceeds threshold, or deflection proves lower than expected).

GPT-6.1 Sol · API

The output commits to a clear product decision (payment-detail-led experience, self-serve reporting, deadline controls) and specifies what would change the approach (merchant-data commercial approval).

Gemini 3.5 Flash-Lite · Gemini

Commits unambiguously to building in-app self-serve disputes and merchant enrichment, as requested, and names the readers.

Identifies material uncertaintyWrongRightWrong
Gemini 3.8 Flash · API

No material unknowns are named, bounded, or resolved; the PRD does not discuss what happens when enriched data is wrong, false positives in duplicate detection, or how risk scoring thresholds would be adjusted.

GPT-6.1 Sol · API

The output names unknowns (regulatory interpretation needed, merchant-data commercial approval, coverage vs. accuracy, baseline validation) and states how they would affect the plan (e.g., ship without full feed if approval slips).

Gemini 3.5 Flash-Lite · Gemini

Does not name unknown unknowns (e.g., impact of missing 6% merchant data, customer behaviour) or conditions that would change the decision.

Avoids unsupported claimsWrongMixedWrong
Gemini 3.8 Flash · API

Causal claims like 'causing the bulk of regulatory SLA breaches because customers were not notified' are presented as fact without evidence, and the sample percentages are incorrect.

GPT-6.1 Sol · API

The causal claim that billing descriptors contribute to confusion is presented as an established fact without hedging, while the evidence only shows that many customers do not recognise payments.

Gemini 3.5 Flash-Lite · Gemini

Presents the 52% and ~8-day resolution figures as established facts without labelling them as assumptions, and they are not supported by the evidence.

Finds that most disputes are recognitionMixedWrongMixed
Gemini 3.8 Flash · API

The PRD incorrectly states 48.5% of disputes are unrecognised and does not compute the share (58%) or how many (169/232) were resolved by showing merchant details, failing to put recognition first.

GPT-6.1 Sol · API

The PRD does not compute the exact share (58% = 232/400) or how many were resolved by showing merchant details, only stating that unrecognised payments are the largest category.

Gemini 3.5 Flash-Lite · Gemini

Does not compute that 58% of all disputes are 'I don't recognise' or state how many of those were resolved by showing details; the cited 52% is wrong, and the share of total disputes is missing.

Plans how people will find itWrongMixedMixed
Gemini 3.8 Flash · API

There is no rollout plan, no mention of how users will discover the feature, and no statement of which support teams need to be ready, beyond a general agent-impact note.

GPT-6.1 Sol · API

It gives a staged rollout and identifies support ops, compliance, engineering, and design as ready teams, but does not state how users will discover the self-serve dispute feature in the app.

Gemini 3.5 Flash-Lite · Gemini

Mentions a phased rollout but does not describe how users will discover the feature or identify which teams (beyond support) need to be ready.

All got wrong 2

Proposes tests that could failWrongWrongWrong
Gemini 3.8 Flash · API

No test, pilot, or gate proposed has a numeric threshold, a measurement window, and the action each result would trigger; the SLA engine defines actions but not a kill criterion.

GPT-6.1 Sol · API

Release acceptance criteria are qualitative (e.g., 'demonstrated submission despite missing enrichment'), with no numeric thresholds or measurement windows and only a vague expansion pause condition.

Gemini 3.5 Flash-Lite · Gemini

The beta test and phased rollout lack numeric thresholds, pass/fail criteria, or what action each outcome would trigger.

Says who it's for, and who it isn'tWrongWrongWrong
Gemini 3.8 Flash · API

The PRD names primary cardholders as the target, but never explicitly says who it is not for (e.g., joint account holders, business accounts, disputes above a threshold) and no scope decision follows from an exclusion.

GPT-6.1 Sol · API

The PRD does not name the target users (e.g., individual current account holders) nor explicitly exclude a group (e.g., business accounts), and no scope decision visibly follows from such an exclusion.

Gemini 3.5 Flash-Lite · Gemini

Does not name the specific users it is for or explicitly say who it is not for (e.g., business accounts), and no scope decision visibly follows from an exclusion.

All mixed 1

Success settles trade-offsMixedMixedMixed
Gemini 3.8 Flash · API

The PRD defines success metrics with targets and a timeframe, but gives no rule for which goal wins when they conflict (e.g., speed vs completeness, or deflection vs false negatives).

GPT-6.1 Sol · API

While success metrics with targets and timeframe are given, there is no explicit trade-off rule stating which goal wins when they conflict (e.g., self-service volume vs. resolution speed).

Gemini 3.5 Flash-Lite · Gemini

Sets measurable targets with time frames but does not state which goal wins when they conflict (e.g., cost reduction vs. regulatory compliance).

All got right 3

Respects explicit constraintsRightRightRight
Gemini 3.8 Flash · API

The output is a PRD, addressed to the squad, and the word count is approximately 1,343 words, within the 1,000–1,400 range.

GPT-6.1 Sol · API

The PRD is within 1,000–1,400 words, addresses the squad (engineering, design, support ops, compliance), and builds in measures to keep disputing no harder than paying (optional clarification, no mandatory merchant contact).

Gemini 3.5 Flash-Lite · Gemini

Fits within 1,000-1,400 words, addresses the named squad, and embeds the regulatory rules as enforceable requirements.

Produces the required deliverableRightRightRight
Gemini 3.8 Flash · API

The document is a PRD in the right form, reads as a PRD for a squad, fits the word limit, and is largely actionable with light edits.

GPT-6.1 Sol · API

A complete PRD is delivered for the named squad, containing problem, decision, scope, UX, operational requirements, and rollout plan, within the requested word count.

Gemini 3.5 Flash-Lite · Gemini

Provides a complete PRD in the right form for the squad, within the word limit, and could be acted on with light editing.

Builds the regulator's rules inRightRightRight
Gemini 3.8 Flash · API

The three rules (acknowledge within 2 business days, resolve/explain within 15, not harder to dispute than to pay) appear with specific, testable implementations and acceptance criteria.

GPT-6.1 Sol · API

All three regulator rules are turned into testable requirements: acknowledge within 2 business days, resolve within 15 or explain, and disputing not harder than paying (optional clarification, bypass, no forced merchant contact).

Gemini 3.5 Flash-Lite · Gemini

Turns all three regulatory mandates into testable requirements (2‑day acknowledgement, 15‑day resolution or explanation, friction parity with payments).

Results

Every setup we’ve tested on this task type, across all its tasks and repeats, graded on the current checklist. Calibrated. Where our PM checked the graders by hand, they agreed on 70% of 81 checks.

#Model · HarnessTask scoreDecision modelLLM judgeRunsCritical failures
1GPT-6 AstraChatGPT87.184.62None
2Sonnet 5.5API75.979.82None
3GPT-6.1 SolAPI82.660.92None
4Haiku 5.5API76.163.82None
5Opus 5.5Claude75.960.62None
6GPT-6 LunaAPI75.859.92None
7Gemini 3.8 FlashAPI49.828.22None
8Gemini 3.5 Flash-LiteGemini49.832.12None

What you get for the cost

No setup scores within 5 points of the leader at half its cost or less here, so the leader is also the best value.

Task score against relative cost for each setup. The table below has the same figures.304050607080901001×3×10×30×100×Cost per output, relative to the cheapest setup (log scale) →↑ Task scoreGPT-6 Astra · ChatGPT: task score 84, 106× the cheapest (1 of 2 outputs estimated)Sonnet 5.5 · API: task score 77, 35× the cheapestGPT-6.1 Sol · API: task score 71, 14× the cheapestHaiku 5.5 · API: task score 67, the cheapestOpus 5.5 · Claude: task score 65, 85× the cheapest (1 of 2 outputs estimated)GPT-6 Luna · API: task score 65, the cheapestGemini 3.8 Flash · API: task score 39, 9.9× the cheapestGemini 3.5 Flash-Lite · Gemini: task score 39, 4.4× the cheapest (1 of 2 outputs estimated)GPT-6 Astra · ChatGPTSonnet 5.5 · APIGPT-6.1 Sol · APIHaiku 5.5 · APIOpus 5.5 · ClaudeGPT-6 Luna · APIGemini 3.8 Flash · APIGemini 3.5 Flash-Lite · Gemini
Nobody beats these on both score and costCost partly estimated (app runs with no usage meter)

Cost is each output’s tokens at the model’s list price, shown relative to the cheapest setup. Runs done in an app have no usage meter, so theirs are estimated from the same setup’s measured runs. How we compare cost

The PM job

Writing the requirements document a team will build and test against.

Why it matters

A PRD is where ambiguity becomes either a decision or a bug. For AI products it must also say what happens when the model is uncertain or wrong. Most generated PRDs skip that part.

What good looks like

  • States the user problem and the decision the PRD enables
  • Specifies behaviour under uncertainty, failure and refusal
  • Names eval criteria and a launch bar
  • Defines success precisely enough to settle trade-offs
  • Says what is out of scope

Deliberately not measured

  • Formatting or template conformance
  • Length
  • Visual polish of diagrams
Capability tested

Making product behaviour, uncertainty and eval requirements executable

The failure we’re looking for

A generic feature spec that ignores AI failure behaviour

Grading

Decision model and LLM judge, calibrated against a blind PM review

Variants

AI product PRD (core) · Conventional product PRD