Humanity’s Last PRD
Latest drop: Gemini 4 Argon · 13 Oct
Tasks / Product Craft

One-shot prototype

Can the configuration produce a working, constraint-compliant prototype in one attempt?

Task type v1.2 · 2 tasksLast changed 27 Sept 2026 · ChangelogDifficulty

What AI gets right here, and what you’ll still have to catch

From 18 graded outputs by 9 models.

Reliably right

  1. Avoids template layout and copy tells100% pass
  2. Shows the no-slots state100% pass
  3. Respects explicit constraints89% pass

Where it slips

  1. Says what it tests and what's faked44% pass
  2. Avoids the generic AI palette78% pass
  3. Stays in scope78% pass

How it’s graded

Each check is built on how world-class product and growth leaders do this work, as they’ve described it on Lenny’s Podcast, and names the leader it comes from.

  1. Says what it tests and what's faked

    Does the reply that comes with the prototype say what question or assumption it is meant to test, and which parts are faked, hard-coded or missing?

    Passes when Names what the prototype is for, and lists the stubbed or hard-coded parts (data, sign-in, saving and the like).

    Karri SaarinenLenny’s Podcast

Plus our standard checks

· and 4 written for each task, which you’ll see in the tasks below.

The tasks

Read the brief, then compare up to three outputs, each with the LLM judge’s verdict on every check. Highlights mark what a PM had to fix.

The brief

Build a clickable prototype of a flow where an employee captures a receipt, checks the amount the app extracted, corrects it, and saves the expense. Deliver a single self-contained HTML file that opens straight into the flow in a browser: no design canvas, artboard or "play" step. Use made-up data.

What the model was given2 items: Scenario, Constraints
ScenarioAlex Morgan is submitting a taxi receipt from 14 September for £38.40. The app's automatic extraction misreads the amount as £83.40.
ConstraintsDesktop web. Receipt capture is simulated: a control such as "Use sample receipt" stands in for the camera or file upload, with no real upload needed. Show the extracted amount and let the employee correct it before saving; the saved expense shows the corrected amount.
What a strong answer doesThe answer key the graders mark against

Capture (simulated), a visible extracted amount, an easy correction, and a saved expense showing the corrected amount.

Critical failures (cap the score at 40)
  • Core interaction broken
Task

Mid-level · v1.1

The outputs

9 setups answered this brief.

Sonnet 5.5API100Judged in the browserGPT-6.1 SolAPI92Judged in the browserHaiku 5.5API92Judged in the browserGemini 4 ArgonAPI92Judged in the browserGPT-6 LunaAPI83Judged in the browserOpus 5.5Claude75Judged in the browserGPT-6 AstraChatGPT75Judged in the browserGemini 3.8 FlashAPI58Judged in the browserGemini 3.5 Flash-LiteGemini42Judged in the browser
Gemini 4 ArgonAPI

5 of 6 checks passed

Output

Open the prototypeThe file the model delivered, in a new tab

Here is a self-contained, interactive HTML prototype for the expense submission and OCR correction flow.

You can save the code block below as an `.html` file (for example, `expense-flow.html`) and open it directly in any desktop web browser.

How to Walk Through the Prototype

1. Capture Receipt (Step 1): Click "Use sample receipt" on the Metro Cab Co. (14 Sep) card. A brief scanning animation simulates the OCR extraction. 2. Review & Correct (Step 2): - On the left, inspect the thermal taxi receipt from 14/09/2025 showing a metered fare of £34.20 + £4.20 surcharge = £38.40 (with a subtle paper crease across the total). - On the right, notice that the app's automatic extraction misread the total as £83.40 and flagged it with a low-confidence warning. - Correct the amount either by typing `38.40` directly into the highlighted Total Amount input or clicking the "Use £38.40 from receipt" quick-action button. 3. Save Expense (Step 3): Click "Save Expense (£38.40)" to commit the item and view Alex Morgan's updated September expense claim alongside the verification audit trail showing the original OCR read (`£83.40`) crossed out in favor of the corrected `£38.40` total.

Grades and run details

Decision model 92
Decision model checks
  • passStays in scope7%
  • partialSays what it tests and what's faked40%
  • passRespects explicit constraintsby hand100%
  • passCore interaction worksby hand100%
  • passAvoids the generic AI paletteby hand100%
  • passAvoids template layout and copy tellsby hand100%
Artefacts
Run
Run
#1
API response time
1.6 min
Submitted
13 Oct 2026

Check by check

Open a check to read the judge’s reasons. “Mixed” means the two graders disagreed.

Mixed 1

Says what it tests and what's fakedMixed

No reason given.

Got right 5

Respects explicit constraintsRight

No reason given.

Core interaction worksRight

No reason given.

Stays in scopeRight

No reason given.

Avoids the generic AI paletteRight

No reason given.

Avoids template layout and copy tellsRight

No reason given.

Results

Every setup we’ve tested on this task type, across all its tasks and repeats, graded on the current checklist. Calibrated. Where our PM checked the graders by hand, they agreed on 78% of 91 checks.

#Model · HarnessTask scoreDecision modelLLM judgeRunsCritical failures
1Sonnet 5.5API92.9–2None
2Haiku 5.5API92.3–2None
3GPT-6.1 SolAPI88.7–2None
4Gemini 4 ArgonAPI88.7–2None
5GPT-6 LunaAPI84.5–2None
6Opus 5.5Claude76.8–2None
7Gemini 3.8 FlashAPI75.6–2None
8GPT-6 AstraChatGPT69.6–2None
9Gemini 3.5 Flash-LiteGemini49.4–2None

What you get for the cost

Haiku 5.5 · API scores within a point of Sonnet 5.5 · API at about a thirtieth of the cost.

Task score against relative cost for each setup. The table below has the same figures.4050607080901001×3×10×30×100×300×Cost per output, relative to the cheapest setup (log scale) →↑ Task scoreSonnet 5.5 · API: task score 93, 351× the cheapestHaiku 5.5 · API: task score 92, 11× the cheapestGPT-6.1 Sol · API: task score 89, 122× the cheapestGemini 4 Argon · API: task score 89, 243× the cheapestGPT-6 Luna · API: task score 85, 7.1× the cheapestOpus 5.5 · Claude: task score 77, 65× the cheapest (2 of 2 outputs estimated)Gemini 3.8 Flash · API: task score 76, 74× the cheapestGPT-6 Astra · ChatGPT: task score 70, 19× the cheapest (2 of 2 outputs estimated)Gemini 3.5 Flash-Lite · Gemini: task score 49, the cheapest (2 of 2 outputs estimated)Sonnet 5.5 · APIHaiku 5.5 · APIGPT-6.1 Sol · APIGemini 4 Argon · APIGPT-6 Luna · APIOpus 5.5 · ClaudeGemini 3.8 Flash · APIGPT-6 Astra · ChatGPTGemini 3.5 Flash-Lite · Gemini
Best valueNobody beats these on both score and costCost partly estimated (app runs with no usage meter)

Cost is each output’s tokens at the model’s list price, shown relative to the cheapest setup. Runs done in an app have no usage meter, so theirs are estimated from the same setup’s measured runs. How we compare cost

The PM job

Getting a clickable prototype in front of users or stakeholders fast.

Why it matters

A prototype that looks right but whose core interaction is broken wastes a research session. This task rewards working behaviour over polish.

What good looks like

  • The core interaction works end to end
  • Respects the stated constraints and scope
  • Handles the empty and error states named in the brief
  • Says what it's testing and what's faked
  • Looks like something you'd share: clear hierarchy, none of the stock AI-template look

Deliberately not measured

  • Production code quality
  • Design to a requested style: the brief never asks for one, so this measures default taste
Capability tested

Turning a spec into a usable interactive artefact

The failure we’re looking for

An attractive shell with a broken or absent core interaction

Grading

Browser checks (interaction and design) and decision model, calibrated against a blind PM review