Humanity’s Last PRD
Latest drop: Gemini 4 Argon · 13 Oct
Tasks / Product Craft

One-shot prototype

Can the configuration produce a working, constraint-compliant prototype in one attempt?

Task type v1.2 · 2 tasksLast changed 27 Sept 2026 · ChangelogDifficulty

What AI gets right here, and what you’ll still have to catch

From 18 graded outputs by 9 models.

Reliably right

  1. Avoids template layout and copy tells100% pass
  2. Shows the no-slots state100% pass
  3. Respects explicit constraints89% pass

Where it slips

  1. Says what it tests and what's faked44% pass
  2. Avoids the generic AI palette78% pass
  3. Stays in scope78% pass

How it’s graded

Each check is built on how world-class product and growth leaders do this work, as they’ve described it on Lenny’s Podcast, and names the leader it comes from.

  1. Says what it tests and what's faked

    Does the reply that comes with the prototype say what question or assumption it is meant to test, and which parts are faked, hard-coded or missing?

    Passes when Names what the prototype is for, and lists the stubbed or hard-coded parts (data, sign-in, saving and the like).

    Karri SaarinenLenny’s Podcast

Plus our standard checks

· and 5 written for each task, which you’ll see in the tasks below.

The tasks

Read the brief, then compare up to three outputs, each with the LLM judge’s verdict on every check. Highlights mark what a PM had to fix.

The brief

Build a clickable prototype that lets a patient move an existing appointment to another available slot in one flow. Deliver a single self-contained HTML file that opens straight into the flow in a browser: no design canvas, artboard or "play" step. Use made-up data.

What the model was given2 items: Scenario, Constraints
ScenarioSam Patel has a GP appointment with Dr Amira Khan at Harbour Medical Centre on Tuesday 6 October at 09:30 and needs to move it. Show the next seven days of availability; at least one of those days is fully booked.
ConstraintsDesigned for a mobile screen, about 390px wide. No login, sign-up or account screens. Show a clear "no slots available" state for a fully booked day. The original appointment stays booked until the patient confirms, and the confirmation shows the new time.
What a strong answer doesThe answer key the graders mark against

A working mobile rebooking flow: pick a new slot, confirm, and see the new time confirmed; a fully booked day says so clearly; nothing beyond rebooking.

Critical failures (cap the score at 40)
  • Core interaction broken
Task

Mid-level · v1.1

The outputs

9 setups answered this brief.

Gemini 3.8 FlashAPI93Judged in the browserHaiku 5.5API93Judged in the browserGPT-6.1 SolAPI86Judged in the browserGPT-6 LunaAPI86Judged in the browserSonnet 5.5API86Judged in the browserGemini 4 ArgonAPI86Judged in the browserOpus 5.5Claude79Judged in the browserGPT-6 AstraChatGPT64Judged in the browserGemini 3.5 Flash-LiteGemini57Judged in the browser
Gemini 4 ArgonAPI

5 of 7 checks passed

Output

Open the prototypeThe file the model delivered, in a new tab

Here is a self-contained, interactive HTML prototype designed for a mobile viewport (~390px wide). It opens directly into Sam Patel's rescheduling flow with interactive day selection, a clearly indicated fully-booked day (Thursday 8 Oct), a review step ensuring the original booking remains active until confirmed, and a final confirmation screen with a calendar download option.

Grades and run details

Decision model 86
Decision model checks
  • partialStays in scope23%
  • partialSays what it tests and what's faked27%
  • passRespects explicit constraintsby hand100%
  • passCore interaction worksby hand100%
  • passShows the no-slots stateby hand100%
  • passAvoids the generic AI paletteby hand100%
  • passAvoids template layout and copy tellsby hand100%
Artefacts
Run
Run
#1
API response time
69 s
Submitted
13 Oct 2026

Check by check

Open a check to read the judge’s reasons. “Mixed” means the two graders disagreed.

Mixed 2

Stays in scopeMixed

No reason given.

Says what it tests and what's fakedMixed

No reason given.

Got right 5

Respects explicit constraintsRight

No reason given.

Core interaction worksRight

No reason given.

Shows the no-slots stateRight

No reason given.

Avoids the generic AI paletteRight

No reason given.

Avoids template layout and copy tellsRight

No reason given.

Results

Every setup we’ve tested on this task type, across all its tasks and repeats, graded on the current checklist. Calibrated. Where our PM checked the graders by hand, they agreed on 78% of 91 checks.

#Model · HarnessTask scoreDecision modelLLM judgeRunsCritical failures
1Sonnet 5.5API92.9–2None
2Haiku 5.5API92.3–2None
3GPT-6.1 SolAPI88.7–2None
4Gemini 4 ArgonAPI88.7–2None
5GPT-6 LunaAPI84.5–2None
6Opus 5.5Claude76.8–2None
7Gemini 3.8 FlashAPI75.6–2None
8GPT-6 AstraChatGPT69.6–2None
9Gemini 3.5 Flash-LiteGemini49.4–2None

What you get for the cost

Haiku 5.5 · API scores within a point of Sonnet 5.5 · API at about a thirtieth of the cost.

Task score against relative cost for each setup. The table below has the same figures.4050607080901001×3×10×30×100×300×Cost per output, relative to the cheapest setup (log scale) →↑ Task scoreSonnet 5.5 · API: task score 93, 351× the cheapestHaiku 5.5 · API: task score 92, 11× the cheapestGPT-6.1 Sol · API: task score 89, 122× the cheapestGemini 4 Argon · API: task score 89, 243× the cheapestGPT-6 Luna · API: task score 85, 7.1× the cheapestOpus 5.5 · Claude: task score 77, 65× the cheapest (2 of 2 outputs estimated)Gemini 3.8 Flash · API: task score 76, 74× the cheapestGPT-6 Astra · ChatGPT: task score 70, 19× the cheapest (2 of 2 outputs estimated)Gemini 3.5 Flash-Lite · Gemini: task score 49, the cheapest (2 of 2 outputs estimated)Sonnet 5.5 · APIHaiku 5.5 · APIGPT-6.1 Sol · APIGemini 4 Argon · APIGPT-6 Luna · APIOpus 5.5 · ClaudeGemini 3.8 Flash · APIGPT-6 Astra · ChatGPTGemini 3.5 Flash-Lite · Gemini
Best valueNobody beats these on both score and costCost partly estimated (app runs with no usage meter)

Cost is each output’s tokens at the model’s list price, shown relative to the cheapest setup. Runs done in an app have no usage meter, so theirs are estimated from the same setup’s measured runs. How we compare cost

The PM job

Getting a clickable prototype in front of users or stakeholders fast.

Why it matters

A prototype that looks right but whose core interaction is broken wastes a research session. This task rewards working behaviour over polish.

What good looks like

  • The core interaction works end to end
  • Respects the stated constraints and scope
  • Handles the empty and error states named in the brief
  • Says what it's testing and what's faked
  • Looks like something you'd share: clear hierarchy, none of the stock AI-template look

Deliberately not measured

  • Production code quality
  • Design to a requested style: the brief never asks for one, so this measures default taste
Capability tested

Turning a spec into a usable interactive artefact

The failure we’re looking for

An attractive shell with a broken or absent core interaction

Grading

Browser checks (interaction and design) and decision model, calibrated against a blind PM review