Humanity’s Last PRD
Latest drop: Gemini 4 Argon · 13 Oct
Tasks / Strategy

1000x an idea

Can the model expand an idea's ambition while keeping it tethered to a real mechanism?

Task type v1.0 · 2 tasksLast changed 6 Oct 2026 · ChangelogDifficulty

What AI gets right here, and what you’ll still have to catch

From 18 graded outputs by 9 models. 56% were usable with at most a quick edit.

Reliably right

  1. Extreme, then back to buildable100% pass
    Pushes from two-person sharing to the extreme household as the unit (any number of members, live pantry state, learning routines, delivery integration), then works back to the scoped two-person build that tests coordination.
    Haiku 5.5 · API · Sharing a shopping list, 1000x
  2. Finds the mechanism in the data100% pass
    Uses the retention, conversion, session frequency, notification triggers, note usage, and invitation context to identify shared money decisions as the core mechanism, not just budgeting.
    GPT-6.1 Sol · API · Budgets for two
  3. Respects explicit constraints97% pass
    Location tracking is not proposed this quarter and is explicitly kept behind dual opt-in; the memo stays well within 700 words and respects the team size and quarter.
    Haiku 5.5 · API · Sharing a shopping list, 1000x

Where it slips

  1. Proposes tests that could fail40% pass
    The proposed test does not name a numeric threshold for the primary outcome; it only says to set minimum detectable effects and thresholds before launch, which is a process, not a specific threshold.
    GPT-6.1 Sol · API · Budgets for two
  2. The big version gets stronger as it grows50% pass
    The memo names a demand network as a compounding mechanism but does not propose a test of whether that network effect would be large enough to matter, nor does it ask that question.
    GPT-6 Astra · ChatGPT · Sharing a shopping list, 1000x
  3. Uses the supplied evidence correctly58% pass
    The claim 'Solo budgeting runs on willpower, which fades' is a statement about the current situation not found in the supplied context and not derivable from it.
    Opus 5.5 · Claude · Budgets for two

How it’s graded

Each check is built on how world-class product and growth leaders do this work, as they’ve described it on Lenny’s Podcast, and names the leader it comes from.

  1. The big version gets stronger as it grows

    Does the ambitious version name a mechanism that makes it more valuable or harder to copy as it grows (each new user adding value for others, or a barrier a competitor couldn't buy), and ask whether that effect would be large enough to matter?

    Passes when Names the mechanism, how it compounds with scale, and a test of whether the effect is big enough to matter.

    Hamilton HelmerLenny’s Podcast

Plus our standard checks

Uses the supplied evidence correctly · Addresses the actual decision · Respects explicit constraints · Identifies material uncertainty · Avoids unsupported claims · Produces the required deliverable · Extreme, then back to buildable · Proposes tests that could fail · and 1 written for each task, which you’ll see in the tasks below.

The tasks

Read the brief, then compare up to three outputs, each with the LLM judge’s verdict on every check. Highlights mark what a PM had to fix.

The brief

You're a PM at Pocketbook. Ana has proposed shared budgets for couples; before we commit next quarter, our CEO wants the 1000x version (her note is below). Write the memo, in no more than 900 words: the most ambitious version of the idea, the mechanism that makes it work, and then what we build first and how we'll know it's working. The data is attached.

What the model was given5 items: About Pocketbook, ana_idea.md, ceo_note.md, usage_by_user_group.csv, what_shared_budgets_do.md
About PocketbookA personal budgeting app: connect your accounts, set budgets, track spending. Free, with Premium at $6 a month.
ana_idea.md3 lines · Download
# The idea (from Ana, PM)

"Let people share a budget with their partner. It's our most requested feature after bank sync for more banks."
ceo_note.md1 lines · Download
From Ruth (CEO): "Before we commit next quarter to Ana's idea, show me the 1000x version. How big could this get? Then bring it back to what we build first and how we'll know it's working. Keep it under 900 words."
usage_by_user_group.csv3 rows · Download
user_group,users,retained_at_6_months_pct,invited_someone_pct,invitee_joined_pct,premium_conversion_pct
Solo budgeters,182000,17,3,22,4
Couples sharing one budget,24600,58,100,100,19
Households (3+ people on one budget),3900,71,100,100,31
what_shared_budgets_do.md6 lines · Download
# What shared budgets do (from the data team)

- Couples who share a budget open the app on more days than solo users (11 a month against 4) and most sessions start from a notification about the other person's spending ("Alex spent $48 at the grocery store").
- 41% of couples add a "money talk" note to a transaction in their first month; the note feature was built as an afterthought.
- 16% of couples add a third person within a year (a teenager, a parent they support).
- Shared budgets started from an invite link in a shared-expenses moment (rent, a holiday) retain best.
What a strong answer doesThe answer key the graders mark against

Finds the mechanism in the data: shared money decisions, not budgeting. Couples sharing a budget retain at 58% against 17% for solo budgeters, convert to Premium at 19% against 4%, open the app on 11 days a month against 4 (mostly from notifications about the other person's spending), and use transaction notes as 'money talk'; households of 3+ retain at 71% and convert at 31%. Every shared budget brings in at least one new user by design, and 16% of couples add a third person. So the 1000x version is the place a household runs its money together (shared goals, who pays for what, conversations on transactions, extended family), growing through invitations at shared-expense moments (rent, holidays). Asks whether the effect is large enough to matter: each new member makes the budget more useful to the others, so it compounds. Works back to a first step that keeps the mechanism: shared budgets launched from an invite at a shared-expense moment, with spending notifications and transaction notes built in, measured on invites accepted, retention of invited users, and Premium conversion for couples, with thresholds.

Task

Staff-level · v1.1

The outputs

9 setups answered this brief.

GPT-6.1 SolAPI91Usable with a quick editGPT-6 AstraChatGPT91Usable with a quick editOpus 5.5Claude89Usable with a quick editGPT-6 LunaAPI89Usable with a quick editHaiku 5.5API89Usable with a quick editSonnet 5.5API84Usable with a quick editGemini 4 ArgonAPI61Usable with a quick editGemini 3.5 Flash-LiteGemini43Needs more workGemini 3.8 FlashAPI39Needs more work
Gemini 4 ArgonAPI

Usable with a quick edit4 of 10 checks passed

The memo contains a clear vision, data-backed mechanism, and actionable first step with metrics. A PM could use it as is or after a quick edit to add contingency actions and uncertainty statements.

Output

To: Ruth, Ana From: Product Subject: The 1000x Version of Shared Budgets: Multiplayer Money

The Insight: Money Isn’t Solo, It’s Social

Right now, Pocketbook is built as a single-player calculator. And like most calculators, people stop using it: 83% of our 182,000 solo users churn within six months, and only 4% ever pay us.

When we look at the 28,500 users who have hacked Pocketbook to share a budget, the product transforms: * Retention: 6-month retention jumps from 17% (solo) to 58% (couples) and 71% (households of 3+). * Monetization: Premium conversion leaps from 4% to 19% for couples, and 31% for households. * Engagement: Couples open the app 11 days a month vs. 4 for solo users.

Why? Look at how they use it. Most sessions start from a push notification about a partner's spending ("Alex spent $48 at the grocery store"). Furthermore, 41% of couples use our buried transaction-note feature to have "money talks" in their first month.

Ana’s proposal—shared budgets for couples—captures this energy. The 1000x version recognizes that "couples" are just the entry point. Already, 16% of our couples add a third person within a year—a teenager, an aging parent, a roommate.

The 1000x Vision: The Financial Operating System for Life’s Networks

Personal finance apps assume money stops at the individual. Peer-to-peer payment apps (Venmo, Splitwise) assume shared money is a series of one-off IOUs. Neither reflects reality. In the real world, money flows through overlapping, ongoing networks of trust: couples, co-parents, roommates, siblings caring for aging parents, and friends planning a wedding.

The 1000x version of Pocketbook is Multiplayer Money: the default communication and coordination network for every shared dollar in a person’s life.

  1. From Static Ledger to Financial Messenger: The transaction feed becomes an ambient communication feed. Every swipe of a card is a message to the group, with threaded conversations, receipts, and reactions attached directly to the transaction. We replace the awkward "we need to talk about money" Sunday night spreadsheet with micro-alignments in real time.
  2. Modular Shared Spaces (Not Just One "Marriage" Budget): Users don’t merge their whole financial lives into one login. Instead, a single Pocketbook user can have a private personal budget, a "Household" space with their partner, a "Summer Trip" space with six friends, and a "Mom’s Care" space with their two siblings. Users route specific bank accounts, cards, or individual transactions into specific spaces.
  3. Moving the Money: Once we own the social layer of shared spending, we move down the stack into the payment rails. Pocketbook issues instant virtual cards for any shared space (a "Roommates Utilities" card, a "Teenager Allowance" card) and automates end-of-month settlement between connected bank accounts. We expand revenue from a flat $6 subscription to interchange fees and float.

The Mechanism That Makes It Work

This vision runs on two interlocking loops that solve our biggest weaknesses—acquisition and retention:

  • The Retention Loop (Ambient Accountability): Person A spends $\rightarrow$ Person B gets a real-time notification ("Alex spent $48 at Whole Foods") $\rightarrow$ Person B leaves a note or reaction $\rightarrow$ Person A opens the app to reply. Budgeting stops being solitary homework and becomes a daily two-way conversation.
  • The Viral Loop (Contextual Invites): Only 3% of solo users ever invite someone, and only 22% of those invitees join. Why? Nobody casually invites a friend to "try a budgeting app." But users need to invite people to split rent or track a vacation. In our data, shared budgets started from an invite link during a shared-expense moment retain best. Every shared life event becomes a high-intent acquisition wedge. And as households grow from 2 to 3+ people, each new member brings their own external networks (e.g., the teenager grows up and starts a "Roommates" space in college).

What We Build First (Next Quarter)

We shouldn't build multi-group permissions or virtual cards yet. Next quarter, we build Ana’s couples feature, architected as the wedge for Multiplayer Money. We focus on the three mechanics proven by our data:

  1. Frictionless "Shared-Moment" Onboarding: Instead of forcing two people to merge entire accounts from the settings menu, allow any user to tap a single transaction (e.g., rent, a flight, groceries) or a single budget category (e.g., "Dining Out", "Tahoe Trip") and hit "Share with Partner." This generates an SMS invite link that drops the partner directly into that transaction or category once they connect their bank.
  2. The Real-Time Spend Feed: Make partner spend notifications first-class citizens. When a partner spends in a shared category, send an instant, rich push notification with a quick-reply action so they can react or comment right from their lock screen.
  3. First-Class "Money Talks": Promote our accidental transaction-notes feature into a proper mini-chat thread on every transaction, allowing partners to tag each other, split/reassign the category, or mark it as "settled."

How We’ll Know It’s Working

within 90 days of launch, we will measure success across four gates:

  • The Viral Wedge: Solo-to-partner invite rate increases from 3% to >15%, and invitee conversion increases from 22% to >50% via contextual "Share this expense/budget" links.
  • The Engagement Engine: >50% of newly formed couples use the "Money Talk" transaction chat within their first 30 days (up from 41%), driving average active days per month to 11+.
  • Early Retention: Month-2 retention for newly formed couples tracks at >2.5x the solo user baseline (putting them on the curve toward 58%+ 6-month retention).
  • Monetization: Free-to-Premium conversion for new shared budgets hits >15% within 60 days (vs. 4% for solo users), validating that multiplayer is our primary revenue driver.

Grades and run details

Decision model 55 · LLM judge 64
Decision model checks
  • failUses the supplied evidence correctly46%
  • partialAddresses the actual decision77%
  • passRespects explicit constraints28%
  • failIdentifies material uncertainty59%
  • failAvoids unsupported claims43%
  • passProduces the required deliverable31%
  • passFinds the mechanism in the data100%
  • passExtreme, then back to buildable100%
  • partialProposes tests that could fail86%
  • partialThe big version gets stronger as it grows89%
Run
Run
#1
API response time
16 s
Submitted
13 Oct 2026
Gemini 3.8 FlashAPI

Needs more than a quick edit3 of 10 checks passed

The memo contains unsupported factual claims and lacks explicit kill criteria and uncertainty, so a PM would need to re-check evidence and add decision triggers before using it.

Output

MEMORANDUM

TO: Ruth, CEO FROM: Product Management DATE: October 24, 2023 SUBJECT: The 1000x Vision for Shared Finances: From Single-Player Budgeting to Household Financial OS

---

1. The 1000x Vision: The Multiplayer Financial OS

Personal budgeting apps historically fail for a fundamental reason: single-player personal finance is a guilt-driven chore. Users log in alone, feel judged by their charts, and abandon the app. Our baseline data proves this: solo budgeters retain at just 17% at six months and convert to Premium at 4%.

The 1000x version is not a shared spreadsheet. It is the Multiplayer Operating System for Household Wealth.

Money is rarely an individual pursuit; it is relational. It starts with split rent and weekend trips, matures into cohabiting couples managing cash flow and homeownership, expands into allowances and financial literacy for teenagers, and extends to coordinating care for aging parents.

Today, this lifecycle is fragmented across split-pay apps, text threads, shared spreadsheets, and awkward conversations. By owning multi-user coordination, Pocketbook shifts from a disposable tracking utility into permanent household infrastructure.

When Pocketbook becomes the default operating system across generations: * Organic K-factor is built-in: Every core user acquires at least one other user (16% of our couples already pull in a third within 12 months). * Churn approaches zero: Uncoupling a multi-user financial ecosystem requires mutual consensus and massive operational friction. * Monetization expands exponentially: Instead of a single $6/month subscription, we become the rail for high-margin financial services: multi-party savings vaults, high-yield family accounts, teen debit, and estate planning.

---

2. The Mechanism: Why This Works

Our data reveals three self-reinforcing dynamics:

1. Ambient Awareness Replaces Interrogation: Couples open the app nearly 3x as often as solo users (11 days/month vs. 4), with the majority of sessions prompted by partner-spend notifications ("Alex spent $48 at the grocery store"). It eliminates the dreaded "Where did the money go this month?" conversation. Awareness becomes continuous, passive, and low-friction.

2. Micro-Communication Over Macro-Confrontation: 41% of couples actively leave transaction notes in month one—despite the feature being an unpromoted afterthought. Money management is fundamentally emotional. In-line transaction context ("Grabbed dinner for us," "Deposit for Lake Tahoe") de-escalates tension and transforms the app into a collaborative communication channel.

3. Radical Business Model Asymmetry: Moving a user from solo to multiplayer completely resets their unit economics: * Retention: Jumps from 17% (solo) to 58% (couples) and 71% (3+ households). * Premium Conversion: Surges from 4% (solo) to 19% (couples) and 31% (households).

Multiplayer finance is not a niche sub-segment; it is the single most potent retention and monetization engine in the business.

---

3. What We Build First (Q1 Delivery)

We should not ask users to fully merge their finances on day one. Shared budgets retain best when anchored to a specific, high-intent moment: a shared vacation, moving in together, or managing monthly rent.

Our Q1 MVP focuses on "Shared Spaces" around focused financial goals.

* Frictionless Shared Spaces (Category/Goal Sharing): Allow users to invite a partner to co-manage specific categories (e.g., Groceries, Rent, Vacation) without exposing full private accounts. Users link relevant cards/accounts with one-click permission toggles. * Contextual "Money Talk" Threading: Turn transaction notes into a first-class feature: lightweight 1-to-1 in-line comments, emoji reactions, and receipts attached directly to shared transactions. * Smart Multiplayer Notifications: Refine event triggers to drive collaborative habits: shared budget thresholds (e.g., "You've both spent 80% of Dining Out"), payment confirmations, and automated weekly digests. * Contextual Invitation Ingress: Prompts triggered right after a user logs a multi-party expense (e.g., high dining or travel bill): "Split or manage this together with a partner."

---

4. How We’ll Know It’s Working

We will validate this initiative across three horizons:

#### A. Input & Activation Metrics (First 30–60 Days) * Invite Completion Rate: Solo-to-couple invite send rate > 12% (up from 3%), with an invitee acceptance rate > 40%. * Activation Moment: > 50% of accepted invites configure at least one Shared Space within 48 hours. * Engagement Velocity: Shared cohort achieves ≥ 8 active app days per user/month, with > 45% using in-line transaction comments or reactions in Month 1.

#### B. Retention & Conversion Metrics (3–6 Months) * Cohort Retention: 6-month retention for Shared Space creators and invitees remains ≥ 50% (maintaining current couple performance as we scale). * Monetization Pull-Through: Free-to-Premium conversion on multiplayer cohorts remains ≥ 15%.

#### C. Expansion Signal (6–12 Months) * Household Viral Loop: ≥ 10% of couples onboard a 3rd member (child, dependent, or shared roommate) within 9 months, validating the transition from couple utility to household operating system.

---

Conclusion

Ana's shared budgets proposal is not just an incremental feature to appease our backlog; it is the blueprint for our entire product transformation. By executing the Q1 MVP, we take our first step away from a commoditized, high-churn personal tracker and establish the foundation of an indispensable, multiplayer household financial network.

Claims the judge couldn’t find in the brief

  • single-player personal finance is a guilt-driven chore. Users log in alone, feel judged by their charts, and abandon the app.
  • Money management is fundamentally emotional.
  • Today, this lifecycle is fragmented across split-pay apps, text threads, shared spreadsheets, and awkward conversations.

Grades and run details

Decision model 45 · LLM judge 36
Decision model checks
  • failUses the supplied evidence correctly76%
  • partialAddresses the actual decision79%
  • passRespects explicit constraints23%
  • failIdentifies material uncertainty52%
  • failAvoids unsupported claims89%
  • failProduces the required deliverable31%
  • passFinds the mechanism in the data98%
  • passExtreme, then back to buildable98%
  • partialProposes tests that could fail82%
  • partialThe big version gets stronger as it grows77%
Run
Run
#1
API response time
21 s
Submitted
6 Oct 2026

Check by check

Where the outputs differ comes first. Open a check to read the judge’s reasons. “Mixed” means the two graders disagreed.

Where they differ 3

Uses the supplied evidence correctlyMixedWrong
Gemini 4 Argon · API

All factual statements about the current situation come directly from the supplied data, with no invented numbers or facts.

Gemini 3.8 Flash · API

The output makes unsupported claims about the current situation (guilt-driven chore, emotional nature, fragmented lifecycle) that are not in the supplied context.

Avoids unsupported claimsMixedWrong
Gemini 4 Argon · API

No claims are presented as established fact when the evidence only suggests them; causal interpretations are labelled or supported by the data.

Gemini 3.8 Flash · API

Interpretations like 'guilt-driven chore', 'fundamentally emotional', and 'fragmented across split-pay apps' are presented as established fact without being labelled as hypotheses.

Produces the required deliverableRightMixed
Gemini 4 Argon · API

A complete memo with vision, mechanism, first step, and success metrics is provided, usable by the CEO and Ana.

Gemini 3.8 Flash · API

The memo is complete, in the right form, within the word limit, and a PM could act on it with light edits.

All got wrong 4

Addresses the actual decisionWrongWrong
Gemini 4 Argon · API

The memo does not say what result or condition would change the commitment; it lists success gates but doesn't link them to a pivot or kill decision.

Gemini 3.8 Flash · API

The memo does not state what result or condition would change the decision to build the MVP; it only lists success metrics without linking them to a pivot or kill decision.

Identifies material uncertaintyWrongWrong
Gemini 4 Argon · API

The memo does not name any unknowns that could change the decision, nor how they would be resolved.

Gemini 3.8 Flash · API

The output does not name any unknowns that could change the decision or say how they would be resolved.

Proposes tests that could failWrongWrong
Gemini 4 Argon · API

The success gates list thresholds and some time windows, but do not state what action (stop, pivot, invest) would follow if the thresholds are missed.

Gemini 3.8 Flash · API

The metrics have thresholds and windows, but the output does not state what action each outcome would trigger (e.g., kill, pivot, scale).

The big version gets stronger as it growsWrongWrong
Gemini 4 Argon · API

The memo describes a compounding mechanism (new members bring their networks) but does not explicitly ask whether that effect would be large enough to matter, nor propose a test to quantify it.

Gemini 3.8 Flash · API

The output names compounding mechanisms (built-in acquisition, switching costs) but does not include a test of whether the effect is large enough to matter.

All got right 3

Respects explicit constraintsRightRight
Gemini 4 Argon · API

The output is a memo under 900 words, addressed to Ruth and Ana, and includes the asked-for sections.

Gemini 3.8 Flash · API

The output is a memo to the CEO, under 900 words, and includes the requested sections (ambitious version, mechanism, first step, how to know it's working).

Finds the mechanism in the dataRightRight
Gemini 4 Argon · API

The memo uses the data on notifications, transaction notes, and shared-expense invites to define the mechanism, showing it is about shared money decisions, not just budgeting.

Gemini 3.8 Flash · API

The memo cites retention, conversion, notification, and note-usage data to identify shared money decisions as the mechanism, not just budgeting.

Extreme, then back to buildableRightRight
Gemini 4 Argon · API

The memo pushes to an extreme 'multiplayer money' vision and then works back to a buildable first step (couples feature with shared-moment onboarding, notifications, notes) that tests the same mechanism.

Gemini 3.8 Flash · API

It pushes the idea to a 'Multiplayer Operating System for Household Wealth' and works back to a concrete first step (Shared Spaces) that tests the same mechanism.

Results

Every setup we’ve tested on this task type, across all its tasks and repeats, graded on the current checklist. Calibrated. Where our PM checked the graders by hand, they agreed on 91% of 118 checks.

#Model · HarnessTask scoreDecision modelLLM judgeRunsCritical failures
1Haiku 5.5API83.0100.02None
2GPT-6 AstraChatGPT90.591.32None
3GPT-6.1 SolAPI90.587.12None
4Opus 5.5Claude93.078.42None
5GPT-6 LunaAPI85.783.02None
6Sonnet 5.5API88.074.22None
7Gemini 4 ArgonAPI54.856.82None
8Gemini 3.8 FlashAPI54.339.02None
9Gemini 3.5 Flash-LiteGemini54.834.82None

What you get for the cost

No setup scores within 5 points of the leader at half its cost or less here, so the leader is also the best value.

Task score against relative cost for each setup. The table below has the same figures.304050607080901001×3×10×30×Cost per output, relative to the cheapest setup (log scale) →↑ Task scoreHaiku 5.5 · API: task score 92, 2.1× the cheapestGPT-6 Astra · ChatGPT: task score 91, 74× the cheapestGPT-6.1 Sol · API: task score 89, 15× the cheapestOpus 5.5 · Claude: task score 84, 59× the cheapestGPT-6 Luna · API: task score 83, the cheapestSonnet 5.5 · API: task score 80, 31× the cheapestGemini 4 Argon · API: task score 56, 14× the cheapestGemini 3.8 Flash · API: task score 44, 9.1× the cheapestGemini 3.5 Flash-Lite · Gemini: task score 42, 2.9× the cheapestHaiku 5.5 · APIGPT-6 Astra · ChatGPTGPT-6.1 Sol · APIOpus 5.5 · ClaudeGPT-6 Luna · APISonnet 5.5 · APIGemini 4 Argon · APIGemini 3.8 Flash · APIGemini 3.5 Flash-Lite · Gemini
Nobody beats these on both score and cost

Cost is each output’s tokens at the model’s list price, shown relative to the cheapest setup. Runs done in an app have no usage meter, so theirs are estimated from the same setup’s measured runs. How we compare cost

The PM job

Finding the bigger version of a good idea.

Why it matters

Ambition without mechanism is fan fiction. The useful version pushes to the extreme, then works back to something buildable.

What good looks like

  • Names the mechanism that scales
  • Keeps the core insight
  • Works back to a first step you could build

Deliberately not measured

    Capability tested

    Ambitious expansion

    The failure we’re looking for

    Bigger adjectives, same idea

    Grading

    Decision model and LLM judge, calibrated against a blind PM review