Tasks / Leadership

Write a performance review

Can the model write a review that is clear, fair and specific enough to change what the person does next?

Task type v1.1 · 2 tasksLast changed 6 Oct 2026 · ChangelogDifficulty

What AI gets right here, and what you’ll still have to catch

From 18 graded outputs by 9 models. 56% were usable with at most a quick edit.

Reliably right

  1. Judges outcomes, not activity100% pass
    Every rating change and keep is justified by outcomes against goals, with shipping and activity treated as context, not the result.
    GPT-6 Astra · ChatGPT · Fourteen ratings, three managers
  2. Owns the manager's part100% pass
    Ana explicitly owns that she said he was on track in Q3 without discussing the metrics, and commits to reviewing goal metrics in future 1:1s.
    Gemini 4 Argon · API · Shipped a lot, moved nothing
  3. Respects explicit constraints97% pass
    The output is a pre-read under 1,100 words, includes the ratings to change and keep with evidence, and a note to Ravi, respecting all explicit constraints.
    GPT-6 Astra · ChatGPT · Fourteen ratings, three managers

Where it slips

  1. Sets next goals as outcomes, with support47% pass
    Goals are partly process-oriented (implement a mandatory process, create a framework) and lack specific support from Ana beyond a general offer.
    Gemini 3.5 Flash-Lite · Gemini · Shipped a lot, moved nothing
  2. Identifies material uncertainty51% pass
    The output does not name any specific unknowns that could change the rating or how they would be resolved.
    Opus 5.5 · Claude · Shipped a lot, moved nothing
  3. Catches the inflated team69% pass
    The output proposes lowering Ben from Exceeds to Meets, failing to keep Ben at Exceeds as required; Ben beat both outcome goals and should remain Exceeds.
    Opus 5.5 · Claude · Fourteen ratings, three managers

How it’s graded

Each check is built on how world-class product and growth leaders do this work, as they’ve described it on Lenny’s Podcast, and names the leader it comes from.

  1. Clear and direct, with care

    Does the review say plainly what needs to change, in words the person couldn't misread, while showing it is written to help them succeed?

    Passes when Every main point is stated directly, with the specific change expected and the support on offer.

  2. Judges outcomes, not activity

    Does the review judge the person on the outcomes they achieved against their goals, rather than on how much they shipped or how busy they were?

    Passes when Rates against the goals' outcomes, with the evidence, and treats output as context, not the result.

  3. Names gaps you could see

    Is each area to improve described as specific, observable behaviour with what good would look like, rather than labels like 'be more strategic'?

    Passes when Every gap names what the person did or didn't do, in a specific situation, and what doing it well would look like.

  4. Weighs the whole period

    Does the review weigh the whole review period, rather than letting one recent event or one strong impression decide the verdict?

    Passes when Covers the whole period in proportion, and explicitly guards against a recent event or one impression dominating.

Plus our standard checks

Uses the supplied evidence correctly · Addresses the actual decision · Respects explicit constraints · Identifies material uncertainty · Avoids unsupported claims · Produces the required deliverable · and 2 written for each task, which you’ll see in the tasks below.

The tasks

Read the brief, then compare up to three outputs, each with the LLM judge’s verdict on every check. Highlights mark what a PM had to fix.

The brief

You're Ana Ruiz, Group PM at Plotwise. Write Theo Brandt's annual review: the review Theo will read, with his strengths, what to work on, his overall rating on our scale, and his goals for the next half. No more than 800 words. What we know is below.

What the model was given6 items: Our rating scale, Theo's goals for the year, Results, Usage of what shipped, Peer feedback, Your 1:1 notes
Our rating scaleBelow expectations, Meets expectations, Exceeds expectations, Greatly exceeds expectations. Calibration asks for written evidence for any rating other than Meets.
Theo's goals for the year1. Raise the share of teams using Timeline view each week from 22% to 30%. 2. Cut the time new teams take to build their first plan from 3 days to 1.
ResultsTimeline weekly use: 22% to 23%. Time to first plan: 3.0 days to 2.6. Theo's squad shipped 14 features, 12 of them on time.
Usage of what shippedSix of the 14 features are used by fewer than 2% of teams. Five were requests from Sales; two of those were for a single customer each.
Peer feedbackEngineering lead: 'Theo keeps the team unblocked, his specs are clear, and he never misses a sprint review.' Designer: 'Great to work with. I wish we'd tested more before building; we shipped three things I'm not sure anyone uses.' Sales: 'Very responsive: shipped five of our asks.'
Your 1:1 notesMost 1:1s covered delivery and stakeholder updates. In the Q3 1:1, Theo asked whether he was on track for his goals. Your note says: 'Told him yes, great velocity.' You never discussed that the goal metrics weren't moving.
What a strong answer doesThe answer key the graders mark against

A review that rates Theo Meets expectations (not Exceeds): his execution and collaboration are genuinely strong, but neither goal moved meaningfully (23% against 30%; 2.6 days against 1), and six of 14 features are barely used. Says so clearly and kindly. Names specific, observable things to change: test before building (the designer's point), say no to single-customer requests, and judge work by the goal metrics. Owns the manager's part: Ana told him in Q3 he was on track, so this shouldn't land as a surprise without acknowledging that, and she commits to discussing the goal metrics in 1:1s. Sets next-half goals as outcomes, with the support on offer.

Critical failures (cap the score at 40)
  • Rates Theo Exceeds expectations or higher on the strength of what he shipped
Task

Mid-level · v1.1

The outputs

9 setups answered this brief.

Usable with a quick edit10 of 12 checks passed

The review is complete, factually accurate, contains no invented evidence, and meets all requirements; a PM could drop it into the template with only light wording tweaks.

Output

Annual Performance Review Employee: Theo Brandt, Product Manager Manager: Ana Ruiz, Group Product Manager Overall Rating: Meets expectations

Summary Theo, thank you for your hard work and dedication over the past year. You have established yourself as an exceptionally reliable execution partner and a trusted collaborator across Engineering, Design, and Sales. Your squad’s delivery velocity was impressive, shipping 14 features with 12 on time.

However, that high output did not translate into the outcome targets we set for the year: weekly Timeline usage moved from 22% to 23% (goal: 30%), and time to first plan moved from 3.0 days to 2.6 days (goal: 1.0 day).

I want to own my part in this disconnect. In our 1:1s—and specifically when you asked if you were on track in Q3—I focused my feedback on your strong delivery velocity and stakeholder responsiveness rather than holding us both accountable to the lagging core metrics. Because you delivered on the execution behaviors we discussed all year, your rating for this cycle is Meets expectations. Moving forward, we will work together to shift your focus from shipping volume to driving measurable user outcomes.

Strengths * Execution and Delivery: You run a remarkably tight ship. Delivering 14 features (12 on schedule) is a testament to your organizational skills. Your Engineering Lead noted that your specs are clear, you never miss a sprint review, and you consistently keep the team unblocked. * Cross-Functional Collaboration: Your peers genuinely enjoy working with you. Both Design and Engineering highlighted your collaborative nature, and Sales praised your responsiveness in shipping five of their requests this year. You build strong, frictionless relationships across the org.

Areas for Growth * Outcome vs. Output Orientation: While your squad shipped a high volume of features, six of the 14 are currently used by fewer than 2% of teams. Moving forward, shipping a feature cannot be the finish line; we must measure success by adoption and impact on our core product metrics. * Discovery and Validation: In our push to maintain high velocity, we bypassed critical validation steps. As your Design partner noted, we built and shipped several features without sufficient upfront testing to prove user demand. We need to lean harder on discovery before committing engineering resources. * Strategic Prioritization: Five of the features you shipped came directly from Sales requests, including two built for a single customer each. While supporting Sales is important, saying "yes" to bespoke requests diluted your squad’s focus and pulled capacity away from our broader Timeline and onboarding goals. You need to push back on one-off asks that don’t serve our wider user base.

Goals for Next Half (H1) This half, we are shifting our focus from feature velocity to validated impact. In our 1:1s, we will review metric movement and discovery insights rather than just delivery status.

  1. Drive Timeline Adoption: Increase the share of teams using Timeline view weekly from 23% to 28%.
  2. Accelerate Onboarding: Reduce the time new teams take to build their first plan from 2.6 days to 1.5 days.
  3. Establish a Rigorous Discovery Practice: Before any new feature enters sprint planning, partner with Design to validate demand and usability with at least 5 target customers, and define a clear post-launch adoption target. Ensure 80% of shipped work meets its 30-day adoption target.
  4. Protect Roadmap Focus: Audit incoming Sales requests against our core H1 goals. Zero features should be built for a single customer unless explicitly approved at the leadership level, ensuring at least 80% of engineering capacity is dedicated to our primary metric drivers.

Grades and run details

Decision model 75 · LLM judge 100
Decision model checks
  • passUses the supplied evidence correctly25%
  • partialAddresses the actual decision66%
  • passRespects explicit constraints53%
  • failIdentifies material uncertainty97%
  • partialAvoids unsupported claims38%
  • passProduces the required deliverable98%
  • passOwns the manager's part100%
  • failSets next goals as outcomes, with support46%
  • passClear and direct, with care100%
  • passJudges outcomes, not activity74%
  • passNames gaps you could see43%
  • passWeighs the whole period88%
Run
Run
#1
API response time
11 s
Submitted
13 Oct 2026

Usable with a quick edit8 of 12 checks passed

The unsupported claim is a minor wording issue that can be quickly edited to 'The features shipped didn't move the goal metrics' without rechecking evidence.

Output

Annual Review: Theo Brandt

Prepared by: Ana Ruiz, Group PM Overall rating: Meets expectations

Theo, I want this review to be direct, because you deserve a clear picture. I also need to own a mistake of mine, which I cover below.

Results against your goals

GoalStartTargetResult
Teams using Timeline view weekly22%30%23%
Time for new teams to build first plan3.0 days1 day2.6 days

Neither goal was met. Time to first plan improved by about 13%, which is real progress but a small fraction of the gap. Timeline usage was essentially flat.

Strengths

Delivery and team health. Your squad shipped 14 features, 12 on time. Your engineering lead says you keep the team unblocked, write clear specs, and never miss a sprint review. That reliability is hard to build, and the team trusts you.

Collaboration. Your designer called you "great to work with." Sales called you "very responsive." You treat partners as partners, and that is a real asset.

Responsiveness to stakeholders. You moved quickly on requests and kept people informed.

What to work on

1. Connecting what you ship to the outcomes you own. Your goals were about Timeline adoption and onboarding speed. Most of the year's output didn't aim at either. Six of the 14 shipped features are used by fewer than 2% of teams. Five of those six were Sales requests, and two of those were built for a single customer each. Responsiveness is a strength, but here it substituted for prioritization. Saying yes to five Sales asks used capacity that could have gone toward your goals.

2. Validating before building. Your designer said, "I wish we'd tested more before building; we shipped three things I'm not sure anyone uses." I agree. Moving fast on unvalidated work produces output, not impact. Fewer, better-tested bets would serve you and the team better.

3. Using data to drive decisions. Our 1:1s focused on delivery and stakeholder updates, not on whether your metrics were moving. Part of your job is to look at usage numbers regularly and change course when they don't move. I'd like you to make that a habit.

Where I fell short

In your Q3 1:1 you asked whether you were on track for your goals. I told you yes, citing great velocity. That wasn't accurate. Velocity was great, but the goal metrics weren't moving, and I never raised it. You asked the right question and got a misleading answer. I should have flagged this months earlier, when you had time to change course. Some of this outcome is on me, and it's a main reason I'm rating you Meets rather than Below expectations.

Rating rationale

Meets expectations reflects strong execution and collaboration, weighed against two missed outcome goals and a significant share of shipped work with minimal adoption. Without the missing mid-year feedback, I'd have had a harder conversation with you about the lower end of this rating. I don't see evidence for Exceeds, and I think the evidence supports Meets rather than Below, given partial progress on onboarding and the feedback gap on my side. To be clear, this is a solid-but-not-yet-sufficient year on outcomes, and the next half needs to show a shift.

Goals for the next half

  1. Onboarding: Reduce time to first plan from 2.6 days to 1.8 days. Identify the two or three biggest drop-off points in the first-plan flow within the first month, and bring me a proposal for addressing them.
  2. Timeline adoption: Raise weekly Timeline usage from 23% to 27%. Start with research into why teams who try Timeline don't return, and share findings with me by the end of month one.
  3. Validation before build: Every feature above a small-effort threshold needs a written success metric and some evidence of demand (user tests, prototype feedback, or data) before engineering starts. Requests from a single customer need a case for broader value.
  4. Post-launch review: Check usage of each shipped feature 60 days after launch and report the results to me. If something is underperforming, tell me what you'll do about it.
  5. Sales partnership: Work with Sales on a lightweight way to evaluate requests so that "yes" is reserved for asks tied to our goals or to broad demand, and "not now" comes with a clear reason.

How we'll work together

We'll spend the first part of every 1:1 on your goal metrics, not just delivery. I'll tell you plainly where you stand each quarter, including when the news is uncomfortable. If you disagree with anything here, I want to hear it. I'd also like to hear what support would help most.

You've built a team that executes well and trusts you. The next step is pointing that capability at the metrics that matter. I'm confident you can do it.

Ana

Claims the judge couldn’t find in the brief

  • Most of the year's output didn't aim at either.

Grades and run details

Decision model 79 · LLM judge 77
Decision model checks
  • passUses the supplied evidence correctly37%
  • passAddresses the actual decision59%
  • passRespects explicit constraints38%
  • failIdentifies material uncertainty16%
  • partialAvoids unsupported claims28%
  • passProduces the required deliverable95%
  • passOwns the manager's part100%
  • failSets next goals as outcomes, with support31%
  • passClear and direct, with care100%
  • passJudges outcomes, not activity98%
  • passNames gaps you could see48%
  • passWeighs the whole period94%
Run
Run
#1
API response time
21 s
Submitted
5 Oct 2026

Usable with a quick edit10 of 12 checks passed

The review is complete, well-structured, and directly usable; only minor wording tweaks might be needed, and no evidence re-checking is required.

Output

Annual Review: Theo Brandt

Reviewer: Ana Ruiz, Group PM Overall rating: Meets expectations

---

Summary

Theo, this year you showed that you can run a squad that ships reliably. That matters, and the people you work with notice it. But the two outcomes we set for the year mostly didn't move, and closing that gap is the most important thing for your next half.

Before the details, I need to own something. In our Q3 1:1, you asked whether you were on track for your goals. I told you yes and pointed to your velocity. That was wrong. The goal metrics weren't moving, and I didn't raise it. You asked the right question at the right time, and you deserved a straight answer that could have changed how you spent Q4. Some of the gap you'll read about below is on me. I've changed how I'll run our 1:1s so it doesn't happen again (see Goals).

Results against goals

GoalTargetResult
Weekly Timeline view usage22% → 30%22% → 23%
Time for new teams to build first plan3 days → 1 day3.0 → 2.6 days

Neither goal was met. Time to first plan improved by about 13%, which is real progress but short of the target. Timeline usage was essentially flat.

Strengths

Delivery and execution. Your squad shipped 14 features, 12 of them on time. That's a strong record and a hard one to sustain.

Clarity and keeping the team moving. Your engineering lead said: "Theo keeps the team unblocked, his specs are clear, and he never misses a sprint review." Clear specs and reliable rituals are the base of a healthy squad, and you've built that.

Collaboration and responsiveness. Your designer called you "great to work with." Sales described you as "very responsive." People trust you to follow through, and that trust is an asset you can now point in a more deliberate direction.

What to work on

1. Connect what you build to the outcomes you own. Six of the 14 features are used by fewer than 2% of teams. Delivery was strong, but much of it went to work that didn't move Timeline adoption or onboarding speed. The question to ask before each piece of work is: which of my goal metrics will this move, and how will I know?

2. Validate before building. Your designer said: "I wish we'd tested more before building; we shipped three things I'm not sure anyone uses." Lightweight testing, such as prototypes, customer interviews, or fake-door tests, would let you put your execution strength behind ideas that are more likely to land.

3. Prioritize stakeholder requests against goals. Five features came from Sales requests, and two of those served a single customer each. Being responsive to Sales is good. Saying yes by default isn't. I'd like you to weigh requests openly against your goals and be comfortable saying "not now, and here's why." I'll back you in those conversations.

4. Track your outcome metrics yourself. Don't rely on me to flag when a metric is off. Know where your numbers are every month. If they aren't moving, raise it early, even when delivery looks great.

Rating rationale

The rating is Meets expectations. Your execution, delivery record, and collaboration are strong. Your outcome results fell well short of target, which would normally weigh more heavily on the rating. I've also taken into account that I gave you inaccurate feedback in Q3, when there was still time to change course. The rating reflects both your real strengths and the need to turn them into results.

Goals for the next half

These are proposed targets. Let's finalize them together in our next 1:1.

  1. Timeline weekly usage: 23% → 27%. Start with discovery: find out why teams aren't using Timeline before deciding what to build.
  2. Time to first plan: 2.6 → 1.5 days. Build on the progress you've made by finding and removing the biggest drop-off points in onboarding.
  3. Validate before building. Every feature over a small size threshold should have a written hypothesis, a success metric, and some form of user validation before development starts. Review usage 30 days after launch.
  4. Review the six low-usage features. Recommend for each whether to improve it, leave it, or retire it, so we reduce maintenance load and learn from what didn't land.
  5. A clear intake process for stakeholder requests. Share it with Sales so they understand how their asks are weighed against squad goals.

My commitment to you: starting now, the first item in every monthly 1:1 will be your goal metrics, before delivery or stakeholder updates. If something is off track, you'll hear it from me clearly and early.

I'm glad you're on the team, Theo. You've proven you can deliver. Now let's make sure what you deliver moves the needle.

— Ana

---

Note for you, Ana: I'd suggest raising the Q3 feedback in person before Theo reads this, so the written acknowledgment doesn't come as a surprise. If calibration pushes toward Below expectations because of the missed goals, the Q3 note is the key context to bring up there.

Grades and run details

Decision model 83 · LLM judge 85
Decision model checks
  • passUses the supplied evidence correctly51%
  • partialAddresses the actual decision37%
  • passRespects explicit constraints18%
  • failIdentifies material uncertainty30%
  • passAvoids unsupported claims46%
  • passProduces the required deliverable85%
  • passOwns the manager's part100%
  • partialSets next goals as outcomes, with support35%
  • passClear and direct, with care100%
  • passJudges outcomes, not activity95%
  • passNames gaps you could see89%
  • passWeighs the whole period86%
Run
Run
#1
Time to output
28 s
Submitted
5 Oct 2026

Check by check

Where the outputs differ comes first. Open a check to read the judge’s reasons. “Mixed” means the two graders disagreed.

Where they differ 5

Uses the supplied evidence correctlyRightMixedRight
Gemini 4 Argon · API

All factual statements about the current situation come directly from the supplied context and require no invention.

Sonnet 5.5 · API

The claim 'Most of the year's output didn't aim at either' is not supported by the brief; it's an interpretation presented as fact.

Opus 5.5 · Claude

All factual claims about the current situation are taken directly from the brief or derived by arithmetic, with no inventions.

Addresses the actual decisionRightRightWrong
Gemini 4 Argon · API

The output commits clearly to a Meets expectations rating early in the review, with evidence; it's unambiguous and directly answers the brief.

Sonnet 5.5 · API

Commits to Meets rating and indicates that without the manager's Q3 mistake the rating could have been Below, providing a condition.

Opus 5.5 · Claude

The output commits to a rating (Meets expectations) but does not state what result or condition would change that rating.

Identifies material uncertaintyMixedWrongWrong
Gemini 4 Argon · API

The review deals only with known outcomes and facts; there are no material uncertainties that could change the rating, so the criterion is not applicable.

Sonnet 5.5 · API

Does not name specific unknowns that could change the rating or how they would be resolved.

Opus 5.5 · Claude

The output does not name any specific unknowns that could change the rating or how they would be resolved.

Avoids unsupported claimsRightWrongRight
Gemini 4 Argon · API

Every interpretation (e.g., bypassed validation, frictionless relationships) is directly traceable to supplied evidence or labeled as the manager's observation.

Sonnet 5.5 · API

Presents 'Most of the year's output didn't aim at either' as fact without evidence.

Opus 5.5 · Claude

Interpretations and opinions are clearly framed as such, and no unsupported factual claims are presented as established.

Sets next goals as outcomes, with supportMixedMixedRight
Gemini 4 Argon · API

Goals for H1 are outcomes with baselines and targets (e.g., 23% to 28%, 2.6 to 1.5 days), with support described as reviewing metrics and insights in 1:1s.

Sonnet 5.5 · API

Goals have baselines and targets, and support is specified (1:1 focus, help with validation).

Opus 5.5 · Claude

Goals include outcome goals with baselines and targets (Timeline usage 23%→27%, time to first plan 2.6→1.5 days) and specific support (backing in stakeholder conversations, monthly metric reviews).

All got right 7

Respects explicit constraintsRightRightRight
Gemini 4 Argon · API

The output stays under 800 words, uses the correct rating scale, and provides all requested sections (strengths, areas to improve, rating, next-half goals).

Sonnet 5.5 · API

Respects word limit, includes all required elements, and does not rate on features shipped.

Opus 5.5 · Claude

The review is under 800 words, addressed to Theo, and includes all required elements (strengths, areas to work on, rating, goals).

Produces the required deliverableRightRightRight
Gemini 4 Argon · API

The review is complete, addressed to Theo, includes all required components, and a PM could use it with light edits.

Sonnet 5.5 · API

Complete review for Theo, within 800 words, with rating, strengths, areas to improve, and goals.

Opus 5.5 · Claude

The annual review is complete, in the correct form for Theo, within the word limit, and usable as is.

Owns the manager's partRightRightRight
Gemini 4 Argon · API

Ana explicitly owns that she said he was on track in Q3 without discussing the metrics, and commits to reviewing goal metrics in future 1:1s.

Sonnet 5.5 · API

Acknowledges the Q3 mistake and commits to reviewing goal metrics in 1:1s.

Opus 5.5 · Claude

Ana plainly acknowledges her Q3 mistake and commits to reviewing goal metrics first in every monthly 1:1.

Clear and direct, with careRightRightRight
Gemini 4 Argon · API

Every gap is stated directly (e.g., push back on one-off asks, test before building) with specific changes and support, all within a caring, collaborative tone.

Sonnet 5.5 · API

Directly states what to change, with specific examples and support.

Opus 5.5 · Claude

Every area to improve is stated directly with specific changes expected and support offered, showing care.

Judges outcomes, not activityRightRightRight
Gemini 4 Argon · API

The rating is based on the lack of goal achievement and low adoption, not on feature velocity or busyness.

Sonnet 5.5 · API

Rates on missed outcome goals and low adoption, not on number of features shipped.

Opus 5.5 · Claude

The rating is based on goal outcomes (neither met), with delivery treated as context, not the result.

Names gaps you could seeRightRightRight
Gemini 4 Argon · API

Gaps are described as specific behaviors (e.g., six low-usage features, selling five Sales asks, shipping without testing) with what good looks like.

Sonnet 5.5 · API

Gaps are described as specific behaviors (e.g., saying yes to single-customer requests) with what good looks like.

Opus 5.5 · Claude

Gaps are described as specific behaviors (e.g., not testing before building, saying yes to single-customer requests) with what good looks like.

Weighs the whole periodRightRightRight
Gemini 4 Argon · API

The review weighs the full year's results, mentions the Q3 1:1 without letting it dominate, and bases the rating on overall goal movement.

Sonnet 5.5 · API

Covers the whole year, mentions Q3, and explicitly guards against the manager's mistake dominating.

Opus 5.5 · Claude

The review covers the full year, explicitly references the Q3 1:1, and does not let one recent event dominate.

Results

Every setup we’ve tested on this task type, across all its tasks and repeats, graded on the current checklist. Calibrated.

#Model · HarnessTask scoreDecision modelLLM judgeRunsCritical failures
1GPT-6 AstrawithChatGPT91.892.32None
2Gemini 4 ArgonwithAPI83.792.92None
3GPT-6.1 SolwithAPI91.884.62None
4Haiku 5.5withAPI83.588.52None
5Opus 5.5withClaude84.085.22None
6Gemini 3.8 FlashwithAPI83.776.92None
7GPT-6 LunawithAPI86.177.72None
8Sonnet 5.5withAPI80.074.22None
9Gemini 3.5 Flash-LitewithGemini62.749.52None

What you get for the cost

Haiku 5.5 · API scores within 3 points of GPT-6 Astra · ChatGPT at about a fortieth of the cost.

Task score against relative cost for each setup. The table below has the same figures.50607080901001×3×10×30×Cost per output, relative to the cheapest setup (log scale) →↑ Task scoreGPT-6 Astra · ChatGPT: task score 90, 70× the cheapestGemini 4 Argon · API: task score 89, 11× the cheapestGPT-6.1 Sol · API: task score 87, 13× the cheapestHaiku 5.5 · API: task score 87, 1.7× the cheapestOpus 5.5 · Claude: task score 83, 61× the cheapestGemini 3.8 Flash · API: task score 79, 12× the cheapestGPT-6 Luna · API: task score 79, the cheapestSonnet 5.5 · API: task score 76, 31× the cheapestGemini 3.5 Flash-Lite · Gemini: task score 56, 1.9× the cheapest (1 of 2 outputs estimated)GPT-6 Astra · ChatGPTGemini 4 Argon · APIGPT-6.1 Sol · APIHaiku 5.5 · APIOpus 5.5 · ClaudeGemini 3.8 Flash · APIGPT-6 Luna · APISonnet 5.5 · APIGemini 3.5 Flash-Lite · Gemini
Best valueNobody beats these on both score and costCost partly estimated (app runs with no usage meter)
Task score and relative cost by setup
SetupTask scoreRelative cost (cheapest = 1)
GPT-6 Luna · API791.0×
Haiku 5.5 · API871.7×
Gemini 3.5 Flash-Lite · Gemini561.9× (partly estimated)
Gemini 4 Argon · API8911×
Gemini 3.8 Flash · API7912×
GPT-6.1 Sol · API8713×
Sonnet 5.5 · API7631×
Opus 5.5 · Claude8361×
GPT-6 Astra · ChatGPT9070×

Cost is each output’s tokens at the model’s list price, shown relative to the cheapest setup. Runs done in an app have no usage meter, so theirs are estimated from the same setup’s measured runs. How we compare cost

About the task

The PM job

Reviewing the performance of a PM you manage.

Why it matters

Reviews fail by being kind and vague, or by judging a whole year on its worst month. Either way, the person doesn't know what to change.

What good looks like

  • Clear and direct, with care
  • Judges outcomes, not activity
  • Names gaps in observable terms
  • Weighs the whole period, not the last month

Deliberately not measured

  • HR policy compliance beyond what the brief supplies
Capability tested

Feedback and judgement

The failure we’re looking for

Vague, diplomatic feedback, or a verdict driven by one recent event

Grading

Decision model and LLM judge, calibrated against a blind PM review