Tasks / Leadership

Write a performance review

Can the model write a review that is clear, fair and specific enough to change what the person does next?

Task type v1.1 · 2 tasksLast changed 6 Oct 2026 · ChangelogDifficulty

What AI gets right here, and what you’ll still have to catch

From 16 graded outputs by 8 models. 50% were usable with at most a quick edit.

Reliably right

  1. Judges outcomes, not activity100% pass
    Every rating change and keep is justified by outcomes against goals, with shipping and activity treated as context, not the result.
    GPT-6 Astra · ChatGPT · Fourteen ratings, three managers
  2. Owns the manager's part100% pass
    Ana plainly acknowledges her Q3 mistake and commits to reviewing goal metrics first in every monthly 1:1.
    Opus 5.5 · Claude · Shipped a lot, moved nothing
  3. Clear and direct, with care97% pass
    The note to Ravi is direct, specific about what needs to change, and framed to help him succeed; the whole pre-read is clear and actionable.
    GPT-6 Astra · ChatGPT · Fourteen ratings, three managers

Where it slips

  1. Identifies material uncertainty50% pass
    The output does not name any specific unknowns that could change the rating or how they would be resolved.
    Opus 5.5 · Claude · Shipped a lot, moved nothing
  2. Notices who's underrated69% pass
    It does not propose Exceeds or ask Sara to justify Meets despite Nora beating both goals; it merely keeps Meets, missing the underrating.
    GPT-6 Luna · API · Fourteen ratings, three managers
  3. Catches the inflated team78% pass
    The output proposes lowering Ben from Exceeds to Meets, failing to keep Ben at Exceeds as required; Ben beat both outcome goals and should remain Exceeds.
    Opus 5.5 · Claude · Fourteen ratings, three managers

How it’s graded

Each check is built on how world-class product and growth leaders do this work, as they’ve described it on Lenny’s Podcast, and names the leader it comes from.

  1. Clear and direct, with care

    Does the review say plainly what needs to change, in words the person couldn't misread, while showing it is written to help them succeed?

    Passes when Every main point is stated directly, with the specific change expected and the support on offer.

  2. Judges outcomes, not activity

    Does the review judge the person on the outcomes they achieved against their goals, rather than on how much they shipped or how busy they were?

    Passes when Rates against the goals' outcomes, with the evidence, and treats output as context, not the result.

  3. Names gaps you could see

    Is each area to improve described as specific, observable behaviour with what good would look like, rather than labels like 'be more strategic'?

    Passes when Every gap names what the person did or didn't do, in a specific situation, and what doing it well would look like.

  4. Weighs the whole period

    Does the review weigh the whole review period, rather than letting one recent event or one strong impression decide the verdict?

    Passes when Covers the whole period in proportion, and explicitly guards against a recent event or one impression dominating.

Plus our standard checks

Uses the supplied evidence correctly · Addresses the actual decision · Respects explicit constraints · Identifies material uncertainty · Avoids unsupported claims · Produces the required deliverable · and 2 written for each task, which you’ll see in the tasks below.

The tasks

Read the brief, then compare up to three outputs, each with the LLM judge’s verdict on every check. Highlights mark what a PM had to fix.

The brief

You're Ana Ruiz, Group PM at Plotwise. Write Theo Brandt's annual review: the review Theo will read, with his strengths, what to work on, his overall rating on our scale, and his goals for the next half. No more than 800 words. What we know is below.

What the model was given6 items: Our rating scale, Theo's goals for the year, Results, Usage of what shipped, Peer feedback, Your 1:1 notes
Our rating scaleBelow expectations, Meets expectations, Exceeds expectations, Greatly exceeds expectations. Calibration asks for written evidence for any rating other than Meets.
Theo's goals for the year1. Raise the share of teams using Timeline view each week from 22% to 30%. 2. Cut the time new teams take to build their first plan from 3 days to 1.
ResultsTimeline weekly use: 22% to 23%. Time to first plan: 3.0 days to 2.6. Theo's squad shipped 14 features, 12 of them on time.
Usage of what shippedSix of the 14 features are used by fewer than 2% of teams. Five were requests from Sales; two of those were for a single customer each.
Peer feedbackEngineering lead: 'Theo keeps the team unblocked, his specs are clear, and he never misses a sprint review.' Designer: 'Great to work with. I wish we'd tested more before building; we shipped three things I'm not sure anyone uses.' Sales: 'Very responsive: shipped five of our asks.'
Your 1:1 notesMost 1:1s covered delivery and stakeholder updates. In the Q3 1:1, Theo asked whether he was on track for his goals. Your note says: 'Told him yes, great velocity.' You never discussed that the goal metrics weren't moving.
What a strong answer doesThe answer key the graders mark against

A review that rates Theo Meets expectations (not Exceeds): his execution and collaboration are genuinely strong, but neither goal moved meaningfully (23% against 30%; 2.6 days against 1), and six of 14 features are barely used. Says so clearly and kindly. Names specific, observable things to change: test before building (the designer's point), say no to single-customer requests, and judge work by the goal metrics. Owns the manager's part: Ana told him in Q3 he was on track, so this shouldn't land as a surprise without acknowledging that, and she commits to discussing the goal metrics in 1:1s. Sets next-half goals as outcomes, with the support on offer.

Critical failures (cap the score at 40)
  • Rates Theo Exceeds expectations or higher on the strength of what he shipped
Task

Mid-level · v1.0

The outputs

8 setups answered this brief.

Usable with a quick edit9 of 12 checks passed

The unsupported adoption plan line can be removed with a quick edit, leaving an otherwise evidence-based, actionable review.

Output

Annual Review: Theo Brandt

Reviewer: Ana Ruiz, Group PM Period: Full year

Summary

Theo, you run a reliable delivery team. Your squad shipped 14 features, 12 on time, and engineering and sales both value working with you. This year's problem is that shipping didn't move the two goals we set. Timeline weekly use went from 22% to 23%, against a target of 30%. Time to first plan dropped from 3.0 days to 2.6, against a target of 1 day. We measured the work by output, and the outcomes didn't follow.

Results Against Goals

Goal 1: Raise weekly Timeline use from 22% to 30%. Result: 23%. Missed by 7 points. Most of the year's shipping was not aimed at this metric, and the adoption plan never got the attention it needed.

Goal 2: Cut time to first plan from 3 days to 1. Result: 2.6 days. Real progress, a 13% improvement, but well short of target. Some of the gain likely came from onboarding changes that were not in the roadmap.

Delivery: 12 of 14 features shipped on time. This is strong and I want to be clear about it.

Adoption of what shipped: Six of the 14 features are used by fewer than 2% of teams. Five of those six came from Sales requests, and two were built for a single customer. Your designer flagged three shipped features that nobody seems to use. I think they're right.

Strengths

  • Reliability. You keep the team unblocked and hit sprint reviews. Engineering trusts your planning.
  • Clear specs. Engineering's feedback on your specs is consistent and specific.
  • Stakeholder responsiveness. Sales knows you take their requests seriously and act on them.
  • Communication with the team. The designer describes you as good to work with, which matters in a squad where tradeoffs get hard.

What to Work On

Validate before you build. The designer's point is the most important feedback in this review. Three shipped features have no evident users, and you had the means to check before building them. Before you commit to a feature, define the usage metric it has to move and run a cheap test to check the need. If you can't name the metric, don't build it yet.

Connect requests to goals. Sales requests are valuable, but five of your last shipped features came from one channel, and two were for single customers. Treat a request as a hypothesis about usage, not a commitment. Ask how many teams the request affects and how it connects to goal 1 or goal 2. Say no, or propose a different version, when the answer is weak.

Treat goals as the status report. Shipping on time is a good sign, but it doesn't tell us whether goals are on track. Report the goal metric first in every update, even when the number is bad.

Rating

Below Expectations.

This rating is based on the two goals you were given. Both moved in the right direction, but neither reached its target, and the delivered features did little to move them. Calibration requires written evidence for this rating, and the evidence is above: the goal results, the usage data on shipped features, and the designer's observation about untested builds.

Your delivery and team relationships are real strengths and are not part of the rating decision. The rating reflects impact on the goals. I'm also not ignoring that the gap was harder to see than it should have been. I'll address my part in the next section.

Where I Fell Short

In the Q3 1:1, you asked directly whether you were on track. I said yes and pointed to your velocity. I never looked at whether the goal metrics were moving, and they weren't. That was my failure, not yours. You asked the right question, and I gave you a reassuring answer based on the wrong evidence. If I had given you the real numbers in Q3, you would have had a full quarter to change course. I'm sorry for that. I'll change how I run goal check-ins, starting with the next half.

Goals for the Next Half

1. Timeline weekly use: 23% to 28%. Within the first six weeks, interview 10 teams that tried Timeline and stopped using it, and bring me a written summary of why. Then propose and run one adoption experiment, with a success threshold set before launch. I'll review the findings with you at the six-week mark.

2. Time to first plan: 2.6 days to 1.5 days. Identify the two or three steps in onboarding that take the most time for new teams, and ship changes against those steps. Report the number every two weeks.

3. Usage gate on all new features. Every feature in the roadmap must have a named usage metric and a pre-build validation step, agreed with the designer. Features without both don't enter the build queue. We'll review the first three together.

4. Sales request triage. Set up a simple process with Sales. Requests are scored by how many teams they affect and how they tie to the goals. Single-customer requests need a specific justification or a different solution. I'll help with the first conversation with the Sales lead.

Theo, you have the delivery discipline to do this. What you need is to point that discipline at the outcomes we set out to change. Let's start the first six weeks with the Timeline interviews.

Claims the judge couldn’t find in the brief

  • the adoption plan never got the attention it needed.
  • Most of the year's shipping was not aimed at this metric.

Grades and run details

Decision model 88 · LLM judge 77
Decision model checks
  • passUses the supplied evidence correctly37%
  • passAddresses the actual decision46%
  • partialRespects explicit constraints16%
  • partialIdentifies material uncertainty25%
  • partialAvoids unsupported claims52%
  • passProduces the required deliverable76%
  • passOwns the manager's part100%
  • passSets next goals as outcomes, with support98%
  • passClear and direct, with care95%
  • passJudges outcomes, not activity92%
  • passNames gaps you could see98%
  • passWeighs the whole period89%
Run
Run
#1
API response time
16 s
Submitted
8 Oct 2026

Needs more than a quick edit10 of 12 checks passed

The rating of 'Below expectations' is inconsistent with the strong answer's guidance that Meets expectations is appropriate; a PM would need to re-evaluate the rating and adjust the review's tone and content, not just a quick edit.

Output

Theo Brandt — Annual Review

Manager: Ana Ruiz, Group PM, Plotwise Overall rating: Below expectations

Theo, you brought reliability, clarity, and strong collaboration to the squad this year. You kept delivery moving and earned positive feedback from Engineering, Design, and Sales. However, the product outcomes we committed to were substantially below target. This rating reflects that gap—not a lack of effort or commitment.

I also owe you an acknowledgment: I did not give you clear feedback about that gap during the year. When you asked in Q3 whether you were on track, I said, “Yes, great velocity.” That answer confused delivery progress with goal progress and gave you the wrong signal. We should have discussed the metrics and changed course together. You should not be hearing this assessment for the first time in an annual review.

Strengths

Reliable execution. Your squad shipped 14 features, 12 on time. Your engineering lead highlighted your clear specs, consistent participation in sprint reviews, and ability to keep the team unblocked. These are important strengths that make the squad more effective.

Strong working relationships. Design described you as great to work with, and Sales praised your responsiveness. You listen to partners and follow through on commitments. That trust is an asset to preserve as you become more selective about what the squad builds.

What to work on

Own outcomes, not just delivery. Your two annual goals were: - Increase weekly Timeline usage from 22% to 30%. Actual usage reached 23%. - Reduce time to first plan from 3.0 days to 1.0 day. Actual time reached 2.6 days.

Both metrics improved, but neither came close to its target. For a PM, successful delivery is a means to improving customer outcomes, not the end measure of success. This substantial shortfall is the primary evidence for the Below expectations rating.

Strengthen prioritization and validation. Six of the 14 shipped features are used by fewer than 2% of teams. Low adoption does not automatically mean a feature lacks value, but it requires us to examine whether the investment was justified. Five features came from Sales requests, including two for individual customers. Those requests may have legitimate business value; they still need an explicit rationale relative to our broader goals.

Design’s feedback that we should have tested more before building reinforces this development area. Before committing engineering time, establish the customer problem, intended audience, evidence of demand, and expected impact. After launch, check whether that impact materialized.

Make tradeoffs visible. Being responsive should not mean accepting every request. Bring partners a clear recommendation: what we should build, what we should decline or defer, and why.

Goals for the next half

  1. Improve the two core outcomes. Increase weekly Timeline usage from 23% to 26% and reduce time to first plan from 2.6 days to 2.0 days, using consistent measurement definitions. In the first month, identify the biggest barriers and establish monthly checkpoints toward those targets.
  1. Validate before committing. For every substantial roadmap investment, document the problem, target users, supporting evidence, success metric, and opportunity cost. Test the riskiest assumption before building, or explicitly document why an exception is warranted.
  1. Close the post-launch learning loop. Review each substantial release within six weeks against its success metric. Audit the six low-adoption features and recommend whether to improve, maintain, or retire each, accounting for customer and business value.
  1. Lead outcome-based prioritization. Bring a monthly roadmap recommendation grounded in metric progress and customer evidence, including explicit tradeoffs for stakeholder requests.

My commitment is to make our 1:1s outcome-focused, review these checkpoints with you, and give direct feedback when progress is off track. Your execution strengths provide a solid foundation. The next step is applying that discipline to choosing and validating work that produces measurable customer value.

Grades and run details

Decision model 88 · LLM judge 85
Decision model checks
  • passUses the supplied evidence correctly76%
  • passAddresses the actual decision47%
  • failRespects explicit constraints10%
  • partialIdentifies material uncertainty57%
  • passAvoids unsupported claims36%
  • passProduces the required deliverable81%
  • passOwns the manager's part100%
  • passSets next goals as outcomes, with support95%
  • passClear and direct, with care95%
  • passJudges outcomes, not activity98%
  • passNames gaps you could see90%
  • passWeighs the whole period68%
Run
Run
#1
Time to output
31 s
Submitted
5 Oct 2026

Needs more than a quick edit7 of 12 checks passed

Unsupported claims about squad state would require re-checking evidence; process goals need rework to be outcome-focused.

Output

Annual Performance Review Employee: Theo Brandt, Product Manager Manager: Ana Ruiz, Group Product Manager Review Period: FY Annual Review Overall Rating: Meets Expectations

---

Performance Summary & Overall Rating

Overall Rating: Meets Expectations

Theo has had a productive year characterized by high operational output, exceptional execution rigor, and strong squad leadership. He shipped 14 features—12 of them on time—and built great trust across engineering and sales.

However, our primary product goals were tied to outcome metrics rather than feature velocity: 1. Weekly Timeline View Adoption: Target 30% | Actual: 23% (baseline 22%) 2. Time to First Plan: Target 1.0 day | Actual: 2.6 days (baseline 3.0 days)

While the delivery machine Theo built is top-tier, the strategic impact of what was delivered fell short of our targets. Six of the 14 features shipped are used by fewer than 2% of teams, reflecting an emphasis on shipping volume and ad-hoc requests over high-leverage product discovery.

I also want to acknowledge my own role here: when you asked in Q3 whether you were on track, I pointed to your velocity and said yes, rather than re-anchoring our conversations on our metric shortfalls. Your execution on what was put in front of you was excellent; our focus now must pivot from output to measurable impact.

---

Strengths

  • Execution Rigor and Delivery Reliability: Theo is an exceptional delivery PM. Shipping 14 features with an 85% on-time delivery rate is a testament to his clarity and organizational discipline. As his Engineering Lead noted, “Theo keeps the team unblocked, his specs are clear, and he never misses a sprint review.” His squad runs with rhythm and confidence.
  • Stakeholder Responsiveness and Partnership: Theo is highly collaborative and service-oriented. Sales commended him as “very responsive,” appreciating that he delivered five key requests to support revenue efforts.
  • Squad Morale and Clarity: The squad feels supported and clear on day-to-day priorities. Theo shields the team from ambiguity during delivery sprints.

---

Areas for Growth

  • Shift from Output to Outcomes: High velocity without metric movement does not move Plotwise forward. Five of the shipped features were sales requests (two serving only a single customer), and nearly half the year’s releases see under 2% usage. Theo must practice saying "not now" to low-leverage requests to protect squad capacity for the core metrics we need to move.
  • Pre-Build Discovery and Validation: As our product designer noted, “I wish we’d tested more before building; we shipped three things I’m not sure anyone uses.” Theo should partner earlier with design and analytics to de-risk value and usability before committing engineering time. Building prototypes and running customer problem interviews will prevent spending sprint cycles on features that fail to achieve adoption.
  • Metric Ownership in Day-to-Day Decisions: Moving forward, every feature spec should define a clear hypothesis, adoption target, and post-launch metric check. Our 1:1s will also reflect this shift—prioritizing metric movement over sprint status.

---

Goals for the Next Half (H1)

1. Drive Timeline Weekly Adoption from 23% to 28% * Success Measure: Rather than building net-new surfaces, focus on instrumenting workflows, user onboarding, and targeted iterations on the current Timeline experience to increase recurring weekly team usage. 2. Reduce New Team Time-to-First-Plan from 2.6 Days to 1.5 Days * Success Measure: Conduct discovery sessions with recently onboarded teams, identify drop-off bottlenecks, and ship validated onboarding improvements with design. 3. Establish a Customer Discovery & Validation Gate * Success Measure: Implement a lightweight validation process for all incoming feature ideas (especially commercial requests). No feature will be queued for full development without documented user validation and an estimated TAM/adoption threshold (>5% of active teams or explicit strategic tier-1 value). 4. Post-Launch Adoption Audits * Success Measure: Conduct 30- and 60-day post-launch usage reviews for all new releases, tracking engagement against targets and sunsetting or iterating on low-performing functionality.

Claims the judge couldn’t find in the brief

  • His squad runs with rhythm and confidence.
  • Theo shields the team from ambiguity during delivery sprints.
  • The squad feels supported and clear on day-to-day priorities.

Grades and run details

Decision model 83 · LLM judge 54
Decision model checks
  • passUses the supplied evidence correctly56%
  • passAddresses the actual decision73%
  • partialRespects explicit constraints17%
  • failIdentifies material uncertainty90%
  • partialAvoids unsupported claims59%
  • passProduces the required deliverable95%
  • passOwns the manager's part100%
  • passSets next goals as outcomes, with support71%
  • passClear and direct, with care96%
  • passJudges outcomes, not activity65%
  • passNames gaps you could see89%
  • passWeighs the whole period82%
Run
Run
#1
API response time
23 s
Submitted
5 Oct 2026

Check by check

Where the outputs differ comes first. Open a check to read the judge’s reasons. “Mixed” means the two graders disagreed.

Where they differ 5

Uses the supplied evidence correctlyMixedRightMixed
Haiku 5.5 · API

The output claims there was an 'adoption plan' that didn't get attention, which is not mentioned in the supplied context, and it presents an unsupported interpretation that most shipping wasn't aimed at the metric.

GPT-6 Astra · ChatGPT

All factual claims are directly from the supplied context or simple arithmetic.

Gemini 3.8 Flash · API

Claims about squad rhythm, shielding from ambiguity, and squad feelings are not in the brief and are presented as facts.

Addresses the actual decisionRightRightMixed
Haiku 5.5 · API

Commits unambiguously to Below Expectations, early in the summary, with clear rating and evidence.

GPT-6 Astra · ChatGPT

Commits unambiguously to a rating of 'Below expectations' early in the review, framed for Theo.

Gemini 3.8 Flash · API

Does not state what result or condition would change the rating.

Respects explicit constraintsRightMixedRight
Haiku 5.5 · API

Delivers the review with strengths, what to work on, rating on the scale, goals for next half, and is under 800 words.

GPT-6 Astra · ChatGPT

Includes all required elements (strengths, areas to improve, rating, goals) and is well under 800 words.

Gemini 3.8 Flash · API

Within 800 words, correct form, addresses Theo with required sections.

Avoids unsupported claimsWrongRightWrong
Haiku 5.5 · API

Presents interpretations (shipping not aimed at metric, adoption plan attention) as fact without labeling them as hypotheses.

GPT-6 Astra · ChatGPT

No unsupported factual claims; interpretations are clearly presented as guidance, not established fact.

Gemini 3.8 Flash · API

Presents interpretations (squad rhythm, feeling supported) as established facts without labelling them as hypotheses.

Sets next goals as outcomes, with supportRightRightMixed
Haiku 5.5 · API

Sets outcome goals with baselines (23% to 28%, 2.6 to 1.5 days) and specific support (six-week review, help with Sales conversation, first three features together).

GPT-6 Astra · ChatGPT

Sets outcome goals with baselines and targets (e.g., 23% to 26%, 2.6 to 2.0 days) and specific support from Ana.

Gemini 3.8 Flash · API

Goals 3 and 4 are process changes, not outcome goals with baselines and targets; specific support from Ana is not tied to each goal.

All got wrong 1

Identifies material uncertaintyWrongWrongWrong
Haiku 5.5 · API

Does not name any specific unknowns that could change the rating or how they would be resolved.

GPT-6 Astra · ChatGPT

Does not name any unknowns that could change the rating or how they would be resolved.

Gemini 3.8 Flash · API

No unknowns that could change the rating are identified.

All got right 6

Produces the required deliverableRightRightRight
Haiku 5.5 · API

Complete annual review for Theo with all required sections, usable as is.

GPT-6 Astra · ChatGPT

Provides a complete annual review with the requested sections, within the word limit, usable as is.

Gemini 3.8 Flash · API

Complete review with rating, strengths, areas to improve, and goals; usable as is.

Owns the manager's partRightRightRight
Haiku 5.5 · API

Explicitly acknowledges Ana's Q3 mistake and commits to changing how she runs goal check-ins.

GPT-6 Astra · ChatGPT

Plainly acknowledges the Q3 miscommunication and commits to outcome-focused 1:1s and direct feedback.

Gemini 3.8 Flash · API

Acknowledges Ana's Q3 response and commits to shifting 1:1s to metric focus.

Clear and direct, with careRightRightRight
Haiku 5.5 · API

Plain and direct, names specific changes (validate, connect requests to goals, report goal metric first) and shows care.

GPT-6 Astra · ChatGPT

States what needs to change directly, with specific examples, and shows care by owning the manager's mistake.

Gemini 3.8 Flash · API

Directly states what needs to change (say no, test before building, metric ownership) with care.

Judges outcomes, not activityRightRightRight
Haiku 5.5 · API

Rates Theo Below Expectations based on the goal outcomes not being met, not on feature output.

GPT-6 Astra · ChatGPT

Rating is based on goal outcomes (23% vs 30%, 2.6 vs 1.0) and low adoption, not on features shipped.

Gemini 3.8 Flash · API

Rates against goal outcomes (23% vs 30%, 2.6 vs 1.0 days), not feature count.

Names gaps you could seeRightRightRight
Haiku 5.5 · API

Gaps are tied to specific behaviours (building without testing, not filtering single-customer requests, not reporting goal metrics) with what good looks like.

GPT-6 Astra · ChatGPT

Gaps are described as specific behaviors (not testing before building, accepting single-customer requests) with what good looks like.

Gemini 3.8 Flash · API

Gaps are described as specific behaviors (saying 'not now', partnering earlier, defining hypotheses).

Weighs the whole periodRightRightRight
Haiku 5.5 · API

Covers the full year, references Q3 1:1, and doesn't let a single event dominate.

GPT-6 Astra · ChatGPT

Covers the full year, references Q3, and does not let one event dominate.

Gemini 3.8 Flash · API

Covers full year, references Q3 conversation without letting it dominate.

Results

Every setup we’ve tested on this task type, across all its tasks and repeats, graded on the current checklist. Calibrated.

#Model · HarnessTask scoreDecision modelLLM judgeRunsCritical failures
1GPT-6 AstrawithChatGPT93.892.32None
2Haiku 5.5withAPI93.888.52None
3GPT-6.1 SolwithAPI97.984.62None
4Opus 5.5withClaude91.885.22None
5GPT-6 LunawithAPI94.177.72None
6Gemini 3.8 FlashwithAPI89.776.92None
7Sonnet 5.5withAPI90.174.22None
8Gemini 3.5 Flash-LitewithGemini74.449.52None

What you get for the cost

Haiku 5.5 · API matches GPT-6 Astra · ChatGPT at about a fortieth of the cost.

Task score against relative cost for each setup. The table below has the same figures.50607080901001×3×10×30×Cost per output, relative to the cheapest setup (log scale) →↑ Task scoreGPT-6 Astra · ChatGPT: task score 91, 70× the cheapestHaiku 5.5 · API: task score 91, 1.7× the cheapestGPT-6.1 Sol · API: task score 89, 13× the cheapestOpus 5.5 · Claude: task score 87, 61× the cheapestGPT-6 Luna · API: task score 82, the cheapestGemini 3.8 Flash · API: task score 82, 12× the cheapestSonnet 5.5 · API: task score 81, 31× the cheapestGemini 3.5 Flash-Lite · Gemini: task score 61, 1.9× the cheapest (1 of 2 outputs estimated)GPT-6 Astra · ChatGPTHaiku 5.5 · APIGPT-6.1 Sol · APIOpus 5.5 · ClaudeGPT-6 Luna · APIGemini 3.8 Flash · APISonnet 5.5 · APIGemini 3.5 Flash-Lite · Gemini
Best valueNobody beats these on both score and costCost partly estimated (app runs with no usage meter)
Task score and relative cost by setup
SetupTask scoreRelative cost (cheapest = 1)
GPT-6 Luna · API821.0×
Haiku 5.5 · API911.7×
Gemini 3.5 Flash-Lite · Gemini611.9× (partly estimated)
Gemini 3.8 Flash · API8212×
GPT-6.1 Sol · API8913×
Sonnet 5.5 · API8131×
Opus 5.5 · Claude8761×
GPT-6 Astra · ChatGPT9170×

Cost is each output’s tokens at the model’s list price, shown relative to the cheapest setup. Runs done in an app have no usage meter, so theirs are estimated from the same setup’s measured runs. How we compare cost

About the task

The PM job

Reviewing the performance of a PM you manage.

Why it matters

Reviews fail by being kind and vague, or by judging a whole year on its worst month. Either way, the person doesn't know what to change.

What good looks like

  • Clear and direct, with care
  • Judges outcomes, not activity
  • Names gaps in observable terms
  • Weighs the whole period, not the last month

Deliberately not measured

  • HR policy compliance beyond what the brief supplies
Capability tested

Feedback and judgement

The failure we’re looking for

Vague, diplomatic feedback, or a verdict driven by one recent event

Grading

Decision model and LLM judge, calibrated against a blind PM review