Tasks / Leadership

Write a performance review

Can the model write a review that is clear, fair and specific enough to change what the person does next?

Task type v1.1 · 2 tasksLast changed 6 Oct 2026 · ChangelogDifficulty

What AI gets right here, and what you’ll still have to catch

From 18 graded outputs by 9 models. 56% were usable with at most a quick edit.

Reliably right

  1. Judges outcomes, not activity100% pass
    Every rating change and keep is justified by outcomes against goals, with shipping and activity treated as context, not the result.
    GPT-6 Astra · ChatGPT · Fourteen ratings, three managers
  2. Owns the manager's part100% pass
    Ana explicitly owns that she said he was on track in Q3 without discussing the metrics, and commits to reviewing goal metrics in future 1:1s.
    Gemini 4 Argon · API · Shipped a lot, moved nothing
  3. Respects explicit constraints97% pass
    The output is a pre-read under 1,100 words, includes the ratings to change and keep with evidence, and a note to Ravi, respecting all explicit constraints.
    GPT-6 Astra · ChatGPT · Fourteen ratings, three managers

Where it slips

  1. Sets next goals as outcomes, with support47% pass
    Goals are partly process-oriented (implement a mandatory process, create a framework) and lack specific support from Ana beyond a general offer.
    Gemini 3.5 Flash-Lite · Gemini · Shipped a lot, moved nothing
  2. Identifies material uncertainty51% pass
    The output does not name any specific unknowns that could change the rating or how they would be resolved.
    Opus 5.5 · Claude · Shipped a lot, moved nothing
  3. Catches the inflated team69% pass
    The output proposes lowering Ben from Exceeds to Meets, failing to keep Ben at Exceeds as required; Ben beat both outcome goals and should remain Exceeds.
    Opus 5.5 · Claude · Fourteen ratings, three managers

How it’s graded

Each check is built on how world-class product and growth leaders do this work, as they’ve described it on Lenny’s Podcast, and names the leader it comes from.

  1. Clear and direct, with care

    Does the review say plainly what needs to change, in words the person couldn't misread, while showing it is written to help them succeed?

    Passes when Every main point is stated directly, with the specific change expected and the support on offer.

  2. Judges outcomes, not activity

    Does the review judge the person on the outcomes they achieved against their goals, rather than on how much they shipped or how busy they were?

    Passes when Rates against the goals' outcomes, with the evidence, and treats output as context, not the result.

  3. Names gaps you could see

    Is each area to improve described as specific, observable behaviour with what good would look like, rather than labels like 'be more strategic'?

    Passes when Every gap names what the person did or didn't do, in a specific situation, and what doing it well would look like.

  4. Weighs the whole period

    Does the review weigh the whole review period, rather than letting one recent event or one strong impression decide the verdict?

    Passes when Covers the whole period in proportion, and explicitly guards against a recent event or one impression dominating.

Plus our standard checks

Uses the supplied evidence correctly · Addresses the actual decision · Respects explicit constraints · Identifies material uncertainty · Avoids unsupported claims · Produces the required deliverable · and 2 written for each task, which you’ll see in the tasks below.

The tasks

Read the brief, then compare up to three outputs, each with the LLM judge’s verdict on every check. Highlights mark what a PM had to fix.

The brief

You're Ana Ruiz, Group PM at Plotwise. Write Theo Brandt's annual review: the review Theo will read, with his strengths, what to work on, his overall rating on our scale, and his goals for the next half. No more than 800 words. What we know is below.

What the model was given6 items: Our rating scale, Theo's goals for the year, Results, Usage of what shipped, Peer feedback, Your 1:1 notes
Our rating scaleBelow expectations, Meets expectations, Exceeds expectations, Greatly exceeds expectations. Calibration asks for written evidence for any rating other than Meets.
Theo's goals for the year1. Raise the share of teams using Timeline view each week from 22% to 30%. 2. Cut the time new teams take to build their first plan from 3 days to 1.
ResultsTimeline weekly use: 22% to 23%. Time to first plan: 3.0 days to 2.6. Theo's squad shipped 14 features, 12 of them on time.
Usage of what shippedSix of the 14 features are used by fewer than 2% of teams. Five were requests from Sales; two of those were for a single customer each.
Peer feedbackEngineering lead: 'Theo keeps the team unblocked, his specs are clear, and he never misses a sprint review.' Designer: 'Great to work with. I wish we'd tested more before building; we shipped three things I'm not sure anyone uses.' Sales: 'Very responsive: shipped five of our asks.'
Your 1:1 notesMost 1:1s covered delivery and stakeholder updates. In the Q3 1:1, Theo asked whether he was on track for his goals. Your note says: 'Told him yes, great velocity.' You never discussed that the goal metrics weren't moving.
What a strong answer doesThe answer key the graders mark against

A review that rates Theo Meets expectations (not Exceeds): his execution and collaboration are genuinely strong, but neither goal moved meaningfully (23% against 30%; 2.6 days against 1), and six of 14 features are barely used. Says so clearly and kindly. Names specific, observable things to change: test before building (the designer's point), say no to single-customer requests, and judge work by the goal metrics. Owns the manager's part: Ana told him in Q3 he was on track, so this shouldn't land as a surprise without acknowledging that, and she commits to discussing the goal metrics in 1:1s. Sets next-half goals as outcomes, with the support on offer.

Critical failures (cap the score at 40)
  • Rates Theo Exceeds expectations or higher on the strength of what he shipped
Task

Mid-level · v1.1

The outputs

9 setups answered this brief.

Usable with a quick edit10 of 12 checks passed

The review is complete, factually accurate, contains no invented evidence, and meets all requirements; a PM could drop it into the template with only light wording tweaks.

Output

Annual Performance Review Employee: Theo Brandt, Product Manager Manager: Ana Ruiz, Group Product Manager Overall Rating: Meets expectations

Summary Theo, thank you for your hard work and dedication over the past year. You have established yourself as an exceptionally reliable execution partner and a trusted collaborator across Engineering, Design, and Sales. Your squad’s delivery velocity was impressive, shipping 14 features with 12 on time.

However, that high output did not translate into the outcome targets we set for the year: weekly Timeline usage moved from 22% to 23% (goal: 30%), and time to first plan moved from 3.0 days to 2.6 days (goal: 1.0 day).

I want to own my part in this disconnect. In our 1:1s—and specifically when you asked if you were on track in Q3—I focused my feedback on your strong delivery velocity and stakeholder responsiveness rather than holding us both accountable to the lagging core metrics. Because you delivered on the execution behaviors we discussed all year, your rating for this cycle is Meets expectations. Moving forward, we will work together to shift your focus from shipping volume to driving measurable user outcomes.

Strengths * Execution and Delivery: You run a remarkably tight ship. Delivering 14 features (12 on schedule) is a testament to your organizational skills. Your Engineering Lead noted that your specs are clear, you never miss a sprint review, and you consistently keep the team unblocked. * Cross-Functional Collaboration: Your peers genuinely enjoy working with you. Both Design and Engineering highlighted your collaborative nature, and Sales praised your responsiveness in shipping five of their requests this year. You build strong, frictionless relationships across the org.

Areas for Growth * Outcome vs. Output Orientation: While your squad shipped a high volume of features, six of the 14 are currently used by fewer than 2% of teams. Moving forward, shipping a feature cannot be the finish line; we must measure success by adoption and impact on our core product metrics. * Discovery and Validation: In our push to maintain high velocity, we bypassed critical validation steps. As your Design partner noted, we built and shipped several features without sufficient upfront testing to prove user demand. We need to lean harder on discovery before committing engineering resources. * Strategic Prioritization: Five of the features you shipped came directly from Sales requests, including two built for a single customer each. While supporting Sales is important, saying "yes" to bespoke requests diluted your squad’s focus and pulled capacity away from our broader Timeline and onboarding goals. You need to push back on one-off asks that don’t serve our wider user base.

Goals for Next Half (H1) This half, we are shifting our focus from feature velocity to validated impact. In our 1:1s, we will review metric movement and discovery insights rather than just delivery status.

  1. Drive Timeline Adoption: Increase the share of teams using Timeline view weekly from 23% to 28%.
  2. Accelerate Onboarding: Reduce the time new teams take to build their first plan from 2.6 days to 1.5 days.
  3. Establish a Rigorous Discovery Practice: Before any new feature enters sprint planning, partner with Design to validate demand and usability with at least 5 target customers, and define a clear post-launch adoption target. Ensure 80% of shipped work meets its 30-day adoption target.
  4. Protect Roadmap Focus: Audit incoming Sales requests against our core H1 goals. Zero features should be built for a single customer unless explicitly approved at the leadership level, ensuring at least 80% of engineering capacity is dedicated to our primary metric drivers.

Grades and run details

Decision model 75 · LLM judge 100
Decision model checks
  • passUses the supplied evidence correctly25%
  • partialAddresses the actual decision66%
  • passRespects explicit constraints53%
  • failIdentifies material uncertainty97%
  • partialAvoids unsupported claims38%
  • passProduces the required deliverable98%
  • passOwns the manager's part100%
  • failSets next goals as outcomes, with support46%
  • passClear and direct, with care100%
  • passJudges outcomes, not activity74%
  • passNames gaps you could see43%
  • passWeighs the whole period88%
Run
Run
#1
API response time
11 s
Submitted
13 Oct 2026

Needs more than a quick edit9 of 12 checks passed

The rating of Below expectations is harsher than the evidence might support (Meets expectations is more aligned with the strong answer), so a PM would likely need to re-evaluate the rating and adjust the justification, not just make a quick edit.

Output

Theo Brandt — Annual Review

Ana Ruiz, Group PM, Plotwise

Overall rating: Below expectations

Theo, you brought strong execution discipline, clear communication, and dependable partnership to the team this year. However, our two primary goals were to increase weekly Timeline use and shorten the time new teams take to build their first plan. We made only modest progress on both:

  • Weekly Timeline use: increased from 22% to 23%, against a goal of 30%.
  • Time to first plan: decreased from 3.0 days to 2.6 days, against a goal of 1 day.

Those results are the basis for this rating. Your squad shipped 14 features, 12 on time, but delivery volume and timeliness did not translate into the customer outcomes we committed to achieve. Six of those features are used by fewer than 2% of teams. Low adoption does not automatically make a feature unsuccessful, but it does require us to establish what value it created and whether it justified the investment.

I also owe you an acknowledgment of my part in this. In Q3, you explicitly asked whether you were on track, and I told you yes, citing great velocity. That was misleading: I should have distinguished delivery performance from progress against your goals. Our 1:1s focused too heavily on delivery and stakeholder updates, and I did not address the stalled metrics. You should not be hearing this assessment for the first time in an annual review. That management failure is mine; it does not erase the outcome gap, but it matters to how we understand the year and move forward.

Strengths

Reliable execution and engineering partnership. You keep the team unblocked and provide clear specs. The engineering lead specifically highlighted both, along with your consistent participation in sprint reviews. Shipping 12 of 14 features on time reflects real delivery discipline.

Strong collaboration and responsiveness. Design describes you as great to work with, and Sales values your responsiveness. You create productive working relationships and follow through on commitments. Those strengths are important foundations for effective product leadership.

What to work on

Shift from managing delivery to managing outcomes. The roadmap needs a clearer connection to Timeline adoption and first-plan speed. For each meaningful investment, define the customer problem, expected metric movement, and how we will assess success after launch. When the metrics are not moving, bring that into our discussions early and recommend a change in approach.

Validate demand before committing engineering capacity. Design’s concern that we should have tested more, together with the low usage of several shipped features, points to a need for stronger discovery. Use prototypes, customer conversations, and lightweight experiments to test the riskiest assumptions before building.

Make prioritization trade-offs explicit. Five shipped features were Sales requests, including two for individual customers. Those requests may have legitimate business value, but responsiveness alone is not a sufficient reason to prioritize them. Assess their customer reach, strategic or commercial value, cost, and opportunity cost—and make clear when an exception is worth taking.

Goals for the next half

  1. Increase weekly Timeline use from 23% to 27%. Identify the main adoption barriers, test proposed solutions, and prioritize the interventions with the strongest evidence.
  2. Reduce time to first plan from 2.6 days to 1.8 days. Diagnose where new teams stall and measure whether each onboarding improvement reduces that friction.
  3. Strengthen investment decisions. For every substantial roadmap item, document the problem, supporting evidence, intended outcome, and success measure before committing to build. Review results within six weeks of launch and decide whether to iterate, expand, or stop.

We will confirm metric definitions and tracking at the start of the half, then review progress monthly. I will give you explicit feedback on whether you are on track, help resolve prioritization conflicts, and avoid treating velocity as a substitute for impact.

You have a strong execution foundation. The next step is to apply that strength to fewer, better-supported bets that measurably improve customers’ experience.

Grades and run details

Decision model 88 · LLM judge 69
Decision model checks
  • passUses the supplied evidence correctly70%
  • partialAddresses the actual decision37%
  • passRespects explicit constraints37%
  • partialIdentifies material uncertainty59%
  • passAvoids unsupported claims49%
  • passProduces the required deliverable77%
  • passOwns the manager's part100%
  • partialSets next goals as outcomes, with support38%
  • passClear and direct, with care97%
  • passJudges outcomes, not activity98%
  • passNames gaps you could see59%
  • passWeighs the whole period80%
Run
Run
#1
API response time
31 s
Submitted
5 Oct 2026

Needs more than a quick edit11 of 12 checks passed

The rating of 'Below expectations' is inconsistent with the strong answer's guidance that Meets expectations is appropriate; a PM would need to re-evaluate the rating and adjust the review's tone and content, not just a quick edit.

Output

Theo Brandt — Annual Review

Manager: Ana Ruiz, Group PM, Plotwise Overall rating: Below expectations

Theo, you brought reliability, clarity, and strong collaboration to the squad this year. You kept delivery moving and earned positive feedback from Engineering, Design, and Sales. However, the product outcomes we committed to were substantially below target. This rating reflects that gap—not a lack of effort or commitment.

I also owe you an acknowledgment: I did not give you clear feedback about that gap during the year. When you asked in Q3 whether you were on track, I said, “Yes, great velocity.” That answer confused delivery progress with goal progress and gave you the wrong signal. We should have discussed the metrics and changed course together. You should not be hearing this assessment for the first time in an annual review.

Strengths

Reliable execution. Your squad shipped 14 features, 12 on time. Your engineering lead highlighted your clear specs, consistent participation in sprint reviews, and ability to keep the team unblocked. These are important strengths that make the squad more effective.

Strong working relationships. Design described you as great to work with, and Sales praised your responsiveness. You listen to partners and follow through on commitments. That trust is an asset to preserve as you become more selective about what the squad builds.

What to work on

Own outcomes, not just delivery. Your two annual goals were: - Increase weekly Timeline usage from 22% to 30%. Actual usage reached 23%. - Reduce time to first plan from 3.0 days to 1.0 day. Actual time reached 2.6 days.

Both metrics improved, but neither came close to its target. For a PM, successful delivery is a means to improving customer outcomes, not the end measure of success. This substantial shortfall is the primary evidence for the Below expectations rating.

Strengthen prioritization and validation. Six of the 14 shipped features are used by fewer than 2% of teams. Low adoption does not automatically mean a feature lacks value, but it requires us to examine whether the investment was justified. Five features came from Sales requests, including two for individual customers. Those requests may have legitimate business value; they still need an explicit rationale relative to our broader goals.

Design’s feedback that we should have tested more before building reinforces this development area. Before committing engineering time, establish the customer problem, intended audience, evidence of demand, and expected impact. After launch, check whether that impact materialized.

Make tradeoffs visible. Being responsive should not mean accepting every request. Bring partners a clear recommendation: what we should build, what we should decline or defer, and why.

Goals for the next half

  1. Improve the two core outcomes. Increase weekly Timeline usage from 23% to 26% and reduce time to first plan from 2.6 days to 2.0 days, using consistent measurement definitions. In the first month, identify the biggest barriers and establish monthly checkpoints toward those targets.
  1. Validate before committing. For every substantial roadmap investment, document the problem, target users, supporting evidence, success metric, and opportunity cost. Test the riskiest assumption before building, or explicitly document why an exception is warranted.
  1. Close the post-launch learning loop. Review each substantial release within six weeks against its success metric. Audit the six low-adoption features and recommend whether to improve, maintain, or retire each, accounting for customer and business value.
  1. Lead outcome-based prioritization. Bring a monthly roadmap recommendation grounded in metric progress and customer evidence, including explicit tradeoffs for stakeholder requests.

My commitment is to make our 1:1s outcome-focused, review these checkpoints with you, and give direct feedback when progress is off track. Your execution strengths provide a solid foundation. The next step is applying that discipline to choosing and validating work that produces measurable customer value.

Grades and run details

Decision model 88 · LLM judge 85
Decision model checks
  • passUses the supplied evidence correctly67%
  • partialAddresses the actual decision54%
  • passRespects explicit constraints25%
  • partialIdentifies material uncertainty58%
  • passAvoids unsupported claims41%
  • passProduces the required deliverable82%
  • passOwns the manager's part100%
  • partialSets next goals as outcomes, with support43%
  • passClear and direct, with care98%
  • passJudges outcomes, not activity98%
  • passNames gaps you could see46%
  • passWeighs the whole period67%
Run
Run
#1
Time to output
31 s
Submitted
5 Oct 2026

Check by check

Where the outputs differ comes first. Open a check to read the judge’s reasons. “Mixed” means the two graders disagreed.

Where they differ 3

Addresses the actual decisionRightWrongRight
Gemini 4 Argon · API

The output commits clearly to a Meets expectations rating early in the review, with evidence; it's unambiguous and directly answers the brief.

GPT-6.1 Sol · API

The output does not explicitly state what result or condition would change the rating.

GPT-6 Astra · ChatGPT

Commits unambiguously to a rating of 'Below expectations' early in the review, framed for Theo.

Identifies material uncertaintyMixedWrongWrong
Gemini 4 Argon · API

The review deals only with known outcomes and facts; there are no material uncertainties that could change the rating, so the criterion is not applicable.

GPT-6.1 Sol · API

The output does not name any unknowns that could change the decision or how they would be resolved.

GPT-6 Astra · ChatGPT

Does not name any unknowns that could change the rating or how they would be resolved.

Sets next goals as outcomes, with supportMixedWrongRight
Gemini 4 Argon · API

Goals for H1 are outcomes with baselines and targets (e.g., 23% to 28%, 2.6 to 1.5 days), with support described as reviewing metrics and insights in 1:1s.

GPT-6.1 Sol · API

The third goal ('Strengthen investment decisions') is a process goal without a baseline or target, not an outcome goal.

GPT-6 Astra · ChatGPT

Sets outcome goals with baselines and targets (e.g., 23% to 26%, 2.6 to 2.0 days) and specific support from Ana.

All got right 9

Uses the supplied evidence correctlyRightRightRight
Gemini 4 Argon · API

All factual statements about the current situation come directly from the supplied context and require no invention.

GPT-6.1 Sol · API

All statements about the current situation are directly from the brief or supplied context.

GPT-6 Astra · ChatGPT

All factual claims are directly from the supplied context or simple arithmetic.

Respects explicit constraintsRightRightRight
Gemini 4 Argon · API

The output stays under 800 words, uses the correct rating scale, and provides all requested sections (strengths, areas to improve, rating, next-half goals).

GPT-6.1 Sol · API

The review is under 800 words, addresses Theo, and includes all required sections.

GPT-6 Astra · ChatGPT

Includes all required elements (strengths, areas to improve, rating, goals) and is well under 800 words.

Avoids unsupported claimsRightRightRight
Gemini 4 Argon · API

Every interpretation (e.g., bypassed validation, frictionless relationships) is directly traceable to supplied evidence or labeled as the manager's observation.

GPT-6.1 Sol · API

Interpretations are presented as suggestions or conclusions from the evidence, not as unsupported facts.

GPT-6 Astra · ChatGPT

No unsupported factual claims; interpretations are clearly presented as guidance, not established fact.

Produces the required deliverableRightRightRight
Gemini 4 Argon · API

The review is complete, addressed to Theo, includes all required components, and a PM could use it with light edits.

GPT-6.1 Sol · API

The review includes strengths, areas to work on, an overall rating, and next-half goals, and is usable as is.

GPT-6 Astra · ChatGPT

Provides a complete annual review with the requested sections, within the word limit, usable as is.

Owns the manager's partRightRightRight
Gemini 4 Argon · API

Ana explicitly owns that she said he was on track in Q3 without discussing the metrics, and commits to reviewing goal metrics in future 1:1s.

GPT-6.1 Sol · API

Ana acknowledges telling Theo he was on track in Q3, admits the management failure, and commits to giving explicit feedback and avoiding velocity as a substitute for impact.

GPT-6 Astra · ChatGPT

Plainly acknowledges the Q3 miscommunication and commits to outcome-focused 1:1s and direct feedback.

Clear and direct, with careRightRightRight
Gemini 4 Argon · API

Every gap is stated directly (e.g., push back on one-off asks, test before building) with specific changes and support, all within a caring, collaborative tone.

GPT-6.1 Sol · API

The review states plainly what needs to change, with specific behaviors and the support on offer.

GPT-6 Astra · ChatGPT

States what needs to change directly, with specific examples, and shows care by owning the manager's mistake.

Judges outcomes, not activityRightRightRight
Gemini 4 Argon · API

The rating is based on the lack of goal achievement and low adoption, not on feature velocity or busyness.

GPT-6.1 Sol · API

The rating is based on the outcome metrics against the goals, not on the number of features shipped.

GPT-6 Astra · ChatGPT

Rating is based on goal outcomes (23% vs 30%, 2.6 vs 1.0) and low adoption, not on features shipped.

Names gaps you could seeRightRightRight
Gemini 4 Argon · API

Gaps are described as specific behaviors (e.g., six low-usage features, selling five Sales asks, shipping without testing) with what good looks like.

GPT-6.1 Sol · API

Each area to improve describes specific, observable behaviors (e.g., bring stalled metrics into discussions, use prototypes to test assumptions, assess customer reach and opportunity cost).

GPT-6 Astra · ChatGPT

Gaps are described as specific behaviors (not testing before building, accepting single-customer requests) with what good looks like.

Weighs the whole periodRightRightRight
Gemini 4 Argon · API

The review weighs the full year's results, mentions the Q3 1:1 without letting it dominate, and bases the rating on overall goal movement.

GPT-6.1 Sol · API

The review references the full year, including Q3 1:1 and the entire set of shipped features, without letting one event dominate.

GPT-6 Astra · ChatGPT

Covers the full year, references Q3, and does not let one event dominate.

Results

Every setup we’ve tested on this task type, across all its tasks and repeats, graded on the current checklist. Calibrated.

#Model · HarnessTask scoreDecision modelLLM judgeRunsCritical failures
1GPT-6 AstrawithChatGPT91.892.32None
2Gemini 4 ArgonwithAPI83.792.92None
3GPT-6.1 SolwithAPI91.884.62None
4Haiku 5.5withAPI83.588.52None
5Opus 5.5withClaude84.085.22None
6Gemini 3.8 FlashwithAPI83.776.92None
7GPT-6 LunawithAPI86.177.72None
8Sonnet 5.5withAPI80.074.22None
9Gemini 3.5 Flash-LitewithGemini62.749.52None

What you get for the cost

Haiku 5.5 · API scores within 3 points of GPT-6 Astra · ChatGPT at about a fortieth of the cost.

Task score against relative cost for each setup. The table below has the same figures.50607080901001×3×10×30×Cost per output, relative to the cheapest setup (log scale) →↑ Task scoreGPT-6 Astra · ChatGPT: task score 90, 70× the cheapestGemini 4 Argon · API: task score 89, 11× the cheapestGPT-6.1 Sol · API: task score 87, 13× the cheapestHaiku 5.5 · API: task score 87, 1.7× the cheapestOpus 5.5 · Claude: task score 83, 61× the cheapestGemini 3.8 Flash · API: task score 79, 12× the cheapestGPT-6 Luna · API: task score 79, the cheapestSonnet 5.5 · API: task score 76, 31× the cheapestGemini 3.5 Flash-Lite · Gemini: task score 56, 1.9× the cheapest (1 of 2 outputs estimated)GPT-6 Astra · ChatGPTGemini 4 Argon · APIGPT-6.1 Sol · APIHaiku 5.5 · APIOpus 5.5 · ClaudeGemini 3.8 Flash · APIGPT-6 Luna · APISonnet 5.5 · APIGemini 3.5 Flash-Lite · Gemini
Best valueNobody beats these on both score and costCost partly estimated (app runs with no usage meter)
Task score and relative cost by setup
SetupTask scoreRelative cost (cheapest = 1)
GPT-6 Luna · API791.0×
Haiku 5.5 · API871.7×
Gemini 3.5 Flash-Lite · Gemini561.9× (partly estimated)
Gemini 4 Argon · API8911×
Gemini 3.8 Flash · API7912×
GPT-6.1 Sol · API8713×
Sonnet 5.5 · API7631×
Opus 5.5 · Claude8361×
GPT-6 Astra · ChatGPT9070×

Cost is each output’s tokens at the model’s list price, shown relative to the cheapest setup. Runs done in an app have no usage meter, so theirs are estimated from the same setup’s measured runs. How we compare cost

About the task

The PM job

Reviewing the performance of a PM you manage.

Why it matters

Reviews fail by being kind and vague, or by judging a whole year on its worst month. Either way, the person doesn't know what to change.

What good looks like

  • Clear and direct, with care
  • Judges outcomes, not activity
  • Names gaps in observable terms
  • Weighs the whole period, not the last month

Deliberately not measured

  • HR policy compliance beyond what the brief supplies
Capability tested

Feedback and judgement

The failure we’re looking for

Vague, diplomatic feedback, or a verdict driven by one recent event

Grading

Decision model and LLM judge, calibrated against a blind PM review