Changelog

Task pages plot each setup's score against what an output costs at list price, as a multiple of the cheapest setup, and name the best value: the cheapest setup within 5 points of the leader at half its cost or less. Cost never changes a score.

Call product-market fit, Price and package a product, Diagnose a retention problem, Write a GTM plan and Competitor analysis join the suite. Every task hands the model files to work through (survey exports, usage and cohort data, win/loss logs, price tests) with the traps in the data. Their checks credit Todd Jackson, Sean Ellis, Rahul Vohra, Madhavan Ramanujam, Bangaly Kaba, Naomi Gleit, April Dunford, Emily Kramer, Jeanne DeWitt Grosser, Arielle Jackson and Hamilton Helmer.

Every task type is now graded on how world-class product and growth leaders work, as they've described it on Lenny's Podcast, with the leader credited by name beside each check. Fifteen new checks join, and task types are grouped into five domains: Product Craft, Growth, Metrics & Experimentation, Strategy and Leadership. A task type now also needs three reviewed verdicts on every one of its checks before it counts as calibrated, so most are provisional until the new checks are reviewed.

The Combined score is now the share of checks an output passes, averaged across the decision model and the LLM judge, and capped at 40 for a critical failure. Our PM no longer scores outputs; they check the checkers. A task type counts as calibrated once the graders agree with them on at least 80% of checks.

Every written output is also graded by an LLM judge from a model family we don't test, so it isn't marking its own homework. It checks each claim against the brief, then answers every check yes or no.