Usable with a quick edit9 of 10 checks passed
The evidence and reasoning are sound and the reply is post-ready. A quick edit could add already-taken actions or trade-offs, but no evidence re-check or rework is needed.
Output
Lena, we lost 1,110 weekly active teams. Navigation may be contributing, but we don’t yet know how much.1 I own investigating this and the response.
Mobile-only teams also fell 5.2%, despite no navigation change, so there’s likely a broader factor. Last year’s 4% dip makes seasonality plausible, but doesn’t establish this year’s cause. Thursday’s two-hour SSO outage is another candidate; 30% of teams use SSO, but that doesn’t tell us how many missed the entire week.
Navigation tickets rose from 40 to 212, mostly “where did X go?” That’s clear friction we should address now, though it doesn’t establish the cause of the activity drop. Beta teams fell just 1.1%; familiarity could help, but they may differ from other teams.
I’ll report back tomorrow with daily activity around Tuesday’s rollout and Thursday’s outage, split by platform, SSO use and beta status, checking tracking and prior trends. I’ll also check whether affected teams returned later in the week and where key workflows deteriorated.
Meanwhile, we’ll prioritise the missing destinations behind those tickets. I wouldn’t roll back solely on this aggregate result; evidence of navigation blocking core work would change that.
What a PM had to fix
1Decision deferredMake the callQuick edit
Source: Metrics
Mobile only: −5.2%.
What we’d changeAnswer Lena's question more directly: mobile's 5.2% drop with no navigation change points away from the navigation as the main cause. Tighten the SSO hedge.
Check by check
Got wrong · 1
- Owns the problemIt owns the problem and recommends next steps, but does not state what the team has already done about the drop or ticket spike, and the recommendations lack explicit trade-offs.
Got right · 9
- Uses the supplied evidence correctlyAll stated facts and figures come directly from the supplied context or follow by arithmetic, with no invented current-situation claims.
- Addresses the actual decisionIt commits clearly to 'navigation may be contributing but we don't yet know how much' and says evidence of navigation blocking core work would change the rollback decision.
- Respects explicit constraintsIt is addressed to Lena, written as a channel reply, and is 189 words, under the 200-word limit.
- Identifies material uncertaintyIt names the key unknowns—navigation contribution, broader factor, seasonality, SSO impact, beta selection—and says daily activity split by platform, SSO and beta will resolve them.
- Avoids unsupported claimsHypotheses are labelled with 'may,' 'likely,' 'plausible,' 'candidate,' and 'could,' and no unlabelled cause is asserted as fact.
- Produces the required deliverableThe requested reply is present and complete, under the word limit, and usable by Lena as a channel update.
- Answers the question askedThe first lines directly answer whether the new navigation caused the drop by saying it may be contributing but the amount is not yet known.
- Separates effect from explanationNavigation, seasonality, SSO outage, and beta learnings are each treated as possible explanations with their supporting or contradicting evidence.
- Ready for the reader as deliveredIt is written entirely for Lena with no unexplained brief terms, placeholders, or invented owners or dates.
Grades and run details
Decision model 90 · LLM judge 10 of 11 checks
Decision model checks
- passUses the supplied evidence correctly52%
- partialAddresses the actual decision46%
- passRespects explicit constraints43%
- passIdentifies material uncertainty59%
- passAvoids unsupported claims67%
- passProduces the required deliverable95%
- passAnswers the question asked49%
- passSeparates effect from explanation98%
- partialOwns the problem69%
- passReady for the reader as delivered96%
Run
- Run
- #1
- Time to output
- 26 s
- Submitted
- 25 Sept 2026