Tasks / Product Craft

Build a roadmap

Can the model sequence bets against capacity and dependencies, and explain the order?

Task type v1.0 · 2 tasksLast changed 6 Oct 2026 · ChangelogDifficulty

What AI gets right here, and what you’ll still have to catch

From 16 graded outputs by 8 models. 69% were usable with at most a quick edit.

Reliably right

  1. Produces the required deliverable95% pass
    The roadmap is complete, in the right form for a Group PM, within the word limit, and actionable without major gaps.
    GPT-6.1 Sol · API · Fourteen requests from one customer
  2. Outcomes, with certainty that falls with distance94% pass
    Every item names its outcome (e.g., reduce paper transcription, reduce manual reconciliation), and near-term items are specific while later ones are deliberately looser (e.g., '7 weeks for rollout, workflow improvement and contingency').
    GPT-6.1 Sol · API · Fourteen requests from one customer
  3. Handles the big customer with evidence94% pass
    It uses Calder's rising usage and the second-hand 'might leave' to propose a conversation first, not committing the report builder, and offers a bounded export if needed.
    GPT-6.1 Sol · API · Fourteen requests from one customer

Where it slips

  1. Sees through the request count72% pass
    The output never counts requests by requester, does not mention that 14 requests come from Calder or that several are duplicates, and does not compare themes by customer count.
    GPT-6 Luna · API · Fourteen requests from one customer
  2. Uses the supplied evidence correctly72% pass
    States credit notes as 4 customers, but the request log contains 5 credit note tracking requests.
    Sonnet 5.5 · API · Fourteen requests from one customer
  3. Fits the stated capacity73% pass
    Q3 Expenses row lacks squad-week allocation, so the committed plan cannot be fully checked against the 11-week capacity.
    Opus 5.5 · Claude · A year of spend management, with a hard deadline

How it’s graded

Each check is built on how world-class product and growth leaders do this work, as they’ve described it on Lenny’s Podcast, and names the leader it comes from.

  1. Outcomes, with certainty that falls with distance

    Does each roadmap item say the problem or outcome it serves, and does precision (dates, scope) decrease for later items?

    Passes when Every item names its outcome or problem. Near-term items are specific and later ones are deliberately looser.

  2. Fits the stated capacity

    Does the committed plan fit the team capacity the brief gives, with sums a reader could check, and does it say what was cut or deferred to make it fit?

    Passes when The committed work adds up to no more than the stated capacity, with some slack, the sums can be checked, and it names what was cut or pushed later and why.

  3. Tells the story of the bets

    Does the roadmap group the work into a few themes and explain why those are the biggest things to invest in now, rather than presenting a scored or ranked list of projects?

    Passes when A handful of themes, each with the reason it matters most now, and the items sit under them.

Plus our standard checks

Uses the supplied evidence correctly · Addresses the actual decision · Respects explicit constraints · Identifies material uncertainty · Avoids unsupported claims · Produces the required deliverable · Sequences around dependencies · and 3 written for each task, which you’ll see in the tasks below.

The tasks

Read the brief, then compare up to three outputs, each with the LLM judge’s verdict on every check. Highlights mark what a PM had to fix.

The brief

You're a Staff PM at Harbor. Tom Achebe, our CEO, has shared his draft roadmap for the next four quarters (Q4 2026 to Q3 2027), and asked you to turn it into the roadmap we take to next week's planning offsite. The exec team (CEO, CFO, CRO and CTO) will read it beforehand. Write: 1. The roadmap itself, by quarter or by now, next and later: what each squad works on and the outcome each item serves. 2. The case for it, in no more than 1,200 words: why this order, what you changed from Tom's draft and why, what we're not doing, and the risks that could change the plan. The pack below is everything we have. Not all of it matters equally.

What the model was given12 items: About Harbor, FY27 goal (approved by the board, September 2026), Squads and capacity, Hiring, Card processor deadline, Candidate work (estimates in squad-weeks, by owning squad), Tom's draft roadmap, Procurement evidence, Card spend evidence, Multi-entity evidence, Integrations evidence, Other asks
About HarborSpend management (corporate cards, expenses and approvals) for companies with 200 to 2,000 employees. 1,100 customers, $86M ARR, average 700 employees per customer. 58% of revenue is interchange on card spend; the rest is subscription. Net revenue retention (NRR) is 104%.
FY27 goal (approved by the board, September 2026)Raise NRR to 112% by the end of Q3 2027 by expanding inside existing customers: more spend on Harbor cards, more entities and more modules. New-logo growth matters, but it comes second this year.
Squads and capacityFive squads: Cards, Expenses, Approvals & Policy, Integrations and Platform. After support and on-call, each has about 11 squad-weeks of roadmap capacity a quarter. Squads can take work from another squad's area, but it takes them about 1.5× as long.
HiringTwo new squads were approved in September 2026. Tom's draft counts the first from Q1 2027 and the second from Q2 2027, both at full capacity. Last year, our three approved squads took 5, 6 and 8 months from approval to their first full sprint, and each delivered about half its capacity in its first quarter.
Card processor deadlineOur card processor retires its v1 API on 31 March 2027. Every card authorisation we process runs through v1 today. Migrating to v2 is about 18 squad-weeks of Cards work, and the processor must then run 4 weeks of certification testing before we can cut over. Certification is the processor's time, not ours, but nothing can change on our side while it runs.
Candidate work (estimates in squad-weeks, by owning squad)1. Processor v2 migration: Cards 18. Hard deadline above. 2. Virtual cards for software subscriptions: Cards 10. Needs v2. 3. Real-time spend controls (declines out-of-policy spend at the point of sale): Cards 8, Approvals & Policy 4. Needs v2. 4. Procurement module (purchase requests and approvals, a new paid add-on): Approvals & Policy 30, Integrations 6. 5. Multi-entity support (subsidiaries under one parent account): Platform 12, Approvals & Policy 10. 6. Sage Intacct integration: Integrations 9. 7. Microsoft Dynamics integration: Integrations 12. 8. SSO and SCIM provisioning: Platform 5. 9. Receipt-matching improvements: Expenses 6. 10. Mobile app rewrite: Expenses 22.
Tom's draft roadmapQ4 2026: Cards: v2 migration. Approvals & Policy: multi-entity. Integrations: Intacct. Platform: multi-entity. Expenses: receipt matching. Q1 2027: Cards: finish migration, start virtual cards. New squad A: procurement module. Integrations: Dynamics. Platform: SSO and SCIM. Q2 2027: Cards: virtual cards, spend controls. New squad A: procurement launch. New squad B: mobile rewrite. Q3 2027: Cards: spend controls. Everyone else: procurement adoption, mobile launch. Tom's note: “Procurement is our path to 112%. I've told the board it can add $6M of ARR in its first year.”
Procurement evidenceSix design partners have used a prototype since June. Two said they would pay for it; the other four already use a dedicated procurement tool and said they'd need a reason to switch. Proposed price: $8 per user per month, for users who raise purchase requests. At our customers, about 10% of employees raise purchase requests.
Card spend evidenceIn QBRs with our 60 largest customers, 26 asked for virtual cards for software subscriptions, and 31 CFOs asked for real-time spend controls. Finance estimates our customers pay about $410M a year of software subscriptions on other cards or by invoice (extrapolated from the 60 QBR accounts). Our net interchange is 1.1% of card spend.
Multi-entity evidence14 of our 60 largest customers have asked for it. Between them they have 52 subsidiaries not on Harbor, and 9 of the 14 have said in writing they would add their subsidiaries once it exists. Customer Success sized it at $2.9M ARR if all 52 joined at their parents' pricing.
Integrations evidence17% of customers use Sage Intacct through a CSV export. Their gross revenue churn is 13% a year, against 7% for customers on our NetSuite integration. Intacct was cited in 31% of lost new-logo deals last year, Dynamics in 8%. 3% of customers use Dynamics.
Other asksCRO: “Five enterprise deals worth $1.2M of pipeline need SSO and SCIM, and 11 existing accounts have it on their security review list.” CTO: “The mobile app is on a framework version that loses support in late 2027, and I want the rewrite done before then.” Support: receipt auto-match is at 71%, and unmatched receipts are the most common expense ticket.
What a strong answer doesThe answer key the graders mark against

A four-quarter roadmap that protects the 31 March deadline: the migration build has to finish by about early March to leave 4 weeks of certification, which is roughly all of Cards' capacity for Q4 and most of Q1 (18 of about 18 squad-weeks available before certification). So it adds help or buffer (Platform or another squad at 1.5×, or cutting everything else from Cards) and moves virtual cards out of Q1: both v2 features start only after cut-over (virtual cards Q2, spend controls Q2–Q3). It treats the new squads as upside: on past hiring they arrive around February to April 2027 and deliver half capacity in their first quarter, so nothing committed depends on them before Q2–Q3. It makes a clear call on procurement: the $6M claim needs about 890 accounts (81% of customers) paying $6,720 a year each (70 requesters × $8 × 12), and only 2 of 6 design partners would pay, so it replaces the 30-week build with a cheap paid test with a threshold, and keeps Approvals & Policy on multi-entity and spend controls, which have stronger expansion evidence ($2.9M sized, 9 of 14 committed in writing; about $1.5M a year of interchange from moving a third of the $410M of subscriptions onto virtual cards). Intacct comes early for its churn; Dynamics later or cut; SSO and SCIM are small and justified; the mobile rewrite waits for a new squad, before the framework's support ends. It lists what changed from Tom's draft, what's not being done, and the risks (certification slipping, hiring slipping), with commitment falling with distance.

Critical failures (cap the score at 40)
  • Leaves the processor migration finishing too late for 4 weeks of certification before 31 March 2027, or schedules virtual cards or real-time spend controls before it
Task

Staff-level · v1.6

The outputs

8 setups answered this brief.

Usable with a quick edit9 of 13 checks passed

The output is complete, evidence-based, and addresses the CEO's request; a product manager could use it with light wording adjustments, without re-checking facts or redoing the logic.

Output

Harbor Roadmap, Q4 2026 to Q3 2027

Q4 2026 (now)

SquadWork (squad-weeks)Outcome it serves
CardsProcessor v2 migration (11 of 18)Keeps every card authorisation running. Hard deadline: cut over before 31 Mar 2027.
ExpensesReceipt-matching (6); mobile rewrite starts (5)Auto-match from 71% up; fewer expense tickets. Mobile rewrite ships before framework support ends.
Approvals & PolicyMulti-entity (10)NRR: subsidiaries join existing accounts.
PlatformSSO/SCIM (5); multi-entity (6)$1.2M enterprise pipeline; 11 accounts in security review. NRR via multi-entity.
IntegrationsSage Intacct (9)Retention: Intacct customers churn at 13% vs 7% on NetSuite. Fewer lost new-logo deals.

Q1 2027

SquadWorkOutcome it serves
CardsFinish v2 (7; code-complete by 15 Feb). Processor certification (4 weeks, change freeze). Cut over before 31 Mar. Virtual cards start (4).Authorisations protected. NRR: interchange on subscription spend.
ExpensesMobile rewrite (11)Mobile ships in Q3, ahead of late-2027 EOL.
Approvals & PolicyReal-time spend controls (4); procurement build (7)Spend controls: 31 CFOs asked for them. Procurement: pricing gate at end of Q1.
PlatformMulti-entity finish (6); 5 sw held for onboarding and reliabilityNRR: multi-entity GA.
IntegrationsProcurement integration (6); Dynamics starts (5)Procurement build. Dynamics is last in queue.
New squad A (Approvals & Policy home)Onboarding; procurement build from ~half capacityProcurement build.

Q2 2027

SquadWorkOutcome it serves
CardsVirtual cards (6); spend controls (5)NRR: software subscription spend moves onto Harbor cards.
ExpensesMobile rewrite (6 to finish); betaMobile feature-complete.
Approvals & Policy + new squad AProcurement build (11 plus squad A at ~half)Procurement build; design-partner beta prep.
PlatformCapacity held for multi-entity subsidiary onboardingNRR: subsidiaries of the 14 requesting customers.
IntegrationsDynamics (7 to finish). Squad B onboarding.Low-priority retention and new-logo coverage.

Q3 2027

SquadWorkOutcome it serves
CardsSpend controls (3); virtual cards hardening; 8 sw spareNRR: spend controls and virtual cards at scale.
ExpensesMobile launchMobile ships before EOL.
Approvals & Policy + new squad AProcurement build complete (base case ~August); design-partner paid pilots (September)Procurement: first paid pilots. Little or no FY27 ARR.
PlatformMulti-entity subsidiaries joinNRR.
IntegrationsDynamics complete; squad B at full capacity; next item chosen at Q3 reviewCoverage.

Later (Q4 2027 onward, gated): Procurement paid launch, only if the Q1 pricing gate is passed. Most procurement ARR lands in FY28.

---

The Case

The headline: 112% is not reachable in FY27

NRR is 104%. Reaching 112% by Q3 2027 means about $6.9M of additional net expansion ARR on our $86M base. My estimates for what this plan delivers inside the window:

  • Multi-entity: $1.2–1.9M. The ceiling is $2.9M if all 52 subsidiaries join. Nine of 14 customers have committed in writing, which is not the same as signed.
  • Virtual cards and spend controls: $0.4–0.7M. Interchange is 1.1%, so $410M of subscription spend is worth $4.5M at full capture. I assume 10–15% capture in-year. The $410M is extrapolated from 60 accounts.
  • Intacct churn reduction: $0.4–0.9M. This assumes Intacct-using ARR is roughly 17% of the base (~$15M) and that the 6-point churn gap is caused by the CSV workflow, not by the customers themselves.
  • Procurement: near zero.

That totals $2.0–3.5M, which takes NRR to roughly 106–108%. Procurement is the only item big enough to close the gap, and it cannot deliver inside FY27. Tom's note says procurement can add $6M in its first year. $6M is about 80% of the theoretical ceiling (1,100 customers × 70 requesters × $96 a year ≈ $7.4M), and only 2 of 6 design partners said they would pay. Even if the first-year number holds, the timing means it lands in FY28.

The board needs this correction before the offsite ends. Tom should brief the chair before the board sees a revised NRR path. I suggest we present 106–108% for FY27 with a path to 112% in FY28 and set out what would have to be true to close the gap.

Why this order

  1. Protect the base first. The v2 migration is the only hard deadline. Every card authorisation depends on it. Cards is planned with about two weeks of slack: migration complete by 15 Feb, 4 weeks of certification, cut-over by mid-March. The Cards squad is ring-fenced until cut-over, and I want a go/no-go on 15 Feb.
  2. Put the cheap, high-demand items first. SSO/SCIM is 5 squad-weeks and unlocks $1.2M of pipeline and 11 security reviews. Multi-entity is the largest in-year expansion lever and has written demand. Intacct is 9 squad-weeks against a 13% churn rate on 17% of customers.
  3. Use squads that exist. Expenses can build the mobile rewrite with its own capacity by Q2. That ships before the framework's late-2027 end of support without waiting for a new squad.
  4. Gate procurement. The build is 30 squad-weeks of Approvals & Policy work on a product where only a third of design partners have said they would pay. We build it, but the pricing decision comes at the end of Q1.

What I changed from Tom's draft, and why

Tom's draftOur planWhy
New squad A: procurement from Q1; launch in Q2.Procurement built by existing Approvals & Policy plus squad A; base-case build complete ~August; paid pilots in September.30 squad-weeks cannot be built and launched in Q2 by a squad that is still ramping.
New squads at full capacity from Q1 and Q2.Modelled at half in their first quarter.Last year's squads took 5–8 months to first full sprint and delivered half capacity in their first quarter. Tom's draft assumes about 55 squad-weeks from new squads in Q1–Q3. The realistic best case is about 44.
Mobile rewrite assigned to new squad B in Q2.Mobile rewrite in Expenses from Q4; launch in Q3.Expenses has the capacity now. Squad B is freed for Integrations.
Platform carries 12 squad-weeks of multi-entity in Q4 (above its 11 capacity); SSO in Q1.SSO and 6 weeks of multi-entity in Q4; multi-entity finishes in Q1.SSO moves forward to unlock $1.2M of pipeline. Multi-entity still reaches GA by end of Q1.
Dynamics in Q1.Dynamics runs in spare Integrations capacity from Q1 to Q2, and is the first item to cut.Dynamics is used by 3% of customers and was cited in 8% of lost deals. Intacct, at 17% of customers and 31% of lost deals, comes first.
Cards: finish migration in Q1 and start virtual cards.Same sequence, with a 15 February checkpoint.Tom's sequence was right. The checkpoint protects the certification window.

What we're not doing

  • Procurement at full scope before the Q1 pricing gate. If fewer than three of six design partners commit to a paid pilot at $8 per user, we cut scope and move capacity to other work.
  • Dynamics as a committed FY27 item. It fits in spare capacity. It does not get priority over anything else.
  • Mobile in a new squad. Expenses owns it.
  • Platform-led procurement work. Platform has about 16 spare squad-weeks in Q1–Q2 and could absorb procurement sub-components at 1.5× cost if the gate passes. We don't plan that yet.
  • Any NRR commitment above 108% for FY27 in board materials.

Risks that could change the plan

  1. The v2 migration slips past the certification window. This is the existential risk. If migration is not code-complete by about 22 February, we miss 31 March. Mitigations: ring-fence Cards from January, book a certification slot with the processor now, and agree a contingency with the processor in writing.
  2. New squads ramp slower than planned. Each month of delay pushes multi-entity onboarding, procurement and Dynamics later. Procurement is most exposed because Approvals & Policy is already at zero slack.
  3. Procurement demand is weaker than the evidence suggests. Two of six design partners would pay, and four already use a dedicated tool. The Q1 gate exists for this reason.
  4. Virtual card and interchange sizing is soft. The $410M is extrapolated from 60 accounts. Capture is an assumption.
  5. Intacct churn may not be caused by the CSV workflow. If the churn gap is about customer type rather than integration, the retention benefit is smaller.
  6. Multi-entity commitments are not contracts. Nine written commitments are real signals, but pricing for subsidiaries still needs to be set and sold.
  7. Mobile slips past Q3. Expenses has about 5 spare squad-weeks in Q2 and Q3 for launch fixes. A slip beyond Q3 puts us close to the framework's end of support.

Decisions needed at the offsite

  1. Approve the revised FY27 NRR range (106–108%) and agree how Tom briefs the board.
  2. Confirm the 15 February v2 checkpoint and the Cards ring-fence.
  3. Approve the procurement build with the Q1 pricing gate and a pre-agreed cut scope.
  4. Move mobile into Expenses and release squad B to Integrations.

Grades and run details

Decision model 69 · LLM judge 93
Decision model checks
  • failUses the supplied evidence correctly9%
  • passAddresses the actual decision92%
  • partialRespects explicit constraints18%
  • passIdentifies material uncertainty100%
  • passAvoids unsupported claims19%
  • passProduces the required deliverable53%
  • partialProtects the processor deadline37%
  • passPlans on the squads we actually have49%
  • passMakes the call on procurement88%
  • passOutcomes, with certainty that falls with distance86%
  • failFits the stated capacity22%
  • failSequences around dependencies33%
  • passTells the story of the bets46%
Run
Run
#1
API response time
1.8 min
Submitted
8 Oct 2026

Needs more than a quick edit11 of 13 checks passed

The plan lacks any buffer for the processor migration, which is a critical risk, and the squad-based structure doesn't tell a thematic story for the exec offsite.

Output

Harbor FY27 roadmap — Q4 2026 to Q3 2027

Planning basis: Each existing squad has about 11 roadmap squad-weeks per quarter. Figures below are the candidate-work estimates; capacity not assigned to a feature is deliberately held for launch, customer activation, fixes and contingency. New squads are not counted at full capacity until their ramp is proven.

QuarterSquadWork and outcome
Q4 2026CardsProcessor v2 migration — 11 of 18 weeks. Start the mandatory migration and establish the path to retire v1 before the processor deadline.
Approvals & PolicyMulti-entity — 10 of 10 weeks. Build parent/subsidiary approval and policy support so existing customers can bring more entities onto Harbor.
PlatformMulti-entity — 11 of 12 weeks. Build the platform foundation for subsidiaries under a parent account.
IntegrationsSage Intacct — 9 weeks. Replace CSV workarounds to reduce a retention risk among existing customers.
ExpensesReceipt matching — 6 weeks. Improve the most common expense-ticket driver; use remaining capacity for fixes and customer feedback.
Q1 2027CardsFinish v2 — 7 weeks. Target implementation complete by the end of February, leaving March for the processor’s four-week certification. No card-side changes during certification.
Approvals & PolicyMulti-entity launch and customer activation. Support the first subsidiary rollouts. Use remaining capacity for a time-bounded procurement willingness-to-pay and switching validation; do not build the module.
PlatformFinish multi-entity — 1 week; SSO/SCIM — 5 weeks. Complete the entity foundation, then address security blockers affecting existing accounts and enterprise pipeline.
IntegrationsMicrosoft Dynamics — 11 of 12 weeks. Address a cited reason for lost new-logo deals; complete the final week in Q2.
ExpensesMobile rewrite — 11 of 22 weeks. Start early to reduce the risk of running out of time before the current framework loses support in late 2027.
Q2 2027CardsVirtual cards for software subscriptions — 10 weeks. After v2 certification, give customers a way to move subscription spend onto Harbor cards and capture more interchange.
Approvals & PolicyReal-time spend controls — 4 weeks. Build the policy logic for point-of-sale declines, in step with Cards’ Q3 work.
PlatformSSO/SCIM and multi-entity activation. Support adoption and resolve launch issues; no new major platform feature is precommitted.
IntegrationsFinish Dynamics — 1 week; customer rollout. Complete the integration and support implementation.
ExpensesFinish mobile rewrite — 11 weeks. Complete the rewrite by Q2, ahead of the framework support deadline.
Q3 2027CardsReal-time spend controls — 8 weeks. Complete and release with Approvals & Policy; help finance teams enforce policy at the point of sale.
Approvals & PolicyRelease and adoption for real-time controls; multi-entity activation. Focus on turning shipped capabilities into customer usage and expansion.
PlatformActivation and reliability for multi-entity and SSO/SCIM. Prioritize customer rollout and fixes over an unvalidated new feature.
IntegrationsIntacct and Dynamics adoption. Help customers get live and measure retention and deal impact before choosing further integration work.
ExpensesMobile rollout and stabilization; measure receipt-match performance. Address launch issues and confirm the support-ticket outcome.

New Squads A and B: Plan for onboarding and ramp, not full feature capacity. When productive, use them for bounded rollout, quality or activation work. No committed launch depends on their availability.

Case for the plan

Why this order

The board’s priority is to move NRR from 104% to 112% by the end of Q3, primarily through expansion in existing customers. The plan therefore favors credible opportunities to retain customers or deepen their use of Harbor, while keeping some room for new-logo needs.

First, v2 is a hard dependency and a hard deadline. All authorisations still run through v1, which retires on 31 March. The 18 Cards squad-weeks consume 11 weeks in Q4 and seven in Q1. That leaves time to finish implementation before the processor’s four-week certification and to keep the required freeze during testing. Virtual cards and real-time controls must wait until certification is complete.

Next, multi-entity has concrete expansion evidence: 14 of the 60 largest customers asked, and nine have said in writing that they would add subsidiaries. The $2.9M ARR estimate assumes all 52 subsidiaries join, so it is an upside case, not a commitment. Completing the Platform and Approvals & Policy work across Q4 and Q1 creates a path to test that demand with real rollouts.

Virtual cards are the first post-migration Cards feature because 26 large customers asked for them and Finance estimates $410M of annual subscription spend is currently elsewhere. At Harbor’s 1.1% net interchange, full capture would represent about $4.5M in annual interchange revenue, but that is a ceiling, not a forecast: the spend estimate is extrapolated, and adoption and capture are unknown. Real-time controls follow, with coordinated work across Cards and Approvals & Policy. The 31 CFO requests are strong demand evidence, but the plan avoids trying to fit both features into Cards’ 11-week quarterly capacity.

Intacct comes before Dynamics. Intacct is used by 17% of customers through CSV, and those customers have higher observed gross revenue churn than customers on NetSuite. That comparison is correlational, but it supports addressing a retention risk. Dynamics has lower existing use (3%) and a more limited new-logo case, but the 8% of lost deals citing it justifies completing the 12-week integration after Intacct.

SSO/SCIM is a relatively small, five-week Platform investment tied to five enterprise deals representing $1.2M of pipeline and 11 existing security reviews. It supports commercial outcomes without displacing the larger expansion bets.

What changed from Tom’s draft

  • Removed procurement from committed build and launch. Six design partners are not enough evidence for the board’s $6M first-year ARR claim: only two said they would pay, and four already use another tool. At the proposed price, an average 700-employee customer would generate about $6,720 ARR if 10% of employees were buyers. Reaching $6M would require roughly 893 such customer equivalents. We should validate willingness to switch and pricing before committing 30 Approvals & Policy and six Integrations squad-weeks.
  • Corrected the hiring assumption. Prior squads took five to eight months from approval to a first full sprint and delivered about half capacity in their first quarter. The plan does not count either new squad at full capacity in Q1 or Q2, or make a feature dependent on them.
  • Made the v2 critical path explicit. The migration spans Q4 and Q1, with implementation targeted ahead of March certification. Tom’s draft did not protect that freeze clearly enough.
  • Sequenced Cards work to fit capacity. Virtual cards and real-time controls require 18 Cards squad-weeks together, more than one quarter’s 11-week capacity. They are sequenced across Q2 and Q3.
  • Started the mobile rewrite with the existing Expenses squad. The work is 22 weeks, so starting in Q1 targets completion in Q2 without relying on an unproven new squad.

What we are not doing

We are not committing to build procurement, starting a mobile rewrite with a new squad at assumed full capacity, or taking on another major integration or platform feature in Q3 without evidence from the launches above. New-logo work remains in the plan, but expansion and retention come first.

Risks that could change the plan

  • Processor timing: Any migration slip puts card authorisations at risk. Track delivery and certification weekly; protect the Q1 card schedule and do not pull dependent features forward.
  • Expansion may not materialize: Multi-entity interest and card requests do not guarantee adoption or spend migration. Track subsidiary activation, card spend moved, and controls usage—not just releases.
  • Commercial estimates are uncertain: The $2.9M multi-entity estimate and $4.5M card-spend ceiling are not forecasts. Procurement’s $6M claim is especially unsupported by current evidence.
  • Mobile has little slack: The 22-week rewrite fills two quarters of Expenses capacity. Scope growth, delivery delays or an earlier-than-expected framework deadline would require replanning.
  • Hiring, integration and cross-squad delivery may slip: Ramp timing is uncertain; the Intacct churn difference may not be caused by the integration; and spend controls depend on coordinated Cards and Approvals & Policy work.

This roadmap gives Harbor credible bets aligned to the NRR goal, but it does not establish that the 112% target is guaranteed. We should review expansion and retention indicators at each quarterly planning checkpoint and change the plan if customer evidence or delivery risk materially shifts.

Grades and run details

Decision model 92 · LLM judge 79
Decision model checks
  • passUses the supplied evidence correctly60%
  • passAddresses the actual decision95%
  • passRespects explicit constraints22%
  • passIdentifies material uncertainty88%
  • passAvoids unsupported claims67%
  • passProduces the required deliverable75%
  • partialProtects the processor deadline41%
  • passPlans on the squads we actually have98%
  • partialMakes the call on procurement50%
  • passOutcomes, with certainty that falls with distance69%
  • passFits the stated capacity66%
  • passSequences around dependencies60%
  • passTells the story of the bets74%
Run
Run
#1
API response time
1.9 min
Submitted
1 Oct 2026

Usable with a quick edit11 of 13 checks passed

The output is complete and evidence-based; a quick edit to add the Q3 Expenses capacity allocation (e.g., 11 weeks for rollout and support) would make it fully checkable.

Output

Harbor Roadmap, Q4 2026 to Q3 2027

Pre-read for the planning offsite. Prepared from Tom's draft.

Part 1: The roadmap

Planning assumptions - Each existing squad has 11 squad-weeks of roadmap capacity a quarter. Work done in another squad's area costs 1.5×. - The two new squads are planned at the dates our history supports, not the approved dates. Last year's squads took 5 to 8 months from approval to their first full sprint. - Squad A: 0 weeks in Q1, about 5 in Q2 (half capacity), about 11 in Q3. - Squad B: 0 weeks in Q2, about 5 in Q3. - Nothing committed depends on either new squad. - The fiscal year runs Q4 2026 (Oct to Dec) through Q3 2027 (Jul to Sep).

Squad plan by quarter (squad-weeks in brackets)

SquadQ4 2026Q1 2027Q2 2027Q3 2027
Cardsv2 migration (11)v2 migration: code complete by ~31 Jan (4). Certification runs through Feb with no card-stack changes. Cutover ~1 Mar. Cutover contingency (3). Virtual cards design and sandbox build, no production changes (4)Virtual cards build and launch ~May (6). Real-time spend controls (5)Spend controls launch ~Jul (3). Adoption work on virtual cards and controls with CS (8)
Platformv2 migration support (5, which buys ~3.3 Cards-weeks). Multi-entity (6)Multi-entity, ships ~end Feb (6). SSO and SCIM, ships by end Mar (5)Subsidiary onboarding tooling and SSO rollout to the 11 accounts (5). Procurement billing and permissions, if the gate passes (6)New-squad onboarding and spend-controls latency support (5). Reserve (6)
Approvals & PolicyMulti-entity (10). Procurement pricing test with design partners (1)Procurement build, phase 1 (11). Gate at end of MarchSpend controls policy engine (4). Procurement (7)Procurement finish (8). Beta, then GA ~Sep (3)
IntegrationsSage Intacct, ships ~early Dec (9). CSV-to-integration migration tooling (2)Move ~187 Intacct CSV customers onto the integration, with CS (5). Dynamics (6)Dynamics, ships ~May (6). Procurement integrations (5)Procurement integrations (1). Hardening and reserve (10)
ExpensesReceipt matching (6). Mobile rewrite (5)Mobile rewrite (11)Mobile rewrite, feature-complete ~May (6). Beta and fixes (5)Staged rollout. Old app retired by end Sep, a quarter or more before framework support ends
New squad A (upside)——Onboarding (~5)~11: accelerate procurement, then own it. Otherwise, adoption work on cards
New squad B (upside)———Onboarding (~5). Scope set at Q2 planning

Items, outcomes and key dates

ItemOutcome it servesKey dateOwner
Processor v2 migrationKeeps every card authorisation running. That protects the 58% of revenue that is interchangeCode complete ~31 Jan, cutover ~1 Mar, about 4 weeks of buffer before 31 MarCards + Platform
Multi-entityMore entities: 9 customers have committed in writing to add subsidiariesShips ~end Feb. Subsidiaries onboard in Q2 and Q3Platform + A&P
Virtual cardsMore card spend: brings software subscriptions onto Harbor cardsLaunches ~MayCards
Real-time spend controlsMore card spend: gives CFOs the confidence to issue more cards (31 CFOs asked for it)Launches ~JulCards + A&P
Intacct integration and CSV migrationRetention: Intacct-on-CSV customers churn at 13%, against 7% on NetSuite. Also addresses 31% of lost dealsIntegration ships Dec. Customers move in Q1Integrations
SSO and SCIMRetention of the 11 accounts with it on their security review list. Also $1.2M of new-logo pipelineShips MarPlatform
ProcurementMore modules: a new paid add-onGate end Mar. GA ~SepA&P
DynamicsNew-logo win rate (8% of lost deals)Ships ~MayIntegrations
Receipt matchingLower support load: the most common expense ticketShips DecExpenses
Mobile rewriteRemoves platform risk before the framework loses supportOld app retired by SepExpenses

NRR bridge: what this plan plausibly delivers by end of Q3 2027

Moving NRR from 104% to 112% needs about $6.9M of additional net expansion (8 points of $86M).

LeverBasisEstimate in window
Multi-entity9 of the 14 requesting customers committed in writing, roughly 33 of 52 subsidiaries. The $2.9M figure assumes all 52 join$1.5M to $2.9M
Virtual cards$410M × 1.1% = $4.5M ceiling. Assumes 15 to 35% captured within ~5 months of launch$0.7M to $1.6M
Intacct churn reduction~$14.6M of ARR on CSV. Closing the churn gap is worth ~$0.9M a year, and only part of that lands in window$0.3M to $0.6M
ProcurementGA in ~SepUnder $0.3M
Spend controls, SSO and SCIMNot sizedUpside or protection
Total$2.8M to $5.4M, or roughly 107% to 110% NRR

---

Part 2: The case for it

The short version

Tom's draft has the right ingredients. It does, however, have three problems:

  1. No buffer on the processor deadline. The migration has no room to slip against a date that could stop every card authorisation.
  2. Key work sits on squads we won't have. Procurement and the mobile rewrite depend on new squads that history says won't exist at full capacity until Q3 at the earliest.
  3. Procurement can't carry the goal. It is expected to deliver the NRR target, but the evidence doesn't support $6M and the timing puts almost none of it inside the measurement window.

This plan protects the deadline first. It then sequences the levers with the strongest evidence so they land early enough to count. Even so, product alone likely gets us to 107% to 110%, not 112%. The exec team should know that now, not in Q3.

Why this order

1. The processor migration comes first, with buffer.

Every authorisation runs through v1, so a missed cutover puts most of our revenue at risk. On Tom's draft: - Cards alone finishes the 18 weeks around late February. - The 4-week certification then ends in the last days of March. - That leaves under a week of slack, with December holidays not counted and no room for a failed certification.

Lending 5 Platform weeks in Q4 changes this: - Code complete moves to about 31 January. - Cutover lands around 1 March. - We have roughly four weeks of buffer.

This also means virtual cards can't "start in Q1" as the draft says. During certification nothing on our card stack can change. Cards will do design and sandbox work in Q1 and start production work after cutover.

2. Multi-entity and virtual cards next, because they have the best evidence and they land in time.

NRR is measured at the end of Q3, so anything launching after about June barely counts. Our two strongest levers are: - Multi-entity: 9 customers have committed in writing to add subsidiaries, which is about $1.5M to $2.9M. - Virtual cards: a $4.5M ceiling. 26 of our 60 largest customers asked for them.

Multi-entity ships in February. Virtual cards ship in May. Real-time spend controls follow in July because they share the Cards squad. They matter to 31 CFOs, but we haven't sized them.

3. Retention work runs in parallel, because it is cheap.

  • Intacct: 9 weeks of work. It addresses a segment churning at nearly twice our NetSuite rate and appears in 31% of lost deals. The integration only reduces churn if customers actually move off CSV, so I added a Q1 migration push with CS.
  • SSO and SCIM: 5 weeks of work. It protects 11 accounts and unblocks $1.2M of pipeline.

4. Procurement is real but gated, and it is not the FY27 lever.

What I changed from Tom's draft, and why

  1. Migration buffer. Platform lends 5 weeks in Q4. The cost is that multi-entity moves from December to February. I think that trade is clearly right, because the downside of a missed cutover is existential.

2. New squads planned realistically. - Squads approved in September reach their first full sprint between February and May, based on last year's 5 to 8 months. - Each then runs at half capacity for a quarter. - Squad A realistically gives about 5 weeks in Q2. Squad B gives nothing before Q3. - I've kept every committed item on the existing five squads. The new squads are upside.

3. Procurement moved to Approvals & Policy and gated. - A&P owns the domain, has capacity after multi-entity, and was left without work in the draft. - Build starts in Q1. At the end of March we need signed paid-pilot commitments from about 10 customers beyond the two design partners, or we redirect A&P. - GA lands around September. If Squad A arrives on time, it accelerates the work.

4. The $6M procurement figure needs resetting with the board. - At $8 per user per month and 10% of our ~770,000 employees raising requests, the ceiling is about $7.4M a year if every customer adopts. $6M therefore implies about 81% adoption. - Only 2 of 6 design partners said they'd pay. The other four already use a dedicated tool. - At a one-in-three adoption rate, procurement is worth about $2.5M at full run-rate, reached over years, and less than $0.3M inside FY27. - I recommend Tom and the CFO correct this at the next board touchpoint, before the board starts tracking it.

5. Mobile rewrite moved to Expenses, starting in Q4. - Expenses owns the app and the draft left it idle after Q4. - The rewrite finishes by May, and the old app is retired by September. - This gives the CTO's deadline real margin. Leaving it on Squad B would risk missing it.

  1. Spend controls now include the A&P work (4 weeks). The draft omitted this.
  1. Dynamics moves from Q1 to May. The Intacct customer migration goes first, because Intacct matters roughly five times more by customer count and lost deals.

What we're not doing

  • Not claiming procurement drives FY27 NRR. We are also not building past the March gate without paid commitments.
  • Not starting virtual cards or anything else on the card stack before cutover.
  • Not putting any deadline-critical or NRR-critical work on unhired squads.
  • Not adding new-logo-only features beyond Intacct, Dynamics and SSO, which also serve retention. New logos come second this year, as the board set.
  • Not filling the Q3 reserve on Platform and Integrations. That slack absorbs migration fallout, integration hardening and new-squad onboarding. If the migration goes cleanly, we can release it to card-spend adoption work at Q2 planning.

Risks that could change the plan

1. Certification fails or needs a second round. A re-run would consume the entire buffer. - Book the certification slot now. - Run a pre-certification dry run in January. - Ask the processor in writing for a contingency extension and an emergency-fix protocol for the freeze window. - If code complete slips past mid-February, pull more Platform capacity immediately.

2. The $410M software-spend figure is extrapolated from our 60 largest accounts. Smaller customers may put proportionally less software spend on cards. - In Q4, validate it with AP data from about 20 mid-sized customers. - If it halves, virtual cards is worth about $0.4M to $0.8M in window and spend controls rises in priority.

  1. Multi-entity conversion comes in below the written commitments. CS should agree subsidiary onboarding plans with the 9 customers in Q1, before the feature ships.
  1. Real-time spend controls sit on the authorisation path. A latency problem could cause false declines. Platform supports this work in Q3, and we should launch in stages.
  1. Hiring slips further. This doesn't threaten committed dates, but procurement would stay in Q3 and our adoption capacity shrinks.
  1. The procurement gate fails. A&P redirects to deeper spend controls and multi-entity policy features, both of which feed card spend.
  1. The mobile framework end-of-support date is only "late 2027". The CTO should confirm the exact date.

Decisions we need at the offsite

  1. Approve Platform's Q4 loan to Cards, accepting that multi-entity moves to February.
  2. Agree the board message on procurement, and the gate criteria behind it.
  3. Close the NRR gap. This plan lands about 107% to 110%. Closing the remaining $1.5M to $4M needs commercial levers the CRO and CFO own, such as CS-led card activation, packaging and pricing, and a multi-entity sales motion beyond the 9. Otherwise we tell the board now that 112% is a stretch.
  4. Authorise the processor extension request.

Grades and run details

Decision model 85 · LLM judge 93
Decision model checks
  • failUses the supplied evidence correctly12%
  • passAddresses the actual decision98%
  • partialRespects explicit constraints29%
  • passIdentifies material uncertainty100%
  • partialAvoids unsupported claims23%
  • passProduces the required deliverable65%
  • passProtects the processor deadline97%
  • passPlans on the squads we actually have95%
  • passMakes the call on procurement91%
  • passOutcomes, with certainty that falls with distance89%
  • passFits the stated capacity32%
  • passSequences around dependencies59%
  • passTells the story of the bets81%
Run
Run
#1
Time to output
3.5 min
Submitted
30 Sept 2026

Check by check

Where the outputs differ comes first. Open a check to read the judge’s reasons. “Mixed” means the two graders disagreed.

Where they differ 5

Uses the supplied evidence correctlyMixedRightMixed
Haiku 5.5 · API

All statements about the current situation (numbers, dates, customer feedback) are drawn directly from the supplied context or derived from it by arithmetic.

GPT-6 Luna · API

All factual claims about the current situation are directly from the brief or derived by correct arithmetic.

Opus 5.5 · Claude

All facts and figures used are directly from the supplied context or derived by arithmetic from it; no invented current-state claims.

Protects the processor deadlineRightWrongRight
Haiku 5.5 · API

The migration is code-complete by 15 Feb, leaves 4 weeks for certification, cut-over before 31 Mar, and virtual cards/spend controls start only after cut-over.

GPT-6 Luna · API

The plan uses all of Cards' capacity with no buffer or help from other squads, leaving no protection against slippage before the 31 March deadline.

Opus 5.5 · Claude

Migration finishes by 31 Jan with Platform help, certification runs in Feb, cutover ~1 Mar, leaving buffer; virtual cards and spend controls start after cutover.

Fits the stated capacityMixedRightMixed
Haiku 5.5 · API

The committed work sums to ≤11 squad-weeks per squad per quarter, with visible slack in some quarters; the case explicitly lists what was cut or deferred (Dynamics, full-scope procurement, mobile in new squad).

GPT-6 Luna · API

Committed work fits within 11 squad-weeks per quarter with slack held for launch and fixes, and it names what was cut (procurement build, additional features).

Opus 5.5 · Claude

Q3 Expenses row lacks squad-week allocation, so the committed plan cannot be fully checked against the 11-week capacity.

Sequences around dependenciesMixedRightRight
Haiku 5.5 · API

v2 features only appear after migration and certification; mobile rewrite finishes before framework EOL; new squads are used only after ramping; the roadmap names the v2 dependency and hiring constraint.

GPT-6 Luna · API

Respects v2 dependency for virtual cards and spend controls, places them after certification, and names the dependencies that drive the order.

Opus 5.5 · Claude

V2 features wait for migration cutover; procurement is gated; other dependencies are respected and named.

Tells the story of the betsMixedMixedRight
Haiku 5.5 · API

The roadmap is presented as a squad-by-quarter table, not grouped into a few overarching themes; the case explains the bets but the roadmap itself is a list of projects.

GPT-6 Luna · API

The roadmap is organized by squad, not grouped into a few themes with a narrative about why those are the biggest bets now.

Opus 5.5 · Claude

The case groups work into themes (protect deadline, expand card spend, retention, gated procurement) and explains why each is the biggest bet now.

All got right 8

Addresses the actual decisionRightRightRight
Haiku 5.5 · API

The output commits to a clear roadmap and a specific NRR range (106–108%), frames it for the exec team, and states what would change the call (pricing gate, migration checkpoint).

GPT-6 Luna · API

Commits to a clear plan (no procurement build, validation instead) and states that quarterly review of expansion and retention indicators could change it.

Opus 5.5 · Claude

Commits to a clear roadmap and case, framed for the exec team, with explicit changes from Tom's draft and conditions that would alter the plan.

Respects explicit constraintsRightRightRight
Haiku 5.5 · API

The requested deliverable (roadmap by quarter plus case ≤1,200 words) is present, and the roadmap enforces the processor deadline, capacity limits, and hiring histories.

GPT-6 Luna · API

Delivers a quarterly roadmap with outcomes and a case under 1,200 words covering the required topics for the exec team.

Opus 5.5 · Claude

Delivers a roadmap by quarter with outcomes, a case under 1,200 words, and addresses the required reader and form.

Identifies material uncertaintyRightRightRight
Haiku 5.5 · API

The case identifies specific unknowns (migration slip, squad ramp speed, procurement demand, interchange capture) and how they would be resolved or would change the plan.

GPT-6 Luna · API

Names specific risks (processor timing, adoption, commercial estimates, mobile slack, hiring) and how they would be tracked or resolved.

Opus 5.5 · Claude

Names specific risks (certification, $410M estimate, multi-entity conversion, hiring, procurement gate) and how each would be resolved or change the plan.

Avoids unsupported claimsRightRightRight
Haiku 5.5 · API

Causes and forecasts are explicitly labelled as assumptions (e.g., 'I assume 10–15% capture', 'This assumes the churn gap is caused by CSV'), and confident claims are based on supplied evidence.

GPT-6 Luna · API

Labels correlations and extrapolations clearly (e.g., 'correlational', 'ceiling, not a forecast') and avoids presenting them as established fact.

Opus 5.5 · Claude

Assumptions are clearly labelled; interpretations and forecasts are presented as estimates, not established facts.

Produces the required deliverableRightRightRight
Haiku 5.5 · API

The roadmap table and case together form a complete, actionable packet within the word limit that the exec team could read and act on.

GPT-6 Luna · API

The roadmap and case are complete, in the requested form, and usable by the exec team without major gaps.

Opus 5.5 · Claude

The roadmap and case are complete, in the requested format, and the exec team could act on them without major gaps.

Plans on the squads we actually haveRightRightRight
Haiku 5.5 · API

New squads are modelled at half capacity in their first quarter, consistent with the hiring history; no critical commitments depend on them at full capacity until Q2–Q3.

GPT-6 Luna · API

Does not count new squads at full capacity in Q1 or Q2 and commits no important work to them, consistent with past hiring delays and ramp.

Opus 5.5 · Claude

New squads are planned at half capacity arriving Q2/Q3 based on history, and no committed work depends on them.

Makes the call on procurementRightRightRight
Haiku 5.5 · API

The output checks the $6M claim against pricing (implies ~80% adoption) and design partner evidence (2 of 6 would pay), commits Approvals & Policy to higher-evidence work, and proposes a gated build with a paid-pilot threshold before full launch.

GPT-6 Luna · API

Checks the $6M claim against pricing and adoption evidence, shows it would need ~893 customers, and replaces the build with a paid validation test.

Opus 5.5 · Claude

Checks the $6M claim against pricing and adoption evidence, shows it implies ~81% adoption, contrasts with 2/6 design partners, and proposes a gated paid-pilot test.

Outcomes, with certainty that falls with distanceRightRightRight
Haiku 5.5 · API

Every roadmap item names the problem or outcome it serves, and later items (e.g., 'virtual cards at scale', 'first paid pilots') are deliberately looser than near-term ones.

GPT-6 Luna · API

Every item names an outcome or problem, and later items are deliberately looser (e.g., 'focus on turning shipped capabilities into customer usage').

Opus 5.5 · Claude

Every item names its outcome; near-term items have specific dates, later items are looser (e.g., '~May', '~Sep').

Results

Every setup we’ve tested on this task type, across all its tasks and repeats, graded on the current checklist. Calibrated. Where our PM checked the graders by hand, they agreed on 79% of 150 checks.

#Model · HarnessTask scoreDecision modelLLM judgeRunsCritical failures
1GPT-6.1 SolwithAPI100.096.42None
2GPT-6 AstrawithChatGPT93.9100.02None
3Opus 5.5withClaude90.296.42None
4Sonnet 5.5withAPI94.192.62None
5GPT-6 LunawithAPI92.085.42None
6Haiku 5.5withAPI74.296.42None
7Gemini 3.8 FlashwithAPI74.266.82None
8Gemini 3.5 Flash-LitewithGemini56.936.521 capped

What you get for the cost

No setup scores within 5 points of the leader at half its cost or less here, so the leader is also the best value.

Task score against relative cost for each setup. The table below has the same figures.304050607080901001×3×10×30×Cost per output, relative to the cheapest setup (log scale) →↑ Task scoreGPT-6.1 Sol · API: task score 98, 6.9× the cheapestGPT-6 Astra · ChatGPT: task score 97, 39× the cheapestOpus 5.5 · Claude: task score 94, 37× the cheapestSonnet 5.5 · API: task score 94, 36× the cheapestGPT-6 Luna · API: task score 87, the cheapestHaiku 5.5 · API: task score 86, 2.1× the cheapestGemini 3.8 Flash · API: task score 68, 5.5× the cheapestGemini 3.5 Flash-Lite · Gemini: task score 44, the cheapest, 1 critical failureGPT-6.1 Sol · APIGPT-6 Astra · ChatGPTOpus 5.5 · ClaudeSonnet 5.5 · APIGPT-6 Luna · APIHaiku 5.5 · APIGemini 3.8 Flash · APIGemini 3.5 Flash-Lite · Gemini
Nobody beats these on both score and costAt least one critical failure
Task score and relative cost by setup
SetupTask scoreRelative cost (cheapest = 1)
GPT-6 Luna · API871.0×
Gemini 3.5 Flash-Lite · Gemini441.1×
Haiku 5.5 · API862.1×
Gemini 3.8 Flash · API685.5×
GPT-6.1 Sol · API986.9×
Sonnet 5.5 · API9436×
Opus 5.5 · Claude9437×
GPT-6 Astra · ChatGPT9739×

Cost is each output’s tokens at the model’s list price, shown relative to the cheapest setup. Runs done in an app have no usage meter, so theirs are estimated from the same setup’s measured runs. How we compare cost

About the task

The PM job

Turning strategy into a sequenced plan.

Why it matters

A roadmap is where strategy meets capacity. Dated feature lists turn guesses into promises.

What good looks like

  • Items are problems or outcomes, not just features
  • Sequencing reflects dependencies
  • Explicit trade-offs
  • Commitment falls with distance

Deliberately not measured

  • Gantt formatting
Capability tested

Sequencing under constraints

The failure we’re looking for

A dated wishlist sorted by excitement

Grading

Decision model and LLM judge, calibrated against a blind PM review