Needs more than a quick edit6 of 11 checks passed
The test lacks a numeric threshold and measurement window, so a PM would need to add those before acting. The paid math is per signup rather than per publishing creator, but still shows the loss.
Output
To: Sam Okoro, Head of Growth Subject: Next-quarter engineering allocation
Recommendation: Put both engineers on the respondent-to-creator loop: ship and rigorously measure “Make a form like this.” Don’t double paid search or build the referral programme this quarter. Keep producing SEO templates through the content team; the available evidence doesn’t show that engineering is the constraint there.
Our primary loop is: a respondent sees a Pollen badge → visits or copies the form → signs up → publishes a form → brings in more respondents. It’s our clearest product-led acquisition loop because creators’ forms distribute the invitation to make a form.
It is not currently compounding. An average active creator publishes 2.22 forms (40,000 / 18,000), producing about 267 respondent exposures per month. Applying the current badge funnel gives roughly 0.085 next-month-active creators per active creator (267 × 0.9% × 11% × 38% × 85%). That’s well below the 1.0 needed for the loop to reproduce itself.
The pilot is promising: using its 1.6% click rate and 52% publish rate, the same calculation would imply about 0.21 next-month-active creators per creator—roughly 2.4× the current rate, if those new publishers retain at the existing 85% rate. That retention is an assumption, not a pilot result. Even so, the loop would still be subcritical: it could help offset churn, but it would not independently drive exponential growth.
Engineering plan - Engineer 1: Build the copy-to-editor flow, preserving the original form’s structure and making it easy to publish a customized copy. - Engineer 2: Instrument the full funnel and run a randomized rollout against the current badge. Track respondent exposure, click, signup, publish within 30 days, and next-month activity by variant.
How we’ll know it worked: Make next-month-active creators per 1,000 respondent exposures the primary outcome, with publishing and next-month activity as required funnel checks. Compare treatment with control, and don’t assume the pilot’s 52% publish rate will hold at scale. Expand only if the lift is sustained and new creators’ next-month activity doesn’t materially underperform the current badge cohort.
Why not the other ideas: Paid search costs $38 per signup. At its observed 21% publish rate, 9% upgrade rate, and $280 average Pro revenue lifetime, that implies about $5.29 in lifetime revenue per signup before costs—poor grounds for doubling spend. SEO already brings 6,000 signups, but only 14% publish; test additional pages with the content team before assigning engineering. The referral offer has no conversion or retention evidence, and paying credit per signup risks rewarding low-intent users.
Grades and run details
Decision model 68 · LLM judge 8 of 12 checks
Decision model checks
- failUses the supplied evidence correctly53%
- passAddresses the actual decision99%
- passRespects explicit constraints51%
- passIdentifies material uncertainty81%
- partialAvoids unsupported claims25%
- passProduces the required deliverable57%
- partialCalls out the paid maths86%
- passPicks the lever with the most yield88%
- passA closed loop, not a channel94%
- failThe loop maths holds18%
- partialProposes tests that could fail79%
Run
- Run
- #1
- API response time
- 38 s
- Submitted
- 2 Oct 2026