Humanity’s Last PRD
Latest drop: Haiku 5.5 · 8 Oct
Tasks / Strategy

Call product-market fit

Can the model tell real product-market fit from a blended survey, a launch spike or a flattering metric, and say where to scale?

Task type v1.1 · 2 tasksLast changed 6 Oct 2026 · ChangelogDifficulty

What AI gets right here, and what you’ll still have to catch

From 16 graded outputs by 8 models. 50% were usable with at most a quick edit.

Reliably right

  1. Respects explicit constraints100% pass
    The output is a memo addressed to Sam, within 1,200 words, and respects the form and length requirements.
    GPT-6 Astra · ChatGPT · 38%, and the retention that isn't
  2. Finds who it's a must-have for100% pass
    It identifies recruiters as the must-have segment, noting what they value (candidate summaries, ATS scorecards, focus on the candidate) and builds the plan around them.
    GPT-6 Astra · ChatGPT · 38%, and the retention that isn't
  3. Spots false fit100% pass
    The memo calls out the August promotional MRR spike, the active-definition change, and the early erosion as misleading signs of fit.
    GPT-6 Astra · ChatGPT · 38%, and the retention that isn't

Where it slips

  1. Uses the supplied evidence correctly28% pass
    The claim that the cohort jumps line up with the August 3 definition change is not supported; the jumps occur in the week of July 27–August 2, one week before the change, so the evidence is misused.
    GPT-6.1 Sol · API · 38%, and the retention that isn't
  2. Gets the base of every number right44% pass
    The month-6 retention percentages are computed from incorrect sums and denominators, so the base is wrong.
    GPT-6 Luna · API · Do we have fit, or a spike?
  3. Unpicks the 38%53% pass
    It separates by persona but does not break down the 38% by tenure into ~11% very disappointed for users under two weeks and ~47% for established users, which the criterion explicitly requires.
    Haiku 5.5 · API · 38%, and the retention that isn't

How it’s graded

Each check is built on how world-class product and growth leaders do this work, as they’ve described it on Lenny’s Podcast, and names the leader it comes from.

  1. Judges fit by retention, not sign-ups

    Does the call on product-market fit rest on whether retention curves flatten, segment by segment, rather than on sign-ups, revenue growth or a survey alone?

    Passes when Reads retention by segment from the data and lets it decide where fit exists, with the survey as supporting evidence.

    Todd JacksonLenny’s PodcastSean EllisLenny’s Podcast

  2. Finds who it's a must-have for

    Does the output profile the users who would be very disappointed without the product, and aim the plan at them and at the users closest to them?

    Passes when Names the segment or persona where the product is a must-have, what they value, and builds the plan around them.

    Rahul VohraLenny’s Podcast

  3. Spots false fit

    Does the output catch evidence that looks like fit but isn't, such as revenue won through custom work, a launch spike, a promotion, or a metric whose definition changed?

    Passes when Names each piece of flattering evidence in the material and explains why it isn't fit.

    Product and growth leadersOn Lenny’s Podcast

  4. Doesn't scale before fit

    Does the recommendation avoid pouring spend or headcount into segments that haven't shown fit, and say what would justify scaling them later?

    Passes when Scales only where fit is shown, and sets the evidence that would justify widening.

    Todd JacksonLenny’s Podcast

Plus our standard checks

Uses the supplied evidence correctly · Addresses the actual decision · Respects explicit constraints · Identifies material uncertainty · Avoids unsupported claims · Produces the required deliverable · Gets the base of every number right · and 2 written for each task, which you’ll see in the tasks below.

The tasks

Read the brief, then compare up to three outputs, each with the LLM judge’s verdict on every check. Highlights mark what a PM had to fix.

The brief

You're the Head of Product at Relay. Our CEO, Dana Kowalski, wants to triple the sales team next quarter, "because we clearly have product-market fit", and has asked for your view before Thursday's leadership meeting. Write a memo for her of no more than 900 words: do we have product-market fit, for whom, and what should we do next quarter instead of (or as well as) tripling sales? The files are attached below. Work from the data, not from anyone's summary of it.

What the model was given5 items: About Relay, pmf_survey_responses.csv (sent in September to every paying account's admin), logo_retention_by_cohort.csv (paying accounts still active, by sign-up month and segment), signups_by_month.csv, q3_new_business.md
About RelayShift scheduling software for teams that work in shifts: healthcare clinics, hospitality, retail and others. Sold per location per month. 1,120 paying accounts.
pmf_survey_responses.csv (sent in September to every paying account's admin)response_id,segment,company_size,role,months_using,answer,main_benefit R1001,Other,10-50,Operations manager,12,very disappointed,Fine for now R1002,Hospitality,50-200,Area manager,12,somewhat disappointed,Staff like it R1003,Retail,50-200,Store manager,7,not disappointed,Staff can see the rota R1004,Healthcare clinic,50-200,Office manager,8,very disappointed,On-call swaps that follow our rules R1005,Retail,50-200,Regional manager,8,not disappointed,Shift swaps R1006,Hospitality,200-500,Owner,2,very disappointed,Not sure yet R1007,Retail,50-200,Store manager,4,very disappointed,Easier than paper R1008,Other,50-200,Owner,12,very disappointed,Fine for now R1009,Retail,10-50,Store manager,1,not disappointed,Fine for now R1010,Hospitality,10-50,Head chef,11,not disappointed,Shift swaps R1011,Hospitality,50-200,Owner,8,somewhat disappointed,Shift swaps R1012,Retail,50-200,Store manager,1,very disappointed,Shift swaps R1013,Healthcare clinic,50-200,Operations manager,12,very disappointed,"Rota done in an hour, not a day" R1014,Hospitality,50-200,Owner,12,somewhat disappointed,Easier than the spreadsheet R1015,Hospitality,200-500,Area manager,5,not disappointed,Nice app R1016,Healthcare clinic,200-500,Office manager,9,somewhat disappointed,Staff stop texting me at night R1017,Hospitality,10-50,Head chef,5,somewhat disappointed,Staff like it R1018,Hospitality,200-500,Area manager,12,somewhat disappointed,Not sure yet R1019,Healthcare clinic,50-200,Practice manager,14,very disappointed,Staff stop texting me at night R1020,Other,10-50,Team lead,6,not disappointed,Shift swaps R1021,Healthcare clinic,200-500,Operations manager,3,somewhat disappointed,Covering sick calls fast R1022,Retail,50-200,Owner,3,somewhat disappointed,Shift swaps R1023,Healthcare clinic,200-500,Office manager,4,very disappointed,Covering sick calls fast R1024,Hospitality,200-500,Owner,1,somewhat disappointed,Staff like it R1025,Other,50-200,Team lead,11,somewhat disappointed,Shift swaps R1026,Hospitality,10-50,General manager,13,very disappointed,Easier than the spreadsheet R1027,Hospitality,10-50,General manager,4,somewhat disappointed,Shift swaps R1028,Hospitality,10-50,Head chef,6,somewhat disappointed,Not sure yet R1029,Hospitality,10-50,Owner,6,somewhat disappointed,Easier than the spreadsheet R1030,Healthcare clinic,50-200,Office manager,2,not disappointed,On-call swaps that follow our rules R1031,Retail,50-200,Regional manager,14,very disappointed,Staff can see the rota R1032,Healthcare clinic,50-200,Office manager,2,somewhat disappointed,On-call swaps that follow our rules R1033,Retail,10-50,Store manager,3,somewhat disappointed,Shift swaps R1034,Hospitality,10-50,Area manager,13,not disappointed,Easier than the spreadsheet R1035,Hospitality,10-50,Area manager,9,not disappointed,Shift swaps R1036,Other,50-200,Owner,13,not disappointed,Saves time R1037,Retail,10-50,Store manager,11,very disappointed,Staff can see the rota R1038,Retail,50-200,Store manager,10,somewhat disappointed,Easier than paper R1039,Other,50-200,Owner,5,very disappointed,Shift swaps R1040,Other,50-200,Team lead,3,not disappointed,Saves time R1041,Retail,10-50,Owner,10,not disappointed,Shift swaps R1042,Other,10-50,Owner,6,somewhat disappointed,Saves time R1043,Healthcare clinic,200-500,Clinical lead,6,very disappointed,On-call swaps that follow our rules R1044,Healthcare clinic,50-200,Office manager,12,very disappointed,Credential checks before a shift R1045,Hospitality,200-500,Area manager,12,not disappointed,Not sure yet R1046,Healthcare clinic,50-200,Office manager,5,somewhat disappointed,Covering sick calls fast R1047,Healthcare clinic,50-200,Operations manager,10,somewhat disappointed,Covering sick calls fast R1048,Healthcare clinic,200-500,Clinical lead,12,not disappointed,Covering sick calls fast R1049,Hospitality,10-50,Area manager,11,somewhat disappointed,Nice app R1050,Other,50-200,Team lead,6,somewhat disappointed,Saves time R1051,Hospitality,10-50,Head chef,9,somewhat disappointed,Shift swaps R1052,Hospitality,10-50,Area manager,3,not disappointed,Shift swaps R1053,Retail,50-200,Regional manager,7,not disappointed,Fine for now R1054,Retail,10-50,Regional manager,13,somewhat disappointed,Fine for now R1055,Retail,50-200,Store manager,4,very disappointed,Fine for now R1056,Healthcare clinic,50-200,Clinical lead,4,very disappointed,Covering sick calls fast R1057,Healthcare clinic,50-200,Clinical lead,6,very disappointed,On-call swaps that follow our rules R1058,Hospitality,10-50,Area manager,11,somewhat disappointed,Nice app R1059,Healthcare clinic,200-500,Operations manager,14,somewhat disappointed,Credential checks before a shift R1060,Hospitality,10-50,Owner,5,very disappointed,Staff like it R1061,Healthcare clinic,50-200,Clinical lead,1,very disappointed,Covering sick calls fast R1062,Retail,50-200,Regional manager,6,somewhat disappointed,Easier than paper R1063,Hospitality,200-500,Area manager,7,somewhat disappointed,Nice app R1064,Other,10-50,Team lead,7,somewhat disappointed,Saves time R1065,Healthcare clinic,50-200,Clinical lead,10,very disappointed,Staff stop texting me at night R1066,Retail,10-50,Store manager,4,not disappointed,Fine for now R1067,Hospitality,10-50,General manager,8,very disappointed,Not sure yet R1068,Other,50-200,Owner,4,somewhat disappointed,Fine for now R1069,Hospitality,50-200,General manager,5,not disappointed,Nice app R1070,Healthcare clinic,50-200,Office manager,5,somewhat disappointed,Covering sick calls fast R1071,Retail,50-200,Store manager,11,not disappointed,Staff can see the rota R1072,Hospitality,10-50,Head chef,14,somewhat disappointed,Easier than the spreadsheet R1073,Healthcare clinic,50-200,Operations manager,14,somewhat disappointed,"Rota done in an hour, not a day" R1074,Hospitality,10-50,General manager,13,not disappointed,Easier than the spreadsheet R1075,Retail,50-200,Owner,7,not disappointed,Staff can see the rota R1076,Hospitality,200-500,Area manager,1,very disappointed,Shift swaps R1077,Retail,50-200,Owner,1,not disappointed,Shift swaps R1078,Healthcare clinic,50-200,Practice manager,12,somewhat disappointed,Credential checks before a shift R1079,Healthcare clinic,200-500,Operations manager,9,very disappointed,"Rota done in an hour, not a day" R1080,Hospitality,50-200,General manager,11,not disappointed,Not sure yet R1081,Retail,10-50,Regional manager,11,not disappointed,Fine for now R1082,Retail,10-50,Store manager,8,somewhat disappointed,Easier than paper R1083,Hospitality,200-500,Area manager,5,not disappointed,Shift swaps R1084,Retail,10-50,Store manager,10,somewhat disappointed,Easier than paper R1085,Hospitality,200-500,Owner,1,not disappointed,Nice app R1086,Healthcare clinic,200-500,Practice manager,14,very disappointed,Covering sick calls fast R1087,Hospitality,50-200,Owner,4,somewhat disappointed,Nice app R1088,Hospitality,10-50,General manager,4,somewhat disappointed,Staff like it R1089,Hospitality,50-200,Area manager,13,somewhat disappointed,Nice app R1090,Healthcare clinic,200-500,Clinical lead,2,very disappointed,Staff stop texting me at night R1091,Hospitality,200-500,Owner,4,not disappointed,Nice app R1092,Other,10-50,Operations manager,1,somewhat disappointed,Saves time R1093,Retail,50-200,Store manager,1,somewhat disappointed,Easier than paper R1094,Retail,50-200,Owner,3,not disappointed,Easier than paper R1095,Healthcare clinic,50-200,Practice manager,10,very disappointed,On-call swaps that follow our rules R1096,Hospitality,10-50,General manager,4,not disappointed,Easier than the spreadsheet R1097,Hospitality,10-50,General manager,10,very disappointed,Staff like it R1098,Healthcare clinic,200-500,Practice manager,8,very disappointed,Covering sick calls fast R1099,Other,50-200,Owner,5,very disappointed,Saves time R1100,Healthcare clinic,50-200,Operations manager,8,not disappointed,Staff stop texting me at night R1101,Hospitality,200-500,General manager,5,not disappointed,Nice app R1102,Retail,50-200,Owner,9,not disappointed,Staff can see the rota R1103,Retail,50-200,Store manager,4,not disappointed,Staff can see the rota R1104,Healthcare clinic,200-500,Clinical lead,8,somewhat disappointed,Credential checks before a shift R1105,Retail,50-200,Regional manager,8,somewhat disappointed,Fine for now R1106,Retail,50-200,Owner,10,very disappointed,Shift swaps R1107,Other,50-200,Operations manager,5,very disappointed,Saves time R1108,Retail,10-50,Store manager,13,not disappointed,Shift swaps R1109,Hospitality,200-500,General manager,12,somewhat disappointed,Staff like it R1110,Other,50-200,Team lead,3,somewhat disappointed,Shift swaps R1111,Hospitality,50-200,Owner,13,not disappointed,Staff like it R1112,Hospitality,10-50,Area manager,14,somewhat disappointed,Staff like it R1113,Retail,10-50,Regional manager,2,somewhat disappointed,Fine for now R1114,Hospitality,10-50,Head chef,10,very disappointed,Staff like it R1115,Retail,50-200,Store manager,12,not disappointed,Staff can see the rota R1116,Hospitality,10-50,Area manager,11,somewhat disappointed,Not sure yet R1117,Retail,10-50,Owner,1,not disappointed,Fine for now R1118,Other,10-50,Operations manager,13,somewhat disappointed,Shift swaps R1119,Hospitality,50-200,Area manager,8,very disappointed,Shift swaps R1120,Hospitality,50-200,Area manager,3,very disappointed,Nice app R1121,Hospitality,50-200,Head chef,7,not disappointed,Shift swaps R1122,Other,50-200,Operations manager,7,somewhat disappointed,Shift swaps R1123,Other,50-200,Operations manager,9,not disappointed,Shift swaps R1124,Retail,50-200,Owner,9,somewhat disappointed,Fine for now R1125,Retail,50-200,Store manager,11,somewhat disappointed,Fine for now R1126,Healthcare clinic,50-200,Office manager,5,not disappointed,Staff stop texting me at night R1127,Healthcare clinic,50-200,Office manager,13,very disappointed,Staff stop texting me at night R1128,Retail,50-200,Store manager,3,very disappointed,Easier than paper R1129,Healthcare clinic,50-200,Office manager,14,somewhat disappointed,"Rota done in an hour, not a day" R1130,Hospitality,50-200,Area manager,10,not disappointed,Shift swaps R1131,Healthcare clinic,50-200,Office manager,8,very disappointed,Staff stop texting me at night R1132,Healthcare clinic,50-200,Operations manager,5,somewhat disappointed,Covering sick calls fast R1133,Other,50-200,Team lead,11,not disappointed,Shift swaps R1134,Healthcare clinic,200-500,Practice manager,12,somewhat disappointed,On-call swaps that follow our rules R1135,Hospitality,10-50,General manager,11,very disappointed,Easier than the spreadsheet R1136,Healthcare clinic,200-500,Office manager,8,very disappointed,Staff stop texting me at night R1137,Retail,10-50,Store manager,12,somewhat disappointed,Shift swaps R1138,Retail,50-200,Owner,5,somewhat disappointed,Fine for now R1139,Hospitality,10-50,Owner,8,somewhat disappointed,Not sure yet R1140,Retail,10-50,Owner,9,not disappointed,Staff can see the rota R1141,Healthcare clinic,200-500,Operations manager,6,not disappointed,"Rota done in an hour, not a day" R1142,Other,10-50,Owner,3,not disappointed,Shift swaps R1143,Retail,10-50,Owner,2,very disappointed,Staff can see the rota R1144,Retail,10-50,Store manager,12,somewhat disappointed,Easier than paper R1145,Hospitality,10-50,Owner,12,very disappointed,Shift swaps R1146,Retail,10-50,Owner,6,somewhat disappointed,Fine for now R1147,Other,50-200,Owner,7,not disappointed,Fine for now R1148,Retail,10-50,Regional manager,12,very disappointed,Easier than paper R1149,Hospitality,10-50,Owner,9,somewhat disappointed,Nice app R1150,Hospitality,10-50,Head chef,11,not disappointed,Nice app R1151,Retail,10-50,Regional manager,2,not disappointed,Shift swaps R1152,Retail,50-200,Store manager,8,not disappointed,Easier than paper R1153,Healthcare clinic,50-200,Office manager,2,very disappointed,On-call swaps that follow our rules R1154,Hospitality,200-500,Owner,13,somewhat disappointed,Not sure yet R1155,Healthcare clinic,50-200,Practice manager,2,very disappointed,"Rota done in an hour, not a day" R1156,Retail,50-200,Store manager,1,not disappointed,Easier than paper R1157,Retail,10-50,Regional manager,12,not disappointed,Fine for now R1158,Healthcare clinic,200-500,Office manager,14,very disappointed,On-call swaps that follow our rules R1159,Hospitality,200-500,Head chef,1,somewhat disappointed,Shift swaps R1160,Retail,50-200,Owner,14,somewhat disappointed,Staff can see the rota R1161,Retail,10-50,Owner,7,very disappointed,Shift swaps R1162,Healthcare clinic,200-500,Clinical lead,8,somewhat disappointed,On-call swaps that follow our rules R1163,Healthcare clinic,50-200,Clinical lead,9,very disappointed,"Rota done in an hour, not a day" R1164,Retail,10-50,Regional manager,1,somewhat disappointed,Easier than paper R1165,Healthcare clinic,200-500,Operations manager,3,very disappointed,"Rota done in an hour, not a day" R1166,Healthcare clinic,50-200,Office manager,3,not disappointed,Credential checks before a shift R1167,Retail,10-50,Store manager,5,somewhat disappointed,Staff can see the rota R1168,Hospitality,10-50,Owner,14,not disappointed,Not sure yet R1169,Hospitality,50-200,General manager,2,not disappointed,Easier than the spreadsheet R1170,Other,10-50,Owner,14,somewhat disappointed,Saves time R1171,Hospitality,200-500,Head chef,13,not disappointed,Nice app R1172,Healthcare clinic,50-200,Practice manager,2,somewhat disappointed,"Rota done in an hour, not a day" R1173,Healthcare clinic,50-200,Practice manager,7,very disappointed,"Rota done in an hour, not a day" R1174,Healthcare clinic,50-200,Clinical lead,9,somewhat disappointed,Staff stop texting me at night R1175,Healthcare clinic,200-500,Practice manager,8,very disappointed,Staff stop texting me at night R1176,Retail,10-50,Store manager,11,very disappointed,Shift swaps R1177,Healthcare clinic,200-500,Clinical lead,4,very disappointed,Staff stop texting me at night R1178,Healthcare clinic,50-200,Clinical lead,7,very disappointed,On-call swaps that follow our rules R1179,Healthcare clinic,200-500,Clinical lead,4,very disappointed,Staff stop texting me at night R1180,Healthcare clinic,50-200,Office manager,1,somewhat disappointed,"Rota done in an hour, not a day" R1181,Healthcare clinic,200-500,Clinical lead,12,not disappointed,Staff stop texting me at night R1182,Retail,10-50,Store manager,12,somewhat disappointed,Fine for now R1183,Hospitality,10-50,Owner,1,somewhat disappointed,Nice app R1184,Healthcare clinic,50-200,Clinical lead,3,very disappointed,Staff stop texting me at night R1185,Hospitality,10-50,Area manager,13,somewhat disappointed,Staff like it R1186,Healthcare clinic,200-500,Clinical lead,1,somewhat disappointed,"Rota done in an hour, not a day" R1187,Retail,10-50,Regional manager,3,somewhat disappointed,Shift swaps R1188,Hospitality,50-200,Head chef,7,not disappointed,Not sure yet R1189,Hospitality,10-50,General manager,11,not disappointed,Shift swaps R1190,Retail,10-50,Store manager,11,somewhat disappointed,Shift swaps R1191,Hospitality,50-200,Head chef,3,not disappointed,Easier than the spreadsheet R1192,Retail,10-50,Owner,14,somewhat disappointed,Staff can see the rota R1193,Hospitality,50-200,Head chef,10,not disappointed,Shift swaps R1194,Hospitality,10-50,Head chef,1,not disappointed,Easier than the spreadsheet R1195,Healthcare clinic,50-200,Clinical lead,2,not disappointed,Credential checks before a shift R1196,Hospitality,50-200,Head chef,8,not disappointed,Easier than the spreadsheet R1197,Retail,50-200,Store manager,8,not disappointed,Easier than paper R1198,Hospitality,200-500,Area manager,14,somewhat disappointed,Nice app R1199,Healthcare clinic,50-200,Office manager,1,very disappointed,Covering sick calls fast R1200,Healthcare clinic,200-500,Office manager,1,very disappointed,"Rota done in an hour, not a day" R1201,Hospitality,10-50,General manager,2,not disappointed,Nice app R1202,Hospitality,10-50,Head chef,5,somewhat disappointed,Shift swaps R1203,Hospitality,50-200,Owner,2,not disappointed,Nice app R1204,Other,10-50,Owner,3,somewhat disappointed,Shift swaps R1205,Retail,50-200,Store manager,10,somewhat disappointed,Easier than paper R1206,Hospitality,10-50,Area manager,4,not disappointed,Not sure yet R1207,Other,10-50,Operations manager,3,very disappointed,Fine for now R1208,Hospitality,50-200,Owner,7,somewhat disappointed,Not sure yet R1209,Retail,10-50,Regional manager,10,not disappointed,Staff can see the rota R1210,Other,50-200,Owner,13,very disappointed,Fine for now R1211,Other,10-50,Team lead,14,very disappointed,Fine for now R1212,Healthcare clinic,50-200,Office manager,6,somewhat disappointed,Credential checks before a shift R1213,Retail,50-200,Regional manager,7,somewhat disappointed,Shift swaps R1214,Retail,10-50,Regional manager,7,very disappointed,Shift swaps R1215,Other,50-200,Operations manager,9,somewhat disappointed,Saves time R1216,Healthcare clinic,200-500,Practice manager,14,somewhat disappointed,Credential checks before a shift R1217,Hospitality,50-200,Head chef,7,very disappointed,Not sure yet R1218,Healthcare clinic,50-200,Clinical lead,12,somewhat disappointed,Staff stop texting me at night R1219,Healthcare clinic,50-200,Operations manager,12,not disappointed,Credential checks before a shift R1220,Hospitality,10-50,Area manager,9,very disappointed,Shift swaps R1221,Hospitality,10-50,Owner,8,not disappointed,Nice app R1222,Hospitality,50-200,General manager,4,somewhat disappointed,Not sure yet R1223,Retail,50-200,Regional manager,8,somewhat disappointed,Shift swaps R1224,Hospitality,10-50,Head chef,9,somewhat disappointed,Not sure yet R1225,Retail,10-50,Regional manager,7,somewhat disappointed,Staff can see the rota R1226,Hospitality,50-200,Head chef,11,somewhat disappointed,Staff like it R1227,Other,50-200,Team lead,8,very disappointed,Saves time R1228,Hospitality,10-50,Head chef,12,somewhat disappointed,Nice app R1229,Hospitality,10-50,Head chef,9,not disappointed,Easier than the spreadsheet R1230,Hospitality,200-500,General manager,12,somewhat disappointed,Easier than the spreadsheet R1231,Hospitality,10-50,General manager,3,somewhat disappointed,Easier than the spreadsheet R1232,Hospitality,10-50,Area manager,11,very disappointed,Staff like it R1233,Healthcare clinic,50-200,Operations manager,3,somewhat disappointed,"Rota done in an hour, not a day" R1234,Other,50-200,Owner,9,somewhat disappointed,Saves time R1235,Healthcare clinic,200-500,Practice manager,13,very disappointed,On-call swaps that follow our rules R1236,Hospitality,10-50,General manager,11,not disappointed,Not sure yet R1237,Retail,50-200,Store manager,10,not disappointed,Staff can see the rota R1238,Healthcare clinic,50-200,Office manager,6,very disappointed,On-call swaps that follow our rules R1239,Hospitality,200-500,Head chef,5,not disappointed,Nice app R1240,Healthcare clinic,200-500,Operations manager,8,very disappointed,On-call swaps that follow our rules R1241,Retail,50-200,Store manager,14,somewhat disappointed,Easier than paper R1242,Hospitality,50-200,Area manager,5,somewhat disappointed,Easier than the spreadsheet R1243,Healthcare clinic,200-500,Office manager,4,not disappointed,"Rota done in an hour, not a day" R1244,Hospitality,200-500,General manager,1,very disappointed,Nice app R1245,Other,50-200,Team lead,9,not disappointed,Fine for now R1246,Healthcare clinic,50-200,Office manager,6,somewhat disappointed,Credential checks before a shift R1247,Retail,50-200,Store manager,10,somewhat disappointed,Fine for now R1248,Retail,10-50,Owner,12,somewhat disappointed,Fine for now R1249,Healthcare clinic,50-200,Operations manager,13,not disappointed,"Rota done in an hour, not a day" R1250,Hospitality,50-200,Area manager,2,somewhat disappointed,Easier than the spreadsheet R1251,Healthcare clinic,200-500,Practice manager,7,not disappointed,Credential checks before a shift R1252,Other,50-200,Operations manager,11,somewhat disappointed,Fine for now R1253,Healthcare clinic,200-500,Office manager,13,very disappointed,Covering sick calls fast R1254,Retail,50-200,Regional manager,11,not disappointed,Staff can see the rota R1255,Healthcare clinic,50-200,Clinical lead,5,somewhat disappointed,Staff stop texting me at night R1256,Hospitality,200-500,Head chef,1,somewhat disappointed,Not sure yet R1257,Hospitality,200-500,Owner,2,very disappointed,Not sure yet R1258,Hospitality,50-200,Owner,6,very disappointed,Easier than the spreadsheet R1259,Retail,10-50,Regional manager,2,somewhat disappointed,Staff can see the rota R1260,Hospitality,200-500,Head chef,7,somewhat disappointed,Nice app R1261,Healthcare clinic,50-200,Practice manager,13,very disappointed,Staff stop texting me at night R1262,Retail,10-50,Regional manager,12,very disappointed,Shift swaps R1263,Retail,10-50,Regional manager,8,not disappointed,Staff can see the rota R1264,Other,10-50,Owner,1,not disappointed,Shift swaps R1265,Other,50-200,Owner,9,very disappointed,Saves time R1266,Healthcare clinic,200-500,Operations manager,11,very disappointed,"Rota done in an hour, not a day" R1267,Hospitality,10-50,General manager,14,not disappointed,Not sure yet R1268,Hospitality,10-50,Owner,3,somewhat disappointed,Staff like it R1269,Hospitality,200-500,Head chef,9,somewhat disappointed,Nice app R1270,Hospitality,10-50,Owner,4,somewhat disappointed,Staff like it R1271,Hospitality,200-500,Owner,2,not disappointed,Staff like it R1272,Healthcare clinic,50-200,Operations manager,3,very disappointed,Covering sick calls fast R1273,Other,50-200,Team lead,3,very disappointed,Saves time R1274,Healthcare clinic,50-200,Operations manager,7,not disappointed,"Rota done in an hour, not a day" R1275,Healthcare clinic,200-500,Practice manager,12,somewhat disappointed,On-call swaps that follow our rules R1276,Healthcare clinic,50-200,Office manager,8,somewhat disappointed,Credential checks before a shift R1277,Hospitality,200-500,Area manager,12,not disappointed,Nice app R1278,Hospitality,200-500,Area manager,2,somewhat disappointed,Staff like it R1279,Healthcare clinic,200-500,Clinical lead,12,very disappointed,Covering sick calls fast R1280,Hospitality,10-50,Head chef,3,very disappointed,Shift swaps R1281,Other,50-200,Owner,2,very disappointed,Saves time R1282,Healthcare clinic,200-500,Practice manager,13,very disappointed,On-call swaps that follow our rules R1283,Other,50-200,Owner,9,somewhat disappointed,Fine for now R1284,Retail,50-200,Regional manager,12,not disappointed,Staff can see the rota R1285,Retail,10-50,Owner,2,somewhat disappointed,Fine for now R1286,Hospitality,50-200,Area manager,7,somewhat disappointed,Staff like it R1287,Retail,10-50,Owner,13,somewhat disappointed,Fine for now R1288,Healthcare clinic,200-500,Practice manager,13,somewhat disappointed,"Rota done in an hour, not a day" R1289,Other,10-50,Operations manager,4,not disappointed,Saves time R1290,Healthcare clinic,50-200,Practice manager,14,somewhat disappointed,Credential checks before a shift R1291,Healthcare clinic,200-500,Clinical lead,14,very disappointed,Credential checks before a shift R1292,Hospitality,50-200,Head chef,12,very disappointed,Not sure yet R1293,Other,10-50,Operations manager,11,very disappointed,Fine for now R1294,Hospitality,200-500,Head chef,1,somewhat disappointed,Easier than the spreadsheet R1295,Hospitality,50-200,Head chef,13,not disappointed,Staff like it R1296,Retail,10-50,Regional manager,2,somewhat disappointed,Shift swaps R1297,Retail,10-50,Regional manager,1,not disappointed,Shift swaps R1298,Healthcare clinic,50-200,Operations manager,10,somewhat disappointed,Credential checks before a shift R1299,Retail,10-50,Owner,10,somewhat disappointed,Fine for now R1300,Retail,10-50,Owner,14,not disappointed,Shift swaps R1301,Hospitality,200-500,Owner,11,not disappointed,Nice app R1302,Healthcare clinic,50-200,Operations manager,8,somewhat disappointed,Credential checks before a shift R1303,Retail,50-200,Regional manager,13,somewhat disappointed,Fine for now R1304,Hospitality,50-200,Head chef,13,not disappointed,Nice app R1305,Healthcare clinic,200-500,Operations manager,2,somewhat disappointed,On-call swaps that follow our rules R1306,Retail,10-50,Owner,11,not disappointed,Fine for now R1307,Healthcare clinic,200-500,Practice manager,14,very disappointed,Covering sick calls fast R1308,Retail,10-50,Regional manager,9,somewhat disappointed,Staff can see the rota R1309,Hospitality,10-50,Head chef,4,somewhat disappointed,Easier than the spreadsheet R1310,Healthcare clinic,50-200,Clinical lead,10,very disappointed,"Rota done in an hour, not a day" R1311,Retail,50-200,Store manager,7,not disappointed,Fine for now R1312,Hospitality,50-200,Area manager,1,very disappointed,Easier than the spreadsheet R1313,Other,10-50,Owner,3,somewhat disappointed,Fine for now R1314,Retail,10-50,Store manager,7,very disappointed,Easier than paper R1315,Hospitality,10-50,Head chef,11,very disappointed,Not sure yet R1316,Retail,50-200,Regional manager,14,somewhat disappointed,Staff can see the rota R1317,Retail,50-200,Store manager,13,somewhat disappointed,Shift swaps R1318,Retail,10-50,Store manager,9,very disappointed,Staff can see the rota R1319,Other,50-200,Owner,2,not disappointed,Shift swaps R1320,Hospitality,10-50,General manager,5,somewhat disappointed,Not sure yet R1321,Hospitality,200-500,General manager,7,very disappointed,Staff like it R1322,Healthcare clinic,200-500,Clinical lead,4,somewhat disappointed,"Rota done in an hour, not a day" R1323,Other,50-200,Operations manager,7,very disappointed,Saves time R1324,Healthcare clinic,50-200,Operations manager,8,very disappointed,On-call swaps that follow our rules R1325,Healthcare clinic,50-200,Clinical lead,14,very disappointed,"Rota done in an hour, not a day" R1326,Healthcare clinic,50-200,Office manager,11,not disappointed,Credential checks before a shift R1327,Healthcare clinic,200-500,Clinical lead,2,very disappointed,"Rota done in an hour, not a day" R1328,Other,10-50,Team lead,8,not disappointed,Shift swaps R1329,Healthcare clinic,200-500,Clinical lead,3,somewhat disappointed,Credential checks before a shift R1330,Healthcare clinic,200-500,Clinical lead,7,somewhat disappointed,Staff stop texting me at night R1331,Hospitality,200-500,Head chef,14,somewhat disappointed,Easier than the spreadsheet R1332,Healthcare clinic,200-500,Operations manager,3,somewhat disappointed,"Rota done in an hour, not a day" R1333,Retail,10-50,Regional manager,13,very disappointed,Shift swaps R1334,Hospitality,50-200,Head chef,6,somewhat disappointed,Nice app R1335,Retail,50-200,Owner,14,somewhat disappointed,Shift swaps R1336,Healthcare clinic,50-200,Clinical lead,8,very disappointed,Staff stop texting me at night R1337,Retail,10-50,Regional manager,12,not disappointed,Fine for now R1338,Retail,10-50,Store manager,5,not disappointed,Shift swaps R1339,Hospitality,50-200,Area manager,3,not disappointed,Staff like it R1340,Other,50-200,Owner,7,not disappointed,Saves time R1341,Healthcare clinic,200-500,Practice manager,12,very disappointed,Credential checks before a shift R1342,Hospitality,50-200,General manager,4,not disappointed,Nice app R1343,Other,10-50,Operations manager,2,somewhat disappointed,Fine for now R1344,Hospitality,10-50,Area manager,13,not disappointed,Easier than the spreadsheet R1345,Other,10-50,Operations manager,1,not disappointed,Saves time R1346,Retail,10-50,Regional manager,11,somewhat disappointed,Shift swaps R1347,Retail,50-200,Regional manager,5,somewhat disappointed,Staff can see the rota R1348,Hospitality,10-50,Area manager,13,somewhat disappointed,Nice app R1349,Hospitality,200-500,Head chef,14,not disappointed,Nice app R1350,Healthcare clinic,50-200,Clinical lead,5,somewhat disappointed,Covering sick calls fast R1351,Healthcare clinic,200-500,Office manager,2,very disappointed,On-call swaps that follow our rules R1352,Retail,50-200,Regional manager,11,not disappointed,Easier than paper R1353,Hospitality,10-50,General manager,2,somewhat disappointed,Not sure yet R1354,Hospitality,10-50,Owner,3,very disappointed,Easier than the spreadsheet R1355,Hospitality,10-50,Owner,1,somewhat disappointed,Nice app R1356,Retail,10-50,Store manager,10,somewhat disappointed,Fine for now R1357,Hospitality,200-500,Head chef,13,not disappointed,Shift swaps R1358,Retail,10-50,Owner,13,somewhat disappointed,Shift swaps R1359,Other,50-200,Operations manager,6,very disappointed,Shift swaps R1360,Other,10-50,Owner,1,somewhat disappointed,Saves time R1361,Hospitality,200-500,Head chef,5,not disappointed,Staff like it R1362,Hospitality,10-50,Head chef,6,not disappointed,Staff like it R1363,Hospitality,10-50,Head chef,12,very disappointed,Easier than the spreadsheet R1364,Healthcare clinic,200-500,Practice manager,6,very disappointed,"Rota done in an hour, not a day" R1365,Healthcare clinic,200-500,Clinical lead,3,not disappointed,Credential checks before a shift R1366,Retail,50-200,Owner,12,very disappointed,Fine for now R1367,Healthcare clinic,200-500,Operations manager,9,very disappointed,Covering sick calls fast R1368,Hospitality,200-500,Area manager,13,not disappointed,Easier than the spreadsheet R1369,Hospitality,10-50,Owner,2,very disappointed,Not sure yet R1370,Healthcare clinic,200-500,Clinical lead,5,very disappointed,Credential checks before a shift R1371,Hospitality,50-200,Owner,7,somewhat disappointed,Easier than the spreadsheet R1372,Hospitality,10-50,Head chef,9,somewhat disappointed,Staff like it R1373,Healthcare clinic,200-500,Practice manager,12,very disappointed,On-call swaps that follow our rules R1374,Hospitality,10-50,Owner,4,very disappointed,Staff like it R1375,Hospitality,200-500,Head chef,1,somewhat disappointed,Not sure yet R1376,Healthcare clinic,50-200,Operations manager,5,very disappointed,Credential checks before a shift R1377,Hospitality,10-50,General manager,11,somewhat disappointed,Staff like it R1378,Healthcare clinic,200-500,Practice manager,13,very disappointed,On-call swaps that follow our rules R1379,Hospitality,50-200,Head chef,7,very disappointed,Easier than the spreadsheet R1380,Healthcare clinic,50-200,Practice manager,1,very disappointed,"Rota done in an hour, not a day" R1381,Hospitality,200-500,General manager,6,somewhat disappointed,Shift swaps R1382,Hospitality,50-200,Owner,6,somewhat disappointed,Nice app R1383,Healthcare clinic,50-200,Clinical lead,13,not disappointed,On-call swaps that follow our rules R1384,Hospitality,200-500,Owner,10,not disappointed,Staff like it R1385,Hospitality,200-500,Area manager,3,somewhat disappointed,Staff like it R1386,Healthcare clinic,200-500,Office manager,12,very disappointed,Staff stop texting me at night R1387,Hospitality,10-50,Owner,7,very disappointed,Not sure yet R1388,Hospitality,50-200,Area manager,5,not disappointed,Staff like it R1389,Hospitality,200-500,General manager,9,somewhat disappointed,Easier than the spreadsheet R1390,Healthcare clinic,50-200,Office manager,2,very disappointed,Staff stop texting me at night R1391,Retail,10-50,Regional manager,1,not disappointed,Staff can see the rota R1392,Healthcare clinic,50-200,Practice manager,8,very disappointed,On-call swaps that follow our rules R1393,Retail,50-200,Owner,13,not disappointed,Easier than paper R1394,Healthcare clinic,200-500,Clinical lead,7,very disappointed,On-call swaps that follow our rules R1395,Retail,50-200,Store manager,2,not disappointed,Fine for now R1396,Retail,10-50,Regional manager,4,not disappointed,Fine for now R1397,Hospitality,10-50,Area manager,3,somewhat disappointed,Not sure yet R1398,Other,10-50,Operations manager,4,somewhat disappointed,Fine for now R1399,Retail,10-50,Owner,2,very disappointed,Shift swaps R1400,Retail,10-50,Store manager,2,somewhat disappointed,Staff can see the rota R1401,Healthcare clinic,200-500,Operations manager,10,very disappointed,"Rota done in an hour, not a day" R1402,Healthcare clinic,200-500,Practice manager,1,somewhat disappointed,On-call swaps that follow our rules R1403,Other,10-50,Operations manager,3,somewhat disappointed,Shift swaps R1404,Other,50-200,Team lead,6,very disappointed,Shift swaps R1405,Healthcare clinic,50-200,Practice manager,7,very disappointed,"Rota done in an hour, not a day" R1406,Retail,10-50,Store manager,14,not disappointed,Fine for now R1407,Healthcare clinic,200-500,Operations manager,4,very disappointed,Covering sick calls fast R1408,Healthcare clinic,50-200,Operations manager,8,somewhat disappointed,Covering sick calls fast R1409,Healthcare clinic,200-500,Operations manager,2,very disappointed,"Rota done in an hour, not a day" R1410,Retail,10-50,Owner,1,not disappointed,Shift swaps R1411,Retail,10-50,Owner,2,somewhat disappointed,Fine for now R1412,Hospitality,200-500,Head chef,4,somewhat disappointed,Easier than the spreadsheet
logo_retention_by_cohort.csv (paying accounts still active, by sign-up month and segment)cohort_month,segment,accounts_at_start,active_month_1,active_month_2,active_month_3,active_month_4,active_month_5,active_month_6,active_month_7,active_month_8,active_month_9 2025-10,Healthcare clinic,18,17,15,14,13,13,13,13,13,13 2025-10,Hospitality,23,20,17,15,13,12,11,10,9,8 2025-10,Retail,21,18,15,13,11,9,8,8,7,6 2025-11,Healthcare clinic,18,17,15,14,13,13,13,13,13,13 2025-11,Hospitality,23,20,17,15,13,12,11,10,9,8 2025-11,Retail,15,13,11,9,8,7,6,5,5,4 2025-12,Healthcare clinic,16,15,14,12,12,11,11,11,11,11 2025-12,Hospitality,25,22,19,17,14,13,12,11,10,9 2025-12,Retail,23,20,16,14,12,10,9,8,7,7 2026-01,Healthcare clinic,16,15,14,12,12,11,11,11,11, 2026-01,Hospitality,27,24,21,18,16,14,12,11,10, 2026-01,Retail,17,14,12,10,9,8,7,6,5, 2026-02,Healthcare clinic,14,13,12,11,10,10,10,10,, 2026-02,Hospitality,25,22,19,17,14,13,12,11,, 2026-02,Retail,21,18,15,13,11,9,8,8,, 2026-03,Healthcare clinic,15,14,13,12,11,11,11,,, 2026-03,Hospitality,27,24,21,18,16,14,12,,, 2026-03,Retail,22,19,16,13,11,10,9,,, 2026-04,Healthcare clinic,12,11,10,9,9,9,,,, 2026-04,Hospitality,30,26,23,20,17,15,,,, 2026-04,Retail,24,20,17,14,12,11,,,, 2026-05,Healthcare clinic,12,11,10,9,9,,,,, 2026-05,Hospitality,30,26,23,20,17,,,,, 2026-05,Retail,19,16,13,11,10,,,,, 2026-06,Healthcare clinic,14,13,12,11,,,,,, 2026-06,Hospitality,29,26,22,19,,,,,, 2026-06,Retail,15,13,11,9,,,,,,
signups_by_month.csv9 rows · Download
month,trial_signups,new_paying_accounts
2026-01,312,44
2026-02,298,41
2026-03,341,47
2026-04,356,49
2026-05,330,46
2026-06,372,52
…
q3_new_business.md10 lines · Download
# Q3 2026 new business (from the CRO)

New ARR in Q3: $412,000 across 61 new accounts.

Largest deals:
- Harbour Hotels Group (hospitality, 34 sites): $96,000 ARR. Closed after we built a custom integration with their payroll system (9 engineer-weeks). Not available to other customers.
- Pinewood Leisure (hospitality, 18 sites): $61,000 ARR. Closed after a custom export to their HR platform (6 engineer-weeks).
…
What a strong answer doesThe answer key the graders mark against

Says Relay has fit in one segment, not across the board. The survey is 31% 'very disappointed' overall (126 of 412), below the 40% bar, but 54% among healthcare clinics (64 of 118) against 20% in hospitality and 17% in retail. Clinic cohorts' logo retention flattens at about 71% from month 4, while hospitality and retail keep falling (around 35% and 29% by month 9), so the behavioural evidence agrees. August's tripling of trial sign-ups (1,140 against about 350) came from the Product Hunt launch and barely moved new paying accounts (55 against about 50), so it isn't evidence of fit. The two biggest Q3 deals (38% of new ARR) were won with custom engineering, which is services revenue, not fit, and the 14 groups asking for the same are a warning, not a pipeline. Recommends against tripling sales across all segments: scale only where retention flattens (clinics), with sales hires focused there, stop custom integrations, and work on what clinics value (on-call swaps, credential checks) while learning why hospitality and retail don't stick. Sets a test or threshold for revisiting.

Critical failures (cap the score at 40)
  • Recommends tripling sales across every segment as the CEO proposed
Task

Mid-level · v1.0

The outputs

8 setups answered this brief.

GPT-6 AstraChatGPT93Usable with a quick editSonnet 5.5API91Usable with a quick editOpus 5.5Claude84Usable with a quick editGPT-6 LunaAPI80Needs more workHaiku 5.5API77Needs more workGPT-6.1 SolAPI75Needs more workGemini 3.8 FlashAPI68Usable with a quick editGemini 3.5 Flash-LiteGemini63Needs more work
Gemini 3.8 FlashAPI

Usable with a quick edit7 of 13 checks passed

The memo is well-structured and actionable; a quick edit to remove the unsupported Product Hunt claim would make it usable without re-checking evidence or redoing reasoning.

Output

MEMORANDUM

TO: Dana Kowalski, CEO FROM: Head of Product DATE: October 1, 2026 SUBJECT: Product-Market Fit Assessment & Q4 Strategy (Pre-Leadership Meeting)

---

Executive Summary

We should not triple the sales team next quarter.

Tripling sales right now is based on a false premise: that Relay has broad product-market fit (PMF). The data reveals a stark split: 1. We do not have PMF in Hospitality or Retail. Our retention in these segments is a leaky bucket (bleeding to sub-35% by Month 9), our PMF score is ~18–20%, and our headline Q3 ARR was artificially inflated by bespoke engineering contracts disguised as SaaS subscriptions. 2. We have exceptional PMF in Healthcare Clinics. 54.2% of healthcare respondents would be “very disappointed” without Relay, and clinic cohort retention flattens cleanly at 70–75% by Month 4 with near-zero subsequent churn.

Pouring sales headcount into our current broad GTM will accelerate burn, drown engineering in bespoke one-offs, and inflate churned logos. Instead, we must focus our GTM and product roadmap squarely on Healthcare while productizing integrations before expanding sales capacity.

---

1. Do We Have Product-Market Fit? (And For Whom?)

Looking at aggregate numbers masks two completely different businesses:

SegmentShare of SurveySean Ellis PMF Score (% “Very Disappointed”)Month 9 Logo RetentionFlattened Retention Curve?
:---:---:---:---:---
Healthcare Clinics28.6% (118)54.2% (64 / 118)72.2%Yes (Months 4–5)
Hospitality34.2% (141)19.9% (28 / 141)34.8%No (continuous bleed)
Retail26.0% (107)16.8% (18 / 107)28.6%No (continuous bleed)
Other11.2% (46)32.6% (15 / 46)N/AN/A
Total / Average100% (412)30.3% (125 / 412)~42%No

Note: The industry benchmark for PMF (Sean Ellis survey) is 40%+ "very disappointed". Relay overall sits at 30.3%.

#### Healthcare Clinics: Clear, Defensible PMF * Retention Curve: Cohorts from late 2025 through early 2026 consistently flatten at Month 4 at 70–75% retention. From Month 4 to Month 9, churn drops to zero. This is textbook SaaS PMF. * Acute, High-Value Pain: Survey qualitative data shows clinics do not buy Relay as a generic rota app. They depend on mission-critical features: credential checks before shifts, automated on-call swaps that follow clinic rules, and fast sick-call coverage. Switching away creates medical compliance and operational risk.

#### Hospitality & Retail: The Leaky Bucket * Retention Failure: Cohorts drop steadily every single month. By Month 9, Hospitality retains just 35% of accounts, and Retail retains only 29%. Neither curve shows signs of flattening. * Low Moat & High Substitutability: Feedback centers on generic benefits ("easier than paper," "nice app," "staff like it"). In these segments, Relay is a commoditized scheduling tool facing brutal churn against entrenched competitors.

---

2. The Danger in the Q3 Numbers

Dana, the Q3 commercial results look impressive on the surface ($412k ARR added across 61 deals), but the underlying mechanics are dangerous:

  1. The Custom Development Trap: $157k ARR (38% of all Q3 new ARR) came from just two hospitality deals (Harbour Hotels and Pinewood Leisure). Closing them cost 15 engineer-weeks of bespoke integrations that cannot be reused.
  2. Consulting Disguised as SaaS: We now have 14 hospitality groups in the pipeline demanding Harbour’s payroll integration. If we triple sales, reps will sell these deals, turning our product team into a bespoke professional services shop.
  3. Low Conversion at High Volume: In August, trial signups jumped 3.2x to 1,140 (from 356 in April), but paying account additions barely moved (from 49 to 55). Conversion cratered from ~14% to 4.8%. Flooding top-of-funnel without targeted positioning wastes acquisition spend.

---

3. What We Should Do Next Quarter

Instead of tripling the sales team across all segments, we should execute a targeted four-point plan:

#### A. Reposition GTM Exclusively Around Healthcare * Stop outbound into generic retail and hospitality. Reallocate all inbound marketing and outbound SDR capacity toward outpatient clinics, specialist practices, and urgent care networks. * Position Relay as Clinical Workforce Management: Rework marketing collateral around compliance, credential verification, and automated clinical swap rules, where we have demonstrated pricing power and retention.

#### B. Controlled Sales Expansion (Hire 2–3 Healthcare Specialists, Not 3x Reps) * Do not hire 10–15 generalist reps. Add 2 to 3 Account Executives with health-tech / clinical SaaS backgrounds dedicated exclusively to medical clinics. * Establish segment-specific quota and qualification criteria that filter out low-retention, high-churn retail accounts.

#### C. Product: Harden the Healthcare Moat & Build Standard API Infrastructure * Healthcare workflow investment: Deepen clinical credential management, rules engines for shift swapping, and automated compliance logging. * Freeze one-off custom enterprise integrations: Refuse custom, single-tenant engineering work for hospitality deals. Assign 1–2 engineers to build a standardized Payroll/HR export API. If enterprise hospitality prospects cannot use standard integrations, we disqualify them until the platform is ready.

#### D. Ringfence Existing High-ARR Hospitality Accounts * Protect the $157k ARR from Harbour and Pinewood by assigning dedicated customer success coverage, but set clear boundaries against scope creep.

---

Conclusion

Tripling sales today will scale our churn, exacerbate custom engineering bottlenecks, and mask our true strengths. By focusing our sales and product efforts entirely on Healthcare Clinics, where product-market fit is proven, we can build a durable, high-margin, high-retention SaaS business.

I look forward to discussing this in detail on Thursday.

Claims the judge couldn’t find in the brief

  • August's tripling of trial sign-ups came from the Product Hunt launch.

Grades and run details

Decision model 69 · LLM judge 64
Decision model checks
  • failUses the supplied evidence correctly22%
  • passAddresses the actual decision36%
  • passRespects explicit constraints28%
  • failIdentifies material uncertainty74%
  • partialAvoids unsupported claims29%
  • passProduces the required deliverable76%
  • passFinds the segment the product fits99%
  • passDiscounts the sign-up spike100%
  • passJudges fit by retention, not sign-ups85%
  • failGets the base of every number right16%
  • passFinds who it's a must-have for100%
  • passSpots false fit99%
  • partialDoesn't scale before fit63%
Run
Run
#1
API response time
52 s
Submitted
6 Oct 2026
GPT-6.1 SolAPI

Needs more than a quick edit10 of 13 checks passed

The memo contains a factual error in the credential-checks count (22 instead of 20), so a PM would need to correct it before using the output.

Output

To: Dana Kowalski, CEO From: Head of Product Subject: Next quarter: scale the clinic wedge, not the whole sales team

Recommendation

Do not triple sales next quarter. Relay has credible evidence of product-market fit among healthcare clinics, but not across shift-based businesses generally. We should concentrate existing sales capacity on clinics, improve their onboarding and core workflows, and test whether hospitality integration demand can become a repeatable product rather than bespoke services.

What the data actually says

1. The survey supports a segment-specific conclusion

Counting the raw responses:

SegmentResponses“Very disappointed”Share
------:---:---:
Healthcare clinic1186353.4%
Hospitality1402820.0%
Retail1031817.5%
Other511631.4%
Total41212530.3%

The commonly used 40% “very disappointed” benchmark is a diagnostic, not proof. Nevertheless, clinics clear it; Relay overall does not.

The clinic value proposition is specific: faster rota creation, rule-following on-call swaps, sick-call coverage, and fewer after-hours messages. Within clinics, 17/25 respondents citing rule-following on-call swaps are very disappointed, versus 5/22 citing credential checks. That suggests a stronger wedge around scheduling and coverage under operational constraints—not that every healthcare feature is equally valuable.

Our best-supported target is healthcare clinics with 50–500 employees, serving the people responsible for scheduling and coverage. Both size bands appear in the clinic responses; we should not invent a narrower winning size or role without further evidence.

Limits matter: only 36.8% of paying accounts responded. These are current paying-account admins, excluding churned customers and frontline staff. The results are account-weighted, not location- or revenue-weighted, and may overstate attachment across the original customer population.

2. Retention independently supports the clinic wedge

For cohorts old enough to have reached each milestone, weighted logo retention is:

SegmentMonth 6: October–March cohortsMonth 9: October–December cohorts
------:---:
Healthcare clinic69/97 = 71.1%37/52 = 71.2%
Hospitality70/150 = 46.7%25/71 = 35.2%
Retail47/119 = 39.5%17/59 = 28.8%

These are different cohort pools, not one continuous aggregate curve. Within the individual clinic cohorts, retention largely stabilizes after months four or five. Hospitality and retail continue losing accounts.

Clinics still lose roughly three in ten accounts early; onboarding is not solved. But survey attachment plus a retention plateau is substantially stronger evidence than either signal alone. Hospitality and retail do not currently justify broad acquisition acceleration.

3. Growth and large deals do not establish repeatability

August produced 1,140 trials versus July’s 365, but only 55 new paying accounts versus 50. September produced 520 trials and 47 paying accounts. Same-month paid/trial ratios are not cohort conversion rates, but the trial spike plainly did not coincide with comparable paid-account growth.

In the Q3 deal file, Harbour and Pinewood contribute $157,000—38.1% of new ARR—from two of 61 accounts. They required 15 engineer-weeks of custom work; Harbour’s integration is unavailable to other customers. Fourteen similar pipeline requests are worth investigating, not treating as fourteen repeatable wins.

There are also unresolved measurement conflicts: monthly signups total 152 Q3 new paying accounts, versus 61 in the deal file, and January–June retention-cohort starting counts do not match monthly new-paying counts. These may reflect different definitions, but we must reconcile them before sizing sales investment.

What we should do next quarter

1. Run a focused clinic growth program with existing sales capacity. Create a clinic-focused sales/product/customer-success pod. Qualify for recurring coverage complexity and rule-bound swaps; sell measurable scheduling outcomes. Avoid speculative enterprise promises. Continue serving other segments, but stop broad paid expansion there until retention improves.

2. Reduce early clinic losses and strengthen the demonstrated value. Instrument first rota publication, staff participation, first successful coverage/swap, and time to value. Interview retained, early-churned, and “somewhat disappointed” clinic customers. Prioritize guided setup, rule configuration, and coverage workflows based on observed failure points—not merely the loudest feature requests.

3. Test hospitality integrations as a product opportunity. Discover the systems and requirements behind all 14 requests. Seek paid pilot commitments for a shared, reusable integration. Proceed only if requirements converge and pricing covers delivery and ongoing support; otherwise price bespoke work explicitly or decline it.

4. Make hiring conditional on repeatability and economics. Within the first month, reconcile account definitions and report segment-level funnel cohorts, acquisition cost, gross-margin-adjusted payback, support/implementation cost, and location/revenue retention. Logo retention alone cannot establish economics for per-location pricing.

At quarter-end, authorize staged clinic sales hiring only if attachment remains strong, mature retention holds, new cohorts activate reliably, and acquisition economics meet an agreed hurdle. Do not pretend next quarter’s new customers already have six-month retention.

Bottom line: We have a promising clinic business to scale deliberately—not evidence that three times as many sellers can efficiently sell Relay to everyone.

Claims the judge couldn’t find in the brief

  • Within clinics, 5/22 citing credential checks are very disappointed.

Grades and run details

Decision model 85 · LLM judge 71
Decision model checks
  • failUses the supplied evidence correctly40%
  • passAddresses the actual decision97%
  • passRespects explicit constraints39%
  • passIdentifies material uncertainty97%
  • passAvoids unsupported claims24%
  • passProduces the required deliverable50%
  • passFinds the segment the product fits95%
  • passDiscounts the sign-up spike100%
  • passJudges fit by retention, not sign-ups55%
  • failGets the base of every number right41%
  • passFinds who it's a must-have for93%
  • passSpots false fit95%
  • passDoesn't scale before fit97%
Run
Run
#1
API response time
1.6 min
Submitted
5 Oct 2026
Haiku 5.5API

Needs more than a quick edit11 of 13 checks passed

A factual error (7 vs 8 'not sure yet' respondents) needs cross-checking with the survey data; the January cohort month-9 retention claim is also unsupported, requiring a minor data re-verification before use.

Output

To: Dana Kowalski, CEO From: Head of Product Re: Product-market fit, and whether to triple sales For: Thursday's leadership meeting

Bottom line

We have product-market fit in one segment: healthcare clinics. We don't have it in hospitality or retail, which together make up 60% of our survey responses. Tripling sales now would mostly add hospitality and retail accounts that churn, and push us further into custom engineering to close deals. I recommend a focused healthcare sales investment, gated hiring elsewhere, and a fix-first plan for hospitality.

What the survey says

We surveyed every paying admin in September. 412 of 1,120 responded (37%). The standard PMF test is whether at least 40% would be "very disappointed" if they could no longer use the product.

SegmentResponsesVery disappointedSomewhatNot disappointed
Healthcare clinic11854%32%14%
Other5032%40%28%
Hospitality14020%44%36%
Retail10417%44%38%
All41231%

Healthcare clears the bar comfortably, and the value is spread across the workflow: on-call swaps that follow clinical rules, rotas done in an hour, sick-call cover, fewer night texts, and credential checks. That's a durable job to be done, not one feature.

Hospitality is weak. Over a third are "not disappointed," and seven of its 28 "very disappointed" respondents name "not sure yet" as their main benefit. Those customers are attached to us but can't say why, which usually means weak retention.

Retail is similar: 38% not disappointed, and its main benefits are generic ("fine for now," "shift swaps").

What the cohort data says

  • Healthcare: roughly 70–75% of accounts are still active at months 8–9, and the curve flattens after about month 4.
  • Hospitality: only 35–37% remain at month 9 in the Oct–Jan cohorts. Newer cohorts are not clearly better at the same age (month 3: 65% for Oct versus 66% for June).
  • Retail: 27–30% remain at month 9 for the 2025 cohorts, with no improvement since.

Growth is coming from volume in segments that don't retain.

What the sales and deal data says

  • Trial-to-paid conversion was steady at about 14% from January to July, producing 41–52 new paying accounts a month.
  • August had 1,140 trials but only 55 paying accounts (4.8%). September had 520 trials and 47 paying (9%). We don't yet know what drove the August spike. Until we do, we shouldn't scale anything that assumes trial volume converts.
  • Q3 new ARR was $412k, but $157k of it (38%) came from two hospitality deals, Harbour Hotels ($96k) and Pinewood Leisure ($61k). Both required custom integrations (15 engineer-weeks in total), and neither is available to other customers.
  • Fourteen hospitality groups are now asking for "the same integration Harbour got." If each needs a comparable build (6–9 engineer-weeks), that's 84–126 engineer-weeks. That is custom services work, not product, and it would crowd out the roadmap.

Two numbers to reconcile before Thursday

  • The CRO reports 61 new accounts in Q3. The signups file shows 152 new paying accounts across July–September (50, 55, 47). One of these definitions is different from the other, and we should not present either until they match.
  • In the cohort file, the Oct and Nov rows for healthcare and hospitality are identical. This may be a copy error.

What I recommend for next quarter

  1. Invest in healthcare, but don't triple the team. Add a small healthcare-focused pod, using our 54% very-disappointed base and retention to build case studies and a clear ICP. Test adjacent regulated shift settings with a few targeted pilots, not a full push.
  1. Fix hospitality before buying more of it. Interview roughly 25 hospitality accounts, including churned ones and the "not sure yet" respondents, to find out what value they're missing. Freeze hospitality headcount until month-6 retention moves from about 48% toward 60%.
  1. No bespoke integrations. Set a rule that custom engineering needs a paid services contract that covers its full cost, plus sign-off from Product. For the 14 requests, ask for signed letters of intent or paid pilots for a shared payroll connector. Build it only if enough commit to justify it.
  1. Keep retail on maintenance. No growth spend until we understand why retention is flat at around 30%.
  1. Investigate the August trial spike and the trial-to-paid drop. Confirm the source, and check whether any sales or marketing change caused it.
  1. Make PMF measurable. Re-run the survey in Q4 among accounts with at least three months' use, with segment and company size on every response. Reconcile the Q3 numbers and the cohort data first.

Hiring gates

I'd tie further sales hiring to evidence, not to a calendar:

  • Healthcare: expand once healthcare retention holds at 70%+ at month 6 and the healthcare pipeline is converting at the 14% trial rate.
  • Hospitality: resume hiring only when month-6 retention reaches about 60% and the re-survey shows at least 40% very disappointed in that segment.
  • Retail: revisit when retention improves, but not before.

Caveats

The survey covers 37% of accounts, and admins who respond may be more engaged than those who don't. The "Other" segment is only 50 responses, so I wouldn't draw conclusions from it. Retention curves are based on a limited number of cohorts.

The short version: we have real product-market fit in healthcare, and a growing customer base that isn't sticking in hospitality or retail. Tripling sales would scale the second problem, not the first.

Claims the judge couldn’t find in the brief

  • Seven of hospitality's 28 very-disappointed respondents name 'not sure yet' as main benefit
  • Hospitality retention is 35–37% at month 9 in the Oct–Jan cohorts

Grades and run details

Decision model 81 · LLM judge 79
Decision model checks
  • failUses the supplied evidence correctly16%
  • passAddresses the actual decision98%
  • passRespects explicit constraints40%
  • passIdentifies material uncertainty98%
  • partialAvoids unsupported claims37%
  • passProduces the required deliverable79%
  • passFinds the segment the product fits100%
  • passDiscounts the sign-up spike98%
  • passJudges fit by retention, not sign-ups37%
  • failGets the base of every number right31%
  • passFinds who it's a must-have for100%
  • passSpots false fit98%
  • passDoesn't scale before fit100%
Run
Run
#1
API response time
1.7 min
Submitted
8 Oct 2026

Check by check

Where the outputs differ comes first. Open a check to read the judge’s reasons. “Mixed” means the two graders disagreed.

Where they differ 5

Addresses the actual decisionMixedRightRight
Gemini 3.8 Flash · API

The memo does not say what result or condition would change the call against tripling sales; it commits to a plan but omits the threshold for revisiting the decision.

GPT-6.1 Sol · API

The memo clearly says 'Do not triple sales next quarter' and gives a conditional plan for scaling clinics later.

Haiku 5.5 · API

The memo unambiguously states product-market fit exists only in healthcare clinics and advises against tripling sales firm-wide, specifying what evidence would change the call.

Identifies material uncertaintyWrongRightRight
Gemini 3.8 Flash · API

The memo does not name specific unknowns that could change the decision or say how they would be resolved.

GPT-6.1 Sol · API

It names the measurement conflicts (signup vs deal counts, cohort vs monthly counts) and sets conditions for future hiring.

Haiku 5.5 · API

It names the August trial spike source, the Q3 account number discrepancy, the cohort data copy error, survey response bias, and specifies re-survey and retention thresholds to resolve them.

Avoids unsupported claimsWrongMixedRight
Gemini 3.8 Flash · API

The Product Hunt launch is presented as an established cause of the sign-up spike, but the evidence only shows the spike without explaining its origin.

GPT-6.1 Sol · API

The 5/22 figure is presented as fact but is not supported by the data; the correct count is 20.

Haiku 5.5 · API

Interpretations like 'weak retention' are qualified; confident claims are limited to what the data supports.

Gets the base of every number rightMixedWrongWrong
Gemini 3.8 Flash · API

All derived percentages (54.2%, 34.8%, 28.6%, 30.3%, 38%, conversion rates) check out against the supplied data.

GPT-6.1 Sol · API

The base for the credential-checks claim is wrong (22 instead of 20), so the derived figure is not computed correctly.

Haiku 5.5 · API

The number of hospitality 'very disappointed' naming 'not sure yet' is stated as 7 but is actually 8; also January cohort lacks month-9 data for the retention claim.

Doesn't scale before fitWrongRightRight
Gemini 3.8 Flash · API

The recommendation scales sales only in healthcare but does not set the evidence (e.g., retention flattening in other segments) that would justify widening later.

GPT-6.1 Sol · API

It recommends scaling only the clinic segment and sets conditions for expanding later.

Haiku 5.5 · API

It recommends scaling only in healthcare and sets explicit evidence thresholds (retention and survey targets) before expanding elsewhere.

All got wrong 1

Uses the supplied evidence correctlyWrongWrongWrong
Gemini 3.8 Flash · API

The memo states the August sign-up spike 'came from the Product Hunt launch', which is not in the supplied context and is an invented fact about the current situation.

GPT-6.1 Sol · API

The claim '5/22 citing credential checks' is incorrect; the data shows 20 respondents with that benefit, not 22.

Haiku 5.5 · API

The claim 'seven of its 28 very disappointed respondents name not sure yet' is incorrect (correct count is 8), and the statement about hospitality month-9 retention includes the January cohort which has no month-9 data.

All got right 7

Respects explicit constraintsRightRightRight
Gemini 3.8 Flash · API

The memo is addressed to the CEO, is under 900 words, and is in the requested memo format.

GPT-6.1 Sol · API

The memo is under 900 words, addressed to the CEO, and respects the requested form.

Haiku 5.5 · API

The memo is addressed to the CEO, stays under 900 words, and directly answers the questions posed.

Produces the required deliverableRightRightRight
Gemini 3.8 Flash · API

The memo is complete, in the right form for the CEO, within the word limit, and actionable with light edits.

GPT-6.1 Sol · API

The memo is complete, within the word limit, and actionable by the CEO.

Haiku 5.5 · API

The memo is complete, correctly formatted for the CEO, and the recommendations could be acted on with light editing.

Finds the segment the product fitsRightRightRight
Gemini 3.8 Flash · API

The memo breaks down survey and retention by segment, finds clinics at 54% very disappointed with flattening retention, and notes the blended 31% hides this.

GPT-6.1 Sol · API

It breaks down survey and retention by segment, finds clinics at ~54% very disappointed and retention flattening, while other segments do not.

Haiku 5.5 · API

The memo breaks down survey and retention by segment, correctly identifying healthcare clinics as the segment with product-market fit.

Discounts the sign-up spikeRightRightRight
Gemini 3.8 Flash · API

The memo compares August trial sign-ups with new paying accounts, notes the conversion drop, and treats the spike as not evidence of fit.

GPT-6.1 Sol · API

It compares August's trial spike with new paying accounts and concludes the spike is not evidence of fit.

Haiku 5.5 · API

It compares August's tripled trial sign-ups with near-flat new paying accounts and notes the spike is not evidence of fit.

Judges fit by retention, not sign-upsRightRightRight
Gemini 3.8 Flash · API

The call on PMF rests on whether retention curves flatten by segment, with the survey as supporting evidence.

GPT-6.1 Sol · API

The call on fit rests on segment-level retention curves flattening, with the survey as supporting evidence.

Haiku 5.5 · API

The call on fit is grounded in segment-level retention curves flattening (or not), with the survey as supporting evidence.

Finds who it's a must-have forRightRightRight
Gemini 3.8 Flash · API

The memo names healthcare clinics as the must-have segment, profiles their valued features, and builds the plan around them.

GPT-6.1 Sol · API

It profiles healthcare clinics with 50–500 employees and the scheduling/coverage persona as the must-have segment.

Haiku 5.5 · API

It profiles healthcare clinics as the must-have segment, lists their specific values (on-call rules, credential checks, etc.), and builds the plan around them.

Spots false fitRightRightRight
Gemini 3.8 Flash · API

The memo identifies the custom-integration deals, the pipeline of copycat requests, and the sign-up spike as false signals of fit and explains why.

GPT-6.1 Sol · API

It identifies the trial spike, custom-integration deals, and pipeline requests as false signals of fit.

Haiku 5.5 · API

It identifies the August trial spike and custom integration revenue as false signals of fit and explains why they are not product-market fit.

Results

Every setup we’ve tested on this task type, across all its tasks and repeats, graded on the current checklist. Calibrated.

#Model · HarnessTask scoreDecision modelLLM judgeRunsCritical failures
1GPT-6 AstraChatGPT90.5100.02None
2Sonnet 5.5API77.990.02None
3Opus 5.5Claude79.782.92None
4Haiku 5.5API77.986.02None
5GPT-6.1 SolAPI90.575.72None
6GPT-6 LunaAPI92.772.42None
7Gemini 3.8 FlashAPI65.062.12None
8Gemini 3.5 Flash-LiteGemini65.151.92None

What you get for the cost

No setup scores within 5 points of the leader at half its cost or less here, so the leader is also the best value.

Task score against relative cost for each setup. The table below has the same figures.50607080901001×3×10×30×100×Cost per output, relative to the cheapest setup (log scale) →↑ Task scoreGPT-6 Astra · ChatGPT: task score 96, 118× the cheapestSonnet 5.5 · API: task score 85, 81× the cheapestOpus 5.5 · Claude: task score 82, 118× the cheapestHaiku 5.5 · API: task score 81, 6.1× the cheapestGPT-6.1 Sol · API: task score 80, 25× the cheapestGPT-6 Luna · API: task score 79, the cheapestGemini 3.8 Flash · API: task score 63, 16× the cheapestGemini 3.5 Flash-Lite · Gemini: task score 56, 2.8× the cheapestGPT-6 Astra · ChatGPTSonnet 5.5 · APIOpus 5.5 · ClaudeHaiku 5.5 · APIGPT-6.1 Sol · APIGPT-6 Luna · APIGemini 3.8 Flash · APIGemini 3.5 Flash-Lite · Gemini
Nobody beats these on both score and cost

Cost is each output’s tokens at the model’s list price, shown relative to the cheapest setup. Runs done in an app have no usage meter, so theirs are estimated from the same setup’s measured runs. How we compare cost

The PM job

Deciding whether you have product-market fit, and for whom.

Why it matters

Scaling before fit amplifies waste. Most teams read fit from sign-ups or a blended survey, and miss the one segment where it's real.

What good looks like

  • Judges fit by retention, not sign-ups
  • Breaks the survey down by segment and tenure
  • Spots revenue and metrics that flatter
  • Scales only where fit is proven

Deliberately not measured

  • Fundraising strategy beyond what the evidence supports
Capability tested

Reading fit from evidence

The failure we’re looking for

Calling fit from a blended number or a growth spike

Grading

Decision model and LLM judge, calibrated against a blind PM review