Write a PRD

Product Requirements Document: AI Meeting Summaries & Action Items

Product: Huddle Author: Product Management Target Audience: Engineering & Design Leads Status: Ready for Kickoff

---

1. Overview & Problem Statement

Huddle currently records and transcribes video meetings for 3,000 B2B customers. However, users still spend 10–15 minutes after every call reviewing transcripts or typing up raw notes to document decisions and action items.

We are building an AI-powered post-meeting summary feature that automatically extracts Key Decisions and Action Items (with assigned owners) from meeting transcripts.

The Core Tension: Speed vs. Accuracy

Sales has requested that we automatically email action items to their owners the moment a meeting ends. However, we have three strict operational realities: 1. Hard Constraint: We must never assign an action item to someone who did not attend the meeting. 2. Data Gap 1 (The 15% problem): 15% of meetings include uninvited guests (e.g., forwarded links, conference rooms) who are missing from the calendar invite. 3. Data Gap 2 (The 8% problem): Speaker diarization/labels are wrong 8% of the time, primarily due to crosstalk or multiple people sharing one room mic.

Product Decision: If we auto-send emails immediately after the meeting ends, the 8% speaker attribution error rate and the 15% calendar mismatch rate will inevitably cause us to violate our hard constraint or embarrass users by emailing the wrong people. Therefore, v1 will use a "Human-in-the-Loop" flow with a fallback auto-send. For high-confidence extractions, emails will send automatically after a 30-minute review window. For low-confidence extractions (crosstalk, unverified guests), we will hold the email until the meeting host approves the summary.

---

2. Goals & Success Metrics

---

3. System Architecture & The "Actual Attendee" Roster

Right now, Huddle relies on the calendar invite for the attendee list. To obey our hard constraint, Engineering must build a unified `Actual_Attendees` service that fires when a meeting ends.

Defining an "Actual Attendee"

A person is only eligible to be assigned an action item if they exist in the `Actual_Attendees` array for that specific meeting session. Calendar invitees who did not join the call are explicitly excluded.

1. Authenticated Huddle Users: Anyone who joins the call while logged into a Huddle account is added to `Actual_Attendees` (solves the 15% uninvited guest problem for logged-in users).
2. Unauthenticated Web Guests: Guests joining via link must type their display name in the pre-call lobby. They are added to `Actual_Attendees` with `email: null`.
3. Conference Room Hardware: When a room system joins (e.g., "4th Floor Conf Room"), the room itself is not an assignable person. (See Edge Cases in Section 4).

---

4. Functional Requirements (Engineering)

4.1 LLM Extraction Pipeline

* Trigger: Fires upon completion of the final meeting transcript. * Inputs: 1. Time-stamped transcript with speaker labels. 2. Diarization confidence scores per utterance. 3. The `ActualAttendees` JSON array (Name, UserID, Email, JoinMethod). 4. The `CalendarInviteesNoShow` array (Explicit negative constraint list). * Prompting & Guardrails: * Decisions: Extract 1–7 bullet points summarizing agreed-upon outcomes. Include timestamp citations. * Action Items: Extract tasks, deadlines (if mentioned), and the owner. * Assignment Logic: The LLM may only populate the `owner` field using an exact match from `ActualAttendees`.

4.2 Handling the Constraints & Edge Cases

• Scenario A: Someone on the call says, "Let's have Sarah do this," when Sarah is NOT in `Actual_Attendees`.
• System Behavior: The system must not assign the task to Sarah. Instead, it assigns the task to the speaker who volunteered Sarah (or leaves it "Unassigned") and rewrites the task text to: "Follow up with Sarah to [do X]."
• Scenario B: The 8% Speaker Label Error (Crosstalk / Low Confidence).
• System Behavior: If an action item is derived from an utterance where someone says "I will do X", the pipeline checks the diarization confidence score for that timestamp block. If confidence is `< 0.85` or overlaps with another speaker within 1.5 seconds, the system sets `owner: Unassigned`, sets `suggestedowner: [SpeakerID]`, and flags the item as `needs_review: true`.
• Scenario C: Multi-person Conference Rooms.
• System Behavior: Audio coming from a single hardware node containing multiple voices cannot reliably attribute "I'll take that" to an individual. If the speaker label is a Room Resource or has multi-voice flags, the LLM may only assign an action item if a name is spoken explicitly in the third person (e.g., "Thanks Dave, so Dave will pull the Q3 numbers" AND "Dave" is matched to a Huddle user via calendar invite + confirmation). Otherwise, set `owner: Unassigned` (`needs_review: true`).
• Scenario D: The 15% Uninvited Guest.
• System Behavior: If "Alex" joined via a forwarded link without logging in, he is in `ActualAttendees` as `{name: "Alex (Guest)", email: null}`. The system can assign action items to "Alex (Guest)" in the UI, but flags the item as `missingemail: true`.

4.3 The Email Dispatch Engine (Addressing the Sales Request)

To satisfy Sales without sending hallucinated assignments, we will implement a Two-Track Dispatch Engine:

• Track 1: High-Confidence Auto-Send (30-Minute Delay)
• Condition: All extracted action items have `owner != Unassigned`, `needsreview == false`, and `missingemail == false`.
• Behavior: The host gets an immediate Slack/Email ping: "Your meeting summary is ready. Action items will be emailed to owners in 30 minutes unless you edit them." If the host makes no changes within 30 minutes (or clicks "Send Now"), emails are dispatched to action item owners.
• Track 2: Low-Confidence Hold
• Condition: One or more action items are flagged `needsreview: true` or `missingemail: true`.
• Behavior: Auto-send is paused. The host is notified: "Your meeting summary has 2 action items that need your confirmation before we email the team." Emails only trigger once the host resolves the flags and clicks "Approve & Send".

---

5. User Experience & Design Requirements

5.1 Pre-Meeting Lobby (Small tweak for Guest ID)

• Unauthenticated Guests: Update the pre-join screen. Require both Full Name and Work Email (Optional) before joining. Explaining "Used to send you meeting notes" will convert a high percentage of the 15% uninvited guests into identifiable users.

5.2 Post-Meeting Summary View

The Meeting Host (and participants) will access the summary inside the Huddle web/desktop app under the "Past Meetings" tab.

* Layout: Split screen. Left side: Video player + searchable transcript. Right side: AI Summary widget. * Summary Widget Sections: 1. Status Banner: Shows dispatch state (e.g., "Auto-sending to 4 owners in 24 mins [Send Now] [Pause]" OR "Approval Required before emailing"). 2. Decisions Made: Bulleted list. Hovering over a decision highlights the corresponding timestamp in the transcript and lets the user click to play that 15-second clip. (Crucial for building user trust). 3. Action Items List: Each row contains: * Checkbox (for completion tracking). * Task description (inline editable). * Assignee Dropdown Pill. * Source timestamp link.

5.3 The Assignee Dropdown & "Needs Review" UI

• Restricted Roster: When a user clicks the Assignee Pill to change or add an owner, the dropdown menu only lists people from the `Actual_Attendees` array.
• Design detail: Below a divider line at the bottom of the dropdown, show calendar invitees who didn't show up in greyed-out, unclickable text with a tooltip: "Did not attend meeting." (This prevents users from thinking the dropdown is broken while enforcing our constraint—unless someone was hiding in a conference room, see below).
• Conference Room Override: If a meeting included a Room system, include an option in the dropdown: `+ Add person who was in [Conf Room Name]`. Clicking this allows the host to pick from the org directory, explicitly confirming that the person was physically present in the room.
• Low-Confidence Visual State: Action items flagged with `needs_review: true` (due to the 8% diarization issue) should have an amber highlight around the Assignee Pill reading: `Suggested: [Name] ?`. Clicking it lets the host confirm with one click.

5.4 The Action Item Email

• Subject: `Action Items from [Meeting Title] ([Date])`
• Body:
• Personalized top section: "Assigned to You (2):" followed by their specific tasks.
• Context section below: "All Decisions & Team Action Items" so they have full context.
• Footer CTA: `[View Full Recording & Transcript in Huddle]`.

---

6. Out of Scope for v1

• 3rd-Party Task Manager Integrations: Pushing tasks to Jira, Asana, Linear, or Salesforce (planned for v1.1 after we validate extraction accuracy).
• Cross-Meeting Action Item Tracking: A global "My Tasks" dashboard across all meetings. v1 is strictly scoped to the single-meeting view and email dispatch.
• Voiceprint Enrollment: Having users record their voice to fix the conference room diarization problem. We will rely on UI fallbacks for v1.

---

7. Open Questions for Kickoff

1. Engineering: What is the p95 latency for transcript generation + LLM summary extraction on a 60-minute call? Sales expects "as soon as the meeting ends"—if our pipeline takes 8 minutes, do we need to adjust the 30-minute auto-send timer?
2. Design: If a meeting has zero decisions and zero action items (e.g., a casual 1:1 catch-up), what should the empty state look like, and should we suppress the post-meeting notification entirely?
3. Engineering: Can our current diarization vendor expose word-level confidence scores via their webhook, or do we need to calculate crosstalk purely by checking overlapping speaker timestamps?

PRD: AI Meeting Summaries with Decisions and Action Items

Status: Draft for engineering and design leads Product: Huddle

1. Problem

Huddle records and transcribes meetings with speaker labels, but attendees still write their own notes afterward. Notes are late, inconsistent, or never written, so decisions get lost and action items have no clear owner. We will generate a summary after each meeting that lists decisions and proposes action items with owners.

2. Goals and non-goals

Goals - Produce a summary within 5 minutes of meeting end: decisions, action items (task, owner, due date if stated), and open questions. - Never assign an action item to someone who was not in the meeting (hard constraint, see §4). - Keep the host's review to under 60 seconds for a typical meeting.

Non-goals (v1) - Fully automatic emailing of action items to owners (see §3). - Cross-meeting task tracking or integrations with Jira, Asana, and similar tools. - Summaries for meetings without a transcript.

3. Decision on the Sales request

Sales asked for action items to be emailed automatically to owners the moment a meeting ends. We are not building that for v1. Instead, we will send the draft summary to the host at meeting end. The host can send it to owners with one click, and owners are emailed on send.

Reasons: - Speaker labels are wrong about 8% of the time. An automatic email would sometimes tell a real person they own a task they never agreed to. Because it goes out with no human check, the error becomes visible to a customer's colleagues. - Email is irreversible, so a wrong or non-attendee assignment cannot be quietly fixed afterward. - The underlying Sales need is that owners hear about their tasks quickly and nothing falls through the cracks. A host prompt at meeting end with one-click send meets that need at the cost of a few seconds.

Path to automation: Auto-send will be an opt-in, per-workspace setting, considered after launch once the gates in §9 are met. It would send only high-confidence, presence-verified items and leave everything else for host review. Engineering should build the send pipeline so that auto-send is later a policy change rather than a rebuild.

4. The hard constraint: who can own an action item

Rule: An action item's owner must be a participant who was actually in the meeting when the item was discussed. Anyone else is ineligible.

This rule cannot rely on the calendar invite, and it cannot rely on the model behaving well.

Attendance source. The eligible-owner roster is built from Huddle's session join/leave logs, not from the calendar invite. This handles both failure modes: - Invitees who never joined are not on the roster. - The roughly 15% of meetings with uninvited joiners include those people, matched to a user account when possible.

Presence at time of item. A person is eligible for an item only if they were in the session at the timestamp of the relevant utterance. Someone who left at minute 10 cannot own an item from minute 40.

Enforcement is structural, not prompt-based. 1. The extraction step receives the roster as a list of participant IDs and must return an owner ID from that list or `null`. 2. A deterministic validator runs after the model. It rejects any owner ID not on the roster or not present at the item's timestamp, and it converts that item to unassigned. 3. The UI and email layers accept only validated owner IDs. There is no free-text owner field on the model output path. 4. Names mentioned in the transcript who were not present ("have Dana look at it") produce an unassigned item. The text may keep the speaker's wording, but Dana gets no assignment and no email.

Shared rooms. A room device usually appears as one participant, so individuals in the room are not in the presence log. We cannot verify that a named person is in the room. Such items become unassigned by default, and the host can attest who was in the room during review (§6). That attestation counts as human-verified presence.

Uninvited joiners without an account (guests) can be eligible owners, shown by display name. They receive email only if the host supplies an address during review. We never guess addresses.

5. Handling speaker-label errors

The 8% label error rate is concentrated in crosstalk and shared rooms. Eligible-owner enforcement does not solve it, because a mislabel can still pick the wrong person who was present. Mitigations:

Confidence tiers (High/Medium/Low) are stored on each item, shown to the host in review, and used later to gate auto-send. Thresholds are set empirically during the evaluation in §9. The tiers themselves are a v1 requirement.

6. User experience

Host flow 1. The meeting ends and the host gets a notification: "Summary ready to review." 2. The review screen lists: - Decisions as short statements, each linked to the transcript moment. - Action items with the task, owner chip, due date if stated, and a confidence indicator. - Unassigned items grouped at the top of the actions list, so gaps are visible. 3. The host can edit text, reassign from a picker limited to eligible people, delete items, or add room attendees ("Who was in the room?"). Adding room attendees makes them eligible for items in the room segments. 4. Send emails each owner their own items plus the decisions. Unassigned items go to no one.

Design requirements - Every item has a "jump to source" link that plays the transcript segment. Reviewing should be verification, not re-listening. - The owner picker never offers non-attendees. If the host wants to assign to an absent person, show a separate option, "Send as a follow-up (not from the meeting)," which is a different item type with clearly different wording and no "you agreed to" language. Design to decide whether this ships in v1; the default is that it does not. - Low-confidence owners appear as suggestions ("Suggested: Sam") and never as assignments. - Email copy says "Action item from Meeting name, date", includes the source excerpt, and offers a "This isn't mine" link that notifies the host. This gives owners a correction path. - If the host does not review, send a reminder at 24 hours. Nothing is sent to owners without host action in v1.

7. Functional requirements

1. Roster service: builds the per-meeting roster with join/leave intervals from session logs and links to user accounts where possible.
2. Extraction: from the transcript and roster, produce decisions, action items, and unassigned items. Each action item has task text, owner ID or null, due date or null, source timestamps, and a confidence tier.
3. Validator: enforces §4 as code with unit tests, independent of the model, and logs every rejection.
4. Review UI: as described in §6, on web and desktop. Mobile is view-only for v1.
5. Delivery: per-owner emails on host send, with a record of who was emailed. Sends are idempotent, and edits after sending trigger an explicit "send update" action.
6. Audit log: stores for each item the model output, validator result, host edits, and send events.
7. Admin setting: workspace-level toggle for summaries, plus retention aligned with existing transcript retention.

8. Edge cases to specify and test

• Invitee never joined but is mentioned as owner.
• Uninvited joiner takes an action; the guest has no account.
• Person leaves and rejoins (multiple intervals).
• Two people share a first name; ownership must resolve by roster, and ambiguity means unassigned.
• Whole meeting in a shared room (all items default unassigned until the host attests).
• Multiple languages, or a transcript with large gaps.
• Nobody commits to anything: show "No action items detected" rather than inventing some.

9. Success metrics and launch gates

Launch gates (must all pass) - Zero non-attendee assignments across a labeled evaluation set of at least 500 meetings, deliberately oversampled for uninvited joiners, shared rooms, no-show invitees, and crosstalk. Any production occurrence afterward is a Sev-1 incident. - Among High-confidence items, wrong-attendee assignment rate of 3% or less (target, to be validated against the evaluation set; propose a stricter figure if data supports it). - Median summary delivery within 5 minutes of meeting end.

Post-launch metrics - Share of meetings where the host sends the summary within 1 hour. - Host edit rate per item, and owner "This isn't mine" rate. - Share of action items left unassigned. Some is healthy, but a high rate means the model is too timid.

Auto-send criteria (future): two consecutive months of High-tier precision above the agreed bar and no non-attendee incidents. Sales and Support are consulted before enabling.

10. Risks and open questions

• Sales disagreement. Sales may see the host step as friction. We should share §3 with them early and offer a pilot of auto-send with a few design-partner accounts after the gates are met.
• Hosts skipping review. If sending rates are low, the fix is better reminders and faster review, not removing the check.
• Presence data quality. Do our session logs reliably capture join/leave for all clients and dial-in users? Engineering to confirm in week 1.
• Guest privacy. Can we email or display guest names without consent, given customer policies? Legal to advise.
• Room attestation. Is host attestation adequate for the constraint, or should room participants confirm individually? Recommend host attestation for v1.
• Follow-up items for absent people. Ship in v1 or defer? Design and Product to decide.

11. Milestones

1. Weeks 1–2: roster service, presence data audit, evaluation set labeling begins.
2. Weeks 3–5: extraction, validator, confidence tiers, offline evaluation.
3. Weeks 4–7: review UI and email delivery, in parallel.
4. Weeks 8–9: internal dogfood, gate review.
5. Week 10: limited beta with design-partner customers, including Sales' accounts.

PRD: AI-Generated Meeting Summaries and Action Items

Audience: Engineering and design leads Product: Huddle Status: Draft

1. Summary

Huddle will generate a post-meeting summary that highlights decisions and action items, identifies owners when it can do so reliably, and can email confirmed action items to their owners automatically. This replaces the manual note-taking people do after calls.

The core safety requirement is non-negotiable: an action must never be assigned to someone who did not attend the meeting. Calendar invitations alone cannot establish attendance: some invitees do not join, and around 15% of meetings include someone who was not invited. Speaker labels also cannot establish identity on their own; they are wrong about 8% of the time, especially during interruptions or when several people share a meeting-room connection.

The product must therefore separate three questions: what was decided, what needs to be done, and who—if anyone—can be safely identified as the owner. When attendance or ownership is uncertain, Huddle must leave the action unassigned rather than guess.

2. Problem and goals

Today, attendees write their own notes after meetings. This is inconsistent, takes time, and makes it easy for decisions and follow-ups to be lost.

Goals - Produce a concise, useful summary after a recorded meeting. - Clearly distinguish decisions from discussion, proposals, and unresolved questions. - Extract action items and assign them only to verified meeting attendees. - When enabled, email confirmed action items to their owners shortly after the meeting ends. - Let users correct mistakes and give feedback.

Non-goals for the first release - Replacing full meeting transcripts or recordings. - Creating tasks in external project-management tools. - Inferring deadlines, priorities, or owners that were not stated. - Automatically assigning work when the owner is unclear. - Resolving disagreements about whether a decision was made without review.

3. Users and primary workflow

Users: meeting attendees who need an accurate recap; organizers who want to review or share it; and action owners who need a clear follow-up.

1. A recorded meeting ends. Huddle gathers the transcript, speaker labels, calendar context, and actual session attendance data.
2. Huddle generates a summary with separate Decisions, Action items, and, where useful, Open questions sections.
3. For each action, Huddle identifies the supporting transcript evidence and checks the proposed owner against the verified attendee roster.
4. If automatic action-item email is enabled for the organization, Huddle emails only actions with a sufficiently clear owner whose attendance is verified.
5. The organizer and attendees can view the summary, correct it, and report errors. Unassigned actions remain visible for follow-up but are not emailed as assigned work.

4. Functional requirements

4.1 Establishing who attended

• Use Huddle’s meeting-session join/leave records as the attendance source of truth. The calendar invite is context, not proof that someone attended.
• Include identifiable people who joined without a calendar invitation. Exclude invitees who did not join.
• Represent an attendee using a verified Huddle account, authenticated guest identity, or another approved identity mechanism.
• If a meeting-room device represents several people and Huddle cannot reliably map speakers to individuals, do not treat the device name or a guessed speaker identity as a person. The organizer may confirm identities or owners in the summary UI.
• If reliable session attendance data is unavailable, fail closed: generate the summary, but do not assign or email action items.

4.2 Generating decisions and action items

• Generate a short overview and the three optional sections: Decisions, Action items, and Open questions.
• Include a decision only when the transcript supports an explicit agreement or choice. Label proposals or unresolved discussion as open questions, not decisions.
• Phrase each action as a concrete task. Include a due date only if one was explicitly stated; otherwise show no due date.
• Attach a brief transcript excerpt and timestamp to each decision and action so users can check the source.
• Do not invent missing details. If the task is clear but no owner is, show it as Owner not confirmed. If the task itself is ambiguous, flag it for review rather than present it as certain.

4.3 Assigning owners safely

• Every proposed owner must match a verified person in the actual attendee roster. Enforce this as a backend validation rule, not just a model instruction or UI check.
• A calendar invitee who did not join must never be assigned. A person who joined without an invite may be assigned if their identity and attendance are verified.
• Speaker labels may help identify who volunteered, but must not independently authorize an assignment. When overlap, room audio, or attribution uncertainty makes ownership unclear, leave the action unassigned.
• Users may assign or change an owner in the UI only to a verified attendee. Record the change and its actor in the audit log.
• Recheck the attendance invariant before saving an assignment and immediately before sending an email.

4.4 Automatic emails

• Provide an organization-level setting to enable automatic action-item emails. Once enabled, send an email after generation to each verified owner with at least one confirmed action.
• Do not send an assignment email for an unassigned action or to anyone not verified as having attended. Do not send the full transcript by default.
• Include the meeting title and date, the recipient’s confirmed actions, any explicitly stated due dates, a link to the Huddle summary, and a way to report an incorrect assignment.
• If the meeting has no verified owners, send no owner emails. The organizer may receive a link to the summary according to organization settings.
• Target delivery within five minutes of the meeting ending at the 95th percentile. If generation or identity checks fail, do not send speculative emails; show the organizer that the summary or email is delayed or needs review.
• Provide clear controls to turn automatic emails off. Email delivery status and failures must be visible to the organizer.

5. UX requirements

The summary should make confidence and uncertainty understandable without exposing raw model scores. Show an Owner confirmed state for verified assignments and Owner not confirmed when a person cannot be safely identified. Present source timestamps as links into the recording or transcript, subject to existing access permissions.

Organizers must be able to edit summary text, remove an incorrect item, or assign an unassigned item to a verified attendee. Edits must be saved with a clear indication of who made them. If an action email has already been sent, do not silently replace the original assignment: notify the organizer that the owner changed and offer a corrected email to the new verified owner. Never email the former owner again unless explicitly requested.

The UI should explain that the calendar list may include people who did not attend and that unidentified room participants require confirmation. Use clear empty states when there are no decisions or actions.

6. Data, privacy, and operational requirements

Use only meeting data already available to Huddle for recording, transcription, and attendance, subject to existing recording consent and organization policies. Apply existing access controls to summaries, transcript evidence, and recording links. Store generated summaries, edits, assignment records, email status, and relevant audit events under Huddle’s existing retention and deletion policies. Do not use action-item content to train models unless the organization’s existing data policy explicitly permits it.

Generation and email must be idempotent: retries must not create duplicate summary items or send duplicate emails. Log the evidence reference, proposed owner, roster-validation result, any human correction, and delivery outcome for troubleshooting and audit.

7. Acceptance criteria

The release is acceptable only if: 1. A calendar invitee who did not join is never assigned or emailed an action. 2. An identifiable, authenticated attendee who joined without an invite can be assigned when the transcript supports the assignment. 3. An unclear speaker in overlapping audio produces an unassigned action, not a guessed owner. 4. A shared-room connection with no reliable person mapping cannot result in an individual assignment without organizer confirmation. 5. Every emailed action has a verified attendee owner and a link to its supporting summary. 6. Email retries do not result in duplicate messages. 7. Users can correct an assignment, and the correction is restricted to verified attendees.

8. Success metrics and rollout

Track summary generation success, time to summary, owner-confirmation rate, action-item correction and removal rates, email delivery time, duplicate-email rate, and user feedback. The primary safety metric is the number of assignments or assignment emails to non-attendees; the target is zero. Also monitor missed or unassigned owners to understand the trade-off between safety and usefulness.

Roll out behind an organization-level feature flag. Start with internal meetings, then a small customer cohort. Review false assignments and user corrections before expanding. Do not broaden automatic-email availability until attendance validation, idempotency, and failure handling pass the acceptance criteria.

9. Dependency to resolve before implementation

Engineering must confirm that Huddle can provide reliable, person-level join/leave records, including authenticated guests and shared-room behavior. If that data is unavailable or insufficient for a meeting, the safe behavior is to generate the summary without assigning or emailing owners.