Write a PRD

Product Requirements Document: AI Meeting Summaries & Action Items

Product: Huddle Author: Product Management Target Audience: Engineering & Design Leads Status: Ready for Kickoff

---

1. Overview & Problem Statement

Huddle currently records and transcribes video meetings for 3,000 B2B customers. However, users still spend 10–15 minutes after every call reviewing transcripts or typing up raw notes to document decisions and action items.

We are building an AI-powered post-meeting summary feature that automatically extracts Key Decisions and Action Items (with assigned owners) from meeting transcripts.

The Core Tension: Speed vs. Accuracy

Sales has requested that we automatically email action items to their owners the moment a meeting ends. However, we have three strict operational realities: 1. Hard Constraint: We must never assign an action item to someone who did not attend the meeting. 2. Data Gap 1 (The 15% problem): 15% of meetings include uninvited guests (e.g., forwarded links, conference rooms) who are missing from the calendar invite. 3. Data Gap 2 (The 8% problem): Speaker diarization/labels are wrong 8% of the time, primarily due to crosstalk or multiple people sharing one room mic.

Product Decision: If we auto-send emails immediately after the meeting ends, the 8% speaker attribution error rate and the 15% calendar mismatch rate will inevitably cause us to violate our hard constraint or embarrass users by emailing the wrong people. Therefore, v1 will use a "Human-in-the-Loop" flow with a fallback auto-send. For high-confidence extractions, emails will send automatically after a 30-minute review window. For low-confidence extractions (crosstalk, unverified guests), we will hold the email until the meeting host approves the summary.

---

2. Goals & Success Metrics

---

3. System Architecture & The "Actual Attendee" Roster

Right now, Huddle relies on the calendar invite for the attendee list. To obey our hard constraint, Engineering must build a unified `Actual_Attendees` service that fires when a meeting ends.

Defining an "Actual Attendee"

A person is only eligible to be assigned an action item if they exist in the `Actual_Attendees` array for that specific meeting session. Calendar invitees who did not join the call are explicitly excluded.

1. Authenticated Huddle Users: Anyone who joins the call while logged into a Huddle account is added to `Actual_Attendees` (solves the 15% uninvited guest problem for logged-in users).
2. Unauthenticated Web Guests: Guests joining via link must type their display name in the pre-call lobby. They are added to `Actual_Attendees` with `email: null`.
3. Conference Room Hardware: When a room system joins (e.g., "4th Floor Conf Room"), the room itself is not an assignable person. (See Edge Cases in Section 4).

---

4. Functional Requirements (Engineering)

4.1 LLM Extraction Pipeline

* Trigger: Fires upon completion of the final meeting transcript. * Inputs: 1. Time-stamped transcript with speaker labels. 2. Diarization confidence scores per utterance. 3. The `ActualAttendees` JSON array (Name, UserID, Email, JoinMethod). 4. The `CalendarInviteesNoShow` array (Explicit negative constraint list). * Prompting & Guardrails: * Decisions: Extract 1–7 bullet points summarizing agreed-upon outcomes. Include timestamp citations. * Action Items: Extract tasks, deadlines (if mentioned), and the owner. * Assignment Logic: The LLM may only populate the `owner` field using an exact match from `ActualAttendees`.

4.2 Handling the Constraints & Edge Cases

• Scenario A: Someone on the call says, "Let's have Sarah do this," when Sarah is NOT in `Actual_Attendees`.
• System Behavior: The system must not assign the task to Sarah. Instead, it assigns the task to the speaker who volunteered Sarah (or leaves it "Unassigned") and rewrites the task text to: "Follow up with Sarah to [do X]."
• Scenario B: The 8% Speaker Label Error (Crosstalk / Low Confidence).
• System Behavior: If an action item is derived from an utterance where someone says "I will do X", the pipeline checks the diarization confidence score for that timestamp block. If confidence is `< 0.85` or overlaps with another speaker within 1.5 seconds, the system sets `owner: Unassigned`, sets `suggestedowner: [SpeakerID]`, and flags the item as `needs_review: true`.
• Scenario C: Multi-person Conference Rooms.
• System Behavior: Audio coming from a single hardware node containing multiple voices cannot reliably attribute "I'll take that" to an individual. If the speaker label is a Room Resource or has multi-voice flags, the LLM may only assign an action item if a name is spoken explicitly in the third person (e.g., "Thanks Dave, so Dave will pull the Q3 numbers" AND "Dave" is matched to a Huddle user via calendar invite + confirmation). Otherwise, set `owner: Unassigned` (`needs_review: true`).
• Scenario D: The 15% Uninvited Guest.
• System Behavior: If "Alex" joined via a forwarded link without logging in, he is in `ActualAttendees` as `{name: "Alex (Guest)", email: null}`. The system can assign action items to "Alex (Guest)" in the UI, but flags the item as `missingemail: true`.

4.3 The Email Dispatch Engine (Addressing the Sales Request)

To satisfy Sales without sending hallucinated assignments, we will implement a Two-Track Dispatch Engine:

• Track 1: High-Confidence Auto-Send (30-Minute Delay)
• Condition: All extracted action items have `owner != Unassigned`, `needsreview == false`, and `missingemail == false`.
• Behavior: The host gets an immediate Slack/Email ping: "Your meeting summary is ready. Action items will be emailed to owners in 30 minutes unless you edit them." If the host makes no changes within 30 minutes (or clicks "Send Now"), emails are dispatched to action item owners.
• Track 2: Low-Confidence Hold
• Condition: One or more action items are flagged `needsreview: true` or `missingemail: true`.
• Behavior: Auto-send is paused. The host is notified: "Your meeting summary has 2 action items that need your confirmation before we email the team." Emails only trigger once the host resolves the flags and clicks "Approve & Send".

---

5. User Experience & Design Requirements

5.1 Pre-Meeting Lobby (Small tweak for Guest ID)

• Unauthenticated Guests: Update the pre-join screen. Require both Full Name and Work Email (Optional) before joining. Explaining "Used to send you meeting notes" will convert a high percentage of the 15% uninvited guests into identifiable users.

5.2 Post-Meeting Summary View

The Meeting Host (and participants) will access the summary inside the Huddle web/desktop app under the "Past Meetings" tab.

* Layout: Split screen. Left side: Video player + searchable transcript. Right side: AI Summary widget. * Summary Widget Sections: 1. Status Banner: Shows dispatch state (e.g., "Auto-sending to 4 owners in 24 mins [Send Now] [Pause]" OR "Approval Required before emailing"). 2. Decisions Made: Bulleted list. Hovering over a decision highlights the corresponding timestamp in the transcript and lets the user click to play that 15-second clip. (Crucial for building user trust). 3. Action Items List: Each row contains: * Checkbox (for completion tracking). * Task description (inline editable). * Assignee Dropdown Pill. * Source timestamp link.

5.3 The Assignee Dropdown & "Needs Review" UI

• Restricted Roster: When a user clicks the Assignee Pill to change or add an owner, the dropdown menu only lists people from the `Actual_Attendees` array.
• Design detail: Below a divider line at the bottom of the dropdown, show calendar invitees who didn't show up in greyed-out, unclickable text with a tooltip: "Did not attend meeting." (This prevents users from thinking the dropdown is broken while enforcing our constraint—unless someone was hiding in a conference room, see below).
• Conference Room Override: If a meeting included a Room system, include an option in the dropdown: `+ Add person who was in [Conf Room Name]`. Clicking this allows the host to pick from the org directory, explicitly confirming that the person was physically present in the room.
• Low-Confidence Visual State: Action items flagged with `needs_review: true` (due to the 8% diarization issue) should have an amber highlight around the Assignee Pill reading: `Suggested: [Name] ?`. Clicking it lets the host confirm with one click.

5.4 The Action Item Email

• Subject: `Action Items from [Meeting Title] ([Date])`
• Body:
• Personalized top section: "Assigned to You (2):" followed by their specific tasks.
• Context section below: "All Decisions & Team Action Items" so they have full context.
• Footer CTA: `[View Full Recording & Transcript in Huddle]`.

---

6. Out of Scope for v1

• 3rd-Party Task Manager Integrations: Pushing tasks to Jira, Asana, Linear, or Salesforce (planned for v1.1 after we validate extraction accuracy).
• Cross-Meeting Action Item Tracking: A global "My Tasks" dashboard across all meetings. v1 is strictly scoped to the single-meeting view and email dispatch.
• Voiceprint Enrollment: Having users record their voice to fix the conference room diarization problem. We will rely on UI fallbacks for v1.

---

7. Open Questions for Kickoff

1. Engineering: What is the p95 latency for transcript generation + LLM summary extraction on a 60-minute call? Sales expects "as soon as the meeting ends"—if our pipeline takes 8 minutes, do we need to adjust the 30-minute auto-send timer?
2. Design: If a meeting has zero decisions and zero action items (e.g., a casual 1:1 catch-up), what should the empty state look like, and should we suppress the post-meeting notification entirely?
3. Engineering: Can our current diarization vendor expose word-level confidence scores via their webhook, or do we need to calculate crosstalk purely by checking overlapping speaker timestamps?

Huddle AI meeting summaries

Product requirements for engineering and design leads

Purpose and product decision

Huddle will generate a reviewable record of meeting decisions and actions from its transcripts. The host checks the draft, resolves ownership and publishes it. Approved actions can then be emailed to their owners.

Huddle serves 3,000 companies whose users currently write notes themselves. Speaker labels are wrong about 8% of the time, and roughly 15% of meetings include an uninvited participant. Calendar membership and transcript attribution therefore cannot establish attendance or ownership.

The MVP will not fulfil Sales’ request for immediate, unreviewed owner emails. It will generate drafts automatically after transcription and send emails automatically after explicit publication. Faster delivery does not justify assigning work to absent people. All requirements and numerical targets below are proposed product decisions.

Scope and intended outcome

Primary users are meeting hosts reviewing records and attendees receiving actions. Success means less time producing dependable notes, with no assignments to non-attendees.

MVP includes English transcripts, decisions, action extraction, attendance verification, host review, publication and owner emails. Exclude live summaries, recurring reminders, task-system integrations and automatic reassignment. Use existing meeting access and retention controls.

An action has one verified owner or remains unassigned. Split genuinely separate responsibilities into separate actions. Do not infer deadlines, commitments or decisions from tentative discussion.

Attendance is a release dependency

Engineering must first confirm whether Huddle retains authenticated participant identities and join/leave events. The supplied calendar list is insufficient. If these records are unavailable, build attendance instrumentation before enabling assignments; summaries may still launch with unassigned actions.

An eligible owner must have a stable person identifier linked to an authenticated, human participant session with a recorded join event for this meeting occurrence. Calendar invitees who never joined are ineligible. Bots and room devices are ineligible. Deduplicate reconnects by identity. Someone who left early remains eligible, although a later discussion does not prove they accepted work.

Uninvited users are eligible when their authenticated join is recorded. Guest display names, shared-room speaker labels and email addresses typed by a reviewer are not identity evidence. Unverified guests and people represented only by a room device remain ineligible in MVP; explain how they can join individually with verified identity for future meetings.

This deliberately sacrifices assignment coverage. The invariant is enforceable against verified participation records, not proof of physical human presence behind an account. If literal physical presence is required, that remains an unresolved product constraint and assignments must not launch under a stronger guarantee.

Ownership and extraction rules

The model extracts structured candidates with supporting transcript spans. Each decision contains its outcome and evidence timestamps. Each action contains a task, supporting spans, optional explicit due date and a nullable candidate participant identifier. Preserve the original date wording alongside any normalised date, using the meeting timezone; ambiguous dates stay unset.

Only explicit commitments or clearly agreed assignments qualify as actions. Suggestions, questions and rejected proposals do not. Where later discussion reverses a decision, present the final supported outcome. If the resolution is unclear, flag it for review instead of declaring a decision.

A model candidate is internal metadata, never an assigned owner. Server validation rejects identifiers outside the eligible attendance set before anything is displayed. The review UI starts every owner field as “Unassigned”; reviewers choose an eligible attendee after checking evidence. Do not preselect names from speaker labels, including for “I’ll do it”.

Render ownership only from validated structured fields. Generated task and decision wording must not embed owner claims that bypass these controls. Unsupported person-specific claims are withheld for review. A request that absent Alex should do something can become an unassigned “Confirm responsibility for…” follow-up only when that follow-up was actually agreed; otherwise omit it as an action.

Review and publication experience

The meeting page shows “Preparing summary” after the call, then “Draft needs review”. The host is the default reviewer; existing authorised meeting editors may also review. If no reviewer is available, retain the draft without sending owner emails.

Display separate Decisions and Actions sections. Each item links to its transcript passage and recording timestamp. Offer edit, delete and add controls. Actions show task, owner and due date. An attendance-only owner picker explains why invitees or unresolved guests cannot be selected. Provide a visible “Needs owner” state rather than silently dropping useful work.

Reviewers must confirm each chosen owner and can publish with unresolved actions. The publish dialogue shows recipients and unresolved-action count, with an explicit “Publish and email owners” control. Actions without owners remain visible but produce no email. Keyboard navigation, labelled controls and screen-reader status announcements are required.

Published records show their revision and approval time. Subsequent edits create a new revision. Ownership changes require the same validation and explicit publication, then notify affected owners with a correction. Never silently replace a published record with regenerated content.

Engineering contract and failure behaviour

Use meeting occurrence IDs, not recurring calendar series IDs. Store transcript version, attendance evidence version, model/prompt version, summary revision, item IDs, evidence spans, owner ID, approval actor/time and delivery status. Keep internal candidates separate from approved owner fields.

Process states are waiting for transcript, generating, needs review, published and failed. Trigger generation only after transcript finalisation. Transcript changes invalidate unpublished approvals and create a fresh draft; published revisions remain intact until a replacement is approved.

Enforce owner eligibility server-side on draft writes, publication, exports and notification dispatch. Manual edits and direct API calls cannot bypass it. If attendance evidence is missing, contradictory or unavailable, block ownership and sending rather than falling back to calendar data. Record rejection reasons for support without exposing transcript content in routine logs.

Use transactional publication and an outbox. Delivery keys combine meeting, published revision and recipient so retries do not duplicate emails. Email only approved actions owned by that recipient, with a permission-checked meeting link. Resolve addresses from verified identities, never model output. Block delivery if permissions or identity validation fail; show the reviewer the failure. Retry transient failures with backoff for up to 24 hours, then offer manual retry.

Treat transcripts as untrusted data: spoken instructions cannot change permissions, choose recipients or trigger tools. Generation has no email capability. Preserve tenant isolation and apply meeting deletion and retention policies to summaries and queued notifications.

Acceptance scenarios

1. Invited Pat never joins but is named in an action: Pat is absent from the picker; API assignment is rejected; no owner email is sent.

1. Uninvited authenticated Lee joins: Lee is eligible and can receive an action after reviewer confirmation and publication.

1. Several people join through one room device: no individual becomes eligible from its label; actions remain unassigned unless independent authenticated participation exists.

1. Overlapping speech mislabels a commitment: no owner is preselected; the reviewer can inspect evidence and choose a verified attendee or leave it unassigned.

1. No decisions or actions are supported: show explicit empty states rather than inventing content. Failed or incomplete transcription shows a retry state and sends nothing.

1. A forged owner ID, missing attendance record or duplicate publish event cannot create an invalid assignment or duplicate email. Revoked access blocks queued delivery.

Evaluation and release

Build a consented, de-identified evaluation set of at least 200 meetings, deliberately including overlap, shared rooms, absent invitees, uninvited guests, duplicate names, reversals and ambiguous commitments. Human annotators label attendance, final decisions, actions, ownership and evidence; adjudicate disagreements independently. Report each difficult slice separately.

Release gates: zero non-attendee assignments or deliveries across end-to-end and adversarial tests; at least 95% precision and 85% recall for supported decisions and actions, measured separately. Zero observed violations is necessary but insufficient: eligibility enforcement must also pass API and notification-path tests. Model confidence alone never permits assignment.

Pilot with ten opt-in companies before staged rollout. Target draft availability within two minutes of final transcript for 95% of meetings up to 60 minutes, and a 50% reduction in median note-preparation time against pilot baseline. Track owner corrections, unassigned-action rate, publication rate and delivery failures. Any non-attendee assignment disables assignments and sending pending investigation. Expand only after two weeks without a critical violation and acceptable review effort; do not reduce abstention merely to improve coverage.

PRD: AI-Generated Meeting Summaries with Decisions and Action Items

Audience: Engineering and Design leads Status: Ready for build planning Owner: Product

1. Summary

Huddle records and transcribes meetings with speaker labels, and the attendee list comes from the calendar invite. Today, people write their own notes after calls, and those notes are often late, incomplete, or missing. This feature generates a summary after each meeting that lists the decisions made and assigns action items to attendees. Action items are emailed to their owners, with Sales' requirement for immediate delivery balanced against the accuracy limits described in Section 4.

2. Goals and Non-Goals

Goals - Produce a summary within 5 minutes of meeting end for 95% of meetings. - Every action item names an owner who was present in the meeting. - Every decision and action item links to the transcript moment that supports it. - Give Sales automatic, immediate delivery for high-confidence action items.

Non-Goals (v1) - Calendar write-back or task creation in external tools (Jira, Asana). Design for it, but do not build it. - Summaries for meetings without a transcript. - Sentiment, coaching, or participation analytics.

3. Definitions

• Attendee: A person present in the meeting, determined by the participant session log (the platform's record of who joined), not only by the calendar invite. The calendar invite is a hint, not proof of attendance.
• Identified participant: A participant tied to a Huddle account or a verified email address.
• Unidentified participant: A guest who joined by name only, with no verified identity.
• Action item: A task with an owner, a description, and optionally a due date, derived from transcript content.
• Decision: A statement that the group agreed on a course of action, with supporting transcript evidence.

4. Known Limits and Design Responses

5. Functional Requirements

FR-1: Roster resolution Build the attendee roster from the participant session log at meeting end. Merge in calendar invitees for context, but do not treat invitees who never joined as attendees. Each roster entry is tagged as identified or unidentified.

FR-2: Summary generation Produce a summary containing: - A 3–5 sentence overview. - A decisions list. Each decision has a one-sentence statement, a timestamp, and a transcript link. Decisions are not assigned to individuals unless the transcript clearly states who made them. - An action items list (FR-3).

FR-3: Action item extraction Each action item includes: description, owner (a roster member), due date if stated, timestamp, transcript link, and attribution confidence (high, medium, low).

FR-4: Hard constraint: no non-attendee owners (must-pass) Before any summary is stored or sent, a deterministic validator checks that every action item owner matches a roster entry from FR-1. If an owner does not match, the item is removed and logged as a pipeline error. This check does not depend on the model or on confidence scores. The model is never the final authority on membership.

FR-5: Unidentified owners If the model attributes an action to an unidentified participant, the item is kept, but the owner is shown as "Unidentified participant (Name as joined)." These items are never emailed. They appear in the organizer's review view only.

FR-6: Confidence gating for delivery Each action item is sent automatically only if all of the following are true: - Owner is an identified roster member. - Attribution confidence is high. - The supporting transcript segment has no overlapping speech flag. - The item's source is not a multi-person room device.

Items that fail any check go to the organizer's review queue. The organizer can confirm, reassign to another roster member, or dismiss. Confirmed items are then sent.

FR-7: Multi-person sources When an action item comes from a multi-person source, the owner field is set to "Confirm owner" and the item is held for review. The review UI lets the organizer or the item's likely owner claim it from the roster.

FR-8: Delivery - Auto-sent items go to the owner's email within 5 minutes of meeting end. - Emails contain: the meeting name, the action item, the due date if any, a link to the summary, and a link to the transcript moment. - Each meeting produces one email per owner, grouping that owner's items. - Emails go only to identified roster members. Admins can disable delivery to external email domains.

FR-9: Review queue Organizers see a summary page with held items, can edit or confirm them, and can see which items were auto-sent. Edits made after sending generate a correction email to the owner.

FR-10: Controls and privacy - Admins can turn summaries on or off per workspace and per meeting type. - Any participant can opt out of their own speech being summarized. Their items are excluded from the summary and their name is removed from the roster. - Summaries and transcripts follow the same retention and access rules as the source recording.

6. Sales Requirement: Immediate Auto-Send

Sales wants action items emailed as soon as the meeting ends. This PRD supports that for high-confidence items (FR-6, FR-8). It does not send low-confidence items immediately, because a wrong owner gets a task they didn't agree to, and an email that is later corrected erodes trust. Sales can see the auto-send rate in the admin dashboard. If the held rate is too high for their workflows, the threshold can be tuned with data from the pilot. It should not be lowered without evaluation.

7. Acceptance Criteria

• AC-1: Across a 300-meeting eval set, zero action items name an owner absent from the participant log. This is a hard gate for launch.
• AC-2: Every decision and action item has a working transcript link that lands within 10 seconds of the cited statement.
• AC-3: For a labeled eval set, owner precision on auto-sent items is at least 97%.
• AC-4: Summaries are generated within 5 minutes of meeting end for 95% of meetings with transcripts under 90 minutes.
• AC-5: An uninvited identified joiner appears in the roster and can receive items. An unidentified joiner never receives email.
• AC-6: Opt-out removes the participant's items and name from the output within one processing cycle.

8. Metrics

• Adoption: Percentage of eligible meetings with a summary delivered.
• Quality: Action items edited or reassigned by the organizer, by confidence band.
• Trust: Post-send corrections per 100 auto-sent items. Target: under 2.
• Usage: Percentage of owners who open the linked transcript from an email.

9. Open Questions

1. Should the review queue notify the organizer by email, Slack, or in-app only?
2. When an owner is an external email address, what is the policy under the admin domain setting?
3. How will we handle meetings where the transcript contains no clear owner for a decision or action?
4. Do we need a per-workspace setting to require organizer approval for all items?
5. Should multi-person room devices be fixed at the hardware or SDK level, which would change FR-7?

10. Phasing

• Phase 1 (pilot, 20 workspaces): FR-1 through FR-5, FR-8 through FR-10, with all items held for organizer review. This validates the constraint and attribution quality.
• Phase 2: Enable confidence gating (FR-6) and auto-send for Sales and other pilot workspaces, based on Phase 1 precision data.
• Phase 3: Multi-person source handling at scale (FR-7), and general availability.