Write a PRD

Product Requirements Document: AI Meeting Summaries & Action Items

Product: Huddle Author: Product Management Target Audience: Engineering & Design Leads Status: Ready for Kickoff

---

1. Overview & Problem Statement

Huddle currently records and transcribes video meetings for 3,000 B2B customers. However, users still spend 10–15 minutes after every call reviewing transcripts or typing up raw notes to document decisions and action items.

We are building an AI-powered post-meeting summary feature that automatically extracts Key Decisions and Action Items (with assigned owners) from meeting transcripts.

The Core Tension: Speed vs. Accuracy

Sales has requested that we automatically email action items to their owners the moment a meeting ends. However, we have three strict operational realities: 1. Hard Constraint: We must never assign an action item to someone who did not attend the meeting. 2. Data Gap 1 (The 15% problem): 15% of meetings include uninvited guests (e.g., forwarded links, conference rooms) who are missing from the calendar invite. 3. Data Gap 2 (The 8% problem): Speaker diarization/labels are wrong 8% of the time, primarily due to crosstalk or multiple people sharing one room mic.

Product Decision: If we auto-send emails immediately after the meeting ends, the 8% speaker attribution error rate and the 15% calendar mismatch rate will inevitably cause us to violate our hard constraint or embarrass users by emailing the wrong people. Therefore, v1 will use a "Human-in-the-Loop" flow with a fallback auto-send. For high-confidence extractions, emails will send automatically after a 30-minute review window. For low-confidence extractions (crosstalk, unverified guests), we will hold the email until the meeting host approves the summary.

---

2. Goals & Success Metrics

---

3. System Architecture & The "Actual Attendee" Roster

Right now, Huddle relies on the calendar invite for the attendee list. To obey our hard constraint, Engineering must build a unified `Actual_Attendees` service that fires when a meeting ends.

Defining an "Actual Attendee"

A person is only eligible to be assigned an action item if they exist in the `Actual_Attendees` array for that specific meeting session. Calendar invitees who did not join the call are explicitly excluded.

1. Authenticated Huddle Users: Anyone who joins the call while logged into a Huddle account is added to `Actual_Attendees` (solves the 15% uninvited guest problem for logged-in users).
2. Unauthenticated Web Guests: Guests joining via link must type their display name in the pre-call lobby. They are added to `Actual_Attendees` with `email: null`.
3. Conference Room Hardware: When a room system joins (e.g., "4th Floor Conf Room"), the room itself is not an assignable person. (See Edge Cases in Section 4).

---

4. Functional Requirements (Engineering)

4.1 LLM Extraction Pipeline

* Trigger: Fires upon completion of the final meeting transcript. * Inputs: 1. Time-stamped transcript with speaker labels. 2. Diarization confidence scores per utterance. 3. The `ActualAttendees` JSON array (Name, UserID, Email, JoinMethod). 4. The `CalendarInviteesNoShow` array (Explicit negative constraint list). * Prompting & Guardrails: * Decisions: Extract 1–7 bullet points summarizing agreed-upon outcomes. Include timestamp citations. * Action Items: Extract tasks, deadlines (if mentioned), and the owner. * Assignment Logic: The LLM may only populate the `owner` field using an exact match from `ActualAttendees`.

4.2 Handling the Constraints & Edge Cases

• Scenario A: Someone on the call says, "Let's have Sarah do this," when Sarah is NOT in `Actual_Attendees`.
• System Behavior: The system must not assign the task to Sarah. Instead, it assigns the task to the speaker who volunteered Sarah (or leaves it "Unassigned") and rewrites the task text to: "Follow up with Sarah to [do X]."
• Scenario B: The 8% Speaker Label Error (Crosstalk / Low Confidence).
• System Behavior: If an action item is derived from an utterance where someone says "I will do X", the pipeline checks the diarization confidence score for that timestamp block. If confidence is `< 0.85` or overlaps with another speaker within 1.5 seconds, the system sets `owner: Unassigned`, sets `suggestedowner: [SpeakerID]`, and flags the item as `needs_review: true`.
• Scenario C: Multi-person Conference Rooms.
• System Behavior: Audio coming from a single hardware node containing multiple voices cannot reliably attribute "I'll take that" to an individual. If the speaker label is a Room Resource or has multi-voice flags, the LLM may only assign an action item if a name is spoken explicitly in the third person (e.g., "Thanks Dave, so Dave will pull the Q3 numbers" AND "Dave" is matched to a Huddle user via calendar invite + confirmation). Otherwise, set `owner: Unassigned` (`needs_review: true`).
• Scenario D: The 15% Uninvited Guest.
• System Behavior: If "Alex" joined via a forwarded link without logging in, he is in `ActualAttendees` as `{name: "Alex (Guest)", email: null}`. The system can assign action items to "Alex (Guest)" in the UI, but flags the item as `missingemail: true`.

4.3 The Email Dispatch Engine (Addressing the Sales Request)

To satisfy Sales without sending hallucinated assignments, we will implement a Two-Track Dispatch Engine:

• Track 1: High-Confidence Auto-Send (30-Minute Delay)
• Condition: All extracted action items have `owner != Unassigned`, `needsreview == false`, and `missingemail == false`.
• Behavior: The host gets an immediate Slack/Email ping: "Your meeting summary is ready. Action items will be emailed to owners in 30 minutes unless you edit them." If the host makes no changes within 30 minutes (or clicks "Send Now"), emails are dispatched to action item owners.
• Track 2: Low-Confidence Hold
• Condition: One or more action items are flagged `needsreview: true` or `missingemail: true`.
• Behavior: Auto-send is paused. The host is notified: "Your meeting summary has 2 action items that need your confirmation before we email the team." Emails only trigger once the host resolves the flags and clicks "Approve & Send".

---

5. User Experience & Design Requirements

5.1 Pre-Meeting Lobby (Small tweak for Guest ID)

• Unauthenticated Guests: Update the pre-join screen. Require both Full Name and Work Email (Optional) before joining. Explaining "Used to send you meeting notes" will convert a high percentage of the 15% uninvited guests into identifiable users.

5.2 Post-Meeting Summary View

The Meeting Host (and participants) will access the summary inside the Huddle web/desktop app under the "Past Meetings" tab.

* Layout: Split screen. Left side: Video player + searchable transcript. Right side: AI Summary widget. * Summary Widget Sections: 1. Status Banner: Shows dispatch state (e.g., "Auto-sending to 4 owners in 24 mins [Send Now] [Pause]" OR "Approval Required before emailing"). 2. Decisions Made: Bulleted list. Hovering over a decision highlights the corresponding timestamp in the transcript and lets the user click to play that 15-second clip. (Crucial for building user trust). 3. Action Items List: Each row contains: * Checkbox (for completion tracking). * Task description (inline editable). * Assignee Dropdown Pill. * Source timestamp link.

5.3 The Assignee Dropdown & "Needs Review" UI

• Restricted Roster: When a user clicks the Assignee Pill to change or add an owner, the dropdown menu only lists people from the `Actual_Attendees` array.
• Design detail: Below a divider line at the bottom of the dropdown, show calendar invitees who didn't show up in greyed-out, unclickable text with a tooltip: "Did not attend meeting." (This prevents users from thinking the dropdown is broken while enforcing our constraint—unless someone was hiding in a conference room, see below).
• Conference Room Override: If a meeting included a Room system, include an option in the dropdown: `+ Add person who was in [Conf Room Name]`. Clicking this allows the host to pick from the org directory, explicitly confirming that the person was physically present in the room.
• Low-Confidence Visual State: Action items flagged with `needs_review: true` (due to the 8% diarization issue) should have an amber highlight around the Assignee Pill reading: `Suggested: [Name] ?`. Clicking it lets the host confirm with one click.

5.4 The Action Item Email

• Subject: `Action Items from [Meeting Title] ([Date])`
• Body:
• Personalized top section: "Assigned to You (2):" followed by their specific tasks.
• Context section below: "All Decisions & Team Action Items" so they have full context.
• Footer CTA: `[View Full Recording & Transcript in Huddle]`.

---

6. Out of Scope for v1

• 3rd-Party Task Manager Integrations: Pushing tasks to Jira, Asana, Linear, or Salesforce (planned for v1.1 after we validate extraction accuracy).
• Cross-Meeting Action Item Tracking: A global "My Tasks" dashboard across all meetings. v1 is strictly scoped to the single-meeting view and email dispatch.
• Voiceprint Enrollment: Having users record their voice to fix the conference room diarization problem. We will rely on UI fallbacks for v1.

---

7. Open Questions for Kickoff

1. Engineering: What is the p95 latency for transcript generation + LLM summary extraction on a 60-minute call? Sales expects "as soon as the meeting ends"—if our pipeline takes 8 minutes, do we need to adjust the 30-minute auto-send timer?
2. Design: If a meeting has zero decisions and zero action items (e.g., a casual 1:1 catch-up), what should the empty state look like, and should we suppress the post-meeting notification entirely?
3. Engineering: Can our current diarization vendor expose word-level confidence scores via their webhook, or do we need to calculate crosstalk purely by checking overlapping speaker timestamps?

PRD: AI Meeting Summaries — Decisions & Action Items

Sep 24, 2026 · @Martin Slaney

Summary

We will generate a summary after every recorded Huddle meeting that lists the decisions made and the action items, each with an owner who was verifiably in the meeting. The organizer reviews the summary and sends it in one click; nothing is emailed without that step in v1.

Problem. Today attendees write their own notes after the call. Notes are inconsistent, often missing, and action items lose their owners within a day. We already have what we need to fix this: a recording, a transcript with speaker labels, and the calendar attendee list.

Why this is not trivial. Two of those inputs are unreliable. Speaker labels are wrong about 8% of the time, mostly during crosstalk and when several people share a meeting-room device. About 15% of meetings include someone who joined without a calendar invite. A naive build would misassign owners often enough to lose trust in the first week.

Goals and metrics

The goal is summaries people send without rewriting, with zero actions assigned to non-attendees.

Goals

• Replace manual post-meeting notes for recorded meetings.
• Every action item has a correct owner or is explicitly marked unassigned.
• Get action items to owners within minutes of the meeting ending.

Non-goals (v1)

• Pushing tasks into Jira, Asana or Linear.
• Summaries for unrecorded meetings or live, in-call summaries.
• Improving diarization itself. We design around the 8% error rate, not fix it.

Success metrics

Owner precision is measured from organizer edits plus a weekly labelled sample of 200 meetings.

Users and core flow

The organizer owns the summary; attendees receive it.

• Organizer: reviews, fixes owners, sends. Usually the calendar invite's creator; can hand off to any attendee.
• Action owner: receives their items by email and in the app.
• Other attendees: receive the full summary once sent.

```mermaid flowchart LR A[Meeting ends] --> B[Transcript +<br/>join log ready] B --> C[Build verified<br/>roster] C --> D[Extract decisions<br/>and actions] D --> E[Roster filter +<br/>confidence check] E --> F[Organizer review] F --> G[Send to owners<br/>and attendees] ```

The roster filter runs as deterministic code after the model, never inside the prompt alone.² Target: summary ready for review within 5 minutes of meeting end for meetings up to 90 minutes.

Requirements

An owner can only be chosen from the verified roster: people who actually joined, not people who were invited.

Verified roster

1. R1. Build the roster from the join log, not the calendar invite.¹ Invited people who never joined are excluded. 2. R2. Each roster entry has an identity tier: - Verified: joined signed in to a Huddle account, or by a dial-in matched to a known number. - Guest: joined with a display name only (most of the 15% uninvited joiners). - Room: a meeting-room device; people behind it are unknown individuals. 3. R3. Match uninvited joiners to workspace accounts by signed-in account only. Never match on display name alone.

Summary content

1. R4. Decisions: a short statement of each decision, with a transcript timestamp link. No owner field.
2. R5. Action items: task, owner (or "Unassigned"), due date if said aloud, timestamp link, and confidence.
3. R6. Every item links to its evidence in the transcript so reviewers can check it in one click.

Assignment rules

1. R7. The model returns a proposed owner per item. Code then checks it against the roster. Any owner not on the roster is replaced with "Unassigned", and the named person is shown as a mention ("mentions: Dave, not in meeting").
2. R8. Owner comes from content first: a named request ("Priya, can you…") beats the speaker label of "I'll do it".
3. R9. Mark an owner low confidence when the evidence depends on a speaker label in an overlapping segment, a Room entry, or a Guest entry.
4. R10. Low-confidence items show as "Suggested: Priya" and need organizer confirmation before any email goes out.
5. R11. Room and Guest entries cannot receive email. The organizer can reassign to a verified person or leave it unassigned.
6. R12. Log every model proposal, filter result and organizer edit. This is our precision measurement.

Handling the known limits

Both limits hit ownership, not decisions, so we contain them at the owner step.

The error compounds per meeting. If each item's owner is wrong 8% of the time, a meeting with 5 action items has a 34% chance of at least one wrong owner (1 − 0.92⁵). That is why confirmation sits before email, not after.

Sales request: auto-email on meeting end

Decision: not in v1 as requested. We ship one-click organizer send now, and gated auto-send in phase 2 once measured precision earns it.

Why not now. An email to the wrong owner is the most visible failure this feature can have. With the error rate above, roughly a third of 5-item meetings would email someone a task that isn't theirs. With external guests, it may land outside the customer's company. Early trust is hard to win back.

What Sales gets instead.

• v1: the organizer gets a "Summary ready" notification within 5 minutes and sends with one click. Goal: sent within 30 minutes (see metrics).
• Phase 2: a workspace setting to auto-send at meeting end, but only items that are high confidence and owned by a Verified attendee. Everything else still waits for the organizer. It ships only when owner precision on high-confidence items is ≥ 97% for 4 consecutive weeks.
• Auto-send never emails Guest or Room entries, or anyone outside the customer's domain, unless an admin enables it.

Design notes

The review screen must make checking owners faster than writing notes.

• Decisions first, then action items grouped by owner, then Unassigned.
• Low-confidence owners look visibly different ("Suggested") and are editable inline from a roster picker, never free text.
• Each item plays its transcript clip on click.
• The Send button states who gets email: "Send to 4 owners and 7 attendees".
• Empty state: if no decisions or actions are found, say so plainly rather than inventing any.

Rollout

Open questions

• [ ] Is the join log's signed-in identity reliable enough to call "Verified"? Engineering to confirm.
• [ ] Who organizes when the invite creator didn't attend? Proposed: first Verified attendee from the invite.
• [ ] Do customers need to disable summaries per meeting for sensitive calls? Legal and Sales to confirm.
• [ ] Retention: how long do we keep drafts and the evidence log?

PRD: AI Meeting Summaries and Action Items

Product: Huddle Audience: Engineering and Design leads Status: Proposed MVP Launch decision: Ship reviewed summaries first; defer immediate, unreviewed owner emails.

1. Problem and opportunity

Huddle serves 3,000 companies. Meetings are recorded and transcribed with speaker labels, but attendees currently write their own notes afterward. This creates duplicated effort, inconsistent records of decisions, and missed follow-through.

We will generate a concise meeting summary containing decisions and action items, with evidence linking each item to the recording.

Two known limitations shape the design:

• Speaker labels are wrong approximately 8% of the time, especially during overlapping speech and shared-room participation.
• Approximately 15% of meetings include someone absent from the calendar invite.

Therefore, neither speaker labels nor calendar invitations are sufficient evidence for assigning work.

Non-negotiable requirement: An action must never be assigned to someone who did not attend the meeting. When attendance or identity cannot be established, leave the action unassigned.

2. Goals and non-goals

Goals

• Reduce time spent producing post-meeting notes.
• Capture decisions accurately, distinguishing decisions from proposals.
• Extract actionable commitments with owners and deadlines when supported.
• Include verified, uninvited attendees without treating all invitees as attendees.
• Make every decision and action inspectable and correctable.
• Deliver owner notifications only after safe assignment and publication.

Non-goals for MVP

• Live summaries or action assignment during meetings.
• Assigning work to absent colleagues, even if requested during the meeting.
• Inferring deadlines or assigning owners based on job title.
• External task-system integrations.
• Automatically publishing unreviewed summaries.
• Resolving identity using voice biometrics.

3. Users and primary workflow

Reviewer: Meeting organizer, or an explicitly designated attendee. Reviews and publishes the summary.

Attendee: Reads the summary, follows evidence links, and reports corrections.

Action owner: Receives published actions assigned to them.

Workflow:

1. Meeting ends; Huddle completes transcript processing.
2. Huddle reconciles actual attendance and generates a private draft.
3. Reviewer checks decisions, unresolved ownership, and attendance exceptions.
4. Reviewer edits and publishes.
5. Huddle makes the summary available to authorized viewers and, where enabled, emails owners their approved actions.

If no eligible reviewer is available, the draft remains unpublished. A non-attending organizer may designate an attending reviewer but cannot resolve attendance exceptions themselves.

4. Attendance and identity requirements

Engineering must establish an attendance ledger separate from the calendar invite.

Each record contains:

• Meeting ID and stable person ID, where resolved.
• Join/leave intervals and participant-session IDs.
• Identity source: authenticated join, verified guest identity, or shared-room roster.
• Verification state and evidence.
• Reviewer confirmation details, where required.

Calendar invitees are suggestions for identity matching, not proof of attendance. A directory search result is also not attendance evidence.

Assignment eligibility

An owner must map to a unique person with verified attendance in that meeting:

• Authenticated individual join: Eligible when attendance telemetry records the person’s session.
• Guest join: Eligible only after identity is resolved and verified; a display name alone is insufficient.
• Shared meeting room: The room endpoint proves room participation, not individual presence. A participating reviewer must explicitly confirm each person physically present before they become eligible.
• Uninvited attendee: Eligible through the same verification rules; invitation status is irrelevant.
• Invited but absent person: Ineligible.

If telemetry is unavailable, identity is ambiguous, or attendance is disputed, block assignment. Keep the action unassigned with an explanation.

The owner picker must show only eligible attendees. Every write path—including generation, manual edits, APIs, and publication—must enforce eligibility server-side. Model confidence must never override this rule.

These controls cannot independently observe who is physically in a room. If trustworthy roster confirmation is unavailable, shared-room individuals remain ineligible rather than weakening the constraint.

5. Summary and extraction requirements

The draft has four sections:

1. Overview: Two to four sentences describing the meeting’s purpose and outcome.
2. Decisions: Clear statements of agreed outcomes.
3. Action items: Task, owner or “Unassigned,” deadline if explicit, and status.
4. Needs review: Uncertain decisions, ownership, identity, or attendance.

Each decision and action must include transcript-span references and recording timestamps. Preserve the original supporting text even when the reviewer edits the final wording.

Decisions

Capture explicit agreement or a clearly accepted conclusion. Do not convert brainstorming, recommendations, or unresolved debate into decisions. If discussion reverses an earlier decision, represent the final outcome and retain supporting evidence.

Actions

Extract a concrete deliverable or next step. Record a deadline only when explicitly stated; normalize relative dates using the meeting’s date and timezone.

Examples:

• “I’ll send the revised deck tomorrow.” → Task with a candidate owner, subject to attribution review.
• “Someone should update the deck.” → Unassigned action.
• “Ask Priya to update it,” when Priya was absent → Unassigned action; note that an absent person was mentioned.
• “We could revisit this next quarter.” → Not an action without a clear commitment.

Speaker labels provide evidence, not authority. Overlap, shared-room speech, ambiguous pronouns, and conflicting commitments must trigger ownership review. Candidate owners are draft suggestions, not assignments, until approved.

6. Review experience

The summary page opens in Draft state with a clear “Review before sharing” banner.

Design must provide:

• Editable decision and action cards.
• Evidence links opening the relevant transcript and recording segment.
• Visible distinction between verified attendance and uncertain attribution.
• “Unassigned” as a first-class owner option.
• A dedicated queue for identity and attendance exceptions.
• A publish preview showing recipients and notification settings.

For each assigned action, the reviewer must approve the owner. Bulk approval is permitted for straightforward items, but flagged attribution requires item-level review.

Attendance verification and ownership approval are separate controls: confirming that someone attended does not prove they accepted a task.

Publication is blocked if any assigned owner is ineligible. Unassigned actions and unresolved decision candidates do not block publication; uncertain decisions must be omitted from the final Decisions section or explicitly labeled unresolved.

7. Email policy and Sales request

Sales requests automatic emails immediately after each meeting. MVP will not send unreviewed action assignments. Given the known speaker errors, immediate delivery would spread incorrect ownership before correction.

Instead:

• Notify the reviewer when the draft is ready.
• Default owner emails to off; enable through workspace settings.
• When enabled, automatically email approved owners after publication.
• Send each owner only their actions, meeting title, deadlines, and an authorized summary link.
• Never email a calendar invitee solely because they were invited.
• Do not send to guests without a verified address and permitted access.

Email dispatch must recheck eligibility and publication version. Use idempotency keys to prevent duplicate sends. Published ownership changes generate a correction for the previous recipient and an updated notification for the new eligible owner.

Future immediate automation requires a separate launch decision and evidence that it preserves the attendance constraint.

8. Technical implementation

Use an asynchronous pipeline:

`Meeting ended → Transcript ready → Attendance reconciled → Draft generated → Reviewed → Published → Notifications`

Proposed records:

• `Summary`: meeting ID, transcript version, state, generated content, revision.
• `Decision`: text, evidence spans, review state.
• `Action`: task, proposed owner, approved owner, deadline, evidence, review state.
• `AttendanceRecord`: identity, evidence, verification state.
• `AuditEvent`: actor, change, timestamp, revision.

Generation returns schema-validated structured data. Treat transcript content as untrusted input: instructions spoken during a meeting must not alter system rules, access controls, or recipient selection.

Owner writes and publication run through a centralized eligibility validator. Notification jobs check the current revision before sending. Attendance corrections invalidate affected assignments and cancel queued notifications. Maintain an audit trail rather than silently rewriting history.

9. Reliability, privacy, and access

Target draft availability: within five minutes of transcript completion for 95% of supported meetings.

Generation failures must leave recordings and transcripts usable, show retry status, and send no owner emails.

Drafts are visible only to reviewers. Published summaries inherit meeting-content access rules; attendance alone does not grant access. Recording evidence links must enforce existing permissions. Apply workspace retention and deletion policies to summaries, attendance evidence, and generated artifacts.

10. Success metrics and acceptance

Proposed launch gates

• Zero assignments to ineligible attendees across automated invariant tests.
• Zero emails to ineligible owners in the pilot; any confirmed incident pauses rollout.
• At least 90% of published decisions require no substantive reviewer correction.
• Median review time below three minutes for meetings under one hour.
• At least 30% less self-reported note-taking time in the pilot.

Measure owner-correction rate separately from attendance violations: a wrong attending owner is still a quality failure.

Required acceptance cases

Test absent invitees, verified uninvited guests, shared rooms, overlapping speakers, duplicate names, failed identity resolution, missing telemetry, absent people named in requests, transcript reprocessing, attendance revoked before dispatch, and repeated notification retries.

In every uncertain-attendance case, expected behavior is an unassigned action and no owner email.

11. Rollout and dependencies

Start with internal meetings, then an opt-in pilot across varied company sizes and shared-room usage. Expand only after launch gates pass.

Before implementation, confirm availability of participant-session telemetry, guest identity verification, room-roster confirmation, reviewer permissions, and verified email addresses. If these dependencies are missing, ship summaries with unassigned actions for affected participants rather than substituting calendar membership as attendance proof.