Introduction
# Zoom RTMS Meeting Assistant
Headless capture service for Zoom meetings using Real-Time Media Streams (RTMS). Receives webhook events, connects to RTMS WebSockets, records all media, and runs AI analysis via OpenClaw.
## Webhook Events Handled
This skill processes two Zoom webhook events:
- **`meeting.rtms_started`** — Zoom sends this when RTMS is activated for a meeting. Contains `server_urls`, `rtms_stream_id`, and `meeting_uuid` needed to connect to the RTMS WebSocket. - **`meeting.rtms_stopped`** — Zoom sends this when RTMS ends (meeting ended or RTMS disabled). Triggers cleanup: closes WebSocket connections, generates screenshare PDF, sends summary notification.
## Webhook Dependency
This skill needs a public webhook endpoint to receive these events from Zoom.
**Preferred:** Use the **ngrok-unofficial-webhook-skill** (`skills/ngrok-unofficial-webhook-skill`). It auto-discovers this skill via `webhookEvents` in `skill.json`, notifies the user, and offers to route events here.
Other webhook solutions (e.g. custom servers, cloud functions) will work but require additional integration to forward payloads to this service.
## Prerequisites
```bash cd skills/zoom-meeting-assistance-rtms-unofficial-community npm install ```
Requires `ffmpeg` for post-meeting media conversion.
## Environment Variables
Set these in the skill's `.env` file:
**Required:** - `ZOOM_SECRET_TOKEN` — Zoom webhook secret token - `ZOOM_CLIENT_ID` — Zoom app Client ID - `ZOOM_CLIENT_SECRET` — Zoom app Client Secret
**Optional:** - `PORT` — Server port (default: `3000`) - `AI_PROCESSING_INTERVAL_MS` — AI analysis frequency in ms (default: `30000`) - `AI_FUNCTION_STAGGER_MS` — Delay between AI calls in ms (default: `5000`) - `AUDIO_DATA_OPT` — `1` = mixed stream, `2` = multi-stream (default: `2`) - `OPENCLAW_NOTIFY_CHANNEL` — Notification channel (default: `whatsapp`) - `OPENCLAW_NOTIFY_TARGET` — Phone number / target for notifications
## Starting the Service
```bash cd skills/zoom-meeting-assistance-rtms-unofficial-community node index.js ```
This starts an Express server listening for Zoom webhook events on `PORT`.
**⚠️ Important:** Before forwarding webhooks to this service, always check if it's running:
```bash # Check if service is listening on port 3000 lsof -i :3000 ```
If nothing is returned, start the service first before forwarding any webhook events.
**Typical flow:** 1. Start the server as a background process 2. Zoom sends `meeting.rtms_started` webhook → service connects to RTMS WebSocket 3. Media streams in real-time: audio, video, transcript, screenshare, chat 4. AI processing runs periodically (dialog suggestions, sentiment, summary) 5. `meeting.rtms_stopped` → service closes connections, generates screenshare PDF
## Recorded Data
All recordings are stored organized by date: ``` skills/zoom-meeting-assistance-rtms-unofficial-community/recordings/YYYY/MM/DD/{streamId}/ ```
Each stream folder contains:
| File | Content | Searchable | |------|---------|-----------| | `metadata.json` | Meeting metadata (UUID, stream ID, operator, start time) | ✅ | | `transcript.txt` | Plain text transcript with timestamps and speaker names | ✅ Best for searching — grep-friendly, one line per utterance | | `transcript.vtt` | VTT format transcript with timing cues | ✅ | | `transcript.srt` | SRT format transcript | ✅ | | `events.log` | Participant join/leave, active speaker changes (JSON lines) | ✅ | | `chat.txt` | Chat messages with timestamps | ✅ | | `ai_summary.md` | AI-generated meeting summary (markdown) | ✅ Key document — read this first for meeting overview | | `ai_dialog.json` | AI dialog suggestions | ✅ | | `ai_sentiment.json` | Sentiment analysis per participant | ✅ | | `mixedaudio.raw` | Mixed audio stream (raw PCM) | ❌ Binary | | `activespeakervideo.h264` | Active speaker video (raw H.264) | ❌ Binary | | `processed/screenshare.pdf` | Deduplicated screenshare frames as PDF | ❌ Binary |
All summaries are also copied to a central folder for easy access: ``` skills/zoom-meeting-assistance-rtms-unofficial-community/summaries/summary_YYYY-MM-DDTHH-MM-SS_{streamId}.md ```
## Searching & Querying Past Meetings
To find and review past meeting data:
```bash # List all recorded meetings by date ls -R recordings/
# List meetings for a specific date ls recordings/2026/01/28/
# Search across all transcripts for a keyword grep -rl "keyword" recordings/*/*/*/*/transcript.txt
# Search for what a specific person said grep "Chun Siong Tan" recordings/*/*/*/*/transcript.txt
# Read a meeting summary cat recordings/YYYY/MM/DD/<streamId>/ai_summary.md
# Search summaries for a topic grep -rl "topic" recordings/*/*/*/*/ai_summary.md
# Check who attended a meeting cat recordings/YYYY/MM/DD/<streamId>/events.log
# Get sentiment for a meeting cat recordings/YYYY/MM/DD/<streamId>/ai_sentiment.json ```
The `.txt`, `.md`, `.json`, and `.log` files are all text-based and searchable. Start with `ai_summary.md` for a quick overview, then drill into `transcript.txt` for specific quotes or details.
## API Endpoints
```bash # Toggle WhatsApp notifications on/off curl -X POST http://localhost:3000/api/notify-toggle -H "Content-Type: application/json" -d '{"enabled": false}'
# Check notification status curl http://localhost:3000/api/notify-toggle ```
## Post-Meeting Processing
When `meeting.rtms_stopped` fires, the service automatically: 1. Generates PDF from screenshare images 2. Converts `mixedaudio.raw` → `mixedaudio.wav` 3. Converts `activespeakervideo.h264` → `activespeakervideo.mp4` 4. Muxes mixed audio + active speaker video into `final_output.mp4`
Manual conversion scripts are available but note that auto-conversion runs on meeting end, so manual re-runs are rarely needed.
## Reading Meeting Data
After or during a meeting, read files from `recordings/YYYY/MM/DD/{streamId}/`:
```bash # List recorded meetings by date ls -R recordings/
# Read transcript cat recordings/YYYY/MM/DD/<streamId>/transcript.txt
# Read AI summary cat recordings/YYYY/MM/DD/<streamId>/ai_summary.md
# Read sentiment analysis cat recordings/YYYY/MM/DD/<streamId>/ai_sentiment.json ```
## Prompt Customization
Want different summary styles or analysis? Customize the AI prompts to fit your needs!
Edit these files to change AI behavior:
| File | Purpose | Example Customizations | |------|---------|----------------------| | `summary_prompt.md` | Meeting summary generation | Bullet points vs prose, focus areas, length | | `query_prompt.md` | Query response formatting | Response style, detail level | | `query_prompt_current_meeting.md` | Real-time meeting analysis | What to highlight during meetings | | `query_prompt_dialog_suggestions.md` | Dialog suggestion style | Formal vs casual, suggestion count | | `query_prompt_sentiment_analysis.md` | Sentiment scoring logic | Custom sentiment categories, thresholds |
**Tip:** Back up the originals before editing, so you can revert if needed.