Introduction
# Topic Monitor
**Monitor what matters. Get notified when it happens.**
Topic Monitor transforms your assistant from reactive to proactive by continuously monitoring topics you care about and intelligently alerting you only when something truly matters.
---
## ⚡ Quick Start (New in v1.2.0!)
**Just want to monitor one topic? One command:**
```bash python3 scripts/quick.py "AI Model Releases" ```
That's it! This creates a topic with sensible defaults: - **Query:** Auto-generated from topic name - **Keywords:** Extracted from topic name - **Frequency:** Daily - **Importance:** Medium - **Channel:** Telegram
### Quick Start Options
```bash # Basic - just a topic name python3 scripts/quick.py "Bitcoin Price"
# With keywords python3 scripts/quick.py "Security CVEs" --keywords "CVE,vulnerability,critical"
# High priority, hourly checks python3 scripts/quick.py "Production Alerts" --frequency hourly --importance high
# Custom query python3 scripts/quick.py "Competitor News" --query "CompanyName product launch funding"
# Different channel python3 scripts/quick.py "Team Updates" --channel discord ```
### Quick Start vs Full Setup
| Feature | Quick Start | Full Setup | |---------|-------------|------------| | Speed | ⚡ 1 command | 📝 Wizard | | Defaults | Smart | Customizable | | Use case | Single topic | Multiple topics | | Configuration | Minimal | Full control |
**After Quick Start, you can always customize:** ```bash python3 scripts/manage_topics.py edit ai-model-releases --frequency hourly ```
---
## Core Capabilities
1. **Topic Configuration** - Define subjects with custom parameters 2. **Scheduled Monitoring** - Automated searches at configurable intervals 3. **AI Importance Scoring** - Smart filtering: immediate alert vs digest vs ignore 4. **Contextual Summaries** - Not just links—meaningful summaries with context 5. **Weekly Digest** - Low-priority findings compiled into readable reports 6. **Memory Integration** - References your past conversations and interests
---
## Full Setup (Interactive Wizard)
For configuring multiple topics or advanced options:
```bash python3 scripts/setup.py ```
The wizard will guide you through:
1. **Topics** - What subjects do you want to monitor? 2. **Search queries** - How to search for each topic 3. **Keywords** - What terms indicate relevance 4. **Frequency** - How often to check (hourly/daily/weekly) 5. **Importance threshold** - When to send alerts (low/medium/high) 6. **Weekly digest** - Compile non-urgent findings into a summary
The wizard creates `config.json` with your preferences. You can always edit it later or use `manage_topics.py` to add/remove topics.
**Example session:** ``` 🔍 Topic Monitor - Setup Wizard
What topics do you want to monitor? > AI Model Releases > Security Vulnerabilities >
--- Topic 1/2: AI Model Releases --- Search query for 'AI Model Releases' [AI Model Releases news updates]: new AI model release announcement Keywords to watch for in 'AI Model Releases'? > GPT, Claude, Llama, release
--- Topic 2/2: Security Vulnerabilities --- Search query for 'Security Vulnerabilities' [Security Vulnerabilities news updates]: CVE critical vulnerability patch Keywords to watch for in 'Security Vulnerabilities'? > CVE, vulnerability, critical, patch
How often should I check for updates? 1. hourly 2. daily * 3. weekly
✅ Setup Complete! ```
## Quick Start
Already know what you're doing? Here's the manual approach:
```bash # Initialize config from template cp config.example.json config.json
# Add a topic python3 scripts/manage_topics.py add "Product Updates" \ --keywords "release,update,patch" \ --frequency daily \ --importance medium
# Test monitoring (dry run) python3 scripts/monitor.py --dry-run
# Set up cron for automatic monitoring python3 scripts/setup_cron.py ```
## Topic Configuration
Each topic has:
- **name** - Display name (e.g., "AI Model Releases") - **query** - Search query (e.g., "new AI model release announcement") - **keywords** - Relevance filters (["GPT", "Claude", "Llama", "release"]) - **frequency** - `hourly`, `daily`, `weekly` - **importance_threshold** - `high` (alert immediately), `medium` (alert if important), `low` (digest only) - **channels** - Where to send alerts (["telegram", "discord"]) - **context** - Why you care (for AI contextual summaries)
### Example config.json
```json { "topics": [ { "id": "ai-models", "name": "AI Model Releases", "query": "new AI model release GPT Claude Llama", "keywords": ["GPT", "Claude", "Llama", "release", "announcement"], "frequency": "daily", "importance_threshold": "high", "channels": ["telegram"], "context": "Following AI developments for work", "alert_on": ["model_release", "major_update"] }, { "id": "tech-news", "name": "Tech Industry News", "query": "technology startup funding acquisition", "keywords": ["startup", "funding", "Series A", "acquisition"], "frequency": "daily", "importance_threshold": "medium", "channels": ["telegram"], "context": "Staying informed on tech trends", "alert_on": ["major_funding", "acquisition"] }, { "id": "security-alerts", "name": "Security Vulnerabilities", "query": "CVE critical vulnerability security patch", "keywords": ["CVE", "vulnerability", "security", "patch", "critical"], "frequency": "hourly", "importance_threshold": "high", "channels": ["telegram", "email"], "context": "DevOps security monitoring", "alert_on": ["critical_cve", "zero_day"] } ], "settings": { "digest_day": "sunday", "digest_time": "18:00", "max_alerts_per_day": 5, "deduplication_window_hours": 72, "learning_enabled": true } } ```
## Scripts
### manage_topics.py
Manage research topics:
```bash # Add topic python3 scripts/manage_topics.py add "Topic Name" \ --query "search query" \ --keywords "word1,word2" \ --frequency daily \ --importance medium \ --channels telegram
# List topics python3 scripts/manage_topics.py list
# Edit topic python3 scripts/manage_topics.py edit eth-price --frequency hourly
# Remove topic python3 scripts/manage_topics.py remove eth-price
# Test topic (preview results without saving) python3 scripts/manage_topics.py test eth-price ```
### monitor.py
Main monitoring script (run via cron):
```bash # Normal run (alerts + saves state) python3 scripts/monitor.py
# Dry run (no alerts, shows what would happen) python3 scripts/monitor.py --dry-run
# Force check specific topic python3 scripts/monitor.py --topic eth-price
# Verbose logging python3 scripts/monitor.py --verbose ```
**How it works:** 1. Reads topics due for checking (based on frequency) 2. Searches using web-search-plus or built-in web_search 3. Scores each result with AI importance scorer 4. High-importance → immediate alert 5. Medium-importance → saved for digest 6. Low-importance → ignored 7. Updates state to prevent duplicate alerts
### digest.py
Generate weekly digest:
```bash # Generate digest for current week python3 scripts/digest.py
# Generate and send python3 scripts/digest.py --send
# Preview without sending python3 scripts/digest.py --preview ```
Output format: ```markdown # Weekly Research Digest - [Date Range]
## 🔥 Highlights
- **AI Models**: Claude 4.5 released with improved reasoning - **Security**: Critical CVE patched in popular framework
## 📊 By Topic
### AI Model Releases - [3 findings this week]
### Security Vulnerabilities - [1 finding this week]
## 💡 Recommendations
Based on your interests, you might want to monitor: - "Kubernetes security" (mentioned 3x this week) ```
### setup_cron.py
Configure automated monitoring:
```bash # Interactive setup python3 scripts/setup_cron.py
# Auto-setup with defaults python3 scripts/setup_cron.py --auto
# Remove cron jobs python3 scripts/setup_cron.py --remove ```
Creates cron entries: ```cron # Topic Monitor - Hourly topics 0 * * * * cd /path/to/skills/topic-monitor && python3 scripts/monitor.py --frequency hourly
# Topic Monitor - Daily topics 0 9 * * * cd /path/to/skills/topic-monitor && python3 scripts/monitor.py --frequency daily
# Topic Monitor - Weekly digest 0 18 * * 0 cd /path/to/skills/topic-monitor && python3 scripts/digest.py --send ```
## AI Importance Scoring
The scorer uses multiple signals to decide alert priority:
### Scoring Signals
**HIGH priority (immediate alert):** - Major breaking news (detected via freshness + keyword density) - Price changes >10% (for finance topics) - Product releases matching your exact keywords - Security vulnerabilities in tools you use - Direct answers to specific questions you asked
**MEDIUM priority (digest-worthy):** - Related news but not urgent - Minor updates to tracked products - Interesting developments in your topics - Tutorial/guide releases - Community discussions with high engagement
**LOW priority (ignore):** - Duplicate news (already alerted) - Tangentially related content - Low-quality sources - Outdated information - Spam/promotional content
### Learning Mode
When enabled (`learning_enabled: true`), the system: 1. Tracks which alerts you interact with 2. Adjusts scoring weights based on your behavior 3. Suggests topic refinements 4. Auto-adjusts importance thresholds
Learning data stored in `.learning_data.json` (privacy-safe, never shared).
## Memory Integration
Topic Monitor connects to your conversation history:
**Example alert:** > 🔔 **Dirac Live Update** > > Version 3.8 released with the room correction improvements you asked about last week. > > **Context:** You mentioned struggling with bass response in your studio. This update includes new low-frequency optimization. > > [Link] | [Full details]
**How it works:** 1. Reads references/memory_hints.md (create this file) 2. Scans recent conversation logs (if available) 3. Matches findings to past context 4. Generates personalized summaries
### memory_hints.md (optional)
Help the AI connect dots:
```markdown # Memory Hints for Topic Monitor
## AI Models - Using Claude for coding assistance - Interested in reasoning improvements - Comparing models for different use cases
## Security - Running production Kubernetes clusters - Need to patch critical CVEs quickly - Interested in zero-day disclosures
## Tech News - Following startup ecosystem - Interested in developer tools space - Tracking potential acquisition targets ```
## Alert Channels
### Telegram
Requires OpenClaw message tool:
```json { "channels": ["telegram"], "telegram_config": { "chat_id": "@your_username", "silent": false, "effects": { "high_importance": "🔥", "medium_importance": "📌" } } } ```
### Discord
Agent-delivered (no webhook in skill config):
`monitor.py` emits `DISCORD_ALERT` JSON payloads, and OpenClaw sends them via the message tool. This matches the Telegram alert flow (structured output, no direct HTTP in skill code).
```json { "channels": ["discord"] } ```
SMTP or API:
```json { "channels": ["email"], "email_config": { "to": "[email protected]", "from": "[email protected]", "smtp_server": "smtp.gmail.com", "smtp_port": 587 } } ```
## Advanced Features
### Alert Conditions
Fine-tune when to alert:
```json { "alert_on": [ "price_change_10pct", "keyword_exact_match", "source_tier_1", "high_engagement" ], "ignore_sources": [ "spam-site.com", "clickbait-news.io" ], "boost_sources": [ "github.com", "arxiv.org", "official-site.com" ] } ```
### Regex Patterns
Match specific patterns:
```json { "patterns": [ "version \\d+\\.\\d+\\.\\d+", "\\$\\d{1,3}(,\\d{3})*", "CVE-\\d{4}-\\d+" ] } ```
### Rate Limiting
Prevent alert fatigue:
```json { "settings": { "max_alerts_per_day": 5, "max_alerts_per_topic_per_day": 2, "quiet_hours": { "start": "22:00", "end": "08:00" } } } ```
## Environment Variables
Configure these environment variables to customize topic-monitor:
| Variable | Default | Description | |----------|---------|-------------| | `TOPIC_MONITOR_TELEGRAM_ID` | — | Your Telegram chat ID for receiving alerts | | `TOPIC_MONITOR_DATA_DIR` | `.data/` in skill dir | Where to store state and findings | | `WEB_SEARCH_PLUS_PATH` | Relative to skill | Path to web-search-plus search.py | | `SERPER_API_KEY` / `TAVILY_API_KEY` / `EXA_API_KEY` / `YOU_API_KEY` / `SEARXNG_INSTANCE_URL` / `WSP_CACHE_DIR` | — | Optional search-provider vars passed via subprocess env allowlist |
**Example setup:** ```bash # Add to ~/.bashrc or .env export TOPIC_MONITOR_TELEGRAM_ID="123456789" export TOPIC_MONITOR_DATA_DIR="/home/user/topic-monitor-data" export WEB_SEARCH_PLUS_PATH="/path/to/skills/web-search-plus/scripts/search.py" ```
## State Management
### .research_state.json
Stored in `TOPIC_MONITOR_DATA_DIR` (default: `.data/` in skill directory).
Tracks: - Last check time per topic - Alerted URLs (deduplication) - Importance scores history - Learning data (if enabled)
Example: ```json { "topics": { "eth-price": { "last_check": "2026-01-28T22:00:00Z", "last_alert": "2026-01-28T15:30:00Z", "alerted_urls": [ "https://example.com/eth-news-1" ], "findings_count": 3, "alerts_today": 1 } }, "deduplication": { "url_hash_map": { "abc123": "2026-01-28T15:30:00Z" } } } ```
### .findings/ directory
Stores digest-worthy findings:
``` .findings/ ├── 2026-01-22_eth-price.json ├── 2026-01-24_fm26-patches.json └── 2026-01-27_ai-breakthroughs.json ```
## Best Practices
1. **Start conservative** - Set `importance_threshold: medium` initially, adjust based on alert quality 2. **Use context field** - Helps AI generate better summaries 3. **Refine keywords** - Add negative keywords to filter noise: `"keywords": ["AI", "-clickbait", "-spam"]` 4. **Enable learning** - Improves over time based on your behavior 5. **Review digest weekly** - Don't ignore the digest—it surfaces patterns 6. **Combine with personal-analytics** - Get topic recommendations based on your chat patterns
## Integration with Other Skills
### web-search-plus
Automatically uses intelligent routing: - Product/price topics → Serper - Research topics → Tavily - Company/startup discovery → Exa
### personal-analytics
Suggests topics based on conversation patterns: > "You've asked about Rust 12 times this month. Want me to monitor 'Rust language updates'?"
## Privacy & Security
- **All data local** - No external services except search APIs - **State files gitignored** - Safe to use in version-controlled workspace - **Memory hints optional** - You control what context is shared - **Learning data stays local** - Never sent to APIs - **Subprocess env allowlist** - monitor forwards only PATH/HOME/LANG/TERM and search-provider keys - **No direct HTTP in skill code** - alerts are emitted as JSON for OpenClaw delivery
## Troubleshooting
**No alerts being sent:** - Check cron is running: `crontab -l` - Verify channel config (Telegram chat ID, topic channel list for Discord/email) - Run with `--dry-run --verbose` to see scoring
**Too many alerts:** - Increase `importance_threshold` - Add rate limiting - Refine keywords (add negative filters) - Enable learning mode
**Missing important news:** - Decrease `importance_threshold` - Increase check frequency - Broaden keywords - Check `.research_state.json` for deduplication issues
**Digest not generating:** - Verify `.findings/` directory exists and has content - Check digest cron schedule - Run manually: `python3 scripts/digest.py --preview`
## Example Workflows
### Track Product Release
```bash python3 scripts/manage_topics.py add "iPhone 17 Release" \ --query "iPhone 17 announcement release date" \ --keywords "iPhone 17,Apple event,September" \ --frequency daily \ --importance high \ --channels telegram \ --context "Planning to upgrade from iPhone 13" ```
### Monitor Competitor
```bash python3 scripts/manage_topics.py add "Competitor Analysis" \ --query "CompetitorCo product launch funding" \ --keywords "CompetitorCo,product,launch,Series,funding" \ --frequency weekly \ --importance medium \ --channels discord,email ```
### Research Topic
```bash python3 scripts/manage_topics.py add "Quantum Computing Papers" \ --query "quantum computing arxiv" \ --keywords "quantum,qubit,arxiv" \ --frequency weekly \ --importance low \ --channels email ```
## Credits
Built for ClawHub. Uses web-search-plus skill for intelligent search routing.