Introduction
# Tavily AI Search
## Overview
Tavily is a search engine specifically optimized for Large Language Models and AI applications. Unlike traditional search APIs, Tavily provides AI-ready results with optional answer generation, clean content extraction, and domain filtering capabilities.
**Key capabilities:** - AI-generated answer summaries from search results - Clean, structured results optimized for LLM processing - Fast (`basic`) and comprehensive (`advanced`) search modes - Domain filtering (include/exclude specific sources) - News-focused search for current events - Image search with relevant visual content - Raw content extraction for deeper analysis
## Architecture
```mermaid graph TB A[User Query] --> B{Search Mode} B -->|basic| C[Fast Search<br/>1-2s response] B -->|advanced| D[Comprehensive Search<br/>5-10s response] C --> E[Tavily API] D --> E E --> F{Topic Filter} F -->|general| G[Broad Web Search] F -->|news| H[News Sources<br/>Last 7 days] G --> I[Domain Filtering] H --> I I --> J{Include Domains?} J -->|yes| K[Filter to Specific Domains] J -->|no| L{Exclude Domains?} K --> M[Search Results] L -->|yes| N[Remove Unwanted Domains] L -->|no| M N --> M M --> O{Response Options} O --> P[AI Answer<br/>Summary] O --> Q[Structured Results<br/>Title, URL, Content, Score] O --> R[Images<br/>if requested] O --> S[Raw HTML Content<br/>if requested] P --> T[Return to Agent] Q --> T R --> T S --> T style E fill:#4A90E2 style P fill:#7ED321 style Q fill:#7ED321 style R fill:#F5A623 style S fill:#F5A623 ```
## Quick Start
### Basic Search
```bash # Simple query with AI answer scripts/tavily_search.py "What is quantum computing?"
# Multiple results scripts/tavily_search.py "Python best practices" --max-results 10 ```
### Advanced Search
```bash # Comprehensive research mode scripts/tavily_search.py "Climate change solutions" --depth advanced
# News-focused search scripts/tavily_search.py "AI developments 2026" --topic news ```
### Domain Filtering
```bash # Search only trusted domains scripts/tavily_search.py "Python tutorials" \ --include-domains python.org docs.python.org realpython.com
# Exclude low-quality sources scripts/tavily_search.py "How to code" \ --exclude-domains w3schools.com geeksforgeeks.org ```
### With Images
```bash # Include relevant images scripts/tavily_search.py "Eiffel Tower architecture" --images ```
## Search Modes
### Basic vs Advanced
| Mode | Speed | Coverage | Use Case | |------|-------|----------|----------| | **basic** | 1-2s | Good | Quick facts, simple queries | | **advanced** | 5-10s | Excellent | Research, complex topics, comprehensive analysis |
**Decision tree:** 1. Need a quick fact or definition? → Use `basic` 2. Researching a complex topic? → Use `advanced` 3. Need multiple perspectives? → Use `advanced` 4. Time-sensitive query? → Use `basic`
### General vs News
| Topic | Time Range | Sources | Use Case | |-------|------------|---------|----------| | **general** | All time | Broad web | Evergreen content, tutorials, documentation | | **news** | Last 7 days | News sites | Current events, recent developments, breaking news |
**Decision tree:** 1. Query contains "latest", "recent", "current", "today"? → Use `news` 2. Looking for historical or evergreen content? → Use `general` 3. Need up-to-date information? → Use `news`
## API Key Setup
### Option 1: Clawdbot Config (Recommended)
Add to your Clawdbot config:
```json { "skills": { "entries": { "tavily": { "enabled": true, "apiKey": "tvly-YOUR_API_KEY_HERE" } } } } ```
Access in scripts via Clawdbot's config system.
### Option 2: Environment Variable
```bash export TAVILY_API_KEY="tvly-YOUR_API_KEY_HERE" ```
Add to `~/.clawdbot/.env` or your shell profile.
### Getting an API Key
1. Visit https://tavily.com 2. Sign up for an account 3. Navigate to your dashboard 4. Generate an API key (starts with `tvly-`) 5. Note your plan's rate limits and credit allocation
## Common Use Cases
### 1. Research & Fact-Finding
```bash # Comprehensive research with answer scripts/tavily_search.py "Explain quantum entanglement" --depth advanced
# Multiple authoritative sources scripts/tavily_search.py "Best practices for REST API design" \ --max-results 10 \ --include-domains github.com microsoft.com google.com ```
### 2. Current Events
```bash # Latest news scripts/tavily_search.py "AI policy updates" --topic news
# Recent developments in a field scripts/tavily_search.py "quantum computing breakthroughs" \ --topic news \ --depth advanced ```
### 3. Domain-Specific Research
```bash # Academic sources only scripts/tavily_search.py "machine learning algorithms" \ --include-domains arxiv.org scholar.google.com ieee.org
# Technical documentation scripts/tavily_search.py "React hooks guide" \ --include-domains react.dev ```
### 4. Visual Research
```bash # Gather visual references scripts/tavily_search.py "modern web design trends" \ --images \ --max-results 10 ```
### 5. Content Extraction
```bash # Get raw HTML content for deeper analysis scripts/tavily_search.py "Python async/await" \ --raw-content \ --max-results 5 ```
## Response Handling
### AI Answer
The AI-generated answer provides a concise summary synthesized from search results:
```python { "answer": "Quantum computing is a type of computing that uses quantum-mechanical phenomena..." } ```
**Use when:** - Need a quick summary - Want synthesized information from multiple sources - Looking for a direct answer to a question
**Skip when** (`--no-answer`): - Only need source URLs - Want to form your own synthesis - Conserving API credits
### Structured Results
Each result includes: - `title`: Page title - `url`: Source URL - `content`: Extracted text snippet - `score`: Relevance score (0-1) - `raw_content`: Full HTML (if `--raw-content` enabled)
### Images
When `--images` is enabled, returns URLs of relevant images found during search.
## Best Practices
### 1. Choose the Right Search Depth
- Start with `basic` for most queries (faster, cheaper) - Escalate to `advanced` only when: - Initial results are insufficient - Topic is complex or nuanced - Need comprehensive coverage
### 2. Use Domain Filtering Strategically
**Include domains for:** - Academic research (`.edu` domains) - Official documentation (official project sites) - Trusted news sources - Known authoritative sources
**Exclude domains for:** - Known low-quality content farms - Irrelevant content types (Pinterest for non-visual queries) - Sites with paywalls or access restrictions
### 3. Optimize for Cost
- Use `basic` depth as default - Limit `max_results` to what you'll actually use - Disable `include_raw_content` unless needed - Cache results locally for repeated queries
### 4. Handle Errors Gracefully
The script provides helpful error messages:
```bash # Missing API key Error: Tavily API key required Setup: Set TAVILY_API_KEY environment variable or pass --api-key
# Package not installed Error: tavily-python package not installed To install: pip install tavily-python ```
## Integration Patterns
### Programmatic Usage
```python from tavily_search import search
result = search( query="What is machine learning?", api_key="tvly-...", search_depth="advanced", max_results=10 )
if result.get("success"): print(result["answer"]) for item in result["results"]: print(f"{item['title']}: {item['url']}") ```
### JSON Output for Parsing
```bash scripts/tavily_search.py "Python tutorials" --json > results.json ```
### Chaining with Other Tools
```bash # Search and extract content scripts/tavily_search.py "React documentation" --json | \ jq -r '.results[].url' | \ xargs -I {} curl -s {} ```
## Comparison with Other Search APIs
**vs Brave Search:** - ✅ AI answer generation - ✅ Raw content extraction - ✅ Better domain filtering - ❌ Slower than Brave - ❌ Costs credits
**vs Perplexity:** - ✅ More control over sources - ✅ Raw content available - ✅ Dedicated news mode - ≈ Similar answer quality - ≈ Similar speed
**vs Google Custom Search:** - ✅ LLM-optimized results - ✅ Answer generation - ✅ Simpler API - ❌ Smaller index - ≈ Similar cost structure
## Troubleshooting
### Script Won't Run
```bash # Make executable chmod +x scripts/tavily_search.py
# Check Python version (requires 3.6+) python3 --version
# Install dependencies pip install tavily-python ```
### API Key Issues
```bash # Verify API key format (should start with tvly-) echo $TAVILY_API_KEY
# Test with explicit key scripts/tavily_search.py "test" --api-key "tvly-..." ```
### Rate Limit Errors
- Check your plan's credit allocation at https://tavily.com - Reduce `max_results` to conserve credits - Use `basic` depth instead of `advanced` - Implement local caching for repeated queries
## Resources
See [api-reference.md](references/api-reference.md) for: - Complete API parameter documentation - Response format specifications - Error handling details - Cost and rate limit information - Advanced usage examples
## Dependencies
- Python 3.6+ - `tavily-python` package (install: `pip install tavily-python`) - Valid Tavily API key
## Credits & Attribution
- Tavily API: https://tavily.com - Python SDK: https://github.com/tavily-ai/tavily-python - Documentation: https://docs.tavily.com