Chat with YouTube Videos 🎬

Want to talk to YouTube videos? This template builds a RAG (Retrieval-Augmented Generation) system that lets you chat with the Mastra AI YouTube channel using AI! Ask questions, get summaries, find topics - just like having a conversation with video transcripts.

🎯 Built for Mastra AI: This template works with the Mastra AI YouTube channel, but you can use it with any YouTube channel!

What This Does 🤖

This template creates a RAG chatbot that knows everything about YouTube videos:

🎯 Chat with Videos: Ask "What did they say about workflows?" and get smart answers from Mastra AI video transcripts.

📚 Learn Mastra: Build something fun while learning Mastra concepts like workflows, agents, and tools.

🔄 Use Any Channel: Want different videos? Just change the video IDs.

How It Works ✨

1. Process Videos: Download and transcribe Mastra AI YouTube videos (or any videos!)
2. Extract & Store: AI pulls out speakers, topics, and key insights, stores them as searchable chunks
3. RAG Chat: When you ask questions, it retrieves relevant transcript chunks first
4. Get YouTube Data: Uses the video IDs from retrieved chunks to fetch metadata, thumbnails, analytics via MCP

Why This is Cool 🚀

• 🎬 Talk to Videos: Get instant answers about Mastra AI content
• 🔍 Smart Search: Find videos by topic, speaker, or just ask in plain English
• 📚 Learn by Doing: Pick up Mastra skills while building something useful
• 🛠️ Ready to Use: Built for real use, not just demos

Quick Start 🏃‍♂️

What You Need 📋

• Node.js 20.9.0+
• PostgreSQL with pgvector
• OpenAI API key
• Deepgram API key
• Smithery.ai account

Get Started 🔧

# Get the code
git clone <repository-url>
cd chat-ytchannel
pnpm install

# Add your API keys
cp .env.example .env
# Edit .env with your keys

# Set up database
pnpm db:push

# Start it up!
pnpm dev

Your API Keys 🔑

Put these in your .env file:

# Database
DATABASE_URL=postgresql://postgres:postgres@localhost:5432/postgres

# AI Services
OPENAI_API_KEY=your_openai_key_here
DEEPGRAM_API_KEY=your_deepgram_key_here

# YouTube Access (you'll need YouTube Data API key when setting up Smithery)
SMITHERY_API_KEY=your_smithery_key_here
SMITHERY_PROFILE=your_profile_here

💡 Note: When setting up your Smithery account, you'll need a YouTube Data API key to access YouTube videos and their metadata.

How to Use It 🎮

1. Process Some Videos 🍿

First, let's teach the RAG system about some Mastra videos:

1. Start the server: pnpm dev
2. Go to the Mastra playground at http://localhost:4111/workflows
3. Find the transcript-workflow and run it
4. Provide these inputs:
   • videoId: Any YouTube video ID (e.g., "dQw4w9WgXcQ")
   • keywords: Array of relevant terms like ["AI", "Mastra", "workflows"]

This will:

• Download the video's audio
• Turn speech into text with Deepgram
• Extract topics, speakers, and summaries with AI
• Split into chunks and store with vector embeddings for RAG

2. Start Chatting! 💬

Once you've processed some videos, go to http://localhost:4111/agents/youtubeAgent/chat and try asking:

For Mastra AI videos:

• "What did they say about building agents?"
• "Find videos about workflows"
• "Summarize the latest tool tutorial"
• "Who talks about memory in the videos?"
• "Show me the most popular video about agents"

The RAG system works like this:

1. Retrieves transcript chunks that match your question from the vector database
2. Gets video IDs from those chunks
3. Fetches YouTube metadata (titles, descriptions, thumbnails, view counts) via MCP
4. Combines everything to give you complete answers with context

💡 Note: You need to process at least one video first using the workflow above, or you won't have any data to chat about!

3. Use Different Channels 🌟

Want to build RAG on other YouTube channels? Just process their videos using the same workflow in the playground with any YouTube video ID and relevant keywords.

What's Inside? 🔧

This template shows you how to use key Mastra features:

The Tech Stack 🛠️

• Workflows: Multi-step processes that handle errors and keep going
• Tools: Functions the AI can call to search and analyze videos
• Agents: Chatbots that remember conversations and use tools smartly
• MCP: Connect to external APIs (like YouTube) seamlessly

The Main Parts ✨

• transcriptWorkflow: Processes videos behind the scenes
• youtubeAgent: Your chat buddy who knows about videos
• videoSearchTool: Smart search that's both fast and accurate

What Powers It ⚡

• Database: PostgreSQL + pgvector for storing video transcripts and embeddings
• AI: OpenAI for chat + Deepgram for transcription
• YouTube: Access via Smithery.ai MCP servers (gets metadata, thumbnails, analytics)
• Framework: Mastra with TypeScript

Database 🗄️

Simple schema that stores everything you need:

-- Videos: transcripts and AI insights
CREATE TABLE videos (
  id TEXT PRIMARY KEY,              -- YouTube video ID
  fullTranscript TEXT,              -- Complete transcript
  metadata JSONB,                   -- AI insights (summary, speakers, topics)
  createdAt TIMESTAMP,
  updatedAt TIMESTAMP
);

-- Chunks: searchable pieces
CREATE TABLE chunks (
  id TEXT PRIMARY KEY,
  videoId TEXT REFERENCES videos(id),
  data JSONB,                       -- Chunk content + metadata
  createdAt TIMESTAMP
);

Customize It 🎨

Add More Metadata

// In transcript-workflow.ts, add more fields
const videoDataSchema = z.object({
  summary: z.string(),
  sentiment: z.enum(["positive", "negative", "neutral"]),
  difficulty: z.enum(["beginner", "intermediate", "advanced"]),
  keyTopics: z.array(z.string()),
  speakers: z.array(z.string()),
});

Add Search Filters

// In retrieval-tool.ts, add date filtering
const queryVideos = async (filters?: {
  speaker?: string;
  tag?: string;
  dateRange?: { start: Date; end: Date };
}) => {
  // Your filtering logic
};

Commands 💻

# Development
pnpm dev                    # Start with playground
pnpm build                  # Build for production
pnpm start                  # Run production

# Database
pnpm db:push               # Update schema
pnpm db:studio             # View data

Common Issues 🔧

"Failed to download video"

• Check the video ID is correct
• Some videos can't be downloaded
• Try a different video

"Database connection failed"

• Make sure PostgreSQL is running
• Check your DATABASE_URL
• Install pgvector extension

"No search results"

• Process some videos first
• Check your API keys work
• Make sure the database has data

What Makes This Special ✨

Learn by Building

• Real Example: Not just theory - you build something useful
• Mastra Patterns: See how workflows, tools, and agents work together
• Best Practices: Learn the right way to structure Mastra apps

Production Ready

• Error Handling: Things break, but the app keeps working
• Type Safety: TypeScript catches bugs before they happen
• Performance: Built to handle lots of videos efficiently

Easy to Extend

• Any Channel: Works with any YouTube channel
• Add Features: Easy to add new search filters or metadata
• Integration: Connect to other services via MCP

Ideas for More 💡

This template can become:

• Multi-channel processor - Handle entire YouTube channels
• Real-time updates - Auto-process new videos
• Analytics dashboard - Track trends and engagement
• Slack bot - Ask questions from your team chat
• API service - Power other apps with video intelligence

Learn More 📚

• Mastra Documentation
• PostgreSQL pgvector
• Deepgram API
• Model Context Protocol

Contributing 🤝

This is a learning template! Feel free to:

• Fork it and make it your own
• Submit improvements
• Share what you built with it

---

Built with ❤️ using Mastra - The AI framework that makes building actually fun.