Imagine a language tutor, a startup coach, or a sales assistant, all available on-demand through a video call.
In this tutorial series from Code with Antonio, we'll guide you through building a video calling application powered entirely by AI agents.
Beyond generating summaries or transcripts after a call, every meeting in this app is a real-time, interactive conversation with a specialized AI.
Users can create custom agents with specific personalities and instructions, such as a "direct, hype-driven" founder persona for a coaching session.
After each call, the application automatically processes the meeting to provide a clean, timestamped summary, a fully searchable transcript, and a complete video replay. It even includes a ChatGPT-like interface that can answer questions about the meeting's content, powered by Stream's Chat SDK.
Built as a SaaS product, this platform runs on a subscription model. It offers a free trial and upgrade options managed through a seamless checkout process.
Let's explore the build.
Technology Stack
The build uses a modern tech stack optimized for real-time AI and SaaS features:
- Framework: Next.js 15 with React 19
- Type Safety: tRPC with TanStack Query
- Database & ORM: Postgres (from Neon) with Drizzle ORM
- Styling: Tailwind CSS v4 with ShadCN/UI for components
- Authentication: BetterAuth
- Payments: Polar
- Real-Time Communication: Stream for Chat and Video SDKs
- Background Jobs: Ingest and Agent Kit
- AI: OpenAI for real-time agents and features
- Code Reviews: CodeRabbit AI
- Version Control: A proper Git workflow with branches and pull requests for each chapter
Overview of the Build
0:00 Intro & Demo
Get a comprehensive overview of the final project, a SaaS video calling application where users interact with AI assistants in real-time. The demonstration highlights key features like custom AI agents, AI-powered summaries, and a full SaaS subscription model.
5:37 Setup
Walk through the initial project setup by confirming the Node.js version and initializing a new Next.js project with TypeScript and Tailwind CSS. Install and initialize ShadCN/UI to create a customizable component library. Conclude by committing the initial project and pushing it to a new GitHub repository.
20:57 Database (Neon, Drizzle ORM)
Establish the project's data layer by provisioning a serverless Postgres database using Neon. Set up and configure Drizzle as the ORM to interact with the database. Define an initial schema and push it to the live database to be verified in the Neon console and Drizzle Studio.
39:07 Auth Setup (Better Auth)
Integrate the BetterAuth library to handle user authentication. Generate the required database schema for auth using the BetterAuth CLI, configure the BetterAuth instance with a Drizzle adapter, and test the email/password signup flow.
01:10:19 Auth UI
Build the authentication UI using ShadCN/UI components and Next.js route groups for a shared layout. Implement client-side form validation with Zod and connect the forms to the authentication backend.
01:57:06 Auth Socials
Create OAuth apps and configure the providers in BetterAuth to add social login capabilities for GitHub and Google. Implement server-side route protection to secure the application based on the user's authentication status.
02:30:39 Dashboard Sidebar
Develop the main dashboard sidebar component and create a custom visual theme by modifying global CSS variables. Implement the user profile button with a dropdown menu for logout and a generated avatar for users without a profile picture.
03:14:04 Dashboard Navbar
Construct the top navigation bar with a button to toggle the sidebar's state. Implement a global command palette for search, accessible via a button and a keyboard shortcut (Cmd/Ctrl + K).
03:33:59 tRPC Setup
Integrate tRPC for end-to-end type-safe API communication. Install and configure tRPC with TanStack React Query for a Next.js App Router environment, setting up server and client utilities. Confirm the setup by calling a tRPC procedure from a client and server component.
03:58:42 Agents Setup
Establish the "Agents" feature by creating its database schema and tRPC procedures for data fetching. Build the main agents page, leveraging server-side prefetching with React Suspense to load and display agent data efficiently. Create reusable LoadingState and ErrorState components for a consistent UI.
04:34:02 Responsive Dialog
Develop a reusable ResponsiveDialog component that shows a desktop dialogue and a drawer on mobile for a better user experience. Apply this responsive pattern to the global command palette to improve its usability on smaller screens. Add a global CSS snippet to restore the cursor: pointer style on buttons for clearer visual feedback.
04:51:24 Agents Form
Implement a protected tRPC procedure to ensure only authenticated users can create agents. Build a reusable form within a responsive dialogue for creating new agents, complete with validation and a dynamic avatar preview. Upon successful creation, invalidate the getMany query to refresh the agent list automatically.
05:37:13 Agents Data Table
Using the Tanstack React Table library, display the list of agents in a data table. Create a reusable DataTable component and an EmptyState component to show when no agents exist. Fix a bug from the previous chapter by adding a mocked meetingCount to the getMany procedure so the table renders correctly.
06:06:31 Agents Filters
Add search and pagination functionality to the agents list. Integrate the nooks library to synchronize UI filter state with URL query parameters, and update the tRPC procedure to handle the filtered requests.
06:50:20 Agent Page
Create a secure dynamic detail page for individual agents so users can only view their own agents. Implement server-side prefetching for fast page loads and build the UI, including breadcrumbs and an action menu with edit/delete options.
07:15:55 Agent Update Delete
Implement the backend tRPC procedures for updating and deleting agents, ensuring users can only modify their agents. Build a reusable useConfirm hook for a confirmation dialogue before deletion and connect the UI to enable full CRUD functionality for agents.
07:37:21 Meetings Setup
Establish the "Meetings" entity by creating its database schema, including a status enum and relationships to users and agents. Set up the necessary pages, tRPC procedures, and initial client view with server-side prefetching for the meetings list.
07:54:18 Meetings Form
Develop the form for creating new meetings, which includes selecting an agent via a custom, searchable CommandSelect component. Implement the meetings.create and meetings.update protected tRPC procedures to handle the backend logic.
08:27:02 Meetings Data Table
Enhance the meetings.getMany tRPC procedure to perform an INNER JOIN with the agents table and calculate call duration. Display the list of meetings in a reusable data table, showing details like the meeting name, status, associated agent, and duration.
08:48:05 Meetings Filters
Add comprehensive filtering to the meetings list by meeting status and agent ID. Implement the filter UI and update the backend to handle these new query parameters, ensuring client and server states are synchronized. Make the filter bar horizontally scrollable on mobile devices to prevent layout issues.
09:13:43 Meeting Page
Develop the individual meeting detail page, including the meetings.remove tRPC procedure. Build the page UI with a header, breadcrumbs, and an action menu to edit or delete the meeting, connecting it to the backend logic.
09:33:00 Meeting Variants
Implement different UI states for the meeting detail page based on the meeting's status (e.g., Upcoming, Active, Canceled, Processing). Use custom SVG assets to visually represent each state, providing clear context to the user.
09:47:29 Video Call (Stream)
Integrate the Stream Video React SDK to add live video calling functionality. Set up the Stream client, generate user tokens for authentication, and create a full-screen call layout. Build the call UI, including a pre-join lobby for device setup, the active call interface with controls, and a post-call screen.
10:36:50 Connecting Agents (Stream, OpenAI)
Connect the AI agent to the live video call using the Stream OpenAI real-time API. Set up a webhook handler using ngrok to listen for Stream events, such as call.session_started, which triggers the agent to join the call. The agent's instructions are passed to the real-time client, enabling it to participate in the conversation.
10:56:01 Background Jobs (Inngest, OpenAI)
Implement background jobs using Inngest to process meetings after they have ended. Configure a webhook to listen for the call.transcription_ready event, which triggers an Inngest function. This function fetches the transcript, uses an OpenAI-powered Agent to generate a summary, and updates the meeting record in the database with the summary and a "completed" status.
11:23:33 Completed State
Build the UI for a completed meeting, organizing the AI-generated summary, video recording, searchable transcript, and an interactive AI chat into a tabbed interface. Integrate the Polar SDK to handle SaaS payments and subscriptions, gate features behind a premium tier, and perform final bug fixes on the global search and data displays. Finally, the application will be deployed to Vercel, configuring all production environment variables and updating webhook URLs in Stream, Google, and GitHub to bring the project live.
Note: The remaining sections are in Part 2 of the Build and Deploy a SaaS AI Agent Platform series.
1:39 Transcript Chat
Implement the final tabs for the completed meeting view, including a searchable transcript that highlights keywords and an interactive "Ask AI" chat. Use Stream Chat and a webhook to power the AI chat, allowing an OpenAI agent to respond to user questions with context from the meeting summary.
27:06 Payments
Integrate the Polar SDK to handle SaaS subscriptions and payments, configuring the BetterAuth plugin to automatically create and sync customers upon signup. Create a premiumProcedure in tRPC to gate features based on subscription status and build a pricing page that displays different plans and handles checkout and customer portal access.
01:25:06 Bugs
Perform final bug fixes before deployment by implementing the global search functionality in the dashboard's command palette to find meetings and agents. Correct the logic for calculating and displaying the number of meetings associated with each agent to ensure data accuracy across the application.
01:33:12 Deployment
Deploy the completed application to Vercel by importing the GitHub repository and adding all necessary environment variables for production. Update the webhook URLs in Stream, Google, and GitHub to point to the new live domain and connect the Inngest app for production-ready background job processing.
What's Next?
Congratulations! You've built a fully functional SaaS AI Agent Platform with:
🤖 Real-time video calls with customizable AI agents.
📝 Automated post-call summaries, recordings, and searchable transcripts.
💬 An interactive AI chat to ask questions about past meetings.
⚙️ A scalable backend with tRPC, Drizzle, and background job processing.
💳 A complete SaaS subscription model with payments via Polar.
🎨 A sleek, responsive UI built with Next.js 15 and Tailwind CSS.
Want to explore more features? Try adding:
- A collaborative whiteboard: Allow video participants to draw content simultaneously in real-time.
- Real-time language translation: Add a feature where the AI agent can translate the conversation for users in real-time.
- Accessibility features: Ensure your AI agent is easy for people of all abilities to use.
Check out part one and part two of the tutorial from Code with Antonio to follow along step-by-step, dive deeper into the code, and extend this project for your use case.