Adding AI Chat Features to a Modern Next.js Application

If one topic has been dominating the news lately, it has been AI. Today, we're diving into an exciting project combining the power AI brings with a modern web app. We will build an AI chat app using Next.js, leveraging Stream’s React Chat SDK and incorporating a sleek UI design inspired by a Dribbble concept⁠ by Tran Mau Tri Tam. Here is a video showing what we will build:

In this tutorial, we'll focus on creating a functional AI chat application with the following key features:

Listening to user messages to trigger LLM responses
AI-generated channel titles based on the prompt the user gives us
LLM-powered responses streamed to the chat channel⁠

Building AI chat applications presents several unique challenges. These include efficiently handling user messages, providing the chat history as the context for the LLM, and streaming AI responses for near-instant, natural interaction.

We'll use Next.js as our framework, showcasing state-of-the-art architecture for building modern web applications⁠⁠. While we'll use a local LLM for this tutorial, the principles we cover can be easily adapted to work with any AI provider that exposes a REST endpoint⁠.

So, whether you're looking to enhance your React and design skills, explore AI integration in web apps, or simply stay ahead of the curve in chat application development, this tutorial has something for you. Let's dive in and start building our AI-powered chat application!

Project Setup

Before we build specific AI features, we need to set up the project correctly. We provide a repository with the final code containing a starter branch. If you want to follow the flow of the post, you can start there and build your way up to the final application.

To set up the project, follow the following steps:

Clone the project repository (using the starter branch):

bash

1
git clone git@github.com:GetStream/nextjs-ai-chat-app.git -b starter

Set up your Stream project: Ensure you have a Stream project set up on the dashboard. If you haven't already, create a new project and set up a user⁠ with the ID TestUser.
Configure environment variables: In the project root, create a .env.local file and fill it with the necessary credentials:

tsx

1
2
NEXT_PUBLIC_STREAM_KEY=[Your Stream API Key]
STREAM_SECRET=[Your Stream API Secret]

Set up a local LLM tool: We'll use a local LLM for this project. We opted for LM Studio, which exposes a route for our application to interact with⁠, but there are many other LLM tools.

We need to create an additional AI user for our AI chat app to send the LLM-generated messages. We created another user called AI in our Stream dashboard⁠. We can give them a profile picture later in the chat.

Now that we have everything set up, let's build and run the project:

Install dependencies:

bash

1
2
3
npm install
# or
yarn install

Start the development server:

bash

1
2
3
npm run dev
# or
yarn dev

Our AI chat app should now be running on http://localhost:3000. We’re all set to start developing the more exciting features of our AI-powered messaging application!

In the next section, we'll dive into implementing real-time message listening.

Responding to User Messages Using an LLM

We need the ability to listen to new user messages in real-time to respond to them. In this section, we'll implement this functionality using React hooks and the Stream Chat SDK.

Let's break down the process step-by-step:

Set up the message listener in MyChat.tsx

We'll use the useEffect hook to set up our message listener. This ensures that we start listening for messages as soon as our component mounts:

tsx

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
useEffect(() => {
  const unsubscribeFunction = channel?.on('message.new', (event) => {
    const messageBody = event?.message?.text;
    if (event?.user?.id === user.id && messageBody) {
      fetch('/api/sendInitialAIResponse', {
        method: 'POST',
        headers: { 'Content-Type': 'application/json' },
        body: JSON.stringify({
          channelId: channel?.id,
          message: messageBody,
        }),
      });
    }
  });

  return () => {
    unsubscribeFunction?.unsubscribe();
  };
}, [channel, user.id]);

Here's what's happening in this code:

We use the useEffect hook to set up and clean up our event listener
We listen for the message.new event on the channel
When a new message arrives, we check if it's from the current user
If it is, we initiate an AI response using the sendInitialAIResponse API route
We return a cleanup function that removes the event listener when the component unmounts

Implement the AI response initiation

Next, let's implement the initiateAIResponse function. For that we create a new file called route.ts in the folder app/api/sendInitialAIResponse. We fill it up with the following code:

tsx

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
import { StreamChat } from 'stream-chat';

export async function POST(req: Request, res: Response) {
  const apiKey = process.env.NEXT_PUBLIC_STREAM_API_KEY;
  const streamSecret = process.env.STREAM_SECRET;
  const { channelId, message } = await req.json();

  const serverClient = StreamChat.getInstance(apiKey, streamSecret);
  const channel = serverClient.channel('messaging', channelId);

  const messageResponse = await channel.sendMessage({
    text: 'Thinking ...',
    user_id: 'ai',
    channelId: channelId,
    message: message,
    isStreaming: true,
  });

  return Response.json({});
}

This route handler does the following:

It checks if the necessary environment variables are there
It extracts the channelId and message from the request body
It sends a "Thinking..." message to the channel as the AI user
It returns a successful response

With these pieces in place, our AI chat app will now listen for new messages and respond with an initial "Thinking..." message. This provides immediate feedback to the user while we prepare the AI-generated response.

Notice that we also set the isStreaming parameter on the message, which we will later use to determine if we should ask for a real answer from the LLM.

In the following sections, we'll explore how to generate meaningful AI responses and stream them back to the client for a seamless chat experience.

Generate channel name from the first user message

One of the key features we'll implement is the ability to generate dynamic channel names based on a user's first message. This will add a personalized touch to each conversation and provide context at a glance.

Most of the necessary code will happen on the backend. Let's break down the implementation inside the route.ts file we just created in the sendInitialAIResponse folder:

Checking for the First Message

Building your own app? Get early access to our Livestream or Video Calling API and launch in days!

In our POST function, we first must determine if the current message is the first one in the channel. After receiving the channel object from the serverClient, we query it for the messages and extract the message count:

tsx

1
2
3
4
5
6
const messageQueryResponse = await channel.query({});
const messageCount = messageQueryResponse.messages.length;

if (messageCount === 1) {
    // Next code snippet goes here
}

Calling the LLM for Name Generation

We'll use our LLM REST endpoint to generate a concise summary if it's the first message. For that, we need to give clear instructions on what we want to achieve. We give it the following prompt and follow it up with the first user message (feel free to optimize it):

bash

1
2
3
4
Give a short summary of the prompt a user is giving you in 3-5 words, 
DO NOT USE MORE WORDS and do not include quotation marks for it. 
It will serve as the title of a chat channel. It needs to be precise 
and really capture the essence of the request. Here is the prompt:

We use this prompt to call the /chat/completions endpoint of the local LLM (in our case, running on localhost:1234 with our LM Studio setup):

tsx

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
const generatedTitle = await fetch(
  'http://localhost:1234/v1/chat/completions',
  {
    method: 'POST',
    headers: {
      'Content-Type': 'application/json',
    },
    body: JSON.stringify({
      messages: [
        {
          role: 'user',
          content: `Give a short summary of the prompt a user is giving you in 3-5 words, DO NOT USER MORE WORDS and do not include quotation marks for it. It will serve as the title of a chat channel. It needs to be precise and really capture the essence of the request. Here is the prompt: ${message}`,
        },
      ],
    }),
  }
);

Updating the Channel Name

Once we have our generated name, we can update the channel:

tsx

1
2
3
4
const generatedJson = await generatedTitle.json();
await channel.updatePartial({
  set: { name: generatedJson?.choices[0].message.content },
});

This process updates the channel name on the backend. Now, we want to reflect this on the client as well. Inside MyChat.tsx, we only want to display the MyChannelHeader component when a name is present in the channel.

For that, we update the line that simply displays the channel header to do this conditionally like this:

bash

1
{channel?.data?.name && <MyChannelHeader />}

By implementing this feature, we created a dynamic and context-aware chat experience. Users can quickly identify each conversation's topic, enhancing our messaging app's overall usability.

Stream LLM-Generated Answers to the Client

We’ve built our first AI integration with a helpful feature for the user. However, one of the critical features of our application is the ability to stream AI-generated responses in real-time. This creates a more engaging and dynamic user experience, similar to what you might see in popular AI chat interfaces like ChatGPT.

So, we want to integrate this into our application as well. Let's break down the implementation process:

1. Initiating the AI Response

When sending a response from the LLM, we already started by sending a Thinking message with an isStreaming flag set to true. This provides immediate feedback to the user that the AI is processing their request. So far, we’ve not been doing anything with that, so in the next step, we will change that.

2. Creating a helper for streaming a response

We want to stream the answer from the LLM to our client, and for that, we first create a helper generator function called streamingFetch, which takes a URL and a request as input and returns an iteration object:

tsx

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
export async function* streamingFetch(
  input: RequestInfo | URL,
  init?: RequestInit
) {
  const response = await fetch(input, init);
  const reader = response.body?.getReader();
  const decoder = new TextDecoder('utf-8');

  if (!reader) return;
  for (;;) {
    const { done, value } = await reader.read();
    if (done) break;

    try {
      yield decoder.decode(value);
    } catch (e: any) {
      console.warn(e.message);
    }
  }
}

Streaming the actual LLM response

We use two state variables to track the state of the response. First, the raw text of the LLM’s answer will be stored in streamingResponse, and then we will also keep track of whether the answer is complete with streamingFinished. We need this to update the final answer on the backend so that we don’t re-request an answer upon the next load.

Then, in the streamMessage function, we request an answer from our streamAIResponse route and extract an iterator using the streamingFetch function we just defined. We iterate over all the values we get and attach them to the previous value of our streamingResponse.

Here is the code for this:

tsx

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
const [streamingResponse, setStreamingResponse] = useState('');
const [streamingFinished, setStreamingFinished] = useState(false);

const streamMessage = useCallback(async (messageToRespondTo: string) => {
  const it = streamingFetch('/api/streamAIResponse', {
    method: 'POST',
    headers: { 'Content-Type': 'application/json' },
    body: JSON.stringify({
      message: messageToRespondTo,
    }),
  });

  for await (let value of it) {
    try {
      const jsonString = value.slice(5);
      setStreamingResponse((prev) => prev + chunk.choices[0].delta.content);
    } catch (e: any) {
      console.warn(e.message);
    }
  }

  setStreamingFinished(true);
}, []);

Initiating an answer from the LLM

This part is a simple useEffect hook that checks if the current message’s isStreaming property is set to true. If yes, it extracts the message and calls the streamMessage function we just defined:

tsx

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
useEffect(() => {
  const isStreaming = message.isStreaming;
  if (isStreaming) {
    const messageToRespondTo = message.message as string;
    if (messageToRespondTo) {
      streamMessage(messageToRespondTo);
    }
  }
}, [
  message,
  message.isStreaming,
  message.channelId,
  message.llmUrl,
  message.message,
  streamMessage,
]);

Updating the message after the entire response was streamed

The final code we need to add is to update the message when the LLM is finished streaming the response. We already have a route that handles that (see the route.ts file in /api/updateMessage).

We set the streamingFinished property to true when we received the last chunk of the answer to add another use effect hook listening for changes in that property. If it’s true we call the endpoint to update the message server-side:

tsx

1
2
3
4
5
6
7
8
9
10
11
12
useEffect(() => {
  if (streamingFinished) {
    fetch('/api/updateMessage', {
      method: 'POST',
      headers: { 'Content-Type': 'application/json' },
      body: JSON.stringify({
        messageId: message.id,
        message: streamingResponse,
      }),
    });
  }
}, [streamingResponse, streamingFinished, message.id]);

With our streaming responses, we update the final message server-side to avoid doing duplicate work.

By implementing this streaming approach, we create a more interactive and engaging chat experience. Users can see the AI's thought process in real-time, which can be particularly satisfying for longer or more complex responses.

Remember to handle potential errors and edge cases in your implementation. For example, you might want to implement a timeout mechanism if the LLM takes too long to respond or handle cases where the connection might be interrupted mid-stream.

Summary

Let's recap the exciting features we've built into our AI chat app:

Real-time Message Listening: We've implemented a system that instantly detects new messages, allowing seamless communication between users and AI.
Dynamic Channel Naming: Our app cleverly generates channel names based on the first message, providing context at a glance and enhancing the user experience.
Streaming AI Responses: We've created a lifelike chat experience by streaming AI-generated responses in real-time, mimicking human-like typing patterns.

These features create a powerful, engaging, and user-friendly messaging application. By implementing these techniques, we’re not just building a chat app but crafting an intelligent, responsive system that can adapt to user input and provide dynamic, context-aware interactions.

The Stream React Chat SDK provides an excellent foundation for developers looking to create their AI-powered projects. It offers robust real-time capabilities and seamless integration with AI services, allowing you to focus on creating unique features rather than wrestling with infrastructure.

As we have seen, it’s easy to integrate AI services seamlessly into the flow. We can use locally running LLMs, but the same principles apply when using popular third-party solutions like OpenAI or Gemini.

Remember, the world of AI and chat applications is constantly evolving. Stay curious, keep experimenting, and your next project could revolutionize our daily interactions with AI! If you’re building an exciting experience, please let us know by tagging us on X.