How an AI Chat App Connects to Language Models

The parts of an AI chat

When you send a message in a chat app, the model does not live inside the text box. The app, a server, and an AI service work together. The app shows the conversation. The server handles private work and sends a request to a model service. The service sends text back, and the app displays it.

An API is the agreed way for two programs to communicate. You can think of it as a menu: it says what you can ask for and what information you need to send. The app sends a request to the model API, and the API returns a response.

A message’s trip through the app

Here is a simplified version of the flow. It is an example to explain the idea, not a copy of Manshverse’s production code:

async function sendMessage(text) {
  showLoadingMessage();

  try {
    const response = await fetch("/api/chat", {
      method: "POST",
      headers: { "Content-Type": "application/json" },
      body: JSON.stringify({ message: text })
    });

    const result = await response.json();
    showReply(result.reply);
  } catch (error) {
    showError("Could not get a reply. Try again.");
  }
}

The app shows that it is working, sends the message, waits for the server, then displays the reply. await lets this function wait without freezing the whole page. The try and catch parts give the app a place to handle a failed connection instead of leaving the person staring at nothing.

A real request also needs to respect the conversation. The app may send earlier messages so the model knows what “that” or “the other option” refers to. The model can only use the context it receives, and long conversations have limits.

Choosing a model

Some apps offer more than one model. They might differ in speed, cost, or the kinds of tasks they handle well. In Manshverse, I worked on a feature called Mansh Mode: it looks at what a person is asking and routes the prompt to a suitable model. It is a bit like choosing the right tool for a job instead of using the same tool for every task.

This routing step is software around the model. The model itself does not decide which service the app should call; the application’s own rules make that choice. The chat can then send the prompt to the selected service and return the answer in the same conversation.

The rough edges matter

Networks are slow sometimes. Services can be busy, a request can be rejected, or the answer can take longer than expected. A good chat app needs a loading state, a useful error message, and a way to try again. It should also stop a very late answer from appearing in the wrong conversation.

There is an important security rule: a private API key should not be placed in browser JavaScript. Anyone can inspect code running in their browser. A server can keep the key private and make the request on the app’s behalf.

What I learned building it

Connecting a model is only one piece of an AI product. The surrounding details—keeping the chat readable, deciding what context to send, showing that a reply is on its way, handling failures, and choosing a model—are what turn a raw model connection into a feature someone can actually use.