How to Build a Chatbot From Scratch

Step 0: prerequisites and project setup

You need Python 3.10 or newer, an OpenAI API key with a few dollars of credit on it, and content worth answering from. That last one is not a formality. A retrieval chatbot is a reading machine, and if there is nothing good to read it will be bad no matter how well the code is written.

Create a directory, a virtual environment and install five packages. Nothing here is a chatbot framework. They are an HTTP client, an HTML parser, an array library, a web server and the vendor SDK.

mkdir scratch-bot && cd scratch-bot
python -m venv .venv
source .venv/bin/activate        # Windows: .venv\Scripts\activate

pip install openai httpx beautifulsoup4 numpy "fastapi[standard]"

export OPENAI_API_KEY="sk-..."   # Windows: setx OPENAI_API_KEY "sk-..."

The shape of what you are building

Four files, each with one job. Read them in this order and nothing later depends on something you have not seen yet.

• ingest.py fetches your pages, splits them into chunks, embeds each chunk and writes index.json.
• bot.py loads that index, searches it for a question, and asks the model to answer from what it found.
• app.py exposes bot.py over a single HTTP endpoint.
• widget.html is the browser side, about 40 lines of plain JavaScript.

This architecture has a name, retrieval-augmented generation, and it is what nearly every serious documentation chatbot does underneath, including ours. If you want the conceptual version before the code, how chatbots work covers the same ground without a terminal.

What this actually costs to run

Two costs matter, and only one of them scales. Embedding your content is a one-off charge measured in cents. Answering questions is a per-conversation charge, and it is the number that decides whether this is worth running.

These are list rates before batching or prompt caching, and they are not directly comparable across vendors because the models differ in capability and tokenise text differently. Treat them as a planning starting point, not a benchmark.

A worked example

One answer from the code below sends about five chunks of roughly 300 tokens each, plus the system prompt and the question, so call it 1,750 input tokens and 150 output tokens. On gpt-5.6-luna that is about $0.0027 per answer, or roughly $2.65 per thousand answers. Indexing 200 pages of about 2,000 tokens each costs under a cent. Following this whole tutorial, including several hundred test questions, should cost you less than five dollars.

Model cost is not project cost. If you want the full picture including the engineering hours, the chatbot development cost breakdown and the chatbot ROI calculator are the honest versions of that arithmetic.

When writing this yourself is the wrong call

Everything after this section assumes you should write it. That assumption is worth testing now rather than in month three, because the code is the cheap part and the reasons not to write it get more expensive the later you notice them. Four stages, in escalating order.

Stage one: nobody is asking often enough for this to pay

Questions arrive in ones and twos, you know every answer, and replying takes less thought than configuring anything would. No retrieval pipeline beats that, and it will not start beating it later just because you enjoyed writing it. Automation pays on volume. Write the missing FAQ page instead, and if the questions stay quiet you have saved yourself a service to run.

Stage two: friction starts, and code is the wrong first fix

The same questions keep landing and answering them has become a chore. Opening an editor is the fun response. The dull one usually works better: write down the questions in the words customers actually use, find the page that answers each, write the ones that are missing, and put a search box over the result. That work is not wasted whatever you decide next, because it is exactly the source material this tutorial would have indexed. A bot has nothing to retrieve until somebody writes the content, and no chunking strategy compensates for a documentation gap.

Stage three: the prototype is live and nobody owns it

This is where hand-built bots do the most damage, and the reason is that the code below is easy to deploy. It works. Then attention moves elsewhere, and it sits in production with no owner, no re-ingest schedule, no spend alert and nobody reading transcripts. Content moves. The index does not. Dependencies drift, the score floor that looked right on your original pages is wrong on the ones you added since, and the first sign of trouble is a customer quoting a stale price back at you. None of that maintenance ever gets estimated, because it does not look like a task. It looks like something that will obviously get done.

Stage four: the question this design cannot answer

Then somebody asks where their order is. Or whether their plan renewed, or what the last invoice covered, or why the card was declined. Retrieval over public pages cannot reach any of it and the fix is not a better prompt. What it needs is identity for an anonymous visitor, an authenticated call into your own systems, a permission model, and a considered view of what a confidently wrong answer about somebody's money costs you. That is a different system from the one on this page. You can build it. Just be honest that you have crossed from a weekend project into software with an on-call rota, and price the crossing before you make it.

Steps 1 to 3: load your content, chunk it, embed it

Chunking is the step people underrate. Retrieval quality is set here, before any model is involved, and a bot that gives vague answers is usually a bot with bad chunks rather than a bot with a bad prompt.

The goal is passages that are self-contained. A chunk should make sense to somebody who reads only that chunk. Split on paragraph boundaries rather than on a fixed character count, keep a small overlap so a sentence spanning two chunks survives in one of them, and aim for something in the region of 1,000 to 1,500 characters. Those numbers are a starting point rather than a law. Reference documentation with short entries wants smaller chunks; long narrative prose wants larger.

ingest.py

# ingest.py - fetch pages, chunk them, embed the chunks, write index.json
import json
import re

import httpx
from bs4 import BeautifulSoup
from openai import OpenAI

client = OpenAI()  # reads OPENAI_API_KEY from the environment
EMBED_MODEL = "text-embedding-3-small"

PAGES = [
    "https://example.com/pricing",
    "https://example.com/docs/getting-started",
    "https://example.com/docs/refunds",
]


def fetch_text(url: str) -> str:
    """Fetch a page and strip it down to readable body text."""
    html = httpx.get(url, follow_redirects=True, timeout=30.0).text
    soup = BeautifulSoup(html, "html.parser")
    for tag in soup(["script", "style", "nav", "header", "footer", "form"]):
        tag.decompose()
    text = soup.get_text(separator="\n")
    return re.sub(r"\n{3,}", "\n\n", text).strip()


def chunk(text: str, target_chars: int = 1200, overlap_chars: int = 200) -> list[str]:
    """Group paragraphs into chunks of roughly target_chars, with overlap."""
    paragraphs = [p.strip() for p in text.split("\n\n") if p.strip()]
    chunks: list[str] = []
    current = ""
    for para in paragraphs:
        if current and len(current) + len(para) + 2 > target_chars:
            chunks.append(current)
            current = current[-overlap_chars:] + "\n\n" + para
        else:
            current = f"{current}\n\n{para}" if current else para
    if current:
        chunks.append(current)
    return chunks


def embed(texts: list[str]) -> list[list[float]]:
    """Embed in batches. The API accepts a list, so do not call it per chunk."""
    vectors: list[list[float]] = []
    for start in range(0, len(texts), 100):
        response = client.embeddings.create(
            model=EMBED_MODEL,
            input=texts[start : start + 100],
        )
        # Sort by index rather than trusting response order.
        for item in sorted(response.data, key=lambda d: d.index):
            vectors.append(item.embedding)
    return vectors


def main() -> None:
    records = []
    for url in PAGES:
        for piece in chunk(fetch_text(url)):
            records.append({"url": url, "text": piece})

    for record, vector in zip(records, embed([r["text"] for r in records])):
        record["embedding"] = vector

    with open("index.json", "w", encoding="utf-8") as handle:
        json.dump(records, handle)
    print(f"Indexed {len(records)} chunks from {len(PAGES)} pages.")


if __name__ == "__main__":
    main()

Run it with python ingest.py. You should get a few dozen chunks from three pages and an index.json a few megabytes in size, because each 1,536-dimension vector is a long list of floats in JSON.

Where this version is illustrative rather than production-ready

• The page list is hard-coded. A real ingester walks a sitemap, respects robots.txt, and handles the pages that only render content in JavaScript.
• A single paragraph longer than target_chars stays whole. Usually that is what you want. If your source has enormous paragraphs, split those on sentence boundaries first.
• JSON on disk is a fine store for a few thousand chunks and a bad one beyond that. The deployment notes below say what to move to.
• There is no incremental update. Every run re-embeds everything, which is cheap at this size and wasteful at scale.

Step 4: retrieval, the system prompt, and generation

Two functions. One finds the relevant passages, the other asks a model to answer using only those passages. The second one is where you decide whether your bot is trustworthy.

Search works by embedding the question with the same model used for the chunks, then comparing that vector against every stored vector. If you normalise all vectors to unit length once, cosine similarity is just a dot product, and a single matrix multiply scores the whole index at once. Brute force sounds naive and is genuinely fine up to a few tens of thousands of chunks, where the multiply takes milliseconds.

bot.py

# bot.py - search the index, then answer only from what was found
import json

import numpy as np
from openai import OpenAI

client = OpenAI()
EMBED_MODEL = "text-embedding-3-small"
CHAT_MODEL = "gpt-5.6-luna"
MIN_SCORE = 0.25  # tune this on your own data, see note below

with open("index.json", encoding="utf-8") as handle:
    RECORDS = json.load(handle)

# One (chunks x 1536) matrix, unit-normalised once at load, so that a dot
# product with a normalised question vector is the cosine similarity.
MATRIX = np.array([r["embedding"] for r in RECORDS], dtype="float32")
MATRIX /= np.linalg.norm(MATRIX, axis=1, keepdims=True)

IDK = "I do not have that in my documentation. I can pass this to a human if you like."

SYSTEM_PROMPT = f"""You are the support assistant for Example Co.

Answer using only the CONTEXT block in the user message. That context is the
complete set of facts available to you. Your own background knowledge about
this company is not reliable and must not be used.

Rules:
- If the context does not contain the answer, reply with exactly this and
  nothing else: {IDK}
- Never state a price, date, limit or policy that is not written in the context.
- After each sentence that uses a source, cite it as a bare URL in brackets.
- Keep answers under 120 words unless the user asks for more detail.
- If the question is not about Example Co, say that is all you cover, and stop.
"""


def search(question: str, k: int = 5) -> list[dict]:
    response = client.embeddings.create(model=EMBED_MODEL, input=question)
    query = np.array(response.data[0].embedding, dtype="float32")
    query /= np.linalg.norm(query)

    scores = MATRIX @ query
    best = np.argsort(-scores)[:k]
    return [
        {
            "url": RECORDS[i]["url"],
            "text": RECORDS[i]["text"],
            "score": float(scores[i]),
        }
        for i in best
        if scores[i] >= MIN_SCORE
    ]


def answer(question: str) -> dict:
    hits = search(question)
    if not hits:
        return {"text": IDK, "sources": [], "usage": None}

    context = "\n\n---\n\n".join(
        f"SOURCE: {hit['url']}\n{hit['text']}" for hit in hits
    )
    response = client.responses.create(
        model=CHAT_MODEL,
        instructions=SYSTEM_PROMPT,
        input=f"CONTEXT:\n{context}\n\nQUESTION: {question}",
    )
    return {
        "text": response.output_text,
        "sources": sorted({hit["url"] for hit in hits}),
        "usage": {
            "input_tokens": response.usage.input_tokens,
            "output_tokens": response.usage.output_tokens,
        },
    }


if __name__ == "__main__":
    import sys

    print(answer(" ".join(sys.argv[1:]) or "What is your refund policy?"))

Why the prompt is written that way

Every line in that system prompt is closing a specific failure. Telling the model its own knowledge is unreliable matters because a model asked about a company it has read about on the open web will happily answer from memory, and that memory may be years out of date. Giving it one exact refusal string means you can detect a refusal with a string comparison instead of trying to classify it later. Naming prices, dates and limits specifically works better than a general instruction not to guess, because those are the categories where a confident invention does real damage.

The score floor is the other half of the safety. Without it, the five nearest chunks are returned no matter how unrelated they are, and the model gets a context block full of noise it will try to be helpful with. With it, an off-topic question retrieves nothing and the bot refuses before spending a single generation token. The value of 0.25 is a starting point and nothing more. Run twenty questions you know the answer to and twenty you know it cannot answer, look at the scores, and put the threshold in the gap. If there is no gap, your chunks are the problem.

Using Claude instead

Only the generation call changes. Anthropic does not publish a first-party embeddings endpoint and points developers at Voyage AI for that, so in a mixed setup the embedding half stays where it is.

# pip install anthropic   (ANTHROPIC_API_KEY in the environment)
import anthropic

claude = anthropic.Anthropic()

message = claude.messages.create(
    model="claude-haiku-4-5",
    max_tokens=1024,
    system=SYSTEM_PROMPT,
    messages=[
        {"role": "user", "content": f"CONTEXT:\n{context}\n\nQUESTION: {question}"}
    ],
)
text = message.content[0].text
usage = {
    "input_tokens": message.usage.input_tokens,
    "output_tokens": message.usage.output_tokens,
}

Steps 5 and 6: an HTTP endpoint and a browser widget

The API key must never reach the browser. That single constraint is why there is a server in the middle at all, and it is the one thing on this page you cannot skip.

app.py

# app.py - run with: fastapi dev app.py
from fastapi import FastAPI, HTTPException
from fastapi.middleware.cors import CORSMiddleware
from pydantic import BaseModel, Field

from bot import answer

app = FastAPI(title="scratch-bot")

app.add_middleware(
    CORSMiddleware,
    allow_origins=["https://example.com"],  # never ["*"] in production
    allow_methods=["POST"],
    allow_headers=["Content-Type"],
)


class Ask(BaseModel):
    question: str = Field(min_length=1, max_length=500)


@app.post("/ask")
def ask(body: Ask) -> dict:
    question = body.question.strip()
    if not question:
        raise HTTPException(status_code=400, detail="Question is empty.")
    return answer(question)

Start it with fastapi dev app.py and you get the endpoint plus interactive docs at /docs, which is the fastest way to test before writing any front-end code. The 500-character cap on questions is not politeness, it is a cost control: without it, one person can paste a novel into your input box and you pay for the tokens.

widget.html

<div id="chat">
  <div id="chat-log" role="log" aria-live="polite"></div>
  <form id="chat-form">
    <input id="chat-input" maxlength="500" autocomplete="off"
           placeholder="Ask a question" aria-label="Ask a question" required>
    <button type="submit">Send</button>
  </form>
</div>

<script>
  var API = "https://your-api.example.com/ask";
  var log = document.getElementById("chat-log");
  var form = document.getElementById("chat-form");
  var input = document.getElementById("chat-input");

  function add(role, text) {
    var el = document.createElement("p");
    el.className = "msg " + role;
    el.textContent = text;   // textContent, not innerHTML: model output is untrusted
    log.appendChild(el);
    log.scrollTop = log.scrollHeight;
    return el;
  }

  form.addEventListener("submit", async function (event) {
    event.preventDefault();
    var question = input.value.trim();
    if (!question) return;

    add("user", question);
    input.value = "";
    var pending = add("bot", "Thinking...");

    try {
      var res = await fetch(API, {
        method: "POST",
        headers: { "Content-Type": "application/json" },
        body: JSON.stringify({ question: question })
      });
      if (!res.ok) throw new Error(res.status);
      var data = await res.json();
      pending.textContent = data.text;
      if (data.sources && data.sources.length) {
        pending.textContent += "\n\nSources: " + data.sources.join(", ");
      }
    } catch (err) {
      pending.textContent = "Something went wrong. Please try again.";
    }
  });
</script>

That is the whole chatbot. Using textContent rather than innerHTML is not a style preference: model output is text you did not write, and your own indexed content can contain markup, so rendering either as HTML is how a support widget becomes a cross-site scripting hole.

Deployment notes

• The index loads into memory at import time, so the process must be restarted after re-ingesting. Fine on one box, wrong the moment you run more than one worker.
• Past roughly fifty thousand chunks, replace the numpy matrix with a real vector store. pgvector if you already run Postgres, sqlite-vec if you want to stay on one file, Qdrant or a hosted equivalent if you want it managed. The search function is the only thing that changes.
• Answers take a few seconds. Stream the response if you can, or the widget feels broken even when it is working.
• Put the ingest script on a schedule. A bot answering from last quarter's pricing page is worse than no bot, because it is confidently wrong in public.
• Set a hard spend limit in your vendor dashboard on day one, before the endpoint is public.

What this tutorial deliberately does not cover

This is the section most tutorials leave out, which is why so many hand-built bots stall at 90 percent. Everything below is real work that stands between the code above and something you would put in front of paying customers.

• Session management. Each request here is independent, so the bot cannot handle "what about the annual plan?" as a follow-up. You need a conversation store, a policy for how much history to resend, and a decision about whether to rewrite follow-up questions into standalone ones before retrieval.
• Rate limiting and abuse. An open endpoint that calls a paid API is a bill somebody else can run up. Per-IP and per-session limits, a daily cap, and a circuit breaker that degrades to a contact form.
• Authentication. Nothing here distinguishes your visitors from a script. If the bot will ever touch account-specific data, that changes the entire design.
• Input sanitisation and prompt injection. A determined user will try to talk the model out of its instructions, and content you indexed from a page you do not control can carry instructions of its own. Strict retrieval limits the damage. It does not remove the class of problem.
• Cost monitoring. The code returns token counts and then throws them away. Log them per conversation, alert on the daily total, and watch for the single question that retrieves twenty chunks.
• Content re-ingestion. Detecting which pages changed, re-embedding only those, and swapping the index without downtime.
• Human escalation. Deciding when to hand over, capturing the conversation so the human is not starting cold, and routing it into whatever your team already uses.
• Analytics. Which questions get asked, which ones return nothing, which answers precede somebody leaving. Without this you cannot tell whether the bot is working, and the list of questions it failed is the most useful document it produces.
• Compliance. Data retention, deletion requests, disclosure that a visitor is talking to a bot, and where conversation logs are stored. This is jurisdiction-dependent and not something to improvise.

None of these are hard individually. Together they are usually a few weeks of engineering plus ongoing maintenance, and that maintenance never ends because your content never stops changing. If you have not budgeted for that, budget for it now rather than in month three.

What getting this wrong costs you later

The token table is the part of this decision you can put in a spreadsheet, which is why it gets more attention than it deserves. The costs that actually hurt are second order, and none of them arrives as an invoice.

The weekend that becomes a quarter is the familiar one. Day one goes so well that everyone extrapolates from it, including you. Then retrieval has to be tuned against real questions rather than the handful you tried, the chunker meets a page that is one enormous table, somebody senior wants a way to correct an answer without a deploy, and the follow-up question turns out to need a second model call nobody designed for. Every item is reasonable on its own. Together they are a roadmap entry you never wrote down, paid for out of whatever you were going to ship instead.

Then there is the prototype that quietly became production. It works, so it goes live, and because it was never formally a project it never gets an owner or a line in anybody's plan. Later the model it calls is superseded and the migration lands on whoever is nearest: re-test the prompt, confirm the refusal string still comes back word for word, check whether the scores shifted when the embedding model changed underneath you. That work has to sit on somebody's calendar. On this route it sits on yours. And the notice will not arrive at a convenient moment, because notices never do.

The mirror image is real too. Buy when you should have built and the one behaviour your product depends on turns out to be the one nobody exposes, which you discover after your content has already moved across. Neither mistake announces itself in month one. So the question worth asking before you write a line is not whether you can build this. It is who will still be maintaining it after your content, your models and your team have all changed. Do they know that yet?

Should you keep building, or use something off the shelf

Now that you know what the code looks like, the build-versus-buy question is answerable rather than theoretical. It comes down to whether the chatbot is part of your product or part of your overhead.

There is a middle path worth naming. Build the version above, run it internally for a fortnight, and read every transcript. You will learn more about what your customers actually ask than any vendor demo will tell you, and that knowledge is portable whichever route you take afterwards. If you then decide to buy, the Intercom alternatives comparison and the SaaS chatbot guide are the next things to read.

Frequently asked questions

Sources

• OpenAI API models and pricing - Model identifiers gpt-5.6-sol, gpt-5.6-terra and gpt-5.6-luna, and their list prices per 1M tokens.
• OpenAI text generation guide - That client.responses.create with instructions and input is the current recommended call, and that Chat Completions is the older API.
• OpenAI embeddings guide - The current embeddings call, the text-embedding-3-small model name and its $0.02 per 1M token list price.
• Anthropic model overview - Claude model identifiers and list prices for Opus 4.8, Sonnet 4.6 and Haiku 4.5.
• Anthropic embeddings documentation - That Anthropic does not offer a first-party embeddings endpoint and points developers to Voyage AI.
• Google Gemini API pricing - List prices for Gemini 2.5 Pro, Gemini 3.5 Flash and Gemini 2.5 Flash-Lite.
• FastAPI documentation - The fastapi[standard] install and the fastapi dev command used to run the API.