Preparing your docs for retrieval

When a visitor asks a question, the model does not see your documentation. It sees a handful of passages that a search picked out of it. The question is embedded, the vector store is searched inside your project's boundary, the shortlist is reranked down to roughly five passages, and those passages — nothing else — become the context the answer is written from.

Which means almost everything people try to fix with prompt engineering is actually a content problem. If the right passage was not retrieved, no instruction in the system prompt will conjure it. If the retrieved passage is ambiguous, the answer will be ambiguous in exactly the same way.

So the work is upstream: make sure the sentence that answers each common question exists, in one place, in a passage that can be retrieved on its own.

Start from your ticket queue, not your documentation tree. Take the twenty questions you answered most often last month and make sure each one has an answer in the knowledge base. That is a better first day than indexing four hundred pages.

• Your help centre or docs site, via a crawl — the bulk of the coverage.
• PDFs and internal documents nobody ever published: manuals, spec sheets, warranty terms.
• Q&A pairs for everything that was never written down — the real refund window, which plan includes the thing, what you actually do about a late delivery. One question, one answer, and they compete with crawled pages on relevance rather than ranking below them.
• Help-centre articles you publish in helpo, which are both the page a customer reads and a source the AI answers from, so there is no second copy to drift.
• Your store catalogue, if you have one.

Each source is split on paragraph boundaries into windows of roughly two thousand characters with a short overlap, then embedded. Nothing is retrieved as a whole document — a chunk is the unit that gets found, and a chunk has to make sense without the page around it.

That is why headings do real work. A page with clear section headings splits into chunks that each answer one question completely. A wall of undivided prose splits somewhere arbitrary, producing two chunks that each half-answer two questions, and neither scores well against either.

Three habits fix most of it. Put the answer in the first sentence under the heading, not in the paragraph that builds up to it. Write headings as the question a customer would ask, because the question is what gets embedded on the other side. And avoid pronouns that reach back across a heading — a chunk beginning "It also applies to annual plans" is unusable on its own.

Crawl anything that lives on the web and that you maintain, so it can be re-read on a schedule. Discovery reads your robots.txt and sitemap, expands a sitemap index, and shows you the full page list so you choose what goes in — indexing everything a sitemap mentions is how a careers page ends up answering a billing question.

Upload anything that is not on the web, or that is authoritative in a file rather than on a page.

Then deselect, because the wrong pages actively degrade answers rather than merely wasting space:

• Marketing and landing pages — they describe benefits, not behaviour, and the AI will quote the pitch back to a customer who has a problem.
• Changelogs and blog posts — they describe how the product used to work, and an old entry mentioning a feature by name will outrank the current documentation for that feature.
• Legal pages — long, rarely what anyone asked about, and they crowd out better passages on vague questions.

For a store, the highest-volume questions are about products, and no help centre contains the answers. A catalogue import reads the storefront's public product API so titles, descriptions, prices, variants and stock become answerable, and the AI can search that catalogue during a conversation and show a product card instead of describing a product in a paragraph.

Treat it like any other source: the answer quality is your product data's quality. A product with an empty description is a product the AI cannot talk about.

Every answer shows the passage it came from. That is a trust feature for the visitor, but it is a debugging tool for you, and it is the fastest way to improve a knowledge base.

When an answer is wrong, open the citation. Nearly always one of three things is true: the cited page is genuinely out of date, and the fix is to edit the page; the right page exists but a worse one was retrieved, and the fix is a clearer heading or a Q&A pair; or nothing in the knowledge base answers the question, and the fix is to write it. None of those three fixes is a prompt change, which is the point.

Below a confidence threshold, the right behaviour is to say so and hand the conversation to a person rather than produce a fluent guess. A refused answer costs you one ticket. An invented returns policy costs you the customer and the trust of everyone they tell.

So treat each refusal as a content bug with a stack trace attached. The visitor has told you, in their own words, about a gap in your documentation. The cheapest fix is usually a Q&A pair written in the words they used, which is live as soon as it is indexed.

Keeping sources current is the other half. A re-crawl fingerprints the normalised text and compares it with the last run: unchanged pages stop there and cost nothing to re-embed, changed pages are re-indexed atomically — new passages go live before the old ones are dropped, so the widget never sees a source with no content. A re-crawl that comes back empty fails rather than emptying the source, and a change that did land can notify your team, so a documentation edit that changes an answer is visible the moment it ships.

Fifteen minutes a week keeps a knowledge base ahead of the queue.

• Read the conversations where the AI handed off. Each one is either a gap or a genuinely human problem — sort them.
• Turn every gap into a Q&A pair, in the customer's phrasing rather than yours.
• Spot-check three answers against their citations. If a citation surprises you, the passage is wrong, not the model.
• Delete a source. Stale content is worse than missing content, because missing content produces a hand-off and stale content produces a confident wrong answer.