---
type: blog
title: "AI Knowledge Base: Check the Sources Behind the Answer"
description: "Build an AI knowledge base around approved store documents, clear audience boundaries, and source checks that distinguish a retrieved document from a correct answer."
date: "2026-10-15"
lastModified: "2026-10-15"
authors: "Runner AI Team"
tags: ["AI Agents", "Knowledge Management", "Ecommerce Operations"]
featured: false
readTime: "6 min read"
thumbnail: "https://storage.googleapis.com/runner-blog/blog/ai-knowledge-base/cover"
thumbnailAlt: "A brass magnifying glass rests over a bookmarked document beside a kraft folder of ivory cards on a warm cream tabletop."
seo:
  title: "AI Knowledge Base: Sources, Permissions, and Answer Checks"
  description: "What is an AI knowledge base? Learn what to include, how retrieval differs from memory, and how to check sources before trusting an answer about your store."
---

An AI knowledge base is a collection of information an assistant can consult when answering questions or preparing work. For a store, that might include approved returns policies, product-care instructions, and brand guidelines. Its value is not the number of documents uploaded. It is whether the assistant receives the right source, respects its audience, and produces an answer that the source actually supports.

A document can be available without being read. It can be read without supporting every sentence in the answer. Keep those distinctions visible before you let an assistant turn a policy into customer-facing copy.

Disclosure: Runner AI publishes this guide. The Runner example describes current source code and recorded Preview QA, not a claim that the feature is available to every account.

## How an AI knowledge base works

A common approach is retrieval-augmented generation, or RAG. The application finds relevant material, supplies it to a language model, and asks the model to answer using that context. [Fin's knowledge-base guide](https://fin.ai/learn/ai-knowledge-base) describes this retrieval-first pattern and explains why content structure matters.

Retrieval methods differ. Some systems use semantic search to find related meanings. Others use text matching, tags, explicit document selection, or a combination. Do not assume that an AI label means every file is searched semantically or included in every answer.

The practical sequence is:

1. **Choose approved sources.** Decide which documents belong to the store and who may use them.
2. **Find relevant material.** Search or select the source for the question being asked.
3. **Supply context.** Pass the permitted content to the assistant.
4. **Generate an answer.** The model uses that context alongside its instructions and other available information.
5. **Check the result.** Compare consequential claims with the source and, where needed, live store records.

The last step does not disappear because retrieval succeeded. A source can be outdated, ambiguous, or about a different product.

## What belongs in a store's AI knowledge base?

Start with a small set of documents whose accuracy you can maintain. Keep each document focused on a question or decision rather than uploading a folder of overlapping drafts.

- **Returns policy.** Include eligibility, deadlines, exceptions, and the contact process. Do not infer an exception that the policy does not grant.
- **Product-care guide.** Record approved materials, cleaning instructions, and exclusions for the specific product or range.
- **Brand guidelines.** Include tone, naming, and approved examples. Writing style does not establish product facts.
- **Supplier notes.** Keep internal handling instructions separate from customer-facing material. Internal access must not imply shopper access.
- **Campaign brief.** Record the approved offer, dates, audience, and exclusions. Retire or revise it when the campaign changes.

A static document should not be the final authority for a changing stock count, an individual order's status, or the amount of a refund. Check the system that owns that record. The same separation applies when a [product appears out of stock](./product-out-of-stock): a general inventory explanation cannot establish whether a particular variant is currently buyable.

Give each document an owner and a review trigger. A policy change should trigger a policy review; a new product material should trigger a care-guide review. [Decagon's guide](https://decagon.ai/blog/knowledge-base-ai) discusses content audits and maintenance as part of preparing a knowledge base for AI. Automated suggestions can help identify work, but a person still needs to approve the business rule.

## Available, read, and supported are three different checks

Consider an illustrative returns question. A store has an approved document that says unworn items can be returned within 30 days of delivery. An internal supplier note discusses damaged packaging.

The assistant answers, "All items can be returned within 60 days, including worn products."

Finding the returns document in a source list would not make that answer correct. The deadline and eligibility both contradict the example policy.

Check each stage separately:

- **In the library** means the application stores the document. It does not establish that the current answer received it.
- **In search results** means retrieval found a candidate. The application may still exclude it from the answer's context.
- **Recorded as read** means the system supplied or recorded the document for the turn. It does not establish that every claim follows from its text.
- **Supported by a passage** means the reviewed claim matches the source. It does not authorize an order-specific action or prove that action completed.

Ask for the relevant passage as well as the source title. Then compare the deadline, product scope, conditions, and exceptions. A citation is a route to evidence, not a substitute for reading it.

## A concrete Runner example, with its limits

Runner's feature-gated Knowledge Library implementation illustrates the distinction between attaching a document and recording its use. In recorded Preview QA for the merged read-tracking change, a sent message retained the titles of its attached documents after a reload. The settled answer showed a **Read 2 Knowledge documents** control. Its list distinguished **Shared with shopper chat** from **Runner only** and offered links to open the documents.

That is evidence of source visibility, not an accuracy score. The implementation records document use for a turn; it does not prove that each generated sentence is entailed by a particular passage. A document attached earlier can also remain in the conversation context and appear for later answers. The count is not necessarily a count of newly searched documents.

The recorded checks also covered a removed document. It stayed visible as **Removed**, without an Open link. That preserves a reference to prior use; it does not mean the deleted body remains available to inspect.

As of this article's source review, this work remains behind a project-level feature gate, with the checked-in default rule off. Do not treat the example as a general-availability announcement. The supporting [Preview QA record](https://github.com/adaptive-machines/runner-monorepo/pull/15801#issuecomment-6166871494) documents the observed states. It also names a gap: a real shopper-chat answer using a document was not checked in that run.

## Test the boundary before sharing more documents

Use harmless test content and a reviewable draft. Do not start with customer records or business secrets.

1. **Ask a covered question.** Verify that the answer matches a specific approved passage, including its exceptions.
2. **Ask an uncovered question.** Look for an explicit uncertainty or request for more information, not an invented policy.
3. **Change the wording.** Ask the same question another way and check whether the relevant source is still selected.
4. **Check the audience.** Confirm that an internal note does not become available to a shopper merely because a merchant can read it.
5. **Revise the source.** Ask again in the intended context and check which version informs the answer. Do not assume an existing chat reloads every document automatically.
6. **Remove a source.** Check future use separately from historical references and retention behavior.

Keep the question, expected source, actual answer, and discrepancy together. That makes a failure actionable. You can distinguish missing content from a retrieval failure or an unsupported inference instead of labeling all three "the AI forgot."

For consequential changes, keep a person at the approval boundary. Our guide to [human-in-the-loop AI](./ai-human-in-the-loop) explains why review should focus on the decision and its consequences, not just whether an answer sounds fluent.

## Related questions

### Is an AI knowledge base the same as AI memory?

No. A knowledge base holds source material for consultation. Persistent memory often holds selected facts or preferences for reuse across tasks. The exact implementation varies. [AI memory, context, and training](./does-ai-have-memory) are separate mechanisms; storing a document or preference does not by itself retrain the model.

### Does uploading a document make every answer use it?

No. The application may require explicit attachment, run a search, filter by audience, or omit content from the current context. Verify the product's selection rule and inspect evidence for the particular answer.

### Can a knowledge base guarantee accurate answers?

No. Good source material gives the assistant something relevant to consult, but incorrect retrieval, stale content, and unsupported interpretation remain possible. Check the claim against the passage before using it to make a promise to a customer.

### Where should I start in Runner?

Begin with one approved document and one draft task, if Knowledge is enabled for your project. Keep customer-facing publication separate from drafting. For broader task context, use the [Runner chats and history guide](https://www.runnerai.com/docs/en/guides/work-with-runner/chats-and-history). For saved preferences rather than source documents, use the [Memory guide](https://www.runnerai.com/docs/en/guides/agent/memory).

The useful first outcome is modest: one answer whose source you can identify, open, and check. Expand the library only when you can maintain that standard.