TL;DR
RAG projects can grow quickly. For this build, Docker Agent’s native RAG capabilities were enough to create a document assistant backed by a local folder.
The complete example is available in the docker-agent-ask-docs repository. This post explains the configuration and what Docker Agent handles for us.
Start with one clear responsibility
The assistant has one job:
documents -> retrieve relevant content -> answer the question
The documents live in docs/. Docker Agent reads the configured knowledge base, retrieves relevant content, and gives the assistant enough context to answer. The generated embeddings database lives in rag/.
The RAG pipeline stays visible here. Docker Agent simply takes care of the first implementation of each step.
The configuration became the application
The repository is small enough to understand at a glance:
docker-agent-ask-docs/
├── README.md
├── docker-agent.yaml
├── docs/
│ └── example.md
└── rag/
└── .gitkeep
There is no application source directory because the configuration is doing most of the work.
The agent is defined in docker-agent.yaml:
agents:
root:
model: gpt-minimal
description: assistant that answers questions about local documents
instruction: |
You are a helpful document assistant.
Answer questions using only information retrieved from the configured
knowledge base. Do not use general knowledge or invent information.
If the knowledge base does not contain enough information to answer
the question, clearly say that the answer was not found in the provided
documents.
If you receive sources from the knowledge base, include them as a
markdown list of links to local files at the very end of your response.
toolsets:
- type: rag
ref: local_docs
This instruction is an important part of the design. Retrieval alone does not define how the assistant should respond.
In this project, the boundary is simple: answer from the retrieved documents, do not fill gaps with general knowledge, and show the files used as sources.
The configuration describes the assistant’s role and its limits.
The folder became the knowledge boundary
The RAG tool points at the local docs directory:
rag:
local_docs:
tool:
description: documents provided by the user
docs:
- ./docs
That makes the folder the assistant’s knowledge boundary.
Add a document and it becomes another possible source. Remove it and that source disappears from the assistant’s context. The assistant has no internet connection and works with the documents in this folder.
The example document is intentionally ordinary:
# Docker Agent Ask Docs
Docker Agent Ask Docs is a document-question-answering example.
The assistant answers questions using information retrieved from documents
stored in the local `docs` directory.
That is enough to verify the retrieval behavior without introducing an unnecessary data set.
The RAG pipeline is one configuration block
The retrieval strategy uses chunked embeddings:
strategies:
- type: chunked-embeddings
embedding_model: openai/text-embedding-3-small
database: ./rag/chunked_embeddings.db
vector_dimensions: 1536
This is the part that would normally require application code and a separate service. Here, I only had to describe the strategy and choose where the generated database should be stored.
The flow looks like this:
example.md
|
v
document chunks
|
v
embeddings
|
v
rag/chunked_embeddings.db
|
v
retrieved context for the agent
The embeddings database is generated locally and ignored by Git. The storage is local, while embedding generation still depends on the configured OpenAI provider.
The database location tells us where retrieval data is stored. It says nothing about where model calls are processed.
Try the assistant
Run the assistant with the Docker Agent configuration:
docker agent run docker-agent.yaml
Once the agent is running, I can ask a question supported by the document:
What does this project demonstrate?
The answer should describe the document-question-answering project and include the local document as a source.
The important test is a question the document cannot answer:
Who will use this system in production?
The assistant should report that the answer was not found in the provided documents.
That refusal is part of the design. A RAG assistant needs to stay within the evidence available to it.
The trade-off of keeping it small
Keeping the project small made one thing very clear: Docker Agent provides the runtime and retrieval workflow, while I decide what belongs in the knowledge base.
I chose the documents, wrote the instructions, defined the retrieval strategy, and decided what the assistant should say when the answer is missing. The configuration removed a lot of plumbing and left the important design decisions visible.
That is a useful trade-off. Instead of spending the first day connecting an API, an ingestion script, and a vector database, I could spend it deciding whether the answers were grounded and useful.
What is missing from the example
This is an early-stage application. It has no custom API, web interface, authentication, document-level permission model, scheduled ingestion, internet search, or multi-agent workflow.
Those features can come later if the use case calls for them. For a folder of project documentation, runbooks, or reference notes, adding them before testing the basic interaction would create unnecessary overhead.
Scale is the main limitation. A larger system would need to handle many users, changing documents, access control, observability, and retrieval quality across a much bigger collection. This setup is a starting point for that work.
When this approach makes sense
I would use this approach when the question is:
Can a document-grounded assistant help with this collection of files?
A folder and a configuration make that question easy to test. Add a few documents, ask questions that should be answerable, and then ask questions that should be refused. The results show whether the knowledge boundary and retrieval behavior make sense.
Useful answers point to the next step: add an interface, expose an API, improve ingestion, or introduce access controls. If the answers fall short, there is very little infrastructure to rework.
Final thought
The result is simple and focused. The instructions define a boundary, the RAG tool supplies context, and the source links make each answer easier to check.
That combination was enough to turn a folder of Markdown files into a working document-question-answering assistant. The complete code and configuration are available in the docker-agent-ask-docs GitHub repository.