RAG
A RAG chunking strategy for technical manuals
Splitting a manual into arbitrary-length pieces makes the right answer get searched in the wrong context. Preserve headings, tables and revision information.
What is RAG and how does it work?
Short answer
RAG (retrieval-augmented generation) means a language model first searches a document store for passages related to the question, then writes its answer from those passages. The retrieved text is added to the prompt alongside the question. When documents change, the index is updated and the model does not need to be retrained.
The basic idea
A language model keeps what it learned during training in its weights. A maintenance manual revised this week, or a procedure published this morning, is not in those weights. RAG fills that gap at query time.
First, documents are split into chunks, each chunk is converted into a numeric vector (an embedding), and the vectors are stored in an index. When a user asks a question, the question is converted into a vector too, and the most similar chunks, usually the top few, are retrieved. The model receives those chunks together with the question and writes the answer.
Its weights do not change; the only thing that changes is the context it sees. If the system records which chunks were used, the answer can cite them.
A worked example
Suppose a hypothetical maintenance manual has 120 pages with an average of 400 words per page. The total is 120 × 400 = 48,000 words. Splitting it into chunks of 500 words gives 48,000 / 500 = 96 chunks.
A technician asks, 'How often is the pump seal replaced?' The system retrieves the 4 most similar chunks (k = 4). Together they hold 4 × 500 = 2,000 words. The model sees 2,000 / 48,000 ≈ 0.042, about 4% of the manual.
If the answer is in one of those chunks, the model can point to the page. If it is not, the answer will be wrong or missing, and the fault lies in the retrieval step.
Limits and common mistakes
Most RAG failures come from retrieval, not generation. If the right passage is not among the top results, the model may still write a fluent answer, and that answer looks like a hallucination. Chunking can also cut a procedure or a table in half, and its meaning is lost.
Old revisions left in the index are retrieved with the same confidence as new ones. Exact strings such as part numbers or alarm codes are often poorly matched by meaning-based search alone; combining it with keyword search is a common fix.
Finally, building a RAG system does not guarantee correct answers. Accuracy has to be shown by measuring on your own questions.
Follow-up questions
What is the difference between RAG and fine-tuning?
RAG fetches knowledge from documents at query time, so updating a document is enough and the answer can cite its source. Fine-tuning changes model weights using example data. It suits output format, language or a specific task behavior better, but updating facts and tracing answers to sources are harder.
What should be prepared before building a RAG system?
Collect current documents with revision numbers and dates, and mark or remove superseded versions. Add metadata such as machine model, line and revision. Write a set of questions with known answers, including some the documents cannot answer, and rerun that set after every change.
Sources
Example numbers are hypothetical, not measured field results.
Read the full guide.
Full library
RAG
Industrial RAG: how to measure answer quality
Evaluate answers from engineering manuals with source relevance, device context and appropriate abstention.