RAG
Industrial RAG: how to measure answer quality
Evaluate answers from engineering manuals with source relevance, device context and appropriate abstention.
A RAG chunking strategy for technical manuals
Splitting a manual into arbitrary-length pieces makes the right answer get searched in the wrong context. Preserve headings, tables and revision information.
In short
- A fixed character count can cut a table in the middle, so split manuals on heading boundaries and keep each chunk’s heading path.
- Fault-code table rows are misleading without their header row, so attach the headers to each row or keep the table whole in one chunk.
- Attach product family, model, revision, language and page metadata to every chunk, because similar fault codes from different families cannot be reliably separated without it.
Keep the structure
Manuals consist of headings, subheadings, tables and warning boxes. Splitting at a fixed character count cuts a table in the middle or separates a warning from its step. Split on heading boundaries and attach to each chunk the heading path it belongs to.
Treat tables and codes separately
Fault-code tables carry meaning row by row. Storing a row without its header row is misleading. Attach the column headers to each table row or keep the table whole within a chunk boundary. For very large tables, form row groups and repeat the header in each.
Add metadata
Attach product family, model, document revision, language and page to every chunk. These fields are needed both for filtering at search time and for citing sources in the answer. A similar fault code from a different product family cannot be reliably separated without metadata.
Measure the chunking
Compare chunk size and overlap not by intuition but with a fixed question set: measure at each setting how often the expected document appears in the top five results. This approach is a suggested engineering method; the best setting depends on the document structure and question type.
Frequently asked
How do I measure the best chunk size for a RAG system on technical manuals?
Compare chunk size and overlap with a fixed question set rather than intuition. At each setting, measure how often the expected document appears in the top five results. The best setting depends on the document structure and question type, so this is a suggested engineering method.
What metadata should each chunk of a technical manual carry?
Attach product family, model, document revision, language and page to every chunk. These fields are needed for filtering at search time and for citing sources in the answer. Without metadata, a similar fault code from a different product family cannot be reliably separated.
Sources and further reading
Pilot steps in this guide are proposed engineering workflows. Examples do not constitute measured field performance.
Questions on this topic
Keep reading.
Full library
RAG
Making LLMs Reliable for Industrial Q&A: Evidence-First Design
General chatbots hallucinate; industrial answers must be verifiable. This article describes the evidence-first design pattern: curated corpora, hybrid retrieval, constrained synthesis, and inspectable output.
RAG
Word Embeddings and Semantic Search: How Modern Retrieval Works
Semantic search matches meaning, not spelling. This article explains embeddings, vector similarity, and why hybrid keyword-plus-vector retrieval is the robust production pattern.