AI Accuracy: Why RAG and Knowledge Quality Matter

Again, the technique used and the capabilities of the underlying model to accurately identify and extract the needed data have a massive impact on results. Numerous benchmarks test different GenAI multimodal and vision models for OCR & table extraction accuracy, with scores ranging from 42%-91% depending on the model used.

Similarly, using OCR results can confuse the AI if the layout and semantics of the document or table are lost. Consider a page formatted with columns of text: OCR results by themselves will provide a line of text read across all the columns on the page, creating “sentences” made from words in different paragraphs. For tables, this can result in fragmented or clipped sets of data, with boundaries between cells, rows, or columns in the table being lost.

In comparison, pre-extracting tables (and, if needed, validating with human-in-the-loop) using the latest computer vision-powered IDP models typically achieve 97%+ extraction accuracy for table data. Likewise, computer vision can interpret document layouts to provide coherent “chunks” of text to LLMs, regardless of paragraphs, sections or page boundaries. This approach provides content to the AI structured in a way that is easy for it to consume, ensuring the visual layout of tables and documents is considered.

Again, for consistently and accuracy, the RAG process needs to both correctly locate the most appropriate table to use *and* provide the results in a format that corresponds to the original table layout and intent.

Can’t the AI just figure it out?

Another consideration is how much content or data is provided to the AI to work with.

With growing “context windows”, Large Language Models can accept more content or data to work with. However, bigger isn’t always better as this article from Chroma Research explores. A phenomenon called “Context Rot” means we should be cautious about just giving ever larger sets of inputs to an LLM – the AI will get much better results from a smaller curated set of data. If you give the AI too much data to work with, it can become “lost in the middle” as it tries to process all the data provided.

RAG provides one way to address this, since it gathers curated results from the Knowledge Base search. However, it also creates a dependency on the Knowledge Base to return the right results; otherwise, the whole downstream agent process is built on sand.

Going beyond basic RAG for AI accuracy

As context windows grow, more input does not guarantee better answers. The dependency shifts to the Knowledge Base. If retrieval is off, reference links can mask gaps, steps may be missed or out of order, and tables passed as images or noisy OCR can trip models. Accuracy rests on curated knowledge, precise retrieval, and completeness and sequencing, not just the model.

A practical path is a centrally governed Knowledge Base paired with sequence and layout aware retrieval, structured extraction of tabular data via IDP with human-in-the-loop validation, and continuous accuracy measurement. With TotalAgility, organisations can centralise and curate content, standardise retrieval patterns including leveraging multi-step search and “graph” relationships between entities or concepts, extract and validate tables, and monitor outcomes while remaining model agnostic, turning experimentation into reliable, auditable answers.