HybridRAG: Knowledge Graphs and Vector Retrieval

HybridRAG: Integrating Knowledge Graphs and Vector Retrieval Augmented Generation for Efficient Information Extraction, by Bhaskarjit Sarmah and co-authors, combines two retrieval paths for question answering over financial documents. The paper evaluates a custom dataset of 400 questions drawn from 50 Nifty 50 Q1 FY2024 earnings-call transcripts. It does not establish the same results for other domains.

Vector retrieval

VectorRAG splits documents into chunks, embeds those chunks, and retrieves the chunks whose vectors are most similar to a query. The language model receives the retrieved text along with the question. This can work well when the answer is distributed across prose, while chunking can lose relationships that span a document’s structure.

Graph retrieval

A knowledge graph represents entities and relationships as nodes and edges. GraphRAG extracts a graph from the documents, finds entities and connected relationships relevant to a query, and supplies that structured context to the language model. In this paper’s implementation, a depth-first search with depth one retrieves a subgraph from the graph.

Graph retrieval can expose explicit relationships, while vector retrieval can provide surrounding prose. Each path can also retrieve irrelevant or incomplete context. The quality of the graph extraction and the retrieval configuration therefore affect the final answer.

HybridRAG

HybridRAG concatenates the contexts returned by VectorRAG and GraphRAG, then sends the combined context to the answer generator.

Reported evaluation

The authors use GPT-3.5-Turbo at temperature 0 with text-embedding-ada-002 and evaluate financial earnings-call Q&A. They report four RAGAS metrics: faithfulness, answer relevance, context precision, and context recall.

HybridRAG matches GraphRAG on faithfulness, improves answer relevance and context recall, and lowers context precision. Relative to VectorRAG, it improves faithfulness and answer relevance, matches context recall, and lowers context precision. These results describe the financial-document setup. Applications beyond finance remain possible research directions.