HybridRAG: Integrating Knowledge Graphs and Vector Retrieval Augmented Generation for Efficient Information Retrieval

Introduction

The paper addresses the challenge of extracting information from financial documents, such as earnings call transcripts, using large language models (LLMs). While LLMs are effective in general NLP tasks, they struggle with domain-specific terminology and complex formats present in financial data. Current approaches, such as VectorRAG (retrieval-augmented generation using vector databases), face limitations in ensuring accuracy and contextual relevance. Knowledge Graphs (KGs) offer structured representations but also have limitations in handling unstructured data. To overcome these issues, the authors propose HybridRAG, which combines the strengths of VectorRAG and GraphRAG, to enhance the performance of Q&A systems in financial contexts.

VectorRAG

VectorRAG employs vector-based retrieval systems for retrieving relevant document chunks by similarity searches. These chunks are converted into embeddings using a pre-trained model and stored in a vector database. For a given query, the most relevant chunks are retrieved and passed as context to the LLM for response generation. This approach works well for abstractive Q&A tasks where the required information is distributed across text but may lose hierarchical context.

GraphRAG

GraphRAG utilizes Knowledge Graphs (KGs) to enhance the retrieval process. A KG is constructed from document triplets representing entities and their relationships. When a query is posed, it retrieves relevant subgraphs containing nodes (entities) and edges (relationships). This structured context is encoded into embeddings and passed to the LLM for generating answers.

Key Steps in GraphRAG:

• Knowledge Graph Construction: Extracts entities and their relationships from text using NLP tasks like entity recognition and relationship extraction. These are converted into graph triplets and stored in a graph database.
• Query Processing: For each query, a depth-first search (DFS) identifies relevant subgraphs.
• Integration with LLM: The extracted subgraph is embedded and used as additional context for generating answers.

Advantages:

• Handles hierarchical and relational data effectively.
• Suitable for extractive Q&A tasks where explicit entities are mentioned.

Limitations:

• Underperforms in abstractive tasks.
• Struggles with queries that lack explicit entities.

HybridRAG

HybridRAG integrates both VectorRAG and GraphRAG. It concatenates the contexts retrieved from both methods and uses this unified context for response generation. This hybrid approach benefits from:

• VectorRAG: Broad coverage and similarity-based retrieval for abstractive tasks.
• GraphRAG: Structured and relationship-rich retrieval for extractive tasks.

Evaluation shows that HybridRAG outperforms both VectorRAG and GraphRAG individually in terms of faithfulness, answer relevancy, and context recall. However, it has slightly lower context precision due to the combined context introducing additional noise.

This approach demonstrates potential beyond financial applications, paving the way for more robust information extraction systems that integrate structured and unstructured data.