OG-RAG: Ontology-Grounded Retrieval for Accurate LLM Responses
OG-RAG: Enhancing LLMs with Ontology-Grounded Retrieval
This blog post summarizes the paper "OG-RAG: Ontology-Grounded Retrieval-Augmented Generation for Large Language Models," which introduces a novel approach to improve the factual accuracy and adaptability of Large Language Models (LLMs) in specialized domains.
Problem Definition
- LLMs often struggle to adapt to specialized knowledge domains without extensive fine-tuning.
- Existing Retrieval-Augmented Generation (RAG) models do not adequately incorporate structured domain knowledge.
- Fine-tuning is computationally expensive and requires significant data curation.
- Current retrieval methods often fail to capture the deep conceptual relationships and nuanced facts necessary for accurate domain-specific retrieval.
Proposed Solution
- OG-RAG: An Ontology-Grounded Retrieval Augmented Generation method that enhances LLM responses by grounding retrieval processes in domain-specific ontologies.
- Key idea: Construct a hypergraph representation of domain documents, where hyperedges encapsulate clusters of factual knowledge grounded using a domain-specific ontology.
- An optimization algorithm retrieves the minimal set of hyperedges to construct a precise, conceptually grounded context for the LLM.
- The method transforms data-mapped ontologies into hypergraphs, using optimized retrieval of relevant fact clusters to reduce hallucinations.
- OG-RAG combines the structured precision of fact-based reasoning with neural flexibility by anchoring unstructured text to domain-specific vocabulary.
- Limitation: Domain-specific ontologies may not always be available or sufficiently comprehensive.
Method
- Hypergraph Construction: Maps domain-specific documents onto a given ontology, using LLMs to map ontology entities to corresponding document text in JSON-LD format.
- Hypergraph Transformation: Flattens the structure of factual-blocks for efficient extraction and attribution using a flattening algorithm.
- Hypergraph-based Retrieval: Retrieves relevant context based on user query Q by identifying relevant hypernodes and forming the relevant context as the set of hyperedges that minimally cover the relevant hypernodes.
- Retrieval-Augmented Generation: Prompts the LLM to use the retrieved context to answer the query.
Results
- Achieved a 55% increase in recall of accurate facts.
- Demonstrated a 40% improvement in response correctness across four different LLMs (GPT-4o-mini, GPT-4o, Llama-3.1-8B, and Llama-3.1-70B).
- Showed a 30% faster attribution of responses to context.
- Realized a 27% boost in fact-based reasoning accuracy.
- OG-RAG outperforms baselines (RAG, RAPTOR, GraphRAG) in almost all cases, boosting the recall of correct entities by 110%.
- OG-RAG performs nearly as efficiently as a simple RAG method, with only a minimal increase of at most 2 seconds during querying time despite being at least 100% better in factual accuracy.
- OG-RAG significantly reduced the time required by 28.8% and increased the human-attributed support by 29.6% on average.
Importance
- OG-RAG enhances response verification by providing precise, fact-based context attribution.
- Enables factual deduction by leveraging domain-adapted facts to infer conclusions based on established knowledge and relationships.
- The method provides easy-to-attribute contexts that require only a little effort from the users to trace the generation of the response.
- OG-RAG significantly improves the factual accuracy of LLM-generated responses, while also enabling faster attribution of answers to their supporting context and more effectively deducing conclusions from domain facts.