AI Engineering Institute

Overview This guide will walk you through creating a simple Retrieval Augmentation Generation (RAG) system using pure Python. We will use an embedding model and a large language model (LLM) to retrieve relevant documents and generate responses based on user queries. https://github.com/adithya-s-k/A...

Introduction Retrieval-enhanced generation (RAG) is a powerful technique that combines the benefits of large language models with the ability to retrieve relevant information from a knowledge base. This approach improves the quality and accuracy of generated responses by basing them on specific retrieved information.a This notebook aims ...

INTRODUCTION BM25 Retrieval Augmented Generation (BM25 RAG) is an advanced technique that combines the BM25 (Best Matching 25) algorithm for information retrieval with a large language model for text generation. By using a validated probabilistic retrieval model, this method improves the accuracy and relevance of the generated responses....

INTRODUCTION Data chunking is a key step in Retrieval Augmented Generation (RAG) systems. It breaks large documents into smaller, manageable pieces for efficient indexing, retrieval, and processing. This README provides an overview of the various chunking methods available in the RAG pipeline. https://github.com/adithya-...

Introduction Evaluation is a key component in the development and optimization of Retrieval Augmentation Generation (RAG) systems. Evaluation involves measuring the performance, accuracy, and quality of all aspects of the RAG process, from retrieval effectiveness to the relevance and authenticity of generated responses. Importance of RAG Evaluation An effective RAG system...

The reordering model will improve the results of semantic ranking by reordering the list of candidate documents based on their semantic match to the user's question. Commonly used bge-reranker-v2-m3 or cohere

Sentence-Window-Based Retriever RAG Approach Introduction The Sentence-Window-Based Retriever RAG (Retrieval-Augmented Generation) approach is a high-level implementation of the RAG framework designed to enhance the context-awareness and coherence of AI-generated responses. The approach combines a large-scale language model with a high ...

  Introduction The Sentence Window-based Retrieval-Augmented Generation (RAG) method is a high-level implementation of the RAG framework that aims to enhance the context-awareness and coherence of AI-generated responses. The method combines the power of large language modeling with efficient information ...

Introduction The Automated Merge Retriever is a high-level implementation of the Enhanced Retrieval Generation (RAG) framework. It aims to enhance the context-awareness and coherence of AI-generated responses by merging potentially fragmented and smaller contexts into larger and more comprehensive ones. https://github.com/adith...

Introduction The Query Transformations User Manual demonstrates a variety of techniques for transforming and disambiguating user queries before they are executed in a Retrieval-Augmented Generation (RAG) query engine, intelligences, or other processes. These transformations can improve the quality and relevance of responses in AI applications. https://github.com/adithya-s-k/AI-...

Introduction Self-Query RAG (SQRAG) is an advanced Retrieval Augmented Generation (RAG) approach that enhances the traditional RAG process by introducing metadata extraction in the ingestion phase and intelligent query parsing in the retrieval phase. https://github.com/adithya-s-k/AI-Engi...

Introduction RAG-Fusion is an advanced information retrieval and text generation methodology built on Retrieval Augmented Generation (RAG). This project implements RAG-Fusion to provide more accurate, contextually relevant and comprehensive responses to user queries. https://github.com/adithya-s-k...

Introduction RAPTOR (Recursive Abstract Processing for Tree-Structured Retrieval Enhanced Generation) is an advanced Retrieval Enhanced Generation (RAG) method. It enhances the traditional RAG process by introducing hierarchical document structuring and summarization techniques. https://github.com/adithya-s-k/AI-Engineering.acade...

ColBERT (Contextualized Post-Cultural Interaction based on BERT) is different from the traditional dense embedding model. Here is a brief description of how ColBERT works: Token-level embedding: Unlike directly creating a single vector for an entire document or query, ColBERT creates embedding vectors for each Token. After...