Appearance

Knowledge Retrieval ​

RAG-powered search through indexed knowledge bases with sub-100ms hybrid search combining keyword matching and semantic understanding.

Overview ​

Knowledge Retrieval is a built-in capability that brings sophisticated RAG (Retrieval-Augmented Generation) capabilities to AI agents. When enabled on your MCP Gateway, agents can perform hybrid search that combines traditional keyword matching with deep semantic understanding, delivering the most relevant information from your knowledge base in milliseconds.

Key Features ​

  • Hybrid Search: Combines BM25 keyword search with vector semantic search
  • Sub-100ms Latency: Optimized for real-time agent interactions
  • Smart Chunking: Automatically segments documents for optimal context
  • Relevance Scoring: Transparent ranking with explainable results
  • LLM Optimization: Returns chunks sized for context windows
  • Multi-Format Support: PDF, DOCX/XLSX/PPTX, Markdown, AsciiDoc, HTML/XHTML, CSV, images (PNG, JPEG, TIFF, BMP, WEBP)

How to Enable ​

  1. Navigate to your MCP Gateway settings
  2. Locate Knowledge Retrieval in the Built-in Capabilities section
  3. Toggle the capability ON
  4. Configure knowledge bases (create and index documents)
  5. Save your configuration

Once enabled, any AI agent connected to your gateway can search indexed knowledge through the MCP protocol.

Supported Input Formats ​

FormatDescription
PDFPortable Document Format files with text extraction
DOCX, XLSX, PPTXMicrosoft Office 2007+ formats based on Office Open XML
MarkdownPlain text formatting syntax (.md files)
AsciiDocLightweight markup language (.adoc files)
HTML, XHTMLWeb markup languages
CSVComma-separated values for structured data
PNG, JPEG, TIFF, BMP, WEBPImage formats processed with OCR for text extraction

All formats are automatically processed and indexed for immediate searchability. Images undergo OCR (Optical Character Recognition) to extract text content for indexing.

What AI Agents Can Do ​

Information Retrieval ​

  • Search across multiple knowledge bases simultaneously
  • Find specific facts, procedures, or documentation
  • Retrieve context-relevant chunks for analysis
  • Access historical versions of documents

Semantic Understanding ​

  • Answer natural language questions
  • Find conceptually similar content
  • Discover related topics and themes
  • Bridge terminology differences

Document Analysis ​

  • Extract key information from large documents
  • Identify patterns across document collections
  • Generate summaries from retrieved chunks
  • Cross-reference multiple sources

Knowledge Management ​

  • Organize information into searchable collections
  • Maintain up-to-date knowledge bases
  • Track document changes and versions
  • Ensure information consistency

Search Types ​

Keyword Search (BM25) ​

Best for specific terms, product names, or exact phrases. Uses traditional information retrieval scoring for precise term matching.

Semantic Search (Vector) ​

Best for concepts, similar meanings, or natural questions. Uses embeddings to find contextually related content.

Hybrid Search (Default) ​

Combines both approaches for optimal results, leveraging the precision of keyword matching with the understanding of semantic search.

Knowledge Base Management ​

Creating Knowledge Bases ​

Agents can create named knowledge bases with specific configurations for chunk size, overlap, and embedding models based on content type and use case.

Document Indexing ​

Documents are automatically processed, chunked, and indexed. The system extracts metadata, generates embeddings, and creates searchable indices.

Metadata Enrichment ​

Automatic extraction of document properties including title, creation date, language, and custom tags for enhanced filtering and retrieval.

Capability Behavior ​

Automatic Features ​

  • Intelligent document chunking based on content structure
  • Automatic language detection and handling
  • Duplicate detection and deduplication
  • Incremental indexing for new content
  • Cache optimization for frequent queries

Search Optimization ​

  • Query expansion with synonyms and related terms
  • Relevance boosting based on metadata
  • Context-aware result ranking
  • Automatic spell correction
  • Result deduplication

Performance Characteristics ​

Index SizeKeyword SearchSemantic SearchHybrid Search
1K docs<10ms<50ms<60ms
10K docs<20ms<70ms<85ms
100K docs<30ms<90ms<100ms
1M docs<50ms<100ms<120ms

Typical Applications ​

Customer Support Systems ​

AI agents use Knowledge Retrieval to instantly find relevant support articles, troubleshooting guides, and FAQ answers, providing accurate responses to customer inquiries based on comprehensive documentation.

Research Assistants ​

Agents search through research papers, technical documentation, and reference materials to compile comprehensive information on topics, supporting literature reviews and technical investigations.

Compliance Monitoring ​

Agents retrieve regulatory requirements, policy documents, and compliance guidelines to ensure operations align with current standards and identify potential violations.

Agents help users navigate large documentation sets, finding specific procedures, API references, configuration guides, and best practices from extensive technical libraries.

Knowledge Synthesis ​

Agents combine information from multiple sources to create comprehensive reports, answer complex questions, and provide multi-faceted analysis on topics.

Advanced Features ​

Filtering Capabilities ​

  • Date range filtering for temporal relevance
  • Tag-based categorization and filtering
  • Metadata-based selection criteria
  • Document type restrictions
  • Language-specific searches

Context Management ​

  • Automatic chunk sizing for LLM context windows
  • Related chunk retrieval for comprehensive context
  • Hierarchical document navigation
  • Section-aware searching

Quality Controls ​

  • Relevance score thresholds
  • Source verification and authority scoring
  • Duplicate content handling
  • Version control and tracking

Pricing ​

Search Operations ​

  • Cost: $0.0015 per search

Storage & Indexing ​

  • Document Storage: $0.10/GB/day
  • Indexing: $0.20 per million tokens
  • Re-indexing: Same as initial indexing

Cost Optimization ​

  • Cache frequently searched queries
  • Use appropriate search types for queries
  • Batch similar searches together
  • Implement relevance thresholds

Limitations ​

  • Max Query Length: 500 characters
  • Max Results: 100 per search
  • Document Size: 10MB per document
  • Index Size: 10GB per knowledge base
  • Concurrent Searches: 100 per second

Prerequisites ​

Before using Knowledge Retrieval:

  1. Create Knowledge Base: Set up named knowledge base with configuration
  2. Index Documents: Add and process documents for searching
  3. Wait for Indexing: Processing completes in seconds to minutes
  4. Start Searching: Query your indexed content

Best Practices ​

Document Preparation ​

  • Clean and structure documents before indexing
  • Add meaningful metadata and tags
  • Split large documents into logical sections
  • Maintain consistent formatting

Query Optimization ​

  • Use specific terms when available
  • Provide context in natural language queries
  • Leverage filters to narrow results
  • Combine search types appropriately

Knowledge Base Organization ​

  • Create separate bases for different domains
  • Regularly update and maintain content
  • Remove outdated information
  • Monitor search performance and adjust

Next Steps ​

Platform Features ​

© 2025 Noorle Inc. All rights reserved.