RAG
Latency
Retrieval
European SaaS provider

Klebbix Hybrid Retrieval System

Query latency fell 68% and embedding and inference costs fell 35%, while relevance accuracy held above 93% across structured and unstructured data. Every tenant stayed fully isolated under GDPR and ISO-27001 requirements.

average query latency: −68%
embedding and inference costs: −35%
relevance accuracy: 93%+
response time per request: <1.2s

Klebbix Hybrid Retrieval System - European SaaS provider, built by MacroCoderz

Overview

A European SaaS provider needed an AI retrieval engine for its internal and client documentation. Its old system used plain keyword search, which could not find relevant results across mixed data formats. Each enterprise tenant stored thousands of files in different formats, so information sat in separate silos. The client needed one RAG platform that could combine text documents, tables and relational records, and keep each customer's data fully isolated.

Core Challenges

Data Fragmentation

Data lived in relational databases, PDFs, Word files and CSV datasets. Standard retrieval pipelines could not connect structured and unstructured sources. Answers were built on incomplete context, and accuracy was low.

Tenant Isolation and Compliance

The system had to serve many tenants, with full encryption and a separate namespace for each. Each tenant’s vector store and relational data had to stay completely apart to meet GDPR and ISO-27001.

Latency and Cost Control

Each tenant held millions of documents, which made embedding and inference expensive. Existing solutions used too many tokens, and query latency went past acceptable enterprise limits. The new platform had to retrieve cheaply and perform predictably at scale.

Solution Overview

Hybrid Retrieval Pipeline

The retrieval layer combines semantic vector matching with relational joins. Qdrant stores the text embeddings and PostgreSQL holds the structured fields. A middleware orchestrator queries both and returns one ranked context.

Security and Multi-Tenancy

Each tenant runs in its own namespace. Data is encrypted with AES-256 at rest and mutual TLS in transit. Azure AD SSO enforces role-based access control (RBAC), with strict authentication and access levels.

Optimization and Cost Efficiency

Redis caching and query thresholding cut repeat embedding requests. Cohere re-ranking sharpens retrieval precision. Together they lower inference cost without losing context accuracy.

Solution Description

Klebbix RAG runs as containerized microservices built with FastAPI. The backend talks directly to PostgreSQL for structured data and to Qdrant for vector embeddings. It runs on Hetzner Cloud, set up for high-availability workloads.

Backend Framework: FastAPI on Ubuntu LTS
Vector Database: Qdrant v1.8
Relational Database: PostgreSQL 15
Re-Ranker: Cohere Rerank-3
Authentication: Azure AD SSO with OIDC
Caching Layer: Redis 7
Monitoring: Grafana and Uptime Kuma
Deployment Environment: Docker Compose on Hetzner CAX31 (8 vCPU, 16 GB RAM)

Process Flow

1
The pipeline parses, chunks and embeds uploaded documents and datasets with OpenAI Ada and Cohere Embed.
2
PostgreSQL stores metadata and structured data. Qdrant indexes the embeddings.
3
Each query runs a hybrid lookup across the vectors and SQL.
4
Cohere re-ranks the results and passes them to GPT-4, which writes the answer.
5
Redis caches repeated queries, which keeps response times steady at under 1.2 seconds per request.

Results

Average query latency68 percent lower
Embedding and inference costs35 percent lower
Tenant data isolation, verified in internal compliance auditsComplete
Relevance accuracy across structured and unstructured formats, from one retrieval layerOver 93 percent

What changed

Klebbix RAG gave the client one production-ready search system for its mixed enterprise data, run securely and efficiently. Teams can query documents, tables and databases at the same time without exposing sensitive tenant data.

The build is a base for multi-tenant AI products that can scale, with precise retrieval, compliance and a modular design that can grow. It remains the reference architecture for later enterprise AI infrastructure projects built on retrieval-augmented generation.

The services behind this buildRAG systems →AI integration services →White-label RAG →

Further readingPinecone vs Weaviate vs pgvector →Evaluating LLM output in production →

Want one of these for your product?

Fixed scope, fixed price. You own the code, prompts and eval data from the first commit.

Book a call →