06Case study — 2026

RAG chat, multi-cloud
a reference architecture where the app never talks to a model directly

Document ingestion, vector search, reranking and streamed answers with sources, every model call going through one LiteLLM proxy that fails over between Bedrock, Azure AI Foundry and Vertex AI, or runs fully local.

Role: Architecture, backend, infrastructure as code
Stack: FastAPI · React · LiteLLM · Qdrant · PostgreSQL · Redis · Terraform
Status: Public reference architecture, a demo, not a product
Links: Repository

The problem

Every team that wants "a chat on our documents" faces the same three decisions: how to get text out of files, where to put the vectors, and how to avoid marrying one model provider. I wanted a small, complete, readable example that makes those decisions explicit, something I could point at in a discussion instead of a whiteboard.

What I built

Ingestion: PDF, DOCX, Markdown and images go through a parser with optional OCR (Tesseract by default, cloud document services behind a variable), a 512-token chunker, embeddings requested through the proxy, and a Qdrant collection; long jobs can move to a Celery worker. A question is embedded, searched (twice the requested candidates), optionally reranked, assembled into a context, and completed with the answer streamed over SSE with its cited sources, conversation history in PostgreSQL.

The whole point is the single exit: a LiteLLM proxy with model aliases, declarative fallback chains (a completion alias that tries Claude, then GPT-4o, then Gemini; an embedding alias across the three clouds), retries, a Redis cache and a master key. The code never sees a provider SDK or a credential; a compose override and an alternate LiteLLM config make the same stack run entirely on a local model server. Terraform provisions the AWS side (VPC, EKS, RDS, ElastiCache, ECR), and raw Kubernetes manifests deploy the services, with OpenTelemetry and Langfuse tracing available but off by default.

fig. 1 — no screenshot, a drawing

Fig. 1 — ingestion on top, a question below · one proxy in front of three providers, and the answer streamed with its sources01

What I learned

That the proxy pattern is the part worth copying, and that it is honest to say what a reference architecture is not: authentication, background workers, reranking and tracing are switches left off, there are no tests, and the vector store is not part of the Terraform. What it does show is the shape, and the shape is what carried into the multimodal chat that came after it, with a model router, assistants and tools.

“The application knows model aliases, not providers; swapping a cloud is a config change.”