How I Built a 'Chat with your PDF' App with Cloudflare Vectorize (RAG)

Large Language Models (LLMs) like Llama 3 are incredibly smart, but they suffer from one major flaw: they only know what they were trained on. If you ask an LLM to summarize a proprietary company PDF or a brand-new research paper, it will either hallucinate or fail. The industry-standard solution to this is Retrieval-Augmented Generation (RAG).

In this technical walkthrough, I'll explain how I built a fast, serverless "Chat with your PDF" application using Cloudflare Workers AI and Cloudflare Vectorize.

What is a RAG Pipeline?

Instead of relying on the AI's internal memory, a RAG pipeline intercepts the user's question, searches a private database for relevant context, and then hands both the context and the question to the AI. The AI simply reads the provided context to formulate its answer.

Step 1: Generating Text Embeddings

Before we can search a PDF, we must convert its text into numbers. This is called a Vector Embedding. An embedding represents the semantic meaning of a sentence. If two sentences mean the same thing, their vector numbers will be geometrically close to each other.

Using a Cloudflare Worker, we take the uploaded PDF text, split it into chunks, and pass it to an embedding model (like `bge-base-en-v1.5`).

// 1. Generate an embedding for a chunk of PDF text
const { data } = await env.AI.run('@cf/baai/bge-base-en-v1.5', {
  text: ["Cloudflare Vectorize is a serverless vector database..."]
});
const embeddingVector = data[0];

Step 2: Storing in Cloudflare Vectorize

Once we have the array of numbers (the vector), we need a specialized database that can perform lightning-fast mathematical comparisons. This is what a Vector Database does. We insert the vector into Cloudflare Vectorize.

// 2. Insert the vector into the database
await env.VECTOR_INDEX.upsert([{
  id: "chunk-123",
  values: embeddingVector,
  metadata: { text: "Cloudflare Vectorize is a serverless vector database..." }
}]);

Step 3: The Semantic Search Query

When the user asks a question, like "What is Vectorize?", we convert their question into a vector using the exact same embedding model. We then query the Vectorize database for the closest mathematical matches (which represent the most semantically relevant text chunks).

// 3. Query the database for the closest matches
const questionEmbedding = await env.AI.run('@cf/baai/bge-base-en-v1.5', { text: ["What is Vectorize?"] });

const matches = await env.VECTOR_INDEX.query(questionEmbedding.data[0], { 
  topK: 3, 
  returnValues: false, 
  returnMetadata: true 
});

// Extract the text from the top 3 matches
const contextText = matches.matches.map(m => m.metadata.text).join("\n");

Step 4: Augmented Generation

Finally, we inject the `contextText` into the system prompt of our LLM (e.g., Llama 3) along with the user's original question. The LLM acts purely as a reading-comprehension engine, utilizing the retrieved PDF text to craft a perfect, hallucination-free response.

Why Cloudflare for RAG?

Traditionally, building a RAG pipeline required stitching together an OpenAI API key, a dedicated Pinecone or Milvus vector database, and an AWS backend. By keeping the embedding model, the vector database, and the LLM generation entirely within the Cloudflare Edge ecosystem, latency is drastically reduced, and data never has to cross the public internet during the generation process.