Retrieval Augmented Generation

Retrieval Augmented Generation (RAG) enriches Large Language Model's (LLM's) responses with your own knowledge base data. Instead of relying only on what the model learned during training, RAG retrieves relevant passages from your data and integrates them directly into the prompt.

This guide builds a RAG pipeline using two IONOS CLOUD services:

• IONOS Managed PostgreSQL with the vector extension (pgvector) as the vector store

• IONOS CLOUD AI Model Hub for embedding generation and text generation

IONOS CLOUD hosts both services in German data centers to ensure your data remains within the country throughout the processing pipeline.

About this guide

This guide is intended for developers. It assumes you have basic knowledge of:

• REST APIs and how to call them

• SQL and PostgreSQL

• A programming language to handle REST API endpoints (for illustration purposes, the guide uses Python and Bash scripting)

You are familiar with:

• Text Generation

• Text Embeddings

• Managed PostgreSQL

By the end of this guide, you will be able to answer customer queries using an LLM which adds data from your own PostgreSQL knowledge base to the answers.

Background

• The IONOS CLOUD AI Model Hub API provides the LLMs and embedding models needed to implement Retrieval Augmented Generation (RAG), eliminating the need for manual hardware management.

• IONOS CLOUD Managed PostgreSQL includes the vector extension (pgvector), which adds a vector data type and similarity operators, your vector store is a standard PostgreSQL table, managed alongside the rest of your relational data.

• IONOS CLOUD hosts both services in German data centers to ensure your data remains within the country throughout the processing pipeline.

Plain text only: The AI Model Hub embedding API accepts plain text. If your source material is in PDF, Word, or another binary format, extract the text yourself (for example with PyMuPDF for PDFs or python-docx for Word documents) before embedding it.

Before you begin

To get started, you need the following:

• A running IONOS CLOUD Managed PostgreSQL cluster. For more information, see Set up a cluster in the DCD.

• The vector extension is activated on your database. For more information, see Activate Extensions and run CREATE EXTENSION vector CASCADE; on your database.

• An IONOS CLOUD API token with access to the AI Model Hub.

• An embedding model. This guide uses BAAI/bge-m3 (1024-dimensional output). For the complete list of available models, see Embedding Models.

• A nLLM for the final answer. For the full list, see Models.

Note: Use the Token Manager in the DCD to generate a new token if you encounter an HTTP 401 Unauthorized error.

Set up your environment

Python

To get started, open your IDE to enter Python code. The examples use the requests, psycopg, and pgvector libraries:

pip install requests "psycopg[binary]" pgvector

Generate a header object to authenticate against the AI Model Hub REST API and open a connection to your PostgreSQL cluster:

# Python example: authentication and PostgreSQL connection
import psycopg
from pgvector.psycopg import register_vector

IONOS_API_TOKEN = "[YOUR API TOKEN HERE]"
PG_DSN = "postgresql://[USER]:[PASSWORD]@[HOST]:5432/[DATABASE]"

header = {
    "Authorization": f"Bearer {IONOS_API_TOKEN}",
    "Content-Type": "application/json",
}

conn = psycopg.connect(PG_DSN, autocommit=True)
register_vector(conn)

After this step, you have a header object to call the AI Model Hub endpoints and a conn to run SQL on your PostgreSQL cluster.

Bash

Step 1: Create the vector store

Create a table for your text chunks and a vector index. The example matches the 1024-dimensional output of BAAI/bge-m3.

Python

# Python example: create the vector store
conn.execute("CREATE EXTENSION IF NOT EXISTS vector;")
conn.execute("""
    CREATE TABLE IF NOT EXISTS documents (
        id          bigserial    PRIMARY KEY,
        content     text         NOT NULL,
        embedding   vector(1024) NOT NULL
    );
""")
conn.execute("""
    CREATE INDEX IF NOT EXISTS documents_embedding_idx
        ON documents
        USING hnsw (embedding vector_cosine_ops);
""")

Bash

Note: If you choose a different embedding model, set the vector dimension to match the model's output size. For more information about each model's dimension, see Embedding Models.

Step 2: Generate embeddings and insert chunks

Split your documents into chunks. For each chunk, generate an embedding through the AI Model Hub and insert both the chunk text and its embedding into the documents table.

Chunking: Embedding a whole document as one vector loses detail. Split long texts into overlapping chunks of a few hundred tokens before embedding. A common starting point is 200–800 tokens per chunk with 10–20 percent overlap; tune this to your documents and your embedding model's context length.

Python

# Python example: Generate embeddings and insert chunks
import requests

EMBEDDINGS_ENDPOINT = "https://openai.inference.de-txl.ionos.com/v1/embeddings"
EMBEDDING_MODEL = "BAAI/bge-m3"

chunks = [
    "IONOS operates data centers in Germany.",
    "The AI Model Hub provides OpenAI-compatible inference endpoints.",
    "pgvector adds a vector data type and similarity operators to PostgreSQL.",
]

response = requests.post(
    EMBEDDINGS_ENDPOINT,
    json={"model": EMBEDDING_MODEL, "input": chunks},
    headers=header,
).json()

with conn.cursor() as cur:
    for chunk, item in zip(chunks, response["data"]):
        cur.execute(
            "INSERT INTO documents (content, embedding) VALUES (%s, %s)",
            (chunk, item["embedding"]),
        )

Bash

Step 3: Retrieve relevant chunks

To answer a user query, embed the query with the same model and run a cosine similarity search. The pgvector <=> operator returns cosine distance: lower values indicate higher similarity.

Python

# Python example: retrieve relevant chunks
USER_QUERY = "Where does IONOS run its AI infrastructure?"
TOP_K = 3

query_embedding = requests.post(
    EMBEDDINGS_ENDPOINT,
    json={"model": EMBEDDING_MODEL, "input": USER_QUERY},
    headers=header,
).json()["data"][0]["embedding"]

rows = conn.execute(
    "SELECT content FROM documents ORDER BY embedding <=> %s LIMIT %s",
    (query_embedding, TOP_K),
).fetchall()

relevant_chunks = [row[0] for row in rows]

Bash

This returns the TOP_K chunks most similar to the user query.

Step 4: Generate the final answer

Combine the retrieved chunks with the user query and send them to an LLM through the OpenAI-compatible /v1/chat/completions endpoint.

Python

# Python example: generate the final answer
CHAT_ENDPOINT = "https://openai.inference.de-txl.ionos.com/v1/chat/completions"
MODEL_ID = "[YOUR MODEL ID HERE]"

context = "\n\n".join(relevant_chunks)

answer = requests.post(
    CHAT_ENDPOINT,
    json={
        "model": MODEL_ID,
        "messages": [
            {
                "role": "system",
                "content": (
                    "Answer the question using only the provided context. "
                    "If the answer is not in the context, say that you do not know."
                ),
            },
            {
                "role": "user",
                "content": f"Context:\n{context}\n\nQuestion: {USER_QUERY}",
            },
        ],
        "temperature": 0.1,
        "max_tokens": 500,
    },
    headers=header,
).json()

print(answer["choices"][0]["message"]["content"])

Bash

The response is a standard OpenAI-compatible chat completion. The answer text is at choices[0].message.content.

Note: The best prompt strongly depends on the LLM used. Adapt your system and user messages to improve results for the specific Large Language Model.

Summary

In this guide, you learned how to use IONOS CLOUD Managed PostgreSQL with pgvector together with the IONOS CLOUD AI Model Hub to implement Retrieval Augmented Generation. Namely, you learned how to:

1. Prepare a PostgreSQL table as a pgvector-backed vector store.

2. Generate embeddings through the AI Model Hub embedding API.

3. Run cosine similarity queries to retrieve the chunks most relevant to a user question.

4. Combine the retrieved context with an LLM to produce a final answer, all within IONOS CLOUD, with your data staying in Germany.