Reranking

The IONOS CLOUD AI Model Hub provides an OpenAI-compatible API that enables reranking of candidate documents against a query. Reranking models score the relevance between a query and each document using a cross-encoder architecture, making them ideal as a precision refinement step after an initial recall phase in two-stage retrieval pipelines.

Supported Reranking Models

The IONOS CLOUD AI Model Hub Models shows all models available for reranking. Refer to the relevant model cards for each reranking model's suitable use cases.

Overview

In this guide, you learn how to rerank documents using the OpenAI-compatible API. It targets developers who already understand:

• REST APIs.
• A programming language for working with REST endpoints (such as Python or Bash).
• The basics of retrieval pipelines and relevance scoring.

By the end, you will be able to:

1. Retrieve a list of available reranking models in the IONOS CLOUD AI Model Hub.
2. Use the API to rerank a set of candidate documents against a query.
3. Use the returned relevance scores to filter or sort results.

Getting Started with Reranking

To use reranking models, first set up your environment and authenticate using the OpenAI-compatible API endpoints.

Step 1: Retrieve available models

Fetch a list of models to see which reranking models are available for your use case:

Python

import requests

IONOS_API_TOKEN = "[YOUR API TOKEN HERE]"

endpoint = "https://openai.inference.de-txl.ionos.com/v1/models"

header = {
    "Authorization": f"Bearer {IONOS_API_TOKEN}",
    "Content-Type": "application/json"
}
requests.get(endpoint, headers=header).json()

Bash

#!/bin/bash

IONOS_API_TOKEN=[YOUR API TOKEN HERE]

curl -H "Authorization: Bearer ${IONOS_API_TOKEN}" \
        --get https://openai.inference.de-txl.ionos.com/v1/models

Output

      {
         "id":"Qwen/Qwen3-VL-Reranker-8B",
         "object":"model",
         "created":1677610602,
      },

This query returns a JSON document that lists each model's name, which you will use to specify a model for reranking in later steps.

Step 2: Rerank documents

To rerank a set of candidate documents, send the query and documents to the /rerank endpoint.

The request accepts the following fields:

Field	Required	Description
model	Yes	The reranking model to use.
query	Yes	The query to rank against. Either a string, or an array of content parts for a multi-modal query.
documents	Yes	Between 1 and 1000 candidate documents. Each is either a string, or an object with a content part array.
top_n	No	Return only the highest-scoring documents. Must be at least 1. Defaults to all documents.
return_documents	No	Include the original document in each result. Defaults to false.

Python

import requests

IONOS_API_TOKEN = "[YOUR API TOKEN HERE]"
MODEL_NAME = "[MODEL NAME HERE]"
QUERY = "What is the capital of France?"
DOCUMENTS = [
    "Paris is the capital of France.",
    "London is the capital of England.",
    "Berlin is the capital of Germany."
]

endpoint = "https://openai.inference.de-txl.ionos.com/v1/rerank"

header = {
    "Authorization": f"Bearer {IONOS_API_TOKEN}",
    "Content-Type": "application/json"
}
body = {
    "model": MODEL_NAME,
    "query": QUERY,
    "documents": DOCUMENTS,
    "top_n": 2,
    "return_documents": True
}
result = requests.post(endpoint, json=body, headers=header).json()

Bash

#!/bin/bash

IONOS_API_TOKEN=[YOUR API TOKEN HERE]
MODEL_NAME=[MODEL NAME HERE]

BODY='{
    "model": "'"$MODEL_NAME"'",
    "query": "What is the capital of France?",
    "documents": [
        "Paris is the capital of France.",
        "London is the capital of England.",
        "Berlin is the capital of Germany."
    ],
    "top_n": 2,
    "return_documents": true
}'

curl -X POST -H "Authorization: Bearer ${IONOS_API_TOKEN}" \
     -H "Content-Type: application/json" \
     -d "$BODY" \
     https://openai.inference.de-txl.ionos.com/v1/rerank

Rank against an Image Query

The query can be an array of content parts instead of a string, which lets you rank documents against an image. Parts use the same names as chat completions: text, image_url, and video_url.

Python

body = {
    "model": MODEL_NAME,
    "query": [
        {"type": "text", "text": "Which product photo matches this reference?"},
        {"type": "image_url", "image_url": {"url": "https://example.com/reference.png"}}
    ],
    "documents": [
        {"content": [{"type": "image_url", "image_url": {"url": "https://example.com/candidate-1.png"}}]},
        {"content": [{"type": "image_url", "image_url": {"url": "https://example.com/candidate-2.png"}}]}
    ]
}
result = requests.post(endpoint, json=body, headers=header).json()

Documents supplied as objects use a content array that supports multiple part types, allowing text and images to be mixed. Use the index field to map results back to your source documents. When return_documents is true, the full document is returned with all original content parts and images.

Image Size and Token Budget

When reranking image documents, the model tokenizes each image at 1 token per 32×32 pixel block. Images exceeding 1,310,720 pixels in total are downscaled proportionally before tokenization. Each (query, document) pair must fit within the 32,768-token context window.

To estimate the token cost of an image:

1. If width × height > 1,310,720, apply the scale factor: scale = √(1,310,720 / (width × height))
2. Round the scaled dimensions down to the nearest multiple of 32.
3. Calculate: tokens = (scaled_width / 32) × (scaled_height / 32)

Image resolution	Total pixels	Tokens after downscaling
512 × 512	262,144	~256
1024 × 768	786,432	~756
1296 × 1936	2,507,616	~1,247
1920 × 1080	2,073,600	~1,247

With ~200 tokens reserved for the query and prompt overhead, a single document can hold approximately 25 images at 1296×1936 px before reaching the context limit. Use smaller images or lower resolutions to fit more documents per request.

The token budget is separate from the size of the request itself. For the accepted formats and the request size limits, see Image Input.

Step 3: Use relevance scores

The returned JSON includes the following key fields:

• results.[..].index: The position of the document in the original input list.
• results.[..].document: The scored document. This object contains either a text field for text documents or a content field for multimodal documents. It is only included when the request sets return_documents to true. Otherwise, use the index field to map each result back to your input.
• results.[..].relevance_score: A score between 0 and 1 indicating how relevant the document is to the query. Higher values indicate greater relevance.
• usage.prompt_tokens: Token count for the input.
• usage.total_tokens: Token count for the entire request.

Using Python, you can filter results above a relevance threshold:

Python

import requests

IONOS_API_TOKEN = "[YOUR API TOKEN HERE]"
MODEL_NAME = "Qwen/Qwen3-VL-Reranker-8B"
QUERY = "What is the capital of France?"
DOCUMENTS = [
    "Paris is the capital of France.",
    "London is the capital of England.",
    "Berlin is the capital of Germany."
]
THRESHOLD = 0.5

endpoint = "https://openai.inference.de-txl.ionos.com/v1/rerank"

header = {
    "Authorization": f"Bearer {IONOS_API_TOKEN}",
    "Content-Type": "application/json"
}
body = {
    "model": MODEL_NAME,
    "query": QUERY,
    "documents": DOCUMENTS,
    "return_documents": True
}
result = requests.post(endpoint, json=body, headers=header).json()

relevant = [
    r for r in result["results"]
    if r["relevance_score"] >= THRESHOLD
]

# [{'index': 0, 'document': {'text': 'Paris is the capital of France.'}, 'relevance_score': 0.9234}]

For a list of possible error codes and handling instructions, see Error Codes.

What You Learned

In this guide, you learned how to:

1. Access available reranking models.
2. Rerank a set of candidate documents against a query.
3. Filter results using relevance score thresholds.

For information on how to use reranking in a full retrieval pipeline with embeddings and document collections, see Retrieval Augmented Generation.