> For the complete documentation index, see [llms.txt](https://docs.ionos.com/cloud/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.ionos.com/cloud/ai/ai-model-hub/how-tos/rate-limits.md).

# Rate Limits

The <code class="expression">space.vars.ionos\_cloud\_ai\_model\_hub</code> offers scalable access to inference features like Chat Completions, Image Generation, and Embeddings through an OpenAI-compatible API. The API enforces rate limits on all incoming requests to ensure performance, fairness, and availability.

This article is intended to help both developers and technical decision-makers understand

* The benefits of rate limits for <code class="expression">space.vars.ionos\_cloud</code> and its customers,
* The types and values of rate limits applied,
* What happens when limits are exceeded?
* And how to track them effectively.

## Benefits of Rate Limits

Rate limits protect system integrity and ensure a reliable experience for all users by:

* **Preventing resource monopolization**: Limits prevent users or applications from consuming disproportionate resources.
* **Mitigating misuse and abuse**: They deter malicious activities such as scraping, brute-force attacks, or spamming.
* **Stabilizing performance under load**: During traffic spikes, limits help maintain responsiveness and availability.
* **Supporting long-term scalability**: Predictable request patterns enable better infrastructure planning and resource allocation.

## IONOS CLOUD AI Model Hub Rate Limits

The <code class="expression">space.vars.ionos\_cloud\_ai\_model\_hub</code> enforces the following default rate limits:

| Endpoint/Scope | Base Limit        | Burst Limit        | Notes                                                               |
| -------------- | ----------------- | ------------------ | ------------------------------------------------------------------- |
| Each model     | 5 requests/second | 10 requests/second | Applies separately to each model, including image generation models |
| Listing models | 5 requests/second | 10 requests/second | Applies to `GET /v1/models`                                         |

### Scopes Explained

* **Each model**: Every model has its own limit. Requests to the same model count toward its limit, whichever endpoint they use. For example, Chat Completions and Completions requests to the same model share one limit, while requests to another model count toward that model's limit.
* **Listing models**: Requests that list the available models have their own limit, separate from the limits of the models.

{% hint style="info" %}
**Contract-based enforcement:** Rate limits are enforced per <code class="expression">space.vars.ionos\_cloud</code> contract, not per user, and separately for each model. Your contract can have custom limits instead of these defaults.
{% endhint %}

### Understanding the Two-Tiered Rate Limit Model

<code class="expression">space.vars.ionos\_cloud</code> applies a two-tiered rate-limiting system consisting of a **base limit** and a **burst limit**.

* **Base limit**: The sustained request rate you are allowed to maintain over time. Staying within this rate ensures uninterrupted access.
* **Burst limit**: The maximum number of requests allowed within a 2-second window, accommodating sudden traffic spikes that exceed the sustained rate. Example: 20 requests in a 2-second burst.

**How it works:**

* The base limit of 5 requests/second is continuously replenished.
* You can temporarily exceed this rate by making up to 10 requests in a 2-second window, using the burst allowance.
* After using a full burst (e.g., 10 requests at once), you must wait for the base tokens to replenish at 5 requests per second, so it takes 2 seconds to regain the full burst capacity.
* The rate limit headers describe the base limit only. `X-RateLimit-Remaining-Requests` counts down from 5 and reaches `0` while the burst allowance still accepts up to 5 more requests. `X-RateLimit-Reset-Requests` includes the time to refill the burst allowance, so immediately after a full burst it shows about 2 seconds. For more information, see [<mark style="color:blue;">Hitting Rate Limits</mark>](#hitting-rate-limits).

This tiered model balances flexibility and fairness by allowing short bursts without compromising system stability.

## Hitting Rate Limits

When rate limits are exceeded, the API returns an `HTTP 429 Too Many Requests` response with a `Retry-After` header that gives the number of seconds to wait before retrying.

Additionally, every API response includes headers that provide real-time information about your current rate limit usage:

| Header                           | Description                                                                                                                                                                                                                                                          |
| -------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `X-RateLimit-Limit-Requests`     | Your base limit: the number of requests allowed per interval, for example `5` for 5 requests per second, without the burst allowance. The actual value depends on your contract.                                                                                     |
| `X-RateLimit-Remaining-Requests` | The number of requests remaining in the base limit, excluding the burst allowance. This value counts down from `X-RateLimit-Limit-Requests` and will not exceed it. To avoid a `429 Too Many Requests` error, pause or throttle your requests when this reaches `0`. |
| `X-RateLimit-Reset-Requests`     | The time until the rate limit is back at its initial state, with the full base limit and burst allowance available again. The value is a duration such as `200ms`, `1.5s`, or `6m0s`.                                                                                |
| `Retry-After`                    | The number of seconds to wait before retrying. This header appears on `429` and `529` responses.                                                                                                                                                                     |
| `X-Request-Id`                   | Identifier for this request, mirroring `X-Trace-ID`. Include it when reporting a problem to support.                                                                                                                                                                 |
| `X-Gateway-Version`              | The build version of the API that served the response.                                                                                                                                                                                                               |

{% hint style="info" %}
**Note:** The rate limit headers appear only on responses that were counted against a limit. A request rejected as malformed does not carry them.
{% endhint %}

Example response headers at the default limit of 5 requests per second, after the first request:

```bash
X-RateLimit-Limit-Requests: 5
X-RateLimit-Remaining-Requests: 4
X-RateLimit-Reset-Requests: 200ms
X-Gateway-Version: 1.4.0
```

Example headers on a `429` response, immediately after a full burst of 10 requests:

```bash
Retry-After: 1
X-RateLimit-Limit-Requests: 5
X-RateLimit-Remaining-Requests: 0
X-RateLimit-Reset-Requests: 1.99s
```

By monitoring these headers, clients can adjust their request rates dynamically and avoid unintentionally breaching limits.

## Service Protection During High Demand (HTTP 529)

To protect system integrity when overall demand for a model is exceptionally high across all customers, the <code class="expression">space.vars.ionos\_cloud\_ai\_model\_hub</code> can temporarily shed load on the platform side. In this case, the API returns an `HTTP 529 Site is Overloaded` response instead of processing the request.

Unlike a `429` response, a `529` response does not mean your contract has exceeded its quota or that your requests alone caused the overload. It means the platform is protecting overall service stability for all customers. Retry the request after the indicated delay.

Every `529` response includes a `Retry-After` header and an error body in the OpenAI-compatible error format:

```json
{
  "error": {
    "type": "server_overloaded",
    "code": "server_overloaded",
    "message": "The service is temporarily overloaded. Please try again shortly."
  }
}
```

| `error.type` / `error.code` | Meaning                                                                                                                         |
| --------------------------- | ------------------------------------------------------------------------------------------------------------------------------- |
| `user_overloaded`           | Your traffic pattern is contributing to the overload; reduce your request rate in addition to retrying.                         |
| `server_overloaded`         | The platform is out of capacity for the requested model, independent of your traffic; retrying later is the only action needed. |

Use `error.code` to decide how to react: back off for `user_overloaded`, or back off and consider failing over to a different model for `server_overloaded`.

{% hint style="info" %}
**Note:** The `message` field is intentionally generic and doesn't include timing information; always rely on the `Retry-After` header to know how long to wait.
{% endhint %}

## Mitigation Strategies and Best Practices

To avoid hitting rate limits and ensure smoother operation of your applications, consider the following mitigation strategies and best practices:

**1. Monitor quota in real time**

* Use rate limit headers to track your remaining quota.
* Distribute requests evenly rather than sending them all at once.

**2. Implement client-side throttling**

* Limit outgoing requests based on rate thresholds.
* Use algorithms like token bucket or leaky bucket to space out requests.

**3. Back off after a `429` or `529`**

* A `429` response carries a `Retry-After` header with the number of seconds until your limit accepts a request again. You must wait this exact duration before resubmitting. Client libraries that honor `Retry-After`, such as the `openai` Python library, manage this automatically.
* A `529` response carries a `Retry-After` header. Wait the number of seconds it gives before resubmitting, rather than a delay of your own choosing, so that clients do not all retry at the same moment.
* Use `X-RateLimit-Remaining-Requests` and `X-RateLimit-Reset-Requests` on successful responses to slow down before you reach the limit.
* A `529` is a platform-side signal rather than a quota issue. Reducing your request rate does not clear it, so retry only after the indicated delay.

**4. Build in resilience**

* Log failed and retried requests to detect rate-limit issues and system bottlenecks.
* Ensure your system can gracefully handle rate-limiting responses and temporary interruptions.

**5. Perform load testing**

* Simulate production traffic to identify potential rate-limit issues.
* Validate throttling and retry logic under high load conditions.

## Summary

Rate limits are essential for maintaining the reliability and scalability of your AI-powered applications. By understanding how the <code class="expression">space.vars.ionos\_cloud\_ai\_model\_hub</code> enforces these limits, you can optimize usage, plan for peak loads, and build resilient integrations.

For more, explore the official API documentation or consider integrating real-time quota monitoring into your observability stack.

* [<mark style="color:blue;">AI Model Hub API Documentation</mark>](https://docs.ionos.com/cloud/api-reference/ai/ai-model-hub/)
* [<mark style="color:blue;">AI Model Hub OpenAI-comp. API Documentation</mark>](https://api.ionos.com/docs/inference-openai/v1/)


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation by asking a question.

Perform an HTTP GET request on the following URL with the `ask` and `goal` query parameters:

```
GET https://docs.ionos.com/cloud/ai/ai-model-hub/how-tos/rate-limits.md?ask=<question>&goal=<user_goal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is what the user is ultimately trying to achieve, the reason they need the answer. Sharing it helps GitBook give you a better, more relevant answer. A goal is most helpful when it describes the outcome the user wants rather than restating the question. For example, with `ask=how do I create an API token`, a goal like `automate deployments from our CI pipeline` lets GitBook tailor the answer to that use case.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.