Tool Calling

The IONOS AI Model Hub supports tool calling, a feature of our OpenAI-compatible text generation API. Tool calling allows Large Language Models (LLMs) to trigger external functions and APIs during conversation, making the models interactive and dynamic. It significantly extends the LLM’s capabilities beyond pre-trained data or vector database retrieval, enabling real-time actions such as retrieving live weather data or interacting with your business systems.

Note: In modern AI APIs, the term "tool calling" has replaced "function calling." This guide uses the updated terminology in accordance with current standards.

Text generation models supporting tool calling

Not all Large Language Models on the AI Model Hub models list support tool calling. To find out which ones are compatible, check the individual model cards.

Overview

In this guide, you will learn how to integrate tool calling with a LLM through the OpenAI-compatible API to generate a mock weather forecast.

This guide is intended for developers with basic knowledge of:

• REST APIs

• A programming language capable of making HTTP requests (Python and Bash examples included)

• IONOS AI Model Hub's OpenAI compatible (text generation API)

• The concept of tool calling

By the end, you will be able to:

1. Define tool definitions that describe available external tools

2. Trigger tool calls based on user queries

3. Parse tool call responses to deliver final answers

Get started with tool calling

First, set up your environment and authenticate using the IONOS OpenAI-compatible API endpoint.

Download the respective code files to easily access tool calling-specific scripts and examples and generate the intended output:

Python Notebook

Download the Python Notebook to explore tool calling with ready-to-use examples.

16KB

ai-model-hub-tool-calling.ipynb

Open

Python Code

Bash Code

Step 1: Retrieve available models

To begin, retrieve a list of available models that support tool calling. This helps you confirm which models we offer for your use case.

Python

import requests

IONOS_API_TOKEN = "[YOUR API TOKEN HERE]"

endpoint = "https://openai.inference.de-txl.ionos.com/v1/models"

header = {
    "Authorization": f"Bearer {IONOS_API_TOKEN}", 
    "Content-Type": "application/json"
}
requests.get(endpoint, headers=header).json()
print(response.json())

Bash

The output will be a JSON document listing each model by name. You will reference one of these model names in later steps to perform text generation and tool calling operations.

Step 2: Define your functions

Tool calling requires you to define external functions that the model can invoke. These functions might fetch external data, trigger workflows, or perform custom computations.

In this example, we define a mock get_weather function that simulates retrieving current weather data.

Python

# Function to get current weather (mock implementation)
def get_weather(city: str, unit: str="celsius"):
    """Get the current weather in a given location"""
    # In a real implementation, this would call a weather API
    # This is a mock implementation
    if city.lower() == "berlin":
        temperature = 22 if unit == "celsius" else 72
        weather_data = {
            "city": city,
            "temperature": temperature,
            "unit": unit,
            "forecast": ["sunny", "windy"],
            "humidity": 70
        }
        return weather_data
    return {"error": "Location not supported"}

Bash

These functions form the core logic your AI model will invoke when processing relevant queries.

Step 3: Derive tool definition

Once you've created your function, define a tool definition that describes it in a format the model can understand. This includes the tool name, description, expected parameters, and parameter types.

Python

# Define the tool schema
tools = [
    {
        "type": "function",
        "function": {
            "name": "get_weather",
            "description": "Get the current weather in a given city",
            "parameters": {
                "type": "object",
                "properties": {
                    "city": {
                        "type": "string",
                        "description": "The city for which the weather should be returned"
                    },
                    "unit": {
                        "type": "string",
                        "enum": ["celsius", "fahrenheit"],
                        "description": "The unit of temperature to use. Is inferred from the city if possible."
                    }
                },
                "required": ["city", "unit"]
            }
        }
    }
]

Bash

Each tool definition must use clear, human-readable names and descriptions. It helps the model determine the most appropriate tool to call and which parameters to request or infer.

In the above example, the model may infer that a query about "Berlin" defaults to Celsius and "New York" to Fahrenheit. If it cannot infer the unit, it may ask the user for clarification.

To use multiple tools, simply add more definitions to the tools list.

Step 4: Enrich user query with available tool definitions

Now, send a user query with your defined tools using the OpenAI-compatible API.

Python

# Python example for tool calling
import requests
import json
from openai import OpenAI

IONOS_API_TOKEN = "[YOUR API TOKEN HERE]"
MODEL_NAME = "[MODEL NAME HERE]"
IONOS_API_URL = 'https://openai.inference.de-txl.ionos.com/v1'

# Initialize the OpenAI client with IONOS API URL
client = OpenAI(base_url=IONOS_API_URL, api_key=IONOS_API_TOKEN)

user_query = "What's the weather like in Berlin?"

body = {
    "model": MODEL_NAME,
    "messages": [
        {"role": "user", "content": user_query}
    ],
    "tools": tools,
    "tool_choice": "auto"  # Let the model decide when to call tools
}

response = client.chat.completions.create(**body)
print(json.dumps(response, default=lambda o: o.__dict__, indent=2))

Bash

The model will decide whether to call the tool based on the user query. For example, if the user asks about Berlin’s weather, the model might infer the unit as Celsius without prompting it explicitly.

Step 5: Process tool call response and execute tool

Once the LLM responds with a tool call, your application must handle the tool call by executing the corresponding function and returning the result to the user.

Python

# Parse tool call and arguments
if response.choices[0].message.tool_calls:
    tool_call = response.choices[0].message.tool_calls[0]
    function_name = tool_call.function.name
    function_args = json.loads(tool_call.function.arguments)
    
    # Call the function
    if function_name == "get_weather":
        tool_response = get_weather(
            city=function_args.get("city"),
            unit=function_args.get("unit")
        )
    print(tool_response)
else:
    print("The model provided a direct response (no tool call):")
    print(response.choices[0].message.content)

Bash

Tool call responses follow a predictable structure, making it easy to extract arguments and determine the function to execute. However, response behavior can vary based on the model, prompt structure, and tool definition quality.

You can also guide model behavior more precisely by:

• Using detailed system prompts

• Providing complete and clear tool descriptions

• Adjusting the temperature or other generation parameters

In real-world use cases, always validate parameters and add safety checks before executing external actions based on user prompts.

Summary

In this guide, you learned how to:

1. Create a mock weather function simulating a real API call

2. Define a tool definition based on the function

3. Prompt the LLM with a user query and tool definition

4. Parse the model’s response to extract tool call details

5. Execute the tool and return structured output

Tool calling enables your application to integrate LLMs with external data sources, APIs, or internal logic. This makes the LLM more powerful, responsive, and capable of handling real-time, or dynamic user scenarios.

For more information about other AI capabilities, see our documentation on text generation and image generation.