Tool Calling

AI Model Hub for Free: From December 1, 2024, to June 30, 2025, IONOS is offering all foundation models in the AI Model Hub for free. Create your contract today and kickstart your AI journey!

The IONOS AI Model Hub supports tool calling, a feature of our OpenAI-compatible text generation API. Tool calling allows Large Language Models (LLMs) to trigger external functions and APIs during conversation, making the models interactive and dynamic. It significantly extends the LLM’s capabilities beyond pre-trained data or vector database retrieval, enabling real-time actions such as retrieving live weather data or interacting with your business systems.

Note: In modern AI APIs, the term "tool calling" has replaced "function calling." This tutorial uses the updated terminology in accordance with current standards.

Text generation models supporting tool calling

Only the following LLMs offer tool calling:

Model Provider	Model Name	Purpose
Meta (License)	Llama 3.3 Instruct (70B)	Best for conversational tasks and natural language processing. Outperforms Llama 3.1 405B at similar quality levels.
Meta (License)	Llama 3.1 Instruct (8B and 405B)	Ideal for chatbots, assistants, and dialogue-driven applications.
Mistral AI (License)	Mistral Instruct v0.3 (7B)	Strong in European languages; supports longer contexts; great for assistant-style use cases.

Unsupported Models: Teuken and Mixtral 8x7B do not support tool calling.

Overview

In this tutorial, you will learn how to integrate tool calling with a LLM through the OpenAI-compatible API to generate a mock weather forecast.

This tutorial is intended for developers with basic knowledge of:

• REST APIs

• A programming language capable of making HTTP requests (Python and Bash examples included)

• IONOS AI Model Hub's OpenAI compatible (text generation API)

• The concept of tool calling

By the end, you will be able to:

1. Define tool definitions that describe available external tools

2. Trigger tool calls based on user queries

3. Parse tool call responses to deliver final answers

Get started with tool calling

First, set up your environment and authenticate using the IONOS OpenAI-compatible API endpoint.

Download the respective code files to easily access tool calling-specific scripts and examples and generate the intended output:

Python Notebook

Download the Python Notebook to explore tool calling with ready-to-use examples.

16KB

ai-model-hub-tool-calling.ipynb

Python Code

Download the standalone Python script for a quick implementation.

4KB

ai-model-hub-tool-calling.py

Bash Code

Download the Bash script for a command-line implementation.

4KB

ai-model-hub-tool-calling.sh

Step 1: Retrieve available models

To begin, retrieve a list of available models that support tool calling. This helps you confirm which models we offer for your use case.

Python

import requests

IONOS_API_TOKEN = "[YOUR API TOKEN HERE]"

endpoint = "https://openai.inference.de-txl.ionos.com/v1/models"

header = {
    "Authorization": f"Bearer {IONOS_API_TOKEN}", 
    "Content-Type": "application/json"
}
requests.get(endpoint, headers=header).json()
print(response.json())

Bash

#!/bin/bash

IONOS_API_TOKEN=[YOUR API TOKEN HERE]

curl -H "Authorization: Bearer ${IONOS_API_TOKEN}" \
     --get https://openai.inference.de-txl.ionos.com/v1/models

The output will be a JSON document listing each model by name. You will reference one of these model names in later steps to perform text generation and tool calling operations.

Step 2: Define your functions

Tool calling requires you to define external functions that the model can invoke. These functions might fetch external data, trigger workflows, or perform custom computations.

In this example, we define a mock get_weather function that simulates retrieving current weather data.

Python

# Function to get current weather (mock implementation)
def get_weather(city: str, unit: str="celsius"):
    """Get the current weather in a given location"""
    # In a real implementation, this would call a weather API
    # This is a mock implementation
    if city.lower() == "berlin":
        temperature = 22 if unit == "celsius" else 72
        weather_data = {
            "city": city,
            "temperature": temperature,
            "unit": unit,
            "forecast": ["sunny", "windy"],
            "humidity": 70
        }
        return weather_data
    return {"error": "Location not supported"}

Bash

# Function to get current weather (mock implementation)
get_weather() {
    local city=$1
    local unit=$2
    
    # Convert location to lowercase for case-insensitive matching
    city_lower=$(echo "$city" | tr '[:upper:]' '[:lower:]')
    
    # Mock data for a few cities
    if [ "$city_lower" = "berlin" ]; then
        if [ "$unit" = "celsius" ]; then
            temperature=22
        else
            temperature=72
        fi
        echo '{"city": $city, "temperature": $temperature, "unit": $unit, "forecast": ["sunny", "windy"], "humidity": 70}'
    else
        echo {"error": "Location not supported"}
    fi
}

These functions form the core logic your AI model will invoke when processing relevant queries.

Step 3: Derive tool definition

Once you've created your function, define a tool definition that describes it in a format the model can understand. This includes the tool name, description, expected parameters, and parameter types.

Python

# Define the tool schema
tools = [
    {
        "type": "function",
        "function": {
            "name": "get_weather",
            "description": "Get the current weather in a given city",
            "parameters": {
                "type": "object",
                "properties": {
                    "city": {
                        "type": "string",
                        "description": "The city for which the weather should be returned"
                    },
                    "unit": {
                        "type": "string",
                        "enum": ["celsius", "fahrenheit"],
                        "description": "The unit of temperature to use. Is inferred from the city if possible."
                    }
                },
                "required": ["city", "unit"]
            }
        }
    }
]

Bash

# Define the tool schema as a JSON string
TOOLS='[
  {
    "type": "function",
    "function": {
      "name": "get_weather",
      "description": "Get the current weather in a given city",
      "parameters": {
        "type": "object",
        "properties": {
          "city": {
            "type": "string",
            "description": "The city for which the weather should be returned"
          },
          "unit": {
            "type": "string",
            "enum": ["celsius", "fahrenheit"],
            "description": "The unit of temperature to use. Is inferred from the city if possible."
          }
        },
        "required": ["city", "unit"]
      }
    }
  }
]'

Each tool definition must use clear, human-readable names and descriptions. It helps the model determine the most appropriate tool to call and which parameters to request or infer.

In the above example, the model may infer that a query about "Berlin" defaults to Celsius and "New York" to Fahrenheit. If it cannot infer the unit, it may ask the user for clarification.

To use multiple tools, simply add more definitions to the tools list.

Step 4: Enrich user query with available tool definitions

Now, send a user query with your defined tools using the OpenAI-compatible API.

Python

# Python example for tool calling
import requests
import json
from openai import OpenAI

IONOS_API_TOKEN = "[YOUR API TOKEN HERE]"
MODEL_NAME = "[MODEL NAME HERE]"
IONOS_API_URL = 'https://openai.inference.de-txl.ionos.com/v1'

# Initialize the OpenAI client with IONOS API URL
client = OpenAI(base_url=IONOS_API_URL, api_key=IONOS_API_TOKEN)

user_query = "What's the weather like in Berlin?"

body = {
    "model": MODEL_NAME,
    "messages": [
        {"role": "user", "content": user_query}
    ],
    "tools": tools,
    "tool_choice": "auto"  # Let the model decide when to call tools
}

response = client.chat.completions.create(**body)
print(json.dumps(response, default=lambda o: o.__dict__, indent=2))

Bash

#!/bin/bash
# Configuration
IONOS_API_TOKEN="[YOUR API TOKEN HERE]"
MODEL_NAME="[MODEL NAME HERE]"
IONOS_API_URL="https://openai.inference.de-txl.ionos.com/v1"

USER_QUERY="What's the weather like in Berlin?"

# Create messages array for the API call
MESSAGES='[
    {"role": "user", "content": "'"$USER_QUERY"'"}
]'

# Create the full request body
REQUEST_BODY="{
    \"model\": \"$MODEL_NAME\",
    \"messages\": $MESSAGES,
    \"tools\": $TOOLS,
    \"tool_choice\": \"auto\"
}"

# Make the API call to IONOS API
RESPONSE=$(curl -s -H "Authorization: Bearer ${IONOS_API_TOKEN}" \
     -H "Content-Type: application/json" \
     -d "$REQUEST_BODY" \
     $IONOS_API_URL/chat/completions)

echo -e "$RESPONSE"

The model will decide whether to call the tool based on the user query. For example, if the user asks about Berlin’s weather, the model might infer the unit as Celsius without prompting it explicitly.

Step 5: Process tool call response and execute tool

Once the LLM responds with a tool call, your application must handle the tool call by executing the corresponding function and returning the result to the user.

Python

# Parse tool call and arguments
if response.choices[0].message.tool_calls:
    tool_call = response.choices[0].message.tool_calls[0]
    function_name = tool_call.function.name
    function_args = json.loads(tool_call.function.arguments)
    
    # Call the function
    if function_name == "get_weather":
        tool_response = get_weather(
            city=function_args.get("city"),
            unit=function_args.get("unit")
        )
    print(tool_response)
else:
    print("The model provided a direct response (no tool call):")
    print(response.choices[0].message.content)

Bash

if echo "$RESPONSE" | jq -e '.choices[0].message.tool_calls' &> /dev/null; then
    TOOL_CALL_ID=$(echo "$RESPONSE" | jq -r '.choices[0].message.tool_calls[0].id')
    FUNCTION_NAME=$(echo "$RESPONSE" | jq -r '.choices[0].message.tool_calls[0].function.name')
    FUNCTION_ARGS=$(echo "$RESPONSE" | jq -r '.choices[0].message.tool_calls[0].function.arguments')
    
    # Extract location and unit from arguments
    LOCATION=$(echo "$FUNCTION_ARGS" | jq -r '.location')
    UNIT=$(echo "$FUNCTION_ARGS" | jq -r '.unit // "celsius"')
    
    # Get weather data
    WEATHER_DATA=$(get_weather "$LOCATION" "$UNIT")
    
    echo -e "Relevant answer: $WEATHER_DATA"
else
    # The model didn't call a tool, just return its response
    DIRECT_RESPONSE=$(echo "$RESPONSE" | jq -r '.choices[0].message.content')
    echo -e "The model provided a direct response (no tool call):"
    echo -e "Relevant answer: $DIRECT_RESPONSE"
fi

Tool call responses follow a predictable structure, making it easy to extract arguments and determine the function to execute. However, response behavior can vary based on the model, prompt structure, and tool definition quality.

You can also guide model behavior more precisely by:

• Using detailed system prompts

• Providing complete and clear tool descriptions

• Adjusting the temperature or other generation parameters

In real-world use cases, always validate parameters and add safety checks before executing external actions based on user prompts.

Summary

In this tutorial, you learned how to:

1. Create a mock weather function simulating a real API call

2. Define a tool definition based on the function

3. Prompt the LLM with a user query and tool definition

4. Parse the model’s response to extract tool call details

5. Execute the tool and return structured output

Tool calling enables your application to integrate LLMs with external data sources, APIs, or internal logic. This makes the LLM more powerful, responsive, and capable of handling real-time, or dynamic user scenarios.

For more information about other AI capabilities, see our documentation on text generation and image generation.