Tool Calling

AI Model Hub for Free: From December 1, 2024, to June 30, 2025, IONOS is offering all foundation models in the AI Model Hub for free. Create your contract today and kickstart your AI journey!

Note: The older term "function calling" is now deprecated in favor of "tool calling" in modern AI APIs. This tutorial uses the current standard terminology. However, "function calling" and "tool calling" stand for the same concept.

In typical conversational use cases, users often want a Large Language Model (LLM) to enrich responses with information from external tools or APIs. Imagine a chatbot that, among other features, answers questions about the current weather. Since LLMs are trained once and not updated daily, they cannot answer such real-time queries independently.

One possible workaround is to use vector databases, which store documents that can be retrieved during inference. But this is not always ideal:

1. Rapidly changing data like weather — Consider a chatbot that should tell users the current weather. Since weather conditions can change multiple times a day, constantly uploading fresh data to a vector database is inefficient and impractical. You need a way to fetch this information in real time, based on the user's question.

2. Structured tasks like meeting scheduling — Imagine a chatbot that helps users schedule meetings. You would not upload everyone's calendar into a vector database. Instead, you would want the model to collect necessary information (like time, participants, and topic) and pass it to a scheduling tool. It is a perfect use case for tool calling, where the LLM provides structured input to trigger an external action.

These use cases are better handled through tool calling.

Tool calling extends the capabilities of LLMs by enabling them to:

1. Recognize when external data or actions are required,

2. Ask the user for parameters needed to perform those actions, and

3. Respond in a structured, machine-readable format to initiate the tool execution.

This creates a seamless user experience where the LLM can fetch real-time data or perform actions outside its built-in capabilities.

Running example

Let us walk through an example where a tool retrieves the current weather. The tool requires two inputs:

• The city name

• The temperature unit (Celsius or Fahrenheit)

Tool definition

To use this tool, you define a tool definition using a JSON schema. The description of the tool includes the tool name, a tool description, relevant parameters, and their types:

{
  "type": "function",
  "function": {
    "name": "get_weather",
    "description": "Get current weather in a given city",
    "parameters": {
      "type": "object",
      "properties": {
        "city": {
          "type": "string",
          "description": "City name"
        },
        "unit": {
          "type": "string",
          "enum": ["celsius", "fahrenheit"]
        }
      },
      "required": ["city"]
    }
  }
}

This definition is sent to the LLM with the user prompt. It tells the model:

• Which tools are available

• What parameters each tool expects

• When to consider using a tool instead of generating a static response

Multiple Tools: You can pass a list of multiple tool definitions to the model. This allows the LLM to choose the appropriate tool for a given query.

Tool calls

The LLM may respond with a tool call when tool definitions are included in the prompt. A tool call appears as follows:

  tool_calls = [
    ChatCompletionMessageToolCall(
      id='chatcmpl-tool-d6edbee352f2432fa7eb5e68c30d712f', 
      function=Function(
        arguments='{"city": "Berlin", "unit": "celsius"}',
        name='get_weather'
      ), 
      type='function'
    )
  ]

The critical part is the function field, which includes:

• The name of the function (get_weather)

• The arguments with values the function should receive

In this example:

• city is "Berlin" (a string)

• unit is "celsius" (one of the allowed enum values)

This process is powerful because:

• The tool call matches the tool definition

• The values are either inferred by the model or collected from the user

Tool execution

At this point, the LLM’s job is done. It doesn’t execute the tool itself. That responsibility falls to your application:

1. Parse the model’s tool call response

2. Trigger the corresponding tool/API

3. Integrate the result into the ongoing conversation

Notice: Like all LLM outputs, tool calls are probabilistic. The model sometimes misunderstands required parameters or omits a tool call even when one is appropriate. Always validate tool calls and their parameters before executing actions on behalf of the user.

Explore further

To see tool calling in action, visit our Tool Calling tutorial.