2.9 Tool Calling - Giving LLMs the Ability to Use External APIs

This recipe is going to cover one of the most exciting features of LLMs today — Tool Calling. In LLMs, tool calling also known as function calling, allows the model to invoke functions you have define to perform specific tasks, such as fetching real-time data from external APIs, performing calculations, or interacting with databases.

Unlike traditional RAG (Retrieval-Augmented Generation) where the model relies on us providing context from external sources, tool calling allows the model to automatically determine which tools at its disposal to use in order to answer the user’s query or perform a task.

This is a very powerful capability and mastering this takes a step closer to building agents and autonomous systems that can perform complex tasks by their own. We, developers, have seen the power of this with coding agents like GitHub Copilot, Claude which can use a variety of tools to help developers write code faster.

In this recipe, we will build a simple AI assistant that can answer user queries by calling various tools we define. Since this is the first time we are introducing tool calling, we will keep the tools simple but useful and practical.

We will implement the following tools:

1. Calculator Tool: Performs basic mathematical operations (add, subtract, multiply, divide).
2. Weather Tool: Fetches real-time weather data from a public API.
3. Web Scraper Tool: Extracts title and meta description from any webpage.
4. Timezone Tool: Gets current time in any timezone using built-in Node.js capabilities.

By the end of this recipe, you should be able to ask the AI assistant questions such as: “What is 25 multiplied by 4?”, “What’s the weather in London?”, “What’s on the homepage of github.com?”, and “What time is it in Tokyo?” and receive accurate responses powered by the respective tools. On top of that, you can combine multiple tools in a single query as well, such as “What’s the weather in New York and Tokyo, and what time is it there?” and receive a complete response.

📋 Prerequisites and Setup

You should have your environment set up for Genkit development. Follow the Getting Started guide if you haven't done so already.

I have a Genkit Starter Template that you can use to quickly spin up a new project with everything configured. You can find it here.

🚀 Getting Started

1. Install Dependencies

pnpm add  cheerio @genkit-ai/google-genai genkit @genkit-ai/express

The above command installs the necessary dependencies for our project:

Dependency	Description
cheerio	A package for parsing and manipulating HTML, used for web scraping.
genkit	The core Genkit library for building AI applications.
@genkit-ai/google-genai	Genkit plugin for integrating Google AI models (Gemini) - you can replace this with model of your choice
@genkit-ai/express	Genkit plugin for integrating with Express.js.

Genkit Tool Definition

In Genkit, tools are defined using the ai.defineTool() method. Each tool has a parameter object containing: name, description, input and output schema, and an implementation callback function that contains the logic for the tool. The implementation function is where you can integrate external APIs, perform calculations, or any other task the tool is designed to do.

Parameter	Description
name	A unique name for the tool.
description	A brief description of what the tool does - the LLM uses this to understand when to call the tool.
input	A Zod schema defining the expected input parameters for the tool.
output	A Zod schema defining the expected output format of the tool.

Let’s define a simple Calculator Tool as an example. Our calculator tool will perform basic mathematical operations such as addition, subtraction, multiplication, and division. With this in mind, our tool would need to accept three inputs: the first number, the second number, and the operation to perform (add, subtract, multiply, divide). The output will be a single number which is the result of the operation.

export const calculatorTool = ai.defineTool(
  {
    name: 'calculator',
    description:
      'Performs basic mathematical calculations. Use this tool for addition, subtraction, multiplication, and division operations.',
    inputSchema: z.object({
      operation: z
        .enum(['add', 'subtract', 'multiply', 'divide'])
        .describe('The mathematical operation to perform'),
      a: z.number().describe('The first number'),
      b: z.number().describe('The second number'),
    }),
    outputSchema: z.object({
      result: z.number().describe('The result of the calculation'),
      expression: z.string().describe('The expression that was evaluated'),
    }),
  },
  async (input) => {
    const { operation, a, b } = input;
    let result: number;

    switch (operation) {
      case 'add':
        result = a + b;
        break;
      case 'subtract':
        result = a - b;
        break;
      case 'multiply':
        result = a * b;
        break;
      case 'divide':
        if (b === 0) {
          throw new Error('Cannot divide by zero');
        }
        result = a / b;
        break;
    }

    const operatorSymbol = {
      add: '+',
      subtract: '-',
      multiply: '*',
      divide: '/',
    }[operation];

    return {
      result,
      expression: `${a} ${operatorSymbol} ${b} = ${result}`,
    };
  },
);

A few things to note about the above code:

• We define the tool using ai.defineTool() method, providing the necessary parameters.
• The input schema uses Zod to define the expected inputs: operation, a, and b. Each input has a description to help the LLM understand its purpose.
• The output schema defines the expected output format: result and expression.
• The implementation function performs the actual calculation based on the provided operation and numbers, and is no different from any regular TypeScript/JavaScript function.

When the LLM needs to perform a calculation, it can call this tool by name (calculator), providing the required inputs. The tool will execute the logic and return the result in the specified output format to the LLM, so that it can generate a response to the user.

Once we have define our tool, we can pass it to our LLM during prompt or generation time. For example:

ai.generate({
  // other params such as prompt, model, etc.
  tools: [calculatorTool],
});

In this recipe, we will use prompts instead of direct generation calls, but the concept is the same. For more details on defining tools, check out the previous recipe on Re-usable Prompts in Genkit.

The LLM will now have access to the calculator tool and can call it automatically when it determines that a calculation is needed to answer the user.

Defining More Tools

Now that we understand how to define a tool, let’s briefly look at how the other tools are defined in this recipe. Each tool follows the same pattern as the calculator tool, with different input/output schemas and implementation logic.

Weather Tool

The Weather Tool fetches real-time weather data from the Open-Meteo API based on the provided city name. It uses the Geocoding API to convert the city name to latitude and longitude, then fetches the current weather data.

export const weatherTool = ai.defineTool(
  {
    name: 'weather',
    description:
      'Fetches current weather information for a specified city using Open-Meteo API. Returns temperature, conditions, humidity, and wind speed.',
    inputSchema: z.object({
      city: z
        .string()
        .describe(
          'The name of the city to get weather for (e.g., "London", "New York", "Tokyo")',
        ),
    }),
    outputSchema: z.object({
      city: z.string(),
      temperature: z.number().describe('Temperature in Celsius'),
      feelsLike: z.number().describe('Apparent temperature in Celsius'),
      humidity: z.number().describe('Relative humidity percentage'),
      windSpeed: z.number().describe('Wind speed in km/h'),
      weatherCode: z.number().describe('WMO weather code'),
    }),
  },
  async (input) => {
    try {
      // Step 1: Geocode the city name to get coordinates
      const geocodeUrl = new URL(
        'https://geocoding-api.open-meteo.com/v1/search',
      );
      geocodeUrl.searchParams.set('name', input.city);
      geocodeUrl.searchParams.set('count', '1');
      geocodeUrl.searchParams.set('language', 'en');
      geocodeUrl.searchParams.set('format', 'json');

      const geocodeResponse = await fetch(geocodeUrl.toString());
      if (!geocodeResponse.ok) {
        throw new Error(`Geocoding failed: ${geocodeResponse.statusText}`);
      }

      const geocodeData = await geocodeResponse.json();
      if (!geocodeData.results || geocodeData.results.length === 0) {
        throw new Error(
          `City "${input.city}" not found. Please check the spelling and try again.`,
        );
      }

      const location = geocodeData.results[0];
      const { latitude, longitude, name } = location;

      // Step 2: Fetch current weather data using coordinates
      const weatherUrl = new URL('https://api.open-meteo.com/v1/forecast');
      weatherUrl.searchParams.set('latitude', latitude.toString());
      weatherUrl.searchParams.set('longitude', longitude.toString());
      weatherUrl.searchParams.set(
        'current',
        'temperature_2m,relative_humidity_2m,apparent_temperature,wind_speed_10m,weather_code',
      );
      weatherUrl.searchParams.set('timezone', 'auto');

      const weatherResponse = await fetch(weatherUrl.toString());
      if (!weatherResponse.ok) {
        throw new Error(`Weather API failed: ${weatherResponse.statusText}`);
      }

      const weatherData = await weatherResponse.json();
      const current = weatherData.current;

      return {
        city: name,
        temperature: Math.round(current.temperature_2m * 10) / 10,
        feelsLike: Math.round(current.apparent_temperature * 10) / 10,
        humidity: current.relative_humidity_2m,
        windSpeed: current.wind_speed_10m,
        weatherCode: current.weather_code,
      };
    } catch (error) {
      if (error instanceof Error && error.message.includes('not found')) {
        throw error;
      }
      throw new Error(
        `Failed to fetch weather data: ${error instanceof Error ? error.message : 'Unknown error'}`,
      );
    }
  },
);

A few things to note about the Weather Tool:

• It uses the Open-Meteo Geocoding API to convert city names to latitude and longitude.
• It then fetches current weather data from the Open-Meteo Weather API using the obtained coordinates.
• It includes error handling to provide clear messages if the city is not found or if the API calls fail.

Web Scraper Tool

The Web Scraper Tool extracts the title and meta description from any webpage using the Cheerio library.

export const webScraperTool = ai.defineTool(
  {
    name: 'webScraper',
    description:
      'Fetches a webpage and extracts its title and meta description. Use this to get basic information about any URL.',
    inputSchema: z.object({
      url: z
        .string()
        .url()
        .describe('The URL of the webpage to scrape (must be a valid URL)'),
    }),
    outputSchema: z.object({
      url: z.string(),
      title: z.string().describe('The page title'),
      description: z
        .string()
        .describe('The meta description or a fallback message'),
      success: z.boolean(),
    }),
  },
  async (input) => {
    try {
      const controller = new AbortController();
      const timeoutId = setTimeout(() => controller.abort(), 10000); // 10 second timeout

      const response = await fetch(input.url, {
        signal: controller.signal,
        headers: {
          'User-Agent': 'Mozilla/5.0 (compatible; GenkitBot/1.0)',
        },
      });

      clearTimeout(timeoutId);

      if (!response.ok) {
        throw new Error(`HTTP ${response.status}: ${response.statusText}`);
      }

      const html = await response.text();
      const $ = cheerio.load(html);

      const title = $('title').text().trim() || 'No title found';
      const description =
        $('meta[name="description"]').attr('content') ||
        $('meta[property="og:description"]').attr('content') ||
        'No description found';

      return {
        url: input.url,
        title,
        description,
        success: true,
      };
    } catch (error) {
      if (error instanceof Error) {
        if (error.name === 'AbortError') {
          throw new Error(`Request timed out for URL: ${input.url}`);
        }
        if (error.message.includes('fetch failed')) {
          throw new Error(`Could not resolve domain for URL: ${input.url}`);
        }
      }
      throw new Error(
        `Failed to scrape webpage: ${error instanceof Error ? error.message : 'Unknown error'}`,
      );
    }
  },
);

a few things to note about the Web Scraper Tool:

• It uses the Cheerio library to parse HTML and extract the title and meta description.
• It includes error handling for network issues, timeouts, and invalid URLs.

Timezone Tool

The Timezone Tool gets the current time in any specified timezone using built-in Node.js capabilities.

export const timezoneTool = ai.defineTool(
  {
    name: 'timezone',
    description:
      'Gets the current date and time in a specified timezone. Use IANA timezone names like "America/New_York", "Europe/London", "Asia/Tokyo".',
    inputSchema: z.object({
      timezone: z
        .string()
        .describe(
          'The IANA timezone name (e.g., "America/New_York", "Europe/London", "Asia/Tokyo")',
        ),
    }),
    outputSchema: z.object({
      timezone: z.string(),
      currentTime: z
        .string()
        .describe('The current time in the specified timezone'),
      date: z.string().describe('The current date in ISO format'),
      offset: z.string().describe('The UTC offset'),
    }),
  },
  async (input) => {
    try {
      const now = new Date();

      // Format time in the specified timezone
      const timeFormatter = new Intl.DateTimeFormat('en-US', {
        timeZone: input.timezone,
        year: 'numeric',
        month: 'long',
        day: 'numeric',
        hour: '2-digit',
        minute: '2-digit',
        second: '2-digit',
        hour12: true,
      });

      // Get UTC offset
      const offsetFormatter = new Intl.DateTimeFormat('en-US', {
        timeZone: input.timezone,
        timeZoneName: 'short',
      });

      const parts = offsetFormatter.formatToParts(now);
      const timeZonePart = parts.find((part) => part.type === 'timeZoneName');
      const offset = timeZonePart?.value || 'Unknown';

      // Get ISO date
      const isoDate = new Date(
        now.toLocaleString('en-US', { timeZone: input.timezone }),
      ).toISOString();

      return {
        timezone: input.timezone,
        currentTime: timeFormatter.format(now),
        date: isoDate.split('T')[0],
        offset,
      };
    } catch (error) {
      throw new Error(
        `Invalid timezone "${input.timezone}". Please use a valid IANA timezone name like "America/New_York" or "Europe/London".`,
      );
    }
  },
);

A few things to note about the Timezone Tool:

• It uses the built-in Intl.DateTimeFormat to format dates and times in the specified timezone.
• It includes error handling for invalid timezone names.

Using the Tools in Prompts

With our tools defined, we can now use them in our prompts. In Genkit, we can pass the tools to the prompt invocation, and the LLM will automatically decide when to call which tool based on the user’s query.

We can create a dotprompt file assistant.prompt to define how the AI assistant should behave and which tools it has access to:

---
input:
  schema:
    query: string
tools:
  - calculator
  - weather
  - webScraper
  - timezone
---

You are a helpful AI assistant with access to various tools.

When a user asks a question:

- Analyze if any tools can help answer the question
- Call the appropriate tool(s) with the correct parameters
- Use the tool results to provide a clear, helpful response
- If no tools are needed, respond naturally from your knowledge

Always be concise and friendly in your responses.

User Query: {{query}}

We are specifying the tools available to the LLM in the tools section. The LLM will use the tool descriptions to determine when to call each tool based on the user’s query.

When invoking the prompt in our flow, we simply pass the user’s query and the tools:

export const assistantFlow = ai.defineFlow(
  {
    name: 'assistantFlow',
    inputSchema: z.object({
      query: z.string().describe("The user's question or request"),
    }),
    outputSchema: z.string().describe('The assistant response to the user'),
  },
  async (input) => {
    console.log('\n🔷 New query received:', input.query);

    // Use dotprompt template - tools are configured in the .prompt file
    const { text } = await assistantPrompt(
      {
        query: input.query,
      },
      {
        // List of tools to provide to the LLM
        tools: [calculatorTool, weatherTool, webScraperTool, timezoneTool],
      },
    );

    console.log(`✅ Response generated`);

    return text;
  },
);

The LLM will autonomously analyze the userQuery, decide if any tools can help, call the appropriate tools with the correct parameters, and use the tool results to generate a response. No manual routing or tool invocation is required!

Now, we can use the Genkit Developer UI to test our tools individually or the entire assistant flow. The LLM should be able to answer a variety of queries by calling the appropriate tools behind the scenes.

Starting the Express Server

We can integrate our assistant flow into an Express server using the @genkit-ai/express plugin. Here’s a simple server setup:

import { startFlowServer } from '@genkit-ai/express';
import { ai } from './genkit';
import { assistantFlow } from './flows';

startFlowServer({
  flows: [assistantFlow],
  port: 3000,
});

And we can start the server with:

pnpm tsx src/index.ts

Remember to replace src/index.ts with the actual path to your server file and also set the GEMINI_API_KEY in your environment variables.

Once the server is running, you can send POST requests to the /assistantFlow endpoint with a JSON body that looks like this:

{
  "data": {
    "query": "What is the weather in Paris and what time is it there?"
  }
}

Here is a curl example:

curl -X POST http://localhost:3000/assistantFlow \
  -H "Content-Type: application/json" \
  -d '{"data": {"query": "What is the weather in Paris and what time is it there?"}}'

We can test various queries to see how the assistant uses the tools to respond, such as:

# Single operation
curl -X POST http://localhost:3400/assistant \
  -H "Content-Type: application/json" \
  -d '{"query": "What is 15 plus 27?"}'

# Multiple operations
curl -X POST http://localhost:3400/assistant \
  -H "Content-Type: application/json" \
  -d '{"query": "What is 100 divided by 4 and then multiplied by 3?"}'

Be creative and try combining different tools in a single query to see how well the assistant can handle complex requests!

Conclusion

In this recipe, we explored how to define and use tools in Genkit to build an AI assistant capable of answering user queries by calling various tools. We covered the definition of four practical tools: Calculator, Weather, Web Scraper, and Timezone. We also saw how to integrate these tools into a prompt and use them in an Express server.

With tool calling, we can create more dynamic and capable AI applications that can interact with external systems and provide real-time information. This opens up many possibilities for building intelligent agents and autonomous systems.

Try extending this recipe by adding more tools or enhancing the existing ones to make the assistant even more powerful such as:

• Adding more mathematical functions to the Calculator Tool
• Integrating more data sources in the Weather Tool
• Enhancing the Web Scraper Tool to extract more information
• Adding timezone conversion capabilities to the Timezone Tool

🚀 Get the Complete Code

This recipe includes a full GitHub repository with working code, setup instructions, and examples. Unlock access to build faster and support this cookbook!

Complete, tested code repositories

Copy-paste ready examples

Lifetime access with updates

Support future recipes & content

Unlock All Recipes - 50% Off Already purchased? Sign in