Call Tools with LLMs

In this guide, you’ll learn how to have an LLM choose a tool to call given a prompt and a list of available tools.

To do this, you’ll build an AI chatbot that can interact with GitHub on your behalf using Arcade’s Github toolkit.

Prerequisites

Create an Arcade account
Get an Arcade API key and take note, you’ll need it in the next steps.

Install Arcade Client

pip install arcadepy

Instantiate the client

Create a new script called call_tools_with_llms.py and instantiate the OpenAI client pointing to Arcade’s endpoint and using the Arcade API key:

import os
from openai import OpenAI
 
# Initialize the OpenAI client, pointing to Arcade
client = OpenAI(
    base_url="https://api.arcade.dev/v1",
    api_key="arcade_api_key",
)

Set the user ID

Arcade needs a unique identifier for your application user (this could be an email address, a UUID, etc). In this example, simply use your email address as the user ID:

user_id = "[email protected]"

Determine which tools will be available to the chatbot agent

We are listing a few Github tools below. For an exhaustive list, refer to the Github toolkit. You could add tools from other toolkits here as well, but we’ll be sticking to Github for this example.

This list restricts which tools the LLM can choose to call based on any user prompt.

If you provide an empty list, the LLM will be able to call any tool available in the Arcade engine, from all toolkits.

tools = [
    "GitHub.SetStarred",
    "Github.ListStargazers",
    "Github.CreateIssue",
    "Github.CreateIssueComment",
    "Github.ListPullRequests",
    "Github.GetPullRequest",
    "Github.UpdatePullRequest",
    "Github.GetRepository",
    "Github.ListRepositoryActivities",
]

Generate a response

Use client.chat.completions.create to generate a response to a user’s prompt:

while True:
    # Ask the user for input
    prompt = input("Enter your prompt (type 'exit' to quit):")
    if prompt.lower() == "exit":
        break
 
    # Let the LLM choose a tool to use based on the user's prompt
    response = client.chat.completions.create(
        messages=[
            {"role": "system", "content": "You are a helpful assistant that can interact with GitHub."},
            {"role": "user", "content": prompt},
        ],
        model="gpt-4o",
        user=user_id,
        tools=tools,
        tool_choice="generate",
    )
 
    print(response.choices[0].message.content)

Try it!

Run the script:

python3 call_tools_with_llms.py

Ask the AI to:

star the ArcadeAI/arcade-ai repo

You’ll be prompted to authorize the connection to GitHub, and then the AI will perform the action.

How it works

When you use tool_choice="generate" and specify a list of tools, Arcade coordinates the interaction between the available tools and the AI model (OpenAI’s GPT-4o, in this case). To your users, it appears that the model magically has new capabilities.

In this example, Arcade Cloud Engine is executing the pre-built Github toolkit. You can also create your own toolkit.

Add more tools

Try adding more tools to the tools list / array to give the AI model more capabilities. As we mentioned earlier, you can add tools from other toolkits as well, such as the Slack toolkit:

For instance, add Slack.SendMessageToChannel and ask the LLM to send a message to a channel about a given Pull Request you retrieved previously from Github:

tools = [
    ...
    "Github.ListRepositoryActivities",
    "Slack.SendMessageToChannel",
]

Next steps

For apps and agents that aren’t directly generating chat completions from user prompts, you can authorize and call tools directly. This is useful when you need more control over the tool calling process.

Introduction Call Tools Directly