Use Stores with LiteLLM (Native Tool Calls)

In this quickstart, we will be creating a simple agent that can get the top posts on Hacker News.

While large language models can generate text, they need additional tools to perform actions like fetching data from Hacker News. Using Stores, we will add tools for querying the Hacker News API.

Hacker News agent

import json
from litellm import completion
import stores

# Load the Hacker News tool index
index = stores.Index(["silanthro/hackernews"])

# Get the response from the model
response = completion(
    model="gemini/gemini-2.0-flash-001",
    messages=[
        {
            "role": "user",
            "content": "What are the top 10 posts on Hacker News today?",
        }
    ],
    tools=index.format_tools("google-gemini"),  # Format tools for Google Gemini
)

# Execute the tool call
tool_call = response.choices[0].message.tool_calls[0]
result = index.execute(
    tool_call.function.name,
    json.loads(tool_call.function.arguments),
)
print(f"Tool output: {result}")

Agent script walkthrough

1. Load the tools

First, we will load the Hacker News tools from the silanthro/hackernews tool index.

index = stores.Index(["silanthro/hackernews"])

You can also load a tool index from a public GitHub repository or load your own custom tools from your repository. Learn more about what a tool index is here.

The Hacker News API doesn't require any API key. If a tool requires an API key, you can pass it via the env_var parameter.

2. Call the model with the tools

Remember to add your Gemini API key (GEMINI_API_KEY) to your .env file.

To format the tools according to the JSON schema required by the Google Gemini API via LiteLLM, we will use index.format_tools("google-gemini").

response = completion(
    model="gemini/gemini-2.0-flash-001",
    messages=[
        {
            "role": "user",
            "content": "What are the top 10 posts on Hacker News today?",
        }
    ],
    tools=index.format_tools("google-gemini"),
)

3. Parse the model response

We can then parse response.choices[0].message.tool_calls to retrieve the tool name and arguments.

response.choices[0].message.tool_calls[0]
ChatCompletionMessageToolCall(
    index=0,
    function=Function(
        arguments='{"num": 10}',
        name='tools.get_top_stories'
    ),
    id='call_random_uuid',
    type='function'
)

4. Execute the function

Finally, we will use index.execute with the tool name and arguments to run the tool.

tool_call = response.choices[0].message.tool_calls[0]
result = index.execute(
    tool_call.function.name,
    json.loads(tool_call.function.arguments),
)

This gives us the tool call result. You can then supply the result to the model and call the model again to get its final response with the supplied information.

Next steps