How to Extend Ollama with Custom Tools
Large Language Models (LLMs) have revolutionized how we interact with AI, but they face a fundamental limitation: they're confined to generating text based on their training data. What if your AI assistant could check the weather, send emails, query databases, or interact with APIs in real-time? This is where tool integration transforms static language models into dynamic, capable AI agents.
Where Traditional LLMs Hit Their Limits
Traditional LLM interactions follow a simple pattern: you ask a question, the model generates a response based on its training data, and that's it. While this works well for many use cases, it creates several significant limitations:
- Knowledge Cutoff Boundaries: Every LLM has a knowledge cutoff date. Ask about recent events, current stock prices, or today's weather, and you'll get outdated or generic responses. The model simply cannot access real-time information.
- Static Capabilities: LLMs excel at reasoning and text generation, but they cannot perform actions in the real world. They can't send emails, make API calls, execute code, or interact with external systems.
- Context Limitations: While LLMs can process large amounts of text, they cannot dynamically fetch additional context when needed.
- Lack of Real-time Interaction: Modern applications require real-time data access. Whether it's checking inventory levels, validating user credentials, or processing payments, LLMs operating in isolation cannot participate in these dynamic workflows.
These limitations mean that despite their impressive capabilities, standalone LLMs often fall short of being truly useful AI assistants in practical, real-world scenarios.
Turning Language Models into Actionable Agents
The solution lies in extending LLMs with custom tools, external functions that the model can call when needed. This approach, often called "function calling" or "tool use," transforms passive language models into active AI agents capable of interacting with the world around them.
- How Tool Integration Works: Instead of just generating text, tool-enhanced LLMs can analyze your request, determine what external actions are needed, call appropriate tools with the right parameters, process the results, and incorporate that information into their response.
- The Ollama Advantage: Ollama, combined with frameworks like LangChain, provides an excellent platform for building these tool-enhanced AI agents. Running locally, you maintain complete control over your data and tools while benefiting from the flexibility of open-source models.
- Intelligent Decision Making: Tool-enhanced LLMs don't just blindly execute functions. They intelligently decide when and how to use tools based on context.
Getting Started
Let's walk through creating a practical AI agent that can extend Ollama with custom tools. We'll build a system that demonstrates the core concepts while being expandable for your specific needs.
Prerequisites
Before we begin, ensure you have the following installed:
# Install required packages
pip install langchain-ollama langchain-core asyncioYou'll also need Ollama running locally with a model downloaded. For this tutorial, we'll use Qwen, but you can adapt it for any model:
# Pull a capable model
ollama pull qwen2.5:32bSetting Up the Foundation
First, let's create the basic structure for our tool-enhanced chatbot. We'll design it as a class-based system for better organization and maintainability:
import asyncio
from typing import List, Dict, Any
from langchain_core.messages import ToolMessage, HumanMessage, SystemMessage, AIMessage, BaseMessage
from langchain_ollama import ChatOllama
class ToolEnhancedChatBot:
def __init__(self, model: str = "qwen2.5:32b", base_url: str = "localhost:11434"):
self.messages: List[BaseMessage] = []
self.client = ChatOllama(
model=model,
temperature=0,
num_ctx=25000,
base_url=base_url
)
self.tools = {}
self._setup_system_message()
def _setup_system_message(self):
system_message = SystemMessage(content=(
"You are an AI assistant with access to custom tools. "
"When a user requests information or actions that require external data or operations, "
"you should use the appropriate tools. Always follow the tool descriptions carefully."
))
self.messages.append(system_message)Creating Custom Tools
The power of tool-enhanced AI lies in the custom tools you create. Let's implement a practical example that sends an HTTP request:
from langchain.tools import tool
import requests
@tool(parse_docstring=True)
def send_request(
base_url: str,
endpoint: str,
method: str = "GET",
headers: dict = None,
params: dict = None,
data: dict = None,
json_body: dict = None
):
"""
Sends an HTTP request to the specified base URL and endpoint with the given parameters.
Args:
base_url (str): The base URL for the API (e.g., "https://api.example.com")
endpoint (str): The API endpoint path (e.g., "/users")
method (str, optional): HTTP method (GET, POST, PUT, DELETE, etc.). Defaults to "GET".
headers (dict, optional): HTTP headers to include in the request. Defaults to None.
params (dict, optional): URL query parameters. Defaults to None.
data (dict or str, optional): Form data or raw body to send. Defaults to None.
json_body (dict, optional): JSON body to send in the request. Defaults to None.
Returns:
dict: A dictionary containing the status code and response body,
or an error message if the request fails.
"""
base_url = base_url.rstrip('/')
url = f"{base_url}{endpoint}"
try:
response = requests.request(
method=method.upper(),
url=url,
headers=headers,
params=params,
data=data,
json=json_body,
)
response.raise_for_status()
content_type = response.headers.get("Content-Type", "")
return {
"status_code": response.status_code,
"body": response.json() if "application/json" in content_type else response.text
}
except requests.exceptions.RequestException as e:
return {"error": str(e)}Integrating Tools with the Chatbot
Now let's extend our chatbot class to handle tool integration and execution:
class ToolEnhancedChatBot:
def __init__(self, model: str = "qwen2.5:32b", base_url: str = "localhost:11434"):
self.messages: List[BaseMessage] = []
self.client = ChatOllama(
model=model,
temperature=0,
num_ctx=25000,
base_url=base_url,
)
# Register available tools
self.available_tools = {
'send_request': send_request
}
# Bind tools to the LLM
self.llm_with_tools = self.client.bind_tools(list(self.available_tools.values()))
self._setup_system_message()
async def _execute_tool_calls(self, tool_calls: List[dict]) -> None:
"""Execute tool calls and add results to message history."""
for tool_call in tool_calls:
tool_name = tool_call["name"].lower()
selected_tool = self.available_tools.get(tool_name)
if not selected_tool:
print(f"Warning: Tool '{tool_name}' not found")
continue
try:
print(f"Executing tool: {tool_name} with args: {tool_call['args']}")
tool_result = selected_tool.invoke(tool_call["args"])
tool_message = ToolMessage(
content=str(tool_result),
tool_call_id=tool_call["id"]
)
print(f"Tool Result: {tool_message.content}")
self.messages.append(tool_message)
except Exception as e:
print(f"Error executing tool '{tool_name}': {e}")
error_message = ToolMessage(
content=f"Error executing {tool_name}: {str(e)}",
tool_call_id=tool_call["id"]
)
self.messages.append(error_message)
async def _process_response(self, response: AIMessage) -> None:
"""Process LLM response and handle any tool calls."""
self.messages.append(response)
if response.tool_calls:
print("AI wants to use tools...")
await self._execute_tool_calls(response.tool_calls)
# Get final response after tool execution
try:
final_response = self.llm_with_tools.invoke(self.messages)
self.messages.append(final_response)
print(f"\nAI: {final_response.content}")
except Exception as e:
print(f"Error getting final response: {e}")
else:
print(f"\nAI: {response.content}")Expanding Your Tool Arsenal
The real power comes from creating tools specific to your needs. Here are some ideas for additional tools you might implement:
- Database Tools: Query your company's database, update records, generate reports.
- File Operations: Read/write files, process documents, manage cloud storage.
- Communication Tools: Send emails, post to Slack, create calendar events.
Each tool follows the same pattern: define the function with proper type hints and docstrings, decorate it with @tool, register it in your chatbot's tool collection, and let the LLM decide when and how to use it.
Final Thoughts
The beauty of this approach is that it scales naturally. You can start with a few basic tools and gradually build a comprehensive suite of capabilities tailored to your specific use case, creating truly powerful AI agents that bridge the gap between conversational AI and practical business applications.