How to Extend Ollama with Custom Tools
Large Language Models (LLMs) have revolutionized how we interact with AI, but they face a fundamental limitation: they’re confined to generating text based on their training data. What if your AI assistant could check the weather, send emails, query databases, or interact with APIs in real-time? This is where tool integration transforms static language models into dynamic, capable AI agents.
Where Traditional LLMs Hit Their Limits
Traditional LLM interactions follow a simple pattern: you ask a question, the model generates a response based on its training data, and that’s it. While this works well for many use cases, it creates several significant limitations:
- Knowledge Cutoff Boundaries: Every LLM has a knowledge cutoff date. Ask about recent events, current stock prices, or today’s weather, and you’ll get outdated or generic responses. The model simply cannot access real-time information.
- Static Capabilities: LLMs excel at reasoning and text generation, but they cannot perform actions in the real world. They can’t send emails, make API calls, execute code, or interact with external systems. This severely limits their practical utility in business applications.
- Context Limitations: While LLMs can process large amounts of text, they cannot dynamically fetch additional context when needed. If you’re discussing a complex project, the model cannot pull up related documents, check project status in your management system, or access relevant data from your databases.
- Lack of Real-time Interaction: Modern applications require real-time data access. Whether it’s checking inventory levels, validating user credentials, or processing payments, LLMs operating in isolation cannot participate in these dynamic workflows.
These limitations mean that despite their impressive capabilities, standalone LLMs often fall short of being truly useful AI assistants in practical, real-world scenarios.
Turning Language Models into Actionable Agents
The solution lies in extending LLMs with custom tools, external functions that the model can call when needed. This approach, often called “function calling” or “tool use,” transforms passive language models into active AI agents capable of interacting with the world around them.
- How Tool Integration Works: Instead of just generating text, tool-enhanced LLMs can analyze your request, determine what external actions are needed, call appropriate tools with the right parameters, process the results, and incorporate that information into their response. This creates a seamless experience where the AI can access live data, perform calculations, trigger actions, and provide up-to-date, contextually relevant responses.
- Real-world Capabilities: With custom tools, your AI assistant can become incredibly practical. It can check your calendar and schedule meetings, query your company’s database for customer information, send notifications through Slack or email, analyze real-time data and generate reports, and integrate with any API your business uses.
- The Ollama Advantage: Ollama, combined with frameworks like LangChain, provides an excellent platform for building these tool-enhanced AI agents. Running locally, you maintain complete control over your data and tools while benefiting from the flexibility of open-source models. You can customize tools for your specific use cases, ensure data privacy and security, scale according to your needs, and integrate deeply with your existing infrastructure.
- Intelligent Decision Making: Perhaps most importantly, tool-enhanced LLMs don’t just blindly execute functions. They intelligently decide when and how to use tools based on context. The model can determine whether a tool is needed for a given request, select the most appropriate tool from available options, extract the correct parameters from natural language, handle errors and retry logic, and combine results from multiple tools into coherent responses.
This creates AI agents that feel truly intelligent, they understand context, make decisions, take actions, and provide meaningful results, all while maintaining the conversational interface that makes LLMs so accessible.
Getting Started:
Let’s walk through creating a practical AI agent that can extend Ollama with custom tools. We’ll build a system that demonstrates the core concepts while being expandable for your specific needs.
Prerequisites
Before we begin, ensure you have the following installed:
pip install langchain-ollama langchain-core asyncio
You’ll also need Ollama running locally with a model downloaded. For this tutorial, we’ll use Qwen, but you can adapt it for any model:
ollama pull qwen2.5:32b
Setting Up the Foundation
First, let’s create the basic structure for our tool-enhanced chatbot. We’ll design it as a class-based system for better organization and maintainability:
import asyncio
from typing import List, Dict, Any
from langchain_core.messages import ToolMessage, HumanMessage, SystemMessage, AIMessage, BaseMessage
from langchain_ollama import ChatOllama
class ToolEnhancedChatBot:
def __init__(self, model: str = “qwen2.5:32b”, base_url: str = “localhost:11434”):
self.messages: List[BaseMessage] = []
self.client = ChatOllama( model=model, temperature=0, num_ctx=25000, base_url=base_url)
self.tools = {}
self._setup_system_message()
def _setup_system_message(self):
system_message = SystemMessage(content=(
“You are an AI assistant with access to custom tools. “
“When a user requests information or actions that require external data or operations, “
“you should use the appropriate tools. Always follow the tool descriptions carefully.”
))
self.messages.append(system_message)
Creating Custom Tools
The power of tool-enhanced AI lies in the custom tools you create. Let’s implement several practical examples that demonstrate different types of functionality:
from langchain.tools import tool
import requests
@tool(parse_docstring=True)
def send_request(
base_url: str,
endpoint: str,
method: str = “GET”,
headers: dict = None,
params: dict = None,
data: dict = None,
json_body: dict = None
):
“””
Sends an HTTP request to the specified base URL and endpoint with the given parameters.
Args:
base_url (str): The base URL for the API (e.g., “https://api.example.com”)
endpoint (str): The API endpoint path (e.g., “/users”)
method (str, optional): HTTP method (GET, POST, PUT, DELETE, etc.). Defaults to “GET”.
headers (dict, optional): HTTP headers to include in the request. Defaults to None.
params (dict, optional): URL query parameters. Defaults to None.
data (dict or str, optional): Form data or raw body to send in the request. Defaults to None.
json_body (dict, optional): JSON body to send in the request. Defaults to None.
Returns:
dict: A dictionary containing the status code and response body,
or an error message if the request fails.
“””
base_url = base_url.rstrip(‘/’)
url = f”{base_url}{endpoint}”
try:
response = requests.request(
method=method.upper(),
url=url,
headers=headers,
params=params,
data=data,
json=json_body,
# timeout=10
)
response.raise_for_status()
content_type = response.headers.get(“Content-Type”, “”)
return {
“status_code”: response.status_code,
“body”: response.json() if “application/json” in content_type else response.text
}
except requests.exceptions.RequestException as e:
return {“error”: str(e)}
Integrating Tools with the Chatbot
Now let’s extend our chatbot class to handle tool integration and execution:
class ToolEnhancedChatBot:
def __init__(self, model: str = “qwen2.5:32b”, base_url: str = “localhost:11434”):
self.messages: List[BaseMessage] = []
self.client = ChatOllama(
model=model,
temperature=0,
num_ctx=25000,
base_url=base_url,
)
# Register available tools
self.available_tools = {
‘send_request’: send_request
}
# Bind tools to the LLM
self.llm_with_tools = self.client.bind_tools(list(self.available_tools.values()))
self._setup_system_message()
async def _execute_tool_calls(self, tool_calls: List[dict]) -> None:
“””Execute tool calls and add results to message history.”””
for tool_call in tool_calls:
tool_name = tool_call[“name”].lower()
selected_tool = self.available_tools.get(tool_name)
if not selected_tool:
print(f”Warning: Tool ‘{tool_name}’ not found”)
continue
try:
print(f”Executing tool: {tool_name} with args: {tool_call[‘args’]}”)
tool_result = selected_tool.invoke(tool_call[“args”])
tool_message = ToolMessage(
content=str(tool_result),
tool_call_id=tool_call[“id”]
)
print(f”Tool Result: {tool_message.content}”)
self.messages.append(tool_message)
except Exception as e:
print(f”Error executing tool ‘{tool_name}’: {e}”)
error_message = ToolMessage(
content=f”Error executing {tool_name}: {str(e)}”,
tool_call_id=tool_call[“id”]
)
self.messages.append(error_message)
async def _process_response(self, response: AIMessage) -> None:
“””Process LLM response and handle any tool calls.”””
self.messages.append(response)
if response.tool_calls:
print(“AI wants to use tools…”)
await self._execute_tool_calls(response.tool_calls)
# Get final response after tool execution
try:
final_response = self.llm_with_tools.invoke(self.messages)
self.messages.append(final_response)
print(f”\nAI: {final_response.content}”)
except Exception as e:
print(f”Error getting final response: {e}”)
else:
print(f”\nAI: {response.content}”)
async def run(self):
“””Main chat loop.”””
print(“🤖 Tool-Enhanced AI Assistant Ready!”)
print(“Available tools: weather lookup, advanced calculations, time queries”)
print(“Type ‘exit’ or ‘quit’ to stop.\n”)
while True:
try:
user_input = input(“\nYou: “)
if user_input.lower() in [“exit”, “quit”]:
print(“Goodbye! 👋”)
break
self.messages.append(HumanMessage(user_input))
response = self.llm_with_tools.invoke(self.messages)
await self._process_response(response)
except KeyboardInterrupt:
print(“\nGoodbye! 👋”)
break
except Exception as e:
print(f”An error occurred: {e}”)
# Main execution
if __name__ == “__main__”:
async def main():
chatbot = ToolEnhancedChatBot()
await chatbot.run()
try:
asyncio.run(main())
except KeyboardInterrupt:
print(“\nGoodbye! 👋”)
Testing Your Tool-Enhanced Agent
HTTP Requests:
Expanding Your Tool Arsenal
The real power comes from creating tools specific to your needs. Here are some ideas for additional tools you might implement:
- Database Tools: Query your company’s database, update records, generate reports
- File Operations: Read/write files, process documents, manage cloud storage
- Communication Tools: Send emails, post to Slack, create calendar events
Each tool follows the same pattern: define the function with proper type hints and docstrings, decorate it with @tool, register it in your chatbot’s tool collection, and let the LLM decide when and how to use it.
Final Thoughts
The beauty of this approach is that it scales naturally. You can start with a few basic tools and gradually build a comprehensive suite of capabilities tailored to your specific use case, creating truly powerful AI agents that bridge the gap between conversational AI and practical business applications.
Author: Viktor Vörös