mirror of
https://github.com/langbot-app/LangBot.git
synced 2026-09-16 23:07:14 +00:00
fix(ollama): use litellm's ollama_chat provider for native tool-calling (#2494)
The Ollama requester declared litellm_provider: ollama, which routes
every request through litellm's legacy /api/generate-based
OllamaConfig. That config's get_supported_openai_params() does not
include "tools"/"tool_choice" at all, so an Ollama-hosted model in a
local-agent pipeline could never receive a structured tool definition
or return a structured tool_calls response - it could only try to
express a tool call as free text (typically inside its own <think>
reasoning), which LangBot then has no way to execute.
litellm's "ollama_chat" provider targets Ollama's modern /api/chat
endpoint instead, which correctly forwards tools/tool_choice and
correctly surfaces the model's native message.tool_calls field.
Verified against a real local Ollama 0.33.2 instance with the exact
system prompt, RAG-augmented user message, and tool set a live
pipeline sends.
Two follow-on fixes needed because the Ollama requester definition is
shared by LLM and text-embedding models:
- get_reasoning_capabilities: match family in ('ollama', 'ollama_chat')
so the reasoning-level UI still works for this provider.
- scan_models: retry {base_url}/v1/models on a 404 from {base_url}/models,
since Ollama's base_url is a bare host (must not include /v1 - that
would break OllamaChatConfig.get_complete_url, which appends /api/chat
to it directly), unlike most other OpenAI-compatible providers whose
base_url already ends in /v1.
- invoke_embedding: litellm's embedding routing has no "ollama_chat"
case, only "ollama". Build the embedding model name with an explicit
custom_llm_provider="ollama" override when the requester is configured
for ollama_chat, so embedding models (e.g. bge-m3) keep working.
Co-authored-by: zx90316 <zx90316@users.noreply.github.com>
This commit is contained in:
@@ -7,7 +7,7 @@ metadata:
|
||||
zh_Hans: Ollama
|
||||
icon: ollama.svg
|
||||
spec:
|
||||
litellm_provider: ollama
|
||||
litellm_provider: ollama_chat
|
||||
config:
|
||||
- name: base_url
|
||||
label:
|
||||
|
||||
Reference in New Issue
Block a user