mirror of
https://github.com/langbot-app/LangBot.git
synced 2026-09-05 09:07:13 +00:00
CWT
601c6975ea
The Ollama requester declared litellm_provider: ollama, which routes
every request through litellm's legacy /api/generate-based
OllamaConfig. That config's get_supported_openai_params() does not
include "tools"/"tool_choice" at all, so an Ollama-hosted model in a
local-agent pipeline could never receive a structured tool definition
or return a structured tool_calls response - it could only try to
express a tool call as free text (typically inside its own <think>
reasoning), which LangBot then has no way to execute.
litellm's "ollama_chat" provider targets Ollama's modern /api/chat
endpoint instead, which correctly forwards tools/tool_choice and
correctly surfaces the model's native message.tool_calls field.
Verified against a real local Ollama 0.33.2 instance with the exact
system prompt, RAG-augmented user message, and tool set a live
pipeline sends.
Two follow-on fixes needed because the Ollama requester definition is
shared by LLM and text-embedding models:
- get_reasoning_capabilities: match family in ('ollama', 'ollama_chat')
so the reasoning-level UI still works for this provider.
- scan_models: retry {base_url}/v1/models on a 404 from {base_url}/models,
since Ollama's base_url is a bare host (must not include /v1 - that
would break OllamaChatConfig.get_complete_url, which appends /api/chat
to it directly), unlike most other OpenAI-compatible providers whose
base_url already ends in /v1.
- invoke_embedding: litellm's embedding routing has no "ollama_chat"
case, only "ollama". Build the embedding model name with an explicit
custom_llm_provider="ollama" override when the requester is configured
for ollama_chat, so embedding models (e.g. bge-m3) keep working.
Co-authored-by: zx90316 <zx90316@users.noreply.github.com>
35 lines
702 B
YAML
35 lines
702 B
YAML
apiVersion: v1
|
|
kind: LLMAPIRequester
|
|
metadata:
|
|
name: ollama-chat
|
|
label:
|
|
en_US: Ollama
|
|
zh_Hans: Ollama
|
|
icon: ollama.svg
|
|
spec:
|
|
litellm_provider: ollama_chat
|
|
config:
|
|
- name: base_url
|
|
label:
|
|
en_US: Base URL
|
|
zh_Hans: 基础 URL
|
|
type: string
|
|
required: true
|
|
default: http://127.0.0.1:11434
|
|
- name: timeout
|
|
label:
|
|
en_US: Timeout
|
|
zh_Hans: 超时时间
|
|
type: integer
|
|
required: true
|
|
default: 120
|
|
alias: "ollama Ollama 本地 local 本地部署 self-hosted llama gguf 私有化"
|
|
support_type:
|
|
- llm
|
|
- text-embedding
|
|
provider_category: self-hosted
|
|
execution:
|
|
python:
|
|
path: ./ollamachat.py
|
|
attr: OllamaChatCompletions
|