Compare commits

..

3 Commits

Author SHA1 Message Date
fdc310 d40348add3 feat(provider): add pipeline reasoning controls 2026-08-01 01:47:24 +08:00
Hyu e3832ca536 style: format workspace identity modules (#2372)
Co-authored-by: dadachann <185672915+dadachann@users.noreply.github.com>
2026-07-31 23:46:03 +08:00
Hyu 5d9fd15671 feat: use workspace identity for telemetry (#2371)
Co-authored-by: dadachann <185672915+dadachann@users.noreply.github.com>
2026-07-31 23:35:14 +08:00
34 changed files with 2369 additions and 92 deletions
@@ -0,0 +1,476 @@
# 模型思考控制设计方案
> 日期:2026-07-31
> 状态:Phase 1 已审核并实现
> 范围:LangBot 主仓库的模型配置、LiteLLM 请求层、Local Agent、Web 管理面板、监控与测试
## 1. 结论
建议为 LangBot 增加一套与厂商参数解耦的“思考策略”模型,并明确区分三个概念:
1. **思考能力**:模型是否支持思考,以及支持开关、档位还是 token 预算。
2. **思考策略**:一次请求选择厂商默认、关闭、开启或指定思考档位。
3. **思考展示**:是否把模型返回的思考内容展示给最终用户。
现有 `remove-think` 只属于第 3 类。它会过滤输出,但不会阻止模型思考,也不会降低思考 token、费用或延迟。新能力不应复用或改写这个字段。
推荐实现原则:
- 默认值为 `provider_default`,不向上游增加任何新参数,现有模型行为完全不变。
- 用户显式选择的策略必须被准确执行;无法准确执行时返回明确错误,不静默降级。
- LangBot 内部只保存统一策略,Provider 请求层负责翻译成各厂商参数。
- `extra_args` 保留为高级逃生口,但不能成为主 UI 的思考配置方式。
- 模型页只管理并展示能力;可写策略归属于 Local Agent 流水线,同一模型可在不同业务中使用不同思考量。
- 原始 reasoning 数据与展示文本分开保存,保证多轮对话、工具调用和签名字段不丢失。
## 2. 调研结论
### 2.1 可验证资料
本次结论基于以下可验证来源:
- OpenAI 官方 Reasoning Guide`reasoning.effort` 的可选值由模型决定,可包括 `none``minimal``low``medium``high``xhigh``max`;低档位偏向低延迟和低 token,高档位偏向质量。
- https://developers.openai.com/api/docs/guides/reasoning#reasoning-effort
- LangBot 锁定的 LiteLLM `1.88.1` 实现。`uv.lock` 已锁定该版本,本地缓存中的适配代码可以确认 LangBot 实际依赖所支持的翻译行为。
- LangBot 当前实现:模型级 `extra_args` 会在 `LiteLLMRequester._build_completion_args()` 中直接合并到 `acompletion()` 参数。
Anthropic、Google 和 LiteLLM 的官方文档域名在本次环境中被浏览器策略禁止访问,因此下表中这些厂商的结论以 LiteLLM `1.88.1` 实际适配代码为准。实施前应再用对应厂商官方文档做一次参数范围核验,尤其是模型代际和允许值。
### 2.2 厂商差异矩阵
| Provider / 生态 | 可控制能力 | LiteLLM 1.88.1 统一入口 | 关键限制 | 建议支持级别 |
| --- | --- | --- | --- | --- |
| OpenAI | 思考档位,部分模型支持 `none` | `reasoning_effort` | 每个模型支持的档位不同,不能把 `none` 当成通用能力 | 首批完整支持 |
| Anthropic | 旧模型使用 extended thinking + token budget;新模型可用 adaptive thinking + effort | `reasoning_effort``thinking` | `none` 表示不发送 thinking;新旧模型的映射不同 | 首批完整支持 |
| Gemini | 2.x 主要映射为 `thinkingBudget`3.x 主要映射为 `thinkingLevel` | `reasoning_effort``thinking` | Gemini 3 的 `none` 可能只能降到最低档,不能保证真正关闭 | 首批支持,但严格限制关闭语义 |
| DeepSeek | 开启/关闭;当前适配不支持预算档位 | `thinking={type: enabled}`;非 `none` effort 会映射成开启 | 多轮思考模式要求回传 `reasoning_content` | 首批开关支持 |
| xAI | 思考档位 | `reasoning_effort` | 仅 reasoning-capable 模型接受 | 首批完整支持 |
| Ollama | `think` 布尔值;部分模型接受 low/medium/high | `reasoning_effort` | 非 gpt-oss 模型的档位可能退化为布尔开关 | 首批支持,按模型能力裁剪 UI |
| OpenRouter | 聚合多厂商的 reasoning 参数 | `reasoning_effort``thinking` | 实际能力由路由后的模型决定 | 首批支持,能力未知时要求测试 |
| Volcengine / Doubao | `thinking.type` 支持 enabled/disabled/auto | LiteLLM `volcengine` 适配器支持 `thinking` | LangBot 当前 manifest 使用 `openai`,不会进入该适配器 | 第二批,先修正路由并回归 |
| Bailian / Qwen | 厂商兼容接口有独立思考开关/预算 | LiteLLM `dashscope` 适配器目前未提供统一 reasoning 映射 | LangBot 当前 manifest 使用 `openai`,只能通过高级参数透传 | 第二批,实施前核对官方字段 |
| 其他 OpenAI-compatible 网关 | 取决于网关 | 尝试标准 `reasoning_effort` | 不能仅凭模型名推断完整能力 | 保守支持,默认不自动开启 |
### 2.3 对 LangBot 的直接含义
不能把这个功能实现成单一 `enable_thinking: bool`,原因如下:
- 有的模型只有开关,有的模型只有档位,有的模型允许精确 token 预算。
- 有的模型本身始终推理,只能降低思考量,无法真正关闭。
- 同一个通用档位在不同厂商会映射成不同的实际预算。
- 聚合网关和自定义 OpenAI-compatible 服务无法可靠地通过模型名识别能力。
- “不展示思考内容”不等于“关闭思考”。
## 3. 当前项目现状
### 3.1 已有能力
- `LLMModel.extra_args` 是 JSON 字段,Web 端已有通用高级参数编辑器。
- `LiteLLMRequester` 会按“模型级 `extra_args`,再调用级 `extra_args`”的顺序合并参数。
- LiteLLM 已统一处理多个 Provider 的 `reasoning_effort``thinking` 和返回的 `reasoning_content`
- `LocalAgentRunner` 的非流式、流式、工具调用和 fallback 路径都经过 `RuntimeProvider.invoke_llm*()`
- `remove-think` 已能控制 `<think>` 或独立 reasoning 内容是否进入展示文本。
- Gemini 工具调用所需的 `provider_specific_fields` / thought signature 已有保留逻辑和单元测试。
### 3.2 现有缺口
- 管理员只能手写 `extra_args`,没有统一语义、能力提示和校验。
- `remove-think` 名称容易被误解为关闭模型思考。
- 模型扫描只识别 `vision``func_call`,没有 reasoning 能力。
- 当前返回处理会把 `reasoning_content` 拼进 `<think>` 文本后删除原字段,可能损失多轮思考所需的结构化数据。
- DeepSeek 思考模式需要在后续轮次回传 `reasoning_content`,当前链路不能保证完整保留。
- Pipeline 只能选择模型,不能针对业务覆盖模型的思考策略。
- 监控只记录总输入/输出 token,没有单独展示 reasoning token。
- 部分 Provider manifest 仍声明为通用 `openai`,导致 LiteLLM 的厂商专用翻译器不会生效。
### 3.3 预计改动地图
| 层 | 主要文件 | 责任 |
| --- | --- | --- |
| 持久化 | `src/langbot/pkg/entity/persistence/model.py``src/langbot/pkg/persistence/alembic/versions/` | 新增 `reasoning_config` JSON 列和 Alembic 迁移 |
| 模型服务 | `src/langbot/pkg/api/http/service/model.py` | CRUD 校验、冲突检测、测试模型时使用统一策略 |
| HTTP 控制器 | `src/langbot/pkg/api/http/controller/groups/provider/models.py` | 继续复用现有模型路由,不新增平行 API |
| 模型管理 | `src/langbot/pkg/provider/modelmgr/modelmgr.py` | 临时模型、数据库模型与扫描结果加载新字段 |
| 请求抽象 | `src/langbot/pkg/provider/modelmgr/requester.py` | 定义能力查询和 reasoning 参数构建接口 |
| LiteLLM 适配 | `src/langbot/pkg/provider/modelmgr/requesters/litellmchat.py` | 能力识别、策略翻译、参数合并、reasoning 返回保留 |
| Provider manifest | `src/langbot/pkg/provider/modelmgr/requesters/*.yaml` | 必要时修正 Provider 路由;相关变更放到独立阶段 |
| Agent 调用 | `src/langbot/pkg/provider/runners/localagent.py` | 所有非流式、流式、工具调用、fallback 路径传递统一策略 |
| Pipeline 元数据 | `src/langbot/templates/metadata/pipeline/ai.yaml` | 第二阶段加入 Pipeline 级覆盖 |
| 输出配置 | `src/langbot/templates/metadata/pipeline/output.yaml` | 保留键名,澄清 `remove-think` 只控制展示 |
| Web 类型/API | `web/src/app/infra/entities/api/index.ts``web/src/app/infra/http/BackendClient.ts` | 增加配置与能力响应类型 |
| 模型 UI | `web/src/app/home/components/models-dialog/` | 能力标记、策略控件、校验、模型测试 |
| i18n | `web/src/i18n/locales/` | 至少补齐英文、简体中文及项目已有覆盖语言 |
| 测试 | `tests/unit_tests/provider/``web/tests/` | 翻译、服务、流式 round-trip、前端状态测试 |
Phase 1 不修改 `langbot-plugin-sdk` 的公共实体或运行时协议。现有 `provider_message.Message.provider_specific_fields` 已可承载 Provider 原始 reasoning 数据;只有后续要把 reasoning 升级为跨插件公开实体时,才需要跨仓库 SDK 变更。
## 4. 领域模型
### 4.1 统一策略
新增 `ReasoningConfig`,保存于 LLM 模型,Pipeline 可提供同结构覆盖。产品层只暴露一个离散档位:
```json
{
"level": "provider_default"
}
```
字段定义:
| 字段 | 类型 | 含义 |
| --- | --- | --- |
| `level` | `provider_default \| disabled \| enabled \| minimal \| low \| medium \| high \| xhigh \| max` | 同时表达开关和思考强度 |
校验规则:
- `provider_default`:不发送任何 reasoning 参数,保持厂商和模型默认行为。
- `disabled`:明确关闭;仅当模型可真正关闭时允许保存/运行。
- `enabled`:明确开启,但由 Provider 决定具体强度,适用于只有开关的模型。
- `minimal``max`:明确开启,并指定强度;仅允许选择模型实际支持的档位。
- 厂商的 `auto` 统一映射为 `provider_default`,不再增加一个重复状态。
- 精确 token 预算不进入主数据结构。少数需要预算的场景继续通过高级参数配置,并由模型测试接口校验。
### 4.2 能力描述
沿用现有 `LLMModel.abilities`,新增 `reasoning` 能力标记。同时由后端在 API 返回中计算只读的 `reasoning_capabilities`
```json
{
"supported": true,
"controls": ["toggle", "effort"],
"efforts": ["none", "low", "medium", "high"],
"can_disable": true,
"source": "litellm"
}
```
设计约束:
- `abilities` 仍是用户可编辑的粗粒度能力,符合现有 `vision``func_call` 模式。
- `reasoning_capabilities` 不持久化,优先从 LiteLLM 模型元数据计算,避免模型升级后数据库残留过期能力。
- 无法识别的自定义模型返回 `supported: null``source: unknown`,不猜测。
- 用户可手动添加 `reasoning` ability,但未知能力模型必须先通过“测试模型”验证显式策略。
- UI 只展示后端声明可用的控件;未知模型保留 Provider Default 和高级参数入口。
### 4.3 持久化
`llm_models` 表新增 JSON 列:
```text
reasoning_config JSON NOT NULL DEFAULT {"level":"provider_default"}
```
使用 Alembic 新迁移,不修改冻结的 legacy migration。
该列作为已实现版本的兼容字段保留;新的模型页不再提供写入口,Local Agent 请求以流水线中按模型 UUID 保存的策略为准。
不建议把内部策略塞进 `extra_args`,原因是当前 `extra_args` 会原样发送给 LiteLLM;使用保留键会让内部元数据泄漏到上游,并使高级参数与产品配置难以区分。
## 5. 配置优先级与请求流程
### 5.1 优先级
```text
Pipeline 当前候选模型策略
↓ 缺少配置时固定为 provider_default
Provider / 模型默认行为
```
请求参数合并顺序:
```text
基础参数
-> 模型 extra_args
-> 调用级 extra_args
-> 统一 reasoning 策略翻译结果(最后应用)
```
统一策略最后应用,可以确保流水线行为不受模型页历史设置影响。为了避免用户困惑,保存和测试时要检测 `extra_args` 中的冲突字段;当 `level != provider_default` 时,发现以下字段应直接报错:
- `reasoning_effort`
- `thinking`
- `reasoning`
- `extra_body` 内已知的 `thinking``enable_thinking``thinking_budget` 等字段
`level == provider_default` 时继续允许这些高级参数,保证旧配置兼容。
### 5.2 翻译层
`pkg/provider/modelmgr/` 内新增独立的 reasoning 规范化模块,职责是:
1. 读取当前流水线候选模型的请求级策略。
2. 查询 `ProviderAPIRequester.get_reasoning_capabilities(model)`
3. 严格校验策略是否可以准确执行。
4. 生成 LiteLLM 参数,不直接发 HTTP。
5. 返回可观测的“最终生效策略”供日志和测试使用。
建议接口:
```python
class ProviderAPIRequester:
def get_reasoning_capabilities(self, model: RuntimeLLMModel) -> ReasoningCapabilities: ...
def build_reasoning_args(
self,
model: RuntimeLLMModel,
config: ReasoningConfig,
) -> dict[str, Any]: ...
```
LiteLLMRequester 默认优先生成统一参数:
- 强度档位:`reasoning_effort=<level>`
- 仅开启:`thinking={"type":"enabled"}` 或 Provider 等价参数
- 关闭:优先 `reasoning_effort="none"`
- 高级参数中的精确预算:`thinking={"type":"enabled","budget_tokens":N}`
Provider 特例只放在 requester 翻译层,不进入 Pipeline 或平台适配器。
### 5.3 Provider 特例
- **Gemini 3**:如果 LiteLLM 能力表不能确认真正关闭,`disabled` 必须报“不支持关闭,可选择 Provider Default 或最低档”,不能把 `none` 静默映射成 low/minimal。
- **DeepSeek**:所有非 `none` 档位最终都只是开启。能力 API 只返回 `toggle`,UI 不显示档位;多轮必须保存并回传 `reasoning_content`
- **Ollama**:仅对明确支持等级的模型展示 effort;其他模型只展示开关。
- **OpenRouter**:以路由后的模型能力为准。模型未知时允许 Provider Default,显式策略必须通过测试接口。
- **Volcengine**:使用 `thinking.type=enabled/disabled/auto`。应先让该 requester 进入 LiteLLM `volcengine` 适配器,或增加等价的明确翻译,不能依赖模型名。
- **Bailian/Qwen**:作为第二批 Provider 专用翻译。实施前核对官方字段、模型范围、预算上下限和流式返回结构,不凭经验写接口。
## 6. 返回数据与思考展示
### 6.1 保留原始 reasoning
当前 `LiteLLMRequester` 会读取 `reasoning_content`,将其拼接成 `<think>` 文本,再删除原字段。建议改为:
```text
上游 reasoning_content
├─ 原样保存在 Message.provider_specific_fields.reasoning_content
└─ 根据 remove-think 决定是否渲染为 <think>...</think>
```
流式路径需要在 accumulator 中分别累计 `content``reasoning_content`,最终消息必须携带结构化 reasoning。不能只依赖已经渲染的 `<think>` 文本反向解析。
这样可以同时满足:
- `remove-think=true` 时用户看不到思考内容,但多轮协议仍能回传必要数据。
- `remove-think=false` 时保持当前用户体验。
- DeepSeek 多轮 thinking 不丢上下文。
- Gemini thought signature、Anthropic thinking block 等 Provider 字段可以继续按结构化方式 round-trip。
### 6.2 现有字段处理
保留数据库和 Pipeline 配置键 `remove-think`,避免破坏兼容。Web 文案改为更准确的:
- 中文:`向用户展示思考过程`
- 英文:`Show reasoning process`
UI 使用正向开关,保存时转换回 `remove-think = !showReasoning`。文案必须强调它只影响展示,不影响模型是否思考、token 或费用。
## 7. Web 管理面板
### 7.1 模型编辑
模型页只承担能力管理和只读展示:
1. `Reasoning` ability 复选框与 Vision、Function Calling 并列,供无法自动识别的自定义模型手动声明能力。
2. 模型卡片使用简短图标或 badge 标识 reasoning 能力。
3. 模型页不提供可写思考挡位,避免模型默认值与流水线策略形成两个控制源。
### 7.2 Local Agent 流水线策略
在 Local Agent 的主模型和每一个 fallback 模型下分别显示紧凑离散滑杆:
1. `Provider 默认` 始终为首个选项;选择它时不向上游增加任何思考参数。
2. 完整档位顺序为:`Provider 默认 / 关闭 / 开启 / 最低 / 低 / 中 / 高 / 极高 / 最大`
3. 前端只渲染后端为该模型返回的可用档位;仅开关模型显示 `Provider 默认 / 关闭 / 开启`
4. 模型不能真正关闭时不提供 `关闭`;能力未知时只显示不可调的 `Provider 默认`
5. 主模型和 fallback 分别保存策略,切换候选模型时不会把一个模型的挡位错误应用到另一个模型。
6. Dify、Coze、Langflow、n8n 等外部 Runner 不显示该控件,因为 LangBot 不直接发起其内部模型请求。
流水线配置保持旧格式兼容,并在模型选择对象中增加按 UUID 保存的映射:
```json
{
"model": {
"primary": "primary-model-uuid",
"fallbacks": ["fallback-model-uuid"],
"reasoning": {
"primary-model-uuid": "high"
}
}
}
```
`provider_default` 不写入映射;缺少 `reasoning` 的旧流水线天然等价于全部使用 Provider 默认。
滑杆交互要求:轨道使用现有主色和中性灰,不使用渐变;当前档位同时显示文字;支持键盘方向键和正确的 ARIA value text;窄屏下不溢出。
### 7.3 i18n
新增文案至少覆盖 `en_US``zh_Hans``ja_JP` 在模型面板现有同类字段已覆盖时同步补齐。不要把厂商参数名直接作为用户文案。
## 8. API、MCP 与 Skill
### 8.1 HTTP API
模型 CRUD 增加:
- 请求字段:`reasoning_config`
- 响应字段:`reasoning_config`
- 只读字段:`reasoning_capabilities`
模型测试接口必须使用与真实请求完全相同的规范化和翻译逻辑,并在失败时返回可操作错误,例如:
```text
Model gemini-3-... cannot disable reasoning.
Supported controls: effort=[low, medium, high].
```
可选增加只读调试信息,仅在测试接口返回:
```json
{
"effective_reasoning": {
"level": "low",
"translated_keys": ["reasoning_effort"]
}
}
```
不得返回 API key、完整请求正文或原始思考内容。
### 8.2 MCP 与技能
当前 MCP 仅列出模型 Provider,没有完整模型 CRUD 工具。如果本次不新增 agent-accessible HTTP 操作,则无需强行新增 MCP 工具。
如果后续让 Agent 修改模型思考策略,则必须同一提交更新:
- `src/langbot/pkg/api/mcp/server.py`
- 对应的 `skills/` 文档
- 参数 schema 和安全说明
## 9. 监控与可观测性
控制思考量后,管理员需要判断质量、延迟和成本是否值得。建议第二阶段增加:
- `reasoning_tokens`:从 `completion_tokens_details.reasoning_tokens` 或 Provider 等价字段提取。
- `effective_reasoning_level`:记录规范化后的生效档位,不记录原始思考内容。
- 模型监控页展示输入 token、可见输出 token、reasoning token、总延迟。
- Provider 不返回细分 token 时显示未知,不推算。
安全要求:日志、监控、debug API 默认都不得记录 reasoning 原文。思考内容可能包含敏感信息或系统提示,不应因为新增配置而扩大持久化范围。
## 10. 兼容与迁移
### 10.1 数据迁移
- 所有现有 LLM 记录迁移为 `{"level":"provider_default"}`
- 不自动解析或迁移现有 `extra_args` 中的 reasoning 参数,避免误判嵌套结构和 Provider 语义。
- UI 检测到旧 `extra_args` reasoning 字段时显示“由高级参数控制”,统一策略保持 Provider Default。
- 用户主动改成统一策略时,要求先移除冲突高级参数。
### 10.2 运行时兼容
- `provider_default` 不产生任何新增请求参数。
- 不改变现有 `remove-think` 的存储键和默认值。
- 不改变已有 Provider 的 `litellm_provider`,除非该 Provider 在专项回归后单独切换。
- `drop_params` 不能用于掩盖显式 reasoning 配置错误;显式策略被丢弃应视为失败。
- 自托管和 toB 环境中的自定义兼容接口保持可用,未知能力不阻止 Provider Default 请求。
## 11. 实施拆分
### Phase 1:统一基础设施与主流 Provider
- Alembic 增加 `llm_models.reasoning_config`
- Backend 模型实体、CRUD、测试接口支持统一配置。
- LiteLLMRequester 增加能力查询、严格校验和参数翻译。
- 支持 OpenAI、Anthropic、Gemini、DeepSeek、xAI、Ollama、OpenRouter 的已验证 LiteLLM 路径。
- 修复结构化 reasoning 的非流式/流式保留。
- 模型面板增加 reasoning ability 与只读能力标识。
- Local Agent 主模型和每个 fallback 增加独立的请求级策略。
### Phase 2:国内 Provider
- 专项核对并支持 Volcengine/Doubao、Bailian/Qwen。
- 对相关 requester 的 `litellm_provider` 变更做独立回归,避免把 reasoning 功能和通用请求行为回归混在一起。
- 补齐扫描结果中的 reasoning capability。
### Phase 3:监控与评估
- 持久化 reasoning token 和生效策略。
- 监控页增加 reasoning 成本/延迟指标。
- 建立不同 effort 的离线质量、首 token 延迟、总耗时和 token 对比基线。
## 12. 测试方案
### 12.1 单元测试
- `ReasoningConfig` 所有合法/非法组合。
- `provider_default` 不产生任何新增参数。
- 显式配置覆盖模型/调用 `extra_args` 的顺序。
- reasoning 配置与高级参数冲突时拒绝。
- OpenAI 档位原样映射。
- Anthropic 档位映射,以及高级参数预算兼容。
- Gemini 2 budget、Gemini 3 level,以及不支持真正关闭时拒绝。
- DeepSeek 只显示/接受 toggle,非 `none` effort 不伪装成不同档位。
- Ollama 布尔与分级模型差异。
- Volcengine enabled/disabled/auto 翻译。
- 未知 Provider 只允许 Provider Default,或在显式测试后使用标准参数。
- 非流式 `reasoning_content` 保存到 `provider_specific_fields`
- 流式 reasoning 分片累计后仍能 round-trip。
- Gemini thought signature 和工具调用现有测试不能回归。
### 12.2 服务与持久化测试
- 新建、读取、更新模型的 `reasoning_config`
- Alembic 从当前 head 升级后默认值正确。
- 模型测试接口与真实 Local Agent 使用同一翻译函数。
- 旧模型、旧 `extra_args``remove-think` 行为不变。
### 12.3 前端测试
- 能力不同的模型显示正确控件。
- 离散滑杆只能停在后端返回的可用档位。
- 当前档位文字、键盘操作和 ARIA value text 正确。
- 仅开关模型、不可关闭模型、完整档位模型分别显示正确刻度。
- fallback 能力不兼容时阻止保存并给出明确提示。
- 中英文文案完整,移动端 Popover 不溢出。
### 12.4 Provider 冒烟测试
至少选取以下真实或可控 mock
- 一个支持 `none` 的 OpenAI reasoning 模型。
- 一个不支持 `none` 的 reasoning 模型。
- 一个 Anthropic adaptive thinking 模型。
- 一个 Gemini 2.x 与一个 Gemini 3.x 模型。
- 一个 DeepSeek hybrid thinking 模型,执行两轮含工具调用对话。
- 一个 Ollama 本地 reasoning 模型。
- 一个 OpenAI-compatible 自定义网关,验证 Provider Default 完全不变。
每个模型比较 Provider Default、最低档、中档、高档或关闭,记录成功率、首 token 延迟、总耗时、总 token 和 reasoning token(若可用)。
## 13. 风险与控制
| 风险 | 影响 | 控制措施 |
| --- | --- | --- |
| 将“最低思考”误当成“关闭” | 用户以为节省了成本,实际仍在推理 | `can_disable` 严格校验,不静默降级 |
| 模型能力表过期 | 新模型无法配置或旧模型报错 | 能力未知时保守;允许测试;升级 LiteLLM 时回归 |
| 高 effort 导致延迟/费用陡增 | 用户体验和预算风险 | 默认 Provider DefaultUI 提示;后续监控 reasoning token |
| `extra_args` 与统一配置冲突 | 实际生效值不可预测 | 保存/测试时拒绝冲突;统一策略最后应用 |
| reasoning 原文进入日志 | 敏感信息泄露 | 不记录原文,只记录策略和 token |
| 多轮 reasoning 丢失 | 工具调用或后续轮次失败/降质 | 结构化保存并 round-trip;流式专项测试 |
| 修改 Provider 路由造成通用回归 | 非 reasoning 请求也受影响 | 国内 Provider 路由放第二阶段,独立提交和回归 |
## 14. 需要审核确认的决策
1. **是否同意三层分离**:能力、策略、展示互不替代,保留 `remove-think` 仅控制展示。
2. **是否同意严格语义**:显式关闭无法准确执行时直接报错,不自动降为最低思考。
3. **是否同意请求级配置**:流水线按模型 UUID 保存挡位,不把产品配置塞进 `extra_args`
4. **是否同意 Runner 边界**:仅 Local Agent 展示控制项,外部 Runner 由其外部系统管理模型策略。
5. **是否同意保守默认**:所有现有模型迁移为 Provider Default,不自动开启、关闭或迁移旧高级参数。
6. **是否把结构化 reasoning 保留纳入第一阶段**:这是 DeepSeek 多轮和工具调用正确性的必要条件,建议必须纳入。
## 15. 推荐审核结果
建议按以上 6 项全部通过,并将 Phase 1 作为一个完整功能单元实施。不要只增加前端开关或只在 `extra_args` 中写 `reasoning_effort`;那样虽然改动小,但会继续混淆展示与推理、无法处理 Provider 差异,也无法保证多轮对话正确性。
+71 -16
View File
@@ -9,6 +9,7 @@ from ....core import app
from ....entity.persistence import model as persistence_model
from ....entity.persistence import pipeline as persistence_pipeline
from ....provider.modelmgr import requester as model_requester
from ....provider.modelmgr import reasoning as model_reasoning
from ....workspace.errors import WorkspaceNotFoundError
from .secrets import mask_secret_value, redact_secrets, restore_secret_placeholders
from .tenant import TenantContext, require_workspace_uuid, scope_statement
@@ -54,6 +55,53 @@ def _redact_model_secrets(model_data: dict) -> dict:
return redacted
def _normalize_llm_reasoning(model_data: dict) -> None:
model_data['reasoning_config'] = model_reasoning.validate_reasoning_config(
model_data.get('reasoning_config'),
model_data.get('abilities'),
model_data.get('extra_args'),
)
def _validate_llm_reasoning_capability(
model_entity: persistence_model.LLMModel,
runtime_provider: model_requester.RuntimeProvider,
) -> None:
config = model_reasoning.normalize_reasoning_config(model_entity.reasoning_config)
if config['level'] == 'provider_default':
return
runtime_model = model_requester.RuntimeLLMModel(
execution_context=runtime_provider.execution_context,
model_entity=model_entity,
provider=runtime_provider,
)
capabilities = runtime_provider.requester.get_reasoning_capabilities(runtime_model)
model_reasoning.validate_reasoning_capabilities(config, capabilities, model_entity.name)
def _reasoning_capabilities(ap: app.Application, model: persistence_model.LLMModel) -> dict:
model_mgr = getattr(ap, 'model_mgr', None)
runtime_models = getattr(model_mgr, 'llm_model_dict', {}) if model_mgr is not None else {}
for runtime_model in runtime_models.values():
if (
runtime_model.model_entity.uuid == model.uuid
and runtime_model.model_entity.workspace_uuid == model.workspace_uuid
):
return runtime_model.provider.requester.get_reasoning_capabilities(runtime_model)
return model_reasoning.default_reasoning_capabilities(
supported='reasoning' in (model.abilities or []),
source='manual' if 'reasoning' in (model.abilities or []) else 'unknown',
)
def _serialize_llm_model(ap: app.Application, model: persistence_model.LLMModel) -> dict:
model_dict = ap.persistence_mgr.serialize_model(persistence_model.LLMModel, model)
model_dict['reasoning_config'] = model_reasoning.normalize_reasoning_config(model_dict.get('reasoning_config'))
model_dict['reasoning_capabilities'] = _reasoning_capabilities(ap, model)
return model_dict
async def _validate_provider_supports(
ap: app.Application,
context: TenantContext,
@@ -147,7 +195,7 @@ class LLMModelsService:
models_list = []
for model in models:
model_dict = self.ap.persistence_mgr.serialize_model(persistence_model.LLMModel, model)
model_dict = _serialize_llm_model(self.ap, model)
provider = providers.get(model.provider_uuid)
if provider:
provider_dict = self.ap.persistence_mgr.serialize_model(persistence_model.ModelProvider, provider)
@@ -178,7 +226,7 @@ class LLMModelsService:
)
)
models = result.all()
serialized = [self.ap.persistence_mgr.serialize_model(persistence_model.LLMModel, m) for m in models]
serialized = [_serialize_llm_model(self.ap, model) for model in models]
return serialized if include_secret else [_redact_model_secrets(model) for model in serialized]
async def create_llm_model(
@@ -214,13 +262,17 @@ class LLMModelsService:
await _require_workspace_provider(self.ap, context, model_data['provider_uuid'])
await _validate_provider_supports(self.ap, context, model_data['provider_uuid'], 'llm')
_normalize_llm_reasoning(model_data)
runtime_provider = await _require_runtime_provider(self.ap, context, model_data['provider_uuid'])
model_entity = persistence_model.LLMModel(**model_data)
_validate_llm_reasoning_capability(model_entity, runtime_provider)
await self.ap.persistence_mgr.execute_async(sqlalchemy.insert(persistence_model.LLMModel).values(**model_data))
runtime_provider = await _require_runtime_provider(self.ap, context, model_data['provider_uuid'])
runtime_llm_model = await self.ap.model_mgr.load_llm_model_with_provider(
context,
persistence_model.LLMModel(**model_data),
model_entity,
runtime_provider,
)
await self.ap.model_mgr.cache_llm_model(context, runtime_llm_model)
@@ -268,7 +320,7 @@ class LLMModelsService:
if model is None:
return None
model_dict = self.ap.persistence_mgr.serialize_model(persistence_model.LLMModel, model)
model_dict = _serialize_llm_model(self.ap, model)
# Get provider
provider_result = await self.ap.persistence_mgr.execute_async(
@@ -323,6 +375,18 @@ class LLMModelsService:
await _require_workspace_provider(self.ap, context, provider_uuid)
await _validate_provider_supports(self.ap, context, provider_uuid, 'llm')
merged_model_data = {
key: value
for key, value in {**existing_model, **model_data, 'provider_uuid': provider_uuid}.items()
if key not in {'provider', 'created_at', 'updated_at', 'reasoning_capabilities'}
}
_normalize_llm_reasoning(merged_model_data)
model_data['reasoning_config'] = merged_model_data['reasoning_config']
runtime_provider = await _require_runtime_provider(self.ap, context, provider_uuid)
model_entity = persistence_model.LLMModel(**_runtime_model_data(model_uuid, merged_model_data))
_validate_llm_reasoning_capability(model_entity, runtime_provider)
result = await self.ap.persistence_mgr.execute_async(
scope_statement(
sqlalchemy.update(persistence_model.LLMModel)
@@ -336,19 +400,9 @@ class LLMModelsService:
raise WorkspaceNotFoundError('Model not found')
await self.ap.model_mgr.remove_llm_model(context, model_uuid)
runtime_provider = await _require_runtime_provider(self.ap, context, provider_uuid)
runtime_llm_model = await self.ap.model_mgr.load_llm_model_with_provider(
context,
persistence_model.LLMModel(
**_runtime_model_data(
model_uuid,
{
key: value
for key, value in {**existing_model, **model_data, 'provider_uuid': provider_uuid}.items()
if key not in {'provider', 'created_at', 'updated_at'}
},
)
),
model_entity,
runtime_provider,
)
await self.ap.model_mgr.cache_llm_model(context, runtime_llm_model)
@@ -376,6 +430,7 @@ class LLMModelsService:
raise WorkspaceNotFoundError('Model not found')
runtime_llm_model = await self.ap.model_mgr.get_model_by_uuid(context, model_uuid)
else:
_normalize_llm_reasoning(model_data)
runtime_llm_model = await self.ap.model_mgr.init_temporary_runtime_llm_model(context, model_data)
extra_args = model_data.get('extra_args', {})
@@ -48,6 +48,12 @@ class LLMModel(Base):
provider_uuid = sqlalchemy.Column(sqlalchemy.String(255), nullable=False)
abilities = sqlalchemy.Column(sqlalchemy.JSON, nullable=False, default=[])
context_length = sqlalchemy.Column(sqlalchemy.Integer, nullable=True)
reasoning_config = sqlalchemy.Column(
sqlalchemy.JSON,
nullable=False,
default=lambda: {'level': 'provider_default'},
server_default=sqlalchemy.text('\'{"level":"provider_default"}\''),
)
extra_args = sqlalchemy.Column(sqlalchemy.JSON, nullable=False, default={})
prefered_ranking = sqlalchemy.Column(sqlalchemy.Integer, nullable=False, default=0)
created_at = sqlalchemy.Column(sqlalchemy.DateTime, nullable=False, server_default=sqlalchemy.func.now())
@@ -67,10 +67,7 @@ def _suspend_postgres_rls(
states: dict[str, tuple[bool, bool]] = {}
for table_name in table_names:
row = conn.execute(
sa.text(
'SELECT relrowsecurity, relforcerowsecurity '
'FROM pg_class WHERE oid = to_regclass(:table_name)'
),
sa.text('SELECT relrowsecurity, relforcerowsecurity FROM pg_class WHERE oid = to_regclass(:table_name)'),
{'table_name': table_name},
).one()
enabled, forced = bool(row.relrowsecurity), bool(row.relforcerowsecurity)
@@ -141,18 +138,11 @@ def upgrade() -> None:
if table_name == 'workspaces':
continue
table = sa.Table(table_name, metadata, autoload_with=conn, extend_existing=True)
conn.execute(
table.update()
.where(table.c.workspace_uuid == old_uuid)
.values(workspace_uuid=canonical_uuid)
)
conn.execute(table.update().where(table.c.workspace_uuid == old_uuid).values(workspace_uuid=canonical_uuid))
if 'metadata' in table_names:
conn.execute(
sa.text(
'UPDATE metadata SET value = :canonical_uuid '
'WHERE key = :key AND value = :old_uuid'
),
sa.text('UPDATE metadata SET value = :canonical_uuid WHERE key = :key AND value = :old_uuid'),
{
'canonical_uuid': canonical_uuid,
'key': _OSS_WORKSPACE_METADATA_KEY,
@@ -0,0 +1,57 @@
"""add llm reasoning config
Revision ID: 0018_llm_reasoning_config
Revises: 0017_oss_workspace_identity
Create Date: 2026-07-27
"""
from __future__ import annotations
import sqlalchemy as sa
from alembic import op
revision = '0018_llm_reasoning_config'
down_revision = '0017_oss_workspace_identity'
branch_labels = None
depends_on = None
_LLM_MODELS = sa.table(
'llm_models',
sa.column('reasoning_config', sa.JSON()),
)
def upgrade() -> None:
conn = op.get_bind()
inspector = sa.inspect(conn)
if 'llm_models' not in inspector.get_table_names():
return
columns = {column['name'] for column in inspector.get_columns('llm_models')}
if 'reasoning_config' in columns:
return
op.add_column(
'llm_models',
sa.Column(
'reasoning_config',
sa.JSON(),
nullable=True,
server_default=sa.text('\'{"level":"provider_default"}\''),
),
)
conn.execute(_LLM_MODELS.update().values(reasoning_config={'level': 'provider_default'}))
with op.batch_alter_table('llm_models') as batch_op:
batch_op.alter_column('reasoning_config', existing_type=sa.JSON(), nullable=False)
def downgrade() -> None:
conn = op.get_bind()
inspector = sa.inspect(conn)
if 'llm_models' not in inspector.get_table_names():
return
columns = {column['name'] for column in inspector.get_columns('llm_models')}
if 'reasoning_config' in columns:
with op.batch_alter_table('llm_models') as batch_op:
batch_op.drop_column('reasoning_config')
@@ -649,6 +649,7 @@ class ModelManager:
provider_uuid=runtime_provider.provider_entity.uuid,
abilities=model_info.get('abilities', []),
context_length=model_info.get('context_length'),
reasoning_config=model_info.get('reasoning_config', {'level': 'provider_default'}),
extra_args=model_info.get('extra_args', {}),
)
return self._build_llm_model(execution_context, model_entity, runtime_provider)
@@ -0,0 +1,110 @@
from __future__ import annotations
import typing
ReasoningLevel = typing.Literal[
'provider_default',
'disabled',
'enabled',
'minimal',
'low',
'medium',
'high',
'xhigh',
'max',
]
REASONING_LEVELS: tuple[str, ...] = (
'provider_default',
'disabled',
'enabled',
'minimal',
'low',
'medium',
'high',
'xhigh',
'max',
)
DEFAULT_REASONING_CONFIG: dict[str, str] = {'level': 'provider_default'}
_CONFLICTING_TOP_LEVEL_ARGS = {'reasoning_effort', 'thinking', 'reasoning'}
_CONFLICTING_EXTRA_BODY_ARGS = {'thinking', 'enable_thinking', 'thinking_budget', 'reasoning'}
def normalize_reasoning_config(value: typing.Any) -> dict[str, str]:
"""Return the canonical model reasoning configuration."""
if value is None:
return dict(DEFAULT_REASONING_CONFIG)
if not isinstance(value, dict):
raise ValueError('reasoning_config must be an object')
unknown_fields = set(value) - {'level'}
if unknown_fields:
raise ValueError(f'Unsupported reasoning_config fields: {", ".join(sorted(unknown_fields))}')
level = value.get('level', 'provider_default')
if level not in REASONING_LEVELS:
raise ValueError(f'Unsupported reasoning level: {level}')
return {'level': typing.cast(str, level)}
def validate_reasoning_config(
value: typing.Any,
abilities: typing.Iterable[str] | None,
extra_args: typing.Any,
) -> dict[str, str]:
"""Validate a model-facing reasoning config and conflicting raw arguments."""
config = normalize_reasoning_config(value)
if config['level'] == 'provider_default':
return config
if 'reasoning' not in set(abilities or []):
raise ValueError('The reasoning ability must be enabled before selecting a reasoning level')
conflicts = find_reasoning_arg_conflicts(extra_args)
if conflicts:
raise ValueError('reasoning_config conflicts with advanced parameters: ' + ', '.join(conflicts))
return config
def find_reasoning_arg_conflicts(extra_args: typing.Any) -> list[str]:
if not isinstance(extra_args, dict):
return []
conflicts = [key for key in sorted(_CONFLICTING_TOP_LEVEL_ARGS) if key in extra_args]
extra_body = extra_args.get('extra_body')
if isinstance(extra_body, dict):
conflicts.extend(f'extra_body.{key}' for key in sorted(_CONFLICTING_EXTRA_BODY_ARGS) if key in extra_body)
return conflicts
def validate_reasoning_capabilities(
config: typing.Any,
capabilities: typing.Mapping[str, typing.Any],
model_name: str,
) -> None:
"""Ensure an explicit reasoning level can be honored by the requester."""
level = normalize_reasoning_config(config)['level']
if level == 'provider_default':
return
available_levels = capabilities.get('levels')
if not isinstance(available_levels, list):
available_levels = []
if capabilities.get('supported') is not True or level not in available_levels:
available_text = ', '.join(str(item) for item in available_levels) or 'provider_default'
raise ValueError(
f'Reasoning level "{level}" is not supported by model {model_name}. Available levels: {available_text}'
)
def default_reasoning_capabilities(
supported: bool = False,
source: str = 'unknown',
) -> dict[str, typing.Any]:
return {
'supported': supported,
'levels': ['provider_default'],
'source': source,
}
@@ -10,6 +10,7 @@ from ...entity.persistence import model as persistence_model
from ...workspace.errors import WorkspaceInvariantError
import langbot_plugin.api.entities.builtin.resource.tool as resource_tool
from . import token
from . import reasoning
import langbot_plugin.api.entities.builtin.pipeline.query as pipeline_query
import langbot_plugin.api.entities.builtin.provider.message as provider_message
@@ -377,11 +378,15 @@ class RuntimeLLMModel:
provider: RuntimeProvider
"""提供商实例"""
reasoning_config_override: dict[str, str] | None
"""Request-scoped reasoning policy supplied by the active pipeline."""
def __init__(
self,
execution_context: ExecutionContext,
model_entity: persistence_model.LLMModel,
provider: RuntimeProvider,
reasoning_config_override: dict[str, str] | None = None,
):
_ensure_same_execution_scope(provider.execution_context, execution_context, resource='LLM model')
if model_entity.workspace_uuid != execution_context.workspace_uuid:
@@ -391,6 +396,7 @@ class RuntimeLLMModel:
self.execution_context = execution_context
self.model_entity = model_entity
self.provider = provider
self.reasoning_config_override = reasoning_config_override
class RuntimeEmbeddingModel:
@@ -482,6 +488,13 @@ class ProviderAPIRequester(metaclass=abc.ABCMeta):
"""
raise NotImplementedError('This provider does not support model scanning')
def get_reasoning_capabilities(self, model: RuntimeLLMModel) -> dict[str, typing.Any]:
"""Return normalized reasoning controls supported by a model."""
return reasoning.default_reasoning_capabilities(
supported='reasoning' in (model.model_entity.abilities or []),
source='manual' if 'reasoning' in (model.model_entity.abilities or []) else 'unknown',
)
@abc.abstractmethod
async def invoke_llm(
self,
@@ -7,7 +7,7 @@ import typing
import litellm
from litellm import acompletion, aembedding, arerank
from .. import errors, requester
from .. import errors, reasoning, requester
from ....utils import httpclient
import langbot_plugin.api.entities.builtin.resource.tool as resource_tool
import langbot_plugin.api.entities.builtin.pipeline.query as pipeline_query
@@ -164,6 +164,19 @@ class LiteLLMRequester(requester.ProviderAPIRequester):
_EMBEDDING_MODEL_HINTS = ('embedding', 'embed', 'bge-', 'e5-', 'm3e', 'gte-', 'text-embedding')
_RERANK_MODEL_HINTS = ('rerank', 're-rank', 're_rank')
_INFERRED_EFFORT_PROVIDERS = frozenset(
{
'anthropic',
'gemini',
'groq',
'mistral',
'openai',
'openrouter',
'together_ai',
'xai',
}
)
_INFERRED_TOGGLE_PROVIDERS = frozenset({'deepseek', 'ollama', 'volcengine'})
default_config: dict[str, typing.Any] = {
'base_url': '',
@@ -201,7 +214,10 @@ class LiteLLMRequester(requester.ProviderAPIRequester):
return False
provider = self._get_custom_llm_provider()
candidates: list[tuple[str, str | None]] = [(model_name, provider)]
candidates: list[tuple[str, str | None]] = [
(candidate, None) for candidate in self._metadata_model_candidates(model_name)
]
candidates.append((model_name, provider))
litellm_model_name = self._build_litellm_model_name(model_name)
if litellm_model_name != model_name:
candidates.append((litellm_model_name, None))
@@ -268,6 +284,14 @@ class LiteLLMRequester(requester.ProviderAPIRequester):
deduped_candidates.append(candidate)
return deduped_candidates
@staticmethod
def _metadata_model_candidates(model_name: str) -> list[str]:
"""Return known equivalent model IDs used only for LiteLLM metadata lookup."""
normalized_model_name = (model_name or '').lower()
if normalized_model_name.startswith('mimo-v2.5'):
return [f'openrouter/xiaomi/{normalized_model_name}']
return []
def _known_context_length_fallback(self, model_name: str) -> int | None:
normalized_model_name = (model_name or '').lower()
if normalized_model_name.startswith('deepseek-v4-'):
@@ -287,7 +311,8 @@ class LiteLLMRequester(requester.ProviderAPIRequester):
if not callable(helper):
return self._known_context_length_fallback(model_name)
candidates = [model_name]
candidates = self._metadata_model_candidates(model_name)
candidates.append(model_name)
litellm_model_name = self._build_litellm_model_name(model_name)
if litellm_model_name != model_name:
candidates.append(litellm_model_name)
@@ -314,6 +339,142 @@ class LiteLLMRequester(requester.ProviderAPIRequester):
def _supports_vision(self, model_name: str) -> bool:
return self._safe_litellm_bool_helper('supports_vision', model_name)
def _supports_reasoning(self, model_name: str) -> bool:
return self._safe_litellm_bool_helper('supports_reasoning', model_name)
def _reasoning_provider(self, model_name: str) -> str:
provider = (self._get_custom_llm_provider() or '').lower()
if provider:
return provider
normalized_name = (model_name or '').lower()
if '/' in normalized_name:
prefix = normalized_name.split('/', 1)[0]
if prefix in {
'anthropic',
'deepseek',
'gemini',
'groq',
'mistral',
'ollama',
'openai',
'openrouter',
'together_ai',
'volcengine',
'xai',
}:
return prefix
candidates = self._metadata_provider_candidates(model_name)
return candidates[0] if candidates else ''
def _safe_model_info(self, model_name: str) -> dict[str, typing.Any]:
helper = getattr(litellm, 'get_model_info', None)
if not callable(helper):
return {}
candidates = [
*self._metadata_model_candidates(model_name),
model_name,
self._build_litellm_model_name(model_name),
]
for candidate in candidates:
try:
info = helper(candidate)
except Exception:
continue
if isinstance(info, dict):
return info
model_dump = getattr(info, 'model_dump', None)
if callable(model_dump):
try:
dumped = model_dump()
if isinstance(dumped, dict):
return dumped
except Exception:
continue
return {}
def get_reasoning_capabilities(self, model: requester.RuntimeLLMModel) -> dict[str, typing.Any]:
model_name = model.model_entity.name
abilities = model.model_entity.abilities or []
detected = self._supports_reasoning(model_name)
declared = 'reasoning' in abilities
provider = self._reasoning_provider(model_name)
inferred = provider in self._INFERRED_EFFORT_PROVIDERS | self._INFERRED_TOGGLE_PROVIDERS
supported = detected or declared or inferred
if not supported:
return reasoning.default_reasoning_capabilities()
normalized_name = model_name.lower()
levels = ['provider_default']
if provider == 'deepseek':
if 'reasoner' not in normalized_name and '-r1' not in normalized_name:
levels.append('disabled')
levels.append('enabled')
elif provider == 'volcengine':
levels.extend(['disabled', 'enabled'])
elif provider == 'ollama':
levels.append('disabled')
if normalized_name.startswith('gpt-oss') or '/gpt-oss' in normalized_name:
levels.extend(['low', 'medium', 'high'])
else:
levels.append('enabled')
elif not detected:
levels.extend(['low', 'medium', 'high'])
else:
model_info = self._safe_model_info(model_name)
supports_none = model_info.get('supports_none_reasoning_effort') is True
if provider == 'anthropic':
supports_none = True
if provider == 'gemini' and 'gemini-3' in normalized_name:
supports_none = False
if supports_none:
levels.append('disabled')
for level in ('minimal', 'low', 'medium', 'high'):
flag = model_info.get(f'supports_{level}_reasoning_effort')
if flag is not False:
levels.append(level)
for level in ('xhigh', 'max'):
if model_info.get(f'supports_{level}_reasoning_effort') is True:
levels.append(level)
return {
'supported': True,
'levels': list(dict.fromkeys(levels)),
'source': 'litellm' if detected else ('provider' if inferred else 'manual'),
}
def _build_reasoning_args(self, model: requester.RuntimeLLMModel) -> dict[str, typing.Any]:
raw_config = getattr(model, 'reasoning_config_override', None)
if raw_config is None:
raw_config = getattr(model.model_entity, 'reasoning_config', None)
if not isinstance(raw_config, dict):
raw_config = None
config = reasoning.normalize_reasoning_config(raw_config)
level = config['level']
if level == 'provider_default':
return {}
capabilities = self.get_reasoning_capabilities(model)
try:
reasoning.validate_reasoning_capabilities(config, capabilities, model.model_entity.name)
except ValueError as exc:
raise errors.RequesterError(str(exc)) from exc
provider = self._reasoning_provider(model.model_entity.name)
if level == 'disabled':
if provider == 'volcengine':
return {'thinking': {'type': 'disabled'}}
return {'reasoning_effort': 'none'}
if level == 'enabled':
if provider in {'deepseek', 'volcengine'}:
return {'thinking': {'type': 'enabled'}}
return {'reasoning_effort': 'low'}
return {'reasoning_effort': level}
def _infer_model_type(self, model_id: str) -> str:
normalized_id = (model_id or '').lower()
if any(kw in normalized_id for kw in self._RERANK_MODEL_HINTS):
@@ -344,6 +505,11 @@ class LiteLLMRequester(requester.ProviderAPIRequester):
)
if supports_provider_reported_vision or self._supports_vision(model_id):
abilities.append('vision')
supports_provider_reported_reasoning = bool(
model_payload and model_payload.get('supports_reasoning') is True
)
if supports_provider_reported_reasoning or self._supports_reasoning(model_id):
abilities.append('reasoning')
scanned_model['abilities'] = abilities
context_length = self._context_length_from_scan_payload(model_payload)
@@ -670,6 +836,21 @@ class LiteLLMRequester(requester.ProviderAPIRequester):
args.update(model.model_entity.extra_args)
args.update(extra_args)
reasoning_args = self._build_reasoning_args(model)
if reasoning_args:
conflicts = reasoning.find_reasoning_arg_conflicts(model.model_entity.extra_args)
conflicts.extend(reasoning.find_reasoning_arg_conflicts(extra_args))
if conflicts:
raise errors.RequesterError(
'reasoning_config conflicts with advanced parameters: ' + ', '.join(dict.fromkeys(conflicts))
)
args.update(reasoning_args)
if 'reasoning_effort' in reasoning_args and self._reasoning_provider(model.model_entity.name) == 'openai':
allowed_openai_params = args.get('allowed_openai_params') or []
if not isinstance(allowed_openai_params, (list, tuple, set)):
raise errors.RequesterError('allowed_openai_params must be an array')
args['allowed_openai_params'] = list(dict.fromkeys([*allowed_openai_params, 'reasoning_effort']))
if funcs:
tools = await self.ap.tool_mgr.generate_tools_for_openai(funcs)
if tools:
@@ -699,6 +880,10 @@ class LiteLLMRequester(requester.ProviderAPIRequester):
content = message_data.get('content', '')
reasoning_content = message_data.get('reasoning_content', None)
if reasoning_content:
provider_fields = dict(message_data.get('provider_specific_fields') or {})
provider_fields['reasoning_content'] = reasoning_content
message_data['provider_specific_fields'] = provider_fields
message_data['content'] = self._process_thinking_content(content, reasoning_content, remove_think)
if 'reasoning_content' in message_data:
@@ -760,13 +945,13 @@ class LiteLLMRequester(requester.ProviderAPIRequester):
delta_content = delta.get('content', '')
reasoning_content = delta.get('reasoning_content', '')
provider_fields = dict(delta.get('provider_specific_fields') or {})
# Handle reasoning_content based on remove_think flag
if reasoning_content:
provider_fields['reasoning_content'] = reasoning_content
if remove_think:
# Skip reasoning content when remove_think is True
chunk_idx += 1
continue
delta_content = None
else:
# Use reasoning_content as the displayed content
delta_content = reasoning_content
@@ -779,7 +964,7 @@ class LiteLLMRequester(requester.ProviderAPIRequester):
tool_calls = self._normalize_stream_tool_calls(delta.get('tool_calls'), tool_call_state)
if chunk_idx == 0 and not delta_content and not tool_calls:
if chunk_idx == 0 and not delta_content and not tool_calls and not provider_fields:
chunk_idx += 1
continue
@@ -791,8 +976,8 @@ class LiteLLMRequester(requester.ProviderAPIRequester):
}
# Preserve provider_specific_fields from delta (e.g., Gemini thought_signatures)
if delta.get('provider_specific_fields'):
chunk_data['provider_specific_fields'] = delta['provider_specific_fields']
if provider_fields:
chunk_data['provider_specific_fields'] = provider_fields
chunk_data = {k: v for k, v in chunk_data.items() if v is not None}
yield provider_message.MessageChunk(**chunk_data)
+37 -2
View File
@@ -6,6 +6,7 @@ import typing
from .. import runner
from ...telemetry import features as telemetry_features
from ..modelmgr import requester as modelmgr_requester
from ..modelmgr import reasoning as modelmgr_reasoning
from ..tools.loaders.native import EXEC_TOOL_NAME
import langbot_plugin.api.entities.builtin.pipeline.query as pipeline_query
import langbot_plugin.api.entities.builtin.provider.message as provider_message
@@ -60,6 +61,7 @@ class _StreamAccumulator:
self.msg_idx = 0
self.accumulated_content = initial_content or ''
self.last_role = 'assistant'
self.provider_specific_fields: dict[str, typing.Any] = {}
self.msg_sequence = msg_sequence
self.remove_think = remove_think
self._think_state = None
@@ -94,6 +96,14 @@ class _StreamAccumulator:
if tool_call.function and tool_call.function.arguments:
self.tool_calls_map[tool_call.id].function.arguments += tool_call.function.arguments
if msg.provider_specific_fields:
for key, value in msg.provider_specific_fields.items():
if key == 'reasoning_content' and isinstance(value, str):
previous = self.provider_specific_fields.get(key, '')
self.provider_specific_fields[key] = f'{previous}{value}'
else:
self.provider_specific_fields[key] = value
if msg.is_final:
self._flush_think_state()
@@ -103,6 +113,7 @@ class _StreamAccumulator:
role=self.last_role,
content=self._maybe_strip_think(self.accumulated_content),
tool_calls=list(self.tool_calls_map.values()) if (self.tool_calls_map and msg.is_final) else None,
provider_specific_fields=(self.provider_specific_fields or None) if msg.is_final else None,
is_final=msg.is_final,
msg_sequence=self.msg_sequence,
)
@@ -115,6 +126,7 @@ class _StreamAccumulator:
role=self.last_role,
content=self._maybe_strip_think(self.accumulated_content),
tool_calls=list(self.tool_calls_map.values()) if self.tool_calls_map else None,
provider_specific_fields=self.provider_specific_fields or None,
msg_sequence=self.msg_sequence,
)
@@ -233,9 +245,10 @@ class LocalAgentRunner(runner.RequestRunner):
execution_context,
query.use_llm_model_uuid,
)
candidates.append(primary)
except ValueError:
self.ap.logger.warning(f'Primary model {query.use_llm_model_uuid} not found')
else:
candidates.append(LocalAgentRunner._apply_pipeline_reasoning_config(query, primary))
# Fallback models
fallback_uuids = (query.variables or {}).get('_fallback_model_uuids', [])
@@ -245,12 +258,34 @@ class LocalAgentRunner(runner.RequestRunner):
execution_context,
fb_uuid,
)
candidates.append(fb_model)
except ValueError:
self.ap.logger.warning(f'Fallback model {fb_uuid} not found, skipping')
else:
candidates.append(LocalAgentRunner._apply_pipeline_reasoning_config(query, fb_model))
return candidates
@staticmethod
def _apply_pipeline_reasoning_config(
query: pipeline_query.Query,
model: modelmgr_requester.RuntimeLLMModel,
) -> modelmgr_requester.RuntimeLLMModel:
local_agent_config = query.pipeline_config.get('ai', {}).get('local-agent', {})
model_config = local_agent_config.get('model', {})
reasoning_by_model = model_config.get('reasoning', {}) if isinstance(model_config, dict) else {}
level = (
reasoning_by_model.get(model.model_entity.uuid, 'provider_default')
if isinstance(reasoning_by_model, dict)
else 'provider_default'
)
reasoning_config = modelmgr_reasoning.normalize_reasoning_config({'level': level})
return modelmgr_requester.RuntimeLLMModel(
execution_context=model.execution_context,
model_entity=model.model_entity,
provider=model.provider,
reasoning_config_override=reasoning_config,
)
async def _invoke_with_fallback(
self,
query: pipeline_query.Query,
+2 -2
View File
@@ -12,5 +12,5 @@ def workspace_identity(execution_context: WorkspaceExecutionContext) -> dict[str
"""Build the canonical telemetry identity for one Workspace execution."""
workspace_uuid = execution_context.workspace_uuid.strip()
if not workspace_uuid:
raise ValueError("Telemetry execution Workspace UUID is empty")
return {"workspace_uuid": workspace_uuid}
raise ValueError('Telemetry execution Workspace UUID is empty')
return {'workspace_uuid': workspace_uuid}
+3 -3
View File
@@ -3,15 +3,15 @@ from __future__ import annotations
import uuid
_INSTANCE_PREFIX = "instance_"
_WORKSPACE_IDENTITY_NAMESPACE = uuid.UUID("8ea04f29-8528-4cc3-bb28-30a838c89d76")
_INSTANCE_PREFIX = 'instance_'
_WORKSPACE_IDENTITY_NAMESPACE = uuid.UUID('8ea04f29-8528-4cc3-bb28-30a838c89d76')
def workspace_uuid_from_instance_id(instance_id: str) -> str:
"""Return the stable OSS Workspace UUID for a persisted instance identity."""
value = instance_id.strip()
if not value:
raise ValueError("LangBot instance identity is empty")
raise ValueError('LangBot instance identity is empty')
candidate = value[len(_INSTANCE_PREFIX) :] if value.startswith(_INSTANCE_PREFIX) else value
try:
@@ -92,6 +92,7 @@ stages:
default:
primary: ''
fallbacks: []
reasoning: {}
- name: max-round
label:
en_US: Max Round
@@ -9,8 +9,11 @@ Run: uv run pytest tests/integration/persistence/test_migrations.py -q
from __future__ import annotations
import json
import pytest
import sqlalchemy
from sqlalchemy import text
from sqlalchemy.ext.asyncio import create_async_engine
from langbot.pkg.entity.persistence.base import Base
@@ -190,6 +193,66 @@ class TestSQLiteMigrationUpgrade:
await run_alembic_upgrade(sqlite_engine, 'head')
assert await get_alembic_current(sqlite_engine) == _get_script_head()
@pytest.mark.asyncio
async def test_reasoning_config_migrates_existing_models(self, sqlite_engine):
"""Upgrade from 0017 backfills reasoning config and keeps a database default."""
async with sqlite_engine.begin() as conn:
await conn.execute(
text(
"""
CREATE TABLE llm_models (
uuid VARCHAR(255) PRIMARY KEY,
name VARCHAR(255) NOT NULL,
provider_uuid VARCHAR(255) NOT NULL,
abilities JSON NOT NULL,
context_length INTEGER,
extra_args JSON NOT NULL,
prefered_ranking INTEGER NOT NULL DEFAULT 0
)
"""
)
)
await conn.execute(
text(
"""
INSERT INTO llm_models (
uuid, name, provider_uuid, abilities, extra_args, prefered_ranking
) VALUES (
'existing-model', 'Existing Model', 'provider', '[]', '{}', 0
)
"""
)
)
await run_alembic_stamp(sqlite_engine, '0017_oss_workspace_identity')
await run_alembic_upgrade(sqlite_engine, 'head')
async with sqlite_engine.begin() as conn:
columns = await conn.run_sync(lambda sync_conn: sqlalchemy.inspect(sync_conn).get_columns('llm_models'))
reasoning_column = next(column for column in columns if column['name'] == 'reasoning_config')
assert reasoning_column['nullable'] is False
existing_value = (
await conn.execute(text("SELECT reasoning_config FROM llm_models WHERE uuid = 'existing-model'"))
).scalar_one()
assert json.loads(existing_value) == {'level': 'provider_default'}
await conn.execute(
text(
"""
INSERT INTO llm_models (
uuid, name, provider_uuid, abilities, extra_args, prefered_ranking
) VALUES (
'new-model', 'New Model', 'provider', '[]', '{}', 0
)
"""
)
)
new_value = (
await conn.execute(text("SELECT reasoning_config FROM llm_models WHERE uuid = 'new-model'"))
).scalar_one()
assert json.loads(new_value) == {'level': 'provider_default'}
class TestSQLiteMigrationFreshDatabase:
"""Tests for fresh database workflow."""
@@ -17,12 +17,14 @@ import pytest
from unittest.mock import AsyncMock, Mock
from types import SimpleNamespace
from langbot.pkg.api.http.context import ExecutionContext
from langbot.pkg.api.http.service.model import (
LLMModelsService,
EmbeddingModelsService,
RerankModelsService,
_parse_provider_api_keys,
_runtime_model_data,
_serialize_llm_model,
_validate_provider_supports,
)
from langbot.pkg.api.http.service import model as model_service_module
@@ -64,15 +66,19 @@ def _create_mock_llm_model(
abilities: list = None,
context_length: int | None = None,
extra_args: dict = None,
reasoning_config: dict = None,
) -> Mock:
"""Helper to create mock LLMModel entity."""
model = Mock(spec=LLMModel)
model.workspace_uuid = WORKSPACE_UUID
model.uuid = model_uuid
model.name = name
model.provider_uuid = provider_uuid
model.abilities = abilities or []
model.context_length = context_length
model.extra_args = extra_args or {}
model.reasoning_config = reasoning_config or {'level': 'provider_default'}
model.prefered_ranking = 0
return model
@@ -156,6 +162,26 @@ def _create_runtime_model_mgr() -> SimpleNamespace:
return manager
def _create_reasoning_runtime_provider(capabilities: dict) -> SimpleNamespace:
execution_context = ExecutionContext(
instance_uuid='instance-test',
workspace_uuid=WORKSPACE_UUID,
placement_generation=1,
)
return SimpleNamespace(
execution_context=execution_context,
provider_entity=ModelProvider(
workspace_uuid=WORKSPACE_UUID,
uuid='provider-uuid',
name='Reasoning Provider',
requester='openai',
base_url='https://api.openai.com',
api_keys=[],
),
requester=SimpleNamespace(get_reasoning_capabilities=Mock(return_value=capabilities)),
)
class TestParseProviderApiKeys:
"""Tests for _parse_provider_api_keys helper function."""
@@ -209,6 +235,42 @@ class TestRuntimeModelData:
assert result['extra_args'] == {'temp': 0.7}
class TestSerializeLLMModel:
def test_includes_runtime_reasoning_capabilities(self):
model = _create_mock_llm_model(
abilities=['reasoning'],
reasoning_config={'level': 'high'},
)
capabilities = {
'supported': True,
'levels': ['provider_default', 'low', 'high'],
'source': 'litellm',
}
runtime_model = SimpleNamespace(
model_entity=model,
provider=SimpleNamespace(
requester=SimpleNamespace(get_reasoning_capabilities=Mock(return_value=capabilities))
),
)
ap = SimpleNamespace(
persistence_mgr=SimpleNamespace(
serialize_model=Mock(
return_value={
'uuid': model.uuid,
'name': model.name,
'reasoning_config': {'level': 'high'},
}
)
),
model_mgr=SimpleNamespace(llm_model_dict={('workspace', model.uuid): runtime_model}),
)
serialized = _serialize_llm_model(ap, model)
assert serialized['reasoning_config'] == {'level': 'high'}
assert serialized['reasoning_capabilities'] == capabilities
class TestLLMModelsServiceGetLLMModels:
"""Tests for LLMModelsService.get_llm_models method."""
@@ -580,6 +642,66 @@ class TestLLMModelsServiceCreateLLMModel:
ap.provider_service.find_or_create_provider.assert_called_once()
assert result_uuid is not None
async def test_create_llm_model_validates_explicit_reasoning_level(self):
ap = SimpleNamespace()
ap.persistence_mgr = SimpleNamespace(execute_async=AsyncMock(return_value=_create_mock_result([])))
runtime_provider = _create_reasoning_runtime_provider(
{
'supported': True,
'levels': ['provider_default', 'low', 'high'],
'source': 'litellm',
}
)
ap.model_mgr = _create_runtime_model_mgr()
ap.model_mgr.provider_dict = {'provider-uuid': runtime_provider}
service = LLMModelsService(ap)
await service.create_llm_model(
WORKSPACE_UUID,
{
'uuid': 'reasoning-model',
'name': 'Reasoning Model',
'provider_uuid': 'provider-uuid',
'abilities': ['reasoning'],
'reasoning_config': {'level': 'high'},
'extra_args': {},
},
preserve_uuid=True,
auto_set_to_default_pipeline=False,
)
runtime_entity = ap.model_mgr.load_llm_model_with_provider.await_args.args[1]
assert runtime_entity.reasoning_config == {'level': 'high'}
async def test_create_llm_model_rejects_unsupported_reasoning_before_insert(self):
ap = SimpleNamespace()
ap.persistence_mgr = SimpleNamespace(execute_async=AsyncMock())
runtime_provider = _create_reasoning_runtime_provider(
{
'supported': True,
'levels': ['provider_default'],
'source': 'manual',
}
)
ap.model_mgr = _create_runtime_model_mgr()
ap.model_mgr.provider_dict = {'provider-uuid': runtime_provider}
service = LLMModelsService(ap)
with pytest.raises(ValueError, match='Available levels: provider_default'):
await service.create_llm_model(
WORKSPACE_UUID,
{
'name': 'Unknown Reasoning Model',
'provider_uuid': 'provider-uuid',
'abilities': ['reasoning'],
'reasoning_config': {'level': 'high'},
'extra_args': {},
},
auto_set_to_default_pipeline=False,
)
ap.persistence_mgr.execute_async.assert_not_awaited()
class TestLLMModelsServiceUpdateLLMModel:
"""Tests for LLMModelsService.update_llm_model method."""
@@ -595,7 +717,10 @@ class TestLLMModelsServiceUpdateLLMModel:
ap.model_mgr.remove_llm_model = AsyncMock()
ap.model_mgr.load_llm_model_with_provider = AsyncMock(return_value=Mock())
ap.persistence_mgr.execute_async = AsyncMock()
existing_model = _create_mock_llm_model()
ap.persistence_mgr.execute_async = AsyncMock(
side_effect=[_create_mock_result(first_item=existing_model), _create_mock_result()]
)
service = LLMModelsService(ap)
service.get_llm_model = AsyncMock(return_value=_existing_llm_data())
@@ -623,7 +748,8 @@ class TestLLMModelsServiceUpdateLLMModel:
ap.model_mgr.provider_dict = {} # Empty
ap.model_mgr.remove_llm_model = AsyncMock()
ap.persistence_mgr.execute_async = AsyncMock()
existing_model = _create_mock_llm_model()
ap.persistence_mgr.execute_async = AsyncMock(return_value=_create_mock_result(first_item=existing_model))
service = LLMModelsService(ap)
service.get_llm_model = AsyncMock(return_value=_existing_llm_data('nonexistent-provider'))
@@ -1304,6 +1304,7 @@ class TestScanModels:
)
requester._supports_function_calling = Mock(side_effect=lambda model_id: model_id == 'gpt-4o')
requester._supports_vision = Mock(side_effect=lambda model_id: model_id == 'gpt-4o')
requester._supports_reasoning = Mock(side_effect=lambda model_id: model_id == 'o3')
requester._safe_context_length = Mock(side_effect=lambda model_id: 128000 if model_id == 'gpt-4o' else None)
mock_response = Mock()
@@ -1311,6 +1312,7 @@ class TestScanModels:
return_value={
'data': [
{'id': 'gpt-4o'},
{'id': 'o3'},
{'id': 'text-embedding-3-small'},
{'id': 'bge-reranker-v2'},
]
@@ -1327,6 +1329,7 @@ class TestScanModels:
by_id = {model['id']: model for model in result['models']}
assert by_id['gpt-4o']['abilities'] == ['func_call', 'vision']
assert by_id['gpt-4o']['context_length'] == 128000
assert by_id['o3']['abilities'] == ['reasoning']
assert by_id['text-embedding-3-small']['type'] == 'embedding'
assert by_id['bge-reranker-v2']['type'] == 'rerank'
@@ -1374,8 +1377,8 @@ class TestScanModels:
)
with patch.object(litellmchat.litellm, 'get_model_info') as mock_get_model_info:
mock_get_model_info.side_effect = (
lambda model: {'max_input_tokens': 131072} if model == 'moonshot/moonshot-v1-128k' else {}
mock_get_model_info.side_effect = lambda model: (
{'max_input_tokens': 131072} if model == 'moonshot/moonshot-v1-128k' else {}
)
assert requester._safe_context_length('moonshot-v1-128k') == 131072
@@ -1404,8 +1407,8 @@ class TestScanModels:
)
with patch.object(litellmchat.litellm, 'supports_function_calling') as mock_supports_function_calling:
mock_supports_function_calling.side_effect = (
lambda model, custom_llm_provider=None: model == 'moonshot/kimi-k2.6' and custom_llm_provider is None
mock_supports_function_calling.side_effect = lambda model, custom_llm_provider=None: (
model == 'moonshot/kimi-k2.6' and custom_llm_provider is None
)
assert requester._supports_function_calling('kimi-k2.6') is True
+132 -1
View File
@@ -249,7 +249,11 @@ async def test_updated_llm_model_is_immediately_usable_by_local_agent_pipeline()
'ai': {
'runner': {'runner': 'local-agent'},
'local-agent': {
'model': {'primary': model_uuid, 'fallbacks': []},
'model': {
'primary': model_uuid,
'fallbacks': [],
'reasoning': {model_uuid: 'high'},
},
'prompt': [],
'knowledge-bases': [],
},
@@ -293,3 +297,130 @@ async def test_updated_llm_model_is_immediately_usable_by_local_agent_pipeline()
candidates = await LocalAgentRunner._get_model_candidates(runner, processed_query)
assert [model.model_entity.uuid for model in candidates] == [model_uuid]
assert candidates[0].reasoning_config_override == {'level': 'high'}
@pytest.mark.asyncio
async def test_local_agent_applies_reasoning_per_fallback_model():
execution_context = ExecutionContext(
instance_uuid='instance-test',
workspace_uuid='workspace-test',
placement_generation=1,
)
provider = Mock(
execution_context=execution_context,
provider_entity=persistence_model.ModelProvider(
workspace_uuid='workspace-test',
uuid='provider',
name='provider',
requester='openai',
base_url='https://example.com',
api_keys=[],
),
)
primary = requester.RuntimeLLMModel(
execution_context,
persistence_model.LLMModel(
workspace_uuid='workspace-test',
uuid='primary-model',
name='primary',
provider_uuid='provider',
abilities=['reasoning'],
extra_args={},
),
provider,
)
fallback = requester.RuntimeLLMModel(
execution_context,
persistence_model.LLMModel(
workspace_uuid='workspace-test',
uuid='fallback-model',
name='fallback',
provider_uuid='provider',
abilities=['reasoning'],
extra_args={},
),
provider,
)
models = {'primary-model': primary, 'fallback-model': fallback}
runner = SimpleNamespace(
ap=SimpleNamespace(
model_mgr=SimpleNamespace(
get_model_by_uuid=AsyncMock(side_effect=lambda _context, model_uuid: models[model_uuid]),
),
logger=Mock(),
)
)
query = SimpleNamespace(
use_llm_model_uuid='primary-model',
variables={'_fallback_model_uuids': ['fallback-model']},
pipeline_config={
'ai': {
'local-agent': {
'model': {
'primary': 'primary-model',
'fallbacks': ['fallback-model'],
'reasoning': {
'primary-model': 'low',
'fallback-model': 'high',
},
}
}
}
},
_execution_context=execution_context,
)
candidates = await LocalAgentRunner._get_model_candidates(runner, query)
assert [candidate.reasoning_config_override for candidate in candidates] == [
{'level': 'low'},
{'level': 'high'},
]
def test_local_agent_rejects_invalid_pipeline_reasoning_level():
execution_context = ExecutionContext(
instance_uuid='instance-test',
workspace_uuid='workspace-test',
placement_generation=1,
)
provider = Mock(
execution_context=execution_context,
provider_entity=persistence_model.ModelProvider(
workspace_uuid='workspace-test',
uuid='provider',
name='provider',
requester='openai',
base_url='https://example.com',
api_keys=[],
),
)
model = requester.RuntimeLLMModel(
execution_context,
persistence_model.LLMModel(
workspace_uuid='workspace-test',
uuid='primary-model',
name='primary',
provider_uuid='provider',
abilities=['reasoning'],
extra_args={},
),
provider,
)
query = SimpleNamespace(
pipeline_config={
'ai': {
'local-agent': {
'model': {
'primary': 'primary-model',
'fallbacks': [],
'reasoning': {'primary-model': 'turbo'},
}
}
}
}
)
with pytest.raises(ValueError, match='Unsupported reasoning level'):
LocalAgentRunner._apply_pipeline_reasoning_config(query, model)
@@ -0,0 +1,408 @@
from __future__ import annotations
from types import SimpleNamespace
from unittest.mock import AsyncMock
import pytest
import langbot_plugin.api.entities.builtin.provider.message as provider_message
from langbot.pkg.api.http.context import ExecutionContext
from langbot.pkg.entity.persistence import model as persistence_model
from langbot.pkg.provider.modelmgr import errors, reasoning, requester
from langbot.pkg.provider.modelmgr.requesters import litellmchat
from langbot.pkg.provider.modelmgr.requesters.litellmchat import LiteLLMRequester
from langbot.pkg.provider.runners.localagent import _StreamAccumulator
def _runtime_model(
request: LiteLLMRequester,
level: str = 'provider_default',
name: str = 'reasoning-model',
abilities: list[str] | None = None,
) -> requester.RuntimeLLMModel:
execution_context = ExecutionContext(
instance_uuid='instance-test',
workspace_uuid='workspace-test',
placement_generation=1,
)
entity = persistence_model.LLMModel(
workspace_uuid='workspace-test',
uuid='reasoning-model',
name=name,
provider_uuid='provider-test',
abilities=abilities if abilities is not None else ['reasoning'],
reasoning_config={'level': level},
extra_args={},
)
provider = SimpleNamespace(
execution_context=execution_context,
provider_entity=persistence_model.ModelProvider(
workspace_uuid='workspace-test',
uuid='provider-test',
name='provider',
requester='openai',
base_url='https://example.com',
api_keys=[],
),
requester=request,
token_mgr=SimpleNamespace(),
)
return requester.RuntimeLLMModel(execution_context, entity, provider)
def _requester(provider: str = '') -> LiteLLMRequester:
return LiteLLMRequester(SimpleNamespace(), {'custom_llm_provider': provider})
def test_reasoning_config_normalization_and_conflicts():
assert reasoning.normalize_reasoning_config(None) == {'level': 'provider_default'}
assert reasoning.normalize_reasoning_config({}) == {'level': 'provider_default'}
assert reasoning.validate_reasoning_config(
{'level': 'high'},
['reasoning'],
{},
) == {'level': 'high'}
with pytest.raises(ValueError, match='Unsupported reasoning level'):
reasoning.normalize_reasoning_config({'level': 'turbo'})
with pytest.raises(ValueError, match='reasoning ability'):
reasoning.validate_reasoning_config({'level': 'low'}, [], {})
with pytest.raises(ValueError, match='extra_body.thinking_budget'):
reasoning.validate_reasoning_config(
{'level': 'low'},
['reasoning'],
{'extra_body': {'thinking_budget': 1024}},
)
def test_manual_reasoning_model_exposes_conservative_effort_levels(monkeypatch):
request = _requester()
monkeypatch.setattr(request, '_supports_reasoning', lambda _: False)
monkeypatch.setattr(request, '_safe_model_info', lambda _: {})
capabilities = request.get_reasoning_capabilities(_runtime_model(request))
assert capabilities == {
'supported': True,
'levels': ['provider_default', 'low', 'medium', 'high'],
'source': 'manual',
}
def test_provider_protocol_exposes_reasoning_for_unknown_model(monkeypatch):
request = _requester('openai')
monkeypatch.setattr(request, '_supports_reasoning', lambda _: False)
monkeypatch.setattr(request, '_safe_model_info', lambda _: pytest.fail('metadata should not be queried'))
capabilities = request.get_reasoning_capabilities(
_runtime_model(request, name='future-reasoning-model', abilities=[])
)
assert capabilities == {
'supported': True,
'levels': ['provider_default', 'low', 'medium', 'high'],
'source': 'provider',
}
def test_unknown_unmarked_model_without_provider_stays_safe(monkeypatch):
request = _requester()
monkeypatch.setattr(request, '_supports_reasoning', lambda _: False)
capabilities = request.get_reasoning_capabilities(
_runtime_model(request, name='unknown-model', abilities=[])
)
assert capabilities == {
'supported': False,
'levels': ['provider_default'],
'source': 'unknown',
}
def test_mimo_native_model_uses_known_equivalent_litellm_metadata(monkeypatch):
request = _requester('openai')
def supports_reasoning(model: str, custom_llm_provider: str | None = None) -> bool:
return model == 'openrouter/xiaomi/mimo-v2.5'
def get_model_info(model: str) -> dict:
if model == 'openrouter/xiaomi/mimo-v2.5':
return {'supports_reasoning': True}
raise ValueError('unknown model')
monkeypatch.setattr(litellmchat.litellm, 'supports_reasoning', supports_reasoning)
monkeypatch.setattr(litellmchat.litellm, 'get_model_info', get_model_info)
capabilities = request.get_reasoning_capabilities(
_runtime_model(request, name='mimo-v2.5', abilities=[])
)
assert capabilities == {
'supported': True,
'levels': ['provider_default', 'minimal', 'low', 'medium', 'high'],
'source': 'litellm',
}
def test_openai_reasoning_levels_follow_litellm_metadata(monkeypatch):
request = _requester('openai')
monkeypatch.setattr(request, '_supports_reasoning', lambda _: True)
monkeypatch.setattr(
request,
'_safe_model_info',
lambda _: {
'supports_none_reasoning_effort': True,
'supports_minimal_reasoning_effort': False,
'supports_low_reasoning_effort': True,
'supports_xhigh_reasoning_effort': True,
},
)
capabilities = request.get_reasoning_capabilities(_runtime_model(request, name='gpt-5'))
assert capabilities['source'] == 'litellm'
assert capabilities['levels'] == [
'provider_default',
'disabled',
'low',
'medium',
'high',
'xhigh',
]
def test_reasoning_argument_translation(monkeypatch):
openai_request = _requester('openai')
monkeypatch.setattr(openai_request, '_supports_reasoning', lambda _: True)
monkeypatch.setattr(
openai_request,
'_safe_model_info',
lambda _: {'supports_none_reasoning_effort': True},
)
assert openai_request._build_reasoning_args(_runtime_model(openai_request, 'provider_default')) == {}
assert openai_request._build_reasoning_args(_runtime_model(openai_request, 'disabled')) == {
'reasoning_effort': 'none'
}
assert openai_request._build_reasoning_args(_runtime_model(openai_request, 'high')) == {'reasoning_effort': 'high'}
deepseek_request = _requester('deepseek')
monkeypatch.setattr(deepseek_request, '_supports_reasoning', lambda _: False)
monkeypatch.setattr(deepseek_request, '_safe_model_info', lambda _: {})
assert deepseek_request._build_reasoning_args(
_runtime_model(deepseek_request, 'enabled', name='deepseek-chat')
) == {'thinking': {'type': 'enabled'}}
def test_pipeline_reasoning_override_takes_precedence(monkeypatch):
request = _requester('openai')
monkeypatch.setattr(request, '_supports_reasoning', lambda _: True)
monkeypatch.setattr(request, '_safe_model_info', lambda _: {})
model = _runtime_model(request, 'high', name='gpt-5')
model.reasoning_config_override = {'level': 'provider_default'}
assert request._build_reasoning_args(model) == {}
model.reasoning_config_override = {'level': 'low'}
assert request._build_reasoning_args(model) == {'reasoning_effort': 'low'}
def test_deepseek_provider_is_inferred_from_model_name(monkeypatch):
request = _requester()
monkeypatch.setattr(request, '_supports_reasoning', lambda _: True)
monkeypatch.setattr(request, '_safe_model_info', lambda _: {})
capabilities = request.get_reasoning_capabilities(_runtime_model(request, name='deepseek-chat'))
assert capabilities['levels'] == ['provider_default', 'disabled', 'enabled']
def test_always_on_reasoning_models_do_not_offer_disabled(monkeypatch):
deepseek_request = _requester('deepseek')
monkeypatch.setattr(deepseek_request, '_supports_reasoning', lambda _: True)
monkeypatch.setattr(deepseek_request, '_safe_model_info', lambda _: {})
deepseek_capabilities = deepseek_request.get_reasoning_capabilities(
_runtime_model(deepseek_request, name='deepseek-r1')
)
assert deepseek_capabilities['levels'] == ['provider_default', 'enabled']
gemini_request = _requester('gemini')
monkeypatch.setattr(gemini_request, '_supports_reasoning', lambda _: True)
monkeypatch.setattr(
gemini_request,
'_safe_model_info',
lambda _: {'supports_none_reasoning_effort': True},
)
gemini_capabilities = gemini_request.get_reasoning_capabilities(_runtime_model(gemini_request, name='gemini-3-pro'))
assert 'disabled' not in gemini_capabilities['levels']
with pytest.raises(errors.RequesterError, match='not supported'):
gemini_request._build_reasoning_args(_runtime_model(gemini_request, 'disabled', name='gemini-3-pro'))
def test_toggle_and_effort_provider_capabilities(monkeypatch):
ollama_request = _requester('ollama')
monkeypatch.setattr(ollama_request, '_supports_reasoning', lambda _: False)
monkeypatch.setattr(ollama_request, '_safe_model_info', lambda _: {})
toggle_capabilities = ollama_request.get_reasoning_capabilities(_runtime_model(ollama_request, name='qwen3'))
assert toggle_capabilities['levels'] == [
'provider_default',
'disabled',
'enabled',
]
assert ollama_request._build_reasoning_args(_runtime_model(ollama_request, 'enabled', name='qwen3')) == {
'reasoning_effort': 'low'
}
effort_capabilities = ollama_request.get_reasoning_capabilities(_runtime_model(ollama_request, name='gpt-oss:20b'))
assert effort_capabilities['levels'] == [
'provider_default',
'disabled',
'low',
'medium',
'high',
]
assert ollama_request._build_reasoning_args(_runtime_model(ollama_request, 'high', name='gpt-oss:20b')) == {
'reasoning_effort': 'high'
}
volcengine_request = _requester('volcengine')
monkeypatch.setattr(volcengine_request, '_supports_reasoning', lambda _: False)
monkeypatch.setattr(volcengine_request, '_safe_model_info', lambda _: {})
assert volcengine_request._build_reasoning_args(
_runtime_model(volcengine_request, 'disabled', name='doubao-seed')
) == {'thinking': {'type': 'disabled'}}
def test_explicit_unsupported_level_raises(monkeypatch):
request = _requester()
monkeypatch.setattr(request, '_supports_reasoning', lambda _: False)
monkeypatch.setattr(request, '_safe_model_info', lambda _: {})
with pytest.raises(errors.RequesterError, match='Available levels: provider_default'):
request._build_reasoning_args(_runtime_model(request, 'high', abilities=[]))
def test_provider_inference_rejects_levels_outside_conservative_profile(monkeypatch):
request = _requester('openai')
monkeypatch.setattr(request, '_supports_reasoning', lambda _: False)
with pytest.raises(errors.RequesterError, match='Available levels: provider_default, low, medium, high'):
request._build_reasoning_args(_runtime_model(request, 'xhigh', abilities=[]))
@pytest.mark.asyncio
async def test_completion_args_reject_reasoning_extra_arg_conflicts(monkeypatch):
request = _requester('openai')
monkeypatch.setattr(request, '_supports_reasoning', lambda _: True)
monkeypatch.setattr(request, '_safe_model_info', lambda _: {})
model = _runtime_model(request, 'high')
model.model_entity.extra_args = {'reasoning_effort': 'low'}
model.provider.token_mgr.get_token = lambda: 'test-token'
with pytest.raises(errors.RequesterError, match='conflicts with advanced parameters'):
await request._build_completion_args(model, [])
@pytest.mark.asyncio
async def test_openai_compatible_reasoning_effort_is_explicitly_allowed(monkeypatch):
request = _requester('openai')
monkeypatch.setattr(request, '_supports_reasoning', lambda _: True)
monkeypatch.setattr(request, '_safe_model_info', lambda _: {})
model = _runtime_model(request, 'high', name='deepseek-v4-flash')
model.model_entity.extra_args = {'allowed_openai_params': ['custom_extension']}
model.provider.token_mgr.get_token = lambda: 'test-token'
args = await request._build_completion_args(model, [])
assert args['reasoning_effort'] == 'high'
assert args['allowed_openai_params'] == ['custom_extension', 'reasoning_effort']
@pytest.mark.asyncio
async def test_provider_default_does_not_allow_or_send_reasoning_effort():
request = _requester('openai')
model = _runtime_model(request, 'provider_default', name='deepseek-v4-flash')
model.provider.token_mgr.get_token = lambda: 'test-token'
args = await request._build_completion_args(model, [])
assert 'reasoning_effort' not in args
assert 'allowed_openai_params' not in args
class _Dumpable:
def __init__(self, data: dict):
self.data = data
def model_dump(self) -> dict:
return dict(self.data)
@pytest.mark.asyncio
async def test_non_stream_reasoning_content_is_preserved(monkeypatch):
request = _requester('deepseek')
request._build_completion_args = AsyncMock(return_value={})
response = SimpleNamespace(
choices=[
SimpleNamespace(
message=_Dumpable(
{
'role': 'assistant',
'content': 'answer',
'reasoning_content': 'private reasoning',
}
)
)
],
usage=None,
)
monkeypatch.setattr(litellmchat, 'acompletion', AsyncMock(return_value=response))
message, _ = await request.invoke_llm(None, _runtime_model(request), [], remove_think=True)
assert message.content == 'answer'
assert message.provider_specific_fields == {'reasoning_content': 'private reasoning'}
@pytest.mark.asyncio
async def test_stream_reasoning_round_trip_with_hidden_display(monkeypatch):
request = _requester('deepseek')
request._build_completion_args = AsyncMock(return_value={})
async def chunks():
yield SimpleNamespace(
choices=[
SimpleNamespace(
delta=_Dumpable({'role': 'assistant', 'reasoning_content': 'private '}),
finish_reason=None,
)
],
usage=None,
)
yield SimpleNamespace(
choices=[
SimpleNamespace(
delta=_Dumpable({'content': 'answer'}),
finish_reason='stop',
)
],
usage=None,
)
monkeypatch.setattr(litellmchat, 'acompletion', AsyncMock(return_value=chunks()))
accumulator = _StreamAccumulator(remove_think=True)
emitted: provider_message.MessageChunk | None = None
async for chunk in request.invoke_llm_stream(
None,
_runtime_model(request),
[],
remove_think=True,
):
emitted = accumulator.add(chunk) or emitted
assert emitted is not None
assert emitted.content == 'answer'
assert emitted.provider_specific_fields == {'reasoning_content': 'private '}
@@ -400,6 +400,7 @@ def test_runtime_llm_model_initialization(runtime_llm_model, fake_persistence_da
assert model.model_entity.abilities == model_entity.abilities
assert model.model_entity.extra_args == model_entity.extra_args
assert model.provider is not None
assert model.reasoning_config_override is None
def test_runtime_llm_model_provider_ref(runtime_llm_model):
@@ -144,6 +144,7 @@ function getValueSchema(spec: DynamicFormValueSpec) {
return z.object({
primary: z.string(),
fallbacks: z.array(z.string()),
reasoning: z.record(z.string()),
});
case DynamicFormItemType.PROMPT_EDITOR:
return z.array(
@@ -488,12 +489,24 @@ export default function DynamicFormComponent({
(v): v is string => typeof v === 'string',
)
: [],
reasoning:
obj.reasoning != null &&
typeof obj.reasoning === 'object' &&
!Array.isArray(obj.reasoning)
? Object.fromEntries(
Object.entries(obj.reasoning).filter(
(entry): entry is [string, string] =>
typeof entry[1] === 'string',
),
)
: {},
};
}
// Legacy string format or any other unexpected type
return {
primary: typeof value === 'string' ? value : '',
fallbacks: [],
reasoning: {},
};
}
if (item.type === 'prompt-editor') {
@@ -25,6 +25,7 @@ import {
EmbeddingModel,
RerankModel,
PluginTool,
ReasoningLevel,
} from '@/app/infra/entities/api';
import { toast } from 'sonner';
import { useTranslation } from 'react-i18next';
@@ -66,6 +67,9 @@ import SettingsDialog, {
} from '@/app/home/components/settings-dialog/SettingsDialog';
import ToolResourceSelectors from '@/app/home/components/dynamic-form/ToolResourceSelectors';
import { LANGBOT_MODELS_PROVIDER_REQUESTER } from '@/app/home/components/models-dialog/types';
import ReasoningLevelPicker, {
REASONING_LEVELS,
} from '@/app/home/components/reasoning/ReasoningLevelPicker';
function hasUsableUuid<T extends { uuid?: string | null }>(
item: T,
@@ -874,7 +878,11 @@ export default function DynamicFormItemComponent({
];
const rawModelValue = field.value;
const modelValue: { primary: string; fallbacks: string[] } =
const modelValue: {
primary: string;
fallbacks: string[];
reasoning: Record<string, ReasoningLevel>;
} =
rawModelValue != null &&
typeof rawModelValue === 'object' &&
!Array.isArray(rawModelValue)
@@ -893,10 +901,31 @@ export default function DynamicFormItemComponent({
.fallbacks as unknown[]
).filter((v): v is string => typeof v === 'string')
: [],
reasoning:
(rawModelValue as Record<string, unknown>).reasoning != null &&
typeof (rawModelValue as Record<string, unknown>).reasoning ===
'object' &&
!Array.isArray(
(rawModelValue as Record<string, unknown>).reasoning,
)
? (Object.fromEntries(
Object.entries(
(rawModelValue as Record<string, unknown>)
.reasoning as Record<string, unknown>,
).filter(
(entry): entry is [string, ReasoningLevel] =>
typeof entry[1] === 'string' &&
REASONING_LEVELS.includes(
entry[1] as ReasoningLevel,
),
),
) as Record<string, ReasoningLevel>)
: {},
}
: {
primary: typeof rawModelValue === 'string' ? rawModelValue : '',
fallbacks: [],
reasoning: {},
};
const renderModelSelect = (
@@ -1043,20 +1072,78 @@ export default function DynamicFormItemComponent({
field.onChange({ ...modelValue, ...patch });
};
const updateModelReasoning = (
modelUuid: string,
level: ReasoningLevel,
) => {
if (!modelUuid) return;
const updated = { ...modelValue.reasoning };
if (level === 'provider_default') {
delete updated[modelUuid];
} else {
updated[modelUuid] = level;
}
updateValue({ reasoning: updated });
};
const replaceModel = (
currentUuid: string,
nextUuid: string,
patch: Partial<typeof modelValue>,
) => {
const nextValue = { ...modelValue, ...patch };
const updatedReasoning = { ...modelValue.reasoning };
const currentModelStillSelected =
nextValue.primary === currentUuid ||
nextValue.fallbacks.includes(currentUuid);
if (
currentUuid &&
currentUuid !== nextUuid &&
!currentModelStillSelected
) {
delete updatedReasoning[currentUuid];
}
updateValue({ ...nextValue, reasoning: updatedReasoning });
};
const renderReasoningPicker = (modelUuid: string) => {
if (!modelUuid) return null;
const model = llmModels.find((candidate) => candidate.uuid === modelUuid);
const currentLevel =
modelValue.reasoning[modelUuid] || 'provider_default';
const availableLevels = model?.reasoning_capabilities?.levels || [
'provider_default',
];
const levels = REASONING_LEVELS.filter(
(level) =>
availableLevels.includes(level) || level === currentLevel,
);
return (
<ReasoningLevelPicker
value={currentLevel}
levels={levels}
onChange={(level) => updateModelReasoning(modelUuid, level)}
/>
);
};
const addFallbackModel = () => {
updateValue({ fallbacks: [...modelValue.fallbacks, ''] });
};
const updateFallbackModel = (index: number, value: string) => {
const updated = [...modelValue.fallbacks];
const currentUuid = updated[index];
updated[index] = value;
updateValue({ fallbacks: updated });
replaceModel(currentUuid, value, { fallbacks: updated });
};
const removeFallbackModel = (index: number) => {
const updated = [...modelValue.fallbacks];
const removedUuid = updated[index];
updated.splice(index, 1);
updateValue({ fallbacks: updated });
replaceModel(removedUuid, '', { fallbacks: updated });
};
const moveFallbackModel = (index: number, direction: 'up' | 'down') => {
@@ -1081,10 +1168,12 @@ export default function DynamicFormItemComponent({
<div className="min-w-0 flex-1">
{renderModelSelect(
modelValue.primary,
(val) => updateValue({ primary: val }),
(val) =>
replaceModel(modelValue.primary, val, { primary: val }),
t('models.selectModel'),
)}
</div>
{renderReasoningPicker(modelValue.primary)}
<Tooltip>
<TooltipTrigger asChild>
<Button
@@ -1118,15 +1207,18 @@ export default function DynamicFormItemComponent({
</p>
{modelValue.fallbacks.map((fbUuid: string, index: number) => (
<div key={index} className="flex min-w-0 items-center gap-2">
<span className="text-xs text-muted-foreground w-4 shrink-0">
<span className="w-4 shrink-0 text-xs text-muted-foreground">
{index + 1}.
</span>
<div className="min-w-0 flex-1">
{renderModelSelect(
fbUuid,
(val) => updateFallbackModel(index, val),
t('models.selectModel'),
)}
<div className="flex min-w-0 flex-1 items-center gap-1.5">
<div className="min-w-0 flex-1">
{renderModelSelect(
fbUuid,
(val) => updateFallbackModel(index, val),
t('models.selectModel'),
)}
</div>
{renderReasoningPicker(fbUuid)}
</div>
<div className="flex gap-1 shrink-0">
<Button
@@ -5,6 +5,56 @@ export type DynamicFormSaveValueSpec = Pick<
'default' | 'name' | 'type'
>;
const reasoningLevels = new Set([
'disabled',
'enabled',
'minimal',
'low',
'medium',
'high',
'xhigh',
'max',
]);
function normalizeModelFallbackValue(value: unknown): {
primary: string;
fallbacks: string[];
reasoning: Record<string, string>;
} {
const raw =
value != null && typeof value === 'object' && !Array.isArray(value)
? (value as Record<string, unknown>)
: {};
const primary =
typeof raw.primary === 'string'
? raw.primary
: typeof value === 'string'
? value
: '';
const fallbacks = Array.isArray(raw.fallbacks)
? raw.fallbacks.filter(
(fallback): fallback is string => typeof fallback === 'string',
)
: [];
const selectedModels = new Set([primary, ...fallbacks].filter(Boolean));
const rawReasoning =
raw.reasoning != null &&
typeof raw.reasoning === 'object' &&
!Array.isArray(raw.reasoning)
? (raw.reasoning as Record<string, unknown>)
: {};
const reasoning = Object.fromEntries(
Object.entries(rawReasoning).filter(
([modelUuid, level]) =>
selectedModels.has(modelUuid) &&
typeof level === 'string' &&
reasoningLevels.has(level),
),
) as Record<string, string>;
return { primary, fallbacks, reasoning };
}
/**
* Build the value snapshot emitted to parent forms for persistence.
* Only single-line string fields trim surrounding whitespace; multiline text
@@ -16,10 +66,14 @@ export function normalizeDynamicFormValuesForSave(
): Record<string, unknown> {
return specs.reduce<Record<string, unknown>>((values, spec) => {
const value = formValues[spec.name] ?? spec.default;
values[spec.name] =
spec.type === 'string' && typeof value === 'string'
? value.trim()
: value;
if (spec.type === 'model-fallback-selector') {
values[spec.name] = normalizeModelFallbackValue(value);
} else {
values[spec.name] =
spec.type === 'string' && typeof value === 'string'
? value.trim()
: value;
}
return values;
}, {});
}
@@ -1,7 +1,7 @@
import { useState, useEffect } from 'react';
import { Plus, Boxes } from 'lucide-react';
import { httpClient, systemInfo } from '@/app/infra/http/HttpClient';
import { ModelProvider } from '@/app/infra/entities/api';
import { ModelProvider, ReasoningConfig } from '@/app/infra/entities/api';
import {
Dialog,
DialogContent,
@@ -15,6 +15,7 @@ import ProviderForm from './component/provider-form/ProviderForm';
import { ProviderCard } from './components';
import {
ExtraArg,
DEFAULT_REASONING_CONFIG,
ModelType,
ScanModelsResult,
SelectedScannedModel,
@@ -285,6 +286,7 @@ export default function ModelsPanel({
name: string,
abilities: string[],
extraArgs: ExtraArg[],
reasoningConfig: ReasoningConfig,
contextLength?: number | null,
) {
if (!name.trim()) {
@@ -300,6 +302,7 @@ export default function ModelsPanel({
name,
provider_uuid: providerUuid,
abilities,
reasoning_config: reasoningConfig,
context_length: parseContextLength(
contextLength,
t('models.contextLengthInvalid'),
@@ -361,6 +364,7 @@ export default function ModelsPanel({
name: item.model.name,
provider_uuid: providerUuid,
abilities: item.abilities,
reasoning_config: DEFAULT_REASONING_CONFIG,
context_length: item.model.context_length ?? null,
extra_args: {},
} as never);
@@ -398,6 +402,7 @@ export default function ModelsPanel({
name: string,
abilities: string[],
extraArgs: ExtraArg[],
reasoningConfig: ReasoningConfig,
contextLength?: number | null,
) {
if (!name.trim()) {
@@ -413,6 +418,7 @@ export default function ModelsPanel({
name,
provider_uuid: providerUuid,
abilities,
reasoning_config: reasoningConfig,
context_length: parseContextLength(
contextLength,
t('models.contextLengthInvalid'),
@@ -469,6 +475,7 @@ export default function ModelsPanel({
modelType: ModelType,
abilities: string[],
extraArgs: ExtraArg[],
reasoningConfig: ReasoningConfig,
) {
setIsTesting(true);
setTestResult(null);
@@ -491,6 +498,7 @@ export default function ModelsPanel({
provider_uuid: '',
provider: providerData,
abilities,
reasoning_config: reasoningConfig,
extra_args: extraArgsObj,
} as never);
} else if (modelType === 'embedding') {
@@ -554,13 +562,21 @@ export default function ModelsPanel({
onSpaceLogin={handleSpaceLogin}
onOpenAddModel={() => setAddModelPopoverOpen(provider.uuid)}
onCloseAddModel={() => setAddModelPopoverOpen(null)}
onAddModel={(modelType, name, abilities, extraArgs, contextLength) =>
onAddModel={(
modelType,
name,
abilities,
extraArgs,
reasoningConfig,
contextLength,
) =>
handleAddModel(
provider.uuid,
modelType,
name,
abilities,
extraArgs,
reasoningConfig,
contextLength,
)
}
@@ -576,6 +592,7 @@ export default function ModelsPanel({
name,
abilities,
extraArgs,
reasoningConfig,
contextLength,
) =>
handleUpdateModel(
@@ -585,6 +602,7 @@ export default function ModelsPanel({
name,
abilities,
extraArgs,
reasoningConfig,
contextLength,
)
}
@@ -593,8 +611,15 @@ export default function ModelsPanel({
onDeleteModel={(modelId, modelType) =>
handleDeleteModel(provider.uuid, modelId, modelType)
}
onTestModel={(name, modelType, abilities, extraArgs) =>
handleTestModel(provider.uuid, name, modelType, abilities, extraArgs)
onTestModel={(name, modelType, abilities, extraArgs, reasoningConfig) =>
handleTestModel(
provider.uuid,
name,
modelType,
abilities,
extraArgs,
reasoningConfig,
)
}
isSubmitting={isSubmitting}
isTesting={isTesting}
@@ -6,6 +6,7 @@ import {
ArrowUpDown,
Eye,
Wrench,
BrainCircuit,
Check,
RefreshCw,
} from 'lucide-react';
@@ -20,8 +21,12 @@ import {
} from '@/components/ui/popover';
import { Tabs, TabsList, TabsTrigger } from '@/components/ui/tabs';
import { useTranslation } from 'react-i18next';
import { ScannedProviderModel } from '@/app/infra/entities/api';
import {
ReasoningConfig,
ScannedProviderModel,
} from '@/app/infra/entities/api';
import {
DEFAULT_REASONING_CONFIG,
ExtraArg,
ModelType,
ScanModelsResult,
@@ -42,6 +47,7 @@ interface AddModelPopoverProps {
name: string,
abilities: string[],
extraArgs: ExtraArg[],
reasoningConfig: ReasoningConfig,
contextLength?: number | null,
) => Promise<void>;
onScanModels: (modelType?: ModelType) => Promise<ScanModelsResult>;
@@ -54,6 +60,7 @@ interface AddModelPopoverProps {
modelType: ModelType,
abilities: string[],
extraArgs: ExtraArg[],
reasoningConfig: ReasoningConfig,
) => Promise<void>;
isSubmitting: boolean;
isTesting: boolean;
@@ -143,11 +150,24 @@ export default function AddModelPopover({
tab === 'llm' && contextLength.trim()
? Number(contextLength.trim())
: null;
await onAddModel(tab, name, abilities, extraArgs, parsedContextLength);
await onAddModel(
tab,
name,
abilities,
extraArgs,
DEFAULT_REASONING_CONFIG,
parsedContextLength,
);
};
const handleTest = async () => {
await onTestModel(name, tab, tab === 'llm' ? abilities : [], extraArgs);
await onTestModel(
name,
tab,
tab === 'llm' ? abilities : [],
extraArgs,
DEFAULT_REASONING_CONFIG,
);
};
const handleScan = async () => {
@@ -322,7 +342,7 @@ export default function AddModelPopover({
{tab === 'llm' && (
<div className="space-y-2">
<Label>{t('models.abilities')}</Label>
<div className="flex gap-4">
<div className="flex flex-wrap gap-4">
<div className="flex items-center gap-2">
<Checkbox
id="add-vision"
@@ -349,6 +369,19 @@ export default function AddModelPopover({
{t('models.functionCallAbility')}
</Label>
</div>
<div className="flex items-center gap-2">
<Checkbox
id="add-reasoning"
checked={abilities.includes('reasoning')}
onCheckedChange={(checked) =>
toggleAbility('reasoning', checked as boolean)
}
/>
<Label htmlFor="add-reasoning" className="text-sm">
<BrainCircuit className="h-3 w-3 inline mr-1" />
{t('models.reasoningAbility')}
</Label>
</div>
</div>
</div>
)}
@@ -1,5 +1,5 @@
import { useState, useEffect } from 'react';
import { Trash2, Eye, Wrench, Check } from 'lucide-react';
import { Trash2, Eye, Wrench, Check, BrainCircuit } from 'lucide-react';
import { Button } from '@/components/ui/button';
import { Input } from '@/components/ui/input';
import { Label } from '@/components/ui/label';
@@ -11,8 +11,17 @@ import {
PopoverTrigger,
} from '@/components/ui/popover';
import { useTranslation } from 'react-i18next';
import { LLMModel, EmbeddingModel } from '@/app/infra/entities/api';
import { ExtraArg, ModelType, TestResult } from '../types';
import {
LLMModel,
EmbeddingModel,
ReasoningConfig,
} from '@/app/infra/entities/api';
import {
DEFAULT_REASONING_CONFIG,
ExtraArg,
ModelType,
TestResult,
} from '../types';
import ExtraArgsEditor from './ExtraArgsEditor';
import { userInfo } from '@/app/infra/http';
@@ -32,12 +41,14 @@ interface ModelItemProps {
name: string,
abilities: string[],
extraArgs: ExtraArg[],
reasoningConfig: ReasoningConfig,
contextLength?: number | null,
) => Promise<void>;
onTestModel: (
name: string,
abilities: string[],
extraArgs: ExtraArg[],
reasoningConfig: ReasoningConfig,
) => Promise<void>;
isSubmitting: boolean;
isTesting: boolean;
@@ -103,7 +114,6 @@ export default function ModelItem({
const [editExtraArgs, setEditExtraArgs] = useState<ExtraArg[]>(
convertExtraArgsToArray(model.extra_args),
);
const isEditOpen = editModelPopoverOpen === model.uuid;
const isDeleteOpen = deleteConfirmOpen === model.uuid;
@@ -133,12 +143,20 @@ export default function ModelItem({
editName,
editAbilities,
editExtraArgs,
modelType === 'llm'
? (model as LLMModel).reasoning_config || DEFAULT_REASONING_CONFIG
: DEFAULT_REASONING_CONFIG,
parsedContextLength,
);
};
const handleTest = async () => {
await onTestModel(editName, editAbilities, editExtraArgs);
await onTestModel(
editName,
editAbilities,
editExtraArgs,
DEFAULT_REASONING_CONFIG,
);
};
const toggleAbility = (ability: string, checked: boolean) => {
@@ -149,6 +167,12 @@ export default function ModelItem({
}
};
const supportsReasoning =
modelType === 'llm' &&
((model as LLMModel).reasoning_capabilities?.supported === true ||
(model as LLMModel).abilities?.includes('reasoning'));
const canSaveModel = !isLangBotModels;
// Check if popover should be disabled (space models when not logged in)
const isPopoverDisabled =
!canManage || (isLangBotModels && userInfo?.account_type !== 'space');
@@ -194,6 +218,12 @@ export default function ModelItem({
<Wrench className="h-3 w-3" />
</Badge>
)}
{supportsReasoning && (
<Badge variant="outline" className="text-xs gap-1">
<BrainCircuit className="h-3 w-3" />
{t('models.reasoningAbility')}
</Badge>
)}
</div>
{canManage && !isLangBotModels && (
<Popover
@@ -270,7 +300,7 @@ export default function ModelItem({
{modelType === 'llm' && (
<div className="space-y-2">
<Label>{t('models.abilities')}</Label>
<div className="flex gap-4">
<div className="flex flex-wrap gap-4">
<div className="flex items-center gap-2">
<Checkbox
id={`edit-vision-${model.uuid}`}
@@ -305,6 +335,23 @@ export default function ModelItem({
{t('models.functionCallAbility')}
</Label>
</div>
<div className="flex items-center gap-2">
<Checkbox
id={`edit-reasoning-${model.uuid}`}
checked={editAbilities.includes('reasoning')}
disabled={isLangBotModels}
onCheckedChange={(checked) =>
toggleAbility('reasoning', checked as boolean)
}
/>
<Label
htmlFor={`edit-reasoning-${model.uuid}`}
className="text-sm"
>
<BrainCircuit className="h-3 w-3 inline mr-1" />
{t('models.reasoningAbility')}
</Label>
</div>
</div>
</div>
)}
@@ -336,7 +383,7 @@ export default function ModelItem({
/>
<div className="flex gap-2">
{!isLangBotModels && (
{canSaveModel && (
<Button
className="flex-1"
size="sm"
@@ -347,7 +394,7 @@ export default function ModelItem({
</Button>
)}
<Button
className={isLangBotModels ? 'w-full' : 'flex-1'}
className={canSaveModel ? 'flex-1' : 'w-full'}
size="sm"
variant="outline"
onClick={handleTest}
@@ -9,7 +9,7 @@ import {
Radar,
} from 'lucide-react';
import { httpClient, systemInfo } from '@/app/infra/http/HttpClient';
import { ModelProvider } from '@/app/infra/entities/api';
import { ModelProvider, ReasoningConfig } from '@/app/infra/entities/api';
import { Button } from '@/components/ui/button';
import {
Collapsible,
@@ -63,6 +63,7 @@ interface ProviderCardProps {
name: string,
abilities: string[],
extraArgs: ExtraArg[],
reasoningConfig: ReasoningConfig,
contextLength?: number | null,
) => Promise<void>;
onScanModels: (modelType?: ModelType) => Promise<ScanModelsResult>;
@@ -78,6 +79,7 @@ interface ProviderCardProps {
name: string,
abilities: string[],
extraArgs: ExtraArg[],
reasoningConfig: ReasoningConfig,
contextLength?: number | null,
) => Promise<void>;
onOpenDeleteConfirm: (modelId: string) => void;
@@ -88,6 +90,7 @@ interface ProviderCardProps {
modelType: ModelType,
abilities: string[],
extraArgs: ExtraArg[],
reasoningConfig: ReasoningConfig,
) => Promise<void>;
isSubmitting: boolean;
isTesting: boolean;
@@ -430,6 +433,7 @@ export default function ProviderCard({
name,
abilities,
extraArgs,
reasoningConfig,
contextLength,
) =>
onUpdateModel(
@@ -438,11 +442,23 @@ export default function ProviderCard({
name,
abilities,
extraArgs,
reasoningConfig,
contextLength,
)
}
onTestModel={(name, abilities, extraArgs) =>
onTestModel(name, 'llm', abilities, extraArgs)
onTestModel={(
name,
abilities,
extraArgs,
reasoningConfig,
) =>
onTestModel(
name,
'llm',
abilities,
extraArgs,
reasoningConfig,
)
}
isSubmitting={isSubmitting}
isTesting={isTesting}
@@ -464,17 +480,34 @@ export default function ProviderCard({
onOpenDeleteConfirm={onOpenDeleteConfirm}
onCloseDeleteConfirm={onCloseDeleteConfirm}
onDeleteModel={() => onDeleteModel(model.uuid, 'embedding')}
onUpdateModel={(name, abilities, extraArgs) =>
onUpdateModel={(
name,
abilities,
extraArgs,
reasoningConfig,
) =>
onUpdateModel(
model.uuid,
'embedding',
name,
abilities,
extraArgs,
reasoningConfig,
)
}
onTestModel={(name, abilities, extraArgs) =>
onTestModel(name, 'embedding', abilities, extraArgs)
onTestModel={(
name,
abilities,
extraArgs,
reasoningConfig,
) =>
onTestModel(
name,
'embedding',
abilities,
extraArgs,
reasoningConfig,
)
}
isSubmitting={isSubmitting}
isTesting={isTesting}
@@ -496,17 +529,34 @@ export default function ProviderCard({
onOpenDeleteConfirm={onOpenDeleteConfirm}
onCloseDeleteConfirm={onCloseDeleteConfirm}
onDeleteModel={() => onDeleteModel(model.uuid, 'rerank')}
onUpdateModel={(name, abilities, extraArgs) =>
onUpdateModel={(
name,
abilities,
extraArgs,
reasoningConfig,
) =>
onUpdateModel(
model.uuid,
'rerank',
name,
abilities,
extraArgs,
reasoningConfig,
)
}
onTestModel={(name, abilities, extraArgs) =>
onTestModel(name, 'rerank', abilities, extraArgs)
onTestModel={(
name,
abilities,
extraArgs,
reasoningConfig,
) =>
onTestModel(
name,
'rerank',
abilities,
extraArgs,
reasoningConfig,
)
}
isSubmitting={isSubmitting}
isTesting={isTesting}
@@ -5,6 +5,7 @@ import {
ModelProvider,
ProviderScanDebugInfo,
ScannedProviderModel,
ReasoningConfig,
} from '@/app/infra/entities/api';
export type ExtraArg = {
@@ -16,6 +17,10 @@ export type ExtraArg = {
export type ModelType = 'llm' | 'embedding' | 'rerank';
export const DEFAULT_REASONING_CONFIG: ReasoningConfig = {
level: 'provider_default',
};
export interface ProviderModels {
llm: LLMModel[];
embedding: EmbeddingModel[];
@@ -53,12 +58,14 @@ export interface ModelItemProps {
name: string,
abilities: string[],
extraArgs: ExtraArg[],
reasoningConfig: ReasoningConfig,
contextLength?: number | null,
) => Promise<void>;
onTest: (
name: string,
abilities: string[],
extraArgs: ExtraArg[],
reasoningConfig: ReasoningConfig,
) => Promise<void>;
isSubmitting: boolean;
isTesting: boolean;
@@ -90,6 +97,7 @@ export interface ProviderCardProps {
name: string,
abilities: string[],
extraArgs: ExtraArg[],
reasoningConfig: ReasoningConfig,
contextLength?: number | null,
) => Promise<void>;
onScanModels: (modelType?: ModelType) => Promise<ScanModelsResult>;
@@ -105,6 +113,7 @@ export interface ProviderCardProps {
name: string,
abilities: string[],
extraArgs: ExtraArg[],
reasoningConfig: ReasoningConfig,
contextLength?: number | null,
) => Promise<void>;
onOpenDeleteConfirm: (modelId: string) => void;
@@ -115,6 +124,7 @@ export interface ProviderCardProps {
modelType: ModelType,
abilities: string[],
extraArgs: ExtraArg[],
reasoningConfig: ReasoningConfig,
) => Promise<void>;
isSubmitting: boolean;
isTesting: boolean;
@@ -0,0 +1,86 @@
.control {
position: relative;
height: 2.5rem;
margin-top: 0.75rem;
}
.track,
.fill {
position: absolute;
top: 50%;
left: 0;
height: 2rem;
transform: translateY(-50%);
border-radius: 1rem;
}
.track {
width: 100%;
background: var(--muted);
}
.fill {
background: var(--primary);
}
.tick {
position: absolute;
top: 50%;
width: 0.3rem;
height: 0.3rem;
transform: translate(-50%, -50%);
border-radius: 50%;
background: color-mix(in oklch, var(--muted-foreground) 46%, transparent);
pointer-events: none;
}
.tickActive {
background: color-mix(in oklch, var(--primary-foreground) 42%, transparent);
}
.input {
position: absolute;
inset: 0;
width: 100%;
height: 100%;
margin: 0;
appearance: none;
background: transparent;
cursor: pointer;
}
.input::-webkit-slider-runnable-track {
height: 2rem;
background: transparent;
}
.input::-webkit-slider-thumb {
width: 2.125rem;
height: 2.125rem;
margin-top: -0.0625rem;
appearance: none;
border: 1px solid var(--border);
border-radius: 50%;
background: var(--background);
box-shadow: 0 1px 4px color-mix(in oklch, var(--foreground) 20%, transparent);
}
.input::-moz-range-track {
height: 2rem;
background: transparent;
}
.input::-moz-range-thumb {
width: 2.125rem;
height: 2.125rem;
border: 1px solid var(--border);
border-radius: 50%;
background: var(--background);
box-shadow: 0 1px 4px color-mix(in oklch, var(--foreground) 20%, transparent);
}
.input:focus-visible {
outline: 2px solid var(--ring);
outline-offset: 2px;
border-radius: 1rem;
}
@@ -0,0 +1,123 @@
import { BrainCircuit, ChevronDown, ChevronRight } from 'lucide-react';
import { useTranslation } from 'react-i18next';
import { ReasoningLevel } from '@/app/infra/entities/api';
import { Button } from '@/components/ui/button';
import {
Popover,
PopoverContent,
PopoverTrigger,
} from '@/components/ui/popover';
import styles from './ReasoningLevelPicker.module.css';
export const REASONING_LEVELS: ReasoningLevel[] = [
'provider_default',
'disabled',
'enabled',
'minimal',
'low',
'medium',
'high',
'xhigh',
'max',
];
export const REASONING_LEVEL_LABEL_KEYS: Record<ReasoningLevel, string> = {
provider_default: 'models.reasoningLevels.providerDefault',
disabled: 'models.reasoningLevels.disabled',
enabled: 'models.reasoningLevels.enabled',
minimal: 'models.reasoningLevels.minimal',
low: 'models.reasoningLevels.low',
medium: 'models.reasoningLevels.medium',
high: 'models.reasoningLevels.high',
xhigh: 'models.reasoningLevels.xhigh',
max: 'models.reasoningLevels.max',
};
interface ReasoningLevelPickerProps {
value: ReasoningLevel;
levels: ReasoningLevel[];
disabled?: boolean;
onChange: (value: ReasoningLevel) => void;
}
export default function ReasoningLevelPicker({
value,
levels,
disabled = false,
onChange,
}: ReasoningLevelPickerProps) {
const { t } = useTranslation();
const safeLevels: ReasoningLevel[] =
levels.length > 0 ? levels : ['provider_default'];
const safeValue: ReasoningLevel = safeLevels.includes(value)
? value
: safeLevels[0];
const currentLabel = t(REASONING_LEVEL_LABEL_KEYS[safeValue]);
const isExplicit = safeValue !== 'provider_default';
const currentIndex = Math.max(0, safeLevels.indexOf(safeValue));
const denominator = Math.max(1, safeLevels.length - 1);
const progress = currentIndex / denominator;
return (
<Popover>
<PopoverTrigger asChild>
<Button
type="button"
variant="outline"
size="sm"
disabled={disabled || safeLevels.length <= 1}
aria-label={`${t('models.reasoningLevel')}: ${currentLabel}`}
className="h-9 w-9 shrink-0 gap-1.5 px-2.5 text-xs font-normal sm:w-auto sm:max-w-36"
>
<BrainCircuit
className={`size-4 shrink-0 ${isExplicit ? 'text-primary' : 'text-muted-foreground'}`}
/>
<span className="hidden min-w-0 truncate sm:block">
{currentLabel}
</span>
<ChevronDown className="hidden size-3.5 shrink-0 text-muted-foreground sm:block" />
</Button>
</PopoverTrigger>
<PopoverContent align="end" className="w-[272px] p-4">
<div className="flex h-5 items-center gap-0.5 text-sm text-muted-foreground">
<span>{currentLabel}</span>
<ChevronRight className="size-3.5" />
</div>
<div className={styles.control}>
<div className={styles.track} />
<div
className={styles.fill}
style={{
width: `calc(17px + (100% - 34px) * ${progress})`,
}}
/>
{safeLevels.map((level, index) => {
const tickProgress = index / denominator;
return (
<span
key={level}
className={`${styles.tick} ${index <= currentIndex ? styles.tickActive : ''}`}
style={{
left: `calc(17px + (100% - 34px) * ${tickProgress})`,
}}
/>
);
})}
<input
className={styles.input}
type="range"
min={0}
max={Math.max(0, safeLevels.length - 1)}
step={1}
value={currentIndex}
aria-label={t('models.reasoningLevel')}
aria-valuetext={currentLabel}
onChange={(event) =>
onChange(safeLevels[Number(event.target.value)])
}
/>
</div>
</PopoverContent>
</Popover>
);
}
+23
View File
@@ -99,9 +99,32 @@ export interface LLMModel {
provider?: ModelProvider;
abilities?: string[];
context_length?: number | null;
reasoning_config?: ReasoningConfig;
reasoning_capabilities?: ReasoningCapabilities;
extra_args?: object;
}
export type ReasoningLevel =
| 'provider_default'
| 'disabled'
| 'enabled'
| 'minimal'
| 'low'
| 'medium'
| 'high'
| 'xhigh'
| 'max';
export interface ReasoningConfig {
level: ReasoningLevel;
}
export interface ReasoningCapabilities {
supported: boolean;
levels: ReasoningLevel[];
source: 'litellm' | 'provider' | 'manual' | 'unknown';
}
export interface ApiRespProviderEmbeddingModels {
models: EmbeddingModel[];
}
+13
View File
@@ -214,6 +214,19 @@ const enUS = {
selectModelAbilities: 'Select model abilities',
visionAbility: 'Vision Ability',
functionCallAbility: 'Function Call',
reasoningAbility: 'Reasoning',
reasoningLevel: 'Reasoning level',
reasoningLevels: {
providerDefault: 'Provider default',
disabled: 'Off',
enabled: 'On',
minimal: 'Minimal',
low: 'Low',
medium: 'Medium',
high: 'High',
xhigh: 'Extra high',
max: 'Maximum',
},
contextLength: 'Context Window',
contextLengthPlaceholder: 'Unknown',
contextLengthInvalid: 'Context window must be a positive integer',
+13
View File
@@ -217,6 +217,19 @@ const jaJP = {
selectModelAbilities: 'モデル機能を選択',
visionAbility: '視覚機能',
functionCallAbility: '関数呼び出し',
reasoningAbility: '推論',
reasoningLevel: '推論レベル',
reasoningLevels: {
providerDefault: 'Provider デフォルト',
disabled: 'オフ',
enabled: 'オン',
minimal: '最小',
low: '低',
medium: '中',
high: '高',
xhigh: '最高',
max: '最大',
},
contextLength: 'コンテキストウィンドウ',
contextLengthPlaceholder: '不明',
contextLengthInvalid:
+13
View File
@@ -204,6 +204,19 @@ const zhHans = {
selectModelAbilities: '选择模型能力',
visionAbility: '视觉能力',
functionCallAbility: '函数调用',
reasoningAbility: '思考能力',
reasoningLevel: '思考档位',
reasoningLevels: {
providerDefault: 'Provider 默认',
disabled: '关闭',
enabled: '开启',
minimal: '最低',
low: '低',
medium: '中',
high: '高',
xhigh: '极高',
max: '最大',
},
contextLength: '上下文窗口',
contextLengthPlaceholder: '未知',
contextLengthInvalid: '上下文窗口必须是正整数',
@@ -34,12 +34,26 @@ test('normalizes only single-line text fields in a dynamic form save snapshot',
{ name: 'multiline', type: 'text', default: '' },
{ name: 'string-list', type: 'array[string]', default: [] },
{ name: 'count', type: 'integer', default: 0 },
{
name: 'model',
type: 'model-fallback-selector',
default: { primary: '', fallbacks: [], reasoning: {} },
},
];
const values = {
'single-line': '\t hello world \n',
multiline: ' keep multiline whitespace \n',
'string-list': [' first ', ' second '],
count: 3,
model: {
primary: 'primary-model',
fallbacks: ['fallback-model'],
reasoning: {
'primary-model': 'high',
'fallback-model': 'provider_default',
'removed-model': 'medium',
},
},
};
assert.deepEqual(normalizeDynamicFormValuesForSave(specs, values), {
@@ -47,5 +61,12 @@ test('normalizes only single-line text fields in a dynamic form save snapshot',
multiline: ' keep multiline whitespace \n',
'string-list': [' first ', ' second '],
count: 3,
model: {
primary: 'primary-model',
fallbacks: ['fallback-model'],
reasoning: {
'primary-model': 'high',
},
},
});
});