feat(models): persist context metadata

The LiteLLM streaming requester only captured usage when a chunk had an
empty `choices` list. Many OpenAI-compatible gateways (e.g. new-api) and
providers send the final usage payload in a chunk that still carries an
empty-delta choice, so streamed calls always recorded 0 tokens in the
monitoring logs/dashboard (non-streaming worked).

- Capture stream usage whenever a chunk carries it, regardless of choices
- Add robust _normalize_usage (dict/obj shapes, derive missing total_tokens)
- Register litellm in bootutils/deps.py (was in pyproject only)
- Add MonitoringService.get_token_statistics + /monitoring/token-statistics
  endpoint: summary, per-model breakdown, token timeseries, and a
  zero-token-success data-quality signal
- Add TokenMonitoring dashboard tab (summary tiles, stacked token chart,
  per-model table) + i18n (en/zh)
- Regression tests for stream usage capture and usage normalization

Verified end-to-end against a real OpenAI-compatible endpoint with
gpt-5.5 and claude-opus-4-8: tokens now recorded non-zero for both
streaming and non-streaming paths.

.github/ISSUE_TEMPLATE/bug-report.yml

@@ -10,6 +10,15 @@ body:
       placeholder: 例如：v3.3.0、CentOS x64 Python 3.10.3、Docker
     validations:
       required: true
+  - type: dropdown
+    attributes:
+      label: 部署版本
+      description: 请选择您使用的 LangBot 部署版本。
+      options:
+        - 社区版
+        - 云服务
+    validations:
+      required: true
   - type: textarea
     attributes:
       label: 异常情况

.github/ISSUE_TEMPLATE/bug-report_en.yml

@@ -10,6 +10,15 @@ body:
       placeholder: "For example: v3.3.0, CentOS x64 Python 3.10.3, Docker"
     validations:
       required: true
+  - type: dropdown
+    attributes:
+      label: Deployment version
+      description: Please select the LangBot deployment version you are using.
+      options:
+        - Community Edition
+        - Cloud Service
+    validations:
+      required: true
   - type: textarea
     attributes:
       label: Exception

Dockerfile

@@ -14,10 +14,22 @@ COPY . .
 
 COPY --from=node /app/web/dist ./web/dist
 
-RUN apt update \
+RUN apt-get update \
-    && apt install gcc -y \
+    && apt-get install -y --no-install-recommends gcc ca-certificates curl gnupg \
+    # Install the Docker CLI (client only) so the optional langbot_box
+    # service can drive the mounted host Docker socket and create sandbox
+    # containers. The same image powers langbot / plugin_runtime / box; only
+    # box uses the client. Arch-aware via dpkg so multi-arch builds work.
+    && install -m 0755 -d /etc/apt/keyrings \
+    && curl -fsSL https://download.docker.com/linux/debian/gpg -o /etc/apt/keyrings/docker.asc \
+    && chmod a+r /etc/apt/keyrings/docker.asc \
+    && echo "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/debian $(. /etc/os-release && echo \"$VERSION_CODENAME\") stable" > /etc/apt/sources.list.d/docker.list \
+    && apt-get update \
+    && apt-get install -y --no-install-recommends docker-ce-cli \
     && python -m pip install --no-cache-dir uv \
     && uv sync \
+    && apt-get purge -y --auto-remove curl gnupg \
+    && rm -rf /var/lib/apt/lists/* \
     && touch /.dockerenv
 
 CMD [ "uv", "run", "--no-sync", "main.py" ]

docs/review/box-architecture.md

@@ -1,8 +1,9 @@
 # Box 系统架构深度分析
 
-> 更新日期: 2026-05-19
+> 更新日期: 2026-06-02
+> 状态更新: 自部署社区版已具备发布条件（box 可选、降级完善、无迁移欠债）；工具调用循环上限、配额遍历异步化、`host_path` 挂载白名单等已落地。剩余多租户 / 安全硬化项见 [SaaS 阻塞项清单](./box-issues.md)。
 > 分支: `feat/sandbox` (LangBot + langbot-plugin-sdk)
-> 相关文档: [问题清单](./box-issues.md) | [Session 作用域](./box-session-scope.md) | [Runtime 对比](./box-vs-plugin-runtime.md) | [测试覆盖](./box-test-coverage.md) | [toB 分析](./box-tob-analysis.md)
+> 相关文档: [SaaS 阻塞项](./box-issues.md) | [Session 作用域](./box-session-scope.md) | [Runtime 对比](./box-vs-plugin-runtime.md) | [测试覆盖](./box-test-coverage.md) | [toB 分析](./box-tob-analysis.md)
 
 ---
 
@@ -163,7 +164,7 @@ BoxService
 
 ### 2.4 policy.py (`pkg/box/policy.py`, 98 行) — 仍是死代码
 
-三层安全策略设计（`SandboxPolicy` / `ToolPolicy` / `ElevatedPolicy`），全项目无任何导入或调用。详见 [问题清单 #1](./box-issues.md)。
+三层安全策略设计（`SandboxPolicy` / `ToolPolicy` / `ElevatedPolicy`），全项目无任何导入或调用。详见 [SaaS 阻塞项 S2](./box-issues.md)。
 
 ### 2.5 SkillManager (`pkg/skill/manager.py`, 186 行)
 
@@ -364,7 +365,7 @@ GitHub 安装路径：HTTP 层（`api/http/service/skill.py`）先 `git clone`
 
 `validate_sandbox_security()`: 黑名单校验 host_path，阻止挂载 `/etc`/`/proc`/`/sys`/`/dev`/`/root`/`/boot` 及 Docker/Podman socket。
 
-**已知缺陷**: 根路径 `/` 未拦截，用户 home 目录未拦截，是 denylist 而非 allowlist 策略。详见 [问题清单 #5](./box-issues.md)。
+**已知缺陷**: 根路径 `/` 未拦截，用户 home 目录未拦截，是 denylist 而非 allowlist 策略。详见 [SaaS 阻塞项 S5](./box-issues.md)。
 
 ### 3.9 Errors (`box/errors.py`, 33 行)
 
@@ -512,7 +513,7 @@ box:
                           #  - skill 列表/读取保持只读可用
                           # BOX__ENABLED 环境变量可覆盖（统一约定）
     backend: 'local'      # 'local' (探测) / 'docker' / 'nsjail' / 'e2b'
-                          # BOX_BACKEND 环境变量优先级更高
+                          # 由 box.backend / BOX__BACKEND 选择后端
     runtime:
         endpoint: ''      # 外部 Runtime 的 WS 基地址 'ws://host:5410'
                           # 留空 = 本地自管 Runtime

docs/review/box-issues.md

@@ -1,157 +1,76 @@
-# Box 系统架构问题清单
+# Box 系统 — SaaS 发布前阻塞项
 
-> 更新日期: 2026-05-19
+> 更新日期: 2026-06-02
 > 分支: `feat/sandbox` (LangBot + langbot-plugin-sdk)
+> 相关文档: [架构分析](./box-architecture.md) | [Session 作用域](./box-session-scope.md) | [Runtime 对比](./box-vs-plugin-runtime.md) | [测试覆盖](./box-test-coverage.md) | [toB 分析](./box-tob-analysis.md)
+
+## 范围说明
+
+**自部署社区版已具备发布条件**：默认 stdio 模式、box 为可选项；box 关闭 / 不可用时后端、前端、工具、skill、stdio-MCP 均能干净降级（清晰报错、不崩溃）；配置向后兼容（旧 `data/config.yaml` 可直接启动）；无新增 ORM 模型、无迁移欠债；市场安装失败不会破坏实例。CI 全绿。
+
+本清单**只保留发布 SaaS / 多租户 / 公网暴露前必须处理的阻塞项**。社区版（可信、单运营者、内网）不受这些项阻塞——它们的风险面在"不可信调用方能直接触达 Box 控制面"或"多租户共享资源"的场景才成立。
+
+## 已解决（社区版发布前）
+
+| 项 | 处理 |
+|----|------|
+| 工具调用循环无上限 (原 #13) | `localagent.py` 增加 `MAX_TOOL_CALL_ROUNDS=128`，超限优雅终止（`cafef1a3`） |
+| 配额校验同步遍历阻塞事件循环 (原 #10) | `_enforce_workspace_quota` 改 async，工作区遍历走 `asyncio.to_thread`（`cafef1a3`） |
+| `host_path` 挂载白名单 (原 #3 的 LangBot 侧) | `pkg/box/service.py` `allowed_mount_roots` 白名单，空列表时拒绝一切宿主挂载 |
+| 重复的 `_is_path_under` (原 #12) | 已去重，仅保留一处定义 |
+| 重连 / 心跳 / Windows 兼容 / nsjail image 字段 / 前端 Box 状态接入 | 见上一轮 review 记录，均已合入 |
 
 ---
 
-## 已解决（自上一轮 review）
+## SaaS 阻塞项
 
-下列原 P0/P1 项在最新分支已被修复，仅作记录：
+### S1. Box 控制面无认证 — Critical
 
-| 原编号 | 问题 | 处理 commit / 说明 |
+- **位置**: SDK `box/server.py` — Action RPC WS (`/rpc/ws`) 与 managed-process relay (`/v1/sessions/{id}/managed-process/{pid}/ws`)
-|--------|------|---------------------|
+- **现状**: 两个 WS handler 在 `ws.prepare` 后直接服务，无任何 token / 鉴权；box 默认绑定 `0.0.0.0:5410`。任何能触达该端口者可发起 `EXEC`、创建 session、attach 任意 session 的 managed-process stdin/stdout、甚至 `SHUTDOWN`。LangBot→box 的 INIT 也未下发任何凭证。
-| #3 | Box 无重连机制 | `_make_connection_callback` 已接入 `runtime_disconnect_callback`；`BoxService._reconnect_loop()` 实现指数退避重连 (`2dfd9d5d`、`c6882cf`) |
+- **缓解现状**: 默认 `docker-compose.yaml` 的 `langbot_box` 未把 5410 发布到宿主（爆炸半径限于内网 bridge）；但 box 挂载了 `/var/run/docker.sock`，同网络的任意服务（含被攻破的插件）→ 宿主 root。若运营者把 5410 发布到宿主或独立以 `0.0.0.0` 起 box，则完全裸奔。
-| #4 | Box 无心跳 | `BoxRuntimeConnector._heartbeat_loop()`，间隔 20s（沿用 Plugin 模式） |
+- **要求**: INIT 时下发 token，两个 WS 路由按连接校验（query/header）。这是 SaaS 的**头号**阻塞项。
-| #10 | Windows 兼容 | connector 增加 Windows 分支 (subprocess + WS)，backend 适配 Windows Docker (`120817a`、`fafb7a4`) |
-| #12 | nsjail image 字段冲突 | `_assert_session_compatible()` 在不支持自定义镜像的 backend 跳过 image 字段 |
-| #22 | 前端无 Box UI | 监控页 `SystemStatusCards.tsx` 已接入 `/api/v1/box/status`；Skill 管理页接入了全部 skill API（sessions/errors API 仍未接入） |
 
----
+### S2. 无 exec 授权模型（policy.py 死代码） — High
 
-## P0 — 合并前建议修复
+- **位置**: LangBot `pkg/box/policy.py`（`SandboxPolicy` / `ToolPolicy` / `ElevatedPolicy` 全项目无引用）；`pkg/provider/tools/loaders/native.py`；`pkg/provider/tools/toolmgr.py`
+- **现状**: 原生工具（`exec/read/write/edit/glob/grep`）按"box 是否可用"全有或全无地暴露，**无 per-pipeline 的 exec 网关 / 工具白名单 / 沙箱模式 / 权限提升控制**。只要 box 可用，任何使用 local-agent + 函数调用模型的 pipeline 都能跑任意 shell。
+- **要求**: 接入 policy.py（或等价机制），按 pipeline 控制是否暴露 `exec`、可用工具白名单、沙箱网络/只读模式。
 
-### 1. policy.py 是死代码
+### S3. 会话资源无界（DoS） — High
 
-- **位置**: `pkg/box/policy.py` (98 行)
+- **#5 session 数量无上限**: SDK `box/runtime.py` `_get_or_create_session` 的 `_sessions` dict 无容量限制——可变 `session_id` 的恶意调用可无限创建容器，耗尽宿主 CPU/内存/PID/磁盘。
-- **现状**: `SandboxPolicy`、`ToolPolicy`、`ElevatedPolicy` 三个类已定义，但全项目无任何导入或调用
+- **#8 无定时回收**: 过期 session 仅在 `_get_or_create_session` 时机会性清理，无独立周期任务；一波创建后转静默会永久泄漏容器。
-- **影响**: 三层安全策略（沙箱模式 / 工具白名单 / 权限提升）完全未生效。当前实际策略仍是"Box 可用就暴露全部 6 个 native tool，不可用就全部隐藏"
+- **要求**: `max_sessions` 上限（拒绝或 LRU），加独立周期 reaper（如 60s）。
-- **建议**: 要么删除死代码，要么接入 NativeToolLoader 的工具暴露 / exec 调用链。如果短期不会接入，至少在 `pkg/box/__init__.py` 显式标注其状态
 
-### 2. WebSocket relay 无认证
+### S4. 工作区配额无内核级限制（TOCTOU） — Med-High
 
-- **位置**: SDK `box/server.py` — Action RPC 路径 `/rpc/ws` 与 managed-process relay `/v1/sessions/{id}/managed-process/{pid}/ws`
+- **位置**: LangBot `pkg/box/service.py` `_enforce_workspace_quota`（应用层 read-then-check）；SDK 侧 `workspace_quota_mb` 仅记录/透传，无 `--storage-opt size=` 等内核/FS 限额
-- **现状**: 任何能访问 5410 端口的客户端都可以连接，attach 任意 session 的 managed process stdin/stdout，或直接发起 EXEC
+- **现状**: 执行前后两次检查之间存在竞态窗口；单条命令（`dd`/`fallocate`）可在检查间隙撑爆磁盘，事后检查只能补救。
-- **影响**: 容器化 / Docker compose 部署中，若 Box runtime 端口外暴露，网络内的攻击者可直接控制沙箱
+- **要求**: Docker `--storage-opt size=` 做内核级限制，或 Redis 原子计数预留式配额。
-- **建议**: 至少加 token 认证（INIT 时下发，WS 连接 query string 或 header 校验）；多 process 后 attach 面更大，更不能裸奔
 
-### 3. security.py 根路径未拦截
+### S5. 挂载校验缺口 — Med-High
 
-- **位置**: SDK `box/security.py` `BLOCKED_HOST_PATHS_POSIX`
+- **位置**: SDK `box/security.py` `_BLOCKED_HOST_PATHS_POSIX`；`box/backend.py` 的 `extra_mounts` 处理
-- **现状**: 黑名单中没有 `/`，`host_path="/"` 可通过校验并挂载整个主机文件系统；用户 home 目录、`/var` 等也未拦截
+- **现状**: ① SDK 黑名单仍不含 `/`（前缀匹配，`host_path="/"` 可通过，挂载整个宿主 fs）；用户 home、`/usr`、`/opt`、`/tmp` 也未拦截。② `validate_sandbox_security` 只校验 `spec.host_path`，**从不遍历 `spec.extra_mounts`**——LangBot 侧 `allowed_mount_roots` 也只校验 `host_path`。当前 `extra_mounts` 仅由 `build_skill_extra_mounts` 内部填充（agent 不可达），但缺乏纵深防御：一旦 S1 的无认证 RPC 被触达，extra_mounts 可挂任意宿主路径，两层都不拦。
-- **建议**: 将 `/` 加入黑名单，或改用白名单策略与 LangBot 侧 `allowed_mount_roots` 二次拦截
+- **要求**: SDK 黑名单加入 `/`（或改白名单）；`extra_mounts` 在 SDK 与 LangBot 两侧都纳入挂载校验。
 
-### 4. INIT 与 backend 初始化的竞态
+### S6. 容器加固缺失 — Med
 
-- **位置**: SDK `box/runtime.py` `init()` 在握手后才下发实际配置；`backend` 在 INIT 之前可能已经按默认值实例化
+- **位置**: SDK `box/backend.py` 的 `docker run` 组装
-- **现状**: commit `5029d9c` 修复了 "init config before backend reuse" 的部分场景，但 backend 重新实例化时若有正在执行的 session，可能命中旧 backend
+- **现状**: 未设置 `--cap-drop=ALL`、`--security-opt=no-new-privileges`、非 root `--user`；叠加挂载 docker.sock，逃逸面偏大。
-- **建议**: 整理 init/handshake 顺序——要么 INIT 完成前不接受任何业务 action，要么允许 backend 配置变更时显式清理现有 session
+- **要求**: 默认加上上述加固 flag（需回归常用 skill 不被破坏）。
 
----
+### S7. 全局锁内执行慢操作（扩展性） — Med
 
-## P1 — 合并后优先跟进
+- **位置**: SDK `box/runtime.py` `_get_or_create_session`：`self._lock` 持有期间调用 `backend.start_session()`（`docker run` / nsjail 启动 / E2B `Sandbox.create`）
+- **影响**: 冷启动（镜像拉取数秒、E2B >1s）期间串行阻塞所有并发请求——多租户负载下整个 Box runtime 停顿。降级表现是延迟而非失败。
+- **要求**: 锁内只做状态检查与注册，容器创建移到锁外。
 
-### 5. Session 数量无上限
+### S8. 其他硬化 / 跟进 — Low
 
-- **位置**: SDK `box/runtime.py` `_get_or_create_session()`
+- **#9** SDK `box/server.py` 直接读 `runtime._sessions` 私有字段、绕过锁，并发下可能读到不一致状态——应加公共访问方法。
-- **现状**: `_sessions` dict 无容量限制，恶意或异常调用可创建无限 session
+- **#16** `pkg/provider/tools/toolmgr.py` `execute_func_call` 按优先级分发，plugin/MCP 若有同名 `exec/read/write/...` 工具会被静默遮蔽——应加命名空间或冲突告警。
-- **建议**: 加 `max_sessions` 配置项，达到上限时拒绝新建或按 LRU 清理
+- **#4** SDK `box/runtime.py` INIT/handshake 与 backend 实例化的残留竞态（仅"纯远程 WS box 先启动、LangBot 后连"场景成立；stdio/compose 路径下 config 经 env 在 spawn 时已就位，无竞态）——应在 INIT 完成前拒绝业务 action。
-
+- **#11** `extra_mounts` 在容器创建时固定（SDK `runtime.py` 兼容性检查不含 extra_mounts）；长生命周期共享 session 后续新激活的 skill 不会挂上（当前缓解：创建时挂上 pipeline 绑定的全部 skill）——动态绑定场景需销毁重建或文档说明。
-### 6. Quota 检查存在 TOCTOU
+- **#21** 集成测试未进 CI：容器实际执行、E2B 真机、managed-process WS attach 仅本地可跑。安全关键路径缺自动化覆盖——SaaS 前建议加 Docker-in-Docker CI stage 或合并前手动 checklist。
-
-- **位置**: `pkg/box/service.py` `_enforce_workspace_quota()`
-- **现状**: 应用层先读磁盘大小再执行命令，两步之间有竞态窗口
-- **建议**: 短期用 Docker `--storage-opt size=` 做内核级限制；长期用 Redis 原子计数器做预留式配额
-
-### 7. 全局锁持有期间执行慢操作
-
-- **位置**: SDK `box/runtime.py` `_get_or_create_session()` — `self._lock` 下调用 `backend.start_session()` (即 `docker run` / `nsjail` 进程启动 / E2B `Sandbox.create`)
-- **影响**: `docker run` 可能耗时数秒（含镜像拉取）、E2B 冷启动通常 > 1s，期间阻塞所有并发请求
-- **建议**: 在 `_lock` 下仅做状态检查和 session 注册，容器创建在锁外执行
-
-### 8. Session 清理是机会性的
-
-- **位置**: SDK `box/runtime.py` `_reap_expired_sessions_locked()` — 仅在 `_get_or_create_session()` 时调用
-- **影响**: 如果长时间无新 session 请求，过期 session（含容器）不会被清理
-- **建议**: 加一个独立的 `asyncio.create_task` 定时清理（如每 60s 一次）
-
-### 9. server.py 直接访问 runtime 私有字段
-
-- **位置**: SDK `box/server.py` — managed-process WS handler 直接读 `runtime._sessions`
-- **影响**: 绕过锁和封装，在并发场景下可能读到不一致状态
-- **建议**: 在 BoxRuntime 上增加公共方法（如 `get_session_managed_process(session_id, process_id)`）
-
-### 10. workspace quota 检查阻塞事件循环
-
-- **位置**: `pkg/box/service.py` `_get_workspace_size_bytes()` — 使用同步 `os.scandir` 递归遍历
-- **影响**: 大工作区可能阻塞 asyncio event loop
-- **建议**: 用 `asyncio.to_thread()` 包装，或用 `aiofiles` 异步扫描
-
-### 11. extra_mounts 一旦容器创建即固定
-
-- **位置**: SDK `box/runtime.py` 的兼容性检查；`pkg/box/service.py:build_skill_extra_mounts()`
-- **现状**: Skill 挂载在容器创建时一次性写入；同一 session 后续 pipeline 切换 skill 列表时，新挂载不会生效（除非销毁重建）
-- **影响**: 用户长时间共享 session 的场景下，新激活的 skill 可能挂不上
-- **建议**: 要么在创建时把 pipeline 绑定的所有 skill 都挂上（实际现状）+ 写入文档；要么变更挂载时强制销毁 session 重建（已被 commit `5029d9c` 部分覆盖，需校验）
-
----
-
-## P2 — 后续迭代
-
-### 12. 重复的 `_is_path_under` 函数
-
-- **位置**: `pkg/box/service.py` 行 30 附近 — 同名函数定义两次
-- **建议**: 删除重复定义
-
-### 13. localagent.py 工具循环无迭代上限
-
-- **位置**: `pkg/provider/runners/localagent.py` `while pending_tool_calls` 循环
-- **影响**: 恶意或混乱的 LLM 可无限产生 tool call，消耗资源
-- **建议**: 加 `max_tool_iterations` 配置项（如默认 50 次）
-
-### 14. localagent.py 中的死代码
-
-- **位置**: `pkg/provider/runners/localagent.py:29-35` 附近 — 旧命名 `SANDBOX_EXEC_TOOL_NAME` 和 `SANDBOX_EXEC_SYSTEM_GUIDANCE`
-- **现状**: 旧命名方案的遗留常量，从未被引用（实际使用 `EXEC_TOOL_NAME` from native.py）
-- **建议**: 删除
-
-### 15. @loader_class 装饰器未使用
-
-- **位置**: `pkg/provider/tools/loader.py` — `preregistered_loaders` 列表和 `@loader_class` 装饰器
-- **现状**: 各 loader 的 `@loader_class` 多数被注释掉，ToolManager 手动实例化所有 loader
-- **建议**: 要么启用装饰器自动注册，要么删除未用的机制
-
-### 16. 工具名冲突风险
-
-- **位置**: `pkg/provider/tools/toolmgr.py` `execute_func_call()` — 按优先级 native → plugin → mcp → skill → skill_authoring 分发
-- **影响**: 如果 plugin 或 MCP 有名为 `exec`/`read`/`write`/`edit`/`glob`/`grep`/`activate` 的工具，会被前序 loader 静默遮蔽
-- **建议**: 加命名空间前缀或冲突检测告警
-
-### 17. client.py 反序列化不一致
-
-- **位置**: SDK `box/client.py` — `execute()` 与其他方法对返回值的反序列化方式不统一（部分手动构造 model，部分用 `model_validate`）
-- **建议**: 统一使用 `model_validate`
-
-### 18. 错误类型还原基于字符串前缀匹配
-
-- **位置**: SDK `box/client.py` `_translate_action_error()`
-- **影响**: 如果 server 端错误消息格式变化，client 会回退到通用 `BoxError`，丢失类型信息
-- **建议**: 在 ActionResponse 中增加结构化的错误类型字段（如 `error_code` 枚举）
-
-### 19. 前端只用到了 status
-
-- **位置**: `web/src/app/home/monitoring/...` 已接入 `/api/v1/box/status`
-- **现状**: `/api/v1/box/sessions` 与 `/api/v1/box/errors` 后端可用、前端未消费
-- **建议**: 在监控页或独立 Box 详情页展示活跃 session 列表与最近错误，提升运维体感
-
-### 20. skill_store 测试覆盖偏薄
-
-- **位置**: SDK `tests/box/test_skill_store.py` 仅 88 行
-- **现状**: 相对 `skill_store.py` 的 647 行实现，单测覆盖度不够；GitHub 安装路径、`source_subdir` / `target_suffix` 组合、损坏 zip 的错误处理等场景未覆盖
-- **建议**: 至少补到核心 path 覆盖（preview/install/list/file CRUD 各 2~3 个 case）
-
-### 21. 集成测试未进 CI
-
-- **位置**: LangBot `tests/integration_tests/box/test_box_integration.py`、`test_box_mcp_integration.py`，SDK 端的 E2B 真机测试
-- **现状**: 容器实际执行、E2B 真实 sandbox、Managed process WS attach 均仅本地能跑
-- **建议**: 加一个可选的 Docker-in-Docker CI stage，或在合并前手动跑 checklist

docs/review/box-session-scope.md

@@ -1,6 +1,7 @@
 # Box Session Scope Design
 
-> Date: 2026-04-18 (last reviewed 2026-05-19)
+> Date: 2026-04-18 (last reviewed 2026-06-02)
+> Status (2026-06-02): the self-hosted community edition is release-ready (box optional, clean degradation, no migration debt). Tool-call loop cap, async quota scan, and the host_path mount allowlist have landed. Remaining multi-tenant / security hardening is tracked in [box-issues.md](./box-issues.md).
 > Branch: `feat/sandbox` (LangBot + langbot-plugin-sdk)
 > Related: [Box Architecture](./box-architecture.md) | [Box vs Plugin Runtime](./box-vs-plugin-runtime.md)
 

docs/review/box-test-coverage.md

@@ -1,6 +1,7 @@
 # Box 系统测试覆盖分析
 
-> 更新日期: 2026-05-19
+> 更新日期: 2026-06-02
+> 状态更新: 自部署社区版已具备发布条件（box 可选、降级完善、无迁移欠债）；工具调用循环上限、配额遍历异步化、`host_path` 挂载白名单等已落地。剩余多租户 / 安全硬化项见 [SaaS 阻塞项清单](./box-issues.md)。
 > 分支: `feat/sandbox` (LangBot + langbot-plugin-sdk)
 
 ---

docs/review/box-tob-analysis.md

@@ -1,6 +1,7 @@
 # Box 系统 toB 商业化分析
 
-> 更新日期: 2026-05-19
+> 更新日期: 2026-06-02
+> 状态更新: 自部署社区版已具备发布条件（box 可选、降级完善、无迁移欠债）；工具调用循环上限、配额遍历异步化、`host_path` 挂载白名单等已落地。剩余多租户 / 安全硬化项见 [SaaS 阻塞项清单](./box-issues.md)。
 > 分支: `feat/sandbox` (LangBot + langbot-plugin-sdk)
 
 ---

docs/review/box-vs-plugin-runtime.md

@@ -1,6 +1,7 @@
 # Box Runtime vs Plugin Runtime: 连接架构对比
 
-> 更新日期: 2026-05-19
+> 更新日期: 2026-06-02
+> 状态更新: 自部署社区版已具备发布条件（box 可选、降级完善、无迁移欠债）；工具调用循环上限、配额遍历异步化、`host_path` 挂载白名单等已落地。剩余多租户 / 安全硬化项见 [SaaS 阻塞项清单](./box-issues.md)。
 > 分支: `feat/sandbox` (LangBot + langbot-plugin-sdk)
 
 ---

pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "langbot"
-version = "4.10.0-beta.1"
+version = "4.10.0"
 description = "Production-grade platform for building agentic IM bots"
 readme = "README.md"
 license-files = ["LICENSE"]
@@ -70,7 +70,7 @@ dependencies = [
     "chromadb>=1.0.0,<2.0.0",
     "qdrant-client (>=1.15.1,<2.0.0)",
     "pyseekdb==1.1.0.post3",
-    "langbot-plugin==0.4.0b1",
+    "langbot-plugin==0.4.1",
     "asyncpg>=0.30.0",
     "line-bot-sdk>=3.19.0",
     "matrix-nio>=0.25.2",
@@ -79,6 +79,7 @@ dependencies = [
     "pymilvus>=2.6.4",
     "pgvector>=0.4.1",
     "botocore>=1.42.39",
+    "litellm>=1.0.0",
 ]
 keywords = [
     "bot",

src/langbot/__init__.py

@@ -1,3 +1,3 @@
 """LangBot - Production-grade platform for building agentic IM bots"""
 
-__version__ = '4.10.0-beta.1'
+__version__ = '4.10.0'

src/langbot/pkg/api/http/controller/groups/monitoring.py

@@ -46,6 +46,30 @@ class MonitoringRouterGroup(group.RouterGroup):
 
             return self.success(data=metrics)
 
+        @self.route('/token-statistics', methods=['GET'], auth_type=group.AuthType.USER_TOKEN)
+        async def get_token_statistics() -> str:
+            """Get detailed token usage statistics (summary, per-model, timeseries)."""
+            bot_ids = quart.request.args.getlist('botId')
+            pipeline_ids = quart.request.args.getlist('pipelineId')
+            start_time_str = quart.request.args.get('startTime')
+            end_time_str = quart.request.args.get('endTime')
+            bucket = quart.request.args.get('bucket', 'hour')
+            if bucket not in ('hour', 'day'):
+                bucket = 'hour'
+
+            start_time = parse_iso_datetime(start_time_str)
+            end_time = parse_iso_datetime(end_time_str)
+
+            stats = await self.ap.monitoring_service.get_token_statistics(
+                bot_ids=bot_ids if bot_ids else None,
+                pipeline_ids=pipeline_ids if pipeline_ids else None,
+                start_time=start_time,
+                end_time=end_time,
+                bucket=bucket,
+            )
+
+            return self.success(data=stats)
+
         @self.route('/messages', methods=['GET'], auth_type=group.AuthType.USER_TOKEN)
         async def get_messages() -> str:
             """Get message logs"""

src/langbot/pkg/api/http/controller/groups/pipelines/websocket_chat.py

@@ -43,8 +43,12 @@ class WebSocketChatRouterGroup(group.RouterGroup):
                     await quart.websocket.send(json.dumps({'type': 'error', 'message': 'WebSocket adapter not found'}))
                     return
 
-                # Find the owning bot for this pipeline (e.g. a web_page_bot)
+                # Dashboard pipeline-debug sessions must always run under the
-                owner_bot = self._find_owner_bot(pipeline_uuid)
+                # built-in websocket_proxy_bot identity. We deliberately do NOT
+                # resolve a web_page_bot owner here — even if one is bound to
+                # the same pipeline, debug requests must not be attributed to
+                # it. The embed widget path (`/api/v1/embed/<bot>/ws/connect`)
+                # is the one that carries the page-bot identity.
 
                 # 注册连接
                 connection = await ws_connection_manager.add_connection(
@@ -73,7 +77,7 @@ class WebSocketChatRouterGroup(group.RouterGroup):
                 )
 
                 # 创建接收和发送任务
-                receive_task = asyncio.create_task(self._handle_receive(connection, websocket_adapter, owner_bot))
+                receive_task = asyncio.create_task(self._handle_receive(connection, websocket_adapter))
                 send_task = asyncio.create_task(self._handle_send(connection))
 
                 # 等待任务完成
@@ -181,14 +185,7 @@ class WebSocketChatRouterGroup(group.RouterGroup):
             except Exception as e:
                 return self.http_status(500, -1, f'Internal server error: {str(e)}')
 
-    def _find_owner_bot(self, pipeline_uuid: str):
+    async def _handle_receive(self, connection, websocket_adapter):
-        """Find a user-created bot (e.g. web_page_bot) that owns this pipeline."""
-        for bot in self.ap.platform_mgr.bots:
-            if bot.bot_entity.adapter == 'web_page_bot' and bot.bot_entity.use_pipeline_uuid == pipeline_uuid:
-                return bot
-        return None
-
-    async def _handle_receive(self, connection, websocket_adapter, owner_bot=None):
         """处理接收消息的任务"""
         try:
             while connection.is_active:
@@ -213,7 +210,10 @@ class WebSocketChatRouterGroup(group.RouterGroup):
                         logger.debug(f'收到消息: {data} from {connection.connection_id}')
 
                         # 处理消息（不等待响应，响应会通过broadcast异步发送）
-                        await websocket_adapter.handle_websocket_message(connection, data, owner_bot=owner_bot)
+                        # owner_bot is intentionally NOT passed: the dashboard
+                        # debug WebSocket must always run under the proxy bot,
+                        # never under a coincidentally-bound web_page_bot.
+                        await websocket_adapter.handle_websocket_message(connection, data)
 
                     elif message_type == 'disconnect':
                         # 客户端主动断开

src/langbot/pkg/api/http/controller/groups/platform/adapters.py

@@ -179,8 +179,6 @@ class AdaptersRouterGroup(group.RouterGroup):
             """Start WeChat QR code login. Returns session_id + QR code data URL."""
             import uuid
             import time
-            import io
-            import base64
 
             from langbot.libs.openclaw_weixin_api.client import OpenClawWeixinClient, DEFAULT_BASE_URL
 
@@ -208,60 +206,32 @@ class AdaptersRouterGroup(group.RouterGroup):
 
             async def run_login():
                 try:
-                    import qrcode as qr_lib
 
-                    for _attempt in range(3):
+                    def on_qrcode(qr_data_url: str, _qr_url: str):
-                        qr_resp = await client.fetch_qrcode()
+                        def _update():
-                        if not qr_resp.qrcode or not qr_resp.qrcode_img_content:
+                            session['qr_data_url'] = qr_data_url
-                            raise Exception('Failed to get QR code from server')
+                            session['expire_at'] = time.time() + 180
-
-                        # Generate QR code image locally
-                        qr = qr_lib.QRCode(error_correction=qr_lib.constants.ERROR_CORRECT_L)
-                        qr.add_data(qr_resp.qrcode_img_content)
-                        qr.make(fit=True)
-                        img = qr.make_image(fill_color='black', back_color='white')
-                        buf = io.BytesIO()
-                        img.save(buf, format='PNG')
-                        b64 = base64.b64encode(buf.getvalue()).decode('utf-8')
-                        data_url = f'data:image/png;base64,{b64}'
-
-                        def _update_qr():
-                            session['qr_data_url'] = data_url
-                            session['expire_at'] = time.time() + 480  # 8 minutes
                             session['status'] = 'waiting'
 
-                        loop.call_soon_threadsafe(_update_qr)
+                        loop.call_soon_threadsafe(_update)
-
-                        # Poll for scan status
-                        deadline = loop.time() + 180
-                        while loop.time() < deadline:
-                            try:
-                                status_resp = await client.poll_qrcode_status(qr_resp.qrcode)
-                            except Exception:
-                                await asyncio.sleep(2)
-                                continue
-
-                            if status_resp.status == 'confirmed' and status_resp.bot_token:
-                                session['status'] = 'success'
-                                session['token'] = status_resp.bot_token
-                                session['base_url'] = status_resp.baseurl or client.base_url
-                                session['account_id'] = status_resp.ilink_bot_id or ''
-                                return
-
-                            if status_resp.status == 'expired':
-                                break  # retry with new QR code
-
-                            await asyncio.sleep(1)
-                        else:
-                            pass  # timeout, retry
-
-                    # All retries exhausted
-                    session['status'] = 'error'
-                    session['error'] = 'QR code login failed: max retries exceeded'
 
+                    result = await client.login(
+                        max_retries=1,
+                        poll_timeout_ms=180_000,
+                        on_qrcode=on_qrcode,
+                    )
+                    session['status'] = 'success'
+                    session['token'] = result.token
+                    session['base_url'] = result.base_url
+                    session['account_id'] = result.account_id
                 except Exception as e:
-                    session['status'] = 'error'
+                    error_message = str(e)
-                    session['error'] = str(e)
+                    if 'expired' in error_message.lower() or 'max retries exceeded' in error_message.lower():
+                        session['status'] = 'expired'
+                        session['error'] = 'QR code expired'
+                    else:
+                        session['status'] = 'error'
+                        session['error'] = error_message
                 finally:
                     await client.close()
 
@@ -295,7 +265,11 @@ class AdaptersRouterGroup(group.RouterGroup):
             if not session:
                 return self.http_status(404, -1, 'Session not found')
 
-            data = {'status': session['status']}
+            data = {
+                'status': session['status'],
+                'qr_data_url': session['qr_data_url'],
+                'expire_at': session['expire_at'],
+            }
 
             if session['status'] == 'success':
                 data['token'] = session['token']
@@ -305,6 +279,9 @@ class AdaptersRouterGroup(group.RouterGroup):
             elif session['status'] == 'error':
                 data['error'] = session['error']
                 _weixin_login_sessions.pop(session_id, None)
+            elif session['status'] == 'expired':
+                data['error'] = session['error']
+                _weixin_login_sessions.pop(session_id, None)
 
             return self.success(data=data)
 

src/langbot/pkg/api/http/service/monitoring.py

@@ -472,6 +472,179 @@ class MonitoringService:
             'active_sessions': active_sessions,
         }
 
+    async def get_token_statistics(
+        self,
+        bot_ids: list[str] | None = None,
+        pipeline_ids: list[str] | None = None,
+        start_time: datetime.datetime | None = None,
+        end_time: datetime.datetime | None = None,
+        bucket: str = 'hour',
+    ) -> dict:
+        """Get detailed token usage statistics for production observability.
+
+        Returns:
+        - summary: aggregate token counters and call/latency stats over the window
+        - by_model: per-model token + call breakdown (sorted by total tokens desc)
+        - timeseries: token usage bucketed by `bucket` ('hour' or 'day')
+
+        Only successful LLM calls are counted toward token totals; error calls are
+        reported separately so a spike in failures is visible without polluting
+        token accounting.
+        """
+        LLMCall = persistence_monitoring.MonitoringLLMCall
+
+        conditions = []
+        if bot_ids:
+            conditions.append(LLMCall.bot_id.in_(bot_ids))
+        if pipeline_ids:
+            conditions.append(LLMCall.pipeline_id.in_(pipeline_ids))
+        if start_time:
+            conditions.append(LLMCall.timestamp >= start_time)
+        if end_time:
+            conditions.append(LLMCall.timestamp <= end_time)
+
+        def _apply(query):
+            if conditions:
+                query = query.where(sqlalchemy.and_(*conditions))
+            return query
+
+        # ---- Summary aggregates ----
+        summary_query = _apply(
+            sqlalchemy.select(
+                sqlalchemy.func.count(LLMCall.id),
+                sqlalchemy.func.coalesce(sqlalchemy.func.sum(LLMCall.input_tokens), 0),
+                sqlalchemy.func.coalesce(sqlalchemy.func.sum(LLMCall.output_tokens), 0),
+                sqlalchemy.func.coalesce(sqlalchemy.func.sum(LLMCall.total_tokens), 0),
+                sqlalchemy.func.coalesce(sqlalchemy.func.sum(LLMCall.duration), 0),
+                sqlalchemy.func.coalesce(sqlalchemy.func.sum(LLMCall.cost), 0.0),
+                sqlalchemy.func.sum(sqlalchemy.case((LLMCall.status == 'success', 1), else_=0)),
+                sqlalchemy.func.sum(sqlalchemy.case((LLMCall.status == 'error', 1), else_=0)),
+                # Count of successful calls that nonetheless recorded zero tokens —
+                # a data-quality signal that usage reporting may be broken upstream.
+                sqlalchemy.func.sum(
+                    sqlalchemy.case(
+                        (sqlalchemy.and_(LLMCall.status == 'success', LLMCall.total_tokens == 0), 1),
+                        else_=0,
+                    )
+                ),
+            )
+        )
+        summary_result = await self.ap.persistence_mgr.execute_async(summary_query)
+        row = summary_result.first()
+        (
+            total_calls,
+            total_input_tokens,
+            total_output_tokens,
+            total_tokens,
+            total_duration,
+            total_cost,
+            success_calls,
+            error_calls,
+            zero_token_success_calls,
+        ) = row if row else (0, 0, 0, 0, 0, 0.0, 0, 0, 0)
+
+        total_calls = total_calls or 0
+        success_calls = success_calls or 0
+        error_calls = error_calls or 0
+        zero_token_success_calls = zero_token_success_calls or 0
+
+        summary = {
+            'total_calls': total_calls,
+            'success_calls': success_calls,
+            'error_calls': error_calls,
+            'total_input_tokens': int(total_input_tokens or 0),
+            'total_output_tokens': int(total_output_tokens or 0),
+            'total_tokens': int(total_tokens or 0),
+            'total_cost': round(float(total_cost or 0.0), 6),
+            'avg_tokens_per_call': int((total_tokens or 0) / total_calls) if total_calls > 0 else 0,
+            'avg_duration_ms': int((total_duration or 0) / total_calls) if total_calls > 0 else 0,
+            'avg_tokens_per_second': round((total_output_tokens or 0) / (total_duration / 1000), 2)
+            if total_duration and total_duration > 0
+            else 0,
+            'zero_token_success_calls': zero_token_success_calls,
+        }
+
+        # ---- Per-model breakdown ----
+        by_model_query = _apply(
+            sqlalchemy.select(
+                LLMCall.model_name,
+                sqlalchemy.func.count(LLMCall.id),
+                sqlalchemy.func.coalesce(sqlalchemy.func.sum(LLMCall.input_tokens), 0),
+                sqlalchemy.func.coalesce(sqlalchemy.func.sum(LLMCall.output_tokens), 0),
+                sqlalchemy.func.coalesce(sqlalchemy.func.sum(LLMCall.total_tokens), 0),
+                sqlalchemy.func.coalesce(sqlalchemy.func.sum(LLMCall.duration), 0),
+                sqlalchemy.func.coalesce(sqlalchemy.func.sum(LLMCall.cost), 0.0),
+                sqlalchemy.func.sum(sqlalchemy.case((LLMCall.status == 'error', 1), else_=0)),
+            ).group_by(LLMCall.model_name)
+        )
+        by_model_result = await self.ap.persistence_mgr.execute_async(by_model_query)
+        by_model = []
+        for mrow in by_model_result.all():
+            (
+                model_name,
+                m_calls,
+                m_in,
+                m_out,
+                m_total,
+                m_duration,
+                m_cost,
+                m_errors,
+            ) = mrow
+            m_calls = m_calls or 0
+            by_model.append(
+                {
+                    'model_name': model_name,
+                    'calls': m_calls,
+                    'error_calls': m_errors or 0,
+                    'input_tokens': int(m_in or 0),
+                    'output_tokens': int(m_out or 0),
+                    'total_tokens': int(m_total or 0),
+                    'cost': round(float(m_cost or 0.0), 6),
+                    'avg_tokens_per_call': int((m_total or 0) / m_calls) if m_calls > 0 else 0,
+                    'avg_duration_ms': int((m_duration or 0) / m_calls) if m_calls > 0 else 0,
+                }
+            )
+        by_model.sort(key=lambda x: x['total_tokens'], reverse=True)
+
+        # ---- Time-bucketed series ----
+        # Use a DB-agnostic bucketing approach: fetch (timestamp, tokens) rows and
+        # aggregate in Python. The window is bounded by the time filter, so this is
+        # cheap for typical dashboard ranges (hours/days).
+        series_query = _apply(
+            sqlalchemy.select(
+                LLMCall.timestamp,
+                LLMCall.input_tokens,
+                LLMCall.output_tokens,
+                LLMCall.total_tokens,
+            ).order_by(LLMCall.timestamp.asc())
+        )
+        series_result = await self.ap.persistence_mgr.execute_async(series_query)
+
+        bucket_fmt = '%Y-%m-%d %H:00' if bucket == 'hour' else '%Y-%m-%d'
+        buckets: dict[str, dict] = {}
+        for srow in series_result.all():
+            ts, s_in, s_out, s_total = srow
+            if ts is None:
+                continue
+            key = ts.strftime(bucket_fmt)
+            b = buckets.setdefault(
+                key,
+                {'bucket': key, 'input_tokens': 0, 'output_tokens': 0, 'total_tokens': 0, 'calls': 0},
+            )
+            b['input_tokens'] += int(s_in or 0)
+            b['output_tokens'] += int(s_out or 0)
+            b['total_tokens'] += int(s_total or 0)
+            b['calls'] += 1
+
+        timeseries = [buckets[k] for k in sorted(buckets.keys())]
+
+        return {
+            'summary': summary,
+            'by_model': by_model,
+            'timeseries': timeseries,
+            'bucket': bucket,
+        }
+
     async def get_messages(
         self,
         bot_ids: list[str] | None = None,

src/langbot/pkg/box/service.py

@@ -168,7 +168,7 @@ class BoxService:
             f'spec={json.dumps(self._summarize_spec(spec), ensure_ascii=False)}'
         )
         try:
-            self._enforce_workspace_quota(spec, phase='before execution')
+            await self._enforce_workspace_quota(spec, phase='before execution')
         except BoxError as exc:
             self._record_error(exc, query)
             raise
@@ -178,7 +178,7 @@ class BoxService:
             self._record_error(exc, query)
             raise
         try:
-            self._enforce_workspace_quota(spec, phase='after execution')
+            await self._enforce_workspace_quota(spec, phase='after execution')
         except BoxError as exc:
             await self._cleanup_exceeded_session(spec)
             self._record_error(exc, query)
@@ -683,7 +683,7 @@ class BoxService:
         _walk(root)
         return total
 
-    def _enforce_workspace_quota(self, spec: BoxSpec, *, phase: str) -> None:
+    async def _enforce_workspace_quota(self, spec: BoxSpec, *, phase: str) -> None:
         if spec.host_path is None or spec.workspace_quota_mb <= 0:
             return
 
@@ -691,7 +691,10 @@ class BoxService:
         if not os.path.isdir(host_path):
             return
 
-        used_bytes = self._get_workspace_size_bytes(host_path)
+        # Walk the workspace off the event loop — this runs on every
+        # quota-enforced exec, and a large tree would otherwise block the whole
+        # asyncio runtime (all bots/pipelines) for the duration of the scan.
+        used_bytes = await asyncio.to_thread(self._get_workspace_size_bytes, host_path)
         limit_bytes = spec.workspace_quota_mb * _MIB
         if used_bytes <= limit_bytes:
             return

src/langbot/pkg/core/bootutils/deps.py

@@ -42,6 +42,7 @@ required_deps = {
     'telegramify_markdown': 'telegramify-markdown',
     'slack_sdk': 'slack_sdk',
     'asyncpg': 'asyncpg',
+    'litellm': 'litellm',
 }
 
 

src/langbot/pkg/entity/persistence/model.py

@@ -31,6 +31,7 @@ class LLMModel(Base):
     name = sqlalchemy.Column(sqlalchemy.String(255), nullable=False)
     provider_uuid = sqlalchemy.Column(sqlalchemy.String(255), nullable=False)
     abilities = sqlalchemy.Column(sqlalchemy.JSON, nullable=False, default=[])
+    context_length = sqlalchemy.Column(sqlalchemy.Integer, nullable=True)
     extra_args = sqlalchemy.Column(sqlalchemy.JSON, nullable=False, default={})
     prefered_ranking = sqlalchemy.Column(sqlalchemy.Integer, nullable=False, default=0)
     created_at = sqlalchemy.Column(sqlalchemy.DateTime, nullable=False, server_default=sqlalchemy.func.now())

src/langbot/pkg/persistence/alembic/versions/0004_add_llm_model_context_length.py

@@ -0,0 +1,30 @@
+"""add llm model context length
+
+Revision ID: 0004_add_llm_model_context_length
+Revises: 0003_add_rerank_models
+Create Date: 2026-06-07
+"""
+
+import sqlalchemy as sa
+from alembic import op
+
+revision = '0004_add_llm_model_context_length'
+down_revision = '0003_add_rerank_models'
+branch_labels = None
+depends_on = None
+
+
+def upgrade() -> None:
+    conn = op.get_bind()
+    inspector = sa.inspect(conn)
+    columns = {column['name'] for column in inspector.get_columns('llm_models')}
+    if 'context_length' not in columns:
+        op.add_column('llm_models', sa.Column('context_length', sa.Integer(), nullable=True))
+
+
+def downgrade() -> None:
+    conn = op.get_bind()
+    inspector = sa.inspect(conn)
+    columns = {column['name'] for column in inspector.get_columns('llm_models')}
+    if 'context_length' in columns:
+        op.drop_column('llm_models', 'context_length')

src/langbot/pkg/persistence/migrations/dbm026_llm_model_context_length.py

@@ -0,0 +1,42 @@
+import sqlalchemy
+from .. import migration
+
+
+@migration.migration_class(26)
+class DBMigrateLLMModelContextLength(migration.DBMigration):
+    """Add context_length column to LLM models"""
+
+    async def upgrade(self):
+        columns = await self._get_columns('llm_models')
+        if 'context_length' not in columns:
+            await self.ap.persistence_mgr.execute_async(
+                sqlalchemy.text('ALTER TABLE llm_models ADD COLUMN context_length INTEGER')
+            )
+
+    async def downgrade(self):
+        columns = await self._get_columns('llm_models')
+        if 'context_length' not in columns:
+            return
+
+        if self.ap.persistence_mgr.db.name == 'postgresql':
+            await self.ap.persistence_mgr.execute_async(
+                sqlalchemy.text('ALTER TABLE llm_models DROP COLUMN IF EXISTS context_length')
+            )
+        else:
+            await self.ap.persistence_mgr.execute_async(
+                sqlalchemy.text('ALTER TABLE llm_models DROP COLUMN context_length')
+            )
+
+    async def _get_columns(self, table_name: str) -> set[str]:
+        if self.ap.persistence_mgr.db.name == 'postgresql':
+            result = await self.ap.persistence_mgr.execute_async(
+                sqlalchemy.text("""
+                    SELECT column_name FROM information_schema.columns
+                    WHERE table_name = :table_name
+                """),
+                {'table_name': table_name},
+            )
+            return {row[0] for row in result.fetchall()}
+
+        result = await self.ap.persistence_mgr.execute_async(sqlalchemy.text(f'PRAGMA table_info({table_name})'))
+        return {row[1] for row in result.fetchall()}

src/langbot/pkg/pipeline/preproc/preproc.py

@@ -109,7 +109,7 @@ class PreProcessor(stage.PipelineStage):
             if llm_model:
                 query.use_llm_model_uuid = llm_model.model_entity.uuid
 
-                if llm_model.model_entity.abilities.__contains__('func_call'):
+                if 'func_call' in (llm_model.model_entity.abilities or []):
                     # Get bound plugins and MCP servers for filtering tools
                     bound_plugins = query.variables.get('_pipeline_bound_plugins', None)
                     bound_mcp_servers = query.variables.get('_pipeline_bound_mcp_servers', None)
@@ -159,11 +159,7 @@ class PreProcessor(stage.PipelineStage):
 
         # Check if this model supports vision, if not, remove all images
         # TODO this checking should be performed in runner, and in this stage, the image should be reserved
-        if (
+        if selected_runner == 'local-agent' and llm_model and 'vision' not in (llm_model.model_entity.abilities or []):
-            selected_runner == 'local-agent'
-            and llm_model
-            and not llm_model.model_entity.abilities.__contains__('vision')
-        ):
             for msg in query.messages:
                 if isinstance(msg.content, list):
                     for me in msg.content:
@@ -181,7 +177,7 @@ class PreProcessor(stage.PipelineStage):
                 plain_text += me.text
             elif isinstance(me, platform_message.Image):
                 if selected_runner != 'local-agent' or (
-                    llm_model and llm_model.model_entity.abilities.__contains__('vision')
+                    llm_model and 'vision' in (llm_model.model_entity.abilities or [])
                 ):
                     if me.base64 is not None:
                         content_list.append(provider_message.ContentElement.from_image_base64(me.base64))
@@ -202,7 +198,7 @@ class PreProcessor(stage.PipelineStage):
                         content_list.append(provider_message.ContentElement.from_text(msg.text))
                     elif isinstance(msg, platform_message.Image):
                         if selected_runner != 'local-agent' or (
-                            llm_model and llm_model.model_entity.abilities.__contains__('vision')
+                            llm_model and 'vision' in (llm_model.model_entity.abilities or [])
                         ):
                             if msg.base64 is not None:
                                 content_list.append(provider_message.ContentElement.from_image_base64(msg.base64))

src/langbot/pkg/platform/sources/lark.py

@@ -881,7 +881,8 @@ class LarkAdapter(abstract_platform_adapter.AbstractMessagePlatformAdapter):
 
         bot_account_id = config['bot_name']
 
-        bot = lark_oapi.ws.Client(config['app_id'], config['app_secret'], event_handler=event_handler)
+        domain = self._resolve_domain(config)
+        bot = lark_oapi.ws.Client(config['app_id'], config['app_secret'], event_handler=event_handler, domain=domain)
         api_client = self.build_api_client(config)
         cipher = AESCipher(config.get('encrypt-key', ''))
         self.request_app_ticket(api_client, config)
@@ -1014,13 +1015,28 @@ class LarkAdapter(abstract_platform_adapter.AbstractMessagePlatformAdapter):
 
         return None
 
+    @staticmethod
+    def _resolve_domain(config) -> str:
+        domain = config.get('domain', lark_oapi.FEISHU_DOMAIN)
+        if domain == 'custom':
+            domain = config.get('custom_domain', '')
+            if not domain:
+                raise ValueError('Custom domain is required when domain is set to "custom"')
+        return domain.rstrip('/')
+
     def build_api_client(self, config):
         app_id = config['app_id']
         app_secret = config['app_secret']
-        api_client = lark_oapi.Client.builder().app_id(app_id).app_secret(app_secret).build()
+        domain = self._resolve_domain(config)
+        api_client = lark_oapi.Client.builder().app_id(app_id).app_secret(app_secret).domain(domain).build()
         if 'isv' == config.get('app_type', 'self'):
             api_client = (
-                lark_oapi.Client.builder().app_id(app_id).app_secret(app_secret).app_type(lark_oapi.AppType.ISV).build()
+                lark_oapi.Client.builder()
+                .app_id(app_id)
+                .app_secret(app_secret)
+                .app_type(lark_oapi.AppType.ISV)
+                .domain(domain)
+                .build()
             )
         return api_client
 

src/langbot/pkg/platform/sources/lark.yaml

@@ -23,6 +23,57 @@ spec:
     en: https://link.langbot.app/en/platforms/lark
     ja: https://link.langbot.app/ja/platforms/lark
   config:
+    - name: domain
+      label:
+        en_US: Platform Domain
+        zh_Hans: 平台域名
+        zh_Hant: 平台域名
+        ja_JP: プラットフォームドメイン
+      description:
+        en_US: Select the open platform domain. Use Feishu for Chinese mainland, Lark for international
+        zh_Hans: 选择开放平台域名，国内使用飞书，海外使用 Lark
+        zh_Hant: 選擇開放平台域名，國內使用飛書，海外使用 Lark
+        ja_JP: オープンプラットフォームのドメインを選択。中国国内は飛書、海外は Lark を使用
+      type: select
+      options:
+        - name: https://open.feishu.cn
+          label:
+            en_US: Feishu (open.feishu.cn)
+            zh_Hans: 飞书 (open.feishu.cn)
+            zh_Hant: 飛書 (open.feishu.cn)
+            ja_JP: 飛書 (open.feishu.cn)
+        - name: https://open.larksuite.com
+          label:
+            en_US: Lark (open.larksuite.com)
+            zh_Hans: Lark (open.larksuite.com)
+            zh_Hant: Lark (open.larksuite.com)
+            ja_JP: Lark (open.larksuite.com)
+        - name: custom
+          label:
+            en_US: Custom
+            zh_Hans: 自定义
+            zh_Hant: 自定義
+            ja_JP: カスタム
+      required: false
+      default: https://open.feishu.cn
+    - name: custom_domain
+      label:
+        en_US: Custom Domain
+        zh_Hans: 自定义域名
+        zh_Hant: 自定義域名
+        ja_JP: カスタムドメイン
+      description:
+        en_US: "Enter the full domain URL, e.g. https://open.example.com"
+        zh_Hans: "输入完整的域名 URL，例如 https://open.example.com"
+        zh_Hant: "輸入完整的域名 URL，例如 https://open.example.com"
+        ja_JP: "完全なドメイン URL を入力（例: https://open.example.com）"
+      type: string
+      required: false
+      default: ""
+      show_if:
+        field: domain
+        operator: eq
+        value: custom
     - name: one-click-create
       label:
         en_US: One-Click Create App
@@ -140,10 +191,10 @@ spec:
         zh_Hant: 應用類型
         ja_JP: アプリタイプ
       description:
-        en_US: Default to self-built application, refer to https://open.feishu.cn/document/platform-overveiw/overview
+        en_US: "Default to self-built application, refer to https://open.feishu.cn/document/platform-overveiw/overview"
-        zh_Hans: 默认为企业自建应用，参考 https://open.feishu.cn/document/platform-overveiw/overview
+        zh_Hans: "默认为企业自建应用，参考 https://open.feishu.cn/document/platform-overveiw/overview"
-        zh_Hant: 預設為企業自建應用，參考 https://open.feishu.cn/document/platform-overveiw/overview
+        zh_Hant: "預設為企業自建應用，參考 https://open.feishu.cn/document/platform-overveiw/overview"
-        ja_JP: デフォルトはカスタムアプリです。詳細は https://open.feishu.cn/document/platform-overveiw/overview を参照してください
+        ja_JP: "デフォルトはカスタムアプリです。詳細は https://open.feishu.cn/document/platform-overveiw/overview を参照してください"
       type: select
       options:
         - name: self

src/langbot/pkg/plugin/connector.py

@@ -103,6 +103,16 @@ class PluginRuntimeConnector(ManagedRuntimeConnector):
 
             self.handler_task = asyncio.create_task(self.handler.run())
             _ = await self.handler.ping()
+            # Push the configured marketplace (Space) URL to the runtime so it
+            # downloads plugins from the same Space LangBot is bound to, rather
+            # than relying on the runtime's own env/default.
+            space_url = self.ap.instance_config.data.get('space', {}).get('url', '').rstrip('/')
+            if space_url:
+                try:
+                    await self.handler.set_runtime_config(cloud_service_url=space_url)
+                    self.ap.logger.info(f'Pushed marketplace URL to plugin runtime: {space_url}')
+                except Exception as e:
+                    self.ap.logger.warning(f'Failed to push runtime config: {e}')
             self.ap.logger.info('Connected to plugin runtime.')
             await self.handler_task
 
@@ -224,30 +234,23 @@ class PluginRuntimeConnector(ManagedRuntimeConnector):
         mcp_data: dict[str, Any],
         task_context: taskmgr.TaskContext | None = None,
     ):
-        """Install an MCP server from marketplace data."""
+        """Install an MCP server from marketplace data.
+
+        Marketplace MCP records carry the runtime-ready ``mode`` and
+        ``extra_args`` directly (the same shape LangBot stores in
+        ``mcp_servers``), so they are used as-is rather than reconstructed.
+        For ``stdio`` this preserves ``command``/``args``/``env``/``box``;
+        for ``http``/``sse`` it preserves ``url``/``headers``/``timeout``/
+        ``ssereadtimeout``.
+        """
         from ..entity.persistence import mcp as persistence_mcp
         import uuid
 
-        config = mcp_data.get('config', {})
+        mode = mcp_data.get('mode') or 'stdio'
-        url = config.get('url', '')
+        extra_args = mcp_data.get('extra_args') or {}
         # Use __ instead of / to avoid URL routing issues with slashes
         name = f'{mcp_data.get("author", "")}__{mcp_data.get("name", "")}'
 
-        # Determine mode from URL
-        if 'sse' in url.lower():
-            mode = 'sse'
-        elif url.startswith('http'):
-            mode = 'http'
-        else:
-            mode = 'stdio'
-
-        # Build extra_args from config
-        extra_args = {
-            'url': url,
-            'timeout': config.get('timeout', 30),
-            'sse_read_timeout': config.get('sse_read_timeout', 300),
-        }
-
         # Check if MCP server already exists
         existing = await self.ap.persistence_mgr.execute_async(
             sqlalchemy.select(persistence_mcp.MCPServer).where(persistence_mcp.MCPServer.name == name)
@@ -376,15 +379,22 @@ class PluginRuntimeConnector(ManagedRuntimeConnector):
                 mcp_resp = await client.get(f'{space_url}/api/v1/marketplace/mcps/{plugin_author}/{plugin_name}')
                 if mcp_resp.status_code == 200:
                     mcp_data = mcp_resp.json().get('data', {}).get('mcp', {})
-                    if mcp_data.get('config'):
+                    if mcp_data.get('mode'):
                         # It's an MCP - create server locally
                         self.ap.logger.info(f'Installing MCP from marketplace: {plugin_author}/{plugin_name}')
                         if task_context:
                             task_context.set_current_action('installing mcp server')
                         await self._install_mcp_from_marketplace(mcp_data, task_context)
+                        # Best-effort install report (bumps marketplace install_count).
+                        try:
+                            await client.post(
+                                f'{space_url}/api/v1/marketplace/mcps/{plugin_author}/{plugin_name}/install'
+                            )
+                        except Exception as report_err:
+                            self.ap.logger.debug(f'Failed to report MCP install: {report_err}')
                         return
                     else:
-                        raise Exception(f'MCP {plugin_author}/{plugin_name} has no config')
+                        raise Exception(f'MCP {plugin_author}/{plugin_name} has no mode')
                 elif mcp_resp.status_code == 404:
                     # Try skill endpoint - download ZIP and install
                     self.ap.logger.info(f'Trying skill endpoint for: {plugin_author}/{plugin_name}')
@@ -449,7 +459,7 @@ class PluginRuntimeConnector(ManagedRuntimeConnector):
                                         )
 
                                     file_bytes = download_resp.content
-                                    self._extract_deps_metadata(file_bytes, task_context)
+                                    self._inspect_plugin_package(file_bytes, task_context)
                                     file_key = await self.handler.send_file(file_bytes, 'lbpkg')
                                     install_info['plugin_file_key'] = file_key
                                     self.ap.logger.info(f'Transfered file {file_key} to plugin runtime')

src/langbot/pkg/plugin/handler.py

@@ -779,6 +779,16 @@ class RuntimeConnectionHandler(handler.Handler):
             timeout=10,
         )
 
+    async def set_runtime_config(self, cloud_service_url: str) -> dict[str, Any]:
+        """Push runtime configuration (e.g. marketplace URL) to the runtime."""
+        return await self.call_action(
+            LangBotToRuntimeAction.SET_RUNTIME_CONFIG,
+            {
+                'cloud_service_url': cloud_service_url,
+            },
+            timeout=10,
+        )
+
     async def install_plugin(
         self, install_source: str, install_info: dict[str, Any]
     ) -> typing.AsyncGenerator[dict[str, Any], None]:

src/langbot/pkg/provider/modelmgr/modelmgr.py

@@ -37,11 +37,41 @@ class ModelManager:
         self.requester_components = []
         self.requester_dict = {}
 
+    @staticmethod
+    def _get_litellm_provider_from_manifest(component: engine.Component | None) -> str | None:
+        if component is None:
+            return None
+
+        spec = getattr(component, 'spec', None) or {}
+        litellm_provider = None
+
+        if isinstance(spec, dict):
+            litellm_provider = spec.get('litellm_provider')
+        else:
+            getter = getattr(spec, 'get', None)
+            if callable(getter):
+                try:
+                    litellm_provider = getter('litellm_provider')
+                except Exception:
+                    litellm_provider = None
+
+        if isinstance(litellm_provider, str) and litellm_provider:
+            return litellm_provider
+        return None
+
     async def initialize(self):
         self.requester_components = self.ap.discover.get_components_by_kind('LLMAPIRequester')
 
         requester_dict: dict[str, type[requester.ProviderAPIRequester]] = {}
         for component in self.requester_components:
+            # Skip components that use litellm_provider (they will use litellmchat.py instead)
+            litellm_provider = self._get_litellm_provider_from_manifest(component)
+            if litellm_provider:
+                self.ap.logger.debug(
+                    f'Skipping Python class loading for {component.metadata.name} '
+                    f'(uses litellm_provider={litellm_provider})'
+                )
+                continue
             requester_dict[component.metadata.name] = component.get_python_component_class()
 
         self.requester_dict = requester_dict
@@ -143,49 +173,83 @@ class ModelManager:
         # get the latest models from space
         space_models = await self.ap.space_service.get_models()
 
-        exists_llm_models_uuids = [m['uuid'] for m in await self.ap.llm_model_service.get_llm_models()]
+        # Index existing models by uuid. Space reuses a model's uuid across
-        exists_embedding_models_uuids = [
+        # renames / re-specs (e.g. the uuid that used to be ``claude-opus-4-6``
-            m['uuid'] for m in await self.ap.embedding_models_service.get_embedding_models()
+        # may later become ``claude-opus-4-7``). So for Space-managed models we
-        ]
+        # upsert: create when the uuid is new, otherwise update name/abilities/
+        # ranking to track Space. Models owned by other providers are never
+        # touched, even on an (unexpected) uuid collision.
+        existing_llm_models = {m['uuid']: m for m in await self.ap.llm_model_service.get_llm_models()}
+        existing_embedding_models = {
+            m['uuid']: m for m in await self.ap.embedding_models_service.get_embedding_models()
+        }
+
+        created = 0
+        updated = 0
 
         for space_model in space_models:
             if space_model.category == 'chat':
-                uuid = space_model.uuid
+                existing = existing_llm_models.get(space_model.uuid)
-
+                if existing is None:
-                if uuid in exists_llm_models_uuids:
+                    # model will be automatically loaded
-                    continue
+                    await self.ap.llm_model_service.create_llm_model(
-
+                        {
-                # model will be automatically loaded
+                            'uuid': space_model.uuid,
-                await self.ap.llm_model_service.create_llm_model(
+                            'name': space_model.model_id,
-                    {
+                            'provider_uuid': space_model_provider.uuid,
-                        'uuid': space_model.uuid,
+                            'abilities': space_model.llm_abilities or [],
+                            'extra_args': {},
+                            'prefered_ranking': space_model.featured_order,
+                        },
+                        preserve_uuid=True,
+                        auto_set_to_default_pipeline=False,
+                    )
+                    created += 1
+                elif existing.get('provider_uuid') == space_model_provider.uuid:
+                    desired = {
                         'name': space_model.model_id,
                         'provider_uuid': space_model_provider.uuid,
                         'abilities': space_model.llm_abilities or [],
-                        'extra_args': {},
                         'prefered_ranking': space_model.featured_order,
-                    },
+                    }
-                    preserve_uuid=True,
+                    if (
-                    auto_set_to_default_pipeline=False,
+                        existing.get('name') != desired['name']
-                )
+                        or list(existing.get('abilities') or []) != list(desired['abilities'])
+                        or existing.get('prefered_ranking') != desired['prefered_ranking']
+                    ):
+                        await self.ap.llm_model_service.update_llm_model(space_model.uuid, dict(desired))
+                        updated += 1
 
             elif space_model.category == 'embedding':
-                uuid = space_model.uuid
+                existing = existing_embedding_models.get(space_model.uuid)
-
+                if existing is None:
-                if uuid in exists_embedding_models_uuids:
+                    # model will be automatically loaded
-                    continue
+                    await self.ap.embedding_models_service.create_embedding_model(
-
+                        {
-                # model will be automatically loaded
+                            'uuid': space_model.uuid,
-                await self.ap.embedding_models_service.create_embedding_model(
+                            'name': space_model.model_id,
-                    {
+                            'provider_uuid': space_model_provider.uuid,
-                        'uuid': space_model.uuid,
+                            'extra_args': {},
+                            'prefered_ranking': space_model.featured_order,
+                        },
+                        preserve_uuid=True,
+                    )
+                    created += 1
+                elif existing.get('provider_uuid') == space_model_provider.uuid:
+                    desired = {
                         'name': space_model.model_id,
                         'provider_uuid': space_model_provider.uuid,
-                        'extra_args': {},
                         'prefered_ranking': space_model.featured_order,
-                    },
+                    }
-                    preserve_uuid=True,
+                    if (
-                )
+                        existing.get('name') != desired['name']
+                        or existing.get('prefered_ranking') != desired['prefered_ranking']
+                    ):
+                        await self.ap.embedding_models_service.update_embedding_model(space_model.uuid, dict(desired))
+                        updated += 1
+
+        if created or updated:
+            self.ap.logger.info(f'Synced models from LangBot Space: {created} added, {updated} updated.')
 
     async def init_temporary_runtime_llm_model(
         self,
@@ -202,6 +266,7 @@ class ModelManager:
                 name=model_info.get('name', ''),
                 provider_uuid='',
                 abilities=model_info.get('abilities', []),
+                context_length=model_info.get('context_length'),
                 extra_args=model_info.get('extra_args', {}),
             ),
             provider=runtime_provider,
@@ -260,13 +325,37 @@ class ModelManager:
         else:
             provider_entity = provider_info
 
-        if provider_entity.requester not in self.requester_dict:
+        # Get requester manifest to check for litellm_provider
-            raise provider_errors.RequesterNotFoundError(provider_entity.requester)
+        requester_manifest = self.get_available_requester_manifest_by_name(provider_entity.requester)
+        litellm_provider = self._get_litellm_provider_from_manifest(requester_manifest)
+
+        # Build config from base_url
+        config = {'base_url': provider_entity.base_url}
+
+        # Check if requester manifest specifies litellm_provider
+        if litellm_provider:
+            from .requesters import litellmchat
+
+            # Use unified LiteLLMRequester with provider prefix
+            # Map litellm_provider (YAML spec) to custom_llm_provider (config)
+            config['custom_llm_provider'] = litellm_provider
+            requester_inst = litellmchat.LiteLLMRequester(
+                ap=self.ap,
+                config=config,
+            )
+            self.ap.logger.debug(
+                f'Using LiteLLMRequester for {provider_entity.requester} '
+                f'with custom_llm_provider={config["custom_llm_provider"]}'
+            )
+        else:
+            # Use original requester class (for backward compatibility)
+            if provider_entity.requester not in self.requester_dict:
+                raise provider_errors.RequesterNotFoundError(provider_entity.requester)
+            requester_inst = self.requester_dict[provider_entity.requester](
+                ap=self.ap,
+                config=config,
+            )
 
-        requester_inst = self.requester_dict[provider_entity.requester](
-            ap=self.ap,
-            config={'base_url': provider_entity.base_url},
-        )
         await requester_inst.initialize()
 
         token_mgr = token.TokenManager(name=provider_entity.uuid, tokens=provider_entity.api_keys or [])
@@ -372,6 +461,7 @@ class ModelManager:
             name=model_info.get('name', ''),
             provider_uuid=model_info.get('provider_uuid', ''),
             abilities=model_info.get('abilities', []),
+            context_length=model_info.get('context_length'),
             extra_args=model_info.get('extra_args', {}),
         )
 

src/langbot/pkg/provider/modelmgr/requester.py

@@ -67,8 +67,8 @@ class RuntimeProvider:
             if isinstance(result, tuple):
                 msg, usage_info = result
                 if usage_info:
-                    input_tokens = usage_info.get('input_tokens', 0)
+                    input_tokens = usage_info.get('prompt_tokens', 0)
-                    output_tokens = usage_info.get('output_tokens', 0)
+                    output_tokens = usage_info.get('completion_tokens', 0)
                 return msg
             else:
                 return result
@@ -128,7 +128,6 @@ class RuntimeProvider:
         start_time = time.time()
         status = 'success'
         error_message = None
-        # Note: Stream doesn't easily provide token counts, set to 0
         input_tokens = 0
         output_tokens = 0
 
@@ -143,6 +142,15 @@ class RuntimeProvider:
                 remove_think=remove_think,
             ):
                 yield chunk
+            # Extract usage from stream if available (stored by LiteLLM requester)
+            if query:
+                if query.variables is None:
+                    query.variables = {}
+                if '_stream_usage' in query.variables:
+                    usage_info = query.variables['_stream_usage']
+                    input_tokens = usage_info.get('prompt_tokens', 0)
+                    output_tokens = usage_info.get('completion_tokens', 0)
+                    del query.variables['_stream_usage']
         except Exception as e:
             status = 'error'
             error_message = str(e)

src/langbot/pkg/provider/modelmgr/requesters/302aichatcmpl.py

@@ -1,17 +0,0 @@
-from __future__ import annotations
-
-import typing
-import openai
-
-from . import chatcmpl
-
-
-class AI302ChatCompletions(chatcmpl.OpenAIChatCompletions):
-    """302.AI ChatCompletion API 请求器"""
-
-    client: openai.AsyncClient
-
-    default_config: dict[str, typing.Any] = {
-        'base_url': 'https://api.302.ai/v1',
-        'timeout': 120,
-    }

src/langbot/pkg/provider/modelmgr/requesters/302aichatcmpl.yaml

@@ -7,6 +7,7 @@ metadata:
     zh_Hans: 302.AI
   icon: 302ai.png
 spec:
+  litellm_provider: openai
   config:
   - name: base_url
     label:

src/langbot/pkg/provider/modelmgr/requesters/anthropicmsgs.py

@@ -1,370 +0,0 @@
-from __future__ import annotations
-
-import typing
-import json
-import platform
-import socket
-import anthropic
-import httpx
-
-from .. import errors, requester
-
-from ....utils import image
-import langbot_plugin.api.entities.builtin.resource.tool as resource_tool
-import langbot_plugin.api.entities.builtin.pipeline.query as pipeline_query
-import langbot_plugin.api.entities.builtin.provider.message as provider_message
-
-
-class AnthropicMessages(requester.ProviderAPIRequester):
-    """Anthropic Messages API 请求器"""
-
-    client: anthropic.AsyncAnthropic
-
-    default_config: dict[str, typing.Any] = {
-        'base_url': 'https://api.anthropic.com',
-        'timeout': 120,
-    }
-
-    async def initialize(self):
-        # 兼容 Windows 缺失 TCP_KEEPINTVL 和 TCP_KEEPCNT 的问题
-        if platform.system() == 'Windows':
-            if not hasattr(socket, 'TCP_KEEPINTVL'):
-                socket.TCP_KEEPINTVL = 0
-            if not hasattr(socket, 'TCP_KEEPCNT'):
-                socket.TCP_KEEPCNT = 0
-        httpx_client = anthropic._base_client.AsyncHttpxClientWrapper(
-            base_url=self.requester_cfg['base_url'],
-            # cast to a valid type because mypy doesn't understand our type narrowing
-            timeout=typing.cast(httpx.Timeout, self.requester_cfg['timeout']),
-            limits=anthropic._constants.DEFAULT_CONNECTION_LIMITS,
-            follow_redirects=True,
-            trust_env=True,
-        )
-
-        self.client = anthropic.AsyncAnthropic(
-            api_key='',
-            http_client=httpx_client,
-            base_url=self.requester_cfg['base_url'],
-        )
-
-    async def invoke_llm(
-        self,
-        query: pipeline_query.Query,
-        model: requester.RuntimeLLMModel,
-        messages: typing.List[provider_message.Message],
-        funcs: typing.List[resource_tool.LLMTool] = None,
-        extra_args: dict[str, typing.Any] = {},
-        remove_think: bool = False,
-    ) -> provider_message.Message:
-        self.client.api_key = model.provider.token_mgr.get_token()
-
-        args = extra_args.copy()
-        args['model'] = model.model_entity.name
-
-        # 处理消息
-
-        # system
-        system_role_message = None
-
-        for i, m in enumerate(messages):
-            if m.role == 'system':
-                system_role_message = m
-
-                break
-
-        if system_role_message:
-            messages.pop(i)
-
-        if isinstance(system_role_message, provider_message.Message) and isinstance(system_role_message.content, str):
-            args['system'] = system_role_message.content
-
-        req_messages = []
-
-        for m in messages:
-            if m.role == 'tool':
-                tool_call_id = m.tool_call_id
-
-                req_messages.append(
-                    {
-                        'role': 'user',
-                        'content': [
-                            {
-                                'type': 'tool_result',
-                                'tool_use_id': tool_call_id,
-                                'is_error': False,
-                                'content': [{'type': 'text', 'text': m.content}],
-                            }
-                        ],
-                    }
-                )
-
-                continue
-
-            msg_dict = m.dict(exclude_none=True)
-
-            if isinstance(m.content, str) and m.content.strip() != '':
-                msg_dict['content'] = [{'type': 'text', 'text': m.content}]
-            elif isinstance(m.content, list):
-                for i, ce in enumerate(m.content):
-                    if ce.type == 'image_base64':
-                        image_b64, image_format = await image.extract_b64_and_format(ce.image_base64)
-
-                        alter_image_ele = {
-                            'type': 'image',
-                            'source': {
-                                'type': 'base64',
-                                'media_type': f'image/{image_format}',
-                                'data': image_b64,
-                            },
-                        }
-                        msg_dict['content'][i] = alter_image_ele
-
-            if m.tool_calls:
-                for tool_call in m.tool_calls:
-                    msg_dict['content'].append(
-                        {
-                            'type': 'tool_use',
-                            'id': tool_call.id,
-                            'name': tool_call.function.name,
-                            'input': json.loads(tool_call.function.arguments),
-                        }
-                    )
-
-                del msg_dict['tool_calls']
-
-            req_messages.append(msg_dict)
-
-        args['messages'] = req_messages
-
-        if 'thinking' in args:
-            args['thinking'] = {'type': 'enabled', 'budget_tokens': 10000}
-
-        if funcs:
-            tools = await self.ap.tool_mgr.generate_tools_for_anthropic(funcs)
-
-            if tools:
-                args['tools'] = tools
-
-        try:
-            resp = await self.client.messages.create(**args)
-
-            args = {
-                'content': '',
-                'role': resp.role,
-            }
-            assert type(resp) is anthropic.types.message.Message
-
-            for block in resp.content:
-                if not remove_think and block.type == 'thinking':
-                    args['content'] = '<think>\n' + block.thinking + '\n</think>\n' + args['content']
-                elif block.type == 'text':
-                    args['content'] += block.text
-                elif block.type == 'tool_use':
-                    assert type(block) is anthropic.types.tool_use_block.ToolUseBlock
-                    tool_call = provider_message.ToolCall(
-                        id=block.id,
-                        type='function',
-                        function=provider_message.FunctionCall(name=block.name, arguments=json.dumps(block.input)),
-                    )
-                    if 'tool_calls' not in args:
-                        args['tool_calls'] = []
-                    args['tool_calls'].append(tool_call)
-
-            return provider_message.Message(**args)
-        except anthropic.AuthenticationError as e:
-            raise errors.RequesterError(f'api-key 无效: {e.message}')
-        except anthropic.BadRequestError as e:
-            raise errors.RequesterError(str(e.message))
-        except anthropic.NotFoundError as e:
-            if 'model: ' in str(e):
-                raise errors.RequesterError(f'模型无效: {e.message}')
-            else:
-                raise errors.RequesterError(f'请求地址无效: {e.message}')
-
-    async def invoke_llm_stream(
-        self,
-        query: pipeline_query.Query,
-        model: requester.RuntimeLLMModel,
-        messages: typing.List[provider_message.Message],
-        funcs: typing.List[resource_tool.LLMTool] = None,
-        extra_args: dict[str, typing.Any] = {},
-        remove_think: bool = False,
-    ) -> provider_message.Message:
-        self.client.api_key = model.provider.token_mgr.get_token()
-
-        args = extra_args.copy()
-        args['model'] = model.model_entity.name
-        args['stream'] = True
-
-        # 处理消息
-
-        # system
-        system_role_message = None
-
-        for i, m in enumerate(messages):
-            if m.role == 'system':
-                system_role_message = m
-
-                break
-
-        if system_role_message:
-            messages.pop(i)
-
-        if isinstance(system_role_message, provider_message.Message) and isinstance(system_role_message.content, str):
-            args['system'] = system_role_message.content
-
-        req_messages = []
-
-        for m in messages:
-            if m.role == 'tool':
-                tool_call_id = m.tool_call_id
-
-                req_messages.append(
-                    {
-                        'role': 'user',
-                        'content': [
-                            {
-                                'type': 'tool_result',
-                                'tool_use_id': tool_call_id,
-                                'is_error': False,  # 暂时直接写false
-                                'content': [
-                                    {'type': 'text', 'text': m.content}
-                                ],  # 这里要是list包裹，应该是多个返回的情况？type类型好像也可以填其他的，暂时只写text
-                            }
-                        ],
-                    }
-                )
-
-                continue
-
-            msg_dict = m.dict(exclude_none=True)
-
-            if isinstance(m.content, str) and m.content.strip() != '':
-                msg_dict['content'] = [{'type': 'text', 'text': m.content}]
-            elif isinstance(m.content, list):
-                for i, ce in enumerate(m.content):
-                    if ce.type == 'image_base64':
-                        image_b64, image_format = await image.extract_b64_and_format(ce.image_base64)
-
-                        alter_image_ele = {
-                            'type': 'image',
-                            'source': {
-                                'type': 'base64',
-                                'media_type': f'image/{image_format}',
-                                'data': image_b64,
-                            },
-                        }
-                        msg_dict['content'][i] = alter_image_ele
-            if isinstance(msg_dict['content'], str) and msg_dict['content'] == '':
-                msg_dict['content'] = []  # 这里不知道为什么会莫名有个空导致content为字符
-            if m.tool_calls:
-                for tool_call in m.tool_calls:
-                    msg_dict['content'].append(
-                        {
-                            'type': 'tool_use',
-                            'id': tool_call.id,
-                            'name': tool_call.function.name,
-                            'input': json.loads(tool_call.function.arguments),
-                        }
-                    )
-
-                del msg_dict['tool_calls']
-
-            req_messages.append(msg_dict)
-        if 'thinking' in args:
-            args['thinking'] = {'type': 'enabled', 'budget_tokens': 10000}
-
-        args['messages'] = req_messages
-
-        if funcs:
-            tools = await self.ap.tool_mgr.generate_tools_for_anthropic(funcs)
-
-            if tools:
-                args['tools'] = tools
-
-        try:
-            role = 'assistant'  # 默认角色
-            # chunk_idx = 0
-            think_started = False
-            think_ended = False
-            finish_reason = False
-            tool_name = ''
-            tool_id = ''
-            async for chunk in await self.client.messages.create(**args):
-                content = ''
-                tool_call = {'id': None, 'function': {'name': None, 'arguments': None}, 'type': 'function'}
-                if isinstance(
-                    chunk, anthropic.types.raw_content_block_start_event.RawContentBlockStartEvent
-                ):  # 记录开始
-                    if chunk.content_block.type == 'tool_use':
-                        if chunk.content_block.name is not None:
-                            tool_name = chunk.content_block.name
-                        if chunk.content_block.id is not None:
-                            tool_id = chunk.content_block.id
-
-                        tool_call['function']['name'] = tool_name
-                        tool_call['function']['arguments'] = ''
-                        tool_call['id'] = tool_id
-
-                    if not remove_think:
-                        if chunk.content_block.type == 'thinking' and not remove_think:
-                            think_started = True
-                        elif chunk.content_block.type == 'text' and chunk.index != 0 and not remove_think:
-                            think_ended = True
-                        continue
-                elif isinstance(chunk, anthropic.types.raw_content_block_delta_event.RawContentBlockDeltaEvent):
-                    if chunk.delta.type == 'thinking_delta':
-                        if think_started:
-                            think_started = False
-                            content = '<think>\n' + chunk.delta.thinking
-                        elif remove_think:
-                            continue
-                        else:
-                            content = chunk.delta.thinking
-                    elif chunk.delta.type == 'text_delta':
-                        if think_ended:
-                            think_ended = False
-                            content = '\n</think>\n' + chunk.delta.text
-                        else:
-                            content = chunk.delta.text
-                    elif chunk.delta.type == 'input_json_delta':
-                        tool_call['function']['arguments'] = chunk.delta.partial_json
-                        tool_call['function']['name'] = tool_name
-                        tool_call['id'] = tool_id
-                elif isinstance(chunk, anthropic.types.raw_content_block_stop_event.RawContentBlockStopEvent):
-                    continue  # 记录raw_content_block结束的
-
-                elif isinstance(chunk, anthropic.types.raw_message_delta_event.RawMessageDeltaEvent):
-                    if chunk.delta.stop_reason == 'end_turn':
-                        finish_reason = True
-                elif isinstance(chunk, anthropic.types.raw_message_stop_event.RawMessageStopEvent):
-                    continue  # 这个好像是完全结束
-                else:
-                    # print(chunk)
-                    self.ap.logger.debug(f'anthropic chunk: {chunk}')
-                    continue
-
-                args = {
-                    'content': content,
-                    'role': role,
-                    'is_final': finish_reason,
-                    'tool_calls': None if tool_call['id'] is None else [tool_call],
-                }
-                # if chunk_idx == 0:
-                #     chunk_idx += 1
-                #     continue
-
-                # assert type(chunk) is anthropic.types.message.Chunk
-
-                yield provider_message.MessageChunk(**args)
-
-            # return llm_entities.Message(**args)
-        except anthropic.AuthenticationError as e:
-            raise errors.RequesterError(f'api-key 无效: {e.message}')
-        except anthropic.BadRequestError as e:
-            raise errors.RequesterError(str(e.message))
-        except anthropic.NotFoundError as e:
-            if 'model: ' in str(e):
-                raise errors.RequesterError(f'模型无效: {e.message}')
-            else:
-                raise errors.RequesterError(f'请求地址无效: {e.message}')

src/langbot/pkg/provider/modelmgr/requesters/anthropicmsgs.yaml

@@ -7,6 +7,7 @@ metadata:
     zh_Hans: Anthropic
   icon: anthropic.svg
 spec:
+  litellm_provider: anthropic
   config:
   - name: base_url
     label:
@@ -24,6 +25,8 @@ spec:
     default: 120
   support_type:
   - llm
+  - text-embedding
+  - rerank
   provider_category: manufacturer
 execution:
   python:

src/langbot/pkg/provider/modelmgr/requesters/baidu.svg

@@ -0,0 +1,5 @@
+<svg width="60" height="50" viewBox="0 0 60 50" xmlns="http://www.w3.org/2000/svg">
+  <rect width="60" height="50" rx="8" fill="#2932E1"/>
+  <text x="30" y="28" font-family="Arial, sans-serif" font-size="10" font-weight="bold" fill="white" text-anchor="middle">Baidu</text>
+  <text x="30" y="40" font-family="Arial, sans-serif" font-size="8" fill="white" text-anchor="middle">ERNIE</text>
+</svg>

src/langbot/pkg/provider/modelmgr/requesters/baiduchatcmpl.yaml

@@ -0,0 +1,30 @@
+apiVersion: v1
+kind: LLMAPIRequester
+metadata:
+  name: baidu-chat-completions
+  label:
+    en_US: Baidu ERNIE
+    zh_Hans: 百度文心一言
+  icon: baidu.svg
+spec:
+  litellm_provider: openai
+  config:
+  - name: base_url
+    label:
+      en_US: Base URL
+      zh_Hans: 基础 URL
+    type: string
+    required: true
+    default: https://aip.baidubce.com/rpc/2.0/ai_custom/v1/wenxinworkshop
+  - name: timeout
+    label:
+      en_US: Timeout
+      zh_Hans: 超时时间
+    type: integer
+    required: true
+    default: 120
+  support_type:
+  - llm
+  - text-embedding
+  - rerank
+  provider_category: manufacturer

src/langbot/pkg/provider/modelmgr/requesters/bailianchatcmpl.py

@@ -1,242 +0,0 @@
-from __future__ import annotations
-
-import typing
-import dashscope
-import openai
-
-from . import modelscopechatcmpl
-from .. import requester
-import langbot_plugin.api.entities.builtin.resource.tool as resource_tool
-import langbot_plugin.api.entities.builtin.pipeline.query as pipeline_query
-import langbot_plugin.api.entities.builtin.provider.message as provider_message
-
-
-class BailianChatCompletions(modelscopechatcmpl.ModelScopeChatCompletions):
-    """阿里云百炼大模型平台 ChatCompletion API 请求器"""
-
-    client: openai.AsyncClient
-
-    default_config: dict[str, typing.Any] = {
-        'base_url': 'https://dashscope.aliyuncs.com/compatible-mode/v1',
-        'timeout': 120,
-    }
-
-    async def _closure_stream(
-        self,
-        query: pipeline_query.Query,
-        req_messages: list[dict],
-        use_model: requester.RuntimeLLMModel,
-        use_funcs: list[resource_tool.LLMTool] = None,
-        extra_args: dict[str, typing.Any] = {},
-        remove_think: bool = False,
-    ) -> provider_message.Message | typing.AsyncGenerator[provider_message.MessageChunk, None]:
-        self.client.api_key = use_model.provider.token_mgr.get_token()
-
-        args = {}
-        args['model'] = use_model.model_entity.name
-
-        if use_funcs:
-            tools = await self.ap.tool_mgr.generate_tools_for_openai(use_funcs)
-
-            if tools:
-                args['tools'] = tools
-
-        # 设置此次请求中的messages
-        messages = req_messages.copy()
-
-        is_use_dashscope_call = False  # 是否使用阿里原生库调用
-        is_enable_multi_model = True  # 是否支持多轮对话
-        use_time_num = 0  # 模型已调用次数，防止存在多文件时重复调用
-        use_time_ids = []  # 已调用的ID列表
-        message_id = 0  # 记录消息序号
-
-        for msg in messages:
-            # print(msg)
-            if 'content' in msg and isinstance(msg['content'], list):
-                for me in msg['content']:
-                    if me['type'] == 'image_base64':
-                        me['image_url'] = {'url': me['image_base64']}
-                        me['type'] = 'image_url'
-                        del me['image_base64']
-                    elif me['type'] == 'file_url' and '.' in me.get('file_name', ''):
-                        # 1. 视频文件推理
-                        # https://bailian.console.aliyun.com/?tab=doc#/doc/?type=model&url=2845871
-                        file_type = me.get('file_name').lower().split('.')[-1]
-                        if file_type in ['mp4', 'avi', 'mkv', 'mov', 'flv', 'wmv']:
-                            me['type'] = 'video_url'
-                            me['video_url'] = {'url': me['file_url']}
-                            del me['file_url']
-                            del me['file_name']
-                            use_time_num += 1
-                            use_time_ids.append(message_id)
-                            is_enable_multi_model = False
-                        # 2. 语音文件识别, 无法通过openai的audio字段传递，暂时不支持
-                        # https://bailian.console.aliyun.com/?tab=doc#/doc/?type=model&url=2979031
-                        elif file_type in [
-                            'aac',
-                            'amr',
-                            'aiff',
-                            'flac',
-                            'm4a',
-                            'mp3',
-                            'mpeg',
-                            'ogg',
-                            'opus',
-                            'wav',
-                            'webm',
-                            'wma',
-                        ]:
-                            me['audio'] = me['file_url']
-                            me['type'] = 'audio'
-                            del me['file_url']
-                            del me['type']
-                            del me['file_name']
-                            is_use_dashscope_call = True
-                            use_time_num += 1
-                            use_time_ids.append(message_id)
-                            is_enable_multi_model = False
-            message_id += 1
-
-        # 使用列表推导式，保留不在 use_time_ids[:-1] 中的元素，仅保留最后一个多媒体消息
-        if not is_enable_multi_model and use_time_num > 1:
-            messages = [msg for idx, msg in enumerate(messages) if idx not in use_time_ids[:-1]]
-
-        if not is_enable_multi_model:
-            messages = [msg for msg in messages if 'resp_message_id' not in msg]
-
-        args['messages'] = messages
-        args['stream'] = True
-
-        # 流式处理状态
-        # tool_calls_map: dict[str, provider_message.ToolCall] = {}
-        chunk_idx = 0
-        thinking_started = False
-        thinking_ended = False
-        role = 'assistant'  # 默认角色
-
-        if is_use_dashscope_call:
-            response = dashscope.MultiModalConversation.call(
-                # 若没有配置环境变量，请用百炼API Key将下行替换为：api_key = "sk-xxx"
-                api_key=use_model.provider.token_mgr.get_token(),
-                model=use_model.model_entity.name,
-                messages=messages,
-                result_format='message',
-                asr_options={
-                    # "language": "zh", # 可选，若已知音频的语种，可通过该参数指定待识别语种，以提升识别准确率
-                    'enable_lid': True,
-                    'enable_itn': False,
-                },
-                stream=True,
-            )
-            content_length_list = []
-            previous_length = 0  # 记录上一次的内容长度
-            for res in response:
-                chunk = res['output']
-                # 解析 chunk 数据
-                if hasattr(chunk, 'choices') and chunk.choices:
-                    choice = chunk.choices[0]
-                    delta_content = choice['message'].content[0]['text']
-                    finish_reason = choice['finish_reason']
-                    content_length_list.append(len(delta_content))
-                else:
-                    delta_content = ''
-                    finish_reason = None
-
-                # 跳过空的第一个 chunk（只有 role 没有内容）
-                if chunk_idx == 0 and not delta_content:
-                    chunk_idx += 1
-                    continue
-
-                # 检查 content_length_list 是否有足够的数据
-                if len(content_length_list) >= 2:
-                    now_content = delta_content[previous_length : content_length_list[-1]]
-                    previous_length = content_length_list[-1]  # 更新上一次的长度
-                else:
-                    now_content = delta_content  # 第一次循环时直接使用 delta_content
-                    previous_length = len(delta_content)  # 更新上一次的长度
-
-                # 构建 MessageChunk - 只包含增量内容
-                chunk_data = {
-                    'role': role,
-                    'content': now_content if now_content else None,
-                    'is_final': bool(finish_reason) and finish_reason != 'null',
-                }
-
-                # 移除 None 值
-                chunk_data = {k: v for k, v in chunk_data.items() if v is not None}
-                yield provider_message.MessageChunk(**chunk_data)
-                chunk_idx += 1
-        else:
-            async for chunk in self._req_stream(args, extra_body=extra_args):
-                # 解析 chunk 数据
-                if hasattr(chunk, 'choices') and chunk.choices:
-                    choice = chunk.choices[0]
-                    delta = choice.delta.model_dump() if hasattr(choice, 'delta') else {}
-                    finish_reason = getattr(choice, 'finish_reason', None)
-                else:
-                    delta = {}
-                    finish_reason = None
-
-                # 从第一个 chunk 获取 role，后续使用这个 role
-                if 'role' in delta and delta['role']:
-                    role = delta['role']
-
-                # 获取增量内容
-                delta_content = delta.get('content', '')
-                reasoning_content = delta.get('reasoning_content', '')
-
-                # 处理 reasoning_content
-                if reasoning_content:
-                    # accumulated_reasoning += reasoning_content
-                    # 如果设置了 remove_think，跳过 reasoning_content
-                    if remove_think:
-                        chunk_idx += 1
-                        continue
-
-                    # 第一次出现 reasoning_content，添加 <think> 开始标签
-                    if not thinking_started:
-                        thinking_started = True
-                        delta_content = '<think>\n' + reasoning_content
-                    else:
-                        # 继续输出 reasoning_content
-                        delta_content = reasoning_content
-                elif thinking_started and not thinking_ended and delta_content:
-                    # reasoning_content 结束，normal content 开始，添加 </think> 结束标签
-                    thinking_ended = True
-                    delta_content = '\n</think>\n' + delta_content
-
-                # 处理工具调用增量
-                if delta.get('tool_calls'):
-                    for tool_call in delta['tool_calls']:
-                        if tool_call['id'] != '':
-                            tool_id = tool_call['id']
-                        if tool_call['function']['name'] is not None:
-                            tool_name = tool_call['function']['name']
-
-                        if tool_call['type'] is None:
-                            tool_call['type'] = 'function'
-                        tool_call['id'] = tool_id
-                        tool_call['function']['name'] = tool_name
-                        tool_call['function']['arguments'] = (
-                            '' if tool_call['function']['arguments'] is None else tool_call['function']['arguments']
-                        )
-
-                # 跳过空的第一个 chunk（只有 role 没有内容）
-                if chunk_idx == 0 and not delta_content and not reasoning_content and not delta.get('tool_calls'):
-                    chunk_idx += 1
-                    continue
-
-                # 构建 MessageChunk - 只包含增量内容
-                chunk_data = {
-                    'role': role,
-                    'content': delta_content if delta_content else None,
-                    'tool_calls': delta.get('tool_calls'),
-                    'is_final': bool(finish_reason),
-                }
-
-                # 移除 None 值
-                chunk_data = {k: v for k, v in chunk_data.items() if v is not None}
-
-                yield provider_message.MessageChunk(**chunk_data)
-                chunk_idx += 1
-                # return

src/langbot/pkg/provider/modelmgr/requesters/bailianchatcmpl.yaml

@@ -7,6 +7,7 @@ metadata:
     zh_Hans: 阿里云百炼
   icon: bailian.png
 spec:
+  litellm_provider: openai
   config:
   - name: base_url
     label:
@@ -24,6 +25,7 @@ spec:
     default: 120
   support_type:
   - llm
+  - text-embedding
   - rerank
   provider_category: maas
 execution:

src/langbot/pkg/provider/modelmgr/requesters/chatcmpl.py

@@ -1,702 +0,0 @@
-from __future__ import annotations
-
-import asyncio
-import typing
-
-import openai
-import openai.types.chat.chat_completion as chat_completion_module
-import httpx
-
-from .. import errors, requester
-import langbot_plugin.api.entities.builtin.resource.tool as resource_tool
-import langbot_plugin.api.entities.builtin.pipeline.query as pipeline_query
-import langbot_plugin.api.entities.builtin.provider.message as provider_message
-
-
-class OpenAIChatCompletions(requester.ProviderAPIRequester):
-    """OpenAI ChatCompletion API 请求器"""
-
-    client: openai.AsyncClient
-
-    default_config: dict[str, typing.Any] = {
-        'base_url': 'https://api.openai.com/v1',
-        'timeout': 120,
-    }
-
-    async def initialize(self):
-        self.client = openai.AsyncClient(
-            api_key=self.init_api_key,
-            base_url=self.requester_cfg['base_url'].replace(' ', ''),
-            timeout=self.requester_cfg['timeout'],
-            http_client=httpx.AsyncClient(trust_env=True, timeout=self.requester_cfg['timeout']),
-        )
-
-    def _mask_api_key(self, api_key: str | None) -> str:
-        if not api_key:
-            return ''
-        if len(api_key) <= 8:
-            return '****'
-        return f'{api_key[:4]}...{api_key[-4:]}'
-
-    def _infer_model_type(self, model_id: str) -> str:
-        normalized_model_id = (model_id or '').lower()
-        embedding_keywords = (
-            'embedding',
-            'embed',
-            'bge-',
-            'e5-',
-            'm3e',
-            'gte-',
-            'multilingual-e5',
-            'text-embedding',
-        )
-        return 'embedding' if any(keyword in normalized_model_id for keyword in embedding_keywords) else 'llm'
-
-    def _infer_model_abilities(self, item: dict[str, typing.Any], model_id: str) -> list[str]:
-        normalized_model_id = (model_id or '').lower()
-        abilities: set[str] = set()
-
-        def _flatten(value: typing.Any) -> list[str]:
-            if value is None:
-                return []
-            if isinstance(value, str):
-                return [value.lower()]
-            if isinstance(value, dict):
-                flattened: list[str] = []
-                for nested_value in value.values():
-                    flattened.extend(_flatten(nested_value))
-                return flattened
-            if isinstance(value, (list, tuple, set)):
-                flattened: list[str] = []
-                for nested_value in value:
-                    flattened.extend(_flatten(nested_value))
-                return flattened
-            return [str(value).lower()]
-
-        capability_tokens = _flatten(item.get('capabilities'))
-        capability_tokens.extend(_flatten(item.get('modalities')))
-        capability_tokens.extend(_flatten(item.get('input_modalities')))
-        capability_tokens.extend(_flatten(item.get('output_modalities')))
-        capability_tokens.extend(_flatten(item.get('supported_generation_methods')))
-        capability_tokens.extend(_flatten(item.get('supported_parameters')))
-        capability_tokens.extend(_flatten(item.get('architecture')))
-
-        combined_tokens = capability_tokens + [normalized_model_id]
-
-        vision_keywords = (
-            'vision',
-            'image',
-            'file',
-            'video',
-            'multimodal',
-            'vl',
-            'ocr',
-            'omni',
-        )
-        function_call_keywords = (
-            'function',
-            'tool',
-            'tools',
-            'tool_choice',
-            'tool_call',
-            'tool-use',
-            'tool_use',
-        )
-
-        if any(any(keyword in token for keyword in vision_keywords) for token in combined_tokens):
-            abilities.add('vision')
-
-        if any(any(keyword in token for keyword in function_call_keywords) for token in combined_tokens):
-            abilities.add('func_call')
-
-        return sorted(abilities)
-
-    def _normalize_modalities(self, value: typing.Any) -> list[str]:
-        normalized: list[str] = []
-
-        def _collect(item: typing.Any):
-            if item is None:
-                return
-            if isinstance(item, str):
-                for part in item.replace('->', ',').replace('+', ',').split(','):
-                    token = part.strip().lower()
-                    if token and token not in normalized:
-                        normalized.append(token)
-                return
-            if isinstance(item, dict):
-                for nested in item.values():
-                    _collect(nested)
-                return
-            if isinstance(item, (list, tuple, set)):
-                for nested in item:
-                    _collect(nested)
-                return
-
-        _collect(value)
-        return normalized
-
-    def _extract_scan_metadata(self, item: dict[str, typing.Any], model_id: str) -> dict[str, typing.Any]:
-        display_name = item.get('name')
-        if not isinstance(display_name, str) or not display_name.strip() or display_name == model_id:
-            display_name = ''
-
-        description = item.get('description')
-        if not isinstance(description, str) or not description.strip():
-            description = ''
-
-        context_length = item.get('context_length')
-        if context_length is None and isinstance(item.get('top_provider'), dict):
-            context_length = item['top_provider'].get('context_length')
-
-        if not isinstance(context_length, int):
-            try:
-                context_length = int(context_length) if context_length is not None else None
-            except (TypeError, ValueError):
-                context_length = None
-
-        input_modalities = self._normalize_modalities(item.get('input_modalities'))
-        output_modalities = self._normalize_modalities(item.get('output_modalities'))
-
-        if isinstance(item.get('architecture'), dict):
-            if not input_modalities:
-                input_modalities = self._normalize_modalities(item['architecture'].get('input_modalities'))
-            if not output_modalities:
-                output_modalities = self._normalize_modalities(item['architecture'].get('output_modalities'))
-
-        owned_by = item.get('owned_by')
-        if not isinstance(owned_by, str) or not owned_by.strip():
-            owned_by = ''
-
-        return {
-            'display_name': display_name or None,
-            'description': description or None,
-            'context_length': context_length,
-            'owned_by': owned_by or None,
-            'input_modalities': input_modalities,
-            'output_modalities': output_modalities,
-        }
-
-    async def scan_models(self, api_key: str | None = None) -> dict[str, typing.Any]:
-        headers = {}
-        if api_key:
-            headers['Authorization'] = f'Bearer {api_key}'
-
-        models_url = f'{self.requester_cfg["base_url"].rstrip("/")}/models'
-        async with httpx.AsyncClient(trust_env=True, timeout=self.requester_cfg['timeout']) as client:
-            response = await client.get(models_url, headers=headers)
-            response.raise_for_status()
-            payload = response.json()
-
-        models = []
-        for item in payload.get('data', []):
-            model_id = item.get('id')
-            if not model_id:
-                continue
-            models.append(
-                {
-                    'id': model_id,
-                    'name': model_id,
-                    'type': self._infer_model_type(model_id),
-                    'abilities': self._infer_model_abilities(item, model_id),
-                    **self._extract_scan_metadata(item, model_id),
-                }
-            )
-
-        models.sort(key=lambda item: (item['type'] != 'llm', item['name'].lower()))
-        return {
-            'models': models,
-            'debug': {
-                'request': {
-                    'method': 'GET',
-                    'url': models_url,
-                    'headers': {
-                        'Authorization': f'Bearer {self._mask_api_key(api_key)}' if api_key else '',
-                    },
-                },
-                'response': payload,
-            },
-        }
-
-    async def _req(
-        self,
-        args: dict,
-        extra_body: dict = {},
-    ) -> chat_completion_module.ChatCompletion:
-        return await self.client.chat.completions.create(**args, extra_body=extra_body)
-
-    async def _req_stream(
-        self,
-        args: dict,
-        extra_body: dict = {},
-    ):
-        async for chunk in await self.client.chat.completions.create(**args, extra_body=extra_body):
-            yield chunk
-
-    async def _make_msg(
-        self,
-        chat_completion: chat_completion_module.ChatCompletion,
-        remove_think: bool = False,
-    ) -> provider_message.Message:
-        if not isinstance(chat_completion, chat_completion_module.ChatCompletion):
-            raise TypeError(f'Expected ChatCompletion, got {type(chat_completion).__name__}: {chat_completion[:16]}')
-
-        chatcmpl_message = chat_completion.choices[0].message.model_dump()
-
-        # 确保 role 字段存在且不为 None
-        if 'role' not in chatcmpl_message or chatcmpl_message['role'] is None:
-            chatcmpl_message['role'] = 'assistant'
-
-        # 处理思维链
-        content = chatcmpl_message.get('content', '')
-        reasoning_content = chatcmpl_message.get('reasoning_content', None)
-
-        processed_content, _ = await self._process_thinking_content(
-            content=content, reasoning_content=reasoning_content, remove_think=remove_think
-        )
-
-        chatcmpl_message['content'] = processed_content
-
-        # 移除 reasoning_content 字段，避免传递给 Message
-        if 'reasoning_content' in chatcmpl_message:
-            del chatcmpl_message['reasoning_content']
-
-        message = provider_message.Message(**chatcmpl_message)
-
-        return message
-
-    async def _process_thinking_content(
-        self,
-        content: str,
-        reasoning_content: str = None,
-        remove_think: bool = False,
-    ) -> tuple[str, str]:
-        """处理思维链内容
-
-        Args:
-            content: 原始内容
-            reasoning_content: reasoning_content 字段内容
-            remove_think: 是否移除思维链
-
-        Returns:
-            (处理后的内容, 提取的思维链内容)
-        """
-        thinking_content = ''
-
-        # 1. 从 reasoning_content 提取思维链
-        if reasoning_content:
-            thinking_content = reasoning_content
-
-        # 2. 从 content 中提取 <think> 标签内容
-        if content and '<think>' in content and '</think>' in content:
-            import re
-
-            think_pattern = r'<think>(.*?)</think>'
-            think_matches = re.findall(think_pattern, content, re.DOTALL)
-            if think_matches:
-                # 如果已有 reasoning_content，则追加
-                if thinking_content:
-                    thinking_content += '\n' + '\n'.join(think_matches)
-                else:
-                    thinking_content = '\n'.join(think_matches)
-                # 移除 content 中的 <think> 标签
-                content = re.sub(think_pattern, '', content, flags=re.DOTALL).strip()
-
-        # 3. 根据 remove_think 参数决定是否保留思维链
-        if remove_think:
-            return content, ''
-        else:
-            # 如果有思维链内容，将其以 <think> 格式添加到 content 开头
-            if thinking_content:
-                content = f'<think>\n{thinking_content}\n</think>\n{content}'.strip()
-            return content, thinking_content
-
-    async def _closure_stream(
-        self,
-        query: pipeline_query.Query,
-        req_messages: list[dict],
-        use_model: requester.RuntimeLLMModel,
-        use_funcs: list[resource_tool.LLMTool] = None,
-        extra_args: dict[str, typing.Any] = {},
-        remove_think: bool = False,
-    ) -> provider_message.MessageChunk:
-        self.client.api_key = use_model.provider.token_mgr.get_token()
-
-        args = {}
-        args['model'] = use_model.model_entity.name
-
-        if use_funcs:
-            tools = await self.ap.tool_mgr.generate_tools_for_openai(use_funcs)
-            if tools:
-                args['tools'] = tools
-
-        # 设置此次请求中的messages
-        messages = req_messages.copy()
-
-        # 检查vision
-        for msg in messages:
-            if 'content' in msg and isinstance(msg['content'], list):
-                for me in msg['content']:
-                    if me['type'] == 'image_base64':
-                        me['image_url'] = {'url': me['image_base64']}
-                        me['type'] = 'image_url'
-                        del me['image_base64']
-
-        args['messages'] = messages
-        args['stream'] = True
-
-        # 流式处理状态
-        # tool_calls_map: dict[str, provider_message.ToolCall] = {}
-        chunk_idx = 0
-        thinking_started = False
-        thinking_ended = False
-        role = 'assistant'  # 默认角色
-        tool_id = ''
-        tool_name = ''
-        # accumulated_reasoning = ''  # 仅用于判断何时结束思维链
-
-        async for chunk in self._req_stream(args, extra_body=extra_args):
-            # 解析 chunk 数据
-
-            if hasattr(chunk, 'choices') and chunk.choices:
-                choice = chunk.choices[0]
-                delta = choice.delta.model_dump() if hasattr(choice, 'delta') else {}
-
-                finish_reason = getattr(choice, 'finish_reason', None)
-            else:
-                delta = {}
-                finish_reason = None
-            # 从第一个 chunk 获取 role，后续使用这个 role
-            if 'role' in delta and delta['role']:
-                role = delta['role']
-
-            # 获取增量内容
-            delta_content = delta.get('content', '')
-            reasoning_content = delta.get('reasoning_content', '')
-
-            # 处理 reasoning_content
-            if reasoning_content:
-                # accumulated_reasoning += reasoning_content
-                # 如果设置了 remove_think，跳过 reasoning_content
-                if remove_think:
-                    chunk_idx += 1
-                    continue
-
-                # 第一次出现 reasoning_content，添加 <think> 开始标签
-                if not thinking_started:
-                    thinking_started = True
-                    delta_content = '<think>\n' + reasoning_content
-                else:
-                    # 继续输出 reasoning_content
-                    delta_content = reasoning_content
-            elif thinking_started and not thinking_ended and delta_content:
-                # reasoning_content 结束，normal content 开始，添加 </think> 结束标签
-                thinking_ended = True
-                delta_content = '\n</think>\n' + delta_content
-
-            # 处理 content 中已有的 <think> 标签（如果需要移除）
-            # if delta_content and remove_think and '<think>' in delta_content:
-            #     import re
-            #
-            #     # 移除 <think> 标签及其内容
-            #     delta_content = re.sub(r'<think>.*?</think>', '', delta_content, flags=re.DOTALL)
-
-            # 处理工具调用增量
-            # delta_tool_calls = None
-            if delta.get('tool_calls'):
-                for tool_call in delta['tool_calls']:
-                    if tool_call['id'] and tool_call['function']['name']:
-                        tool_id = tool_call['id']
-                        tool_name = tool_call['function']['name']
-                    else:
-                        tool_call['id'] = tool_id
-                        tool_call['function']['name'] = tool_name
-                    if tool_call['type'] is None:
-                        tool_call['type'] = 'function'
-
-            # 跳过空的第一个 chunk（只有 role 没有内容）
-            if chunk_idx == 0 and not delta_content and not reasoning_content and not delta.get('tool_calls'):
-                chunk_idx += 1
-                continue
-            # 构建 MessageChunk - 只包含增量内容
-            chunk_data = {
-                'role': role,
-                'content': delta_content if delta_content else None,
-                'tool_calls': delta.get('tool_calls'),
-                'is_final': bool(finish_reason),
-            }
-
-            # 移除 None 值
-            chunk_data = {k: v for k, v in chunk_data.items() if v is not None}
-
-            yield provider_message.MessageChunk(**chunk_data)
-            chunk_idx += 1
-
-    async def _closure(
-        self,
-        query: pipeline_query.Query,
-        req_messages: list[dict],
-        use_model: requester.RuntimeLLMModel,
-        use_funcs: list[resource_tool.LLMTool] = None,
-        extra_args: dict[str, typing.Any] = {},
-        remove_think: bool = False,
-    ) -> tuple[provider_message.Message, dict]:
-        self.client.api_key = use_model.provider.token_mgr.get_token()
-
-        args = {}
-        args['model'] = use_model.model_entity.name
-
-        if use_funcs:
-            tools = await self.ap.tool_mgr.generate_tools_for_openai(use_funcs)
-
-            if tools:
-                args['tools'] = tools
-
-        # 设置此次请求中的messages
-        messages = req_messages.copy()
-
-        # 检查vision
-        for msg in messages:
-            if 'content' in msg and isinstance(msg['content'], list):
-                for me in msg['content']:
-                    if me['type'] == 'image_base64':
-                        me['image_url'] = {'url': me['image_base64']}
-                        me['type'] = 'image_url'
-                        del me['image_base64']
-
-        args['messages'] = messages
-
-        # 发送请求
-
-        resp = await self._req(args, extra_body=extra_args)
-        # 处理请求结果
-        message = await self._make_msg(resp, remove_think)
-
-        # Extract token usage from response
-        usage_info = {}
-        if hasattr(resp, 'usage') and resp.usage:
-            usage_info['input_tokens'] = resp.usage.prompt_tokens or 0
-            usage_info['output_tokens'] = resp.usage.completion_tokens or 0
-            usage_info['total_tokens'] = resp.usage.total_tokens or 0
-
-        return message, usage_info
-
-    async def invoke_llm(
-        self,
-        query: pipeline_query.Query,
-        model: requester.RuntimeLLMModel,
-        messages: typing.List[provider_message.Message],
-        funcs: typing.List[resource_tool.LLMTool] = None,
-        extra_args: dict[str, typing.Any] = {},
-        remove_think: bool = False,
-    ) -> tuple[provider_message.Message, dict]:
-        """Invoke LLM and return message with usage info"""
-        req_messages = []  # req_messages 仅用于类内，外部同步由 query.messages 进行
-        for m in messages:
-            msg_dict = m.dict(exclude_none=True)
-            content = msg_dict.get('content')
-            if isinstance(content, list):
-                # 检查 content 列表中是否每个部分都是文本
-                if all(isinstance(part, dict) and part.get('type') == 'text' for part in content):
-                    # 将所有文本部分合并为一个字符串
-                    msg_dict['content'] = '\n'.join(part['text'] for part in content)
-            req_messages.append(msg_dict)
-
-        try:
-            msg, usage_info = await self._closure(
-                query=query,
-                req_messages=req_messages,
-                use_model=model,
-                use_funcs=funcs,
-                extra_args=extra_args,
-                remove_think=remove_think,
-            )
-            return msg, usage_info
-        except asyncio.TimeoutError:
-            raise errors.RequesterError('请求超时')
-        except openai.BadRequestError as e:
-            error_message = str(e.message) if hasattr(e, 'message') else str(e)
-            if 'context_length_exceeded' in str(e):
-                raise errors.RequesterError(f'上文过长，请重置会话: {error_message}')
-            else:
-                raise errors.RequesterError(f'请求参数错误: {error_message}')
-        except openai.AuthenticationError as e:
-            error_message = str(e.message) if hasattr(e, 'message') else str(e)
-            raise errors.RequesterError(f'无效的 api-key: {error_message}')
-        except openai.NotFoundError as e:
-            error_message = str(e.message) if hasattr(e, 'message') else str(e)
-            raise errors.RequesterError(f'请求路径错误: {error_message}')
-        except openai.RateLimitError as e:
-            error_message = str(e.message) if hasattr(e, 'message') else str(e)
-            raise errors.RequesterError(f'请求过于频繁或余额不足: {error_message}')
-        except openai.APIConnectionError as e:
-            error_message = f'连接错误: {str(e)}'
-            raise errors.RequesterError(error_message)
-        except openai.APIError as e:
-            error_message = str(e.message) if hasattr(e, 'message') else str(e)
-            raise errors.RequesterError(f'请求错误: {error_message}')
-
-    async def invoke_embedding(
-        self,
-        model: requester.RuntimeEmbeddingModel,
-        input_text: list[str],
-        extra_args: dict[str, typing.Any] = {},
-    ) -> tuple[list[list[float]], dict]:
-        """调用 Embedding API, returns (embeddings, usage_info)"""
-        self.client.api_key = model.provider.token_mgr.get_token()
-
-        args = {
-            'model': model.model_entity.name,
-            'input': input_text,
-        }
-
-        if model.model_entity.extra_args:
-            args.update(model.model_entity.extra_args)
-
-        args.update(extra_args)
-
-        try:
-            resp = await self.client.embeddings.create(**args)
-
-            # Extract usage info
-            usage_info = {}
-            if hasattr(resp, 'usage') and resp.usage:
-                usage_info['prompt_tokens'] = resp.usage.prompt_tokens or 0
-                usage_info['total_tokens'] = resp.usage.total_tokens or 0
-
-            return [d.embedding for d in resp.data], usage_info
-        except asyncio.TimeoutError:
-            raise errors.RequesterError('请求超时')
-        except openai.BadRequestError as e:
-            raise errors.RequesterError(f'请求参数错误: {e.message}')
-
-    async def invoke_llm_stream(
-        self,
-        query: pipeline_query.Query,
-        model: requester.RuntimeLLMModel,
-        messages: typing.List[provider_message.Message],
-        funcs: typing.List[resource_tool.LLMTool] = None,
-        extra_args: dict[str, typing.Any] = {},
-        remove_think: bool = False,
-    ) -> provider_message.MessageChunk:
-        req_messages = []  # req_messages 仅用于类内，外部同步由 query.messages 进行
-        for m in messages:
-            msg_dict = m.dict(exclude_none=True)
-            content = msg_dict.get('content')
-            if isinstance(content, list):
-                # 检查 content 列表中是否每个部分都是文本
-                if all(isinstance(part, dict) and part.get('type') == 'text' for part in content):
-                    # 将所有文本部分合并为一个字符串
-                    msg_dict['content'] = '\n'.join(part['text'] for part in content)
-            req_messages.append(msg_dict)
-
-        try:
-            async for item in self._closure_stream(
-                query=query,
-                req_messages=req_messages,
-                use_model=model,
-                use_funcs=funcs,
-                extra_args=extra_args,
-                remove_think=remove_think,
-            ):
-                yield item
-
-        except asyncio.TimeoutError:
-            raise errors.RequesterError('请求超时')
-        except openai.BadRequestError as e:
-            if 'context_length_exceeded' in e.message:
-                raise errors.RequesterError(f'上文过长，请重置会话: {e.message}')
-            else:
-                raise errors.RequesterError(f'请求参数错误: {e.message}')
-        except openai.AuthenticationError as e:
-            raise errors.RequesterError(f'无效的 api-key: {e.message}')
-        except openai.NotFoundError as e:
-            raise errors.RequesterError(f'请求路径错误: {e.message}')
-        except openai.RateLimitError as e:
-            raise errors.RequesterError(f'请求过于频繁或余额不足: {e.message}')
-        except openai.APIError as e:
-            raise errors.RequesterError(f'请求错误: {e.message}')
-
-    async def invoke_rerank(
-        self,
-        model: requester.RuntimeRerankModel,
-        query: str,
-        documents: typing.List[str],
-        extra_args: dict[str, typing.Any] = {},
-    ) -> typing.List[dict]:
-        """Standard /rerank endpoint (Jina/Cohere/SiliconFlow/Voyage/DashScope compatible)
-
-        Supports extra_args from model.extra_args:
-        - rerank_url: full URL override (e.g. "https://dashscope.aliyuncs.com/compatible-api/v1/reranks")
-        - rerank_path: path override appended to base_url (e.g. "reranks" instead of default "rerank")
-        - Any other fields are merged into the request payload.
-        """
-        api_key = model.provider.token_mgr.get_token()
-        base_url = self.requester_cfg.get('base_url', '').rstrip('/')
-        timeout = self.requester_cfg.get('timeout', 120)
-
-        merged_args = {}
-        if model.model_entity.extra_args:
-            merged_args.update(model.model_entity.extra_args)
-        if extra_args:
-            merged_args.update(extra_args)
-
-        rerank_url = merged_args.pop('rerank_url', None)
-        rerank_path = merged_args.pop('rerank_path', 'rerank')
-        if not rerank_url:
-            rerank_url = f'{base_url}/{rerank_path}'
-
-        headers = {
-            'Content-Type': 'application/json',
-            'Authorization': f'Bearer {api_key}',
-        }
-
-        payload = {
-            'model': model.model_entity.name,
-            'query': query,
-            'documents': documents[:64],
-            'top_n': min(len(documents), 64),
-        }
-
-        if merged_args:
-            payload.update(merged_args)
-
-        try:
-            async with httpx.AsyncClient(trust_env=True, timeout=timeout) as client:
-                resp = await client.post(rerank_url, headers=headers, json=payload)
-                resp.raise_for_status()
-                data = resp.json()
-
-            results = self._parse_rerank_response(data)
-
-            if results:
-                scores = [r.get('relevance_score', 0.0) for r in results]
-                min_score = min(scores)
-                max_score = max(scores)
-                if max_score - min_score > 1e-6:
-                    for r in results:
-                        r['relevance_score'] = (r['relevance_score'] - min_score) / (max_score - min_score)
-
-            return results
-        except httpx.HTTPStatusError as e:
-            raise errors.RequesterError(f'Rerank request failed: {e.response.status_code} - {e.response.text}')
-        except httpx.TimeoutException:
-            raise errors.RequesterError('Rerank request timed out')
-        except Exception as e:
-            raise errors.RequesterError(f'Rerank request error: {str(e)}')
-
-    @staticmethod
-    def _parse_rerank_response(data: dict) -> typing.List[dict]:
-        """Parse rerank response from various providers.
-
-        Handles:
-        - Jina/Cohere/SiliconFlow: {"results": [{"index", "relevance_score"}]}
-        - Voyage AI: {"data": [{"index", "relevance_score"}]}
-        - DashScope: {"output": {"results": [{"index", "relevance_score"}]}}
-        """
-        if 'results' in data:
-            return data['results']
-        if 'data' in data:
-            return data['data']
-        if 'output' in data and isinstance(data['output'], dict):
-            return data['output'].get('results', [])
-        return []

src/langbot/pkg/provider/modelmgr/requesters/chatcmpl.yaml

@@ -7,6 +7,7 @@ metadata:
     zh_Hans: OpenAI
   icon: openai.svg
 spec:
+  litellm_provider: openai
   config:
   - name: base_url
     label:

src/langbot/pkg/provider/modelmgr/requesters/coherererank.yaml

@@ -7,6 +7,7 @@ metadata:
     zh_Hans: Cohere
   icon: cohere.svg
 spec:
+  litellm_provider: cohere
   config:
   - name: base_url
     label:

src/langbot/pkg/provider/modelmgr/requesters/compsharechatcmpl.py

@@ -1,17 +0,0 @@
-from __future__ import annotations
-
-import typing
-import openai
-
-from . import chatcmpl
-
-
-class CompShareChatCompletions(chatcmpl.OpenAIChatCompletions):
-    """CompShare ChatCompletion API 请求器"""
-
-    client: openai.AsyncClient
-
-    default_config: dict[str, typing.Any] = {
-        'base_url': 'https://api.modelverse.cn/v1',
-        'timeout': 120,
-    }

src/langbot/pkg/provider/modelmgr/requesters/compsharechatcmpl.yaml

@@ -7,6 +7,7 @@ metadata:
     zh_Hans: 优云智算
   icon: compshare.png
 spec:
+  litellm_provider: openai
   config:
   - name: base_url
     label:
@@ -24,6 +25,8 @@ spec:
     default: 120
   support_type:
   - llm
+  - text-embedding
+  - rerank
   provider_category: maas
 execution:
   python:

src/langbot/pkg/provider/modelmgr/requesters/deepseekchatcmpl.py

@@ -1,67 +0,0 @@
-from __future__ import annotations
-
-import typing
-
-from . import chatcmpl
-from .. import errors, requester
-import langbot_plugin.api.entities.builtin.resource.tool as resource_tool
-import langbot_plugin.api.entities.builtin.pipeline.query as pipeline_query
-import langbot_plugin.api.entities.builtin.provider.message as provider_message
-
-
-class DeepseekChatCompletions(chatcmpl.OpenAIChatCompletions):
-    """Deepseek ChatCompletion API 请求器"""
-
-    default_config: dict[str, typing.Any] = {
-        'base_url': 'https://api.deepseek.com',
-        'timeout': 120,
-    }
-
-    async def _closure(
-        self,
-        query: pipeline_query.Query,
-        req_messages: list[dict],
-        use_model: requester.RuntimeLLMModel,
-        use_funcs: list[resource_tool.LLMTool] = None,
-        extra_args: dict[str, typing.Any] = {},
-        remove_think: bool = False,
-    ) -> tuple[provider_message.Message, dict]:
-        self.client.api_key = use_model.provider.token_mgr.get_token()
-
-        args = {}
-        args['model'] = use_model.model_entity.name
-
-        if use_funcs:
-            tools = await self.ap.tool_mgr.generate_tools_for_openai(use_funcs)
-
-            if tools:
-                args['tools'] = tools
-
-        # 设置此次请求中的messages
-        messages = req_messages
-
-        # deepseek 不支持多模态，把content都转换成纯文字
-        for m in messages:
-            if 'content' in m and isinstance(m['content'], list):
-                m['content'] = ' '.join([c['text'] for c in m['content'] if 'text' in c])
-
-        args['messages'] = messages
-
-        # 发送请求
-        resp = await self._req(args, extra_body=extra_args)
-
-        # print(resp)
-
-        if resp is None:
-            raise errors.RequesterError('接口返回为空，请确定模型提供商服务是否正常')
-        # 处理请求结果
-        message = await self._make_msg(resp, remove_think)
-
-        # Extract token usage from response
-        usage_info = {}
-        if hasattr(resp, 'usage') and resp.usage:
-            usage_info['input_tokens'] = resp.usage.prompt_tokens or 0
-            usage_info['output_tokens'] = resp.usage.completion_tokens or 0
-            usage_info['total_tokens'] = resp.usage.total_tokens or 0
-
-        return message, usage_info

src/langbot/pkg/provider/modelmgr/requesters/deepseekchatcmpl.yaml

@@ -7,6 +7,7 @@ metadata:
     zh_Hans: DeepSeek
   icon: deepseek.svg
 spec:
+  litellm_provider: deepseek
   config:
   - name: base_url
     label:
@@ -24,6 +25,8 @@ spec:
     default: 120
   support_type:
   - llm
+  - text-embedding
+  - rerank
   provider_category: manufacturer
 execution:
   python:

src/langbot/pkg/provider/modelmgr/requesters/doubao.svg

@@ -0,0 +1,4 @@
+<svg width="60" height="50" viewBox="0 0 60 50" xmlns="http://www.w3.org/2000/svg">
+  <rect width="60" height="50" rx="8" fill="#3B82F6"/>
+  <text x="30" y="32" font-family="Arial, sans-serif" font-size="12" font-weight="bold" fill="white" text-anchor="middle">豆包</text>
+</svg>

src/langbot/pkg/provider/modelmgr/requesters/doubaochatcmpl.yaml

@@ -0,0 +1,30 @@
+apiVersion: v1
+kind: LLMAPIRequester
+metadata:
+  name: doubao-chat-completions
+  label:
+    en_US: ByteDance Doubao
+    zh_Hans: 字节豆包
+  icon: doubao.svg
+spec:
+  litellm_provider: openai
+  config:
+  - name: base_url
+    label:
+      en_US: Base URL
+      zh_Hans: 基础 URL
+    type: string
+    required: true
+    default: https://ark.cn-beijing.volces.com/api/v3
+  - name: timeout
+    label:
+      en_US: Timeout
+      zh_Hans: 超时时间
+    type: integer
+    required: true
+    default: 120
+  support_type:
+  - llm
+  - text-embedding
+  - rerank
+  provider_category: manufacturer

src/langbot/pkg/provider/modelmgr/requesters/geminichatcmpl.py

@@ -1,205 +0,0 @@
-from __future__ import annotations
-
-import typing
-import httpx
-
-from . import chatcmpl
-
-import uuid
-
-from .. import requester
-import langbot_plugin.api.entities.builtin.provider.message as provider_message
-import langbot_plugin.api.entities.builtin.pipeline.query as pipeline_query
-import langbot_plugin.api.entities.builtin.resource.tool as resource_tool
-
-
-class GeminiChatCompletions(chatcmpl.OpenAIChatCompletions):
-    """Google Gemini API 请求器"""
-
-    default_config: dict[str, typing.Any] = {
-        'base_url': 'https://generativelanguage.googleapis.com/v1beta/openai',
-        'timeout': 120,
-    }
-
-    async def scan_models(self, api_key: str | None = None) -> dict[str, typing.Any]:
-        models_url = 'https://generativelanguage.googleapis.com/v1beta/models'
-        params = {'key': api_key} if api_key else {}
-
-        all_models: list[dict[str, typing.Any]] = []
-        next_page_token = ''
-        last_payload: dict[str, typing.Any] = {}
-
-        async with httpx.AsyncClient(trust_env=True, timeout=self.requester_cfg['timeout']) as client:
-            while True:
-                request_params = dict(params)
-                if next_page_token:
-                    request_params['pageToken'] = next_page_token
-
-                response = await client.get(models_url, params=request_params)
-                response.raise_for_status()
-                payload = response.json()
-                last_payload = payload
-
-                for item in payload.get('models', []):
-                    model_name = item.get('name', '')
-                    model_id = model_name.replace('models/', '', 1)
-                    if not model_id:
-                        continue
-
-                    supported_methods = item.get('supportedGenerationMethods', []) or []
-                    if 'embedContent' in supported_methods and 'generateContent' not in supported_methods:
-                        model_type = 'embedding'
-                    else:
-                        model_type = 'llm'
-
-                    all_models.append(
-                        {
-                            'id': model_id,
-                            'name': model_id,
-                            'type': model_type,
-                            'abilities': self._infer_model_abilities(item, model_id),
-                            'display_name': item.get('displayName') or None,
-                            'description': item.get('description') or None,
-                            'context_length': item.get('inputTokenLimit'),
-                            'input_modalities': self._normalize_modalities(item.get('inputModalities')),
-                            'output_modalities': self._normalize_modalities(item.get('outputModalities')),
-                        }
-                    )
-
-                next_page_token = payload.get('nextPageToken', '')
-                if not next_page_token:
-                    break
-
-        all_models.sort(key=lambda item: (item['type'] != 'llm', item['name'].lower()))
-        return {
-            'models': all_models,
-            'debug': {
-                'request': {
-                    'method': 'GET',
-                    'url': models_url,
-                    'query': {'key': self._mask_api_key(api_key)} if api_key else {},
-                },
-                'response': last_payload,
-            },
-        }
-
-    async def _closure_stream(
-        self,
-        query: pipeline_query.Query,
-        req_messages: list[dict],
-        use_model: requester.RuntimeLLMModel,
-        use_funcs: list[resource_tool.LLMTool] = None,
-        extra_args: dict[str, typing.Any] = {},
-        remove_think: bool = False,
-    ) -> provider_message.MessageChunk:
-        self.client.api_key = use_model.provider.token_mgr.get_token()
-
-        args = {}
-        args['model'] = use_model.model_entity.name
-
-        if use_funcs:
-            tools = await self.ap.tool_mgr.generate_tools_for_openai(use_funcs)
-            if tools:
-                args['tools'] = tools
-
-        # 设置此次请求中的messages
-        messages = req_messages.copy()
-
-        # 检查vision
-        for msg in messages:
-            if 'content' in msg and isinstance(msg['content'], list):
-                for me in msg['content']:
-                    if me['type'] == 'image_base64':
-                        me['image_url'] = {'url': me['image_base64']}
-                        me['type'] = 'image_url'
-                        del me['image_base64']
-
-        args['messages'] = messages
-        args['stream'] = True
-
-        # 流式处理状态
-        # tool_calls_map: dict[str, provider_message.ToolCall] = {}
-        chunk_idx = 0
-        thinking_started = False
-        thinking_ended = False
-        role = 'assistant'  # 默认角色
-        tool_id = ''
-        tool_name = ''
-        # accumulated_reasoning = ''  # 仅用于判断何时结束思维链
-
-        async for chunk in self._req_stream(args, extra_body=extra_args):
-            # 解析 chunk 数据
-
-            if hasattr(chunk, 'choices') and chunk.choices:
-                choice = chunk.choices[0]
-                delta = choice.delta.model_dump() if hasattr(choice, 'delta') else {}
-
-                finish_reason = getattr(choice, 'finish_reason', None)
-            else:
-                delta = {}
-                finish_reason = None
-            # 从第一个 chunk 获取 role，后续使用这个 role
-            if 'role' in delta and delta['role']:
-                role = delta['role']
-
-            # 获取增量内容
-            delta_content = delta.get('content', '')
-            reasoning_content = delta.get('reasoning_content', '')
-
-            # 处理 reasoning_content
-            if reasoning_content:
-                # accumulated_reasoning += reasoning_content
-                # 如果设置了 remove_think，跳过 reasoning_content
-                if remove_think:
-                    chunk_idx += 1
-                    continue
-
-                # 第一次出现 reasoning_content，添加 <think> 开始标签
-                if not thinking_started:
-                    thinking_started = True
-                    delta_content = '<think>\n' + reasoning_content
-                else:
-                    # 继续输出 reasoning_content
-                    delta_content = reasoning_content
-            elif thinking_started and not thinking_ended and delta_content:
-                # reasoning_content 结束，normal content 开始，添加 </think> 结束标签
-                thinking_ended = True
-                delta_content = '\n</think>\n' + delta_content
-
-            # 处理 content 中已有的 <think> 标签（如果需要移除）
-            # if delta_content and remove_think and '<think>' in delta_content:
-            #     import re
-            #
-            #     # 移除 <think> 标签及其内容
-            #     delta_content = re.sub(r'<think>.*?</think>', '', delta_content, flags=re.DOTALL)
-
-            # 处理工具调用增量
-            # delta_tool_calls = None
-            if delta.get('tool_calls'):
-                for tool_call in delta['tool_calls']:
-                    if tool_call['id'] == '' and tool_id == '':
-                        tool_id = str(uuid.uuid4())
-                    if tool_call['function']['name']:
-                        tool_name = tool_call['function']['name']
-                    tool_call['id'] = tool_id
-                    tool_call['function']['name'] = tool_name
-                    if tool_call['type'] is None:
-                        tool_call['type'] = 'function'
-
-            # 跳过空的第一个 chunk（只有 role 没有内容）
-            if chunk_idx == 0 and not delta_content and not reasoning_content and not delta.get('tool_calls'):
-                chunk_idx += 1
-                continue
-            # 构建 MessageChunk - 只包含增量内容
-            chunk_data = {
-                'role': role,
-                'content': delta_content if delta_content else None,
-                'tool_calls': delta.get('tool_calls'),
-                'is_final': bool(finish_reason),
-            }
-
-            # 移除 None 值
-            chunk_data = {k: v for k, v in chunk_data.items() if v is not None}
-
-            yield provider_message.MessageChunk(**chunk_data)
-            chunk_idx += 1

src/langbot/pkg/provider/modelmgr/requesters/geminichatcmpl.yaml

@@ -7,6 +7,7 @@ metadata:
     zh_Hans: Google Gemini
   icon: gemini.svg
 spec:
+  litellm_provider: gemini
   config:
   - name: base_url
     label:
@@ -24,6 +25,8 @@ spec:
     default: 120
   support_type:
   - llm
+  - text-embedding
+  - rerank
   provider_category: manufacturer
 execution:
   python:

src/langbot/pkg/provider/modelmgr/requesters/giteeaichatcmpl.py

@@ -1,15 +0,0 @@
-from __future__ import annotations
-
-
-import typing
-
-from . import ppiochatcmpl
-
-
-class GiteeAIChatCompletions(ppiochatcmpl.PPIOChatCompletions):
-    """Gitee AI ChatCompletions API 请求器"""
-
-    default_config: dict[str, typing.Any] = {
-        'base_url': 'https://ai.gitee.com/v1',
-        'timeout': 120,
-    }

src/langbot/pkg/provider/modelmgr/requesters/giteeaichatcmpl.yaml

@@ -7,6 +7,7 @@ metadata:
     zh_Hans: Gitee AI
   icon: giteeai.svg
 spec:
+  litellm_provider: openai
   config:
   - name: base_url
     label:

src/langbot/pkg/provider/modelmgr/requesters/groq.svg

@@ -0,0 +1,4 @@
+<svg width="60" height="50" viewBox="0 0 60 50" xmlns="http://www.w3.org/2000/svg">
+  <rect width="60" height="50" rx="8" fill="#F97316"/>
+  <text x="30" y="32" font-family="Arial, sans-serif" font-size="14" font-weight="bold" fill="white" text-anchor="middle">Groq</text>
+</svg>

src/langbot/pkg/provider/modelmgr/requesters/groqchatcmpl.yaml

@@ -0,0 +1,30 @@
+apiVersion: v1
+kind: LLMAPIRequester
+metadata:
+  name: groq-chat-completions
+  label:
+    en_US: Groq
+    zh_Hans: Groq
+  icon: groq.svg
+spec:
+  litellm_provider: groq
+  config:
+  - name: base_url
+    label:
+      en_US: Base URL
+      zh_Hans: 基础 URL
+    type: string
+    required: true
+    default: https://api.groq.com/openai/v1
+  - name: timeout
+    label:
+      en_US: Timeout
+      zh_Hans: 超时时间
+    type: integer
+    required: true
+    default: 120
+  support_type:
+  - llm
+  - text-embedding
+  - rerank
+  provider_category: manufacturer

src/langbot/pkg/provider/modelmgr/requesters/iflytek.svg

@@ -0,0 +1,5 @@
+<svg width="60" height="50" viewBox="0 0 60 50" xmlns="http://www.w3.org/2000/svg">
+  <rect width="60" height="50" rx="8" fill="#0066FF"/>
+  <text x="30" y="28" font-family="Arial, sans-serif" font-size="10" font-weight="bold" fill="white" text-anchor="middle">iFlytek</text>
+  <text x="30" y="40" font-family="Arial, sans-serif" font-size="8" fill="white" text-anchor="middle">Spark</text>
+</svg>

src/langbot/pkg/provider/modelmgr/requesters/iflytekchatcmpl.yaml

@@ -0,0 +1,30 @@
+apiVersion: v1
+kind: LLMAPIRequester
+metadata:
+  name: iflytek-chat-completions
+  label:
+    en_US: iFlytek Spark
+    zh_Hans: 讯飞星火
+  icon: iflytek.svg
+spec:
+  litellm_provider: openai
+  config:
+  - name: base_url
+    label:
+      en_US: Base URL
+      zh_Hans: 基础 URL
+    type: string
+    required: true
+    default: https://spark-api-open.xf-yun.com/v1
+  - name: timeout
+    label:
+      en_US: Timeout
+      zh_Hans: 超时时间
+    type: integer
+    required: true
+    default: 120
+  support_type:
+  - llm
+  - text-embedding
+  - rerank
+  provider_category: manufacturer

src/langbot/pkg/provider/modelmgr/requesters/jiekouaichatcmpl.py

@@ -1,208 +0,0 @@
-from __future__ import annotations
-
-import openai
-import typing
-
-from . import chatcmpl
-from .. import requester
-import openai.types.chat.chat_completion as chat_completion
-import re
-import langbot_plugin.api.entities.builtin.provider.message as provider_message
-import langbot_plugin.api.entities.builtin.pipeline.query as pipeline_query
-import langbot_plugin.api.entities.builtin.resource.tool as resource_tool
-
-
-class JieKouAIChatCompletions(chatcmpl.OpenAIChatCompletions):
-    """接口 AI ChatCompletion API 请求器"""
-
-    client: openai.AsyncClient
-
-    default_config: dict[str, typing.Any] = {
-        'base_url': 'https://api.jiekou.ai/openai',
-        'timeout': 120,
-    }
-
-    is_think: bool = False
-
-    async def _make_msg(
-        self,
-        chat_completion: chat_completion.ChatCompletion,
-        remove_think: bool,
-    ) -> provider_message.Message:
-        chatcmpl_message = chat_completion.choices[0].message.model_dump()
-        # print(chatcmpl_message.keys(), chatcmpl_message.values())
-
-        # 确保 role 字段存在且不为 None
-        if 'role' not in chatcmpl_message or chatcmpl_message['role'] is None:
-            chatcmpl_message['role'] = 'assistant'
-
-        reasoning_content = chatcmpl_message['reasoning_content'] if 'reasoning_content' in chatcmpl_message else None
-
-        # deepseek的reasoner模型
-        chatcmpl_message['content'] = await self._process_thinking_content(
-            chatcmpl_message['content'], reasoning_content, remove_think
-        )
-
-        # 移除 reasoning_content 字段，避免传递给 Message
-        if 'reasoning_content' in chatcmpl_message:
-            del chatcmpl_message['reasoning_content']
-
-        message = provider_message.Message(**chatcmpl_message)
-
-        return message
-
-    async def _process_thinking_content(
-        self,
-        content: str,
-        reasoning_content: str = None,
-        remove_think: bool = False,
-    ) -> tuple[str, str]:
-        """处理思维链内容
-
-        Args:
-            content: 原始内容
-            reasoning_content: reasoning_content 字段内容
-            remove_think: 是否移除思维链
-
-        Returns:
-            处理后的内容
-        """
-        if remove_think:
-            content = re.sub(r'<think>.*?</think>', '', content, flags=re.DOTALL)
-        else:
-            if reasoning_content is not None:
-                content = '<think>\n' + reasoning_content + '\n</think>\n' + content
-        return content
-
-    async def _make_msg_chunk(
-        self,
-        delta: dict[str, typing.Any],
-        idx: int,
-    ) -> provider_message.MessageChunk:
-        # 处理流式chunk和完整响应的差异
-        # print(chat_completion.choices[0])
-
-        # 确保 role 字段存在且不为 None
-        if 'role' not in delta or delta['role'] is None:
-            delta['role'] = 'assistant'
-
-        reasoning_content = delta['reasoning_content'] if 'reasoning_content' in delta else None
-
-        delta['content'] = '' if delta['content'] is None else delta['content']
-        # print(reasoning_content)
-
-        # deepseek的reasoner模型
-
-        if reasoning_content is not None:
-            delta['content'] += reasoning_content
-
-        message = provider_message.MessageChunk(**delta)
-
-        return message
-
-    async def _closure_stream(
-        self,
-        query: pipeline_query.Query,
-        req_messages: list[dict],
-        use_model: requester.RuntimeLLMModel,
-        use_funcs: list[resource_tool.LLMTool] = None,
-        extra_args: dict[str, typing.Any] = {},
-        remove_think: bool = False,
-    ) -> provider_message.Message | typing.AsyncGenerator[provider_message.MessageChunk, None]:
-        self.client.api_key = use_model.provider.token_mgr.get_token()
-
-        args = {}
-        args['model'] = use_model.model_entity.name
-
-        if use_funcs:
-            tools = await self.ap.tool_mgr.generate_tools_for_openai(use_funcs)
-
-            if tools:
-                args['tools'] = tools
-
-        # 设置此次请求中的messages
-        messages = req_messages.copy()
-
-        # 检查vision
-        for msg in messages:
-            if 'content' in msg and isinstance(msg['content'], list):
-                for me in msg['content']:
-                    if me['type'] == 'image_base64':
-                        me['image_url'] = {'url': me['image_base64']}
-                        me['type'] = 'image_url'
-                        del me['image_base64']
-
-        args['messages'] = messages
-        args['stream'] = True
-
-        # tool_calls_map: dict[str, provider_message.ToolCall] = {}
-        chunk_idx = 0
-        thinking_started = False
-        thinking_ended = False
-        role = 'assistant'  # 默认角色
-        async for chunk in self._req_stream(args, extra_body=extra_args):
-            # 解析 chunk 数据
-            if hasattr(chunk, 'choices') and chunk.choices:
-                choice = chunk.choices[0]
-                delta = choice.delta.model_dump() if hasattr(choice, 'delta') else {}
-                finish_reason = getattr(choice, 'finish_reason', None)
-            else:
-                delta = {}
-                finish_reason = None
-
-            # 从第一个 chunk 获取 role，后续使用这个 role
-            if 'role' in delta and delta['role']:
-                role = delta['role']
-
-            # 获取增量内容
-            delta_content = delta.get('content', '')
-            # reasoning_content = delta.get('reasoning_content', '')
-
-            if remove_think:
-                if delta['content'] is not None:
-                    if '<think>' in delta['content'] and not thinking_started and not thinking_ended:
-                        thinking_started = True
-                        continue
-                    elif delta['content'] == r'</think>' and not thinking_ended:
-                        thinking_ended = True
-                        continue
-                    elif thinking_ended and delta['content'] == '\n\n' and thinking_started:
-                        thinking_started = False
-                        continue
-                    elif thinking_started and not thinking_ended:
-                        continue
-
-            # delta_tool_calls = None
-            if delta.get('tool_calls'):
-                for tool_call in delta['tool_calls']:
-                    if tool_call['id'] and tool_call['function']['name']:
-                        tool_id = tool_call['id']
-                        tool_name = tool_call['function']['name']
-
-                    if tool_call['id'] is None:
-                        tool_call['id'] = tool_id
-                    if tool_call['function']['name'] is None:
-                        tool_call['function']['name'] = tool_name
-                    if tool_call['function']['arguments'] is None:
-                        tool_call['function']['arguments'] = ''
-                    if tool_call['type'] is None:
-                        tool_call['type'] = 'function'
-
-            # 跳过空的第一个 chunk（只有 role 没有内容）
-            if chunk_idx == 0 and not delta_content and not delta.get('tool_calls'):
-                chunk_idx += 1
-                continue
-
-            # 构建 MessageChunk - 只包含增量内容
-            chunk_data = {
-                'role': role,
-                'content': delta_content if delta_content else None,
-                'tool_calls': delta.get('tool_calls'),
-                'is_final': bool(finish_reason),
-            }
-
-            # 移除 None 值
-            chunk_data = {k: v for k, v in chunk_data.items() if v is not None}
-
-            yield provider_message.MessageChunk(**chunk_data)
-            chunk_idx += 1

src/langbot/pkg/provider/modelmgr/requesters/jiekouaichatcmpl.yaml

@@ -7,6 +7,7 @@ metadata:
     zh_Hans: 接口 AI
   icon: jiekouai.png
 spec:
+  litellm_provider: openai
   config:
   - name: base_url
     label:

src/langbot/pkg/provider/modelmgr/requesters/jinarerank.yaml

@@ -7,6 +7,7 @@ metadata:
     zh_Hans: Jina
   icon: jina.svg
 spec:
+  litellm_provider: openai
   config:
   - name: base_url
     label:

src/langbot/pkg/provider/modelmgr/requesters/litellmchat.py

@@ -0,0 +1,644 @@
+"""LiteLLM unified requester for chat, embedding, and rerank."""
+
+from __future__ import annotations
+
+import typing
+
+import litellm
+from litellm import acompletion, aembedding, arerank
+
+from .. import errors, requester
+import langbot_plugin.api.entities.builtin.resource.tool as resource_tool
+import langbot_plugin.api.entities.builtin.pipeline.query as pipeline_query
+import langbot_plugin.api.entities.builtin.provider.message as provider_message
+
+
+class LiteLLMRequester(requester.ProviderAPIRequester):
+    """LiteLLM unified API requester supporting chat, embedding, and rerank."""
+
+    _EMBEDDING_MODEL_HINTS = ('embedding', 'embed', 'bge-', 'e5-', 'm3e', 'gte-', 'text-embedding')
+    _RERANK_MODEL_HINTS = ('rerank', 're-rank', 're_rank')
+
+    default_config: dict[str, typing.Any] = {
+        'base_url': '',
+        'timeout': 120,
+        'custom_llm_provider': '',
+        'drop_params': False,
+        'num_retries': 0,
+        'api_version': '',
+    }
+
+    async def initialize(self):
+        """Initialize LiteLLM client settings."""
+        # LiteLLM doesn't require explicit client initialization
+        # Configuration is passed per-request via litellm params
+        pass
+
+    def _build_litellm_model_name(self, model_name: str, custom_llm_provider: str | None = None) -> str:
+        """Build LiteLLM model name with provider prefix if needed."""
+        provider = custom_llm_provider or self.requester_cfg.get('custom_llm_provider', '')
+        if provider:
+            # LiteLLM format: provider/model_name
+            if model_name.startswith(f'{provider}/'):
+                return model_name
+            return f'{provider}/{model_name}'
+        # If no custom provider, assume model_name already includes prefix or is OpenAI-compatible
+        return model_name
+
+    def _get_custom_llm_provider(self) -> str | None:
+        return self.requester_cfg.get('custom_llm_provider') or None
+
+    def _safe_litellm_bool_helper(self, helper_name: str, model_name: str) -> bool:
+        """Call a LiteLLM boolean capability helper without letting metadata gaps fail requests."""
+        helper = getattr(litellm, helper_name, None)
+        if not callable(helper):
+            return False
+
+        provider = self._get_custom_llm_provider()
+        candidates: list[tuple[str, str | None]] = [(model_name, provider)]
+        litellm_model_name = self._build_litellm_model_name(model_name)
+        if litellm_model_name != model_name:
+            candidates.append((litellm_model_name, None))
+        for metadata_provider in self._metadata_provider_candidates(model_name):
+            candidates.append((f'{metadata_provider}/{model_name}', None))
+
+        tried_candidates: set[tuple[str, str | None]] = set()
+        for candidate_model, candidate_provider in candidates:
+            candidate_key = (candidate_model, candidate_provider)
+            if candidate_key in tried_candidates:
+                continue
+            tried_candidates.add(candidate_key)
+            try:
+                if bool(helper(model=candidate_model, custom_llm_provider=candidate_provider)):
+                    return True
+            except Exception:
+                continue
+        return False
+
+    def _context_length_from_scan_payload(self, model_payload: dict[str, typing.Any] | None) -> int | None:
+        if not model_payload:
+            return None
+
+        for field_name in ('context_length', 'context_window', 'max_context_length'):
+            value = model_payload.get(field_name)
+            if isinstance(value, bool):
+                continue
+            if isinstance(value, int) and value > 0:
+                return value
+            if isinstance(value, str) and value.isdigit():
+                parsed_value = int(value)
+                if parsed_value > 0:
+                    return parsed_value
+        return None
+
+    def _metadata_provider_candidates(self, model_name: str) -> list[str]:
+        normalized_model_name = (model_name or '').lower()
+        candidates = []
+        if normalized_model_name.startswith(('moonshot-', 'kimi-')):
+            candidates.append('moonshot')
+        if normalized_model_name.startswith('deepseek-'):
+            candidates.append('deepseek')
+
+        base_url = self.requester_cfg.get('base_url', '').lower()
+        if 'moonshot' in base_url:
+            candidates.append('moonshot')
+        if 'deepseek' in base_url:
+            candidates.append('deepseek')
+
+        deduped_candidates = []
+        for candidate in candidates:
+            if candidate not in deduped_candidates:
+                deduped_candidates.append(candidate)
+        return deduped_candidates
+
+    def _known_context_length_fallback(self, model_name: str) -> int | None:
+        normalized_model_name = (model_name or '').lower()
+        if normalized_model_name.startswith('deepseek-v4-'):
+            return 1_000_000
+        if normalized_model_name.startswith(('kimi-k2.5', 'kimi-k2.6')):
+            return 256 * 1024
+        if normalized_model_name.startswith('moonshot-v1-8k'):
+            return 8 * 1024
+        if normalized_model_name.startswith('moonshot-v1-32k'):
+            return 32 * 1024
+        if normalized_model_name.startswith('moonshot-v1-128k') or normalized_model_name == 'moonshot-v1-auto':
+            return 128 * 1024
+        return None
+
+    def _safe_context_length(self, model_name: str) -> int | None:
+        helper = getattr(litellm, 'get_max_tokens', None)
+        if not callable(helper):
+            return self._known_context_length_fallback(model_name)
+
+        candidates = [model_name]
+        litellm_model_name = self._build_litellm_model_name(model_name)
+        if litellm_model_name != model_name:
+            candidates.append(litellm_model_name)
+        for provider in self._metadata_provider_candidates(model_name):
+            candidates.append(f'{provider}/{model_name}')
+
+        tried_candidates = []
+        for candidate in candidates:
+            if candidate in tried_candidates:
+                continue
+            tried_candidates.append(candidate)
+            try:
+                max_tokens = helper(candidate)
+            except Exception:
+                continue
+            if isinstance(max_tokens, int) and max_tokens > 0:
+                return max_tokens
+        return self._known_context_length_fallback(model_name)
+
+    def _supports_function_calling(self, model_name: str) -> bool:
+        return self._safe_litellm_bool_helper('supports_function_calling', model_name)
+
+    def _supports_vision(self, model_name: str) -> bool:
+        return self._safe_litellm_bool_helper('supports_vision', model_name)
+
+    def _infer_model_type(self, model_id: str) -> str:
+        normalized_id = (model_id or '').lower()
+        if any(kw in normalized_id for kw in self._RERANK_MODEL_HINTS):
+            return 'rerank'
+        if any(kw in normalized_id for kw in self._EMBEDDING_MODEL_HINTS):
+            return 'embedding'
+        return 'llm'
+
+    def _enrich_scanned_model(
+        self,
+        model_id: str,
+        model_payload: dict[str, typing.Any] | None = None,
+    ) -> dict[str, typing.Any]:
+        model_type = self._infer_model_type(model_id)
+        scanned_model: dict[str, typing.Any] = {
+            'id': model_id,
+            'name': model_id,
+            'type': model_type,
+        }
+
+        if model_type == 'llm':
+            abilities = []
+            if self._supports_function_calling(model_id):
+                abilities.append('func_call')
+            supports_provider_reported_vision = bool(
+                model_payload
+                and (model_payload.get('supports_image_in') is True or model_payload.get('supports_vision') is True)
+            )
+            if supports_provider_reported_vision or self._supports_vision(model_id):
+                abilities.append('vision')
+            scanned_model['abilities'] = abilities
+
+            context_length = self._context_length_from_scan_payload(model_payload)
+            if context_length is None:
+                context_length = self._safe_context_length(model_id)
+            if context_length is not None:
+                scanned_model['context_length'] = context_length
+
+        return scanned_model
+
+    def _convert_messages(self, messages: typing.List[provider_message.Message]) -> list[dict]:
+        """Convert LangBot messages to LiteLLM/OpenAI format."""
+        req_messages = []
+        for m in messages:
+            msg_dict = m.dict(exclude_none=True)
+            content = msg_dict.get('content')
+
+            if isinstance(content, list):
+                for part in content:
+                    if isinstance(part, dict) and part.get('type') == 'image_base64':
+                        part['image_url'] = {'url': part['image_base64']}
+                        part['type'] = 'image_url'
+                        del part['image_base64']
+
+            req_messages.append(msg_dict)
+
+        return req_messages
+
+    def _process_thinking_content(self, content: str, reasoning_content: str | None, remove_think: bool) -> str:
+        """Process thinking/reasoning content.
+
+        Args:
+            content: The main content from response
+            reasoning_content: Separate reasoning content from model
+            remove_think: If True, remove thinking markers; if False, preserve them
+
+        Returns:
+            Processed content string
+        """
+        # Extract and handle thinking tags
+        if content and 'CRETIRE_REASONING_BEGINk' in content and 'CRETIRE_REASONING_ENDk' in content:
+            import re
+
+            think_pattern = r'CRETIRE_REASONING_BEGINk(.*?)CRETIRE_REASONING_ENDk'
+
+            if remove_think:
+                # Remove thinking tags and their content from output
+                content = re.sub(think_pattern, '', content, flags=re.DOTALL).strip()
+            # else: preserve thinking content as-is
+
+        # Handle separate reasoning_content field
+        # Currently we don't include reasoning_content in user-facing output regardless of remove_think
+        # because it's typically internal model reasoning, not user-visible thinking
+        return content or ''
+
+    @staticmethod
+    def _normalize_usage(usage: typing.Any) -> dict:
+        """Normalize a LiteLLM/OpenAI usage object into a plain token dict.
+
+        Handles several real-world shapes returned by different upstreams:
+        - object with ``prompt_tokens`` / ``completion_tokens`` / ``total_tokens`` attrs
+        - dict with the same keys
+        - missing ``total_tokens`` (derived from prompt + completion)
+        - ``None`` / partially-populated usage (defaults to 0)
+        """
+        if usage is None:
+            return {'prompt_tokens': 0, 'completion_tokens': 0, 'total_tokens': 0}
+
+        def _get(key: str) -> typing.Any:
+            if isinstance(usage, dict):
+                return usage.get(key)
+            return getattr(usage, key, None)
+
+        prompt_tokens = _get('prompt_tokens') or 0
+        completion_tokens = _get('completion_tokens') or 0
+        total_tokens = _get('total_tokens') or 0
+
+        # Some providers omit total_tokens in streaming usage; derive it.
+        if not total_tokens:
+            total_tokens = prompt_tokens + completion_tokens
+
+        return {
+            'prompt_tokens': int(prompt_tokens),
+            'completion_tokens': int(completion_tokens),
+            'total_tokens': int(total_tokens),
+        }
+
+    def _extract_usage(self, response) -> dict:
+        """Extract usage info from a non-streaming LiteLLM response."""
+        return self._normalize_usage(getattr(response, 'usage', None))
+
+    @staticmethod
+    def _as_dict(value: typing.Any) -> dict:
+        if value is None:
+            return {}
+        if isinstance(value, dict):
+            return value
+        if hasattr(value, 'model_dump'):
+            return value.model_dump()
+        return {}
+
+    def _normalize_stream_tool_calls(
+        self,
+        raw_tool_calls: typing.Any,
+        tool_call_state: dict[int, dict[str, str]],
+    ) -> list[dict] | None:
+        """Fill OpenAI-style streaming tool-call deltas so MessageChunk can validate them."""
+        if not raw_tool_calls:
+            return None
+
+        normalized = []
+        for fallback_index, raw_tool_call in enumerate(raw_tool_calls):
+            tool_call = self._as_dict(raw_tool_call)
+            index = tool_call.get('index')
+            if not isinstance(index, int):
+                index = fallback_index
+
+            state = tool_call_state.setdefault(index, {'id': '', 'type': 'function', 'name': ''})
+            if tool_call.get('id'):
+                state['id'] = tool_call['id']
+            if tool_call.get('type'):
+                state['type'] = tool_call['type']
+
+            function = self._as_dict(tool_call.get('function'))
+            if function.get('name'):
+                state['name'] = function['name']
+
+            arguments = function.get('arguments')
+            if arguments is None:
+                arguments = ''
+            elif not isinstance(arguments, str):
+                arguments = str(arguments)
+
+            if not state['id'] or not state['name']:
+                continue
+
+            normalized.append(
+                {
+                    'id': state['id'],
+                    'type': state['type'] or 'function',
+                    'function': {
+                        'name': state['name'],
+                        'arguments': arguments,
+                    },
+                }
+            )
+
+        return normalized or None
+
+    def _build_common_args(self, args: dict, include_retry_params: bool = True) -> dict:
+        """Apply common requester config to args dict."""
+        if self.requester_cfg.get('base_url'):
+            args['api_base'] = self.requester_cfg['base_url']
+        if self.requester_cfg.get('timeout'):
+            args['timeout'] = self.requester_cfg['timeout']
+        if include_retry_params:
+            if self.requester_cfg.get('drop_params'):
+                args['drop_params'] = self.requester_cfg['drop_params']
+            if self.requester_cfg.get('num_retries'):
+                args['num_retries'] = self.requester_cfg['num_retries']
+            if self.requester_cfg.get('api_version'):
+                args['api_version'] = self.requester_cfg['api_version']
+        return args
+
+    def _handle_litellm_error(self, e: Exception) -> None:
+        """Convert LiteLLM exceptions to RequesterError. Never returns, always raises."""
+        # Check more specific exceptions first (they inherit from base exceptions)
+        if isinstance(e, litellm.ContextWindowExceededError):
+            raise errors.RequesterError(f'上下文长度超限: {str(e)}')
+        if isinstance(e, litellm.BadRequestError):
+            raise errors.RequesterError(f'请求参数错误: {str(e)}')
+        if isinstance(e, litellm.AuthenticationError):
+            raise errors.RequesterError(f'API key 无效: {str(e)}')
+        if isinstance(e, litellm.NotFoundError):
+            raise errors.RequesterError(f'模型或路径无效: {str(e)}')
+        if isinstance(e, litellm.RateLimitError):
+            raise errors.RequesterError(f'请求过于频繁或余额不足: {str(e)}')
+        if isinstance(e, litellm.Timeout):
+            raise errors.RequesterError(f'请求超时: {str(e)}')
+        if isinstance(e, litellm.APIConnectionError):
+            raise errors.RequesterError(f'连接错误: {str(e)}')
+        if isinstance(e, litellm.APIError):
+            raise errors.RequesterError(f'API 错误: {str(e)}')
+        raise errors.RequesterError(f'未知错误: {str(e)}')
+
+    async def _build_completion_args(
+        self,
+        model: requester.RuntimeLLMModel,
+        messages: typing.List[provider_message.Message],
+        funcs: typing.List[resource_tool.LLMTool] = None,
+        extra_args: dict[str, typing.Any] = {},
+        stream: bool = False,
+    ) -> dict:
+        """Build common completion arguments for invoke_llm and invoke_llm_stream."""
+        req_messages = self._convert_messages(messages)
+        model_name = self._build_litellm_model_name(model.model_entity.name)
+        api_key = model.provider.token_mgr.get_token()
+
+        args = {
+            'model': model_name,
+            'messages': req_messages,
+            'api_key': api_key,
+        }
+        if stream:
+            args['stream'] = True
+            args['stream_options'] = {'include_usage': True}
+        self._build_common_args(args)
+
+        # Apply model-level extra_args first, then call-level extra_args
+        if model.model_entity.extra_args:
+            args.update(model.model_entity.extra_args)
+        args.update(extra_args)
+
+        if funcs:
+            tools = await self.ap.tool_mgr.generate_tools_for_openai(funcs)
+            if tools:
+                args['tools'] = tools
+                args.setdefault('tool_choice', 'auto')
+
+        return args
+
+    async def invoke_llm(
+        self,
+        query: pipeline_query.Query,
+        model: requester.RuntimeLLMModel,
+        messages: typing.List[provider_message.Message],
+        funcs: typing.List[resource_tool.LLMTool] = None,
+        extra_args: dict[str, typing.Any] = {},
+        remove_think: bool = False,
+    ) -> tuple[provider_message.Message, dict]:
+        """Invoke LLM and return message with usage info."""
+        args = await self._build_completion_args(model, messages, funcs, extra_args, stream=False)
+
+        try:
+            response = await acompletion(**args)
+
+            message_data = response.choices[0].message.model_dump()
+            if 'role' not in message_data or message_data['role'] is None:
+                message_data['role'] = 'assistant'
+
+            content = message_data.get('content', '')
+            reasoning_content = message_data.get('reasoning_content', None)
+            message_data['content'] = self._process_thinking_content(content, reasoning_content, remove_think)
+
+            if 'reasoning_content' in message_data:
+                del message_data['reasoning_content']
+
+            message = provider_message.Message(**message_data)
+            usage_info = self._extract_usage(response)
+
+            return message, usage_info
+
+        except Exception as e:
+            self._handle_litellm_error(e)
+
+    async def invoke_llm_stream(
+        self,
+        query: pipeline_query.Query,
+        model: requester.RuntimeLLMModel,
+        messages: typing.List[provider_message.Message],
+        funcs: typing.List[resource_tool.LLMTool] = None,
+        extra_args: dict[str, typing.Any] = {},
+        remove_think: bool = False,
+    ) -> provider_message.MessageChunk:
+        """Invoke LLM streaming and yield chunks."""
+        args = await self._build_completion_args(model, messages, funcs, extra_args, stream=True)
+
+        chunk_idx = 0
+        role = 'assistant'
+        tool_call_state: dict[int, dict[str, str]] = {}
+
+        try:
+            response = await acompletion(**args)
+            async for chunk in response:
+                # Capture usage whenever a chunk carries it.
+                #
+                # Important: many OpenAI-compatible gateways (e.g. new-api) and
+                # providers send the final usage payload in a chunk that STILL
+                # contains a (empty-delta) choice, not an empty `choices` list.
+                # The previous implementation only captured usage when `choices`
+                # was empty, so streamed calls always recorded 0 tokens.
+                # We therefore capture usage independently of `choices`, and then
+                # fall through to also process any content this chunk may carry.
+                if getattr(chunk, 'usage', None):
+                    usage_info = self._normalize_usage(chunk.usage)
+                    if query is not None:
+                        if query.variables is None:
+                            query.variables = {}
+                        query.variables['_stream_usage'] = usage_info
+
+                if not hasattr(chunk, 'choices') or not chunk.choices:
+                    continue
+
+                choice = chunk.choices[0]
+                delta = choice.delta.model_dump() if hasattr(choice, 'delta') else {}
+                finish_reason = getattr(choice, 'finish_reason', None)
+
+                if 'role' in delta and delta['role']:
+                    role = delta['role']
+
+                delta_content = delta.get('content', '')
+                reasoning_content = delta.get('reasoning_content', '')
+
+                # Handle reasoning_content based on remove_think flag
+                if reasoning_content:
+                    if remove_think:
+                        # Skip reasoning content when remove_think is True
+                        chunk_idx += 1
+                        continue
+                    else:
+                        # Use reasoning_content as the displayed content
+                        delta_content = reasoning_content
+
+                tool_calls = self._normalize_stream_tool_calls(delta.get('tool_calls'), tool_call_state)
+
+                if chunk_idx == 0 and not delta_content and not tool_calls:
+                    chunk_idx += 1
+                    continue
+
+                chunk_data = {
+                    'role': role,
+                    'content': delta_content if delta_content else None,
+                    'tool_calls': tool_calls,
+                    'is_final': bool(finish_reason),
+                }
+
+                chunk_data = {k: v for k, v in chunk_data.items() if v is not None}
+                yield provider_message.MessageChunk(**chunk_data)
+                chunk_idx += 1
+
+        except Exception as e:
+            self._handle_litellm_error(e)
+
+    async def invoke_embedding(
+        self,
+        model: requester.RuntimeEmbeddingModel,
+        input_text: list[str],
+        extra_args: dict[str, typing.Any] = {},
+    ) -> tuple[list[list[float]], dict]:
+        """Invoke embedding and return vectors with usage info."""
+        model_name = self._build_litellm_model_name(model.model_entity.name)
+        api_key = model.provider.token_mgr.get_token()
+
+        args = {
+            'model': model_name,
+            'input': input_text,
+            'api_key': api_key,
+        }
+        self._build_common_args(args, include_retry_params=False)
+
+        if model.model_entity.extra_args:
+            args.update(model.model_entity.extra_args)
+
+        args.update(extra_args)
+
+        try:
+            response = await aembedding(**args)
+
+            embeddings = [d.embedding for d in response.data]
+            usage_info = self._extract_usage(response)
+
+            return embeddings, usage_info
+
+        except Exception as e:
+            self._handle_litellm_error(e)
+
+    async def invoke_rerank(
+        self,
+        model: requester.RuntimeRerankModel,
+        query: str,
+        documents: typing.List[str],
+        extra_args: dict[str, typing.Any] = {},
+    ) -> typing.List[dict]:
+        """Invoke rerank and return relevance scores."""
+        model_name = self._build_litellm_model_name(model.model_entity.name)
+        api_key = model.provider.token_mgr.get_token()
+
+        args = {
+            'model': model_name,
+            'query': query,
+            'documents': documents,
+            'api_key': api_key,
+            'top_n': min(len(documents), 64),
+        }
+        self._build_common_args(args, include_retry_params=False)
+
+        if model.model_entity.extra_args:
+            args.update(model.model_entity.extra_args)
+
+        args.update(extra_args)
+
+        try:
+            response = await arerank(**args)
+
+            results = []
+            for r in response.results:
+                results.append(
+                    {
+                        'index': r.get('index', 0),
+                        'relevance_score': r.get('relevance_score', 0.0),
+                    }
+                )
+
+            if results:
+                scores = [r['relevance_score'] for r in results]
+                min_score = min(scores)
+                max_score = max(scores)
+                if max_score - min_score > 1e-6:
+                    for r in results:
+                        r['relevance_score'] = (r['relevance_score'] - min_score) / (max_score - min_score)
+
+            return results
+
+        except Exception as e:
+            self._handle_litellm_error(e)
+
+    async def scan_models(self, api_key: str | None = None) -> dict[str, typing.Any]:
+        """Scan models supported by the provider."""
+        import httpx
+
+        base_url = self.requester_cfg.get('base_url', '').rstrip('/')
+        timeout = self.requester_cfg.get('timeout', 120)
+
+        if not base_url:
+            raise errors.RequesterError('Base URL required for model scanning')
+
+        headers = {}
+        if api_key:
+            headers['Authorization'] = f'Bearer {api_key}'
+
+        models_url = f'{base_url}/models'
+
+        try:
+            async with httpx.AsyncClient(trust_env=True, timeout=timeout) as client:
+                response = await client.get(models_url, headers=headers)
+                response.raise_for_status()
+                payload = response.json()
+
+            models = []
+            for item in payload.get('data', []):
+                model_id = item.get('id')
+                if not model_id:
+                    continue
+
+                models.append(self._enrich_scanned_model(model_id, item))
+
+            models.sort(key=lambda x: (x['type'] != 'llm', x['name'].lower()))
+
+            return {'models': models}
+
+        except httpx.HTTPStatusError as e:
+            raise errors.RequesterError(f'Model scan failed: {e.response.status_code}')
+        except httpx.TimeoutException:
+            raise errors.RequesterError('Model scan timeout')
+        except Exception as e:
+            raise errors.RequesterError(f'Model scan error: {str(e)}')

src/langbot/pkg/provider/modelmgr/requesters/litellmchat.yaml

@@ -0,0 +1,64 @@
+apiVersion: v1
+kind: LLMAPIRequester
+metadata:
+  name: litellm-chat
+  label:
+    en_US: LiteLLM (Unified)
+    zh_Hans: LiteLLM (统一请求器)
+  icon: litellm.svg
+spec:
+  config:
+  - name: base_url
+    label:
+      en_US: Base URL
+      zh_Hans: 基础 URL
+    type: string
+    required: false
+    default: ''
+  - name: timeout
+    label:
+      en_US: Timeout
+      zh_Hans: 超时时间
+    type: integer
+    required: true
+    default: 120
+  - name: custom_llm_provider
+    label:
+      en_US: Custom Provider
+      zh_Hans: 自定义 Provider
+    type: string
+    required: false
+    default: ''
+    description:
+      en_US: Force provider type (e.g., anthropic, openai, gemini)
+      zh_Hans: 强制指定 provider 类型（如 anthropic, openai, gemini）
+  - name: drop_params
+    label:
+      en_US: Drop Unsupported Params
+      zh_Hans: 丢弃不支持参数
+    type: boolean
+    required: false
+    default: false
+  - name: num_retries
+    label:
+      en_US: Number of Retries
+      zh_Hans: 重试次数
+    type: integer
+    required: false
+    default: 0
+  - name: api_version
+    label:
+      en_US: API Version
+      zh_Hans: API 版本
+    type: string
+    required: false
+    default: ''
+  support_type:
+  - llm
+  - text-embedding
+  - rerank
+  provider_category: unified
+execution:
+  python:
+    path: ./litellmchat.py
+    attr: LiteLLMRequester

src/langbot/pkg/provider/modelmgr/requesters/lmstudiochatcmpl.py

@@ -1,17 +0,0 @@
-from __future__ import annotations
-
-import typing
-import openai
-
-from . import chatcmpl
-
-
-class LmStudioChatCompletions(chatcmpl.OpenAIChatCompletions):
-    """LMStudio ChatCompletion API 请求器"""
-
-    client: openai.AsyncClient
-
-    default_config: dict[str, typing.Any] = {
-        'base_url': 'http://127.0.0.1:1234/v1',
-        'timeout': 120,
-    }

src/langbot/pkg/provider/modelmgr/requesters/lmstudiochatcmpl.yaml

@@ -7,6 +7,7 @@ metadata:
     zh_Hans: LM Studio
   icon: lmstudio.webp
 spec:
+  litellm_provider: openai
   config:
   - name: base_url
     label:

src/langbot/pkg/provider/modelmgr/requesters/mimo.svg

@@ -0,0 +1,4 @@
+<svg width="60" height="50" viewBox="0 0 60 50" xmlns="http://www.w3.org/2000/svg">
+  <rect width="60" height="50" rx="8" fill="#FF6700"/>
+  <text x="30" y="32" font-family="Arial, sans-serif" font-size="18" font-weight="bold" fill="white" text-anchor="middle">MiMo</text>
+</svg>

src/langbot/pkg/provider/modelmgr/requesters/mimochatcmpl.yaml

@@ -0,0 +1,30 @@
+apiVersion: v1
+kind: LLMAPIRequester
+metadata:
+  name: mimo-chat-completions
+  label:
+    en_US: Xiaomi MiMo
+    zh_Hans: 小米 MiMo
+  icon: mimo.svg
+spec:
+  litellm_provider: openai
+  config:
+  - name: base_url
+    label:
+      en_US: Base URL
+      zh_Hans: 基础 URL
+    type: string
+    required: true
+    default: https://api.xiaomimimo.com/v1
+  - name: timeout
+    label:
+      en_US: Timeout
+      zh_Hans: 超时时间
+    type: integer
+    required: true
+    default: 120
+  support_type:
+  - llm
+  - text-embedding
+  - rerank
+  provider_category: manufacturer

src/langbot/pkg/provider/modelmgr/requesters/minimax.svg

@@ -0,0 +1,4 @@
+<svg width="60" height="50" viewBox="0 0 60 50" xmlns="http://www.w3.org/2000/svg">
+  <rect width="60" height="50" rx="8" fill="#4F46E5"/>
+  <text x="30" y="32" font-family="Arial, sans-serif" font-size="12" font-weight="bold" fill="white" text-anchor="middle">MiniMax</text>
+</svg>

src/langbot/pkg/provider/modelmgr/requesters/minimaxchatcmpl.yaml

@@ -0,0 +1,30 @@
+apiVersion: v1
+kind: LLMAPIRequester
+metadata:
+  name: minimax-chat-completions
+  label:
+    en_US: MiniMax
+    zh_Hans: MiniMax
+  icon: minimax.svg
+spec:
+  litellm_provider: openai
+  config:
+  - name: base_url
+    label:
+      en_US: Base URL
+      zh_Hans: 基础 URL
+    type: string
+    required: true
+    default: https://api.minimax.chat/v1
+  - name: timeout
+    label:
+      en_US: Timeout
+      zh_Hans: 超时时间
+    type: integer
+    required: true
+    default: 120
+  support_type:
+  - llm
+  - text-embedding
+  - rerank
+  provider_category: manufacturer

src/langbot/pkg/provider/modelmgr/requesters/mistral.svg

@@ -0,0 +1,5 @@
+<svg width="60" height="50" viewBox="0 0 60 50" xmlns="http://www.w3.org/2000/svg">
+  <rect width="60" height="50" rx="8" fill="#FF6B35"/>
+  <text x="30" y="28" font-family="Arial, sans-serif" font-size="10" font-weight="bold" fill="white" text-anchor="middle">Mistral</text>
+  <text x="30" y="40" font-family="Arial, sans-serif" font-size="8" fill="white" text-anchor="middle">AI</text>
+</svg>

src/langbot/pkg/provider/modelmgr/requesters/mistralchatcmpl.yaml

@@ -0,0 +1,30 @@
+apiVersion: v1
+kind: LLMAPIRequester
+metadata:
+  name: mistral-chat-completions
+  label:
+    en_US: Mistral AI
+    zh_Hans: Mistral AI
+  icon: mistral.svg
+spec:
+  litellm_provider: mistral
+  config:
+  - name: base_url
+    label:
+      en_US: Base URL
+      zh_Hans: 基础 URL
+    type: string
+    required: true
+    default: https://api.mistral.ai/v1
+  - name: timeout
+    label:
+      en_US: Timeout
+      zh_Hans: 超时时间
+    type: integer
+    required: true
+    default: 120
+  support_type:
+  - llm
+  - text-embedding
+  - rerank
+  provider_category: manufacturer

src/langbot/pkg/provider/modelmgr/requesters/modelscopechatcmpl.py

@@ -1,561 +0,0 @@
-from __future__ import annotations
-
-import asyncio
-import typing
-
-import openai
-import openai.types.chat.chat_completion as chat_completion
-import httpx
-
-from .. import entities, errors, requester
-import langbot_plugin.api.entities.builtin.resource.tool as resource_tool
-import langbot_plugin.api.entities.builtin.pipeline.query as pipeline_query
-import langbot_plugin.api.entities.builtin.provider.message as provider_message
-
-
-class ModelScopeChatCompletions(requester.ProviderAPIRequester):
-    """ModelScope ChatCompletion API 请求器"""
-
-    client: openai.AsyncClient
-
-    default_config: dict[str, typing.Any] = {
-        'base_url': 'https://api-inference.modelscope.cn/v1',
-        'timeout': 120,
-    }
-
-    async def initialize(self):
-        self.client = openai.AsyncClient(
-            api_key=self.init_api_key,
-            base_url=self.requester_cfg['base_url'],
-            timeout=self.requester_cfg['timeout'],
-            http_client=httpx.AsyncClient(trust_env=True, timeout=self.requester_cfg['timeout']),
-        )
-
-    def _mask_api_key(self, api_key: str | None) -> str:
-        if not api_key:
-            return ''
-        if len(api_key) <= 8:
-            return '****'
-        return f'{api_key[:4]}...{api_key[-4:]}'
-
-    def _infer_model_type(self, model_id: str) -> str:
-        normalized_model_id = (model_id or '').lower()
-        embedding_keywords = (
-            'embedding',
-            'embed',
-            'bge-',
-            'e5-',
-            'm3e',
-            'gte-',
-            'multilingual-e5',
-            'text-embedding',
-        )
-        return 'embedding' if any(keyword in normalized_model_id for keyword in embedding_keywords) else 'llm'
-
-    def _infer_model_abilities(self, item: dict[str, typing.Any], model_id: str) -> list[str]:
-        normalized_model_id = (model_id or '').lower()
-        abilities: set[str] = set()
-
-        def _flatten(value: typing.Any) -> list[str]:
-            if value is None:
-                return []
-            if isinstance(value, str):
-                return [value.lower()]
-            if isinstance(value, dict):
-                flattened: list[str] = []
-                for nested_value in value.values():
-                    flattened.extend(_flatten(nested_value))
-                return flattened
-            if isinstance(value, (list, tuple, set)):
-                flattened: list[str] = []
-                for nested_value in value:
-                    flattened.extend(_flatten(nested_value))
-                return flattened
-            return [str(value).lower()]
-
-        capability_tokens = _flatten(item.get('capabilities'))
-        capability_tokens.extend(_flatten(item.get('modalities')))
-        capability_tokens.extend(_flatten(item.get('input_modalities')))
-        capability_tokens.extend(_flatten(item.get('output_modalities')))
-        capability_tokens.extend(_flatten(item.get('supported_generation_methods')))
-        capability_tokens.extend(_flatten(item.get('supported_parameters')))
-        capability_tokens.extend(_flatten(item.get('architecture')))
-
-        combined_tokens = capability_tokens + [normalized_model_id]
-
-        vision_keywords = ('vision', 'image', 'file', 'video', 'multimodal', 'vl', 'ocr', 'omni')
-        function_call_keywords = ('function', 'tool', 'tools', 'tool_choice', 'tool_call', 'tool-use', 'tool_use')
-
-        if any(any(keyword in token for keyword in vision_keywords) for token in combined_tokens):
-            abilities.add('vision')
-
-        if any(any(keyword in token for keyword in function_call_keywords) for token in combined_tokens):
-            abilities.add('func_call')
-
-        return sorted(abilities)
-
-    def _normalize_modalities(self, value: typing.Any) -> list[str]:
-        normalized: list[str] = []
-
-        def _collect(item: typing.Any):
-            if item is None:
-                return
-            if isinstance(item, str):
-                for part in item.replace('->', ',').replace('+', ',').split(','):
-                    token = part.strip().lower()
-                    if token and token not in normalized:
-                        normalized.append(token)
-                return
-            if isinstance(item, dict):
-                for nested in item.values():
-                    _collect(nested)
-                return
-            if isinstance(item, (list, tuple, set)):
-                for nested in item:
-                    _collect(nested)
-                return
-
-        _collect(value)
-        return normalized
-
-    def _extract_scan_metadata(self, item: dict[str, typing.Any], model_id: str) -> dict[str, typing.Any]:
-        display_name = item.get('name')
-        if not isinstance(display_name, str) or not display_name.strip() or display_name == model_id:
-            display_name = ''
-
-        description = item.get('description')
-        if not isinstance(description, str) or not description.strip():
-            description = ''
-
-        context_length = item.get('context_length')
-        if context_length is None and isinstance(item.get('top_provider'), dict):
-            context_length = item['top_provider'].get('context_length')
-
-        if not isinstance(context_length, int):
-            try:
-                context_length = int(context_length) if context_length is not None else None
-            except (TypeError, ValueError):
-                context_length = None
-
-        input_modalities = self._normalize_modalities(item.get('input_modalities'))
-        output_modalities = self._normalize_modalities(item.get('output_modalities'))
-
-        if isinstance(item.get('architecture'), dict):
-            if not input_modalities:
-                input_modalities = self._normalize_modalities(item['architecture'].get('input_modalities'))
-            if not output_modalities:
-                output_modalities = self._normalize_modalities(item['architecture'].get('output_modalities'))
-
-        owned_by = item.get('owned_by')
-        if not isinstance(owned_by, str) or not owned_by.strip():
-            owned_by = ''
-
-        return {
-            'display_name': display_name or None,
-            'description': description or None,
-            'context_length': context_length,
-            'owned_by': owned_by or None,
-            'input_modalities': input_modalities,
-            'output_modalities': output_modalities,
-        }
-
-    async def scan_models(self, api_key: str | None = None) -> dict[str, typing.Any]:
-        headers = {}
-        if api_key:
-            headers['Authorization'] = f'Bearer {api_key}'
-
-        models_url = f'{self.requester_cfg["base_url"].rstrip("/")}/models'
-        async with httpx.AsyncClient(trust_env=True, timeout=self.requester_cfg['timeout']) as client:
-            response = await client.get(models_url, headers=headers)
-            response.raise_for_status()
-            payload = response.json()
-
-        models = []
-        for item in payload.get('data', []):
-            model_id = item.get('id')
-            if not model_id:
-                continue
-            models.append(
-                {
-                    'id': model_id,
-                    'name': model_id,
-                    'type': self._infer_model_type(model_id),
-                    'abilities': self._infer_model_abilities(item, model_id),
-                    **self._extract_scan_metadata(item, model_id),
-                }
-            )
-
-        models.sort(key=lambda item: (item['type'] != 'llm', item['name'].lower()))
-        return {
-            'models': models,
-            'debug': {
-                'request': {
-                    'method': 'GET',
-                    'url': models_url,
-                    'headers': {
-                        'Authorization': f'Bearer {self._mask_api_key(api_key)}' if api_key else '',
-                    },
-                },
-                'response': payload,
-            },
-        }
-
-    async def _req(
-        self,
-        query: pipeline_query.Query,
-        args: dict,
-        extra_body: dict = {},
-        remove_think: bool = False,
-    ) -> list[dict[str, typing.Any]]:
-        args['stream'] = True
-
-        chunk = None
-
-        pending_content = ''
-
-        tool_calls = []
-
-        resp_gen: openai.AsyncStream = await self.client.chat.completions.create(**args, extra_body=extra_body)
-
-        chunk_idx = 0
-        thinking_started = False
-        thinking_ended = False
-        tool_id = ''
-        tool_name = ''
-        message_delta = {}
-        async for chunk in resp_gen:
-            if not chunk or not chunk.id or not chunk.choices or not chunk.choices[0] or not chunk.choices[0].delta:
-                continue
-
-            delta = chunk.choices[0].delta.model_dump() if hasattr(chunk.choices[0], 'delta') else {}
-            reasoning_content = delta.get('reasoning_content')
-            # 处理 reasoning_content
-            if reasoning_content:
-                # accumulated_reasoning += reasoning_content
-                # 如果设置了 remove_think，跳过 reasoning_content
-                if remove_think:
-                    chunk_idx += 1
-                    continue
-
-                # 第一次出现 reasoning_content，添加 <think> 开始标签
-                if not thinking_started:
-                    thinking_started = True
-                    pending_content += '<think>\n' + reasoning_content
-                else:
-                    # 继续输出 reasoning_content
-                    pending_content += reasoning_content
-            elif thinking_started and not thinking_ended and delta.get('content'):
-                # reasoning_content 结束，normal content 开始，添加 </think> 结束标签
-                thinking_ended = True
-                pending_content += '\n</think>\n' + delta.get('content')
-
-            if delta.get('content') is not None:
-                pending_content += delta.get('content')
-
-            if delta.get('tool_calls') is not None:
-                for tool_call in delta.get('tool_calls'):
-                    if tool_call['id'] != '':
-                        tool_id = tool_call['id']
-                    if tool_call['function']['name'] is not None:
-                        tool_name = tool_call['function']['name']
-                    if tool_call['function']['arguments'] is None:
-                        continue
-                    tool_call['id'] = tool_id
-                    tool_call['name'] = tool_name
-                    for tc in tool_calls:
-                        if tc['index'] == tool_call['index']:
-                            tc['function']['arguments'] += tool_call['function']['arguments']
-                            break
-                    else:
-                        tool_calls.append(tool_call)
-
-            if chunk.choices[0].finish_reason is not None:
-                break
-        message_delta['content'] = pending_content
-        message_delta['role'] = 'assistant'
-
-        message_delta['tool_calls'] = tool_calls if tool_calls else None
-        return [message_delta]
-
-    async def _make_msg(
-        self,
-        chat_completion: list[dict[str, typing.Any]],
-    ) -> provider_message.Message:
-        chatcmpl_message = chat_completion[0]
-
-        # 确保 role 字段存在且不为 None
-        if 'role' not in chatcmpl_message or chatcmpl_message['role'] is None:
-            chatcmpl_message['role'] = 'assistant'
-
-        message = provider_message.Message(**chatcmpl_message)
-
-        return message
-
-    async def _closure(
-        self,
-        query: pipeline_query.Query,
-        req_messages: list[dict],
-        use_model: requester.RuntimeLLMModel,
-        use_funcs: list[resource_tool.LLMTool] = None,
-        extra_args: dict[str, typing.Any] = {},
-        remove_think: bool = False,
-    ) -> tuple[provider_message.Message, dict]:
-        self.client.api_key = use_model.provider.token_mgr.get_token()
-
-        args = {}
-        args['model'] = use_model.model_entity.name
-
-        if use_funcs:
-            tools = await self.ap.tool_mgr.generate_tools_for_openai(use_funcs)
-
-            if tools:
-                args['tools'] = tools
-
-        # 设置此次请求中的messages
-        messages = req_messages.copy()
-
-        # 检查vision
-        for msg in messages:
-            if 'content' in msg and isinstance(msg['content'], list):
-                for me in msg['content']:
-                    if me['type'] == 'image_base64':
-                        me['image_url'] = {'url': me['image_base64']}
-                        me['type'] = 'image_url'
-                        del me['image_base64']
-
-        args['messages'] = messages
-
-        # 发送请求
-        resp = await self._req(query, args, extra_body=extra_args, remove_think=remove_think)
-
-        # 处理请求结果
-        message = await self._make_msg(resp)
-
-        # ModelScope uses streaming, usage info not available
-        usage_info = {}
-
-        return message, usage_info
-
-    async def _req_stream(
-        self,
-        args: dict,
-        extra_body: dict = {},
-    ) -> chat_completion.ChatCompletion:
-        async for chunk in await self.client.chat.completions.create(**args, extra_body=extra_body):
-            yield chunk
-
-    async def _closure_stream(
-        self,
-        query: pipeline_query.Query,
-        req_messages: list[dict],
-        use_model: requester.RuntimeLLMModel,
-        use_funcs: list[resource_tool.LLMTool] = None,
-        extra_args: dict[str, typing.Any] = {},
-        remove_think: bool = False,
-    ) -> provider_message.Message | typing.AsyncGenerator[provider_message.MessageChunk, None]:
-        self.client.api_key = use_model.provider.token_mgr.get_token()
-
-        args = {}
-        args['model'] = use_model.model_entity.name
-
-        if use_funcs:
-            tools = await self.ap.tool_mgr.generate_tools_for_openai(use_funcs)
-
-            if tools:
-                args['tools'] = tools
-
-        # 设置此次请求中的messages
-        messages = req_messages.copy()
-
-        # 检查vision
-        for msg in messages:
-            if 'content' in msg and isinstance(msg['content'], list):
-                for me in msg['content']:
-                    if me['type'] == 'image_base64':
-                        me['image_url'] = {'url': me['image_base64']}
-                        me['type'] = 'image_url'
-                        del me['image_base64']
-
-        args['messages'] = messages
-        args['stream'] = True
-
-        # 流式处理状态
-        # tool_calls_map: dict[str, provider_message.ToolCall] = {}
-        chunk_idx = 0
-        thinking_started = False
-        thinking_ended = False
-        role = 'assistant'  # 默认角色
-        # accumulated_reasoning = ''  # 仅用于判断何时结束思维链
-
-        async for chunk in self._req_stream(args, extra_body=extra_args):
-            # 解析 chunk 数据
-            if hasattr(chunk, 'choices') and chunk.choices:
-                choice = chunk.choices[0]
-                delta = choice.delta.model_dump() if hasattr(choice, 'delta') else {}
-                finish_reason = getattr(choice, 'finish_reason', None)
-            else:
-                delta = {}
-                finish_reason = None
-
-            # 从第一个 chunk 获取 role，后续使用这个 role
-            if 'role' in delta and delta['role']:
-                role = delta['role']
-
-            # 获取增量内容
-            delta_content = delta.get('content', '')
-            reasoning_content = delta.get('reasoning_content', '')
-
-            # 处理 reasoning_content
-            if reasoning_content:
-                # accumulated_reasoning += reasoning_content
-                # 如果设置了 remove_think，跳过 reasoning_content
-                if remove_think:
-                    chunk_idx += 1
-                    continue
-
-                # 第一次出现 reasoning_content，添加 <think> 开始标签
-                if not thinking_started:
-                    thinking_started = True
-                    delta_content = '<think>\n' + reasoning_content
-                else:
-                    # 继续输出 reasoning_content
-                    delta_content = reasoning_content
-            elif thinking_started and not thinking_ended and delta_content:
-                # reasoning_content 结束，normal content 开始，添加 </think> 结束标签
-                thinking_ended = True
-                delta_content = '\n</think>\n' + delta_content
-
-            # 处理 content 中已有的 <think> 标签（如果需要移除）
-            # if delta_content and remove_think and '<think>' in delta_content:
-            #     import re
-            #
-            #     # 移除 <think> 标签及其内容
-            #     delta_content = re.sub(r'<think>.*?</think>', '', delta_content, flags=re.DOTALL)
-
-            # 处理工具调用增量
-            if delta.get('tool_calls'):
-                for tool_call in delta['tool_calls']:
-                    if tool_call['id'] != '':
-                        tool_id = tool_call['id']
-                    if tool_call['function']['name'] is not None:
-                        tool_name = tool_call['function']['name']
-
-                    if tool_call['type'] is None:
-                        tool_call['type'] = 'function'
-                    tool_call['id'] = tool_id
-                    tool_call['function']['name'] = tool_name
-                    tool_call['function']['arguments'] = (
-                        '' if tool_call['function']['arguments'] is None else tool_call['function']['arguments']
-                    )
-
-            # 跳过空的第一个 chunk（只有 role 没有内容）
-            if chunk_idx == 0 and not delta_content and not reasoning_content and not delta.get('tool_calls'):
-                chunk_idx += 1
-                continue
-
-            # 构建 MessageChunk - 只包含增量内容
-            chunk_data = {
-                'role': role,
-                'content': delta_content if delta_content else None,
-                'tool_calls': delta.get('tool_calls'),
-                'is_final': bool(finish_reason),
-            }
-
-            # 移除 None 值
-            chunk_data = {k: v for k, v in chunk_data.items() if v is not None}
-
-            yield provider_message.MessageChunk(**chunk_data)
-            chunk_idx += 1
-            # return
-
-    async def invoke_llm(
-        self,
-        query: pipeline_query.Query,
-        model: entities.LLMModelInfo,
-        messages: typing.List[provider_message.Message],
-        funcs: typing.List[resource_tool.LLMTool] = None,
-        extra_args: dict[str, typing.Any] = {},
-        remove_think: bool = False,
-    ) -> provider_message.Message:
-        req_messages = []  # req_messages 仅用于类内，外部同步由 query.messages 进行
-        for m in messages:
-            msg_dict = m.dict(exclude_none=True)
-            content = msg_dict.get('content')
-            if isinstance(content, list):
-                # 检查 content 列表中是否每个部分都是文本
-                if all(isinstance(part, dict) and part.get('type') == 'text' for part in content):
-                    # 将所有文本部分合并为一个字符串
-                    msg_dict['content'] = '\n'.join(part['text'] for part in content)
-            req_messages.append(msg_dict)
-
-        try:
-            return await self._closure(
-                query=query,
-                req_messages=req_messages,
-                use_model=model,
-                use_funcs=funcs,
-                extra_args=extra_args,
-                remove_think=remove_think,
-            )
-        except asyncio.TimeoutError:
-            raise errors.RequesterError('请求超时')
-        except openai.BadRequestError as e:
-            if 'context_length_exceeded' in e.message:
-                raise errors.RequesterError(f'上文过长，请重置会话: {e.message}')
-            else:
-                raise errors.RequesterError(f'请求参数错误: {e.message}')
-        except openai.AuthenticationError as e:
-            raise errors.RequesterError(f'无效的 api-key: {e.message}')
-        except openai.NotFoundError as e:
-            raise errors.RequesterError(f'请求路径错误: {e.message}')
-        except openai.RateLimitError as e:
-            raise errors.RequesterError(f'请求过于频繁或余额不足: {e.message}')
-        except openai.APIError as e:
-            raise errors.RequesterError(f'请求错误: {e.message}')
-
-    async def invoke_llm_stream(
-        self,
-        query: pipeline_query.Query,
-        model: requester.RuntimeLLMModel,
-        messages: typing.List[provider_message.Message],
-        funcs: typing.List[resource_tool.LLMTool] = None,
-        extra_args: dict[str, typing.Any] = {},
-        remove_think: bool = False,
-    ) -> provider_message.MessageChunk:
-        req_messages = []  # req_messages 仅用于类内，外部同步由 query.messages 进行
-        for m in messages:
-            msg_dict = m.dict(exclude_none=True)
-            content = msg_dict.get('content')
-            if isinstance(content, list):
-                # 检查 content 列表中是否每个部分都是文本
-                if all(isinstance(part, dict) and part.get('type') == 'text' for part in content):
-                    # 将所有文本部分合并为一个字符串
-                    msg_dict['content'] = '\n'.join(part['text'] for part in content)
-            req_messages.append(msg_dict)
-
-        try:
-            async for item in self._closure_stream(
-                query=query,
-                req_messages=req_messages,
-                use_model=model,
-                use_funcs=funcs,
-                extra_args=extra_args,
-                remove_think=remove_think,
-            ):
-                yield item
-
-        except asyncio.TimeoutError:
-            raise errors.RequesterError('请求超时')
-        except openai.BadRequestError as e:
-            if 'context_length_exceeded' in e.message:
-                raise errors.RequesterError(f'上文过长，请重置会话: {e.message}')
-            else:
-                raise errors.RequesterError(f'请求参数错误: {e.message}')
-        except openai.AuthenticationError as e:
-            raise errors.RequesterError(f'无效的 api-key: {e.message}')
-        except openai.NotFoundError as e:
-            raise errors.RequesterError(f'请求路径错误: {e.message}')
-        except openai.RateLimitError as e:
-            raise errors.RequesterError(f'请求过于频繁或余额不足: {e.message}')
-        except openai.APIError as e:
-            raise errors.RequesterError(f'请求错误: {e.message}')

src/langbot/pkg/provider/modelmgr/requesters/modelscopechatcmpl.yaml

@@ -7,6 +7,7 @@ metadata:
     zh_Hans: 魔搭社区
   icon: modelscope.svg
 spec:
+  litellm_provider: openai
   config:
   - name: base_url
     label:
@@ -31,6 +32,8 @@ spec:
     default: 120
   support_type:
   - llm
+  - text-embedding
+  - rerank
   provider_category: maas
 execution:
   python:

src/langbot/pkg/provider/modelmgr/requesters/moonshotchatcmpl.py

@@ -1,67 +0,0 @@
-from __future__ import annotations
-
-import typing
-
-
-from . import chatcmpl
-from .. import requester
-import langbot_plugin.api.entities.builtin.resource.tool as resource_tool
-import langbot_plugin.api.entities.builtin.pipeline.query as pipeline_query
-import langbot_plugin.api.entities.builtin.provider.message as provider_message
-
-
-class MoonshotChatCompletions(chatcmpl.OpenAIChatCompletions):
-    """Moonshot ChatCompletion API 请求器"""
-
-    default_config: dict[str, typing.Any] = {
-        'base_url': 'https://api.moonshot.cn/v1',
-        'timeout': 120,
-    }
-
-    async def _closure(
-        self,
-        query: pipeline_query.Query,
-        req_messages: list[dict],
-        use_model: requester.RuntimeLLMModel,
-        use_funcs: list[resource_tool.LLMTool] = None,
-        extra_args: dict[str, typing.Any] = {},
-        remove_think: bool = False,
-    ) -> tuple[provider_message.Message, dict]:
-        self.client.api_key = use_model.provider.token_mgr.get_token()
-
-        args = {}
-        args['model'] = use_model.model_entity.name
-
-        if use_funcs:
-            tools = await self.ap.tool_mgr.generate_tools_for_openai(use_funcs)
-
-            if tools:
-                args['tools'] = tools
-
-        # 设置此次请求中的messages
-        messages = req_messages
-
-        # deepseek 不支持多模态，把content都转换成纯文字
-        for m in messages:
-            if 'content' in m and isinstance(m['content'], list):
-                m['content'] = ' '.join([c['text'] for c in m['content']])
-
-        # 删除空的，不知道干嘛的，直接删了。
-        # messages = [m for m in messages if m["content"].strip() != "" and ('tool_calls' not in m or not m['tool_calls'])]
-
-        args['messages'] = messages
-
-        # 发送请求
-        resp = await self._req(args, extra_body=extra_args)
-
-        # 处理请求结果
-        message = await self._make_msg(resp, remove_think)
-
-        # Extract token usage from response
-        usage_info = {}
-        if hasattr(resp, 'usage') and resp.usage:
-            usage_info['input_tokens'] = resp.usage.prompt_tokens or 0
-            usage_info['output_tokens'] = resp.usage.completion_tokens or 0
-            usage_info['total_tokens'] = resp.usage.total_tokens or 0
-
-        return message, usage_info

src/langbot/pkg/provider/modelmgr/requesters/moonshotchatcmpl.yaml

@@ -7,6 +7,7 @@ metadata:
     zh_Hans: 月之暗面
   icon: moonshot.png
 spec:
+  litellm_provider: openai
   config:
   - name: base_url
     label:
@@ -24,6 +25,8 @@ spec:
     default: 120
   support_type:
   - llm
+  - text-embedding
+  - rerank
   provider_category: manufacturer
 execution:
   python:

src/langbot/pkg/provider/modelmgr/requesters/newapichatcmpl.py

@@ -1,17 +0,0 @@
-from __future__ import annotations
-
-import typing
-import openai
-
-from . import chatcmpl
-
-
-class NewAPIChatCompletions(chatcmpl.OpenAIChatCompletions):
-    """New API ChatCompletion API 请求器"""
-
-    client: openai.AsyncClient
-
-    default_config: dict[str, typing.Any] = {
-        'base_url': 'http://localhost:3000/v1',
-        'timeout': 120,
-    }

src/langbot/pkg/provider/modelmgr/requesters/newapichatcmpl.yaml

@@ -7,6 +7,7 @@ metadata:
     zh_Hans: New API
   icon: newapi.png
 spec:
+  litellm_provider: openai
   config:
   - name: base_url
     label:

src/langbot/pkg/provider/modelmgr/requesters/ollamachat.py

@@ -1,314 +0,0 @@
-from __future__ import annotations
-
-import asyncio
-import os
-import typing
-from typing import Union, Mapping, Any, AsyncIterator
-import uuid
-import json
-
-import ollama
-import httpx
-
-from .. import errors, requester
-import langbot_plugin.api.entities.builtin.resource.tool as resource_tool
-import langbot_plugin.api.entities.builtin.pipeline.query as pipeline_query
-import langbot_plugin.api.entities.builtin.provider.message as provider_message
-
-REQUESTER_NAME: str = 'ollama-chat'
-
-
-class OllamaChatCompletions(requester.ProviderAPIRequester):
-    """Ollama平台 ChatCompletion API请求器"""
-
-    client: ollama.AsyncClient
-
-    default_config: dict[str, typing.Any] = {
-        'base_url': 'http://127.0.0.1:11434',
-        'timeout': 120,
-    }
-
-    async def initialize(self):
-        os.environ['OLLAMA_HOST'] = self.requester_cfg['base_url']
-        self.client = ollama.AsyncClient(timeout=self.requester_cfg['timeout'])
-
-    def _infer_model_type(self, model_id: str) -> str:
-        normalized_model_id = (model_id or '').lower()
-        embedding_keywords = ('embedding', 'embed', 'bge-', 'e5-', 'm3e', 'gte-', 'text-embedding')
-        return 'embedding' if any(keyword in normalized_model_id for keyword in embedding_keywords) else 'llm'
-
-    def _infer_model_abilities(self, item: dict[str, typing.Any], model_id: str) -> list[str]:
-        normalized_model_id = (model_id or '').lower()
-        abilities: set[str] = set()
-        details = item.get('details', {}) or {}
-        families = details.get('families', []) or []
-        tokens = [normalized_model_id, str(details.get('family', '')).lower()]
-        tokens.extend(str(family).lower() for family in families)
-
-        if any(keyword in token for token in tokens for keyword in ('vision', 'vl', 'omni', 'llava', 'ocr')):
-            abilities.add('vision')
-        if any(keyword in token for token in tokens for keyword in ('tool', 'function')):
-            abilities.add('func_call')
-        return sorted(abilities)
-
-    async def scan_models(self, api_key: str | None = None) -> dict[str, typing.Any]:
-        del api_key
-        models_url = f'{self.requester_cfg["base_url"].rstrip("/")}/api/tags'
-
-        async with httpx.AsyncClient(trust_env=True, timeout=self.requester_cfg['timeout']) as client:
-            response = await client.get(models_url)
-            response.raise_for_status()
-            payload = response.json()
-
-        models: list[dict[str, typing.Any]] = []
-        for item in payload.get('models', []):
-            model_id = item.get('model') or item.get('name')
-            if not model_id:
-                continue
-            models.append(
-                {
-                    'id': model_id,
-                    'name': item.get('name', model_id),
-                    'type': self._infer_model_type(model_id),
-                    'abilities': self._infer_model_abilities(item, model_id),
-                }
-            )
-
-        models.sort(key=lambda item: (item['type'] != 'llm', item['name'].lower()))
-        return {
-            'models': models,
-            'debug': {
-                'request': {
-                    'method': 'GET',
-                    'url': models_url,
-                },
-                'response': payload,
-            },
-        }
-
-    async def _req(
-        self,
-        args: dict,
-    ) -> Union[Mapping[str, Any], AsyncIterator[Mapping[str, Any]]]:
-        return await self.client.chat(**args)
-
-    async def _closure(
-        self,
-        query: pipeline_query.Query,
-        req_messages: list[dict],
-        use_model: requester.RuntimeLLMModel,
-        use_funcs: list[resource_tool.LLMTool] = None,
-        extra_args: dict[str, typing.Any] = {},
-        remove_think: bool = False,
-    ) -> provider_message.Message:
-        args = extra_args.copy()
-        args['model'] = use_model.model_entity.name
-
-        messages: list[dict] = req_messages.copy()
-        for msg in messages:
-            if 'content' in msg and isinstance(msg['content'], list):
-                text_content: list = []
-                image_urls: list = []
-                for me in msg['content']:
-                    if me['type'] == 'text':
-                        text_content.append(me['text'])
-                    elif me['type'] == 'image_base64':
-                        image_urls.append(me['image_base64'])
-
-                msg['content'] = '\n'.join(text_content)
-                msg['images'] = [url.split(',')[1] for url in image_urls]
-            if 'tool_calls' in msg:  # LangBot 内部以 str 存储 tool_calls 的参数，这里需要转换为 dict
-                for tool_call in msg['tool_calls']:
-                    tool_call['function']['arguments'] = json.loads(tool_call['function']['arguments'])
-        args['messages'] = messages
-
-        args['tools'] = []
-        if use_funcs:
-            tools = await self.ap.tool_mgr.generate_tools_for_openai(use_funcs)
-            if tools:
-                args['tools'] = tools
-
-        resp = await self._req(args)
-        message: provider_message.Message = await self._make_msg(resp)
-        return message
-
-    async def _make_msg(self, chat_completions: ollama.ChatResponse) -> provider_message.Message:
-        message: ollama.Message = chat_completions.message
-        if message is None:
-            raise ValueError("chat_completions must contain a 'message' field")
-
-        ret_msg: provider_message.Message = None
-
-        if message.content is not None:
-            ret_msg = provider_message.Message(role='assistant', content=message.content)
-        if message.tool_calls is not None and len(message.tool_calls) > 0:
-            tool_calls: list[provider_message.ToolCall] = []
-
-            for tool_call in message.tool_calls:
-                tool_calls.append(
-                    provider_message.ToolCall(
-                        id=uuid.uuid4().hex,
-                        type='function',
-                        function=provider_message.FunctionCall(
-                            name=tool_call.function.name,
-                            arguments=json.dumps(tool_call.function.arguments),
-                        ),
-                    )
-                )
-            ret_msg.tool_calls = tool_calls
-
-        return ret_msg
-
-    async def _prepare_messages(
-        self,
-        messages: typing.List[provider_message.Message],
-    ) -> list[dict]:
-        """Prepare messages for Ollama API request."""
-        req_messages: list = []
-        for m in messages:
-            msg_dict: dict = m.dict(exclude_none=True)
-            content: Any = msg_dict.get('content')
-            if isinstance(content, list):
-                if all(isinstance(part, dict) and part.get('type') == 'text' for part in content):
-                    msg_dict['content'] = '\n'.join(part['text'] for part in content)
-            req_messages.append(msg_dict)
-        return req_messages
-
-    async def invoke_llm(
-        self,
-        query: pipeline_query.Query,
-        model: requester.RuntimeLLMModel,
-        messages: typing.List[provider_message.Message],
-        funcs: typing.List[resource_tool.LLMTool] = None,
-        extra_args: dict[str, typing.Any] = {},
-        remove_think: bool = False,
-    ) -> provider_message.Message:
-        req_messages = await self._prepare_messages(messages)
-        try:
-            return await self._closure(
-                query=query,
-                req_messages=req_messages,
-                use_model=model,
-                use_funcs=funcs,
-                extra_args=extra_args,
-                remove_think=remove_think,
-            )
-        except asyncio.TimeoutError:
-            raise errors.RequesterError('请求超时')
-
-    async def invoke_llm_stream(
-        self,
-        query: pipeline_query.Query,
-        model: requester.RuntimeLLMModel,
-        messages: typing.List[provider_message.Message],
-        funcs: typing.List[resource_tool.LLMTool] = None,
-        extra_args: dict[str, typing.Any] = {},
-        remove_think: bool = False,
-    ) -> provider_message.MessageChunk:
-        req_messages = await self._prepare_messages(messages)
-
-        try:
-            args = extra_args.copy()
-            args['model'] = model.model_entity.name
-
-            # Process messages for Ollama format
-            msgs: list[dict] = req_messages.copy()
-            for msg in msgs:
-                if 'content' in msg and isinstance(msg['content'], list):
-                    text_content: list = []
-                    image_urls: list = []
-                    for me in msg['content']:
-                        if me['type'] == 'text':
-                            text_content.append(me['text'])
-                        elif me['type'] == 'image_base64':
-                            image_urls.append(me['image_base64'])
-                    msg['content'] = '\n'.join(text_content)
-                    msg['images'] = [url.split(',')[1] for url in image_urls]
-                if 'tool_calls' in msg:
-                    for tool_call in msg['tool_calls']:
-                        tool_call['function']['arguments'] = json.loads(tool_call['function']['arguments'])
-            args['messages'] = msgs
-
-            args['tools'] = []
-            if funcs:
-                tools = await self.ap.tool_mgr.generate_tools_for_openai(funcs)
-                if tools:
-                    args['tools'] = tools
-
-            args['stream'] = True
-
-            chunk_idx = 0
-            thinking_started = False
-            thinking_ended = False
-            role = 'assistant'
-
-            async for chunk in await self.client.chat(**args):
-                message: ollama.Message = chunk.message
-                done = chunk.done
-
-                delta_content = message.content or ''
-                reasoning_content = getattr(message, 'thinking', '') or ''
-
-                # Handle reasoning/thinking content
-                if reasoning_content:
-                    if remove_think:
-                        chunk_idx += 1
-                        continue
-
-                    if not thinking_started:
-                        thinking_started = True
-                        delta_content = '<think>\n' + reasoning_content
-                    else:
-                        delta_content = reasoning_content
-                elif thinking_started and not thinking_ended and delta_content:
-                    thinking_ended = True
-                    delta_content = '\n</think>\n' + delta_content
-
-                # Handle tool calls
-                tool_calls_data = None
-                if message.tool_calls:
-                    tool_calls_data = []
-                    for tc in message.tool_calls:
-                        tool_calls_data.append(
-                            {
-                                'id': uuid.uuid4().hex,
-                                'type': 'function',
-                                'function': {
-                                    'name': tc.function.name,
-                                    'arguments': json.dumps(tc.function.arguments),
-                                },
-                            }
-                        )
-
-                # Skip empty first chunk
-                if chunk_idx == 0 and not delta_content and not reasoning_content and not tool_calls_data:
-                    chunk_idx += 1
-                    continue
-
-                chunk_data = {
-                    'role': role,
-                    'content': delta_content if delta_content else None,
-                    'tool_calls': tool_calls_data,
-                    'is_final': bool(done),
-                }
-                chunk_data = {k: v for k, v in chunk_data.items() if v is not None}
-
-                yield provider_message.MessageChunk(**chunk_data)
-                chunk_idx += 1
-
-        except asyncio.TimeoutError:
-            raise errors.RequesterError('请求超时')
-
-    async def invoke_embedding(
-        self,
-        model: requester.RuntimeEmbeddingModel,
-        input_text: list[str],
-        extra_args: dict[str, typing.Any] = {},
-    ) -> list[list[float]]:
-        return (
-            await self.client.embed(
-                model=model.model_entity.name,
-                input=input_text,
-                **extra_args,
-            )
-        ).embeddings

src/langbot/pkg/provider/modelmgr/requesters/ollamachat.yaml

@@ -7,6 +7,7 @@ metadata:
     zh_Hans: Ollama
   icon: ollama.svg
 spec:
+  litellm_provider: ollama
   config:
   - name: base_url
     label:

src/langbot/pkg/provider/modelmgr/requesters/openrouterchatcmpl.py

@@ -1,25 +0,0 @@
-from __future__ import annotations
-
-import typing
-import openai
-
-from . import modelscopechatcmpl
-
-
-class OpenRouterChatCompletions(modelscopechatcmpl.ModelScopeChatCompletions):
-    """OpenRouter ChatCompletion API 请求器"""
-
-    client: openai.AsyncClient
-
-    default_config: dict[str, typing.Any] = {
-        'base_url': 'https://openrouter.ai/api/v1',
-        'timeout': 120,
-    }
-
-    async def scan_models(self, api_key: str | None = None) -> dict[str, typing.Any]:
-        original_base_url = self.requester_cfg.get('base_url', '')
-        self.requester_cfg['base_url'] = 'https://openrouter.ai/api/v1'
-        try:
-            return await super().scan_models(api_key)
-        finally:
-            self.requester_cfg['base_url'] = original_base_url

src/langbot/pkg/provider/modelmgr/requesters/openrouterchatcmpl.yaml

@@ -7,6 +7,7 @@ metadata:
     zh_Hans: OpenRouter
   icon: openrouter.svg
 spec:
+  litellm_provider: openai
   config:
   - name: base_url
     label:

src/langbot/pkg/provider/modelmgr/requesters/ppiochatcmpl.py

@@ -1,208 +0,0 @@
-from __future__ import annotations
-
-import openai
-import typing
-
-from . import chatcmpl
-from .. import requester
-import openai.types.chat.chat_completion as chat_completion
-import re
-import langbot_plugin.api.entities.builtin.provider.message as provider_message
-import langbot_plugin.api.entities.builtin.pipeline.query as pipeline_query
-import langbot_plugin.api.entities.builtin.resource.tool as resource_tool
-
-
-class PPIOChatCompletions(chatcmpl.OpenAIChatCompletions):
-    """欧派云 ChatCompletion API 请求器"""
-
-    client: openai.AsyncClient
-
-    default_config: dict[str, typing.Any] = {
-        'base_url': 'https://api.ppinfra.com/v3/openai',
-        'timeout': 120,
-    }
-
-    is_think: bool = False
-
-    async def _make_msg(
-        self,
-        chat_completion: chat_completion.ChatCompletion,
-        remove_think: bool,
-    ) -> provider_message.Message:
-        chatcmpl_message = chat_completion.choices[0].message.model_dump()
-        # print(chatcmpl_message.keys(), chatcmpl_message.values())
-
-        # 确保 role 字段存在且不为 None
-        if 'role' not in chatcmpl_message or chatcmpl_message['role'] is None:
-            chatcmpl_message['role'] = 'assistant'
-
-        reasoning_content = chatcmpl_message['reasoning_content'] if 'reasoning_content' in chatcmpl_message else None
-
-        # deepseek的reasoner模型
-        chatcmpl_message['content'] = await self._process_thinking_content(
-            chatcmpl_message['content'], reasoning_content, remove_think
-        )
-
-        # 移除 reasoning_content 字段，避免传递给 Message
-        if 'reasoning_content' in chatcmpl_message:
-            del chatcmpl_message['reasoning_content']
-
-        message = provider_message.Message(**chatcmpl_message)
-
-        return message
-
-    async def _process_thinking_content(
-        self,
-        content: str,
-        reasoning_content: str = None,
-        remove_think: bool = False,
-    ) -> tuple[str, str]:
-        """处理思维链内容
-
-        Args:
-            content: 原始内容
-            reasoning_content: reasoning_content 字段内容
-            remove_think: 是否移除思维链
-
-        Returns:
-            处理后的内容
-        """
-        if remove_think:
-            content = re.sub(r'<think>.*?</think>', '', content, flags=re.DOTALL)
-        else:
-            if reasoning_content is not None:
-                content = '<think>\n' + reasoning_content + '\n</think>\n' + content
-        return content
-
-    async def _make_msg_chunk(
-        self,
-        delta: dict[str, typing.Any],
-        idx: int,
-    ) -> provider_message.MessageChunk:
-        # 处理流式chunk和完整响应的差异
-        # print(chat_completion.choices[0])
-
-        # 确保 role 字段存在且不为 None
-        if 'role' not in delta or delta['role'] is None:
-            delta['role'] = 'assistant'
-
-        reasoning_content = delta['reasoning_content'] if 'reasoning_content' in delta else None
-
-        delta['content'] = '' if delta['content'] is None else delta['content']
-        # print(reasoning_content)
-
-        # deepseek的reasoner模型
-
-        if reasoning_content is not None:
-            delta['content'] += reasoning_content
-
-        message = provider_message.MessageChunk(**delta)
-
-        return message
-
-    async def _closure_stream(
-        self,
-        query: pipeline_query.Query,
-        req_messages: list[dict],
-        use_model: requester.RuntimeLLMModel,
-        use_funcs: list[resource_tool.LLMTool] = None,
-        extra_args: dict[str, typing.Any] = {},
-        remove_think: bool = False,
-    ) -> provider_message.Message | typing.AsyncGenerator[provider_message.MessageChunk, None]:
-        self.client.api_key = use_model.provider.token_mgr.get_token()
-
-        args = {}
-        args['model'] = use_model.model_entity.name
-
-        if use_funcs:
-            tools = await self.ap.tool_mgr.generate_tools_for_openai(use_funcs)
-
-            if tools:
-                args['tools'] = tools
-
-        # 设置此次请求中的messages
-        messages = req_messages.copy()
-
-        # 检查vision
-        for msg in messages:
-            if 'content' in msg and isinstance(msg['content'], list):
-                for me in msg['content']:
-                    if me['type'] == 'image_base64':
-                        me['image_url'] = {'url': me['image_base64']}
-                        me['type'] = 'image_url'
-                        del me['image_base64']
-
-        args['messages'] = messages
-        args['stream'] = True
-
-        # tool_calls_map: dict[str, provider_message.ToolCall] = {}
-        chunk_idx = 0
-        thinking_started = False
-        thinking_ended = False
-        role = 'assistant'  # 默认角色
-        async for chunk in self._req_stream(args, extra_body=extra_args):
-            # 解析 chunk 数据
-            if hasattr(chunk, 'choices') and chunk.choices:
-                choice = chunk.choices[0]
-                delta = choice.delta.model_dump() if hasattr(choice, 'delta') else {}
-                finish_reason = getattr(choice, 'finish_reason', None)
-            else:
-                delta = {}
-                finish_reason = None
-
-            # 从第一个 chunk 获取 role，后续使用这个 role
-            if 'role' in delta and delta['role']:
-                role = delta['role']
-
-            # 获取增量内容
-            delta_content = delta.get('content', '')
-            # reasoning_content = delta.get('reasoning_content', '')
-
-            if remove_think:
-                if delta['content'] is not None:
-                    if '<think>' in delta['content'] and not thinking_started and not thinking_ended:
-                        thinking_started = True
-                        continue
-                    elif delta['content'] == r'</think>' and not thinking_ended:
-                        thinking_ended = True
-                        continue
-                    elif thinking_ended and delta['content'] == '\n\n' and thinking_started:
-                        thinking_started = False
-                        continue
-                    elif thinking_started and not thinking_ended:
-                        continue
-
-            # delta_tool_calls = None
-            if delta.get('tool_calls'):
-                for tool_call in delta['tool_calls']:
-                    if tool_call['id'] and tool_call['function']['name']:
-                        tool_id = tool_call['id']
-                        tool_name = tool_call['function']['name']
-
-                    if tool_call['id'] is None:
-                        tool_call['id'] = tool_id
-                    if tool_call['function']['name'] is None:
-                        tool_call['function']['name'] = tool_name
-                    if tool_call['function']['arguments'] is None:
-                        tool_call['function']['arguments'] = ''
-                    if tool_call['type'] is None:
-                        tool_call['type'] = 'function'
-
-            # 跳过空的第一个 chunk（只有 role 没有内容）
-            if chunk_idx == 0 and not delta_content and not delta.get('tool_calls'):
-                chunk_idx += 1
-                continue
-
-            # 构建 MessageChunk - 只包含增量内容
-            chunk_data = {
-                'role': role,
-                'content': delta_content if delta_content else None,
-                'tool_calls': delta.get('tool_calls'),
-                'is_final': bool(finish_reason),
-            }
-
-            # 移除 None 值
-            chunk_data = {k: v for k, v in chunk_data.items() if v is not None}
-
-            yield provider_message.MessageChunk(**chunk_data)
-            chunk_idx += 1

src/langbot/pkg/provider/modelmgr/requesters/ppiochatcmpl.yaml

@@ -7,6 +7,7 @@ metadata:
     zh_Hans: 派欧云
   icon: ppio.svg
 spec:
+  litellm_provider: openai
   config:
   - name: base_url
     label:

src/langbot/pkg/provider/modelmgr/requesters/qhaigcchatcmpl.py

@@ -1,17 +0,0 @@
-from __future__ import annotations
-
-import openai
-import typing
-
-from . import chatcmpl
-
-
-class QHAIGCChatCompletions(chatcmpl.OpenAIChatCompletions):
-    """启航 AI ChatCompletion API 请求器"""
-
-    client: openai.AsyncClient
-
-    default_config: dict[str, typing.Any] = {
-        'base_url': 'https://api.qhaigc.com/v1',
-        'timeout': 120,
-    }

src/langbot/pkg/provider/modelmgr/requesters/qhaigcchatcmpl.yaml

@@ -7,6 +7,7 @@ metadata:
     zh_Hans: 启航 AI
   icon: qhaigc.png
 spec:
+  litellm_provider: openai
   config:
   - name: base_url
     label:

src/langbot/pkg/provider/modelmgr/requesters/qiniuchatcmpl.py

@@ -2,19 +2,16 @@ from __future__ import annotations
 
 import typing
 
-import openai
+from . import litellmchat
-
-from . import chatcmpl
 
 
-class QiniuChatCompletions(chatcmpl.OpenAIChatCompletions):
+class QiniuChatCompletions(litellmchat.LiteLLMRequester):
     """七牛云 ChatCompletion API 请求器"""
 
-    client: openai.AsyncClient
-
     default_config: dict[str, typing.Any] = {
         'base_url': 'https://api.qnaigc.com/v1',
         'timeout': 120,
+        'custom_llm_provider': 'openai',
     }
 
     async def scan_models(self, api_key: str | None = None) -> dict[str, typing.Any]:

src/langbot/pkg/provider/modelmgr/requesters/shengsuanyun.py

@@ -1,32 +0,0 @@
-from __future__ import annotations
-
-import openai
-import typing
-
-from . import chatcmpl
-import openai.types.chat.chat_completion as chat_completion
-
-
-class ShengSuanYunChatCompletions(chatcmpl.OpenAIChatCompletions):
-    """胜算云(ModelSpot.AI) ChatCompletion API 请求器"""
-
-    client: openai.AsyncClient
-
-    default_config: dict[str, typing.Any] = {
-        'base_url': 'https://router.shengsuanyun.com/api/v1',
-        'timeout': 120,
-    }
-
-    async def _req(
-        self,
-        args: dict,
-        extra_body: dict = {},
-    ) -> chat_completion.ChatCompletion:
-        return await self.client.chat.completions.create(
-            **args,
-            extra_body=extra_body,
-            extra_headers={
-                'HTTP-Referer': 'https://langbot.app',
-                'X-Title': 'LangBot',
-            },
-        )

src/langbot/pkg/provider/modelmgr/requesters/shengsuanyun.yaml

@@ -7,6 +7,7 @@ metadata:
     zh_Hans: 胜算云
   icon: shengsuanyun.svg
 spec:
+  litellm_provider: openai
   config:
   - name: base_url
     label:

src/langbot/pkg/provider/modelmgr/requesters/siliconflowchatcmpl.py

@@ -1,17 +0,0 @@
-from __future__ import annotations
-
-import typing
-import openai
-
-from . import chatcmpl
-
-
-class SiliconFlowChatCompletions(chatcmpl.OpenAIChatCompletions):
-    """SiliconFlow ChatCompletion API 请求器"""
-
-    client: openai.AsyncClient
-
-    default_config: dict[str, typing.Any] = {
-        'base_url': 'https://api.siliconflow.cn/v1',
-        'timeout': 120,
-    }

src/langbot/pkg/provider/modelmgr/requesters/siliconflowchatcmpl.yaml

@@ -7,6 +7,7 @@ metadata:
     zh_Hans: 硅基流动
   icon: siliconflow.svg
 spec:
+  litellm_provider: openai
   config:
   - name: base_url
     label:

src/langbot/pkg/provider/modelmgr/requesters/spacechatcmpl.py

@@ -1,17 +0,0 @@
-from __future__ import annotations
-
-import typing
-import openai
-
-from . import chatcmpl
-
-
-class LangBotSpaceChatCompletions(chatcmpl.OpenAIChatCompletions):
-    """LangBot Space ChatCompletion API 请求器"""
-
-    client: openai.AsyncClient
-
-    default_config: dict[str, typing.Any] = {
-        'base_url': 'https://api.langbot.cloud/v1',
-        'timeout': 120,
-    }

src/langbot/pkg/provider/modelmgr/requesters/spacechatcmpl.yaml

@@ -7,6 +7,7 @@ metadata:
     zh_Hans: Space
   icon: space.webp
 spec:
+  litellm_provider: openai
   config:
   - name: base_url
     label:

src/langbot/pkg/provider/modelmgr/requesters/tencent.svg

@@ -0,0 +1,5 @@
+<svg width="60" height="50" viewBox="0 0 60 50" xmlns="http://www.w3.org/2000/svg">
+  <rect width="60" height="50" rx="8" fill="#0052D9"/>
+  <text x="30" y="28" font-family="Arial, sans-serif" font-size="10" font-weight="bold" fill="white" text-anchor="middle">Tencent</text>
+  <text x="30" y="40" font-family="Arial, sans-serif" font-size="8" fill="white" text-anchor="middle">Hunyuan</text>
+</svg>

src/langbot/pkg/provider/modelmgr/requesters/tencentchatcmpl.yaml

@@ -0,0 +1,30 @@
+apiVersion: v1
+kind: LLMAPIRequester
+metadata:
+  name: tencent-chat-completions
+  label:
+    en_US: Tencent Hunyuan
+    zh_Hans: 腾讯混元
+  icon: tencent.svg
+spec:
+  litellm_provider: openai
+  config:
+  - name: base_url
+    label:
+      en_US: Base URL
+      zh_Hans: 基础 URL
+    type: string
+    required: true
+    default: https://hunyuan.tencentcloudapi.com/v1
+  - name: timeout
+    label:
+      en_US: Timeout
+      zh_Hans: 超时时间
+    type: integer
+    required: true
+    default: 120
+  support_type:
+  - llm
+  - text-embedding
+  - rerank
+  provider_category: manufacturer

src/langbot/pkg/provider/modelmgr/requesters/together.svg

@@ -0,0 +1,5 @@
+<svg width="60" height="50" viewBox="0 0 60 50" xmlns="http://www.w3.org/2000/svg">
+  <rect width="60" height="50" rx="8" fill="#8B5CF6"/>
+  <text x="30" y="28" font-family="Arial, sans-serif" font-size="10" font-weight="bold" fill="white" text-anchor="middle">Together</text>
+  <text x="30" y="40" font-family="Arial, sans-serif" font-size="8" fill="white" text-anchor="middle">AI</text>
+</svg>

src/langbot/pkg/provider/modelmgr/requesters/togetherchatcmpl.yaml

@@ -0,0 +1,30 @@
+apiVersion: v1
+kind: LLMAPIRequester
+metadata:
+  name: together-chat-completions
+  label:
+    en_US: Together AI
+    zh_Hans: Together AI
+  icon: together.svg
+spec:
+  litellm_provider: together_ai
+  config:
+  - name: base_url
+    label:
+      en_US: Base URL
+      zh_Hans: 基础 URL
+    type: string
+    required: true
+    default: https://api.together.xyz/v1
+  - name: timeout
+    label:
+      en_US: Timeout
+      zh_Hans: 超时时间
+    type: integer
+    required: true
+    default: 120
+  support_type:
+  - llm
+  - text-embedding
+  - rerank
+  provider_category: manufacturer

src/langbot/pkg/provider/modelmgr/requesters/tokenpony.yaml

@@ -7,6 +7,7 @@ metadata:
     zh_Hans: 小马算力
   icon: tokenpony.svg
 spec:
+  litellm_provider: openai
   config:
   - name: base_url
     label:

src/langbot/pkg/provider/modelmgr/requesters/tokenponychatcmpl.py

@@ -1,17 +0,0 @@
-from __future__ import annotations
-
-import typing
-import openai
-
-from . import chatcmpl
-
-
-class TokenPonyChatCompletions(chatcmpl.OpenAIChatCompletions):
-    """TokenPony ChatCompletion API 请求器"""
-
-    client: openai.AsyncClient
-
-    default_config: dict[str, typing.Any] = {
-        'base_url': 'https://api.tokenpony.cn/v1',
-        'timeout': 120,
-    }

src/langbot/pkg/provider/modelmgr/requesters/volcarkchatcmpl.py

@@ -1,17 +0,0 @@
-from __future__ import annotations
-
-import typing
-import openai
-
-from . import chatcmpl
-
-
-class VolcArkChatCompletions(chatcmpl.OpenAIChatCompletions):
-    """火山方舟大模型平台 ChatCompletion API 请求器"""
-
-    client: openai.AsyncClient
-
-    default_config: dict[str, typing.Any] = {
-        'base_url': 'https://ark.cn-beijing.volces.com/api/v3',
-        'timeout': 120,
-    }

src/langbot/pkg/provider/modelmgr/requesters/volcarkchatcmpl.yaml

@@ -7,6 +7,7 @@ metadata:
     zh_Hans: 火山方舟
   icon: volcark.svg
 spec:
+  litellm_provider: openai
   config:
   - name: base_url
     label:
@@ -24,6 +25,8 @@ spec:
     default: 120
   support_type:
   - llm
+  - text-embedding
+  - rerank
   provider_category: maas
 execution:
   python:

src/langbot/pkg/provider/modelmgr/requesters/voyageairerank.yaml

@@ -7,6 +7,7 @@ metadata:
     zh_Hans: Voyage AI
   icon: voyageai.svg
 spec:
+  litellm_provider: openai
   config:
   - name: base_url
     label:

src/langbot/pkg/provider/modelmgr/requesters/xaichatcmpl.py

@@ -1,17 +0,0 @@
-from __future__ import annotations
-
-import typing
-import openai
-
-from . import chatcmpl
-
-
-class XaiChatCompletions(chatcmpl.OpenAIChatCompletions):
-    """xAI ChatCompletion API 请求器"""
-
-    client: openai.AsyncClient
-
-    default_config: dict[str, typing.Any] = {
-        'base_url': 'https://api.x.ai/v1',
-        'timeout': 120,
-    }

src/langbot/pkg/provider/modelmgr/requesters/xaichatcmpl.yaml

@@ -7,6 +7,7 @@ metadata:
     zh_Hans: xAI
   icon: xai.svg
 spec:
+  litellm_provider: openai
   config:
   - name: base_url
     label:
@@ -24,6 +25,8 @@ spec:
     default: 120
   support_type:
   - llm
+  - text-embedding
+  - rerank
   provider_category: manufacturer
 execution:
   python:

src/langbot/pkg/provider/modelmgr/requesters/yi.svg

@@ -0,0 +1,5 @@
+<svg width="60" height="50" viewBox="0 0 60 50" xmlns="http://www.w3.org/2000/svg">
+  <rect width="60" height="50" rx="8" fill="#10B981"/>
+  <text x="30" y="28" font-family="Arial, sans-serif" font-size="10" font-weight="bold" fill="white" text-anchor="middle">01.AI</text>
+  <text x="30" y="40" font-family="Arial, sans-serif" font-size="8" fill="white" text-anchor="middle">Yi</text>
+</svg>

src/langbot/pkg/provider/modelmgr/requesters/yichatcmpl.yaml

@@ -0,0 +1,30 @@
+apiVersion: v1
+kind: LLMAPIRequester
+metadata:
+  name: yi-chat-completions
+  label:
+    en_US: 01.AI Yi
+    zh_Hans: 零一万物
+  icon: yi.svg
+spec:
+  litellm_provider: openai
+  config:
+  - name: base_url
+    label:
+      en_US: Base URL
+      zh_Hans: 基础 URL
+    type: string
+    required: true
+    default: https://api.lingyiwanwu.com/v1
+  - name: timeout
+    label:
+      en_US: Timeout
+      zh_Hans: 超时时间
+    type: integer
+    required: true
+    default: 120
+  support_type:
+  - llm
+  - text-embedding
+  - rerank
+  provider_category: manufacturer

src/langbot/pkg/provider/modelmgr/requesters/zhipuaichatcmpl.py

@@ -1,17 +0,0 @@
-from __future__ import annotations
-
-import typing
-import openai
-
-from . import chatcmpl
-
-
-class ZhipuAIChatCompletions(chatcmpl.OpenAIChatCompletions):
-    """智谱AI ChatCompletion API 请求器"""
-
-    client: openai.AsyncClient
-
-    default_config: dict[str, typing.Any] = {
-        'base_url': 'https://open.bigmodel.cn/api/paas/v4',
-        'timeout': 120,
-    }