Compare commits

..

4 Commits

Author SHA1 Message Date
huanghuoguoguo 2fe4b117a4 refactor(skill): move orchestration into core 2026-09-06 12:50:37 +08:00
huanghuoguoguo cba842fbee style: format core skill orchestration 2026-09-05 23:18:17 +08:00
huanghuoguoguo a61b0f0068 refactor(skill): compose generic box mounts in core 2026-09-05 23:09:46 +08:00
huanghuoguoguo c15f668126 feat(skill): use SDK store without box execution 2026-09-05 19:49:35 +08:00
120 changed files with 1882 additions and 9066 deletions
+16 -7
View File
@@ -169,14 +169,17 @@ The Plugin Runtime supports stdio and WebSocket control transports. Direct local
## Box Runtime and Skills ## Box Runtime and Skills
Box is the sandbox subsystem used by native agent tools, stdio MCP servers, skill authoring, and managed processes. Box is the optional sandbox subsystem used by native execution tools, stdio MCP servers, agent-side skill authoring, and managed processes. Skill storage and read-only access are Core responsibilities and remain available without Box.
In this repo: In this repo:
- `pkg/box/service.py` is the application-facing facade for exec, sessions, managed processes, skill CRUD, status, reconnects, quotas, mounts, and sandbox profiles. - `pkg/box/service.py` is the application-facing facade for exec, sessions, managed processes, status, reconnects, quotas, generic mounts, and sandbox profiles.
- `pkg/box/connector.py` connects to the Box Runtime over stdio, Windows subprocess+WebSocket, or remote WebSocket. - `pkg/box/connector.py` connects to the Box Runtime over stdio, Windows subprocess+WebSocket, or remote WebSocket.
- `pkg/provider/tools/loaders/native.py`, `mcp_stdio.py`, and skill loaders depend on Box availability. - `pkg/provider/tools/loaders/native.py` is the Core orchestration seam: the
- `pkg/skill/manager.py` loads skills from the Box runtime, falling back to local `data/skills` when needed. Skill loader supplies generic read-only mounts to Box execution. `mcp_stdio.py`
and execution-backed tools depend on Box availability.
- `pkg/skill/repository.py` is the thin async/Workspace adapter over the Plugin SDK's execution-independent `SkillStore`; `skills.root` owns its location independently of Box.
- `pkg/skill/manager.py` caches the Core repository catalog for progressive disclosure. Activation and read-only resource tools do not require Box; script execution and Workspace mutation still do.
Durable Box Workspace storage is shared across placement generations, but Durable Box Workspace storage is shared across placement generations, but
sandbox sessions and managed processes are generation-scoped. LangBot validates sandbox sessions and managed processes are generation-scoped. LangBot validates
@@ -187,11 +190,17 @@ retires stale processes and closes already-attached relays.
In `langbot-plugin-sdk`: In `langbot-plugin-sdk`:
- `src/langbot_plugin/box/server.py` implements `lbp box` and the WebSocket endpoints on `:5410`. - `src/langbot_plugin/box/server.py` implements `lbp box` and the WebSocket endpoints on `:5410`.
- `src/langbot_plugin/box/runtime.py` owns sandbox sessions and managed processes. - `src/langbot_plugin/box/runtime.py` owns sandbox sessions, generic read-only mounts, and managed processes.
- `backend.py`, `nsjail_backend.py`, and `e2b_backend.py` implement sandbox backends. - `backend.py`, `nsjail_backend.py`, and `e2b_backend.py` implement sandbox backends.
- `skill_store.py` manages skill packages from the Box side. - `src/langbot_plugin/skill_store.py` is consumed by Core, not Box. Core turns
selected package roots into generic read-only mounts; Box does not understand
Skill names, metadata, revisions, files, or CRUD.
Important config keys live under `box:` in `src/langbot/templates/config.yaml`: `box.enabled`, `box.backend`, `box.runtime.endpoint`, and `box.local.*`. Start LangBot with `--standalone-box` when connecting to an externally launched Box runtime. Skill storage uses `skills.root`. Box execution config lives under `box:`:
`box.enabled`, `box.backend`, `box.runtime.endpoint`, and `box.local.*`. The old
`box.local.skills_root` key is read only as an online-upgrade fallback and is
marked for removal in the next major version. Start LangBot with
`--standalone-box` when connecting to an externally launched Box runtime.
## HTTP API, Web UI, and MCP Server ## HTTP API, Web UI, and MCP Server
+9 -3
View File
@@ -27,8 +27,8 @@ services:
# The Box sandbox runtime is optional. It is only started when you run # The Box sandbox runtime is optional. It is only started when you run
# ``docker compose --profile box up`` (or ``docker compose --profile all # ``docker compose --profile box up`` (or ``docker compose --profile all
# up``). With Box off, LangBot keeps the dashboard / skills list visible # up``). With Box off, LangBot keeps skill management, activation, and
# (read-only) but disables sandbox tools, skill add/edit and stdio MCP — # read-only resources available but disables execution tools and stdio MCP —
# set ``box.enabled: false`` in ``data/config.yaml`` (or # set ``box.enabled: false`` in ``data/config.yaml`` (or
# ``BOX__ENABLED=false`` in the langbot service env below) to match. # ``BOX__ENABLED=false`` in the langbot service env below) to match.
langbot_box: langbot_box:
@@ -73,6 +73,10 @@ services:
container_name: langbot container_name: langbot
volumes: volumes:
- ./data:/app/data - ./data:/app/data
# Core owns the SkillRepository even when the Box profile is disabled.
# Keep this path identical to langbot_box so optional execution can
# consume the same Workspace-scoped package revisions.
- ${LANGBOT_BOX_ROOT:-${PWD}/data/box}:${LANGBOT_BOX_ROOT:-${PWD}/data/box}
restart: on-failure restart: on-failure
environment: environment:
- TZ=Asia/Shanghai - TZ=Asia/Shanghai
@@ -92,7 +96,9 @@ services:
# box.* and are forwarded to the Box runtime via INIT RPC. # box.* and are forwarded to the Box runtime via INIT RPC.
- BOX__LOCAL__HOST_ROOT=${LANGBOT_BOX_ROOT:-${PWD}/data/box} - BOX__LOCAL__HOST_ROOT=${LANGBOT_BOX_ROOT:-${PWD}/data/box}
- BOX__LOCAL__DEFAULT_WORKSPACE=default - BOX__LOCAL__DEFAULT_WORKSPACE=default
- BOX__LOCAL__SKILLS_ROOT=skills # TODO(next-major): default LANGBOT_SKILLS_ROOT to ${PWD}/data/skills
# after the historical Box storage path no longer needs zero-copy upgrades.
- SKILLS__ROOT=${LANGBOT_SKILLS_ROOT:-${LANGBOT_BOX_ROOT:-${PWD}/data/box}/skills}
- BOX__LOCAL__ALLOWED_MOUNT_ROOTS=${LANGBOT_BOX_ROOT:-${PWD}/data/box} - BOX__LOCAL__ALLOWED_MOUNT_ROOTS=${LANGBOT_BOX_ROOT:-${PWD}/data/box}
- BOX__DOCKER__CPU_LIMIT_ENABLED=${LANGBOT_BOX_DOCKER_CPU_LIMIT_ENABLED:-true} - BOX__DOCKER__CPU_LIMIT_ENABLED=${LANGBOT_BOX_DOCKER_CPU_LIMIT_ENABLED:-true}
ports: ports:
+8 -6
View File
@@ -214,8 +214,9 @@ spec:
# Deployment for LangBot Box (sandbox) runtime # Deployment for LangBot Box (sandbox) runtime
# #
# The Box runtime backs LangBot's sandbox tools (exec / read / write / edit / # The Box runtime backs LangBot's sandbox tools (exec / read / write / edit /
# glob / grep), the `activate` skill tool, skill add/edit, and stdio-mode MCP # glob / grep), Skill script execution, and stdio-mode MCP servers. Skill
# servers. It is OPTIONAL: if you do not deploy it, set `BOX__ENABLED=false` on # activation, resources, and management remain Core-owned without Box. Box is
# OPTIONAL: if you do not deploy it, set `BOX__ENABLED=false` on
# the langbot Deployment (or `box.enabled: false` in config.yaml) so the # the langbot Deployment (or `box.enabled: false` in config.yaml) so the
# dashboard renders cleanly with sandbox features disabled. # dashboard renders cleanly with sandbox features disabled.
# #
@@ -448,14 +449,15 @@ spec:
key: token key: token
# box.local.* config — forwarded to the Box runtime via INIT RPC. The # box.local.* config — forwarded to the Box runtime via INIT RPC. The
# host_root MUST match the box-root hostPath mountPath below AND the box # host_root MUST match the box-root hostPath mountPath below AND the box
# Deployment's box-root mountPath, so that skill package paths resolve # Deployment's box-root mountPath, so generic package mount paths
# identically on both sides and on the node's Docker daemon. # resolve identically on both sides and on the node's Docker daemon.
- name: BOX__LOCAL__HOST_ROOT - name: BOX__LOCAL__HOST_ROOT
value: "/app/data/box" value: "/app/data/box"
- name: BOX__LOCAL__DEFAULT_WORKSPACE - name: BOX__LOCAL__DEFAULT_WORKSPACE
value: "default" value: "default"
- name: BOX__LOCAL__SKILLS_ROOT - name: SKILLS__ROOT
value: "skills" # TODO(next-major): use /app/data/skills after the legacy path window.
value: "/app/data/box/skills"
- name: BOX__LOCAL__ALLOWED_MOUNT_ROOTS - name: BOX__LOCAL__ALLOWED_MOUNT_ROOTS
value: "/app/data/box" value: "/app/data/box"
volumeMounts: volumeMounts:
-17
View File
@@ -88,23 +88,6 @@ Each endpoint accepts **either**:
1. **User Token** (via `Authorization: Bearer <user_jwt_token>`) - for web UI and authenticated users 1. **User Token** (via `Authorization: Bearer <user_jwt_token>`) - for web UI and authenticated users
2. **API Key** (via `X-API-Key` or `Authorization: Bearer <api_key>`) - for external services 2. **API Key** (via `X-API-Key` or `Authorization: Bearer <api_key>`) - for external services
### Inspecting API Key Identity
`GET /api/v1/system/context` validates an API key (user JWT not accepted) and returns its bound identity without requiring resource permissions:
```json
{
"code": 0,
"msg": "ok",
"data": {
"instance_uuid": "...",
"workspace_uuid": "...",
"api_key_id": "...",
"permissions": ["..."]
}
}
```
## Example: Model Management ## Example: Model Management
### List All LLM Models ### List All LLM Models
-65
View File
@@ -1,65 +0,0 @@
# ChatGPT / Codex subscription
LangBot's **OpenAI Codex** model provider uses **Sign in with ChatGPT** and the account's Codex entitlement. It is separate from the existing OpenAI API-key provider: subscribing to ChatGPT does not supply an OpenAI Platform API key, and API-key billing is unchanged.
## Connect an account
1. Open **Models**, choose **Add Provider**, and select **OpenAI Codex**.
2. Enter a provider name and choose **Save and sign in**. This saves the provider before authorization, so an interrupted login can be retried from its settings.
3. Open the OpenAI authorization link and enter the one-time code displayed in LangBot. Sign in on OpenAI's site, not in LangBot.
4. If OpenAI asks you to enable device-code authorization, enable it in your ChatGPT account's security settings, or contact your workspace administrator.
5. Keep the LangBot dialog open until it confirms the connection, then finish the form.
6. Use the existing **Scan models** or **Add model** controls, test the model, and select it in a pipeline as usual. Only LLM models are supported by this provider.
The device-code flow also works when LangBot runs remotely or in Docker: the browser does not need to reach a localhost OAuth callback on the server. Serve the LangBot management panel over HTTPS when accessing it remotely.
The account's model catalog is authoritative. A model listed elsewhere or entered manually is not a guarantee that this account has access. Scan errors are reported rather than replaced with a fabricated available-model list.
## Reconnect and disconnect
Open the provider's existing settings to sign in again or disconnect. LangBot refreshes expiring access tokens automatically. A revoked or invalid refresh grant requires another sign-in; transient network failures are not proof that the grant was revoked.
**Disconnect** removes this provider's locally stored authorization. It does not log the account out of other applications or revoke the account globally. Canceling a pending sign-in is separate from disconnecting an existing account. Removing a provider also removes its authorization; the normal rule that models must be removed first still applies.
A saved provider can remain disconnected. Scanning or invoking it then returns a sign-in-required error; LangBot does not silently switch to paid API-key billing.
## Usage and deployment boundary
Calls consume the connected account's included Codex usage and remain subject to OpenAI's plan limits, model availability, workspace policies, and terms. Token counts recorded by LangBot are request usage, not a measurement of remaining subscription quota or an OpenAI invoice.
Use this integration for your own authorized account and trusted workflows. Third-party sign-in support is not permission to pool accounts, resell subscription quota, or redistribute one subscription as a shared API service. For a public or commercial multi-user service, use the appropriate OpenAI API or separately authorized enterprise arrangement. The provider remains a Workspace resource in LangBot: consider who can invoke its models before connecting a personal account.
## Credential handling and API surface
- OAuth credentials are stored server-side separately from provider API keys. Provider and model reads do not supply OAuth access, refresh, or ID tokens.
- Authorization uses a fixed OpenAI origin. The Codex provider does not accept a custom base URL or manually supplied API keys.
- Authentication controls require an authenticated LangBot browser user with `provider_secret.manage` in the selected Workspace. Pending attempts are scoped to the Workspace, provider, and initiating user.
- Browser storage must not contain OAuth tokens. Treat the server database and its backups as sensitive application data.
- MCP and LangBot API keys do not expose the browser-only OAuth controls. Agents may inspect configured providers and models with the existing tools, but a human connects the subscription in the management panel.
The provider-scoped authentication routes are under `/api/v1/provider/providers/{uuid}/codex`:
| Method | Suffix | Purpose |
| --- | --- | --- |
| GET | `/status` | Read local connection state without returning credentials |
| POST | `/device` | Start device authorization |
| POST | `/device/poll` | Poll the initiating user's authorization attempt |
| DELETE | `/device/{authorization_id}` | Cancel only that pending attempt |
| DELETE | `/auth` | Remove local authorization |
Use the returned polling interval and expiration time. An expired attempt must be restarted. These routes are not a general-purpose subscription-to-API gateway.
## References
- [OpenAI Codex authentication](https://developers.openai.com/codex/auth): ChatGPT versus API-key access and device-code login.
- [Hermes Agent providers](https://hermes-agent.nousresearch.com/docs/integrations/providers/): subscription device authentication and refresh recovery.
- [OpenClaw OpenAI provider](https://docs.openclaw.ai/providers/openai): subscription and API-key route distinctions.
- [New API](https://github.com/QuantumNous/new-api): reference for Codex protocol compatibility; its gateway/account-pooling product model is not adopted here.
## 中文快速说明
在「模型」中添加提供商,选择 **OpenAI Codex**,填写名称并点击「保存并登录」。打开 OpenAI 授权页面,输入 LangBot 显示的一次性验证码,完成授权后回到原对话框。随后照常扫描或添加模型、测试模型,并在流水线中选择它。
无需填写 API Key,也无需为远程服务器配置 localhost 回调。登录中断后可以从该提供商的设置中重试;断开连接只删除 LangBot 中保存的授权。调用消耗所登录账号的 Codex 额度,受账号实际权限和 OpenAI 限制约束,不会自动转用按量付费的 OpenAI API。
此功能用于自己的授权账号及可信工作流,不应将个人订阅作为面向多个用户转售或共享的 API 服务。提供商仍是 LangBot 工作空间内的资源,连接个人账号前请确认模型的使用范围。
Binary file not shown.

Before

Width:  |  Height:  |  Size: 73 KiB

+39 -38
View File
@@ -22,6 +22,7 @@
│ │ │ (shared 容器, 多 process) │ │ │ │ (shared 容器, 多 process) │
│ │ │ │ │ │ │ │
│ │ ├──> SkillToolLoader (activate 工具) │ │ │ ├──> SkillToolLoader (activate 工具) │
│ │ │ └─ build_execution_mounts() │
│ │ │ │ │ │ │ │
│ │ ├──> SkillAuthoringToolLoader │ │ │ ├──> SkillAuthoringToolLoader │
│ │ │ │ │ │ │ │
@@ -33,7 +34,7 @@
│ ├─ Workspace quota 检查 │ │ ├─ Workspace quota 检查 │
│ ├─ 输出截断 (head+tail) │ │ ├─ 输出截断 (head+tail) │
│ ├─ Session ID 模板解析 (resolve_box_session_id) │ │ ├─ Session ID 模板解析 (resolve_box_session_id) │
│ ├─ 技能挂载组装 (build_skill_extra_mounts) │ │ ├─ 通用只读挂载接收 (read_only_mounts)
│ ├─ 重连循环 (_reconnect_loop, 指数退避) │ │ ├─ 重连循环 (_reconnect_loop, 指数退避) │
│ └─ BoxRuntimeConnector │ │ └─ BoxRuntimeConnector │
│ ├─ 心跳 loop (20s ping) │ │ ├─ 心跳 loop (20s ping) │
@@ -41,7 +42,7 @@
│ │ Action RPC (stdio 或 WebSocket) │ │ │ Action RPC (stdio 或 WebSocket) │
│ │ │ │
│ SkillManager (skill_mgr) │ │ SkillManager (skill_mgr) │
│ └─ 从 Box runtime 拉取 skills, 不可用时回落 data/skills │ │ └─ 从 Core SkillRepository 加载 Workspace-scoped skills │
└──────────────────────────────────────────────────────────────────┘ └──────────────────────────────────────────────────────────────────┘
@@ -59,10 +60,8 @@
│ NsjailBackend ──┘ (本地 CLI 或 fallback 到容器内 CLI) │ │ NsjailBackend ──┘ (本地 CLI 或 fallback 到容器内 CLI) │
│ E2BBackend (云沙箱, 需要 E2B_API_KEY) │ │ E2BBackend (云沙箱, 需要 E2B_API_KEY) │
│ │ │ │
BoxSkillStore Generic mount admission
list / get / create / update / delete allow-list + read-only + normalized target validation
│ ├─ scan_skill_directory / read_skill_file / write_skill_file │
│ └─ preview_skill_zip / install_skill_zip (zip 或 GitHub) │
│ │ │ │
│ aiohttp 单端口服务 (默认 :5410): │ │ aiohttp 单端口服务 (默认 :5410): │
│ /rpc/ws — Action RPC │ │ /rpc/ws — Action RPC │
@@ -85,7 +84,7 @@
**核心设计原则**: **核心设计原则**:
- Box Runtime 作为独立进程运行,通过 Action RPC 与 LangBot 主进程通信,两者复用 SDK 的 IO 层(Handler → Connection → Controller - Box Runtime 作为独立进程运行,通过 Action RPC 与 LangBot 主进程通信,两者复用 SDK 的 IO 层(Handler → Connection → Controller
- 一个 session_id 对应一个容器/沙箱实例。同一 session 内可并存多条 mount 与多个 managed process - 一个 session_id 对应一个容器/沙箱实例。同一 session 内可并存多条 mount 与多个 managed process
- Skill / 默认 exec / MCP Server 共享同一个 session 容器(详见 [box-session-scope.md](./box-session-scope.md) - Skill 仅是 Core 组装 mount 的业务来源;Box 与默认 exec / MCP Server 共享通用 session 和 mount 机制(详见 [box-session-scope.md](./box-session-scope.md)
--- ---
@@ -93,7 +92,7 @@
### 2.1 BoxService (`pkg/box/service.py`, 722 行) ### 2.1 BoxService (`pkg/box/service.py`, 722 行)
应用层门面,协调 Profile、安全校验、配额、连接、Skill 挂载与 Session 模板: 应用层门面,协调 Profile、安全校验、配额、连接、Core 生成的只读挂载与 Session 模板:
主要公开方法(按定义顺序): 主要公开方法(按定义顺序):
@@ -105,7 +104,7 @@ BoxService
├─ available (property) 连接状态 ├─ available (property) 连接状态
├─ resolve_box_session_id(query) 从 pipeline 模板解析 session_id ├─ resolve_box_session_id(query) 从 pipeline 模板解析 session_id
├─ build_skill_extra_mounts(query) 组装 pipeline-bound skill 的挂载列表 ├─ execute_tool(..., read_only_mounts=...) 接收 Core 组装的通用只读挂载
├─ execute_tool(parameters, query) Agent 调用 exec 时的入口 ├─ execute_tool(parameters, query) Agent 调用 exec 时的入口
│ ├─ _apply_profile / build_spec │ ├─ _apply_profile / build_spec
@@ -122,12 +121,6 @@ BoxService
├─ stop_managed_process(session_id, pid) 单独停止某个 managed process ├─ stop_managed_process(session_id, pid) 单独停止某个 managed process
├─ get_managed_process_websocket_url(...) 返回 WS attach URL ├─ get_managed_process_websocket_url(...) 返回 WS attach URL
├─ list_skills() / get_skill(name) Skill 元数据
├─ create_skill / update_skill / delete_skill Skill CRUD
├─ scan_skill_directory(path) 扫描目录
├─ list_skill_files / read_skill_file / write_skill_file
├─ preview_skill_zip / install_skill_zip zip / GitHub 安装
├─ shutdown() / dispose() 清理:RPC SHUTDOWN + 进程终止 ├─ shutdown() / dispose() 清理:RPC SHUTDOWN + 进程终止
├─ get_status() / get_sessions() / get_recent_errors() ├─ get_status() / get_sessions() / get_recent_errors()
└─ get_system_guidance() LLM 系统提示 └─ get_system_guidance() LLM 系统提示
@@ -137,7 +130,7 @@ BoxService
**输出截断**: 默认 4000 字符上限,保留前 60% + 后 40%,中间插入 `[...truncated...]` **输出截断**: 默认 4000 字符上限,保留前 60% + 后 40%,中间插入 `[...truncated...]`
**Skill 挂载合并**: `execute_tool()` 调用时,`build_skill_extra_mounts(query)`把当前 pipeline-bound 的所有 skill 的 `package_root` 作为 `extra_mounts` 加入 BoxSpec,挂在 `/workspace/.skills/<name>`。LLM 通过 `activate` 工具显式激活某个 skill 后,工具调用才允许引用这个 skill 的虚拟路径。 **Skill 挂载合并**: native loader 调用 `skill.build_execution_mounts()`把当前 pipeline-bound 的所有 skill 的 `package_root` 转成普通只读 mount,再通过 `BoxService.execute_tool(..., read_only_mounts=...)` 交给 Box,挂在 `/workspace/.skills/<name>`。LLM 通过 `activate` 工具显式激活某个 skill 后,工具调用才允许引用这个 skill 的虚拟路径BoxService 和 Box Runtime 都不知道这些 mount 来自 Skill
### 2.2 BoxRuntimeConnector (`pkg/box/connector.py`, 357 行) ### 2.2 BoxRuntimeConnector (`pkg/box/connector.py`, 357 行)
@@ -171,11 +164,10 @@ BoxService
``` ```
SkillManager SkillManager
├─ initialize() 调用 reload_skills() ├─ initialize() 调用 reload_skills()
├─ reload_skills() Box runtime list_skills() ├─ reload_skills() 从 Core SkillRepository 加载
│ 不可用则回落 data/skills/ 扫描 ├─ refresh_skill_from_disk() 检查单 skill 的缓存状态
├─ refresh_skill_from_disk() 单 skill 重新加载
├─ get_skill_by_name(name) ├─ get_skill_by_name(name)
└─ get_managed_skills_root() 返回 Box 视角的 skills_root 路径 └─ build_skill_aware_prompt_addition() 生成渐进披露索引
``` ```
skill 元数据通过 `parse_frontmatter` 解析 `SKILL.md` 头部(`name` / `description` / `instructions`),不再做整体扫描的代价(典型 < 50 个)。 skill 元数据通过 `parse_frontmatter` 解析 `SKILL.md` 头部(`name` / `description` / `instructions`),不再做整体扫描的代价(典型 < 50 个)。
@@ -295,7 +287,7 @@ start_managed_process(session, spec):
单端口 aiohttp 服务(默认 5410),通过路径区分(commit `8c71ec5` 合并端口): 单端口 aiohttp 服务(默认 5410),通过路径区分(commit `8c71ec5` 合并端口):
1. **Action RPC** (`/rpc/ws`): `BoxServerHandler` 处理所有 action,包括 `INIT` 配置注入、skill store 操作等 1. **Action RPC** (`/rpc/ws`): `BoxServerHandler` 处理 `INIT`、exec、session、managed-process 与状态等通用 action
2. **WS Relay** (`/v1/sessions/{id}/managed-process/ws``/v1/sessions/{id}/managed-process/{pid}/ws`): 双向桥接 WebSocket ↔ 指定 managed process stdin/stdout 2. **WS Relay** (`/v1/sessions/{id}/managed-process/ws``/v1/sessions/{id}/managed-process/{pid}/ws`): 双向桥接 WebSocket ↔ 指定 managed process stdin/stdout
stdio 模式同样会在 5410 启动 aiohttp,专门承担 managed process attachAction RPC 走 stdin/stdout。 stdio 模式同样会在 5410 启动 aiohttp,专门承担 managed process attachAction RPC 走 stdin/stdout。
@@ -304,7 +296,7 @@ stdio 模式同样会在 5410 启动 aiohttp,专门承担 managed process atta
`ActionRPCBoxClient` 封装 `Handler.call_action()` 调用: `ActionRPCBoxClient` 封装 `Handler.call_action()` 调用:
- 25+ 方法对应 25+ 个 RPC actionexec / session / managed-process / skill / status / shutdown - 方法对应 exec / session / managed-process / status / shutdown 等通用 RPC action,不暴露 Skill CRUD
- 错误还原: `_translate_action_error()` 通过字符串前缀匹配还原 SDK 侧异常类型 - 错误还原: `_translate_action_error()` 通过字符串前缀匹配还原 SDK 侧异常类型
- `execute()` timeout = 300s,其他默认 15s - `execute()` timeout = 300s,其他默认 15s
- `BoxRuntimeClient` 是 ABC,供后续可能的非 RPC 实现复用 - `BoxRuntimeClient` 是 ABC,供后续可能的非 RPC 实现复用
@@ -343,23 +335,29 @@ stdio 模式同样会在 5410 启动 aiohttp,专门承担 managed process atta
`BoxSpec` 校验器: `workdir` 默认继承 `mount_path``host_path` 支持 POSIX 和 Windows 路径;设置 `host_path``workdir` 必须在 `mount_path` 下。 `BoxSpec` 校验器: `workdir` 默认继承 `mount_path``host_path` 支持 POSIX 和 Windows 路径;设置 `host_path``workdir` 必须在 `mount_path` 下。
### 3.7 BoxSkillStore (`box/skill_store.py`, 647 行) ### 3.7 SkillStore (`langbot_plugin.skill_store`)
新增模块(commit `4ab3502`),把 skill 持久化收归 Box runtime Skill 包存储最初位于 Box Runtimeissue #2410 将通用实现抽到 Plugin SDK 顶层,由 Core 独占存储和 revision 语义
``` ```
BoxSkillStore SkillStore
├─ list_skills() / get_skill(name) ├─ list_skills() / get_skill(name)
├─ create_skill(data) / update_skill(name, data) / delete_skill(name) ├─ create_skill(data) / update_skill(name, data) / delete_skill(name)
├─ scan_skill_directory(path) 扫描目录返回候选 skill 包列表 ├─ scan_skill_directory(path) 扫描目录返回候选 skill 包列表
├─ list_skill_files(name, path) 浏览 skill 内文件树 ├─ list_skill_resources(name, path, revision) 按 revision 浏览只读资源
├─ read_skill_file(name, path) / write_skill_file(name, path, content) ├─ read_skill_resource(name, path, revision) 按 revision 读取 UTF-8 资源
├─ read_skill_file(...) / write_skill_file(...) 管理侧文件接口
├─ preview_skill_zip(zip_bytes, ...) 不落盘预览 zip 内容 ├─ preview_skill_zip(zip_bytes, ...) 不落盘预览 zip 内容
└─ install_skill_zip(zip_bytes, ...) 解压、校验、复制到 skills_root └─ install_skill_zip(zip_bytes, ...) 解压、校验、复制到 skills_root
└─ 支持 source_subdir / target_suffixcommit 1aa043f └─ 支持 source_subdir / target_suffixcommit 1aa043f
``` ```
GitHub 安装路径HTTP 层(`api/http/service/skill.py`)先 `git clone` 拉取,再走 `install_skill_zip` 或 directory 路径。Skill 文件存放于 `box.local.skills_root`(默认 `skills`,相对 `host_root`),容器内对应 `/workspace/.skills/` GitHub 安装路径由 Core HTTP 层下载归档,再交给 SkillRepository。Skill 文件位于独立的 `skills.root`,执行时由 Core 组装成通用只读 `BoxMountSpec` 并挂载到 `/workspace/.skills/`。Box 的模型、客户端和 Runtime 不包含 `skill_name`、Skill CRUD、revision 或 `SKILL.md` 语义。Core 与 Box Runtime SDK 按同一发布单元同步升级;Box 不保留旧 Skill RPC,也会拒绝旧的 Skill-aware payload 字段
仍保留的兼容仅用于已有数据的在线升级,不用于混版本协议:
1. Core 暂时读取 `box.local.skills_root`,兼容尚未生成 `skills.root` 的持久化配置。
2. 新安装默认暂时沿用历史 `./data/box/skills`,避免升级时搬迁已安装 Skill;下一大版本切换到 `./data/skills`
### 3.8 Security (`box/security.py`, 52 行) ### 3.8 Security (`box/security.py`, 52 行)
@@ -463,7 +461,7 @@ BuildAppStage.run(ap)
├─ ap.tool_mgr = tool_mgr ├─ ap.tool_mgr = tool_mgr
├─ ... (platform, pipeline) ... ├─ ... (platform, pipeline) ...
├─ SkillManager.initialize() (从 Box runtime 加载 skill 列表) ├─ SkillManager.initialize() (从 Core SkillRepository 加载 skill 列表)
└─ ... (RAG, HTTP, plugins) ... └─ ... (RAG, HTTP, plugins) ...
``` ```
@@ -480,7 +478,7 @@ except Exception as e:
logger.warning(f"Box runtime unavailable: {e}") logger.warning(f"Box runtime unavailable: {e}")
``` ```
**静默降级**: Box 初始化失败不会阻止应用启动,仅导致 6 个 native tool、所有 Skill 工具和 MCP-in-Box 工具不暴露给 LLM。与 Plugin 的行为不同(Plugin 失败会抛异常)。 **静默降级**: Box 初始化失败不会阻止应用启动6 个 native tool、`register_skill` 和 MCP-in-Box 工具不暴露给 LLM`activate` 与 Skill 只读资源工具继续由 Core 提供。与 Plugin 的行为不同(Plugin 失败会抛异常)。
### 5.3 销毁流程 ### 5.3 销毁流程
@@ -504,13 +502,16 @@ Box 额外做了 RPC SHUTDOWN 通知 Runtime 主动清理容器,比 Plugin 的
### config.yaml (重构后) ### config.yaml (重构后)
```yaml ```yaml
skills:
root: './data/box/skills' # Core-ownedBox 关闭时仍可管理/读取
box: box:
enabled: true # 整个 Box 子系统的总开关。设为 false 时: enabled: true # 整个 Box 子系统的总开关。设为 false 时:
# - 不连接远程 Box runtime,不 fork 本地 stdio 子进程 # - 不连接远程 Box runtime,不 fork 本地 stdio 子进程
# - sandbox 工具 (exec/read/write/edit/glob/grep) 不暴露给 LLM # - sandbox 工具 (exec/read/write/edit/glob/grep) 不暴露给 LLM
# - skill 添加/编辑 / GitHub 安装 / 文件写入全部拒绝 # - Agent 从 sandbox 注册 skill 的能力不可用
# - stdio 模式的 MCP server 启动时报错(http/sse 模式不受影响) # - stdio 模式的 MCP server 启动时报错(http/sse 模式不受影响)
# - skill 列表/读取保持只读可用 # - skill 管理、激活和只读资源保持可用
# BOX__ENABLED 环境变量可覆盖(统一约定) # BOX__ENABLED 环境变量可覆盖(统一约定)
backend: 'local' # 'local' (探测) / 'docker' / 'nsjail' / 'e2b' backend: 'local' # 'local' (探测) / 'docker' / 'nsjail' / 'e2b'
# 由 box.backend / BOX__BACKEND 选择后端 # 由 box.backend / BOX__BACKEND 选择后端
@@ -522,7 +523,6 @@ box:
image: '' # 覆盖 profile 默认 image image: '' # 覆盖 profile 默认 image
host_root: './data/box' # 工作区挂载根,Docker 部署需绝对路径 host_root: './data/box' # 工作区挂载根,Docker 部署需绝对路径
default_workspace: '' # 默认 '<host_root>/default' default_workspace: '' # 默认 '<host_root>/default'
skills_root: 'skills' # Box 管理的 skill 包目录(相对 host_root
allowed_mount_roots: # 默认 ['<host_root>'] allowed_mount_roots: # 默认 ['<host_root>']
- './data/box' - './data/box'
- '/tmp' - '/tmp'
@@ -561,16 +561,17 @@ volumes:
| 消费方 | Box 可用 | Box 不可用(disabled 或 failed) | | 消费方 | Box 可用 | Box 不可用(disabled 或 failed) |
|---|---|---| |---|---|---|
| native exec/read/write/edit/glob/grep 工具 | 暴露给 LLM | **不暴露** | | native exec/read/write/edit/glob/grep 工具 | 暴露给 LLM | **不暴露** |
| `activate` / `register_skill` 工具 | 暴露给 LLM | **不暴露** | | `activate` / Skill resource 工具 | 暴露给 LLM | 暴露给 LLM |
| `register_skill` 工具 | 暴露给 LLM | **不暴露**;直接调用返回 `sandbox_unavailable` |
| stdio MCP server | 在 Box 内启动 | **`_init_stdio_python_server` 抛 RuntimeError** 拒绝;不退化到宿主 stdio | | stdio MCP server | 在 Box 内启动 | **`_init_stdio_python_server` 抛 RuntimeError** 拒绝;不退化到宿主 stdio |
| http/sse MCP server | 正常 | 正常(不依赖 Box) | | http/sse MCP server | 正常 | 正常(不依赖 Box) |
| Skill 列表/读取 (`list_skills`/`get_skill`/`read_skill_file`) | 走 Box runtime | 走 LangBot 本地 `data/skills/` 只读 fallback | | Skill 列表/读取 (`list_skills`/`get_skill`/`read_skill_file`) | 走 Core SkillRepository | 走 Core SkillRepository |
| Skill 创建/编辑/安装/写文件 | 走 Box runtime | **HTTP 400** + 明确错误信息(`_require_box_for_write`) | | Skill 创建/编辑/安装/写文件 | 走 Core SkillRepository | 走 Core SkillRepository |
| Pipeline AI 配置中 `box-session-id-template` | 正常生效 | **前端 banner** 提示字段无效 | | Pipeline AI 配置中 `box-session-id-template` | 正常生效 | **前端 banner** 提示字段无效 |
| Pipeline 扩展页 `enable_all_skills` / 绑定 skill | 可编辑 | **前端禁用** + banner | | Pipeline 扩展页 `enable_all_skills` / 绑定 skill | 可编辑 | 可编辑 |
| 仪表盘 Box 状态卡片 | 绿点 / "已连接" | 灰点 / "已禁用"(disabled) 或 红点 / "已断开"(failed) | | 仪表盘 Box 状态卡片 | 绿点 / "已连接" | 灰点 / "已禁用"(disabled) 或 红点 / "已断开"(failed) |
> 后端拒写的边界条件:如果 `ap.box_service` **完全没装**(老式 dev mode,没经过 BuildAppStage),`_require_box_for_write` 视作 no-op,保留 `data/skills/` 本地路径——以兼容历史测试与最小化设置。生产环境总会装 `ap.box_service`,因此该 fallback 不会被触发 > Core 的 SkillRepository 是 `langbot_plugin.skill_store.SkillStore` 的异步 Workspace 适配层。默认 `skills.root` 保持原 `data/box/skills/tenants/...` 布局,升级时无需移动已安装 Skill;旧 `box.local.skills_root` 仅作为在线升级 fallback,并将在下一大版本删除。Box 只消费 Core 下发的通用只读 mount
### Pipeline 配置 (templates/metadata/pipeline/ai.yaml) ### Pipeline 配置 (templates/metadata/pipeline/ai.yaml)
+2 -2
View File
@@ -52,8 +52,8 @@
### S5. 挂载校验缺口 — Med-High ### S5. 挂载校验缺口 — Med-High
- **位置**: SDK `box/security.py` `_BLOCKED_HOST_PATHS_POSIX``box/backend.py``extra_mounts` 处理 - **位置**: SDK `box/security.py` `_BLOCKED_HOST_PATHS_POSIX``box/backend.py``extra_mounts` 处理
- **现状**: ① SDK 黑名单仍不含 `/`(前缀匹配,`host_path="/"` 可通过,挂载整个宿主 fs);用户 home、`/usr``/opt``/tmp` 也未拦截。② `validate_sandbox_security` 只校验 `spec.host_path`**从不遍历 `spec.extra_mounts`**——LangBot 侧 `allowed_mount_roots` 也只校验 `host_path`。当前 `extra_mounts` 仅由 `build_skill_extra_mounts` 内部填充(agent 不可达),但缺乏纵深防御:一旦 S1 的无认证 RPC 被触达,extra_mounts 可挂任意宿主路径,两层都不拦 - **现状**: grant-enforced 模式已经由 Core 与 Runtime 双重校验通用只读 mount(绝对路径 allow-list、存在性、只读模式、规范化且位于 `/workspace` 下的目标);它不再有 Skill 特例。遗留风险仅在 admission-disabled 的低信任直连场景:通用 `extra_mounts` 仍未统一套用 grant-enforced 白名单
- **要求**: SDK 黑名单加入 `/`(或改白名单)`extra_mounts` 在 SDK 与 LangBot 两侧都纳入挂载校验 - **要求**: admission-disabled 的外部控制面也复用同一套通用 mount 校验;SDK 黑名单加入 `/`(或全面改白名单)。
### S6. 容器加固缺失 — Med ### S6. 容器加固缺失 — Med
+1 -1
View File
@@ -18,7 +18,7 @@ has shipped the design largely as written:
| Docker / nsjail / E2B backends apply extra mounts | ✅ Shipped | Last gap closed by SDK commit `0fea9b1` (E2B) | | Docker / nsjail / E2B backends apply extra mounts | ✅ Shipped | Last gap closed by SDK commit `0fea9b1` (E2B) |
| `box-session-id-template` in `local-agent` pipeline config | ✅ Shipped | `templates/metadata/pipeline/ai.yaml`, default `{launcher_type}_{launcher_id}` | | `box-session-id-template` in `local-agent` pipeline config | ✅ Shipped | `templates/metadata/pipeline/ai.yaml`, default `{launcher_type}_{launcher_id}` |
| `BoxService.resolve_box_session_id(query)` | ✅ Shipped | `pkg/box/service.py:166` | | `BoxService.resolve_box_session_id(query)` | ✅ Shipped | `pkg/box/service.py:166` |
| `BoxService.build_skill_extra_mounts(query)` | ✅ Shipped | `pkg/box/service.py:189` | | `skill.build_execution_mounts(ap, query)` | ✅ Shipped | Core composes read-only packages; Box receives generic mounts |
| Skill exec uses unified container + extra mounts | ✅ Shipped | `pkg/provider/tools/loaders/native.py` skill branch | | Skill exec uses unified container + extra mounts | ✅ Shipped | `pkg/provider/tools/loaders/native.py` skill branch |
| MCP-in-Box uses shared persistent session, multi-process | ✅ Shipped (earlier than originally scoped) | SDK commit `529088e`, LangBot `mcp_stdio.py:_build_box_session_id` | | MCP-in-Box uses shared persistent session, multi-process | ✅ Shipped (earlier than originally scoped) | SDK commit `529088e`, LangBot `mcp_stdio.py:_build_box_session_id` |
| `BoxManagedProcessSpec.process_id` + multi-process per session | ✅ Shipped | `BoxRuntime` keeps `managed_processes: dict[pid, _ManagedProcess]` | | `BoxManagedProcessSpec.process_id` + multi-process per session | ✅ Shipped | `BoxRuntime` keeps `managed_processes: dict[pid, _ManagedProcess]` |
+1 -1
View File
@@ -51,7 +51,7 @@
| BoxService workspace quota | 优秀 | 前置/后置配额检查、超额清理 | | BoxService workspace quota | 优秀 | 前置/后置配额检查、超额清理 |
| BoxService 输出截断 | 优秀 | 短/精确边界/长输出、独立 stderr | | BoxService 输出截断 | 优秀 | 短/精确边界/长输出、独立 stderr |
| BoxService 可观测性 | 优秀 | 状态报告、error ring buffer、buffer 上限 | | BoxService 可观测性 | 优秀 | 状态报告、error ring buffer、buffer 上限 |
| BoxService session 模板 | 良好 | `resolve_box_session_id` + `build_skill_extra_mounts` 在 service / native / mcp 三处都有覆盖 | | BoxService session / mount contract | 良好 | `resolve_box_session_id` + generic `read_only_mounts`; Skill mount composition is covered in the Core loader |
| RPC client/server 协议 | 优秀 | execute/get_sessions/delete/create/conflict error | | RPC client/server 协议 | 优秀 | execute/get_sessions/delete/create/conflict error |
| BoxRuntimeConnector | 良好 | local/remote 模式、Docker 平台、relay URL、心跳与重连回调 | | BoxRuntimeConnector | 良好 | local/remote 模式、Docker 平台、relay URL、心跳与重连回调 |
| BoxWorkspaceSession | 良好 | payload 构建、managed process 路径重写、stage host file | | BoxWorkspaceSession | 良好 | payload 构建、managed process 路径重写、stage host file |
-34
View File
@@ -43,8 +43,6 @@ Two kinds of key are accepted:
Invalid, revoked, or expired keys get `401 Unauthorized`. A valid key whose Invalid, revoked, or expired keys get `401 Unauthorized`. A valid key whose
scopes do not authorize a tool gets `403 Forbidden`. scopes do not authorize a tool gets `403 Forbidden`.
To inspect key identity and permissions, call `GET /api/v1/system/context` with the API key.
## Client configuration ## Client configuration
```json ```json
@@ -88,38 +86,6 @@ already have a default pipeline.
4. Use `list_*` tools to discover, then `get_*` / `create_*` / `update_*` / 4. Use `list_*` tools to discover, then `get_*` / `create_*` / `update_*` /
`delete_*` as needed. `delete_*` as needed.
## ChatGPT / Codex subscription providers
`list_model_providers` can return the `openai-codex` requester. Its OAuth
credentials are server-only and are not provider API keys. Never ask a user
to paste ChatGPT access tokens, refresh tokens, or a Codex auth cache into an
MCP tool or model configuration.
A human connects or disconnects the subscription through **Models → provider
settings** in the LangBot web UI. The provider-scoped `/codex/*` authentication
routes deliberately require a browser-user session and are not exposed as MCP
tools or authorized by a LangBot API key. Once connected, models are managed
and selected through the normal provider/model workflow. A disconnected
provider must be reauthorized; do not silently replace it with API-key billing.
See [ChatGPT / Codex subscription](../../../docs/CODEX_SUBSCRIPTION.md) for setup,
usage limits, and the personal-account versus shared-service boundary.
## Provider deletion
The curated MCP surface currently lists providers but has no provider-deletion
tool. In the web UI, **Edit Provider → Delete** asks for confirmation before
removing that provider and all its LLM, embedding, and rerank models. This is
irreversible; never interpret a request to edit a provider as authorization to
delete it.
The equivalent HTTP operation is
`DELETE /api/v1/provider/providers/{uuid}?cascade=true`, requiring
`resource.manage` in the authenticated Workspace. Omitting `cascade` preserves
the existing refusal to delete providers that still have models. Cloud-managed
providers remain protected. Cascade deletion removes stored Codex authorization
state as well; it is not the same operation as disconnecting an account.
## Implementation & maintenance (for LangBot developers) ## Implementation & maintenance (for LangBot developers)
- Server: `src/langbot/pkg/api/mcp/server.py` (FastMCP). Tools call the service - Server: `src/langbot/pkg/api/mcp/server.py` (FastMCP). Tools call the service
@@ -15,7 +15,6 @@ from ....workspace.collaboration import MembershipPermissionError, WorkspaceColl
from ....workspace.errors import WorkspaceNotFoundError from ....workspace.errors import WorkspaceNotFoundError
from ....cloud.entitlements import EntitlementUnavailableError from ....cloud.entitlements import EntitlementUnavailableError
from ....core.errors import TaskCapacityError from ....core.errors import TaskCapacityError
from ....provider.modelmgr.codex_errors import CodexProviderError
from ..authz import ( from ..authz import (
AuthenticationDeniedError, AuthenticationDeniedError,
AuthorizationError, AuthorizationError,
@@ -248,8 +247,6 @@ class RouterGroup(abc.ABC):
return await f(*args, **kwargs) return await f(*args, **kwargs)
except Exception as e: # 自动 500 except Exception as e: # 自动 500
if isinstance(e, CodexProviderError):
return self.http_status(e.status_code, e.error_code, str(e))
if isinstance(e, AuthorizationError): if isinstance(e, AuthorizationError):
return self.http_status(e.status_code, e.error_code, str(e)) return self.http_status(e.status_code, e.error_code, str(e))
if isinstance(e, WorkspaceNotFoundError): if isinstance(e, WorkspaceNotFoundError):
@@ -3,7 +3,6 @@ import quart
from ....authz import Permission, has_permission from ....authz import Permission, has_permission
from ....context import RequestContext from ....context import RequestContext
from ... import group from ... import group
from .query import resolve_include_secret
@group.group_class('models/llm', '/api/v1/provider/models/llm') @group.group_class('models/llm', '/api/v1/provider/models/llm')
@@ -17,12 +16,7 @@ class LLMModelsRouterGroup(group.RouterGroup):
) )
async def _(request_context: RequestContext) -> str: async def _(request_context: RequestContext) -> str:
provider_uuid = quart.request.args.get('provider_uuid') provider_uuid = quart.request.args.get('provider_uuid')
include_secret, error = resolve_include_secret( include_secret = has_permission(request_context, Permission.PROVIDER_SECRET_MANAGE)
quart.request.args.get('include_secret'),
permitted=has_permission(request_context, Permission.PROVIDER_SECRET_MANAGE),
)
if error:
return self.http_status(400, -1, error)
if provider_uuid: if provider_uuid:
models = await self.ap.llm_model_service.get_llm_models_by_provider( models = await self.ap.llm_model_service.get_llm_models_by_provider(
request_context, request_context,
@@ -59,16 +53,10 @@ class LLMModelsRouterGroup(group.RouterGroup):
permission=Permission.RESOURCE_VIEW, permission=Permission.RESOURCE_VIEW,
) )
async def _(model_uuid: str, request_context: RequestContext) -> str: async def _(model_uuid: str, request_context: RequestContext) -> str:
include_secret, error = resolve_include_secret(
quart.request.args.get('include_secret'),
permitted=has_permission(request_context, Permission.PROVIDER_SECRET_MANAGE),
)
if error:
return self.http_status(400, -1, error)
model = await self.ap.llm_model_service.get_llm_model( model = await self.ap.llm_model_service.get_llm_model(
request_context, request_context,
model_uuid, model_uuid,
include_secret=include_secret, include_secret=has_permission(request_context, Permission.PROVIDER_SECRET_MANAGE),
) )
if model is None: if model is None:
return self.http_status(404, -1, 'model not found') return self.http_status(404, -1, 'model not found')
@@ -123,12 +111,7 @@ class EmbeddingModelsRouterGroup(group.RouterGroup):
) )
async def _(request_context: RequestContext) -> str: async def _(request_context: RequestContext) -> str:
provider_uuid = quart.request.args.get('provider_uuid') provider_uuid = quart.request.args.get('provider_uuid')
include_secret, error = resolve_include_secret( include_secret = has_permission(request_context, Permission.PROVIDER_SECRET_MANAGE)
quart.request.args.get('include_secret'),
permitted=has_permission(request_context, Permission.PROVIDER_SECRET_MANAGE),
)
if error:
return self.http_status(400, -1, error)
if provider_uuid: if provider_uuid:
models = await self.ap.embedding_models_service.get_embedding_models_by_provider( models = await self.ap.embedding_models_service.get_embedding_models_by_provider(
request_context, request_context,
@@ -165,16 +148,10 @@ class EmbeddingModelsRouterGroup(group.RouterGroup):
permission=Permission.RESOURCE_VIEW, permission=Permission.RESOURCE_VIEW,
) )
async def _(model_uuid: str, request_context: RequestContext) -> str: async def _(model_uuid: str, request_context: RequestContext) -> str:
include_secret, error = resolve_include_secret(
quart.request.args.get('include_secret'),
permitted=has_permission(request_context, Permission.PROVIDER_SECRET_MANAGE),
)
if error:
return self.http_status(400, -1, error)
model = await self.ap.embedding_models_service.get_embedding_model( model = await self.ap.embedding_models_service.get_embedding_model(
request_context, request_context,
model_uuid, model_uuid,
include_secret=include_secret, include_secret=has_permission(request_context, Permission.PROVIDER_SECRET_MANAGE),
) )
if model is None: if model is None:
return self.http_status(404, -1, 'model not found') return self.http_status(404, -1, 'model not found')
@@ -231,12 +208,7 @@ class RerankModelsRouterGroup(group.RouterGroup):
) )
async def _(request_context: RequestContext) -> str: async def _(request_context: RequestContext) -> str:
provider_uuid = quart.request.args.get('provider_uuid') provider_uuid = quart.request.args.get('provider_uuid')
include_secret, error = resolve_include_secret( include_secret = has_permission(request_context, Permission.PROVIDER_SECRET_MANAGE)
quart.request.args.get('include_secret'),
permitted=has_permission(request_context, Permission.PROVIDER_SECRET_MANAGE),
)
if error:
return self.http_status(400, -1, error)
if provider_uuid: if provider_uuid:
models = await self.ap.rerank_models_service.get_rerank_models_by_provider( models = await self.ap.rerank_models_service.get_rerank_models_by_provider(
request_context, request_context,
@@ -273,16 +245,10 @@ class RerankModelsRouterGroup(group.RouterGroup):
permission=Permission.RESOURCE_VIEW, permission=Permission.RESOURCE_VIEW,
) )
async def _(model_uuid: str, request_context: RequestContext) -> str: async def _(model_uuid: str, request_context: RequestContext) -> str:
include_secret, error = resolve_include_secret(
quart.request.args.get('include_secret'),
permitted=has_permission(request_context, Permission.PROVIDER_SECRET_MANAGE),
)
if error:
return self.http_status(400, -1, error)
model = await self.ap.rerank_models_service.get_rerank_model( model = await self.ap.rerank_models_service.get_rerank_model(
request_context, request_context,
model_uuid, model_uuid,
include_secret=include_secret, include_secret=has_permission(request_context, Permission.PROVIDER_SECRET_MANAGE),
) )
if model is None: if model is None:
return self.http_status(404, -1, 'model not found') return self.http_status(404, -1, 'model not found')
@@ -3,86 +3,11 @@ import quart
from ....authz import Permission, has_permission from ....authz import Permission, has_permission
from ....context import RequestContext from ....context import RequestContext
from ... import group from ... import group
from .query import resolve_include_secret
@group.group_class('models/providers', '/api/v1/provider/providers') @group.group_class('models/providers', '/api/v1/provider/providers')
class ModelProvidersRouterGroup(group.RouterGroup): class ModelProvidersRouterGroup(group.RouterGroup):
async def initialize(self) -> None: async def initialize(self) -> None:
# Subscription authorization is an interactive, browser-user-only surface.
@self.route(
'/<provider_uuid>/codex/status',
methods=['GET'],
auth_type=group.AuthType.USER_TOKEN,
permission=Permission.PROVIDER_SECRET_MANAGE,
)
async def codex_status(provider_uuid: str, request_context: RequestContext):
try:
return self.success(
data=await self.ap.provider_service.codex_auth.status(request_context, provider_uuid)
)
except ValueError as exc:
return self.http_status(400, -1, str(exc))
@self.route(
'/<provider_uuid>/codex/device',
methods=['POST'],
auth_type=group.AuthType.USER_TOKEN,
permission=Permission.PROVIDER_SECRET_MANAGE,
)
async def codex_device(provider_uuid: str, request_context: RequestContext):
try:
return self.success(
data=await self.ap.provider_service.codex_auth.start(request_context, provider_uuid)
)
except ValueError as exc:
return self.http_status(400, -1, str(exc))
@self.route(
'/<provider_uuid>/codex/device/poll',
methods=['POST'],
auth_type=group.AuthType.USER_TOKEN,
permission=Permission.PROVIDER_SECRET_MANAGE,
)
async def codex_poll(provider_uuid: str, request_context: RequestContext):
body = await quart.request.get_json()
if not isinstance(body, dict):
return self.http_status(400, -1, 'JSON object required')
try:
return self.success(
data=await self.ap.provider_service.codex_auth.poll(
request_context, provider_uuid, body.get('authorization_id')
)
)
except ValueError as exc:
return self.http_status(400, -1, str(exc))
@self.route(
'/<provider_uuid>/codex/auth',
methods=['DELETE'],
auth_type=group.AuthType.USER_TOKEN,
permission=Permission.PROVIDER_SECRET_MANAGE,
)
async def codex_disconnect(provider_uuid: str, request_context: RequestContext):
try:
await self.ap.provider_service.codex_auth.disconnect(request_context, provider_uuid)
return self.success()
except ValueError as exc:
return self.http_status(400, -1, str(exc))
@self.route(
'/<provider_uuid>/codex/device/<authorization_id>',
methods=['DELETE'],
auth_type=group.AuthType.USER_TOKEN,
permission=Permission.PROVIDER_SECRET_MANAGE,
)
async def codex_cancel(provider_uuid: str, authorization_id: str, request_context: RequestContext):
try:
await self.ap.provider_service.codex_auth.cancel(request_context, provider_uuid, authorization_id)
return self.success()
except ValueError as exc:
return self.http_status(400, -1, str(exc))
@self.route( @self.route(
'', '',
methods=['GET'], methods=['GET'],
@@ -90,15 +15,9 @@ class ModelProvidersRouterGroup(group.RouterGroup):
permission=Permission.RESOURCE_VIEW, permission=Permission.RESOURCE_VIEW,
) )
async def _(request_context: RequestContext) -> str: async def _(request_context: RequestContext) -> str:
include_secret, error = resolve_include_secret(
quart.request.args.get('include_secret'),
permitted=has_permission(request_context, Permission.PROVIDER_SECRET_MANAGE),
)
if error:
return self.http_status(400, -1, error)
providers = await self.ap.provider_service.get_providers( providers = await self.ap.provider_service.get_providers(
request_context, request_context,
include_secret=include_secret, include_secret=has_permission(request_context, Permission.PROVIDER_SECRET_MANAGE),
) )
for provider in providers: for provider in providers:
counts = await self.ap.provider_service.get_provider_model_counts(request_context, provider['uuid']) counts = await self.ap.provider_service.get_provider_model_counts(request_context, provider['uuid'])
@@ -128,16 +47,10 @@ class ModelProvidersRouterGroup(group.RouterGroup):
permission=Permission.RESOURCE_VIEW, permission=Permission.RESOURCE_VIEW,
) )
async def _(provider_uuid: str, request_context: RequestContext) -> str: async def _(provider_uuid: str, request_context: RequestContext) -> str:
include_secret, error = resolve_include_secret(
quart.request.args.get('include_secret'),
permitted=has_permission(request_context, Permission.PROVIDER_SECRET_MANAGE),
)
if error:
return self.http_status(400, -1, error)
provider = await self.ap.provider_service.get_provider( provider = await self.ap.provider_service.get_provider(
request_context, request_context,
provider_uuid, provider_uuid,
include_secret=include_secret, include_secret=has_permission(request_context, Permission.PROVIDER_SECRET_MANAGE),
) )
if provider is None: if provider is None:
return self.http_status(404, -1, 'provider not found') return self.http_status(404, -1, 'provider not found')
@@ -169,15 +82,7 @@ class ModelProvidersRouterGroup(group.RouterGroup):
) )
async def _(provider_uuid: str, request_context: RequestContext) -> str: async def _(provider_uuid: str, request_context: RequestContext) -> str:
try: try:
cascade_values = quart.request.args.getlist('cascade') await self.ap.provider_service.delete_provider(request_context, provider_uuid)
if cascade_values:
if len(cascade_values) != 1 or cascade_values[0] not in ('true', 'false'):
return self.http_status(400, -1, 'cascade must be a single true or false value')
await self.ap.provider_service.delete_provider(
request_context, provider_uuid, cascade=cascade_values[0] == 'true'
)
else:
await self.ap.provider_service.delete_provider(request_context, provider_uuid)
return self.success() return self.success()
except ValueError as e: except ValueError as e:
return self.http_status(400, -1, str(e)) return self.http_status(400, -1, str(e))
@@ -1,15 +0,0 @@
from __future__ import annotations
def resolve_include_secret(raw_value: str | None, *, permitted: bool) -> tuple[bool, str | None]:
"""Resolve the optional secret projection query parameter."""
if raw_value is None:
return permitted, None
value = raw_value.strip().lower()
if value == 'false':
return False, None
if value == 'true':
return permitted, None
return False, 'include_secret must be either true or false'
@@ -2,9 +2,6 @@ from __future__ import annotations
import quart import quart
from langbot.pkg.cloud.entitlements import EntitlementFeatureUnavailableError
from langbot_plugin.box.errors import BoxError
from ...authz import Permission from ...authz import Permission
from ...context import RequestContext from ...context import RequestContext
from .. import group from .. import group
@@ -24,12 +21,7 @@ class SkillsRouterGroup(group.RouterGroup):
async def list_skills(request_context: RequestContext) -> quart.Response: async def list_skills(request_context: RequestContext) -> quart.Response:
try: try:
skills = await self.ap.skill_service.list_skills(request_context) skills = await self.ap.skill_service.list_skills(request_context)
except EntitlementFeatureUnavailableError: except ValueError as exc:
# Plans without managed sandbox support have no runnable skills.
# Treat that capability absence as an empty collection so the
# shared UI can render normally instead of surfacing a 500.
return self.success(data={'skills': []})
except (ValueError, BoxError) as exc:
return self.http_status(400, -1, str(exc)) return self.http_status(400, -1, str(exc))
return self.success(data={'skills': skills}) return self.success(data={'skills': skills})
@@ -47,7 +39,7 @@ class SkillsRouterGroup(group.RouterGroup):
try: try:
skill = await self.ap.skill_service.create_skill(request_context, data) skill = await self.ap.skill_service.create_skill(request_context, data)
return self.success(data={'skill': skill}) return self.success(data={'skill': skill})
except (ValueError, BoxError) as exc: except ValueError as exc:
return self.http_status(400, -1, str(exc)) return self.http_status(400, -1, str(exc))
@self.route( @self.route(
@@ -59,7 +51,7 @@ class SkillsRouterGroup(group.RouterGroup):
async def get_skill(skill_name: str, request_context: RequestContext) -> quart.Response: async def get_skill(skill_name: str, request_context: RequestContext) -> quart.Response:
try: try:
skill = await self.ap.skill_service.get_skill(request_context, skill_name) skill = await self.ap.skill_service.get_skill(request_context, skill_name)
except (ValueError, BoxError) as exc: except ValueError as exc:
return self.http_status(400, -1, str(exc)) return self.http_status(400, -1, str(exc))
if not skill: if not skill:
return self.http_status(404, -1, 'Skill not found') return self.http_status(404, -1, 'Skill not found')
@@ -77,13 +69,13 @@ class SkillsRouterGroup(group.RouterGroup):
try: try:
skill = await self.ap.skill_service.update_skill(request_context, skill_name, data) skill = await self.ap.skill_service.update_skill(request_context, skill_name, data)
return self.success(data={'skill': skill}) return self.success(data={'skill': skill})
except (ValueError, BoxError) as exc: except ValueError as exc:
return self.http_status(400, -1, str(exc)) return self.http_status(400, -1, str(exc))
try: try:
await self.ap.skill_service.delete_skill(request_context, skill_name) await self.ap.skill_service.delete_skill(request_context, skill_name)
return self.success() return self.success()
except (ValueError, BoxError) as exc: except ValueError as exc:
return self.http_status(400, -1, str(exc)) return self.http_status(400, -1, str(exc))
@self.route( @self.route(
@@ -105,7 +97,7 @@ class SkillsRouterGroup(group.RouterGroup):
include_hidden=include_hidden, include_hidden=include_hidden,
) )
return self.success(data=result) return self.success(data=result)
except (ValueError, BoxError) as exc: except ValueError as exc:
return self.http_status(400, -1, str(exc)) return self.http_status(400, -1, str(exc))
@self.route( @self.route(
@@ -118,7 +110,7 @@ class SkillsRouterGroup(group.RouterGroup):
try: try:
result = await self.ap.skill_service.read_skill_file(request_context, skill_name, path) result = await self.ap.skill_service.read_skill_file(request_context, skill_name, path)
return self.success(data=result) return self.success(data=result)
except (ValueError, BoxError) as exc: except ValueError as exc:
return self.http_status(400, -1, str(exc)) return self.http_status(400, -1, str(exc))
@self.route( @self.route(
@@ -136,7 +128,7 @@ class SkillsRouterGroup(group.RouterGroup):
try: try:
result = await self.ap.skill_service.write_skill_file(request_context, skill_name, path, content) result = await self.ap.skill_service.write_skill_file(request_context, skill_name, path, content)
return self.success(data=result) return self.success(data=result)
except (ValueError, BoxError) as exc: except ValueError as exc:
return self.http_status(400, -1, str(exc)) return self.http_status(400, -1, str(exc))
@self.route( @self.route(
@@ -170,7 +162,7 @@ class SkillsRouterGroup(group.RouterGroup):
try: try:
skill = await self.ap.skill_service.install_from_github(request_context, data) skill = await self.ap.skill_service.install_from_github(request_context, data)
return self.success(data={'skills': skill}) return self.success(data={'skills': skill})
except (ValueError, BoxError) as exc: except ValueError as exc:
return self.http_status(400, -1, str(exc)) return self.http_status(400, -1, str(exc))
except Exception: except Exception:
raise raise
@@ -194,7 +186,7 @@ class SkillsRouterGroup(group.RouterGroup):
try: try:
preview = await self.ap.skill_service.preview_install_from_github(request_context, data) preview = await self.ap.skill_service.preview_install_from_github(request_context, data)
return self.success(data={'skills': preview}) return self.success(data={'skills': preview})
except (ValueError, BoxError) as exc: except ValueError as exc:
return self.http_status(400, -1, str(exc)) return self.http_status(400, -1, str(exc))
except Exception: except Exception:
raise raise
@@ -219,7 +211,7 @@ class SkillsRouterGroup(group.RouterGroup):
source_paths=form.getlist('source_paths'), source_paths=form.getlist('source_paths'),
) )
return self.success(data={'skills': skill}) return self.success(data={'skills': skill})
except (ValueError, BoxError) as exc: except ValueError as exc:
return self.http_status(400, -1, str(exc)) return self.http_status(400, -1, str(exc))
except Exception: except Exception:
raise raise
@@ -242,7 +234,7 @@ class SkillsRouterGroup(group.RouterGroup):
filename=file.filename or '', filename=file.filename or '',
) )
return self.success(data={'skills': preview}) return self.success(data={'skills': preview})
except (ValueError, BoxError) as exc: except ValueError as exc:
return self.http_status(400, -1, str(exc)) return self.http_status(400, -1, str(exc))
except Exception: except Exception:
raise raise
@@ -261,5 +253,5 @@ class SkillsRouterGroup(group.RouterGroup):
try: try:
result = await self.ap.skill_service.scan_directory_async(request_context, path) result = await self.ap.skill_service.scan_directory_async(request_context, path)
return self.success(data=result) return self.success(data=result)
except (ValueError, BoxError) as exc: except ValueError as exc:
return self.http_status(400, -1, str(exc)) return self.http_status(400, -1, str(exc))
@@ -7,116 +7,14 @@ from .. import group
from .....utils import constants from .....utils import constants
from .....entity.persistence.metadata import WorkspaceMetadata from .....entity.persistence.metadata import WorkspaceMetadata
from ...authz import Permission from ...authz import Permission
from ...context import PrincipalType, RequestContext from ...context import RequestContext
from .....provider.tools.loaders.mcp_policy import stdio_mcp_enabled from .....provider.tools.loaders.mcp_policy import stdio_mcp_enabled
from .....workspace.invitation_delivery import InvitationDeliveryService from .....workspace.invitation_delivery import InvitationDeliveryService
SYSTEM_CAPABILITY_OPERATIONS = (
'bot.list',
'bot.get',
'bot.create',
'bot.update',
'bot.delete',
'pipeline.list',
'pipeline.get',
'pipeline.create',
'pipeline.update',
'pipeline.delete',
'pipeline.copy',
'task.list',
'task.get',
'knowledge_base.list',
'knowledge_base.get',
'knowledge_base.create',
'knowledge_base.update',
'knowledge_base.delete',
'knowledge_base.file.list',
'knowledge_base.file.store',
'knowledge_base.file.delete',
'knowledge_base.retrieve',
'file.document.upload',
'plugin.install.github',
'plugin.install.marketplace',
'plugin.install.local',
'plugin.upgrade',
'plugin.get',
'plugin.list',
'plugin.config.get',
'plugin.config.update',
'plugin.logs',
'plugin.delete',
'provider.list',
'provider.get',
'provider.create',
'provider.update',
'provider.delete',
'provider.scan_models',
'model.llm.list',
'model.llm.get',
'model.llm.create',
'model.llm.update',
'model.llm.delete',
'model.llm.test',
'model.embedding.list',
'model.embedding.get',
'model.embedding.create',
'model.embedding.update',
'model.embedding.delete',
'model.embedding.test',
'model.rerank.list',
'model.rerank.get',
'model.rerank.create',
'model.rerank.update',
'model.rerank.delete',
'model.rerank.test',
'skill.list',
'skill.get',
'skill.create',
'skill.update',
'skill.delete',
'skill.files.list',
'skill.files.read',
'skill.files.write',
'skill.preview',
'skill.install.github',
'skill.install.upload',
'mcp_server.list',
'mcp_server.get',
'mcp_server.create',
'mcp_server.update',
'mcp_server.delete',
'mcp_server.resources',
'mcp_server.resource_templates',
'mcp_server.resource_read',
'mcp_server.logs',
'mcp_server.test',
)
@group.group_class('system', '/api/v1/system') @group.group_class('system', '/api/v1/system')
class SystemRouterGroup(group.RouterGroup): class SystemRouterGroup(group.RouterGroup):
async def initialize(self) -> None: async def initialize(self) -> None:
@self.route('/context', methods=['GET'], auth_type=group.AuthType.API_KEY)
async def _(request_context: RequestContext) -> str:
return self.success(
data={
'instance_uuid': request_context.instance_uuid,
'workspace_uuid': request_context.workspace_uuid,
'api_key_id': request_context.principal.api_key_uuid,
'permissions': sorted(request_context.workspace.permissions),
}
)
@self.route('/capabilities', methods=['GET'], auth_type=group.AuthType.API_KEY)
async def _() -> str:
return self.success(
data={
'schema_version': 1,
'operations': {operation: {'supported': True} for operation in SYSTEM_CAPABILITY_OPERATIONS},
}
)
@self.route('/info', methods=['GET'], auth_type=group.AuthType.NONE) @self.route('/info', methods=['GET'], auth_type=group.AuthType.NONE)
async def _() -> str: async def _() -> str:
# Read wizard_status and wizard_progress from metadata table # Read wizard_status and wizard_progress from metadata table
@@ -325,7 +223,7 @@ class SystemRouterGroup(group.RouterGroup):
@self.route( @self.route(
'/tasks', '/tasks',
methods=['GET'], methods=['GET'],
auth_type=group.AuthType.USER_TOKEN_OR_API_KEY, auth_type=group.AuthType.USER_TOKEN,
permission=Permission.RESOURCE_VIEW, permission=Permission.RESOURCE_VIEW,
) )
async def _(request_context: RequestContext) -> str: async def _(request_context: RequestContext) -> str:
@@ -344,23 +242,18 @@ class SystemRouterGroup(group.RouterGroup):
instance_uuid=request_context.instance_uuid, instance_uuid=request_context.instance_uuid,
workspace_uuid=request_context.workspace_uuid, workspace_uuid=request_context.workspace_uuid,
placement_generation=request_context.placement_generation, placement_generation=request_context.placement_generation,
public=request_context.principal.principal_type == PrincipalType.API_KEY,
) )
) )
@self.route( @self.route(
'/tasks/<task_id>', '/tasks/<task_id>',
methods=['GET'], methods=['GET'],
auth_type=group.AuthType.USER_TOKEN_OR_API_KEY, auth_type=group.AuthType.USER_TOKEN,
permission=Permission.RESOURCE_VIEW, permission=Permission.RESOURCE_VIEW,
) )
async def _(task_id: str, request_context: RequestContext) -> str: async def _(task_id: str, request_context: RequestContext) -> str:
try:
task_index = int(task_id)
except (TypeError, ValueError):
return self.http_status(404, 404, 'Task not found')
task = self.ap.task_mgr.get_task_by_id( task = self.ap.task_mgr.get_task_by_id(
task_index, int(task_id),
instance_uuid=request_context.instance_uuid, instance_uuid=request_context.instance_uuid,
workspace_uuid=request_context.workspace_uuid, workspace_uuid=request_context.workspace_uuid,
placement_generation=request_context.placement_generation, placement_generation=request_context.placement_generation,
@@ -369,8 +262,6 @@ class SystemRouterGroup(group.RouterGroup):
if task is None: if task is None:
return self.http_status(404, 404, 'Task not found') return self.http_status(404, 404, 'Task not found')
if request_context.principal.principal_type == PrincipalType.API_KEY:
return self.success(data=task.to_public_dict())
return self.success(data=task.to_dict()) return self.success(data=task.to_dict())
@self.route( @self.route(
@@ -2,8 +2,6 @@ import quart
import argon2 import argon2
import asyncio import asyncio
import datetime import datetime
import hmac
import time
import uuid import uuid
from urllib.parse import parse_qs, urlsplit from urllib.parse import parse_qs, urlsplit
@@ -13,33 +11,6 @@ from ...context import RequestContext
from .....cloud.launch import SpaceLaunchError from .....cloud.launch import SpaceLaunchError
from ...service.user import ControlPlaneDirectoryRequiredError, PublicRegistrationClosedError from ...service.user import ControlPlaneDirectoryRequiredError, PublicRegistrationClosedError
# Fixed-window admission quota for the unauthenticated reset-password endpoint (#2392).
# The admission check and slot bump share ONE synchronous critical section with no await
# points, so concurrent bursts within a single event loop cannot slip past accounting.
# Every admitted attempt consumes quota (regardless of success), which throttles both the
# legacy 24-bit keyspace exhaustion and brute-force on modern high-entropy keys.
# NOTE: this state is process-local; multi-worker deployments need a shared limiter upstream.
_MAX_RESET_ATTEMPTS_PER_WINDOW = 5
_RESET_WINDOW_SECONDS = 15 * 60
_reset_password_state: dict = {'window_started_at': 0.0, 'attempts': 0}
def _admit_reset_attempt(now: float) -> bool:
"""Atomically reserve one reset-password admission slot.
Must stay await-free: running to completion without suspension makes the
check-and-increment atomic under the single-threaded event loop.
"""
st = _reset_password_state
if now - st['window_started_at'] >= _RESET_WINDOW_SECONDS:
st['window_started_at'] = now
st['attempts'] = 0
if st['attempts'] >= _MAX_RESET_ATTEMPTS_PER_WINDOW:
return False
st['attempts'] += 1
return True
@group.group_class('user', '/api/v1/user') @group.group_class('user', '/api/v1/user')
class UserRouterGroup(group.RouterGroup): class UserRouterGroup(group.RouterGroup):
@@ -110,12 +81,6 @@ class UserRouterGroup(group.RouterGroup):
@self.route('/reset-password', methods=['POST'], auth_type=group.AuthType.NONE) @self.route('/reset-password', methods=['POST'], auth_type=group.AuthType.NONE)
async def _() -> str: async def _() -> str:
# Admit (or reject) BEFORE touching the body or any service call (#2392):
# rejecting requests never reach the slow path, and quota accounting happens
# synchronously at entry, closing the post-await race of burst requests.
if not _admit_reset_attempt(time.monotonic()):
return self.http_status(429, -1, 'Too many attempts, try again later')
json_data = await quart.request.json json_data = await quart.request.json
user_email = json_data['user'] user_email = json_data['user']
@@ -133,18 +98,7 @@ class UserRouterGroup(group.RouterGroup):
if user_obj is None: if user_obj is None:
return self.http_status(400, -1, 'User not found') return self.http_status(400, -1, 'User not found')
stored_key = self.ap.instance_config.data['system']['recovery_key'] if recovery_key != self.ap.instance_config.data['system']['recovery_key']:
try:
key_matches = (
isinstance(recovery_key, str)
and isinstance(stored_key, str)
and hmac.compare_digest(recovery_key.encode(), stored_key.encode())
)
except UnicodeEncodeError:
# JSON can contain lone surrogates, which are not valid UTF-8.
key_matches = False
if not key_matches:
return self.http_status(403, -1, 'Invalid recovery key') return self.http_status(403, -1, 'Invalid recovery key')
await self.ap.user_service.reset_password(user_email, new_password) await self.ap.user_service.reset_password(user_email, new_password)
+9 -16
View File
@@ -446,19 +446,15 @@ class MCPService:
persisted_session = runtime_mcp_session persisted_session = runtime_mcp_session
async def _refresh_and_report() -> None: async def _refresh_and_report() -> None:
try: needs_start = persisted_session.status == MCPSessionStatus.ERROR or persisted_session.session is None
needs_start = ( if needs_start:
persisted_session.status == MCPSessionStatus.ERROR or persisted_session.session is None await persisted_session.start()
) else:
if needs_start: try:
await persisted_session.refresh()
except Exception:
await persisted_session.start() await persisted_session.start()
else: ctx.metadata['runtime_info'] = persisted_session.get_runtime_info_dict()
try:
await persisted_session.refresh()
except Exception:
await persisted_session.start()
finally:
ctx.metadata['runtime_info'] = persisted_session.get_runtime_info_dict()
coroutine = _refresh_and_report() coroutine = _refresh_and_report()
else: else:
@@ -475,11 +471,8 @@ class MCPService:
async def _run_and_cleanup() -> None: async def _run_and_cleanup() -> None:
try: try:
await test_session.start() await test_session.start()
finally:
# start() raises for a failed connection. Preserve the
# terminal runtime state so the UI can render actionable
# failure phases such as OAuth-required.
ctx.metadata['runtime_info'] = test_session.get_runtime_info_dict() ctx.metadata['runtime_info'] = test_session.get_runtime_info_dict()
finally:
try: try:
await test_session.shutdown() await test_session.shutdown()
except Exception as exc: except Exception as exc:
+50 -129
View File
@@ -1,6 +1,5 @@
from __future__ import annotations from __future__ import annotations
import asyncio
import uuid import uuid
import traceback import traceback
@@ -8,10 +7,8 @@ import sqlalchemy
from ....cloud.model_catalog import LANGBOT_MODELS_PROVIDER_REQUESTER from ....cloud.model_catalog import LANGBOT_MODELS_PROVIDER_REQUESTER
from ....core import app from ....core import app
from ....core.task_boundary import create_detached_task
from ....entity.persistence import model as persistence_model from ....entity.persistence import model as persistence_model
from ....workspace.errors import WorkspaceNotFoundError from ....workspace.errors import WorkspaceNotFoundError
from ....provider.modelmgr.codex_auth import CodexAuth, REQUESTER as CODEX_REQUESTER, validate_config
from .secrets import contains_secret_placeholder, redact_secrets, restore_secret_placeholders from .secrets import contains_secret_placeholder, redact_secrets, restore_secret_placeholders
from .tenant import TenantContext, require_workspace_uuid, scope_statement from .tenant import TenantContext, require_workspace_uuid, scope_statement
@@ -23,8 +20,6 @@ class ModelProviderService:
def __init__(self, ap: app.Application) -> None: def __init__(self, ap: app.Application) -> None:
self.ap = ap self.ap = ap
self.codex_auth = CodexAuth(ap)
self._deletion_tasks: set[asyncio.Task[None]] = set()
def _is_cloud_runtime(self) -> bool: def _is_cloud_runtime(self) -> bool:
mode = getattr(self.ap.persistence_mgr, 'mode', None) mode = getattr(self.ap.persistence_mgr, 'mode', None)
@@ -121,30 +116,14 @@ class ModelProviderService:
provider_data = provider_data.copy() provider_data = provider_data.copy()
if self._system_requester_is_reserved(provider_data.get('requester')): if self._system_requester_is_reserved(provider_data.get('requester')):
raise ValueError('space-chat-completions is reserved for the Cloud-managed LangBot Models provider') raise ValueError('space-chat-completions is reserved for the Cloud-managed LangBot Models provider')
validate_config(provider_data)
provider_data['uuid'] = str(uuid.uuid4()) provider_data['uuid'] = str(uuid.uuid4())
provider_data['workspace_uuid'] = require_workspace_uuid(context) provider_data['workspace_uuid'] = require_workspace_uuid(context)
provider_data['api_keys'] = self._normalize_api_keys( provider_data['api_keys'] = self._normalize_api_keys(
restore_secret_placeholders(provider_data.get('api_keys'), sensitive=True) restore_secret_placeholders(provider_data.get('api_keys'), sensitive=True)
) )
if provider_data.get('requester') == CODEX_REQUESTER: await self.ap.persistence_mgr.execute_async(
async with self.ap.persistence_mgr.tenant_uow(provider_data['workspace_uuid']): sqlalchemy.insert(persistence_model.ModelProvider).values(**provider_data)
await self.ap.persistence_mgr.execute_async( )
sqlalchemy.insert(persistence_model.ModelProvider).values(**provider_data)
)
await self.ap.persistence_mgr.execute_async(
sqlalchemy.insert(persistence_model.CodexCredential).values(
workspace_uuid=provider_data['workspace_uuid'],
provider_uuid=provider_data['uuid'],
payload={},
version=0,
lease_until=0,
)
)
else:
await self.ap.persistence_mgr.execute_async(
sqlalchemy.insert(persistence_model.ModelProvider).values(**provider_data)
)
# load to runtime # load to runtime
runtime_provider = await self.ap.model_mgr.load_provider(context, provider_data) runtime_provider = await self.ap.model_mgr.load_provider(context, provider_data)
@@ -159,17 +138,6 @@ class ModelProviderService:
raise ValueError('space-chat-completions is reserved for the Cloud-managed LangBot Models provider') raise ValueError('space-chat-completions is reserved for the Cloud-managed LangBot Models provider')
provider_data.pop('uuid', None) provider_data.pop('uuid', None)
provider_data.pop('workspace_uuid', None) provider_data.pop('workspace_uuid', None)
if {'requester', 'base_url', 'api_keys'} & provider_data.keys():
current = await self.get_provider(context, provider_uuid, include_secret=True)
if current is None:
raise WorkspaceNotFoundError('Provider not found')
if CODEX_REQUESTER in (current.get('requester'), provider_data.get('requester')):
if provider_data.get('requester', current.get('requester')) != current.get('requester'):
raise ValueError('Create a separate provider to change the ChatGPT authentication type')
merged = {**current, **provider_data}
validate_config(merged)
provider_data['base_url'] = merged['base_url']
provider_data['api_keys'] = []
if 'api_keys' in provider_data: if 'api_keys' in provider_data:
submitted_keys = provider_data.get('api_keys') submitted_keys = provider_data.get('api_keys')
if contains_secret_placeholder(submitted_keys, sensitive=True): if contains_secret_placeholder(submitted_keys, sensitive=True):
@@ -195,107 +163,60 @@ class ModelProviderService:
raise WorkspaceNotFoundError('Provider not found') raise WorkspaceNotFoundError('Provider not found')
await self.ap.model_mgr.reload_provider(context, provider_uuid) await self.ap.model_mgr.reload_provider(context, provider_uuid)
async def delete_provider(self, context: TenantContext, provider_uuid: str, cascade: bool = False) -> None: async def delete_provider(self, context: TenantContext, provider_uuid: str) -> None:
"""Delete a provider, optionally deleting all its Workspace-scoped models.""" """Delete a provider (only if no models reference it)"""
await self._assert_provider_mutable(context, provider_uuid)
workspace_uuid = require_workspace_uuid(context) workspace_uuid = require_workspace_uuid(context)
persistence = self.ap.persistence_mgr # Check if any models use this provider
model_types = ( llm_result = await self.ap.persistence_mgr.execute_async(
(persistence_model.LLMModel, 'LLM', 'remove_llm_model'), scope_statement(
(persistence_model.EmbeddingModel, 'Embedding', 'remove_embedding_model'), sqlalchemy.select(persistence_model.LLMModel).where(
(persistence_model.RerankModel, 'Rerank', 'remove_rerank_model'), persistence_model.LLMModel.provider_uuid == provider_uuid
),
persistence_model.LLMModel,
workspace_uuid,
)
) )
deleted_models: list[tuple[str, list[str]]] = [] if llm_result.first() is not None:
async with persistence.tenant_uow(workspace_uuid): raise ValueError('Cannot delete provider: LLM models still reference it')
# Check ownership before touching children. Lock the provider on PostgreSQL
# so concurrent model inserts cannot race the reference check/deletion. embedding_result = await self.ap.persistence_mgr.execute_async(
provider_result = await persistence.execute_async( scope_statement(
scope_statement( sqlalchemy.select(persistence_model.EmbeddingModel).where(
sqlalchemy.select(persistence_model.ModelProvider.requester) persistence_model.EmbeddingModel.provider_uuid == provider_uuid
.where(persistence_model.ModelProvider.uuid == provider_uuid) ),
.with_for_update(), persistence_model.EmbeddingModel,
persistence_model.ModelProvider, workspace_uuid,
workspace_uuid,
)
) )
provider = provider_result.first() )
if provider is None: if embedding_result.first() is not None:
raise WorkspaceNotFoundError('Provider not found') raise ValueError('Cannot delete provider: Embedding models still reference it')
if self._system_requester_is_reserved(provider.requester):
raise ValueError('LangBot Models is managed by Cloud and cannot be modified')
for model_type, label, remover in model_types: rerank_result = await self.ap.persistence_mgr.execute_async(
result = await persistence.execute_async( scope_statement(
scope_statement( sqlalchemy.select(persistence_model.RerankModel).where(
sqlalchemy.select(model_type.uuid).where(model_type.provider_uuid == provider_uuid), persistence_model.RerankModel.provider_uuid == provider_uuid
model_type, ),
workspace_uuid, persistence_model.RerankModel,
) workspace_uuid,
)
model_uuids = list(result.scalars())
if model_uuids and not cascade:
raise ValueError(f'Cannot delete provider: {label} models still reference it')
if model_uuids:
# Model services have no pipeline/KB deletion side effects: they
# delete the scoped row and evict its runtime cache. Defer eviction
# here rather than calling those services before our commit.
await persistence.execute_async(
scope_statement(
sqlalchemy.delete(model_type).where(model_type.provider_uuid == provider_uuid),
model_type,
workspace_uuid,
)
)
deleted_models.append((remover, model_uuids))
# Explicit cleanup also works on legacy SQLite connections without FK
# enforcement; never load or serialize the private credential payload.
await persistence.execute_async(
scope_statement(
sqlalchemy.delete(persistence_model.CodexCredential).where(
persistence_model.CodexCredential.provider_uuid == provider_uuid
),
persistence_model.CodexCredential,
workspace_uuid,
)
) )
result = await persistence.execute_async( )
scope_statement( if rerank_result.first() is not None:
sqlalchemy.delete(persistence_model.ModelProvider).where( raise ValueError('Cannot delete provider: Rerank models still reference it')
persistence_model.ModelProvider.uuid == provider_uuid
), result = await self.ap.persistence_mgr.execute_async(
persistence_model.ModelProvider, scope_statement(
workspace_uuid, sqlalchemy.delete(persistence_model.ModelProvider).where(
) persistence_model.ModelProvider.uuid == provider_uuid
),
persistence_model.ModelProvider,
workspace_uuid,
) )
if result.rowcount == 0: )
raise WorkspaceNotFoundError('Provider not found') if getattr(result, 'rowcount', None) == 0:
raise WorkspaceNotFoundError('Provider not found')
async def remove_runtime() -> None: await self.ap.model_mgr.remove_provider(context, provider_uuid)
async with persistence.tenant_scope(workspace_uuid):
for remover, model_uuids in deleted_models:
for model_uuid in model_uuids:
await getattr(self.ap.model_mgr, remover)(context, model_uuid)
# This also closes the requester's HTTP client; models go first.
await self.ap.model_mgr.remove_provider(context, provider_uuid)
if persistence.current_session() is None:
await remove_runtime()
else:
# A nested UoW has not committed yet. Reuse the rollback-cancelled gate
# and detached context boundary instead of evicting uncommitted data.
task = create_detached_task(
remove_runtime(),
after_commit_manager=persistence,
workspace_uuid=workspace_uuid,
)
self._deletion_tasks.add(task)
def completed(task: asyncio.Task[None]) -> None:
self._deletion_tasks.discard(task)
if not task.cancelled() and task.exception() is not None:
self.ap.logger.error('Failed to remove deleted provider runtime', exc_info=task.exception())
task.add_done_callback(completed)
async def get_provider_model_counts(self, context: TenantContext, provider_uuid: str) -> dict: async def get_provider_model_counts(self, context: TenantContext, provider_uuid: str) -> dict:
"""Get count of models using this provider""" """Get count of models using this provider"""
+37 -69
View File
@@ -25,6 +25,7 @@ _PUBLIC_SKILL_FIELDS = (
'description', 'description',
'instructions', 'instructions',
'package_root', 'package_root',
'revision',
'created_at', 'created_at',
'updated_at', 'updated_at',
) )
@@ -53,38 +54,11 @@ class SkillService:
def __init__(self, ap: app.Application) -> None: def __init__(self, ap: app.Application) -> None:
self.ap = ap self.ap = ap
def _box_service(self): def _repository(self):
box_service = getattr(self.ap, 'box_service', None) repository = getattr(self.ap, 'skill_repository', None)
if box_service is not None and getattr(box_service, 'available', False): if repository is None:
return box_service raise ValueError('Skill repository is not initialised')
return None return repository
def _require_box(self, action: str):
"""Return the Box service or raise if it is not available.
Box is the only source of truth for skills. Every read and write
operation goes through it — there is no local-filesystem fallback.
"""
box_service = self._box_service()
if box_service is not None:
return box_service
ap_box = getattr(self.ap, 'box_service', None)
if ap_box is None:
reason = 'not initialised'
elif not getattr(ap_box, 'enabled', True):
reason = 'disabled in config (box.enabled = false)'
else:
connector_error = getattr(ap_box, '_connector_error', '') or 'currently unavailable'
reason = f'unavailable: {connector_error}'
raise ValueError(
f'{action} requires the Box runtime, which is {reason}. '
f'Enable Box in config.yaml (box.enabled = true) and ensure the '
f'runtime is reachable before retrying.'
)
def _require_box_for_write(self, action: str) -> None:
"""Backwards-compatible alias preserved for clarity at call sites."""
self._require_box(action)
async def _execution_context(self, context: TenantContext) -> ExecutionContext: async def _execution_context(self, context: TenantContext) -> ExecutionContext:
workspace_uuid = require_workspace_uuid(context) workspace_uuid = require_workspace_uuid(context)
@@ -113,20 +87,11 @@ class SkillService:
async def list_skills(self, context: TenantContext) -> list[dict]: async def list_skills(self, context: TenantContext) -> list[dict]:
execution_context = await self._execution_context(context) execution_context = await self._execution_context(context)
# When Box is unavailable, surface an empty list rather than raising — return [self._serialize_skill(skill) for skill in await self._repository().list_skills(execution_context)]
# the skills page should render cleanly, and the UI separately renders
# a "Box disabled / unavailable" banner via useBoxStatus.
box_service = self._box_service()
if box_service is None:
return []
return [self._serialize_skill(skill) for skill in await box_service.list_skills(execution_context)]
async def get_skill(self, context: TenantContext, skill_name: str) -> Optional[dict]: async def get_skill(self, context: TenantContext, skill_name: str) -> Optional[dict]:
execution_context = await self._execution_context(context) execution_context = await self._execution_context(context)
box_service = self._box_service() skill = await self._repository().get_skill(execution_context, skill_name, snapshot=True)
if box_service is None:
return None
skill = await box_service.get_skill(execution_context, skill_name)
return self._serialize_skill(skill) if skill else None return self._serialize_skill(skill) if skill else None
async def get_skill_by_name(self, context: TenantContext, name: str) -> Optional[dict]: async def get_skill_by_name(self, context: TenantContext, name: str) -> Optional[dict]:
@@ -134,22 +99,25 @@ class SkillService:
async def create_skill(self, context: TenantContext, data: dict) -> dict: async def create_skill(self, context: TenantContext, data: dict) -> dict:
execution_context = await self._execution_context(context) execution_context = await self._execution_context(context)
box_service = self._require_box('Creating a skill') created = await self._repository().create_skill(execution_context, data)
created = await box_service.create_skill(execution_context, data) await self._reload_skills(execution_context)
return self._serialize_skill(created)
async def import_skill_directory(self, context: TenantContext, path: str, data: dict) -> dict:
execution_context = await self._execution_context(context)
created = await self._repository().import_skill_directory(execution_context, path, data)
await self._reload_skills(execution_context) await self._reload_skills(execution_context)
return self._serialize_skill(created) return self._serialize_skill(created)
async def update_skill(self, context: TenantContext, skill_name: str, data: dict) -> dict: async def update_skill(self, context: TenantContext, skill_name: str, data: dict) -> dict:
execution_context = await self._execution_context(context) execution_context = await self._execution_context(context)
box_service = self._require_box('Editing a skill') updated = await self._repository().update_skill(execution_context, skill_name, data)
updated = await box_service.update_skill(execution_context, skill_name, data)
await self._reload_skills(execution_context) await self._reload_skills(execution_context)
return self._serialize_skill(updated) return self._serialize_skill(updated)
async def delete_skill(self, context: TenantContext, skill_name: str) -> bool: async def delete_skill(self, context: TenantContext, skill_name: str) -> bool:
execution_context = await self._execution_context(context) execution_context = await self._execution_context(context)
box_service = self._require_box('Deleting a skill') await self._repository().delete_skill(execution_context, skill_name)
await box_service.delete_skill(execution_context, skill_name)
await self._reload_skills(execution_context) await self._reload_skills(execution_context)
return True return True
@@ -162,24 +130,27 @@ class SkillService:
max_entries: int = 200, max_entries: int = 200,
) -> dict: ) -> dict:
execution_context = await self._execution_context(context) execution_context = await self._execution_context(context)
box_service = self._require_box('Browsing skill files') return await self._repository().list_skill_files(
return await box_service.list_skill_files(execution_context, skill_name, path, include_hidden, max_entries) execution_context,
skill_name,
path,
include_hidden,
max_entries,
)
async def read_skill_file(self, context: TenantContext, skill_name: str, path: str) -> dict: async def read_skill_file(self, context: TenantContext, skill_name: str, path: str) -> dict:
execution_context = await self._execution_context(context) execution_context = await self._execution_context(context)
box_service = self._require_box('Reading a skill file') return await self._repository().read_skill_file(execution_context, skill_name, path)
return await box_service.read_skill_file(execution_context, skill_name, path)
async def write_skill_file(self, context: TenantContext, skill_name: str, path: str, content: str) -> dict: async def write_skill_file(self, context: TenantContext, skill_name: str, path: str, content: str) -> dict:
execution_context = await self._execution_context(context) execution_context = await self._execution_context(context)
box_service = self._require_box('Editing skill files') result = await self._repository().write_skill_file(execution_context, skill_name, path, content)
result = await box_service.write_skill_file(execution_context, skill_name, path, content)
await self._reload_skills(execution_context) await self._reload_skills(execution_context)
return result return result
async def install_from_github(self, context: TenantContext, data: dict) -> list[dict]: async def install_from_github(self, context: TenantContext, data: dict) -> list[dict]:
execution_context = await self._execution_context(context) execution_context = await self._execution_context(context)
box_service = self._require_box('Installing a skill from GitHub') repository = self._repository()
owner = str(data['owner']).strip() owner = str(data['owner']).strip()
repo = str(data['repo']).strip() repo = str(data['repo']).strip()
release_tag = str(data.get('release_tag', '')).strip() release_tag = str(data.get('release_tag', '')).strip()
@@ -198,7 +169,7 @@ class SkillService:
zip_bytes = await self._download_github_asset(asset_url) zip_bytes = await self._download_github_asset(asset_url)
filename = f'{repo}-{release_tag.lstrip("v").replace("/", "-") or "source"}.zip' filename = f'{repo}-{release_tag.lstrip("v").replace("/", "-") or "source"}.zip'
installed = await box_service.install_skill_zip( installed = await repository.install_skill_zip(
execution_context, execution_context,
zip_bytes, zip_bytes,
filename, filename,
@@ -211,7 +182,7 @@ class SkillService:
async def preview_install_from_github(self, context: TenantContext, data: dict) -> list[dict]: async def preview_install_from_github(self, context: TenantContext, data: dict) -> list[dict]:
execution_context = await self._execution_context(context) execution_context = await self._execution_context(context)
box_service = self._require_box('Previewing a skill from GitHub') repository = self._repository()
owner = str(data['owner']).strip() owner = str(data['owner']).strip()
repo = str(data['repo']).strip() repo = str(data['repo']).strip()
release_tag = str(data.get('release_tag', '')).strip() release_tag = str(data.get('release_tag', '')).strip()
@@ -228,7 +199,7 @@ class SkillService:
source_subdir = str(data.get('source_subdir', '') or '').strip() source_subdir = str(data.get('source_subdir', '') or '').strip()
zip_bytes = await self._download_github_asset(asset_url) zip_bytes = await self._download_github_asset(asset_url)
return await box_service.preview_skill_zip( return await repository.preview_skill_zip(
execution_context, execution_context,
zip_bytes, zip_bytes,
f'{repo}-{release_tag.lstrip("v").replace("/", "-") or "source"}.zip', f'{repo}-{release_tag.lstrip("v").replace("/", "-") or "source"}.zip',
@@ -245,8 +216,7 @@ class SkillService:
source_path: str = '', source_path: str = '',
) -> list[dict]: ) -> list[dict]:
execution_context = await self._execution_context(context) execution_context = await self._execution_context(context)
box_service = self._require_box('Installing a skill from upload') installed = await self._repository().install_skill_zip(
installed = await box_service.install_skill_zip(
execution_context, execution_context,
file_bytes, file_bytes,
filename, filename,
@@ -264,8 +234,7 @@ class SkillService:
filename: str, filename: str,
) -> list[dict]: ) -> list[dict]:
execution_context = await self._execution_context(context) execution_context = await self._execution_context(context)
box_service = self._require_box('Previewing a skill upload') return await self._repository().preview_skill_zip(execution_context, file_bytes, filename)
return await box_service.preview_skill_zip(execution_context, file_bytes, filename)
async def _install_github_skill_md( async def _install_github_skill_md(
self, self,
@@ -276,14 +245,14 @@ class SkillService:
repo: str, repo: str,
data: dict, data: dict,
) -> list[dict]: ) -> list[dict]:
box_service = self._require_box('Installing a skill from GitHub') repository = self._repository()
zip_bytes, filename, _package_name = await self._download_github_skill_directory_as_zip( zip_bytes, filename, _package_name = await self._download_github_skill_directory_as_zip(
asset_url, asset_url,
owner=owner, owner=owner,
repo=repo, repo=repo,
) )
installed = await box_service.install_skill_zip( installed = await repository.install_skill_zip(
context, context,
zip_bytes, zip_bytes,
filename, filename,
@@ -302,13 +271,13 @@ class SkillService:
owner: str, owner: str,
repo: str, repo: str,
) -> list[dict]: ) -> list[dict]:
box_service = self._require_box('Previewing a skill from GitHub') repository = self._repository()
zip_bytes, _filename, package_name = await self._download_github_skill_directory_as_zip( zip_bytes, _filename, package_name = await self._download_github_skill_directory_as_zip(
asset_url, asset_url,
owner=owner, owner=owner,
repo=repo, repo=repo,
) )
return await box_service.preview_skill_zip(context, zip_bytes, f'{package_name}.zip', target_suffix='') return await repository.preview_skill_zip(context, zip_bytes, f'{package_name}.zip', target_suffix='')
async def reload_skills(self, context: TenantContext) -> list[dict]: async def reload_skills(self, context: TenantContext) -> list[dict]:
execution_context = await self._execution_context(context) execution_context = await self._execution_context(context)
@@ -317,8 +286,7 @@ class SkillService:
async def scan_directory_async(self, context: TenantContext, path: str) -> dict: async def scan_directory_async(self, context: TenantContext, path: str) -> dict:
execution_context = await self._execution_context(context) execution_context = await self._execution_context(context)
box_service = self._require_box('Scanning a skill directory') return await self._repository().scan_skill_directory(execution_context, path)
return await box_service.scan_skill_directory(execution_context, path)
async def _reload_skills(self, context: TenantContext) -> None: async def _reload_skills(self, context: TenantContext) -> None:
skill_mgr = getattr(self.ap, 'skill_mgr', None) skill_mgr = getattr(self.ap, 'skill_mgr', None)
+3 -4
View File
@@ -147,10 +147,9 @@ class BoxRuntimeConnector(ManagedRuntimeConnector):
- An explicit ``runtime.endpoint`` was configured - An explicit ``runtime.endpoint`` was configured
When this is True the Box runtime lives in a separate process with its When this is True the Box runtime lives in a separate process with its
own filesystem view (container, pod sidecar, or remote host), so paths own filesystem view (container, pod sidecar, or remote host), so only
it reports (e.g. skill ``package_root``) are NOT resolvable on the explicitly shared paths are usable on both sides. When False, Box runs
LangBot side. When False, Box runs as a stdio child process that shares as a stdio child process that shares LangBot's filesystem.
LangBot's filesystem.
""" """
return bool( return bool(
self.configured_runtime_endpoint self.configured_runtime_endpoint
+45 -211
View File
@@ -28,8 +28,10 @@ from langbot_plugin.box.errors import BoxAdmissionError, BoxError, BoxValidation
from langbot_plugin.box.models import ( from langbot_plugin.box.models import (
BUILTIN_PROFILES, BUILTIN_PROFILES,
BoxExecutionResult, BoxExecutionResult,
BoxHostMountMode,
BoxManagedProcessInfo, BoxManagedProcessInfo,
BoxManagedProcessSpec, BoxManagedProcessSpec,
BoxMountSpec,
BoxProfile, BoxProfile,
BoxSpec, BoxSpec,
) )
@@ -169,7 +171,7 @@ class BoxService:
self._connector_error = 'Box runtime is disabled in config (box.enabled = false)' self._connector_error = 'Box runtime is disabled in config (box.enabled = false)'
self.ap.logger.info( self.ap.logger.info(
'Box runtime disabled by config; sandbox features (exec/read/write/edit, ' 'Box runtime disabled by config; sandbox features (exec/read/write/edit, '
'skill add/edit, stdio MCP) will be unavailable.' 'stdio MCP, executable package scripts) will be unavailable.'
) )
return return
try: try:
@@ -275,10 +277,6 @@ class BoxService:
await self._purge_attachment_dirs() await self._purge_attachment_dirs()
self._available = True self._available = True
self._connector_error = '' self._connector_error = ''
skill_mgr = getattr(self.ap, 'skill_mgr', None)
reload_skills = getattr(skill_mgr, 'reload_skills', None)
if callable(reload_skills) and not self._cloud_managed:
await reload_skills()
self.ap.logger.info('Box runtime reconnected, sandbox features restored.') self.ap.logger.info('Box runtime reconnected, sandbox features restored.')
return return
except Exception as exc: except Exception as exc:
@@ -358,19 +356,17 @@ class BoxService:
"""Whether LangBot and the Box runtime share a filesystem view. """Whether LangBot and the Box runtime share a filesystem view.
This is True only when Box runs as a local stdio child process of This is True only when Box runs as a local stdio child process of
LangBot (same container/host). In that case paths the Box runtime LangBot (same container/host). In that case host paths resolve
reports — notably skill ``package_root`` — resolve identically on the identically on both sides and Core may perform local filesystem work.
LangBot side, so LangBot may validate them against its own filesystem.
It is False for every separated deployment (Docker Compose, k8s It is False for every separated deployment (Docker Compose, k8s
sidecar, ``--standalone-box``, or an explicit ``runtime.endpoint``), sidecar, ``--standalone-box``, or an explicit ``runtime.endpoint``),
where the Box runtime owns its own filesystem and LangBot must trust where only explicitly shared and identically mounted roots can cross
the paths it reports rather than checking them locally. the process boundary.
When Box is wired up with an injected client (tests, custom embeds) When Box is wired up with an injected client (tests, custom embeds)
there is no connector to introspect; we conservatively report False so there is no connector to introspect; we conservatively report False.
LangBot never wrongly drops Box-reported skills. An explicit override An explicit override can be set via ``_shares_filesystem_with_box`` (used by tests and any
can be set via ``_shares_filesystem_with_box`` (used by tests and any
embedder that knows the real topology). embedder that knows the real topology).
""" """
if self._shares_filesystem_with_box_override is not None: if self._shares_filesystem_with_box_override is not None:
@@ -483,11 +479,18 @@ class BoxService:
self, self,
context: TenantContext, context: TenantContext,
spec_payload: dict, spec_payload: dict,
*,
trusted_read_only_mounts: list[dict] | None = None,
) -> dict: ) -> dict:
"""Reject tenant-owned policy fields and apply the Cloud hard policy.""" """Reject tenant-owned policy fields and apply the Cloud hard policy."""
payload = dict(spec_payload) payload = dict(spec_payload)
trusted_mounts = self._normalize_trusted_read_only_mounts(trusted_read_only_mounts or [])
if not self._cloud_managed: if not self._cloud_managed:
if trusted_mounts:
if payload.get('extra_mounts'):
raise BoxValidationError('extra_mounts and trusted_read_only_mounts cannot both be supplied')
payload['extra_mounts'] = trusted_mounts
return payload return payload
policy = self._admission_policy policy = self._admission_policy
if policy is None: if policy is None:
@@ -537,7 +540,7 @@ class BoxService:
'network': 'off', 'network': 'off',
'host_path': canonical_host_path, 'host_path': canonical_host_path,
'mount_path': '/workspace', 'mount_path': '/workspace',
'extra_mounts': [], 'extra_mounts': trusted_mounts,
'persistent': True, 'persistent': True,
'timeout_sec': min(timeout, policy.max_timeout_sec), 'timeout_sec': min(timeout, policy.max_timeout_sec),
'cpus': policy.cpus, 'cpus': policy.cpus,
@@ -549,6 +552,22 @@ class BoxService:
) )
return payload return payload
def _normalize_trusted_read_only_mounts(self, mounts: list[dict]) -> list[dict]:
"""Validate Core-composed artifacts before crossing into Box."""
normalized: list[dict] = []
for raw_mount in mounts:
mount = BoxMountSpec.model_validate(raw_mount)
if mount.mode != BoxHostMountMode.READ_ONLY:
raise BoxAdmissionError('Core-composed additional mounts must be read-only')
host_path = os.path.realpath(mount.host_path)
if not os.path.isdir(host_path):
raise BoxAdmissionError('Core-composed read-only mount source is unavailable')
if not any(_is_path_under(host_path, root) for root in self.allowed_mount_roots):
raise BoxAdmissionError('Core-composed read-only mount source is outside allowed_mount_roots')
normalized.append(mount.model_copy(update={'host_path': host_path}).model_dump(mode='json'))
return normalized
def _reject_cloud_managed_process(self) -> None: def _reject_cloud_managed_process(self) -> None:
if self._cloud_managed: if self._cloud_managed:
raise BoxAdmissionError('Managed processes are disabled for Cloud sandboxes') raise BoxAdmissionError('Managed processes are disabled for Cloud sandboxes')
@@ -565,13 +584,18 @@ class BoxService:
query: pipeline_query.Query, query: pipeline_query.Query,
*, *,
skip_host_mount_validation: bool = False, skip_host_mount_validation: bool = False,
trusted_read_only_mounts: list[dict] | None = None,
) -> dict: ) -> dict:
if not self._available: if not self._available:
raise BoxError( raise BoxError(
'Box runtime is not available. Configure an available Box backend before using Box features.' 'Box runtime is not available. Configure an available Box backend before using Box features.'
) )
execution_context = await self._validated_execution_context(self._query_execution_context(query)) execution_context = await self._validated_execution_context(self._query_execution_context(query))
spec_payload = self._managed_policy_payload(execution_context, spec_payload) spec_payload = self._managed_policy_payload(
execution_context,
spec_payload,
trusted_read_only_mounts=trusted_read_only_mounts,
)
await self._require_validated_workspace_sandbox(execution_context) await self._require_validated_workspace_sandbox(execution_context)
if spec_payload.get('host_path') in (None, ''): if spec_payload.get('host_path') in (None, ''):
tenant_workspace = self._tenant_workspace(execution_context) tenant_workspace = self._tenant_workspace(execution_context)
@@ -658,70 +682,12 @@ class BoxService:
variables.setdefault('global', 'global') variables.setdefault('global', 'global')
return template.format_map(collections.defaultdict(lambda: 'unknown', variables)) return template.format_map(collections.defaultdict(lambda: 'unknown', variables))
def build_skill_extra_mounts(self, query: pipeline_query.Query) -> list[dict]:
"""Build extra_mounts entries for all pipeline-bound skills.
This ensures that when a container is first created it already has
all skill packages mounted, regardless of which skill is currently
activated.
Path validation is filesystem-topology dependent. When LangBot and the
Box runtime share a filesystem (local stdio mode), a skill whose
``package_root`` is missing or no longer a directory is skipped with a
warning instead of being passed through to the backend. Without that
guard the three backends behave inconsistently on a stale mount: nsjail
refuses to start the sandbox (failing every exec in the session),
Docker silently auto-creates a root-owned empty directory on the host,
and E2B silently skips the upload — none of which surfaces an
actionable error.
When Box runs as a separate process (Docker Compose, k8s sidecar,
``--standalone-box``, or a remote ``runtime.endpoint``), the
``package_root`` reported by ``list_skills`` is the Box runtime's own
filesystem path and is NOT resolvable on the LangBot side. Validating
it locally would wrongly drop every skill, so LangBot trusts the path
and lets the Box runtime resolve it. The Box runtime only ever reports
skills it discovered on its own filesystem, so the path is valid there
by construction.
"""
if self._cloud_managed:
return []
skill_mgr = getattr(self.ap, 'skill_mgr', None)
if skill_mgr is None:
return []
from ..provider.tools.loaders import skill as skill_loader
validate_locally = self.shares_filesystem_with_box
visible_skills = skill_loader.get_visible_skills(self.ap, query)
mounts: list[dict] = []
for skill_name, skill_data in visible_skills.items():
package_root = str(skill_data.get('package_root', '') or '').strip()
if not package_root:
continue
if validate_locally and not os.path.isdir(package_root):
self.ap.logger.warning(
f'Skill "{skill_name}" package_root missing on filesystem '
f'({package_root}); skipping mount to prevent sandbox failures. '
f'The skill cache may be stale — consider reloading skills.'
)
continue
mounts.append(
{
'host_path': package_root,
'mount_path': f'/workspace/.skills/{skill_name}',
'mode': 'rw',
}
)
return mounts
async def execute_tool( async def execute_tool(
self, self,
parameters: dict, parameters: dict,
query: pipeline_query.Query, query: pipeline_query.Query,
*, *,
skill_name: str | None = None, read_only_mounts: list[dict] | None = None,
) -> dict: ) -> dict:
"""Execute an agent-facing ``exec`` tool call. """Execute an agent-facing ``exec`` tool call.
@@ -729,8 +695,6 @@ class BoxService:
``BoxSpec.cmd`` field and injects the session id from the query. ``BoxSpec.cmd`` field and injects the session id from the query.
""" """
spec_payload: dict = {'cmd': parameters['command']} spec_payload: dict = {'cmd': parameters['command']}
if skill_name is not None:
spec_payload['skill_name'] = skill_name
# Pass through allowed agent-facing fields # Pass through allowed agent-facing fields
for key in ('workdir', 'timeout_sec', 'env'): for key in ('workdir', 'timeout_sec', 'env'):
@@ -740,11 +704,11 @@ class BoxService:
# Inject context the agent must not control # Inject context the agent must not control
spec_payload.setdefault('session_id', self.resolve_box_session_id(query)) spec_payload.setdefault('session_id', self.resolve_box_session_id(query))
# Mount all pipeline-bound skills so they are available in the container return await self.execute_spec_payload(
if 'extra_mounts' not in spec_payload: spec_payload,
spec_payload['extra_mounts'] = self.build_skill_extra_mounts(query) query,
trusted_read_only_mounts=read_only_mounts,
return await self.execute_spec_payload(spec_payload, query) )
async def execute_in_context( async def execute_in_context(
self, self,
@@ -1275,8 +1239,6 @@ class BoxService:
'timeout_sec': 120, 'timeout_sec': 120,
'session_id': self.resolve_box_session_id(query), 'session_id': self.resolve_box_session_id(query),
} }
if 'extra_mounts' not in spec_payload:
spec_payload['extra_mounts'] = self.build_skill_extra_mounts(query)
try: try:
spec = self.build_spec(spec_payload) spec = self.build_spec(spec_payload)
result = await self.client.execute(spec) result = await self.client.execute(spec)
@@ -1527,126 +1489,6 @@ class BoxService:
self._runtime_connector.get_relay_headers(action_context), self._runtime_connector.get_relay_headers(action_context),
) )
async def list_skills(self, context: TenantContext) -> list[dict]:
execution_context = await self._validated_skill_execution_context(context)
return await self.client.list_skills(action_context=self._action_context(execution_context))
async def get_skill(self, context: TenantContext, name: str) -> dict | None:
execution_context = await self._validated_skill_execution_context(context)
return await self.client.get_skill(name, action_context=self._action_context(execution_context))
async def create_skill(self, context: TenantContext, skill: dict) -> dict:
execution_context = await self._validated_skill_execution_context(context)
payload = dict(skill)
payload.pop('workspace_uuid', None)
if self._cloud_managed and str(payload.get('package_root', '') or '').strip():
raise BoxAdmissionError('Cloud skill package_root is runtime-owned')
if self._cloud_managed:
payload.pop('package_root', None)
return await self.client.create_skill(payload, action_context=self._action_context(execution_context))
async def update_skill(self, context: TenantContext, name: str, skill: dict) -> dict:
execution_context = await self._validated_skill_execution_context(context)
payload = dict(skill)
payload.pop('workspace_uuid', None)
if self._cloud_managed:
# The runtime already owns the package path for an existing skill.
# A serialized read response may contain it, but it is never an
# authority-bearing update field in shared Cloud mode.
payload.pop('package_root', None)
return await self.client.update_skill(
name,
payload,
action_context=self._action_context(execution_context),
)
async def delete_skill(self, context: TenantContext, name: str) -> None:
execution_context = await self._validated_skill_execution_context(context)
await self.client.delete_skill(name, action_context=self._action_context(execution_context))
async def scan_skill_directory(self, context: TenantContext, path: str) -> dict:
execution_context = await self._validated_skill_execution_context(context)
if self._cloud_managed:
raise BoxAdmissionError('Scanning arbitrary host skill directories is disabled in Cloud')
return await self.client.scan_skill_directory(path, action_context=self._action_context(execution_context))
async def _validated_skill_execution_context(self, context: TenantContext) -> ExecutionContext:
execution_context = await self._validated_execution_context(context)
await self._require_validated_workspace_sandbox(execution_context)
return execution_context
async def list_skill_files(
self,
context: TenantContext,
name: str,
path: str = '.',
include_hidden: bool = False,
max_entries: int = 200,
) -> dict:
execution_context = await self._validated_skill_execution_context(context)
return await self.client.list_skill_files(
name,
path,
include_hidden,
max_entries,
action_context=self._action_context(execution_context),
)
async def read_skill_file(self, context: TenantContext, name: str, path: str) -> dict:
execution_context = await self._validated_skill_execution_context(context)
return await self.client.read_skill_file(
name,
path,
action_context=self._action_context(execution_context),
)
async def write_skill_file(self, context: TenantContext, name: str, path: str, content: str) -> dict:
execution_context = await self._validated_skill_execution_context(context)
return await self.client.write_skill_file(
name,
path,
content,
action_context=self._action_context(execution_context),
)
async def preview_skill_zip(
self,
context: TenantContext,
file_bytes: bytes,
filename: str,
source_subdir: str = '',
target_suffix: str = 'upload',
) -> list[dict]:
execution_context = await self._validated_skill_execution_context(context)
return await self.client.preview_skill_zip(
file_bytes,
filename,
source_subdir,
target_suffix,
action_context=self._action_context(execution_context),
)
async def install_skill_zip(
self,
context: TenantContext,
file_bytes: bytes,
filename: str,
source_paths: list[str] | None = None,
source_path: str = '',
source_subdir: str = '',
target_suffix: str = 'upload',
) -> list[dict]:
execution_context = await self._validated_skill_execution_context(context)
return await self.client.install_skill_zip(
file_bytes,
filename,
source_paths,
source_path,
source_subdir,
target_suffix,
action_context=self._action_context(execution_context),
)
def _serialize_result(self, result: BoxExecutionResult) -> dict: def _serialize_result(self, result: BoxExecutionResult) -> dict:
stdout, stdout_truncated = self._truncate(result.stdout) stdout, stdout_truncated = self._truncate(result.stdout)
stderr, stderr_truncated = self._truncate(result.stderr) stderr, stderr_truncated = self._truncate(result.stderr)
@@ -1783,14 +1625,6 @@ class BoxService:
default_workspace = os.path.join(self.host_root, default_workspace) default_workspace = os.path.join(self.host_root, default_workspace)
return os.path.realpath(os.path.abspath(default_workspace)) return os.path.realpath(os.path.abspath(default_workspace))
def get_skills_root(self) -> str | None:
skills_root = str(self._local_config().get('skills_root', '') or 'skills').strip()
if not skills_root:
skills_root = 'skills'
if not os.path.isabs(skills_root) and self.host_root is not None:
skills_root = os.path.join(self.host_root, skills_root)
return os.path.realpath(os.path.abspath(skills_root))
def _load_enabled(self) -> bool: def _load_enabled(self) -> bool:
"""Read ``box.enabled`` (top-level, not ``box.local.*``). Default True """Read ``box.enabled`` (top-level, not ``box.local.*``). Default True
— disabling is opt-in. Accepts bool, ``'true'``/``'false'`` strings, — disabling is opt-in. Accepts bool, ``'true'``/``'false'`` strings,
+6 -185
View File
@@ -1,43 +1,28 @@
"""Reusable workspace/session helpers built on top of Box. """Reusable workspace/session helpers built on top of Box.
This module is the middle layer between the raw Box runtime primitives and This module is the middle layer between raw Box runtime primitives and
application-specific flows such as skills or MCP stdio. application-specific consumers.
It intentionally stays generic: It intentionally stays generic:
- path and virtualenv rewriting are workspace concerns - path and virtualenv rewriting are workspace concerns
- Python project detection/bootstrap are workspace concerns - Python project detection/bootstrap are workspace concerns
- session exec / managed-process helpers are workspace concerns - session exec / managed-process helpers are workspace concerns
Higher layers add their own semantics on top, for example: Higher layers add their own semantics on top; BoxWorkspaceSession retains only
- skills choose a stable per-skill session id and use repeated exec workspace, execution, and managed-process concepts.
- MCP stdio chooses how to prepare dependencies and attaches to a managed process
""" """
from __future__ import annotations from __future__ import annotations
import os import os
import textwrap
from typing import Any from typing import Any
PYTHON_MANIFEST_FILES = ( from ..utils.python_workspace import list_python_manifest_files
'requirements.txt',
'pyproject.toml',
'setup.py',
'setup.cfg',
)
_VENV_DIRS = frozenset({'.venv', 'venv', 'env', '.env'}) _VENV_DIRS = frozenset({'.venv', 'venv', 'env', '.env'})
_VENV_BIN_DIRS = frozenset({'bin', 'Scripts'}) _VENV_BIN_DIRS = frozenset({'bin', 'Scripts'})
def normalize_host_path(path: str | None) -> str:
if path is None:
return ''
stripped = str(path).strip()
if not stripped:
return ''
return os.path.realpath(os.path.abspath(stripped))
def rewrite_mounted_path(path: str, host_path: str | None, *, mount_path: str = '/workspace') -> str: def rewrite_mounted_path(path: str, host_path: str | None, *, mount_path: str = '/workspace') -> str:
"""Translate a host path into the path visible inside the sandbox mount.""" """Translate a host path into the path visible inside the sandbox mount."""
if not host_path or not path: if not host_path or not path:
@@ -98,13 +83,6 @@ def rewrite_venv_command(command: str, host_path: str | None, *, mount_path: str
return rewrite_mounted_path(normalized_command, host_path, mount_path=mount_path) return rewrite_mounted_path(normalized_command, host_path, mount_path=mount_path)
def list_python_manifest_files(host_path: str | None) -> list[str]:
normalized_root = normalize_host_path(host_path)
if not normalized_root:
return []
return [filename for filename in PYTHON_MANIFEST_FILES if os.path.isfile(os.path.join(normalized_root, filename))]
def classify_python_workspace(host_path: str | None) -> str | None: def classify_python_workspace(host_path: str | None) -> str | None:
"""Return the generic Python workspace shape, without app-specific policy.""" """Return the generic Python workspace shape, without app-specific policy."""
manifest_files = set(list_python_manifest_files(host_path)) manifest_files = set(list_python_manifest_files(host_path))
@@ -117,163 +95,6 @@ def classify_python_workspace(host_path: str | None) -> str | None:
return None return None
def should_prepare_python_env(host_path: str | None) -> bool:
normalized_root = normalize_host_path(host_path)
if not normalized_root:
return False
if os.path.isdir(os.path.join(normalized_root, '.venv')):
return True
return bool(list_python_manifest_files(normalized_root))
def wrap_python_command_with_env(
command: str,
*,
mount_path: str = '/workspace',
state_path: str | None = None,
) -> str:
"""Wrap a command with a reusable sandbox-local Python env bootstrap.
``mount_path`` is always the source tree used for manifest hashing and
installation. ``state_path`` may point at a separate writable directory
for read-only source mounts; when omitted, legacy mutable-workspace behavior
stores the environment beside the source.
"""
writable_state_path = state_path or mount_path
bootstrap = textwrap.dedent(
f"""
set -e
_LB_VENV_DIR="{writable_state_path}/.venv"
_LB_META_DIR="{writable_state_path}/.langbot"
_LB_META_FILE="$_LB_META_DIR/python-env.json"
_LB_LOCK_DIR="$_LB_META_DIR/python-env.lock"
_LB_TMP_DIR="{writable_state_path}/.tmp"
_LB_PIP_CACHE_DIR="{writable_state_path}/.cache/pip"
mkdir -p "$_LB_META_DIR" "$_LB_TMP_DIR" "$_LB_PIP_CACHE_DIR"
_LB_SYSTEM_PYTHON="$(command -v python3 || command -v python || true)"
if [ -z "$_LB_SYSTEM_PYTHON" ]; then
echo "python3 or python is required to prepare the workspace Python environment" >&2
exit 127
fi
export TMPDIR="$_LB_TMP_DIR"
export TEMP="$_LB_TMP_DIR"
export TMP="$_LB_TMP_DIR"
export PIP_CACHE_DIR="$_LB_PIP_CACHE_DIR"
_lb_python_meta() {{
"$_LB_SYSTEM_PYTHON" - <<'PY'
import hashlib
import json
import os
import sys
root = "{mount_path}"
max_manifest_bytes = 10 * 1024 * 1024
digest = hashlib.sha256()
manifest_files = []
for rel in ("requirements.txt", "pyproject.toml", "setup.py", "setup.cfg"):
path = os.path.join(root, rel)
if not os.path.isfile(path):
continue
if os.path.getsize(path) > max_manifest_bytes:
raise RuntimeError(
f"Python project manifest exceeds {{max_manifest_bytes}} bytes: {{rel}}"
)
manifest_files.append(rel)
with open(path, "rb") as handle:
digest.update(rel.encode("utf-8"))
digest.update(b"\\0")
while chunk := handle.read(1024 * 1024):
digest.update(chunk)
digest.update(b"\\0")
print(
json.dumps(
{{
"python_executable": sys.executable,
"python_version": list(sys.version_info[:3]),
"manifest_files": manifest_files,
"manifest_sha256": digest.hexdigest(),
}},
sort_keys=True,
)
)
PY
}}
_LB_CURRENT_META="$(_lb_python_meta)"
_LB_NEEDS_BOOTSTRAP=0
if [ ! -x "$_LB_VENV_DIR/bin/python" ]; then
_LB_NEEDS_BOOTSTRAP=1
elif [ ! -f "$_LB_META_FILE" ]; then
_LB_NEEDS_BOOTSTRAP=1
elif [ "$(cat "$_LB_META_FILE")" != "$_LB_CURRENT_META" ]; then
_LB_NEEDS_BOOTSTRAP=1
fi
if [ "$_LB_NEEDS_BOOTSTRAP" -eq 1 ]; then
_LB_LOCK_WAIT=0
while ! mkdir "$_LB_LOCK_DIR" 2>/dev/null; do
if [ "$_LB_LOCK_WAIT" -ge 120 ]; then
_LB_LOCK_OWNER="$(cat "$_LB_LOCK_DIR/pid" 2>/dev/null || true)"
if [ -n "$_LB_LOCK_OWNER" ] && kill -0 "$_LB_LOCK_OWNER" 2>/dev/null; then
echo "Timed out waiting for active Python environment lock: $_LB_LOCK_DIR" >&2
exit 1
fi
echo "Timed out waiting for Python environment lock, clearing stale lock: $_LB_LOCK_DIR" >&2
rm -rf "$_LB_LOCK_DIR" 2>/dev/null || true
if mkdir "$_LB_LOCK_DIR" 2>/dev/null; then
break
fi
echo "Timed out waiting for Python environment lock: $_LB_LOCK_DIR" >&2
exit 1
fi
sleep 1
_LB_LOCK_WAIT=$((_LB_LOCK_WAIT + 1))
done
printf '%s\\n' "$$" > "$_LB_LOCK_DIR/pid" 2>/dev/null || true
_lb_cleanup_lock() {{
rm -rf "$_LB_LOCK_DIR" >/dev/null 2>&1 || true
}}
trap _lb_cleanup_lock EXIT INT TERM
_LB_CURRENT_META="$(_lb_python_meta)"
_LB_NEEDS_BOOTSTRAP=0
if [ ! -x "$_LB_VENV_DIR/bin/python" ]; then
_LB_NEEDS_BOOTSTRAP=1
elif [ ! -f "$_LB_META_FILE" ]; then
_LB_NEEDS_BOOTSTRAP=1
elif [ "$(cat "$_LB_META_FILE")" != "$_LB_CURRENT_META" ]; then
_LB_NEEDS_BOOTSTRAP=1
fi
if [ "$_LB_NEEDS_BOOTSTRAP" -eq 1 ]; then
rm -rf "$_LB_VENV_DIR"
"$_LB_SYSTEM_PYTHON" -m venv "$_LB_VENV_DIR"
. "$_LB_VENV_DIR/bin/activate"
python -m pip install --upgrade pip setuptools wheel
if [ -f "{mount_path}/requirements.txt" ]; then
python -m pip install -r "{mount_path}/requirements.txt"
elif [ -f "{mount_path}/pyproject.toml" ] || [ -f "{mount_path}/setup.py" ] || [ -f "{mount_path}/setup.cfg" ]; then
python -m pip install "{mount_path}"
fi
printf '%s' "$_LB_CURRENT_META" > "$_LB_META_FILE"
fi
fi
export VIRTUAL_ENV="$_LB_VENV_DIR"
export PATH="$_LB_VENV_DIR/bin:$PATH"
{command}
"""
).strip()
return bootstrap + '\n'
class BoxWorkspaceSession: class BoxWorkspaceSession:
"""High-level handle for one reusable workspace-backed Box session. """High-level handle for one reusable workspace-backed Box session.
+2
View File
@@ -43,6 +43,7 @@ from . import entities as core_entities
from ..rag.knowledge import kbmgr as rag_mgr from ..rag.knowledge import kbmgr as rag_mgr
from ..rag.service import RAGRuntimeService from ..rag.service import RAGRuntimeService
from ..vector import mgr as vectordb_mgr from ..vector import mgr as vectordb_mgr
from ..skill import repository as skill_repository
from ..telemetry import telemetry as telemetry_module from ..telemetry import telemetry as telemetry_module
from ..survey import manager as survey_module from ..survey import manager as survey_module
from ..skill import manager as skill_mgr from ..skill import manager as skill_mgr
@@ -84,6 +85,7 @@ class Application:
# TODO move to pipeline # TODO move to pipeline
tool_mgr: llm_tool_mgr.ToolManager = None tool_mgr: llm_tool_mgr.ToolManager = None
box_service: box_service_module.BoxService = None box_service: box_service_module.BoxService = None
skill_repository: skill_repository.SkillRepository = None
# ======= Config manager ======= # ======= Config manager =======
+2
View File
@@ -26,6 +26,7 @@ from ...api.http.service import knowledge as knowledge_service
from ...api.http.service import mcp as mcp_service from ...api.http.service import mcp as mcp_service
from ...api.http.service import apikey as apikey_service from ...api.http.service import apikey as apikey_service
from ...api.http.service import webhook as webhook_service from ...api.http.service import webhook as webhook_service
from ...skill import repository as skill_repository
from ...api.http.service import monitoring as monitoring_service from ...api.http.service import monitoring as monitoring_service
from ...api.http.service import skill as skill_service from ...api.http.service import skill as skill_service
from ...skill import manager as skill_mgr from ...skill import manager as skill_mgr
@@ -127,6 +128,7 @@ class BuildAppStage(stage.BootingStage):
webhook_service_inst = webhook_service.WebhookService(ap) webhook_service_inst = webhook_service.WebhookService(ap)
ap.webhook_service = webhook_service_inst ap.webhook_service = webhook_service_inst
ap.skill_repository = skill_repository.SkillRepository(ap)
skill_service_inst = skill_service.SkillService(ap) skill_service_inst = skill_service.SkillService(ap)
ap.skill_service = skill_service_inst ap.skill_service = skill_service_inst
+1 -20
View File
@@ -1,18 +1,9 @@
from __future__ import annotations from __future__ import annotations
import logging
import secrets import secrets
from .. import stage, app from .. import stage, app
# This stage runs before SetupLoggerStage, so ap.logger is still None here;
# the module logger falls back to the stderr lastResort handler.
_logger = logging.getLogger(__name__)
# 32 symbols without 0/O or 1/I; eight independent draws provide 40 random bits.
_RECOVERY_KEY_ALPHABET = '23456789ABCDEFGHJKLMNPQRSTUVWXYZ'
_RECOVERY_KEY_LENGTH = 8
@stage.stage_class('GenKeysStage') @stage.stage_class('GenKeysStage')
class GenKeysStage(stage.BootingStage): class GenKeysStage(stage.BootingStage):
@@ -29,15 +20,5 @@ class GenKeysStage(stage.BootingStage):
ap.instance_config.data['system']['recovery_key'] = '' ap.instance_config.data['system']['recovery_key'] = ''
if not ap.instance_config.data['system']['recovery_key']: if not ap.instance_config.data['system']['recovery_key']:
# Keep recovery practical to type. Security also requires the reset ap.instance_config.data['system']['recovery_key'] = secrets.token_hex(3).upper()
# endpoint's concurrency-safe quota (five admissions per 15 minutes).
ap.instance_config.data['system']['recovery_key'] = ''.join(
secrets.choice(_RECOVERY_KEY_ALPHABET) for _ in range(_RECOVERY_KEY_LENGTH)
)
await ap.instance_config.dump_config() await ap.instance_config.dump_config()
elif len(ap.instance_config.data['system']['recovery_key']) < _RECOVERY_KEY_LENGTH:
_logger.warning(
'Low-entropy legacy recovery key detected (length < 8); '
'regenerate system.recovery_key in the configuration file '
'with a strong random value (#2392)'
)
+12 -49
View File
@@ -1,7 +1,6 @@
from __future__ import annotations from __future__ import annotations
import asyncio import asyncio
import json
import typing import typing
import datetime import datetime
import time import time
@@ -198,41 +197,6 @@ class TaskWrapper:
}, },
} }
def to_public_dict(self) -> dict:
"""Return the stable task projection exposed to API-key callers."""
if self.task.cancelled():
status = 'cancelled'
error = {'type': 'task_cancelled', 'message': 'Task was cancelled'}
result = None
elif not self.task.done():
status = 'running'
error = None
result = None
else:
exception = self.assume_exception()
if exception is not None:
status = 'failed'
error = {'type': 'task_failed', 'message': 'Task execution failed'}
result = None
else:
status = 'succeeded'
error = None
result = self.assume_result()
try:
json.dumps(result)
except (TypeError, ValueError):
result = None
return {
'id': self.id,
'task_type': self.task_type,
'kind': self.kind,
'status': status,
'error': error,
'result': result,
'created_at': self.created_at,
}
def cancel(self): def cancel(self):
self.task.cancel() self.task.cancel()
@@ -361,20 +325,19 @@ class AsyncTaskManager:
instance_uuid: str | None = None, instance_uuid: str | None = None,
workspace_uuid: str | None = None, workspace_uuid: str | None = None,
placement_generation: int | None = None, placement_generation: int | None = None,
public: bool = False,
) -> dict: ) -> dict:
tasks = [ return {
t.to_public_dict() if public else t.to_dict() 'tasks': [
for t in self.tasks t.to_dict()
if (type is None or t.task_type == type) for t in self.tasks
and (kind is None or t.kind == kind) if (type is None or t.task_type == type)
and (instance_uuid is None or t.instance_uuid == instance_uuid) and (kind is None or t.kind == kind)
and (workspace_uuid is None or t.workspace_uuid == workspace_uuid) and (instance_uuid is None or t.instance_uuid == instance_uuid)
and (placement_generation is None or t.placement_generation == placement_generation) and (workspace_uuid is None or t.workspace_uuid == workspace_uuid)
] and (placement_generation is None or t.placement_generation == placement_generation)
if public: ],
return {'tasks': tasks} 'id_index': TaskWrapper._id_index,
return {'tasks': tasks, 'id_index': TaskWrapper._id_index} }
def get_stats(self) -> dict: def get_stats(self) -> dict:
completed = sum(1 for t in self.tasks if t.task.done()) completed = sum(1 for t in self.tasks if t.task.done())
@@ -33,28 +33,6 @@ class ModelProvider(Base):
) )
class CodexCredential(Base):
"""Server-only OAuth state. Never joined into provider/model serialization."""
__tablename__ = 'codex_credentials'
provider_uuid = sqlalchemy.Column(sqlalchemy.String(255), primary_key=True)
workspace_uuid = sqlalchemy.Column(sqlalchemy.String(36), nullable=False)
payload = sqlalchemy.Column(sqlalchemy.JSON, nullable=False, default=dict)
version = sqlalchemy.Column(sqlalchemy.Integer, nullable=False, default=0)
lease_owner = sqlalchemy.Column(sqlalchemy.String(64), nullable=True)
lease_until = sqlalchemy.Column(sqlalchemy.Float, nullable=False, default=0)
__table_args__ = (
sqlalchemy.ForeignKeyConstraint(
['workspace_uuid', 'provider_uuid'],
['model_providers.workspace_uuid', 'model_providers.uuid'],
name='fk_codex_credentials_workspace_provider',
ondelete='CASCADE',
),
sqlalchemy.Index('ix_codex_credentials_workspace', 'workspace_uuid'),
)
class LLMModel(Base): class LLMModel(Base):
"""LLM model""" """LLM model"""
@@ -1,48 +0,0 @@
"""Add isolated server-only Codex credentials and tenant RLS.
Revision ID: 0022_codex_credentials
Revises: 0021_merge_reasoning_config
"""
from alembic import op
import sqlalchemy as sa
revision = '0022_codex_credentials'
down_revision = '0021_merge_reasoning_config'
branch_labels = None
depends_on = None
def upgrade() -> None:
conn = op.get_bind()
# Fresh startup creates ORM metadata before running Alembic.
if 'codex_credentials' not in sa.inspect(conn).get_table_names():
op.create_table(
'codex_credentials',
sa.Column('provider_uuid', sa.String(255), primary_key=True),
sa.Column('workspace_uuid', sa.String(36), nullable=False),
sa.Column('payload', sa.JSON(), nullable=False),
sa.Column('version', sa.Integer(), nullable=False),
sa.Column('lease_owner', sa.String(64), nullable=True),
sa.Column('lease_until', sa.Float(), nullable=False),
sa.ForeignKeyConstraint(
['workspace_uuid', 'provider_uuid'],
['model_providers.workspace_uuid', 'model_providers.uuid'],
name='fk_codex_credentials_workspace_provider',
ondelete='CASCADE',
),
)
op.create_index('ix_codex_credentials_workspace', 'codex_credentials', ['workspace_uuid'])
if conn.dialect.name == 'postgresql':
op.execute('ALTER TABLE codex_credentials ENABLE ROW LEVEL SECURITY')
op.execute('ALTER TABLE codex_credentials FORCE ROW LEVEL SECURITY')
op.execute('DROP POLICY IF EXISTS langbot_workspace_isolation ON codex_credentials')
expression = "workspace_uuid::text = NULLIF(current_setting('langbot.workspace_uuid', true), '')"
op.execute(
f'CREATE POLICY langbot_workspace_isolation ON codex_credentials '
f'FOR ALL USING ({expression}) WITH CHECK ({expression})'
)
def downgrade() -> None:
op.drop_table('codex_credentials')
-1
View File
@@ -62,7 +62,6 @@ _ALEMBIC_TENANT_TABLES = {
'binary_storages', 'binary_storages',
'mcp_servers', 'mcp_servers',
'model_providers', 'model_providers',
'codex_credentials',
'llm_models', 'llm_models',
'embedding_models', 'embedding_models',
'rerank_models', 'rerank_models',
+1 -33
View File
@@ -51,7 +51,6 @@ TENANT_TABLE_COLUMNS: dict[str, str] = {
'binary_storages': 'workspace_uuid', 'binary_storages': 'workspace_uuid',
'mcp_servers': 'workspace_uuid', 'mcp_servers': 'workspace_uuid',
'model_providers': 'workspace_uuid', 'model_providers': 'workspace_uuid',
'codex_credentials': 'workspace_uuid',
'llm_models': 'workspace_uuid', 'llm_models': 'workspace_uuid',
'embedding_models': 'workspace_uuid', 'embedding_models': 'workspace_uuid',
'rerank_models': 'workspace_uuid', 'rerank_models': 'workspace_uuid',
@@ -853,30 +852,7 @@ class TenantScopedAsyncSession(sqlalchemy_asyncio.AsyncSession):
self._require_owner_task() self._require_owner_task()
self._enter_internal_access() self._enter_internal_access()
try: try:
# Retain the actual connection before COMMIT: after a failed SQLite await transaction.commit()
# COMMIT the logical transaction is inactive, but the DBAPI writer
# can still hold PENDING/RESERVED locks. Session.close()/rollback()
# alone can then return that poisoned connection to the pool.
connection = await super().connection()
try:
await transaction.commit()
except BaseException as exc:
cleanup = asyncio.create_task(connection.invalidate())
# Invalidation does not access the task-owned Session. Shield
# physical cleanup, including against repeated cancellation,
# before the owner closes the Session and releases its scope.
while not cleanup.done():
try:
await asyncio.shield(cleanup)
except asyncio.CancelledError:
continue
except BaseException:
break
try:
cleanup.result()
except BaseException as cleanup_error:
exc.add_note(f'Failed to invalidate transaction connection: {cleanup_error!r}')
raise
finally: finally:
self._exit_internal_access() self._exit_internal_access()
@@ -1393,7 +1369,6 @@ class TenantUnitOfWork:
state.mark_rollback_only(exc_value) state.mark_rollback_only(exc_value)
rollback_only = state.rollback_only rollback_only = state.rollback_only
committed = False committed = False
transaction_error: BaseException | None = None
try: try:
if exc_type is None and not rollback_only: if exc_type is None and not rollback_only:
await typing.cast(TenantScopedAsyncSession, session)._commit_owned_transaction( await typing.cast(TenantScopedAsyncSession, session)._commit_owned_transaction(
@@ -1406,9 +1381,6 @@ class TenantUnitOfWork:
_UOW_SESSION_CONTROL_CAPABILITY, _UOW_SESSION_CONTROL_CAPABILITY,
transaction, transaction,
) )
except BaseException as exc:
transaction_error = exc
raise
finally: finally:
try: try:
if self._active_transaction is not None and self._context_token is not None: if self._active_transaction is not None and self._context_token is not None:
@@ -1416,10 +1388,6 @@ class TenantUnitOfWork:
await typing.cast(TenantScopedAsyncSession, session)._close_owned_session( await typing.cast(TenantScopedAsyncSession, session)._close_owned_session(
_UOW_SESSION_CONTROL_CAPABILITY _UOW_SESSION_CONTROL_CAPABILITY
) )
except BaseException as cleanup_error:
if transaction_error is None:
raise
transaction_error.add_note(f'Failed to close transaction Session: {cleanup_error!r}')
finally: finally:
if self._database_operation_token is not None: if self._database_operation_token is not None:
_DATABASE_OPERATION_TRANSACTION.reset(self._database_operation_token) _DATABASE_OPERATION_TRANSACTION.reset(self._database_operation_token)
+4 -2
View File
@@ -328,6 +328,8 @@ class PreProcessor(stage.PipelineStage):
# relied on this injection; without it the LLM never discovers # relied on this injection; without it the LLM never discovers
# the skills are there and just calls native tools instead. # the skills are there and just calls native tools instead.
if selected_runner == 'local-agent' and self.ap.skill_mgr: if selected_runner == 'local-agent' and self.ap.skill_mgr:
available_tool_names = {tool.name for tool in query.use_funcs}
query.variables['_skill_execution_available'] = 'exec' in available_tool_names
skill_execution_context = get_query_execution_context(query) skill_execution_context = get_query_execution_context(query)
await self.ap.skill_mgr.ensure_loaded(skill_execution_context) await self.ap.skill_mgr.ensure_loaded(skill_execution_context)
pipeline_data = await self.ap.pipeline_service.get_pipeline( pipeline_data = await self.ap.pipeline_service.get_pipeline(
@@ -349,7 +351,7 @@ class PreProcessor(stage.PipelineStage):
skill_execution_context, skill_execution_context,
bound_skills=bound_skills, bound_skills=bound_skills,
) )
if skill_addition: if skill_addition and 'activate' in available_tool_names:
self._append_to_system_prompt(query.prompt.messages, skill_addition) self._append_to_system_prompt(query.prompt.messages, skill_addition)
self.ap.logger.debug( self.ap.logger.debug(
f'Skill index injected into system prompt: ' f'Skill index injected into system prompt: '
@@ -357,7 +359,7 @@ class PreProcessor(stage.PipelineStage):
f'bound_skills={bound_skills or "all"} ' f'bound_skills={bound_skills or "all"} '
f'loaded_skills={len(self.ap.skill_mgr.get_skills(skill_execution_context))}' f'loaded_skills={len(self.ap.skill_mgr.get_skills(skill_execution_context))}'
) )
else: elif 'activate' in available_tool_names:
self.ap.logger.debug( self.ap.logger.debug(
f'No skills available for prompt injection: ' f'No skills available for prompt injection: '
f'pipeline={query.pipeline_uuid} ' f'pipeline={query.pipeline_uuid} '
+2 -2
View File
@@ -682,8 +682,8 @@ class MatrixAdapter(abstract_platform_adapter.AbstractMessagePlatformAdapter):
lines.append(f'[{bridge.user_id}] 跳过(未配置登录命令或无DM房间)') lines.append(f'[{bridge.user_id}] 跳过(未配置登录命令或无DM房间)')
continue continue
# Use configured logout command, fallback to deriving from login command # Use configured logout command, fallback to deriving from login command
logout_cmd = bridge.logout_command or bridge.login_command.replace('login', 'logout') logout_cmd = bridge.logout_command or bridge.login_command.replace('login', 'logout')
lines.append(f'[{bridge.user_id}] 发送 "{logout_cmd}"...') lines.append(f'[{bridge.user_id}] 发送 "{logout_cmd}"...')
# Cancel existing tasks # Cancel existing tasks
@@ -1,375 +0,0 @@
from __future__ import annotations
import asyncio
import json
import re
import typing
from urllib.parse import urlsplit, urlunsplit
import aiohttp
import langbot_plugin.api.definition.abstract.platform.adapter as abstract_platform_adapter
import langbot_plugin.api.definition.abstract.platform.event_logger as abstract_platform_logger
import langbot_plugin.api.entities.builtin.platform.entities as platform_entities
import langbot_plugin.api.entities.builtin.platform.events as platform_events
import langbot_plugin.api.entities.builtin.platform.message as platform_message
_MATTERMOST_MAX_POST_LENGTH = 16_383
_MENTION_BOUNDARY = r'(?<![\w.-])@{username}(?![\w.-])'
def _normalize_server_url(server_url: str) -> str:
"""Return a validated Mattermost server URL without a trailing slash."""
url = server_url.strip().rstrip('/')
parsed = urlsplit(url)
if parsed.scheme not in {'http', 'https'} or not parsed.netloc:
raise ValueError('Mattermost server_url must be an absolute HTTP(S) URL')
return url
def _websocket_url(server_url: str) -> str:
parsed = urlsplit(server_url)
scheme = 'wss' if parsed.scheme == 'https' else 'ws'
return urlunsplit((scheme, parsed.netloc, f'{parsed.path}/api/v4/websocket', '', ''))
class MattermostMessageConverter(abstract_platform_adapter.AbstractMessageConverter):
"""Translate Mattermost post text to and from LangBot message chains."""
@staticmethod
async def yiri2target(message_chain: platform_message.MessageChain) -> str:
parts: list[str] = []
for component in message_chain:
if isinstance(component, platform_message.Plain):
parts.append(component.text)
elif isinstance(component, platform_message.Image) and component.url:
# Mattermost renders image URLs in Markdown messages.
parts.append(component.url)
elif isinstance(component, platform_message.File) and component.url:
parts.append(component.url)
return ''.join(parts)
@staticmethod
async def target2yiri(post: dict, bot_username: str) -> platform_message.MessageChain:
text = str(post.get('message') or '')
components: list[typing.Any] = [
platform_message.Source(
id=str(post.get('id') or ''),
time=float(post.get('create_at') or 0) / 1000,
)
]
if bot_username:
mention_pattern = re.compile(_MENTION_BOUNDARY.format(username=re.escape(bot_username)), re.IGNORECASE)
if mention_pattern.search(text):
components.append(platform_message.At(target=bot_username))
text = mention_pattern.sub('', text).strip()
if text:
components.append(platform_message.Plain(text=text))
return platform_message.MessageChain(components)
class MattermostEventConverter(abstract_platform_adapter.AbstractEventConverter):
@staticmethod
async def yiri2target(event: platform_events.MessageEvent) -> dict:
return event.source_platform_object
@staticmethod
async def target2yiri(
post: dict,
channel: dict,
sender_name: str,
bot_username: str,
) -> platform_events.MessageEvent:
message_chain = await MattermostMessageConverter.target2yiri(post, bot_username)
timestamp = float(post.get('create_at') or 0) / 1000
sender_id = str(post.get('user_id') or '')
channel_type = channel.get('type')
if channel_type == 'D':
return platform_events.FriendMessage(
sender=platform_entities.Friend(id=sender_id, nickname=sender_name or sender_id, remark=''),
message_chain=message_chain,
time=timestamp,
source_platform_object={'post': post, 'channel': channel},
)
return platform_events.GroupMessage(
sender=platform_entities.GroupMember(
id=sender_id,
member_name=sender_name or sender_id,
permission=platform_entities.Permission.Member,
group=platform_entities.Group(
id=str(post.get('channel_id') or ''),
name=str(channel.get('display_name') or channel.get('name') or post.get('channel_id') or ''),
permission=platform_entities.Permission.Member,
),
special_title='',
),
message_chain=message_chain,
time=timestamp,
source_platform_object={'post': post, 'channel': channel},
)
class MattermostAdapter(abstract_platform_adapter.AbstractMessagePlatformAdapter):
"""Mattermost Bot Account adapter using the v4 REST and WebSocket APIs."""
server_url: str = ''
access_token: str = ''
session: aiohttp.ClientSession | None = None
listeners: dict[typing.Type[platform_events.Event], typing.Callable] = {}
channel_cache: dict[str, dict] = {}
stream_post_ids: dict[str, str] = {}
bot_username: str = ''
_running: bool = False
message_converter: MattermostMessageConverter = MattermostMessageConverter()
event_converter: MattermostEventConverter = MattermostEventConverter()
def __init__(self, config: dict, logger: abstract_platform_logger.AbstractEventLogger):
server_url = _normalize_server_url(str(config.get('server_url') or ''))
access_token = str(config.get('access_token') or '').strip()
if not access_token:
raise ValueError('Mattermost adapter requires an access_token')
super().__init__(
config=config,
logger=logger,
server_url=server_url,
access_token=access_token,
bot_account_id='',
session=None,
listeners={},
channel_cache={},
stream_post_ids={},
bot_username='',
_running=False,
)
async def _get_session(self) -> aiohttp.ClientSession:
if self.session is None or self.session.closed:
self.session = aiohttp.ClientSession(
headers={'Authorization': f'Bearer {self.access_token}'},
raise_for_status=False,
)
return self.session
async def _api_request(
self,
method: str,
path: str,
*,
payload: dict | None = None,
) -> dict:
session = await self._get_session()
async with session.request(method, f'{self.server_url}/api/v4{path}', json=payload) as response:
raw_body = await response.text()
if response.status >= 400:
# Mattermost returns a useful JSON error, but never include request headers/tokens in errors.
try:
error = json.loads(raw_body).get('message', raw_body)
except json.JSONDecodeError:
error = raw_body
raise RuntimeError(f'Mattermost API {method} {path} failed ({response.status}): {error}')
if not raw_body:
return {}
return json.loads(raw_body)
async def _load_bot_identity(self) -> None:
user = await self._api_request('GET', '/users/me')
self.bot_account_id = str(user.get('id') or '')
self.bot_username = str(user.get('username') or '')
if not self.bot_account_id:
raise RuntimeError('Mattermost API did not return a bot user ID')
async def _get_channel(self, channel_id: str) -> dict:
if channel_id not in self.channel_cache:
self.channel_cache[channel_id] = await self._api_request('GET', f'/channels/{channel_id}')
return self.channel_cache[channel_id]
async def _post_message(self, channel_id: str, text: str, root_id: str = '') -> dict:
if not text:
return {}
if len(text) > _MATTERMOST_MAX_POST_LENGTH:
raise ValueError(f'Mattermost messages cannot exceed {_MATTERMOST_MAX_POST_LENGTH} characters')
payload = {'channel_id': channel_id, 'message': text}
if root_id:
payload['root_id'] = root_id
return await self._api_request('POST', '/posts', payload=payload)
async def _get_direct_channel_id(self, user_id: str) -> str:
if not self.bot_account_id:
await self._load_bot_identity()
channel = await self._api_request(
'POST',
'/channels/direct',
payload={'user_ids': [self.bot_account_id, user_id]},
)
channel_id = str(channel.get('id') or '')
if not channel_id:
raise RuntimeError('Mattermost did not return a direct-message channel ID')
self.channel_cache[channel_id] = channel
return channel_id
async def send_message(self, target_type: str, target_id: str, message: platform_message.MessageChain):
if target_type not in {'person', 'group'}:
raise ValueError("Mattermost target_type must be 'person' or 'group'")
text = await self.message_converter.yiri2target(message)
channel_id = str(target_id)
if target_type == 'person':
channel_id = await self._get_direct_channel_id(channel_id)
await self._post_message(channel_id, text)
async def reply_message(
self,
message_source: platform_events.MessageEvent,
message: platform_message.MessageChain,
quote_origin: bool = False,
):
source = await self.event_converter.yiri2target(message_source)
post = source['post']
text = await self.message_converter.yiri2target(message)
# A message received inside a Mattermost thread must remain in that thread. When
# quote_origin is requested, make the response a reply to the source root post.
root_id = str(post.get('root_id') or '')
if quote_origin and not root_id:
root_id = str(post.get('id') or '')
await self._post_message(str(post['channel_id']), text, root_id)
async def create_message_card(self, message_id: str, event: platform_events.MessageEvent) -> bool:
source = await self.event_converter.yiri2target(event)
post = source['post']
root_id = str(post.get('root_id') or post.get('id') or '')
created = await self._post_message(str(post['channel_id']), 'Thinking…', root_id)
if created.get('id'):
self.stream_post_ids[str(message_id)] = str(created['id'])
return True
return False
async def reply_message_chunk(
self,
message_source: platform_events.MessageEvent,
bot_message,
message: platform_message.MessageChain,
quote_origin: bool = False,
is_final: bool = False,
):
response_id = str(bot_message.resp_message_id)
text = await self.message_converter.yiri2target(message)
if not text:
return
post_id = self.stream_post_ids.get(response_id)
if post_id:
await self._api_request('PUT', f'/posts/{post_id}', payload={'id': post_id, 'message': text})
else:
source = await self.event_converter.yiri2target(message_source)
post = source['post']
root_id = str(post.get('root_id') or '')
if quote_origin and not root_id:
root_id = str(post.get('id') or '')
created = await self._post_message(str(post['channel_id']), text, root_id)
post_id = str(created.get('id') or '')
if post_id:
self.stream_post_ids[response_id] = post_id
if is_final and getattr(bot_message, 'tool_calls', None) is None:
self.stream_post_ids.pop(response_id, None)
async def is_stream_output_supported(self) -> bool:
return bool(self.config.get('enable_stream_reply', True))
def register_listener(
self,
event_type: typing.Type[platform_events.Event],
callback: typing.Callable[
[platform_events.Event, abstract_platform_adapter.AbstractMessagePlatformAdapter], typing.Awaitable[None]
],
):
self.listeners[event_type] = callback
def unregister_listener(
self,
event_type: typing.Type[platform_events.Event],
callback: typing.Callable[
[platform_events.Event, abstract_platform_adapter.AbstractMessagePlatformAdapter], typing.Awaitable[None]
],
):
self.listeners.pop(event_type, None)
async def _dispatch_post(self, payload: dict) -> None:
data = payload.get('data') or {}
try:
post = json.loads(data.get('post') or '{}')
except (TypeError, json.JSONDecodeError):
await self.logger.error('Mattermost received a posted event with an invalid post payload')
return
if not post or str(post.get('user_id') or '') == self.bot_account_id:
return
channel_id = str(post.get('channel_id') or '')
if not channel_id:
return
try:
channel = await self._get_channel(channel_id)
event = await self.event_converter.target2yiri(
post,
channel,
str(data.get('sender_name') or post.get('user_id') or ''),
self.bot_username,
)
callback = self.listeners.get(type(event))
if callback:
result = callback(event, self)
if asyncio.iscoroutine(result):
await result
except Exception as exc:
await self.logger.error(f'Error handling Mattermost post: {exc}')
async def _run_websocket_once(self) -> None:
session = await self._get_session()
async with session.ws_connect(_websocket_url(self.server_url), heartbeat=30) as websocket:
await websocket.send_json(
{
'seq': 1,
'action': 'authentication_challenge',
'data': {'token': self.access_token},
}
)
async for message in websocket:
if message.type == aiohttp.WSMsgType.TEXT:
try:
payload = json.loads(message.data)
except json.JSONDecodeError:
continue
if payload.get('event') == 'posted':
await self._dispatch_post(payload)
elif message.type in {aiohttp.WSMsgType.CLOSED, aiohttp.WSMsgType.CLOSE, aiohttp.WSMsgType.ERROR}:
break
async def run_async(self):
self._running = True
await self._load_bot_identity()
await self.logger.info(f'Mattermost bot connected: @{self.bot_username} ({self.bot_account_id})')
retry_delay = 1
while self._running:
try:
await self._run_websocket_once()
retry_delay = 1
except asyncio.CancelledError:
raise
except Exception as exc:
if self._running:
await self.logger.error(f'Mattermost WebSocket disconnected: {exc}')
await asyncio.sleep(retry_delay)
retry_delay = min(retry_delay * 2, 30)
async def kill(self) -> bool:
self._running = False
if self.session and not self.session.closed:
await self.session.close()
return True
@@ -1 +0,0 @@
<?xml version="1.0" encoding="UTF-8"?><svg id="Artwork" xmlns="http://www.w3.org/2000/svg" viewBox="0 0 140 140"><defs><style>.cls-1{fill:#1e325c;fill-rule:evenodd;}</style></defs><path class="cls-1" d="M111.11,13.36l.74,14.86c12.04,13.3,16.8,32.15,10.81,49.86-8.95,26.44-38.46,40.33-65.92,31.04-27.46-9.29-42.45-38.26-33.5-64.7,6.01-17.77,21.32-29.87,39.05-33.07L71.87.03C41.99-.77,13.8,17.77,3.72,47.55c-12.4,36.6,7.24,76.33,43.85,88.73,36.6,12.4,76.33-7.24,88.73-43.85,10.07-29.74-1-61.55-25.14-79.07h-.03Z"/><path class="cls-1" d="M93.95,57.21l-.51-20.77-.41-11.95-.28-10.35s.07-4.99-.11-6.16c-.03-.25-.11-.44-.21-.62,0-.03-.02-.05-.03-.07,0-.02-.03-.05-.03-.07-.2-.33-.49-.59-.89-.72s-.8-.1-1.17.05h-.02s-.08.03-.13.07c-.16.08-.34.2-.51.36-.85.82-3.84,4.83-3.84,4.83l-6.5,8.06-7.59,9.25-13.02,16.19s-5.98,7.46-4.65,16.64c1.31,9.18,8.15,13.65,13.43,15.44,5.29,1.79,13.43,2.38,20.05-4.11,6.62-6.49,6.4-16.04,6.4-16.04l.02-.02Z"/></svg>

Before

Width:  |  Height:  |  Size: 938 B

@@ -1,75 +0,0 @@
apiVersion: v1
kind: MessagePlatformAdapter
metadata:
name: mattermost
label:
en_US: Mattermost
zh_Hans: Mattermost
zh_Hant: Mattermost
ja_JP: Mattermost
th_TH: Mattermost
vi_VN: Mattermost
es_ES: Mattermost
icon: mattermost.svg
description:
en_US: Mattermost Bot Account adapter using the v4 REST and WebSocket APIs. Add me to the teams and channels where you want me to interact. Please use a browser or desktop application to do this.
zh_Hans: 使用 Mattermost v4 REST API 与 WebSocket 的 Bot Account 适配器。请将我添加到您想要我互动的团队与频道。请使用浏览器或桌面应用进行操作。
zh_Hant: 使用 Mattermost v4 REST API 與 WebSocket 的 Bot Account 介面卡。請將我加入您希望我互動的團隊與頻道。請使用瀏覽器或桌面應用程式操作。
ja_JP: Mattermost v4 REST API と WebSocket を使用する Bot Account アダプター。利用させたいチームとチャンネルに私を追加してください。ブラウザまたはデスクトップアプリで操作してください。
th_TH: อะแดปเตอร์ Bot Account ของ Mattermost ผ่าน v4 REST API และ WebSocket โปรดเพิ่มฉันไปยังทีมและช่องที่คุณต้องการให้ฉันโต้ตอบ โปรดดำเนินการผ่านเบราว์เซอร์หรือแอปเดสก์ท็อป
vi_VN: Bộ điều hợp Bot Account Mattermost sử dụng REST API v4 và WebSocket. Hãy thêm tôi vào các nhóm và kênh mà bạn muốn tôi tương tác. Vui lòng thao tác bằng trình duyệt hoặc ứng dụng máy tính để bàn.
es_ES: Adaptador de Bot Account de Mattermost mediante REST API v4 y WebSocket. Añádeme a los equipos y canales en los que quieras que interactúe. Hazlo desde un navegador o la aplicación de escritorio.
spec:
categories:
- global
- popular
config:
- name: server_url
label:
en_US: Mattermost Server URL
zh_Hans: Mattermost 服务器地址
zh_Hant: 位址伺服器 Mattermost
ja_JP: Mattermost サーバー URL
th_TH: URL เซิร์ฟเวอร์ Mattermost
vi_VN: URL máy chủ Mattermost
es_ES: URL del servidor Mattermost
description:
en_US: The base URL of the Mattermost server, for example https://mattermost.example.com
zh_Hans: Mattermost 服务器基础地址,例如 https://mattermost.example.com
type: string
required: true
default: ""
- name: access_token
label:
en_US: Bot Access Token
zh_Hans: Bot 访问令牌
zh_Hant: Bot 存取權杖
ja_JP: Bot アクセストークン
th_TH: โทเค็นการเข้าถึงของบอต
vi_VN: Mã truy cập Bot
es_ES: Token de acceso del bot
description:
en_US: The personal access token generated for the Mattermost Bot Account
zh_Hans: 为 Mattermost Bot Account 生成的个人访问令牌
type: string
required: true
default: ""
- name: enable_stream_reply
label:
en_US: Enable Stream Reply
zh_Hans: 启用流式回复
zh_Hant: 啟用串流回覆
ja_JP: ストリーミング返信を有効化
th_TH: เปิดใช้งานการตอบกลับแบบสตรีม
vi_VN: Bật phản hồi luồng
es_ES: Activar respuesta en streaming
description:
en_US: Update a Mattermost post while LangBot generates a response
zh_Hans: 在 LangBot 生成回复时持续更新同一条 Mattermost 消息
type: boolean
required: false
default: true
execution:
python:
path: ./mattermost.py
attr: MattermostAdapter
@@ -1,420 +0,0 @@
"""ChatGPT device auth with server-only credentials and cross-process refresh leases.
Network I/O never holds a DB transaction. A persisted CAS lease serializes refresh
and poll; cancel fences device exchanges but waits for existing-token refreshes.
"""
from __future__ import annotations
import asyncio
import base64
import json
import math
import secrets
import time
from contextlib import asynccontextmanager
from datetime import datetime, timezone
import httpx
import sqlalchemy as sa
from ...entity.persistence.model import CodexCredential, ModelProvider
from ...api.http.context import PrincipalType, RequestContext
from ...api.http.authz import Permission, has_permission
from ...api.http.service.tenant import require_workspace_uuid
from ...workspace.errors import WorkspaceNotFoundError
REQUESTER = 'openai-codex'
BASE_URL = 'https://chatgpt.com/backend-api/codex'
ISSUER = 'https://auth.openai.com'
CLIENT_ID = 'app_EMoamEEZ73f0CkXaXp7hrann'
LOGIN_REQUIRED = 'ChatGPT sign-in required. Open this provider and sign in again.'
LEASE_SECONDS = 90
def validate_config(data: dict) -> None:
if data.get('requester') != REQUESTER:
return
if data.get('base_url') not in (None, '', BASE_URL):
raise ValueError('Codex uses the fixed ChatGPT endpoint; custom base URLs are not supported')
if data.get('api_keys') not in (None, [], ''):
raise ValueError('Codex uses ChatGPT sign-in, not API keys')
data['base_url'] = BASE_URL
data['api_keys'] = []
def _claims(token: str) -> dict:
"""Read routing metadata, NOT trusted LangBot identity, from issuer tokens."""
try:
part = token.split('.')[1]
value = json.loads(base64.urlsafe_b64decode(part + '=' * (-len(part) % 4)))
return value if isinstance(value, dict) else {}
except (ValueError, IndexError, TypeError):
return {}
def _tokens(data: dict, previous: dict | None = None) -> dict:
previous = previous or {}
access = data.get('access_token')
refresh = data.get('refresh_token') or previous.get('refresh_token')
account = None
for token in (access, data.get('id_token')):
namespace = _claims(token or '').get('https://api.openai.com/auth', {})
if isinstance(namespace, dict) and isinstance(namespace.get('chatgpt_account_id'), str):
account = namespace['chatgpt_account_id']
break
account = account or previous.get('account_id')
try:
expires_at = (
time.time() + float(data['expires_in'])
if data.get('expires_in') is not None
else float(_claims(access or '').get('exp', 0))
)
except (TypeError, ValueError):
expires_at = 0
if (
not all(isinstance(v, str) and v for v in (access, refresh, account))
or not math.isfinite(expires_at)
or expires_at <= time.time()
):
raise ValueError('ChatGPT returned an incomplete authorization. Please sign in again.')
return {
'access_token': access,
'refresh_token': refresh,
'account_id': account,
'expires_at': expires_at,
'connection_id': previous.get('connection_id') or secrets.token_urlsafe(24),
}
class CodexAuth:
def __init__(self, ap):
self.ap = ap
def _where(self, workspace: str, provider: str):
return (CodexCredential.workspace_uuid == workspace, CodexCredential.provider_uuid == provider)
async def _execute(self, statement):
# SQLAlchemy/driver/serialization errors may embed the entire secret payload.
try:
return await self.ap.persistence_mgr.execute_async(statement)
except Exception:
raise ValueError('ChatGPT credential storage failed. Please retry.') from None
async def _read(self, workspace: str, provider: str) -> dict | None:
result = await self._execute(sa.select(CodexCredential).where(*self._where(workspace, provider)))
try:
row = result.first()
return dict(row._mapping) if row is not None else None
except Exception:
raise ValueError('ChatGPT credential storage failed. Please retry.') from None
async def _provider(self, context, provider: str, *, user: bool = False) -> str:
workspace = require_workspace_uuid(context)
if user and (
not isinstance(context, RequestContext)
or context.principal.principal_type != PrincipalType.ACCOUNT
or not context.account_uuid
or not has_permission(context, Permission.PROVIDER_SECRET_MANAGE)
):
raise ValueError('ChatGPT authorization requires an authorized workspace user')
result = await self._execute(
sa.select(ModelProvider.requester).where(
ModelProvider.workspace_uuid == workspace, ModelProvider.uuid == provider
)
)
kind = result.scalar()
if kind is None:
raise WorkspaceNotFoundError('Provider not found')
if kind != REQUESTER:
raise ValueError('This provider does not use ChatGPT sign-in')
return workspace
@asynccontextmanager
async def _lease(self, workspace: str, provider: str, *, refresh: bool = False):
owner = ('refresh:' if refresh else 'device:') + secrets.token_urlsafe(32)
deadline = time.monotonic() + 65
while True:
now = time.time()
result = await self._execute(
sa.update(CodexCredential)
.where(
*self._where(workspace, provider),
sa.or_(CodexCredential.lease_owner.is_(None), CodexCredential.lease_until < now),
)
.values(lease_owner=owner, lease_until=now + LEASE_SECONDS)
)
if result.rowcount == 1:
break
if await self._read(workspace, provider) is None:
raise ValueError(LOGIN_REQUIRED)
if time.monotonic() >= deadline:
raise ValueError('ChatGPT authorization is busy. Please retry shortly.')
await asyncio.sleep(0.1)
try:
yield owner
finally:
await self._execute(
sa.update(CodexCredential)
.where(*self._where(workspace, provider), CodexCredential.lease_owner == owner)
.values(lease_owner=None, lease_until=0)
)
async def _save(self, workspace: str, provider: str, owner: str, payload: dict) -> None:
result = await self._execute(
sa.update(CodexCredential)
.where(
*self._where(workspace, provider),
CodexCredential.lease_owner == owner,
CodexCredential.lease_until > time.time(),
)
.values(payload=payload, version=CodexCredential.version + 1)
)
if result.rowcount != 1:
raise ValueError('ChatGPT authorization was cancelled or replaced. Please retry.')
async def _post(self, path: str, *, data=None, json_body=None) -> httpx.Response:
try:
async with httpx.AsyncClient(timeout=20, follow_redirects=False) as client:
return await asyncio.wait_for(
client.post(
ISSUER + path,
data=data,
json=json_body,
headers={'Accept': 'application/json', 'User-Agent': 'LangBot'},
),
25,
)
except (httpx.HTTPError, TimeoutError):
raise ValueError('ChatGPT authorization network error. Please retry.') from None
@staticmethod
def _json(response: httpx.Response) -> dict:
try:
value = response.json()
if not isinstance(value, dict):
raise ValueError
return value
except ValueError:
raise ValueError('ChatGPT returned an invalid authorization response') from None
async def status(self, context, provider: str) -> dict:
workspace = await self._provider(context, provider, user=True)
row = await self._read(workspace, provider)
payload = row['payload'] if row else {}
tokens = payload.get('tokens')
connected = bool(tokens and not payload.get('invalid'))
return {
'status': 'connected' if connected else 'expired' if payload.get('invalid') else 'disconnected',
'connected': connected,
'expires_at': tokens.get('expires_at') if tokens else None,
}
async def start(self, context, provider: str) -> dict:
workspace = await self._provider(context, provider, user=True)
async with self._lease(workspace, provider) as owner:
response = await self._post('/api/accounts/deviceauth/usercode', json_body={'client_id': CLIENT_ID})
if response.status_code != 200:
raise ValueError('Unable to start ChatGPT device login. Enable device code login in ChatGPT settings.')
data = self._json(response)
try:
code = data.get('user_code') or data['usercode']
device = data['device_auth_id']
interval = max(5, min(60, int(data.get('interval') or 5)))
if not isinstance(code, str) or not isinstance(device, str) or not code or not device:
raise ValueError
except (KeyError, ValueError, TypeError):
raise ValueError('ChatGPT returned an invalid device code') from None
now = time.time()
try:
expiry = data.get('expires_at')
if expiry is None:
expiry = now + float(data.get('expires_in', 900))
try:
expires_at = float(expiry)
except ValueError:
parsed = datetime.fromisoformat(expiry.replace('Z', '+00:00'))
if parsed.tzinfo is None:
parsed = parsed.replace(tzinfo=timezone.utc)
expires_at = parsed.timestamp()
if not math.isfinite(expires_at) or expires_at <= now:
raise ValueError
expires_at = min(now + 900, expires_at)
except (ValueError, TypeError):
raise ValueError('ChatGPT returned an invalid device code expiry') from None
pending = {
'authorization_id': secrets.token_urlsafe(32),
'user_code': code,
'device_auth_id': device,
'account_uuid': context.account_uuid,
'interval': interval,
'expires_at': expires_at,
'next_poll_at': now + interval,
}
row = await self._read(workspace, provider)
payload = dict(row['payload'])
payload['pending'] = pending
await self._save(workspace, provider, owner, payload)
return {k: pending[k] for k in ('authorization_id', 'user_code', 'interval', 'expires_at')} | {
'verification_uri': ISSUER + '/codex/device'
}
@staticmethod
def _attempt(payload: dict, context, authorization_id: str) -> dict | None:
pending = payload.get('pending')
if not pending or pending.get('authorization_id') != authorization_id:
return None
if pending.get('account_uuid') != context.account_uuid:
raise WorkspaceNotFoundError('Authorization not found')
return pending
async def poll(self, context, provider: str, authorization_id: str) -> dict:
workspace = await self._provider(context, provider, user=True)
if not isinstance(authorization_id, str) or not authorization_id:
raise ValueError('authorization_id is required')
async with self._lease(workspace, provider) as owner:
row = await self._read(workspace, provider)
payload = dict(row['payload'])
pending = self._attempt(payload, context, authorization_id)
if pending is None:
completed = payload.get('completed', {})
if (
completed.get('authorization_id') == authorization_id
and completed.get('account_uuid') == context.account_uuid
):
return {'status': 'connected'}
return {'status': 'expired'}
now = time.time()
if pending['expires_at'] <= now or pending.get('consumed'):
payload.pop('pending', None)
await self._save(workspace, provider, owner, payload)
return {'status': 'expired'}
if pending['next_poll_at'] > now:
return {'status': 'pending', 'interval': pending['interval']}
pending['next_poll_at'] = now + pending['interval']
await self._save(workspace, provider, owner, payload)
response = await self._post(
'/api/accounts/deviceauth/token',
json_body={'device_auth_id': pending['device_auth_id'], 'user_code': pending['user_code']},
)
if response.status_code in (403, 404, 429):
if response.status_code == 429:
pending['interval'] = min(60, pending['interval'] + 5)
pending['next_poll_at'] = time.time() + pending['interval']
await self._save(workspace, provider, owner, payload)
return {'status': 'pending', 'interval': pending['interval']}
if response.status_code != 200:
payload.pop('pending', None)
await self._save(workspace, provider, owner, payload)
raise ValueError('ChatGPT device authorization failed. Please start again.')
data = self._json(response)
if not data.get('authorization_code') or not data.get('code_verifier'):
payload.pop('pending', None)
await self._save(workspace, provider, owner, payload)
raise ValueError('ChatGPT returned an incomplete device authorization')
# Keep an attempt tombstone so cancel can preempt exchange, but never replay a code.
pending['consumed'] = True
await self._save(workspace, provider, owner, payload)
response = await self._post(
'/oauth/token',
data={
'grant_type': 'authorization_code',
'client_id': CLIENT_ID,
'code': data['authorization_code'],
'code_verifier': data['code_verifier'],
'redirect_uri': ISSUER + '/deviceauth/callback',
},
)
if response.status_code != 200:
raise ValueError('ChatGPT token exchange failed. Please start sign-in again.')
tokens = _tokens(self._json(response))
await self._save(
workspace,
provider,
owner,
{
'tokens': tokens,
'completed': {'authorization_id': authorization_id, 'account_uuid': context.account_uuid},
},
)
return {'status': 'connected'}
async def disconnect(self, context, provider: str) -> None:
workspace = await self._provider(context, provider, user=True)
await self._execute(
sa.update(CodexCredential)
.where(*self._where(workspace, provider))
.values(payload={}, lease_owner=None, lease_until=0, version=CodexCredential.version + 1)
)
async def cancel(self, context, provider: str, authorization_id: str) -> None:
workspace = await self._provider(context, provider, user=True)
deadline = time.monotonic() + 65
while time.monotonic() < deadline:
row = await self._read(workspace, provider)
if row is None:
return
old = row['payload']
if self._attempt(old, context, authorization_id) is None:
return
lease_owner = row['lease_owner']
if lease_owner and lease_owner.startswith('refresh:') and row['lease_until'] > time.time():
# A rotated refresh token must be committed before removing the attempt.
await asyncio.sleep(0.1)
continue
payload = dict(old)
payload.pop('pending', None)
result = await self._execute(
sa.update(CodexCredential)
.where(
*self._where(workspace, provider),
CodexCredential.version == row['version'],
# Lease acquisition does not change version; fence that race too.
CodexCredential.lease_owner == lease_owner,
)
.values(payload=payload, lease_owner=None, lease_until=0, version=CodexCredential.version + 1)
)
if result.rowcount == 1:
return
raise ValueError('Authorization changed concurrently. Please retry cancellation.')
async def access(self, context, provider: str, *, rejected_token: str | None = None) -> dict:
workspace = await self._provider(context, provider)
row = await self._read(workspace, provider)
payload = row['payload'] if row else {}
tokens = payload.get('tokens')
if not tokens or payload.get('invalid'):
raise ValueError(LOGIN_REQUIRED)
if tokens['expires_at'] > time.time() + 120 and tokens['access_token'] != rejected_token:
return tokens
async with self._lease(workspace, provider, refresh=True) as owner:
row = await self._read(workspace, provider)
payload = dict(row['payload'])
tokens = payload.get('tokens')
if not tokens or payload.get('invalid'):
raise ValueError(LOGIN_REQUIRED)
if tokens['expires_at'] > time.time() + 120 and tokens['access_token'] != rejected_token:
return tokens
response = await self._post(
'/oauth/token',
data={'grant_type': 'refresh_token', 'client_id': CLIENT_ID, 'refresh_token': tokens['refresh_token']},
)
error = self._json(response).get('error') if response.status_code in (400, 401, 403) else None
error_code = error.get('code') if isinstance(error, dict) else error
if error_code in (
'invalid_grant',
'refresh_token_reused',
'refresh_token_expired',
'refresh_token_revoked',
):
payload['invalid'] = True
payload.pop('tokens', None)
payload.pop('completed', None)
await self._save(workspace, provider, owner, payload)
raise ValueError(LOGIN_REQUIRED)
if response.status_code != 200:
raise ValueError('ChatGPT token refresh temporarily failed. Please retry.')
refreshed = _tokens(self._json(response), tokens)
payload['tokens'] = refreshed
await self._save(workspace, provider, owner, payload)
return refreshed
@@ -1,10 +0,0 @@
"""Explicitly safe Codex failures; never construct messages from upstream bodies."""
class CodexProviderError(ValueError):
"""A known provider failure safe to expose at the HTTP boundary."""
def __init__(self, message: str, status_code: int = 502, error_code: str = 'codex_upstream_failure'):
super().__init__(message)
self.status_code = status_code
self.error_code = error_code
+5 -31
View File
@@ -18,7 +18,7 @@ from ...discover import engine
from ...entity.errors import provider as provider_errors from ...entity.errors import provider as provider_errors
from ...entity.persistence import model as persistence_model from ...entity.persistence import model as persistence_model
from ...workspace.entities import WorkspaceExecutionBinding from ...workspace.entities import WorkspaceExecutionBinding
from ...workspace.errors import WorkspaceError, WorkspaceInvariantError, WorkspaceNotFoundError from ...workspace.errors import WorkspaceError, WorkspaceInvariantError
from . import requester, token from . import requester, token
@@ -638,32 +638,10 @@ class ModelManager:
) -> requester.RuntimeLLMModel: ) -> requester.RuntimeLLMModel:
execution_context = await self.resolve_execution_context(context) execution_context = await self.resolve_execution_context(context)
provider_info = {**model_info.get('provider', {}), 'workspace_uuid': execution_context.workspace_uuid} provider_info = {**model_info.get('provider', {}), 'workspace_uuid': execution_context.workspace_uuid}
provider_uuid = model_info.get('provider_uuid') or provider_info.get('uuid') runtime_provider = await self._build_provider(
inline_codex = provider_info.get('requester') == 'openai-codex' execution_context,
provider_entity = persistence_model.ModelProvider(**provider_info) persistence_model.ModelProvider(**provider_info),
if provider_uuid: )
if provider_info.get('uuid') and provider_info['uuid'] != provider_uuid:
raise ValueError('Conflicting provider identities')
result = await self.ap.persistence_mgr.execute_async(
sqlalchemy.select(persistence_model.ModelProvider).where(
persistence_model.ModelProvider.workspace_uuid == execution_context.workspace_uuid,
persistence_model.ModelProvider.uuid == provider_uuid,
)
)
saved_provider = result.first()
if saved_provider is None:
if inline_codex or model_info.get('provider_uuid'):
raise WorkspaceNotFoundError('Provider not found')
else:
saved_provider = self._coerce_provider(saved_provider, execution_context)
if saved_provider.requester == 'openai-codex':
# OAuth identity and transport configuration are server-owned.
provider_entity = saved_provider
elif inline_codex:
raise ValueError('This provider does not use ChatGPT sign-in')
elif inline_codex:
raise WorkspaceNotFoundError('Provider not found')
runtime_provider = await self._build_provider(execution_context, provider_entity)
model_entity = persistence_model.LLMModel( model_entity = persistence_model.LLMModel(
workspace_uuid=execution_context.workspace_uuid, workspace_uuid=execution_context.workspace_uuid,
uuid=model_info.get('uuid', ''), uuid=model_info.get('uuid', ''),
@@ -745,10 +723,6 @@ class ModelManager:
'requester_name': provider_entity.requester, 'requester_name': provider_entity.requester,
} }
if provider_entity.requester == 'openai-codex':
config['provider_uuid'] = provider_entity.uuid
config['workspace_uuid'] = context.workspace_uuid
if litellm_provider: if litellm_provider:
from .requesters import litellmchat from .requesters import litellmchat
@@ -1,423 +0,0 @@
"""Native ChatGPT Codex Responses/SSE requester (never Chat Completions)."""
from __future__ import annotations
import asyncio
import json
import secrets
import time
from collections import OrderedDict
import httpx
import langbot
import langbot_plugin.api.entities.builtin.provider.message as pm
from .. import requester, reasoning
from ..codex_auth import BASE_URL, CodexAuth, LOGIN_REQUIRED
from ..codex_errors import CodexProviderError
async def sse_events(response):
"""Decode SSE records, including CRLF, comments, and multiline data."""
data = []
size = 0
async for line in response.aiter_lines():
if not line:
if data:
text = '\n'.join(data)
if text == '[DONE]':
return
try:
event = json.loads(text)
if not isinstance(event, dict):
raise ValueError
except ValueError:
raise ValueError('Codex returned an invalid stream event') from None
yield event
data, size = [], 0
elif line.startswith('data:'):
value = line[5:]
if value.startswith(' '):
value = value[1:]
size += len(value)
if size > 4 * 1024 * 1024:
raise ValueError('Codex stream event exceeds the size limit')
data.append(value)
# SSE requires the blank separator; unterminated records cannot prove completion.
def _content(message):
content = message.content
if isinstance(content, str):
return [{'type': 'output_text' if message.role == 'assistant' else 'input_text', 'text': content}]
result = []
for part in content or []:
if part.type == 'text':
result.append(
{'type': 'output_text' if message.role == 'assistant' else 'input_text', 'text': part.text or ''}
)
elif part.type == 'image_url' and part.image_url is not None:
result.append({'type': 'input_image', 'image_url': part.image_url.url})
elif part.type == 'image_base64' and part.image_base64:
value = part.image_base64
result.append(
{
'type': 'input_image',
'image_url': value if value.startswith('data:') else 'data:image/png;base64,' + value,
}
)
else:
raise ValueError('Codex supports text and images only; this message contains unsupported content')
return result
def _tool(item):
try:
return pm.ToolCall(
id=item['call_id'],
type='function',
function=pm.FunctionCall(name=item['name'], arguments=item.get('arguments') or ''),
)
except (KeyError, ValueError, TypeError):
raise ValueError('Codex returned an invalid function call') from None
def _usage(response):
usage = response.get('usage') or {}
return {
'prompt_tokens': usage.get('input_tokens', 0),
'completion_tokens': usage.get('output_tokens', 0),
'total_tokens': usage.get('total_tokens', usage.get('input_tokens', 0) + usage.get('output_tokens', 0)),
'prompt_tokens_details': usage.get('input_tokens_details', {}),
'completion_tokens_details': usage.get('output_tokens_details', {}),
}
class CodexRequester(requester.ProviderAPIRequester):
async def initialize(self):
self.auth = CodexAuth(self.ap)
self.workspace = self.requester_cfg['workspace_uuid']
self.provider = self.requester_cfg['provider_uuid']
# Opaque replay data stays server-side; handles are scoped to the same query,
# model and OAuth connection. No token or encrypted reasoning enters messages.
self._replay = OrderedDict()
async def aclose(self):
self._replay.clear()
def get_reasoning_capabilities(self, model):
return {
'supported': True,
'levels': ['provider_default', 'low', 'medium', 'high', 'xhigh'],
'source': 'provider',
}
@staticmethod
def _headers(tokens, *, stream=False):
return {
'Authorization': 'Bearer ' + tokens['access_token'],
'ChatGPT-Account-ID': tokens['account_id'],
'User-Agent': 'LangBot/' + langbot.__version__,
'originator': 'langbot',
'OpenAI-Beta': 'responses=experimental',
'Accept': 'text/event-stream' if stream else 'application/json',
}
@staticmethod
def _http_error(status):
if status == 401:
# Upstream authentication is not LangBot authentication: HTTP401 would
# make the browser discard its own valid user session.
return CodexProviderError(LOGIN_REQUIRED, 400, 'codex_reauthentication_required')
if status == 429:
return CodexProviderError(
'ChatGPT request was limited (rate limit or usage restriction). Please retry later or check your plan.',
429,
'codex_rate_limited',
)
if status == 403:
return CodexProviderError(
'ChatGPT denied this request. Check subscription and workspace permissions.',
403,
'codex_access_denied',
)
if status == 400:
return CodexProviderError(
'ChatGPT rejected the model or request. Check the selected model and request settings.',
400,
'codex_invalid_request',
)
return CodexProviderError('ChatGPT Codex upstream request failed. Please retry later.')
async def _response_error(self, response):
# Inspect only a bounded 429 error record and an allowlisted machine code.
# Never expose upstream prose, reset metadata, headers or credentials.
if response.status_code == 429:
payload = bytearray()
async for chunk in response.aiter_bytes():
if len(payload) + len(chunk) > 8192:
return self._http_error(429)
payload.extend(chunk)
try:
data = json.loads(payload)
error = data.get('error') if isinstance(data, dict) else None
if isinstance(error, dict) and (
error.get('type') == 'usage_limit_reached' or error.get('code') == 'usage_limit_reached'
):
return CodexProviderError(
'ChatGPT subscription usage limit reached. Please retry later or check your plan.',
429,
'codex_usage_limit_reached',
)
except (ValueError, UnicodeError):
pass
return self._http_error(response.status_code)
def _scope(self, query, model, tokens):
return (
id(query),
getattr(query, 'query_id', None),
model.model_entity.name,
tokens.get('connection_id'),
tokens['account_id'],
)
def _body(self, query, model, messages, funcs, extra_args, tokens):
args = {**(model.model_entity.extra_args or {}), **(extra_args or {})}
# Never permit credentials, transport overrides, store/history or arbitrary
# SDK kwargs to be smuggled through model advanced parameters.
allowed = {'reasoning', 'text', 'parallel_tool_calls', 'tool_choice'}
unknown = set(args) - allowed
if unknown:
raise ValueError('Unsupported Codex advanced parameters: ' + ', '.join(sorted(unknown)))
instructions = []
items = []
scope = self._scope(query, model, tokens)
for message in messages:
if message.role in ('system', 'developer'):
instructions.append('\n'.join(p['text'] for p in _content(message) if 'text' in p))
continue
if message.role == 'tool':
if not message.tool_call_id:
raise ValueError('Codex tool results require a tool_call_id')
output = (
message.content
if isinstance(message.content, str)
else json.dumps([p.model_dump(exclude_none=True) for p in message.content or []])
)
items.append({'type': 'function_call_output', 'call_id': message.tool_call_id, 'output': output or ''})
continue
if message.role not in ('assistant', 'user'):
raise ValueError('Unsupported Codex message role')
handle = (message.provider_specific_fields or {}).get('codex_replay_id')
cached = self._replay.get(handle) if isinstance(handle, str) else None
if query is not None and cached and cached[0] == scope and cached[1] > time.time():
items.extend(cached[2])
continue
content = _content(message)
if content:
items.append({'type': 'message', 'role': message.role, 'content': content})
for call in message.tool_calls or []:
items.append(
{
'type': 'function_call',
'call_id': call.id,
'name': call.function.name,
'arguments': call.function.arguments,
}
)
body = {
**args,
'model': model.model_entity.name,
'instructions': '\n\n'.join(instructions),
'input': items,
'store': False,
'stream': True,
'include': ['reasoning.encrypted_content'],
}
level = reasoning.normalize_reasoning_config(getattr(model.model_entity, 'reasoning_config', None))['level']
if level != 'provider_default':
reasoning.validate_reasoning_capabilities(
{'level': level}, self.get_reasoning_capabilities(model), model.model_entity.name
)
body['reasoning'] = {'effort': level, 'summary': 'auto'}
if funcs:
body['tools'] = [
{
'type': 'function',
'name': f.name,
'description': f.description,
'parameters': f.parameters,
'strict': False,
}
for f in funcs
]
return body
async def _events(self, query, model, messages, funcs, extra_args):
tokens = await self.auth.access(self.workspace, self.provider)
try:
async with asyncio.timeout(300), httpx.AsyncClient(timeout=120, follow_redirects=False) as client:
for attempt in range(2):
body = self._body(query, model, messages, funcs, extra_args, tokens)
async with client.stream(
'POST', BASE_URL + '/responses', json=body, headers=self._headers(tokens, stream=True)
) as response:
if response.status_code == 401 and attempt == 0:
tokens = await self.auth.access(
self.workspace, self.provider, rejected_token=tokens['access_token']
)
continue
if response.status_code != 200:
raise await self._response_error(response)
async for event in sse_events(response):
yield event, tokens
return
except (httpx.HTTPError, TimeoutError):
raise ValueError('ChatGPT Codex network error or timeout. Please retry.') from None
async def _chunks(self, query, model, messages, funcs, extra_args, remove_think, usage_out):
text = ''
seen_calls = set()
output_items = {}
response_id = None
async for event, tokens in self._events(query, model, messages, funcs, extra_args):
kind = event.get('type')
response = event.get('response') or {}
response_id = response.get('id') or response_id
if kind in ('error', 'response.failed', 'response.incomplete'):
raise CodexProviderError('ChatGPT Codex response failed or was incomplete. Please retry.')
if kind == 'response.output_text.delta':
delta = event.get('delta', '')
text += delta
yield pm.MessageChunk(role='assistant', content=delta, resp_message_id=response_id)
elif kind in ('response.reasoning_summary_text.delta', 'response.reasoning_text.delta'):
if not remove_think:
yield pm.MessageChunk(
role='assistant',
content='',
provider_specific_fields={'reasoning_content': event.get('delta', '')},
)
elif kind == 'response.output_item.done':
item = event.get('item') or {}
output_items[event.get('output_index', len(output_items))] = item
if item.get('type') == 'function_call' and item.get('call_id') not in seen_calls:
seen_calls.add(item.get('call_id'))
yield pm.MessageChunk(role='assistant', content='', tool_calls=[_tool(item)])
elif kind in ('response.completed', 'response.done'):
if response.get('status') not in (None, 'completed'):
raise CodexProviderError('ChatGPT Codex response was not completed')
output = response.get('output') or [output_items[k] for k in sorted(output_items)]
for item in output:
if item.get('type') == 'function_call' and item.get('call_id') not in seen_calls:
seen_calls.add(item.get('call_id'))
yield pm.MessageChunk(role='assistant', content='', tool_calls=[_tool(item)])
# Some servers send only the terminal output, without text deltas.
final_text = ''.join(
p.get('text', '')
for item in output
if item.get('type') == 'message'
for p in item.get('content', [])
if p.get('type') == 'output_text'
)
if not text and final_text:
text = final_text
yield pm.MessageChunk(role='assistant', content=text, resp_message_id=response_id)
usage_out.update(_usage(response))
if query is not None:
if query.variables is None:
query.variables = {}
query.variables[requester.STREAM_USAGE_QUERY_VARIABLE] = dict(usage_out)
fields = None
if query is not None and output:
handle = secrets.token_urlsafe(24)
self._replay[handle] = (self._scope(query, model, tokens), time.time() + 3600, output)
while len(self._replay) > 64:
self._replay.popitem(last=False)
fields = {'codex_replay_id': handle}
yield pm.MessageChunk(
role='assistant',
content='',
all_content=text,
is_final=True,
resp_message_id=response_id,
provider_specific_fields=fields,
)
return
raise CodexProviderError('ChatGPT Codex stream ended before completion. Please retry.')
async def invoke_llm_stream(self, query, model, messages, funcs=None, extra_args=None, remove_think=False):
async for chunk in self._chunks(query, model, messages, funcs, extra_args, remove_think, {}):
yield chunk
async def invoke_llm(self, query, model, messages, funcs=None, extra_args=None, remove_think=False):
usage = {}
text = ''
calls = []
fields = {}
response_id = None
async for chunk in self._chunks(query, model, messages, funcs, extra_args, remove_think, usage):
text += chunk.content or ''
calls.extend(chunk.tool_calls or [])
response_id = chunk.resp_message_id or response_id
for key, value in (chunk.provider_specific_fields or {}).items():
fields[key] = fields.get(key, '') + value if key == 'reasoning_content' else value
return pm.Message(
role='assistant',
content=text,
tool_calls=calls or None,
resp_message_id=response_id,
provider_specific_fields=fields or None,
), usage
async def scan_models(self, api_key=None):
tokens = await self.auth.access(self.workspace, self.provider)
try:
async with asyncio.timeout(90), httpx.AsyncClient(timeout=30, follow_redirects=False) as client:
for attempt in range(2):
response = await client.get(
BASE_URL + '/models',
params={'client_version': langbot.__version__},
headers=self._headers(tokens),
)
if response.status_code == 401 and attempt == 0:
tokens = await self.auth.access(
self.workspace, self.provider, rejected_token=tokens['access_token']
)
continue
if response.status_code != 200:
raise await self._response_error(response)
data = response.json()
if not isinstance(data, dict) or not isinstance(data.get('models'), list):
raise ValueError('ChatGPT returned an invalid model catalog')
result = {}
for item in data['models']:
name = item.get('slug') or item.get('id')
if not isinstance(name, str) or not name or item.get('visibility') == 'hide':
continue
modalities = item.get('input_modalities') or ['text']
abilities = ['func_call']
if 'image' in modalities:
abilities.append('vision')
if item.get('supported_reasoning_levels'):
abilities.append('reasoning')
result[name] = {
'id': name,
'name': name,
'type': 'llm',
'abilities': abilities,
'display_name': item.get('display_name'),
'description': item.get('description'),
'context_length': item.get('context_window'),
'input_modalities': modalities,
'output_modalities': ['text'],
'owned_by': 'openai',
}
return {'models': list(result.values()), 'debug': None}
except (httpx.HTTPError, TimeoutError):
raise ValueError('ChatGPT model discovery network error. Please retry.') from None
except (ValueError, TypeError, KeyError, AttributeError) as exc:
# Never echo upstream response bodies (which may contain credentials).
if isinstance(exc, ValueError) and str(exc).startswith(('ChatGPT', 'Codex')):
raise
raise ValueError('ChatGPT returned an invalid model catalog') from None
@@ -1,27 +0,0 @@
apiVersion: v1
kind: LLMAPIRequester
metadata:
name: openai-codex
label:
en_US: OpenAI Codex
zh_Hans: OpenAI Codex
ja_JP: OpenAI Codex
icon: openai.svg
spec:
config:
- name: base_url
label:
en_US: ChatGPT endpoint
zh_Hans: ChatGPT 服务地址
ja_JP: ChatGPT エンドポイント
type: string
required: false
default: https://chatgpt.com/backend-api/codex
alias: "openai codex ChatGPT subscription OAuth 订阅"
support_type:
- llm
provider_category: manufacturer
execution:
python:
path: ./codex.py
attr: CodexRequester
@@ -90,7 +90,7 @@ class LangflowAPIRunner(runner.RequestRunner):
} }
# 如果配置中有tweaks,则添加到负载中 # 如果配置中有tweaks,则添加到负载中
tweaks = json.loads(self.pipeline_config['ai']['langflow-api'].get('tweaks') or '{}') tweaks = json.loads(self.pipeline_config['ai']['langflow-api'].get('tweaks'))
if tweaks: if tweaks:
payload['tweaks'] = tweaks payload['tweaks'] = tweaks
+1 -56
View File
@@ -1,6 +1,5 @@
from __future__ import annotations from __future__ import annotations
import dataclasses
import enum import enum
import json import json
import math import math
@@ -207,13 +206,6 @@ class MCPSessionStatus(enum.Enum):
ERROR = 'error' ERROR = 'error'
@dataclasses.dataclass(frozen=True)
class MCPOAuthChallenge:
"""Bearer challenge metadata returned by an OAuth-protected MCP server."""
resource_metadata_url: str | None
class _TransportReconnect(Exception): class _TransportReconnect(Exception):
"""Internal signal: the Box stdio WS transport dropped but the managed """Internal signal: the Box stdio WS transport dropped but the managed
process is still alive. Triggers a lightweight transport reconnect that process is still alive. Triggers a lightweight transport reconnect that
@@ -273,7 +265,6 @@ class RuntimeMCPSession:
_ready_event: asyncio.Event _ready_event: asyncio.Event
error_message: str | None = None error_message: str | None = None
_public_error_code: str = 'runtime_error'
error_phase: MCPSessionErrorPhase | None = None error_phase: MCPSessionErrorPhase | None = None
@@ -519,13 +510,6 @@ class RuntimeMCPSession:
await self._init_streamable_http_server() await self._init_streamable_http_server()
return return
except Exception as e: except Exception as e:
if self._extract_oauth_challenge(e) is not None:
self.error_phase = MCPSessionErrorPhase.OAUTH_REQUIRED
self.ap.logger.info(
f'MCP server {self.server_name}: remote server requires OAuth authorization; '
'not falling back to SSE'
)
raise
if not self._should_fallback_to_sse(e): if not self._should_fallback_to_sse(e):
self.ap.logger.info( self.ap.logger.info(
f'MCP server {self.server_name}: Streamable HTTP transport failed ' f'MCP server {self.server_name}: Streamable HTTP transport failed '
@@ -646,7 +630,6 @@ class RuntimeMCPSession:
except Exception as e: except Exception as e:
self.status = MCPSessionStatus.ERROR self.status = MCPSessionStatus.ERROR
self.error_message = str(e) self.error_message = str(e)
self._public_error_code = self._classify_public_error(e)
self.ap.logger.error(f'Error in MCP session lifecycle {self.server_name}: {e}\n{traceback.format_exc()}') self.ap.logger.error(f'Error in MCP session lifecycle {self.server_name}: {e}\n{traceback.format_exc()}')
# Do NOT set _ready_event here — let _lifecycle_loop_with_retry # Do NOT set _ready_event here — let _lifecycle_loop_with_retry
# handle retries first. It will set the event when all retries # handle retries first. It will set the event when all retries
@@ -769,11 +752,6 @@ class RuntimeMCPSession:
except Exception as e: except Exception as e:
if self._shutdown_event.is_set(): if self._shutdown_event.is_set():
return # Shutdown requested, don't retry return # Shutdown requested, don't retry
if self.error_phase == MCPSessionErrorPhase.OAUTH_REQUIRED:
self.retry_count = attempt + 1
self.status = MCPSessionStatus.ERROR
self._ready_event.set()
return
if self.error_phase == MCPSessionErrorPhase.BOX_UNAVAILABLE: if self.error_phase == MCPSessionErrorPhase.BOX_UNAVAILABLE:
box_service = getattr(self.ap, 'box_service', None) box_service = getattr(self.ap, 'box_service', None)
if box_service is not None and getattr(box_service, 'enabled', True): if box_service is not None and getattr(box_service, 'enabled', True):
@@ -854,39 +832,6 @@ class RuntimeMCPSession:
else: else:
yield exc yield exc
@staticmethod
def _classify_public_error(exc: BaseException) -> str:
"""Expose a safe category without transport URLs, headers, or arguments."""
for leaf in RuntimeMCPSession._iter_exception_leaves(exc):
if isinstance(leaf, httpx.HTTPStatusError):
return f'http_{leaf.response.status_code}'
if isinstance(leaf, (httpx.TimeoutException, TimeoutError)):
return 'connection_timeout'
if isinstance(leaf, httpx.ConnectError):
return 'connection_unreachable'
return 'runtime_error'
@staticmethod
def _extract_oauth_challenge(exc: BaseException) -> MCPOAuthChallenge | None:
"""Extract an OAuth Bearer challenge from a remote MCP connection failure."""
for leaf in RuntimeMCPSession._iter_exception_leaves(exc):
if not isinstance(leaf, httpx.HTTPStatusError) or leaf.response.status_code != 401:
continue
for header in leaf.response.headers.get_list('www-authenticate'):
bearer_match = re.search(r'(?:^|,)\s*Bearer(?:\s|,|$)', header, flags=re.IGNORECASE)
if bearer_match is None:
continue
metadata_match = re.search(
r'(?:^|,)\s*resource_metadata\s*=\s*(?:"([^"]+)"|([^,\s]+))',
header[bearer_match.end() :],
flags=re.IGNORECASE,
)
if metadata_match is None:
continue
resource_metadata_url = metadata_match.group(1) or metadata_match.group(2)
return MCPOAuthChallenge(resource_metadata_url=resource_metadata_url)
return None
@staticmethod @staticmethod
def _should_fallback_to_sse(exc: BaseException) -> bool: def _should_fallback_to_sse(exc: BaseException) -> bool:
"""Whether a Streamable HTTP failure matches legacy-SSE fallback. """Whether a Streamable HTTP failure matches legacy-SSE fallback.
@@ -1429,7 +1374,7 @@ class RuntimeMCPSession:
# environment values. Detailed diagnostics belong in AUDIT_VIEW # environment values. Detailed diagnostics belong in AUDIT_VIEW
# logs; resource-list responses expose only a stable status. # logs; resource-list responses expose only a stable status.
'error_message': 'MCP runtime failed' if self.error_message else None, 'error_message': 'MCP runtime failed' if self.error_message else None,
'error_code': self._public_error_code if self.error_message else None, 'error_code': 'runtime_error' if self.error_message else None,
'error_phase': self.error_phase.value if self.error_phase else None, 'error_phase': self.error_phase.value if self.error_phase else None,
'retry_count': self.retry_count, 'retry_count': self.retry_count,
'tool_count': len(self.get_tools()), 'tool_count': len(self.get_tools()),
@@ -18,12 +18,11 @@ from ....box.workspace import (
BoxWorkspaceSession, BoxWorkspaceSession,
classify_python_workspace, classify_python_workspace,
infer_workspace_host_path, infer_workspace_host_path,
normalize_host_path,
rewrite_mounted_path, rewrite_mounted_path,
rewrite_venv_command, rewrite_venv_command,
unwrap_venv_path, unwrap_venv_path,
wrap_python_command_with_env,
) )
from ....utils.python_workspace import normalize_host_path, wrap_python_command_with_env
if TYPE_CHECKING: if TYPE_CHECKING:
from .mcp import RuntimeMCPSession from .mcp import RuntimeMCPSession
@@ -52,7 +51,6 @@ class MCPSessionErrorPhase(enum.Enum):
MCP_INIT = 'mcp_init' MCP_INIT = 'mcp_init'
RUNTIME = 'runtime' RUNTIME = 'runtime'
TOOL_CALL = 'tool_call' TOOL_CALL = 'tool_call'
OAUTH_REQUIRED = 'oauth_required'
# Stdio MCP refused because Box is disabled in config or currently # Stdio MCP refused because Box is disabled in config or currently
# unavailable. Not transient — retries would be pointless. The frontend # unavailable. Not transient — retries would be pointless. The frontend
# uses this phase to render a localized actionable message instead of # uses this phase to render a localized actionable message instead of
@@ -32,7 +32,7 @@ EDIT_TOOL_NAME = 'edit'
GLOB_TOOL_NAME = 'glob' GLOB_TOOL_NAME = 'glob'
GREP_TOOL_NAME = 'grep' GREP_TOOL_NAME = 'grep'
_ALL_TOOL_NAMES = {EXEC_TOOL_NAME, READ_TOOL_NAME, WRITE_TOOL_NAME, EDIT_TOOL_NAME, GLOB_TOOL_NAME, GREP_TOOL_NAME} SANDBOX_TOOL_NAMES = {EXEC_TOOL_NAME, READ_TOOL_NAME, WRITE_TOOL_NAME, EDIT_TOOL_NAME, GLOB_TOOL_NAME, GREP_TOOL_NAME}
# Skip these dirs during grep walk to avoid noise # Skip these dirs during grep walk to avoid noise
_SKIP_DIRS = {'.git', 'node_modules', '__pycache__', '.venv', 'venv', '.tox', 'dist', 'build'} _SKIP_DIRS = {'.git', 'node_modules', '__pycache__', '.venv', 'venv', '.tox', 'dist', 'build'}
@@ -260,7 +260,11 @@ class NativeToolLoader(loader.ToolLoader):
return list(self._tools) return list(self._tools)
async def has_tool(self, name: str) -> bool: async def has_tool(self, name: str) -> bool:
return name in _ALL_TOOL_NAMES and await self._is_sandbox_available() return name in SANDBOX_TOOL_NAMES and await self._is_sandbox_available()
@staticmethod
def recognizes_tool(name: str) -> bool:
return name in SANDBOX_TOOL_NAMES
async def invoke_tool(self, name: str, parameters: dict, query: pipeline_query.Query): async def invoke_tool(self, name: str, parameters: dict, query: pipeline_query.Query):
require_sandbox = getattr( require_sandbox = getattr(
@@ -325,20 +329,11 @@ class NativeToolLoader(loader.ToolLoader):
if not package_root: if not package_root:
raise ValueError(f'Activated skill "{selected_skill_name}" has no package_root.') raise ValueError(f'Activated skill "{selected_skill_name}" has no package_root.')
# Pass only the logical name across the authenticated Core→Runtime
# boundary. In Cloud mode the shared Box Runtime resolves the
# Workspace-scoped package root and constructs the read-only mount;
# Core host paths are never accepted as mount authority.
# Wrap command with Python venv bootstrap if the skill has a Python project. # Wrap command with Python venv bootstrap if the skill has a Python project.
# The venv is created inside the skill's mount path. # The venv is created inside the skill's mount path.
skill_mount = f'/workspace/.skills/{selected_skill_name}' skill_mount = f'/workspace/.skills/{selected_skill_name}'
python_project = selected_skill.get('python_project') is True python_project = selected_skill.get('python_project') is True
if 'python_project' not in selected_skill and bool( if 'python_project' not in selected_skill:
getattr(self.ap.box_service, 'shares_filesystem_with_box', False)
):
# Backward compatibility for a same-process OSS Runtime that
# predates trusted Box metadata. Never probe a path reported by
# an external Runtime from the Core filesystem.
python_project = skill_loader.should_prepare_skill_python_env(package_root) python_project = skill_loader.should_prepare_skill_python_env(package_root)
if python_project: if python_project:
parameters = dict(parameters) parameters = dict(parameters)
@@ -354,12 +349,9 @@ class NativeToolLoader(loader.ToolLoader):
result = await self.ap.box_service.execute_tool( result = await self.ap.box_service.execute_tool(
parameters, parameters,
query, query,
skill_name=selected_skill_name, read_only_mounts=skill_loader.build_execution_mounts(self.ap, query),
) )
result = self._normalize_exec_result(result) result = self._normalize_exec_result(result)
if selected_skill is not None:
self._refresh_skill_from_disk(query, selected_skill)
return result return result
def _resolve_host_location( def _resolve_host_location(
@@ -381,10 +373,7 @@ class NativeToolLoader(loader.ToolLoader):
box_service = self.ap.box_service box_service = self.ap.box_service
if selected_skill is not None: if selected_skill is not None:
if not self._can_interpret_skill_host_paths(): if not self._can_interpret_skill_host_paths():
raise ValueError( raise ValueError('Secure Core host file operations are unavailable on this platform.')
'Skill package paths are owned by the Box Runtime; '
'this operation requires a Runtime skill-file API.'
)
host_root = selected_skill.get('package_root') host_root = selected_skill.get('package_root')
workspace_anchor = None workspace_anchor = None
else: else:
@@ -422,11 +411,9 @@ class NativeToolLoader(loader.ToolLoader):
return selected_skill, relative return selected_skill, relative
def _can_interpret_skill_host_paths(self) -> bool: def _can_interpret_skill_host_paths(self) -> bool:
"""Require an explicitly proven shared Core/Runtime filesystem view.""" """Return whether Core can use its no-follow host file primitives."""
return _SECURE_HOST_FILE_OPS_AVAILABLE and bool( return _SECURE_HOST_FILE_OPS_AVAILABLE
getattr(self.ap.box_service, 'shares_filesystem_with_box', False)
)
def _should_use_box_workspace_files(self, selected_skill: dict | None) -> bool: def _should_use_box_workspace_files(self, selected_skill: dict | None) -> bool:
if selected_skill is not None: if selected_skill is not None:
@@ -1123,20 +1110,11 @@ else:
include_visible=True, include_visible=True,
include_activated=True, include_activated=True,
) )
if skill_request is not None and hasattr(self.ap.box_service, 'read_skill_file'): skill_repository = getattr(self.ap, 'skill_repository', None)
if skill_request is not None and skill_repository is not None:
selected_skill, relative = skill_request selected_skill, relative = skill_request
if self._can_interpret_skill_host_paths():
host_location = self._resolve_skill_host_location(selected_skill, relative)
else:
host_location = None
if host_location is not None:
try:
return await asyncio.to_thread(self._read_host_location, host_location, parameters)
except FileNotFoundError:
pass
try: try:
result = await self.ap.box_service.read_skill_file( result = await skill_repository.read_skill_file(
self._execution_context(query), self._execution_context(query),
selected_skill['name'], selected_skill['name'],
relative, relative,
@@ -1144,7 +1122,7 @@ else:
return self._build_read_result_from_text(str(result.get('content', '')), parameters) return self._build_read_result_from_text(str(result.get('content', '')), parameters)
except Exception: except Exception:
try: try:
result = await self.ap.box_service.list_skill_files( result = await skill_repository.list_skill_files(
self._execution_context(query), self._execution_context(query),
selected_skill['name'], selected_skill['name'],
relative, relative,
@@ -1178,12 +1156,13 @@ else:
include_visible=False, include_visible=False,
include_activated=True, include_activated=True,
) )
if skill_request is not None and hasattr(self.ap.box_service, 'write_skill_file'): skill_repository = getattr(self.ap, 'skill_repository', None)
if skill_request is not None and skill_repository is not None:
if encoding != 'text': if encoding != 'text':
return {'ok': False, 'error': 'base64 writes to skill packages are not supported.'} return {'ok': False, 'error': 'base64 writes to skill packages are not supported.'}
selected_skill, relative = skill_request selected_skill, relative = skill_request
execution_context = self._execution_context(query) execution_context = self._execution_context(query)
await self.ap.box_service.write_skill_file(execution_context, selected_skill['name'], relative, content) await skill_repository.write_skill_file(execution_context, selected_skill['name'], relative, content)
await self.ap.skill_mgr.reload_skills(execution_context) await self.ap.skill_mgr.reload_skills(execution_context)
return {'ok': True, 'path': path} return {'ok': True, 'path': path}
@@ -1216,14 +1195,10 @@ else:
include_visible=False, include_visible=False,
include_activated=True, include_activated=True,
) )
if ( if skill_request is not None and getattr(self.ap, 'skill_repository', None) is not None:
skill_request is not None
and hasattr(self.ap.box_service, 'read_skill_file')
and hasattr(self.ap.box_service, 'write_skill_file')
):
selected_skill, relative = skill_request selected_skill, relative = skill_request
try: try:
result = await self.ap.box_service.read_skill_file( result = await self.ap.skill_repository.read_skill_file(
self._execution_context(query), self._execution_context(query),
selected_skill['name'], selected_skill['name'],
relative, relative,
@@ -1238,7 +1213,7 @@ else:
return {'ok': False, 'error': f'old_string matches {count} locations; provide a more unique string.'} return {'ok': False, 'error': f'old_string matches {count} locations; provide a more unique string.'}
new_content = content.replace(old_string, new_string, 1) new_content = content.replace(old_string, new_string, 1)
execution_context = self._execution_context(query) execution_context = self._execution_context(query)
await self.ap.box_service.write_skill_file( await self.ap.skill_repository.write_skill_file(
execution_context, execution_context,
selected_skill['name'], selected_skill['name'],
relative, relative,
@@ -1,10 +1,14 @@
from __future__ import annotations from __future__ import annotations
import os
import re import re
import typing import typing
from ....box import workspace as box_workspace
from ....api.http.context import ExecutionContext from ....api.http.context import ExecutionContext
from ....utils.python_workspace import (
should_prepare_python_env,
wrap_python_command_with_env,
)
if typing.TYPE_CHECKING: if typing.TYPE_CHECKING:
from ....core import app from ....core import app
@@ -57,6 +61,30 @@ def get_visible_skill(ap: app.Application, query: pipeline_query.Query, skill_na
return get_visible_skills(ap, query).get(skill_name) return get_visible_skills(ap, query).get(skill_name)
def build_execution_mounts(ap: app.Application, query: pipeline_query.Query) -> list[dict]:
"""Translate visible Core-owned packages into generic read-only mounts."""
mounts: list[dict] = []
for skill_name, skill_data in get_visible_skills(ap, query).items():
package_root = str(skill_data.get('package_root', '') or '').strip()
if not package_root:
continue
if not os.path.isdir(package_root):
ap.logger.warning(
f'Skill "{skill_name}" package_root missing on the Core filesystem '
f'({package_root}); skipping its execution mount. Reload the skill catalog.'
)
continue
mounts.append(
{
'host_path': package_root,
'mount_path': get_virtual_skill_mount_path(skill_name),
'mode': 'ro',
}
)
return mounts
def get_activated_skills(query: pipeline_query.Query) -> dict[str, dict]: def get_activated_skills(query: pipeline_query.Query) -> dict[str, dict]:
if query.variables is None: if query.variables is None:
return {} return {}
@@ -198,7 +226,7 @@ def build_skill_session_id(skill_data: dict, query: pipeline_query.Query) -> str
def should_prepare_skill_python_env(package_root: str | None) -> bool: def should_prepare_skill_python_env(package_root: str | None) -> bool:
return box_workspace.should_prepare_python_env(package_root) return should_prepare_python_env(package_root)
def wrap_skill_command_with_python_env( def wrap_skill_command_with_python_env(
@@ -207,7 +235,7 @@ def wrap_skill_command_with_python_env(
mount_path: str = '/workspace', mount_path: str = '/workspace',
state_path: str | None = None, state_path: str | None = None,
) -> str: ) -> str:
return box_workspace.wrap_python_command_with_env( return wrap_python_command_with_env(
command, command,
mount_path=mount_path, mount_path=mount_path,
state_path=state_path, state_path=state_path,
@@ -6,7 +6,6 @@ import typing
import langbot_plugin.api.entities.builtin.resource.tool as resource_tool import langbot_plugin.api.entities.builtin.resource.tool as resource_tool
from .. import loader from .. import loader
from .availability import is_box_backend_available
from ....api.http.context import ExecutionContext from ....api.http.context import ExecutionContext
# Align with Claude Code's Skill tool design: # Align with Claude Code's Skill tool design:
@@ -15,12 +14,23 @@ from ....api.http.context import ExecutionContext
# - This protects KV Cache and follows industry standard # - This protects KV Cache and follows industry standard
ACTIVATE_SKILL_TOOL_NAME = 'activate' ACTIVATE_SKILL_TOOL_NAME = 'activate'
LIST_SKILL_RESOURCES_TOOL_NAME = 'list_skill_resources'
READ_SKILL_RESOURCE_TOOL_NAME = 'read_skill_resource'
REGISTER_SKILL_TOOL_NAME = 'register_skill' REGISTER_SKILL_TOOL_NAME = 'register_skill'
SKILL_TOOL_NAMES = { READ_ONLY_SKILL_TOOL_NAMES = {
ACTIVATE_SKILL_TOOL_NAME, ACTIVATE_SKILL_TOOL_NAME,
LIST_SKILL_RESOURCES_TOOL_NAME,
READ_SKILL_RESOURCE_TOOL_NAME,
}
SANDBOX_SKILL_TOOL_NAMES = {
REGISTER_SKILL_TOOL_NAME, REGISTER_SKILL_TOOL_NAME,
} }
SKILL_TOOL_NAMES = READ_ONLY_SKILL_TOOL_NAMES | SANDBOX_SKILL_TOOL_NAMES
_SKILL_EXECUTION_AVAILABLE_KEY = '_skill_execution_available'
_SKILL_RESOURCE_BYTES_READ_KEY = '_skill_resource_bytes_read'
_MAX_SKILL_RESOURCE_FILE_BYTES = 256 * 1024
_MAX_SKILL_RESOURCE_RUN_BYTES = 1024 * 1024
class SkillToolLoader(loader.ToolLoader): class SkillToolLoader(loader.ToolLoader):
@@ -28,62 +38,73 @@ class SkillToolLoader(loader.ToolLoader):
def __init__(self, ap): def __init__(self, ap):
super().__init__(ap) super().__init__(ap)
self._tools: list[resource_tool.LLMTool] = [] self._read_only_tools: list[resource_tool.LLMTool] = []
self._sandbox_available: bool = False self._sandbox_tools: list[resource_tool.LLMTool] = []
async def initialize(self): async def initialize(self):
# Check if sandbox backend is available (same check as native tools) if self._is_available():
self._sandbox_available = await self._check_sandbox_available() self._read_only_tools = [
if self._sandbox_available:
self._tools = [
self._build_activate_skill_tool(), self._build_activate_skill_tool(),
self._build_register_skill_tool(), self._build_list_skill_resources_tool(),
self._build_read_skill_resource_tool(),
] ]
self._sandbox_tools = [self._build_register_skill_tool()]
else: else:
self.ap.logger.info( self.ap.logger.info('Skill tools are unavailable because the Core SkillRepository is not initialized.')
'Skill tools (activate/register_skill) are NOT available. '
'No sandbox backend (Docker/nsjail/E2B) is ready. '
'Trusted local development may explicitly select box.backend=host.'
)
async def _check_sandbox_available(self) -> bool: async def get_tools(
"""Check if the box backend is truly available (not just the runtime).""" self,
return await is_box_backend_available(self.ap) bound_plugins: list[str] | None = None,
*,
async def get_tools(self, bound_plugins: list[str] | None = None) -> list[resource_tool.LLMTool]: sandbox_available: bool | None = None,
if not await self._is_available(): ) -> list[resource_tool.LLMTool]:
if not self._is_available():
return [] return []
if not self._tools: if not self._read_only_tools:
self._tools = [ await self.initialize()
self._build_activate_skill_tool(), tools = list(self._read_only_tools)
self._build_register_skill_tool(), if sandbox_available:
] tools.extend(self._sandbox_tools)
return list(self._tools) return tools
async def has_tool(self, name: str) -> bool: async def get_tool(self, name: str, *, sandbox_available: bool | None = None):
return await self._is_available() and name in SKILL_TOOL_NAMES for tool in await self.get_tools(sandbox_available=sandbox_available):
if tool.name == name:
return tool
return None
async def _is_available(self) -> bool: async def has_tool(self, name: str, *, sandbox_available: bool | None = None) -> bool:
"""Check if skill tools should be available. if not self._is_available() or name not in SKILL_TOOL_NAMES:
Skill tools require both a skill manager and a sandbox backend.
"""
if not self._has_skill_manager():
return False return False
self._sandbox_available = await self._check_sandbox_available() return name in READ_ONLY_SKILL_TOOL_NAMES or bool(sandbox_available)
return self._sandbox_available
@staticmethod
def is_sandbox_tool(name: str) -> bool:
return name in SANDBOX_SKILL_TOOL_NAMES
@staticmethod
def recognizes_tool(name: str) -> bool:
return name in SKILL_TOOL_NAMES
def _is_available(self) -> bool:
return self._has_skill_manager() and getattr(self.ap, 'skill_repository', None) is not None
async def invoke_tool(self, name: str, parameters: dict, query) -> typing.Any: async def invoke_tool(self, name: str, parameters: dict, query) -> typing.Any:
require_sandbox = getattr(
getattr(self.ap, 'box_service', None),
'require_workspace_sandbox',
None,
)
if callable(require_sandbox):
await require_sandbox(self._execution_context(query))
if name == ACTIVATE_SKILL_TOOL_NAME: if name == ACTIVATE_SKILL_TOOL_NAME:
return await self._invoke_activate_skill(parameters, query) return await self._invoke_activate_skill(parameters, query)
if name == LIST_SKILL_RESOURCES_TOOL_NAME:
return await self._invoke_list_skill_resources(parameters, query)
if name == READ_SKILL_RESOURCE_TOOL_NAME:
return await self._invoke_read_skill_resource(parameters, query)
if name == REGISTER_SKILL_TOOL_NAME: if name == REGISTER_SKILL_TOOL_NAME:
require_sandbox = getattr(
getattr(self.ap, 'box_service', None),
'require_workspace_sandbox',
None,
)
if not callable(require_sandbox):
return self._sandbox_unavailable_result(name)
await require_sandbox(self._execution_context(query))
return await self._invoke_register_skill(parameters, query) return await self._invoke_register_skill(parameters, query)
raise ValueError(f'Unknown skill tool: {name}') raise ValueError(f'Unknown skill tool: {name}')
@@ -108,6 +129,27 @@ class SkillToolLoader(loader.ToolLoader):
def _has_skill_manager(self) -> bool: def _has_skill_manager(self) -> bool:
return getattr(self.ap, 'skill_mgr', None) is not None return getattr(self.ap, 'skill_mgr', None) is not None
@staticmethod
def _sandbox_unavailable_result(name: str) -> dict:
return {
'ok': False,
'code': 'sandbox_unavailable',
'tool': name,
'message': 'This operation requires Box execution, but Box is not configured or available.',
}
async def _execution_available(self, query) -> bool:
variables = getattr(query, 'variables', None)
if isinstance(variables, dict) and _SKILL_EXECUTION_AVAILABLE_KEY in variables:
return bool(variables[_SKILL_EXECUTION_AVAILABLE_KEY])
checker = getattr(getattr(self.ap, 'box_service', None), 'is_workspace_sandbox_available', None)
if not callable(checker):
return False
try:
return bool(await checker(self._execution_context(query)))
except Exception:
return False
async def _invoke_activate_skill(self, parameters: dict, query) -> typing.Any: async def _invoke_activate_skill(self, parameters: dict, query) -> typing.Any:
"""Activate a skill and return SKILL.md content via Tool Result.""" """Activate a skill and return SKILL.md content via Tool Result."""
skill_name = str(parameters.get('skill_name', '') or '').strip() skill_name = str(parameters.get('skill_name', '') or '').strip()
@@ -116,42 +158,111 @@ class SkillToolLoader(loader.ToolLoader):
from . import skill as skill_loader from . import skill as skill_loader
skill_data = skill_loader.get_visible_skill(self.ap, query, skill_name) visible_skill = skill_loader.get_visible_skill(self.ap, query, skill_name)
if skill_data is None: if visible_skill is None:
visible_skills = skill_loader.get_visible_skills(self.ap, query) visible_skills = skill_loader.get_visible_skills(self.ap, query)
available_names = ', '.join(sorted(visible_skills.keys())) or 'none' available_names = ', '.join(sorted(visible_skills.keys())) or 'none'
raise ValueError(f'Skill "{skill_name}" not found. Available skills: {available_names}') raise ValueError(f'Skill "{skill_name}" not found. Available skills: {available_names}')
# Register activated skill for sandbox mount path resolution skill_data = await self.ap.skill_repository.get_skill(
self._execution_context(query),
skill_name,
snapshot=True,
)
if skill_data is None:
raise ValueError(f'Skill "{skill_name}" is no longer available; reload the skill catalog.')
skill_loader.register_activated_skill(query, skill_data) skill_loader.register_activated_skill(query, skill_data)
# Return SKILL.md content as Tool Result (injects into context)
instructions = skill_data.get('instructions', '') instructions = skill_data.get('instructions', '')
package_root = skill_data.get('package_root', '') revision = str(skill_data.get('revision', '') or '')
mount_path = skill_loader.get_virtual_skill_mount_path(skill_name) execution_available = await self._execution_available(query)
mount_path = skill_loader.get_virtual_skill_mount_path(skill_name) if execution_available else None
# Build Tool Result content
result_content = f'<command-message>The "{skill_name}" skill is activated</command-message>\n' result_content = f'<command-message>The "{skill_name}" skill is activated</command-message>\n'
result_content += '<skill-activation>\n' result_content += '<skill-activation>\n'
result_content += f'<skill-name>{skill_name}</skill-name>\n' result_content += f'<skill-name>{skill_name}</skill-name>\n'
result_content += f'<mount-path>{mount_path}</mount-path>\n' result_content += f'<revision>{revision}</revision>\n'
result_content += f'<package-root>{package_root}</package-root>\n' result_content += '<resources-readable>true</resources-readable>\n'
result_content += f'<execution-available>{str(execution_available).lower()}</execution-available>\n'
result_content += f'\n## Instructions\n{instructions}\n' result_content += f'\n## Instructions\n{instructions}\n'
result_content += '\n## Runtime Context\n' result_content += '\n## Runtime Context\n'
result_content += f'The skill package is mounted at {mount_path}. Use the standard tools to interact with it:\n' result_content += '- Use `list_skill_resources` and `read_skill_resource` for read-only package resources.\n'
result_content += f'- Use `read` to inspect files under {mount_path}\n' if execution_available:
result_content += f'- Use `exec` with workdir set to {mount_path} to run commands in that package\n' result_content += (
result_content += '- Use `write` and `edit` on that path when the instructions require updating files\n' f'- Box execution is available; executable package files will be mounted at {mount_path}.\n'
)
else:
result_content += (
'- Box execution is unavailable. Do not attempt to run scripts or modify Workspace files.\n'
)
result_content += '</skill-activation>\n' result_content += '</skill-activation>\n'
return { return {
'activated': True, 'activated': True,
'skill_name': skill_name, 'skill_name': skill_name,
'mount_path': mount_path, 'mount_path': mount_path,
'revision': revision,
'capabilities': {
'instructions_readable': True,
'resources_readable': True,
'execution_available': execution_available,
},
'activated_skill_names': skill_loader.get_activated_skill_names(query), 'activated_skill_names': skill_loader.get_activated_skill_names(query),
'content': result_content, 'content': result_content,
} }
@staticmethod
def _activated_skill(parameters: dict, query) -> dict:
from . import skill as skill_loader
skill_name = str(parameters.get('skill_name', '') or '').strip()
if not skill_name:
raise ValueError('skill_name is required')
skill_data = skill_loader.get_activated_skill(query, skill_name)
if skill_data is None:
raise ValueError(f'Skill "{skill_name}" must be activated before its resources can be read.')
requested_revision = str(parameters.get('revision', '') or '').strip()
activated_revision = str(skill_data.get('revision', '') or '').strip()
if requested_revision and requested_revision != activated_revision:
raise ValueError('revision must match the activated skill revision')
return skill_data
async def _invoke_list_skill_resources(self, parameters: dict, query) -> dict:
skill_data = self._activated_skill(parameters, query)
return await self.ap.skill_repository.list_skill_resources(
self._execution_context(query),
skill_data['name'],
str(parameters.get('path', '.') or '.'),
expected_revision=skill_data.get('revision'),
)
async def _invoke_read_skill_resource(self, parameters: dict, query) -> dict:
skill_data = self._activated_skill(parameters, query)
path = str(parameters.get('path', '') or '').strip()
if not path:
raise ValueError('path is required')
result = await self.ap.skill_repository.read_skill_resource(
self._execution_context(query),
skill_data['name'],
path,
expected_revision=skill_data.get('revision'),
)
content = str(result.get('content', ''))
size = len(content.encode('utf-8'))
if size > _MAX_SKILL_RESOURCE_FILE_BYTES:
raise ValueError('Skill resource exceeds the per-file read limit')
variables = getattr(query, 'variables', None)
if not isinstance(variables, dict):
variables = {}
query.variables = variables
total = int(variables.get(_SKILL_RESOURCE_BYTES_READ_KEY, 0) or 0) + size
if total > _MAX_SKILL_RESOURCE_RUN_BYTES:
raise ValueError('Skill resource reads exceed the per-run limit')
variables[_SKILL_RESOURCE_BYTES_READ_KEY] = total
result['size'] = size
return result
async def _invoke_register_skill(self, parameters: dict, query) -> typing.Any: async def _invoke_register_skill(self, parameters: dict, query) -> typing.Any:
"""Register a skill from sandbox directory to data/skills/.""" """Register a skill from sandbox directory to data/skills/."""
sandbox_path = str(parameters.get('path', '') or '').strip() sandbox_path = str(parameters.get('path', '') or '').strip()
@@ -176,14 +287,14 @@ class SkillToolLoader(loader.ToolLoader):
raise ValueError('skill name is required') raise ValueError('skill name is required')
# Create the skill # Create the skill
created = await skill_service.create_skill( created = await skill_service.import_skill_directory(
execution_context, execution_context,
host_path,
{ {
'name': skill_name, 'name': skill_name,
'display_name': str(parameters.get('display_name') or scanned.get('display_name', '')).strip(), 'display_name': str(parameters.get('display_name') or scanned.get('display_name', '')).strip(),
'description': str(parameters.get('description') or scanned.get('description', '')).strip(), 'description': str(parameters.get('description') or scanned.get('description', '')).strip(),
'instructions': str(parameters.get('instructions') or scanned.get('instructions', '')), 'instructions': str(parameters.get('instructions') or scanned.get('instructions', '')),
'package_root': host_path,
}, },
) )
@@ -249,6 +360,42 @@ class SkillToolLoader(loader.ToolLoader):
func=lambda parameters: parameters, func=lambda parameters: parameters,
) )
def _build_list_skill_resources_tool(self) -> resource_tool.LLMTool:
return resource_tool.LLMTool(
name=LIST_SKILL_RESOURCES_TOOL_NAME,
human_desc='List activated skill resources',
description='List read-only files in an activated skill package without starting a sandbox.',
parameters={
'type': 'object',
'properties': {
'skill_name': {'type': 'string', 'description': 'The activated skill name.'},
'path': {'type': 'string', 'description': 'Relative directory path. Defaults to the package root.'},
'revision': {'type': 'string', 'description': 'Optional revision returned by activate.'},
},
'required': ['skill_name'],
'additionalProperties': False,
},
func=lambda parameters: parameters,
)
def _build_read_skill_resource_tool(self) -> resource_tool.LLMTool:
return resource_tool.LLMTool(
name=READ_SKILL_RESOURCE_TOOL_NAME,
human_desc='Read an activated skill resource',
description='Read a UTF-8 text resource from an activated skill package without starting a sandbox.',
parameters={
'type': 'object',
'properties': {
'skill_name': {'type': 'string', 'description': 'The activated skill name.'},
'path': {'type': 'string', 'description': 'File path relative to the skill package root.'},
'revision': {'type': 'string', 'description': 'Optional revision returned by activate.'},
},
'required': ['skill_name', 'path'],
'additionalProperties': False,
},
func=lambda parameters: parameters,
)
def _build_register_skill_tool(self) -> resource_tool.LLMTool: def _build_register_skill_tool(self) -> resource_tool.LLMTool:
return resource_tool.LLMTool( return resource_tool.LLMTool(
name=REGISTER_SKILL_TOOL_NAME, name=REGISTER_SKILL_TOOL_NAME,
+31 -9
View File
@@ -66,6 +66,15 @@ class ToolManager:
except Exception: except Exception:
return False return False
@staticmethod
def _sandbox_unavailable_result(name: str) -> dict[str, typing.Any]:
return {
'ok': False,
'code': 'sandbox_unavailable',
'tool': name,
'message': 'This operation requires Box execution, but Box is not configured or available.',
}
async def initialize(self): async def initialize(self):
from langbot.pkg.utils import importutil from langbot.pkg.utils import importutil
from langbot.pkg.provider.tools import loaders from langbot.pkg.provider.tools import loaders
@@ -102,8 +111,8 @@ class ToolManager:
sandbox_available = await self._workspace_sandbox_available(context) sandbox_available = await self._workspace_sandbox_available(context)
if sandbox_available: if sandbox_available:
all_functions.extend(await self.native_tool_loader.get_tools()) all_functions.extend(await self.native_tool_loader.get_tools())
if include_skill_authoring and sandbox_available: if include_skill_authoring:
all_functions.extend(await self.skill_tool_loader.get_tools()) all_functions.extend(await self.skill_tool_loader.get_tools(sandbox_available=sandbox_available))
all_functions.extend(await self.plugin_tool_loader.get_tools(bound_plugins)) all_functions.extend(await self.plugin_tool_loader.get_tools(bound_plugins))
all_functions.extend( all_functions.extend(
await self.mcp_tool_loader.get_tools( await self.mcp_tool_loader.get_tools(
@@ -142,8 +151,12 @@ class ToolManager:
sandbox_available = await self._workspace_sandbox_available(context) sandbox_available = await self._workspace_sandbox_available(context)
if sandbox_available: if sandbox_available:
append_tools('builtin', 'LangBot', await self.native_tool_loader.get_tools()) append_tools('builtin', 'LangBot', await self.native_tool_loader.get_tools())
if include_skill_authoring and sandbox_available: if include_skill_authoring:
append_tools('skill', 'LangBot', await self.skill_tool_loader.get_tools()) append_tools(
'skill',
'LangBot',
await self.skill_tool_loader.get_tools(sandbox_available=sandbox_available),
)
catalog.extend(await self.plugin_tool_loader.get_tool_catalog(bound_plugins)) catalog.extend(await self.plugin_tool_loader.get_tool_catalog(bound_plugins))
if self.mcp_tool_loader: if self.mcp_tool_loader:
@@ -168,10 +181,9 @@ class ToolManager:
tool = await active_loader.get_tool(name) tool = await active_loader.get_tool(name)
if tool: if tool:
return tool return tool
if sandbox_available: tool = await self.skill_tool_loader.get_tool(name, sandbox_available=sandbox_available)
tool = await self.skill_tool_loader.get_tool(name) if tool:
if tool: return tool
return tool
return await self.mcp_tool_loader.get_tool(context, name) return await self.mcp_tool_loader.get_tool(context, name)
@@ -310,7 +322,10 @@ class ToolManager:
query=query, query=query,
invoke=lambda: self.mcp_tool_loader.invoke_tool(name, parameters, query), invoke=lambda: self.mcp_tool_loader.invoke_tool(name, parameters, query),
) )
if sandbox_available and await self.skill_tool_loader.has_tool(name): if await self.skill_tool_loader.has_tool(name, sandbox_available=sandbox_available):
variables = getattr(query, 'variables', None)
if isinstance(variables, dict):
variables['_skill_execution_available'] = sandbox_available
telemetry_features.increment(query, 'tool_calls', 'skill') telemetry_features.increment(query, 'tool_calls', 'skill')
return await self._invoke_tool_with_monitoring( return await self._invoke_tool_with_monitoring(
source='skill', source='skill',
@@ -319,6 +334,13 @@ class ToolManager:
query=query, query=query,
invoke=lambda: self.skill_tool_loader.invoke_tool(name, parameters, query), invoke=lambda: self.skill_tool_loader.invoke_tool(name, parameters, query),
) )
recognizes_native = getattr(self.native_tool_loader, 'recognizes_tool', None)
is_sandbox_skill_tool = getattr(self.skill_tool_loader, 'is_sandbox_tool', None)
if not sandbox_available and (
(callable(recognizes_native) and recognizes_native(name) is True)
or (callable(is_sandbox_skill_tool) and is_sandbox_skill_tool(name) is True)
):
return self._sandbox_unavailable_result(name)
raise ToolNotFoundError(name) raise ToolNotFoundError(name)
async def shutdown(self): async def shutdown(self):
+3 -1
View File
@@ -1,3 +1,5 @@
from .manager import SkillManager from .manager import SkillManager
from .repository import SkillRepository
__all__ = ['SkillManager']
__all__ = ['SkillManager', 'SkillRepository']
+6 -21
View File
@@ -1,14 +1,12 @@
from __future__ import annotations from __future__ import annotations
import os
from ..api.http.context import ExecutionContext from ..api.http.context import ExecutionContext
from ..api.http.service.tenant import TenantContext, require_workspace_uuid from ..api.http.service.tenant import TenantContext, require_workspace_uuid
from ..core import app from ..core import app
class SkillManager: class SkillManager:
"""Workspace-scoped in-memory view of Box-managed skill packages.""" """Workspace-scoped in-memory view of Core-managed skill packages."""
ap: app.Application ap: app.Application
@@ -63,33 +61,20 @@ class SkillManager:
self._skills_by_scope.pop(existing_key, None) self._skills_by_scope.pop(existing_key, None)
self._skills_by_scope[key] = {} self._skills_by_scope[key] = {}
box_service = getattr(self.ap, 'box_service', None) repository = getattr(self.ap, 'skill_repository', None)
if box_service is None or not getattr(box_service, 'available', False): if repository is None:
self.ap.logger.info( self.ap.logger.info('Skill repository unavailable; skill cache will remain empty.')
f'Box runtime unavailable; skill cache is empty for Workspace {execution_context.workspace_uuid}.'
)
return return
validate_locally = bool(getattr(box_service, 'shares_filesystem_with_box', False))
try: try:
dropped = 0
skills: dict[str, dict] = {} skills: dict[str, dict] = {}
for skill_data in await box_service.list_skills(execution_context): for skill_data in await repository.list_skills(execution_context):
skill_name = skill_data.get('name') skill_name = skill_data.get('name')
if not skill_name: if not skill_name:
continue continue
package_root = str(skill_data.get('package_root', '') or '').strip()
if validate_locally and package_root and not os.path.isdir(package_root):
self.ap.logger.warning(
f'Skill "{skill_name}" reported by Box runtime but package_root '
f'missing on LangBot filesystem ({package_root}); dropping from cache.'
)
dropped += 1
continue
skills[skill_name] = skill_data skills[skill_name] = skill_data
self._skills_by_scope[key] = skills self._skills_by_scope[key] = skills
suffix = f' ({dropped} dropped due to missing package_root)' if dropped else '' self.ap.logger.info(f'Loaded {len(skills)} skills for Workspace {execution_context.workspace_uuid}')
self.ap.logger.info(f'Loaded {len(skills)} skills for Workspace {execution_context.workspace_uuid}{suffix}')
except Exception as exc: except Exception as exc:
self.ap.logger.warning(f'Failed to load skills for Workspace {execution_context.workspace_uuid}: {exc}') self.ap.logger.warning(f'Failed to load skills for Workspace {execution_context.workspace_uuid}: {exc}')
+207
View File
@@ -0,0 +1,207 @@
from __future__ import annotations
import asyncio
import os
import weakref
from langbot_plugin.skill_store import (
SkillRevisionMismatchError,
SkillStore,
skill_namespace,
)
from ..api.http.context import ExecutionContext
from ..api.http.service.tenant import TenantContext, require_workspace_uuid
from ..utils.bounded_executor import blocking_work_scope, run_blocking_atomic
class SkillRepository:
"""Async, Workspace-scoped adapter around the SDK SkillStore."""
def __init__(self, ap) -> None:
self.ap = ap
config = getattr(getattr(ap, 'instance_config', None), 'data', {}) or {}
self._local_config = (config.get('box') or {}).get('local') or {}
self._skills_config = config.get('skills') or {}
self._store = SkillStore(self._skills_root())
self._locks: weakref.WeakValueDictionary[str, asyncio.Lock] = weakref.WeakValueDictionary()
def _workspace_lock(self, namespace: str) -> asyncio.Lock:
lock = self._locks.get(namespace)
if lock is None:
lock = asyncio.Lock()
self._locks[namespace] = lock
return lock
def _host_root(self) -> str:
configured = str(self._local_config.get('host_root') or './data/box').strip()
return os.path.realpath(os.path.abspath(os.path.expanduser(configured)))
def _skills_root(self) -> str:
configured = str(self._skills_config.get('root') or '').strip()
if not configured:
# Online-upgrade bridge for installations whose persisted config
# predates the standalone Skill domain.
# TODO(next-major): remove box.local.skills_root fallback.
legacy = str(self._local_config.get('skills_root') or '').strip()
configured = legacy or 'skills'
if not os.path.isabs(configured):
configured = os.path.join(self._host_root(), configured)
return os.path.realpath(os.path.abspath(os.path.expanduser(configured)))
def _default_workspace(self) -> str:
configured = str(self._local_config.get('default_workspace') or '').strip()
if not configured:
configured = os.path.join(self._host_root(), 'default')
elif not os.path.isabs(configured):
configured = os.path.join(self._host_root(), configured)
return os.path.realpath(os.path.abspath(os.path.expanduser(configured)))
@staticmethod
def _execution_context(context: TenantContext) -> ExecutionContext:
workspace_uuid = require_workspace_uuid(context)
instance_uuid = str(getattr(context, 'instance_uuid', '') or '').strip()
generation = getattr(context, 'placement_generation', None)
if not instance_uuid:
raise ValueError('Skill operations require an explicit instance UUID')
if isinstance(generation, bool) or not isinstance(generation, int) or generation <= 0:
raise ValueError('Skill operations require a positive placement generation')
return ExecutionContext(
instance_uuid=instance_uuid,
workspace_uuid=workspace_uuid,
placement_generation=generation,
bot_uuid=getattr(context, 'bot_uuid', None),
pipeline_uuid=getattr(context, 'pipeline_uuid', None),
query_uuid=getattr(context, 'query_uuid', None),
entitlement_revision=getattr(context, 'entitlement_revision', 0),
)
@classmethod
def _namespace(cls, context: TenantContext) -> str:
execution_context = cls._execution_context(context)
return skill_namespace(
execution_context.instance_uuid,
execution_context.workspace_uuid,
)
async def _validated_execution_context(self, context: TenantContext) -> ExecutionContext:
execution_context = self._execution_context(context)
binding = await self.ap.workspace_service.get_execution_binding(
execution_context.workspace_uuid,
expected_generation=execution_context.placement_generation,
)
if (
binding.instance_uuid != execution_context.instance_uuid
or str(getattr(binding, 'workspace_uuid', '') or '') != execution_context.workspace_uuid
or getattr(binding, 'placement_generation', None) != execution_context.placement_generation
):
raise ValueError('Skill execution context belongs to a stale Workspace placement')
return execution_context
def _workspace_root(self, namespace: str) -> str:
return os.path.join(self._default_workspace(), 'tenants', namespace)
async def _call(self, context: TenantContext, method_name: str, *args, **kwargs):
execution_context = await self._validated_execution_context(context)
namespace = self._namespace(execution_context)
def invoke():
method = getattr(self._store.scoped(namespace), method_name)
return method(*args, **kwargs)
async with self._workspace_lock(namespace):
with blocking_work_scope(f'skill:{namespace}'):
return await run_blocking_atomic(invoke)
async def list_skills(self, context: TenantContext) -> list[dict]:
return await self._call(context, 'list_skills')
async def get_skill(self, context: TenantContext, name: str, *, snapshot: bool = False) -> dict | None:
return await self._call(context, 'get_skill_snapshot' if snapshot else 'get_skill', name)
async def create_skill(self, context: TenantContext, skill: dict) -> dict:
return await self._call(context, 'create_skill', skill)
async def import_skill_directory(self, context: TenantContext, path: str, skill: dict) -> dict:
namespace = self._namespace(context)
return await self._call(
context,
'import_skill_directory',
path,
skill,
source_root=self._workspace_root(namespace),
)
async def update_skill(self, context: TenantContext, name: str, skill: dict) -> dict:
return await self._call(context, 'update_skill', name, skill)
async def delete_skill(self, context: TenantContext, name: str) -> None:
await self._call(context, 'delete_skill', name)
async def scan_skill_directory(self, context: TenantContext, path: str) -> dict:
namespace = self._namespace(context)
return await self._call(
context,
'scan_import_directory',
path,
source_root=self._workspace_root(namespace),
)
async def list_skill_files(
self,
context: TenantContext,
name: str,
path: str = '.',
include_hidden: bool = False,
max_entries: int = 200,
) -> dict:
return await self._call(context, 'list_skill_files', name, path, include_hidden, max_entries)
async def read_skill_file(self, context: TenantContext, name: str, path: str) -> dict:
return await self._call(context, 'read_skill_file', name, path)
async def list_skill_resources(
self,
context: TenantContext,
name: str,
path: str = '.',
*,
expected_revision: str | None = None,
) -> dict:
return await self._call(
context,
'list_skill_resources',
name,
path,
False,
200,
expected_revision=expected_revision,
)
async def read_skill_resource(
self,
context: TenantContext,
name: str,
path: str,
*,
expected_revision: str | None = None,
) -> dict:
return await self._call(
context,
'read_skill_resource',
name,
path,
expected_revision=expected_revision,
)
async def write_skill_file(self, context: TenantContext, name: str, path: str, content: str) -> dict:
return await self._call(context, 'write_skill_file', name, path, content)
async def preview_skill_zip(self, context: TenantContext, file_bytes: bytes, filename: str, **kwargs) -> list[dict]:
return await self._call(context, 'preview_zip_upload', file_bytes=file_bytes, filename=filename, **kwargs)
async def install_skill_zip(self, context: TenantContext, file_bytes: bytes, filename: str, **kwargs) -> list[dict]:
return await self._call(context, 'install_zip_upload', file_bytes=file_bytes, filename=filename, **kwargs)
__all__ = ['SkillRepository', 'SkillRevisionMismatchError']
+1 -1
View File
@@ -198,7 +198,7 @@ async def build_heartbeat_payload(
except Exception: except Exception:
features['plugin_count'] = -1 features['plugin_count'] = -1
# Skill count (from Box runtime via skill manager) # Skill count (from the Core SkillRepository cache)
try: try:
skill_mgr = getattr(ap, 'skill_mgr', None) skill_mgr = getattr(ap, 'skill_mgr', None)
if skill_mgr is not None: if skill_mgr is not None:
+191
View File
@@ -0,0 +1,191 @@
"""Python project detection and sandbox-local environment bootstrap helpers."""
from __future__ import annotations
import os
import textwrap
PYTHON_MANIFEST_FILES = (
'requirements.txt',
'pyproject.toml',
'setup.py',
'setup.cfg',
)
def normalize_host_path(path: str | None) -> str:
if path is None:
return ''
stripped = str(path).strip()
if not stripped:
return ''
return os.path.realpath(os.path.abspath(stripped))
def list_python_manifest_files(host_path: str | None) -> list[str]:
normalized_root = normalize_host_path(host_path)
if not normalized_root:
return []
return [filename for filename in PYTHON_MANIFEST_FILES if os.path.isfile(os.path.join(normalized_root, filename))]
def should_prepare_python_env(host_path: str | None) -> bool:
normalized_root = normalize_host_path(host_path)
if not normalized_root:
return False
if os.path.isdir(os.path.join(normalized_root, '.venv')):
return True
return bool(list_python_manifest_files(normalized_root))
def wrap_python_command_with_env(
command: str,
*,
mount_path: str = '/workspace',
state_path: str | None = None,
) -> str:
"""Wrap a command with a reusable sandbox-local Python env bootstrap."""
writable_state_path = state_path or mount_path
bootstrap = textwrap.dedent(
f"""
set -e
_LB_VENV_DIR="{writable_state_path}/.venv"
_LB_META_DIR="{writable_state_path}/.langbot"
_LB_META_FILE="$_LB_META_DIR/python-env.json"
_LB_LOCK_DIR="$_LB_META_DIR/python-env.lock"
_LB_TMP_DIR="{writable_state_path}/.tmp"
_LB_PIP_CACHE_DIR="{writable_state_path}/.cache/pip"
mkdir -p "$_LB_META_DIR" "$_LB_TMP_DIR" "$_LB_PIP_CACHE_DIR"
_LB_SYSTEM_PYTHON="$(command -v python3 || command -v python || true)"
if [ -z "$_LB_SYSTEM_PYTHON" ]; then
echo "python3 or python is required to prepare the workspace Python environment" >&2
exit 127
fi
export TMPDIR="$_LB_TMP_DIR"
export TEMP="$_LB_TMP_DIR"
export TMP="$_LB_TMP_DIR"
export PIP_CACHE_DIR="$_LB_PIP_CACHE_DIR"
_lb_python_meta() {{
"$_LB_SYSTEM_PYTHON" - <<'PY'
import hashlib
import json
import os
import sys
root = "{mount_path}"
max_manifest_bytes = 10 * 1024 * 1024
digest = hashlib.sha256()
manifest_files = []
for rel in ("requirements.txt", "pyproject.toml", "setup.py", "setup.cfg"):
path = os.path.join(root, rel)
if not os.path.isfile(path):
continue
if os.path.getsize(path) > max_manifest_bytes:
raise RuntimeError(
f"Python project manifest exceeds {{max_manifest_bytes}} bytes: {{rel}}"
)
manifest_files.append(rel)
with open(path, "rb") as handle:
digest.update(rel.encode("utf-8"))
digest.update(b"\0")
while chunk := handle.read(1024 * 1024):
digest.update(chunk)
digest.update(b"\0")
print(
json.dumps(
{{
"python_executable": sys.executable,
"python_version": list(sys.version_info[:3]),
"manifest_files": manifest_files,
"manifest_sha256": digest.hexdigest(),
}},
sort_keys=True,
)
)
PY
}}
_LB_CURRENT_META="$(_lb_python_meta)"
_LB_NEEDS_BOOTSTRAP=0
if [ ! -x "$_LB_VENV_DIR/bin/python" ]; then
_LB_NEEDS_BOOTSTRAP=1
elif [ ! -f "$_LB_META_FILE" ]; then
_LB_NEEDS_BOOTSTRAP=1
elif [ "$(cat "$_LB_META_FILE")" != "$_LB_CURRENT_META" ]; then
_LB_NEEDS_BOOTSTRAP=1
fi
if [ "$_LB_NEEDS_BOOTSTRAP" -eq 1 ]; then
_LB_LOCK_WAIT=0
while ! mkdir "$_LB_LOCK_DIR" 2>/dev/null; do
if [ "$_LB_LOCK_WAIT" -ge 120 ]; then
_LB_LOCK_OWNER="$(cat "$_LB_LOCK_DIR/pid" 2>/dev/null || true)"
if [ -n "$_LB_LOCK_OWNER" ] && kill -0 "$_LB_LOCK_OWNER" 2>/dev/null; then
echo "Timed out waiting for active Python environment lock: $_LB_LOCK_DIR" >&2
exit 1
fi
echo "Timed out waiting for Python environment lock, clearing stale lock: $_LB_LOCK_DIR" >&2
rm -rf "$_LB_LOCK_DIR" 2>/dev/null || true
if mkdir "$_LB_LOCK_DIR" 2>/dev/null; then
break
fi
echo "Timed out waiting for Python environment lock: $_LB_LOCK_DIR" >&2
exit 1
fi
sleep 1
_LB_LOCK_WAIT=$((_LB_LOCK_WAIT + 1))
done
printf '%s\n' "$$" > "$_LB_LOCK_DIR/pid" 2>/dev/null || true
_lb_cleanup_lock() {{
rm -rf "$_LB_LOCK_DIR" >/dev/null 2>&1 || true
}}
trap _lb_cleanup_lock EXIT INT TERM
_LB_CURRENT_META="$(_lb_python_meta)"
_LB_NEEDS_BOOTSTRAP=0
if [ ! -x "$_LB_VENV_DIR/bin/python" ]; then
_LB_NEEDS_BOOTSTRAP=1
elif [ ! -f "$_LB_META_FILE" ]; then
_LB_NEEDS_BOOTSTRAP=1
elif [ "$(cat "$_LB_META_FILE")" != "$_LB_CURRENT_META" ]; then
_LB_NEEDS_BOOTSTRAP=1
fi
if [ "$_LB_NEEDS_BOOTSTRAP" -eq 1 ]; then
rm -rf "$_LB_VENV_DIR"
"$_LB_SYSTEM_PYTHON" -m venv "$_LB_VENV_DIR"
. "$_LB_VENV_DIR/bin/activate"
python -m pip install --upgrade pip setuptools wheel
if [ -f "{mount_path}/requirements.txt" ]; then
python -m pip install -r "{mount_path}/requirements.txt"
elif [ -f "{mount_path}/pyproject.toml" ] || [ -f "{mount_path}/setup.py" ] || [ -f "{mount_path}/setup.cfg" ]; then
python -m pip install "{mount_path}"
fi
printf '%s' "$_LB_CURRENT_META" > "$_LB_META_FILE"
fi
fi
export VIRTUAL_ENV="$_LB_VENV_DIR"
export PATH="$_LB_VENV_DIR/bin:$PATH"
{command}
"""
).strip()
return bootstrap + '\n'
__all__ = [
'PYTHON_MANIFEST_FILES',
'list_python_manifest_files',
'normalize_host_path',
'should_prepare_python_env',
'wrap_python_command_with_env',
]
+11 -5
View File
@@ -323,13 +323,20 @@ monitoring:
# Prevent one large Workspace backlog from monopolizing PostgreSQL. # Prevent one large Workspace backlog from monopolizing PostgreSQL.
# Supports MONITORING__AUTO_CLEANUP__MAX_BATCHES_PER_TABLE_PER_RUN. # Supports MONITORING__AUTO_CLEANUP__MAX_BATCHES_PER_TABLE_PER_RUN.
max_batches_per_table_per_run: 4 max_batches_per_table_per_run: 4
skills:
# Core-owned SkillStore. Skill discovery, activation, resources, CRUD and
# revisions remain available when Box is disabled.
# TODO(next-major): change the fresh-install default to './data/skills'
# after the online-upgrade window for the historical Box path closes.
root: './data/box/skills'
box: box:
# Master switch for the Box sandbox runtime. When false, LangBot does NOT # Master switch for the Box sandbox runtime. When false, LangBot does NOT
# attempt to connect to a remote Box runtime nor start a local stdio Box # attempt to connect to a remote Box runtime nor start a local stdio Box
# subprocess. Disabling Box also disables every feature that depends on it: # subprocess. Disabling Box disables execution-backed features: native
# the native sandbox tools (exec/read/write/edit/glob/grep), the activate # sandbox tools (exec/read/write/edit/glob/grep), agent-side skill
# skill tool, skill add/edit, and stdio-mode MCP servers. Skills can still # registration, and stdio-mode MCP servers. Skill management, activation,
# be listed read-only and http/sse MCP servers continue to work. # and read-only package resources remain available without Box.
enabled: true enabled: true
# 'host' runs commands directly as the Box Runtime user without sandbox # 'host' runs commands directly as the Box Runtime user without sandbox
# isolation. It is never auto-selected and is only for trusted local # isolation. It is never auto-selected and is only for trusted local
@@ -378,7 +385,6 @@ box:
image: '' # Custom local sandbox image. Leave empty to use the profile default. image: '' # Custom local sandbox image. Leave empty to use the profile default.
host_root: './data/box' # Base host directory for local workspace mounts. Docker deployments should override this with an absolute host path. host_root: './data/box' # Base host directory for local workspace mounts. Docker deployments should override this with an absolute host path.
default_workspace: '' # Defaults to '<host_root>/default'. Relative paths are resolved under host_root. default_workspace: '' # Defaults to '<host_root>/default'. Relative paths are resolved under host_root.
skills_root: 'skills' # Box-owned skill package directory. Relative paths are resolved under host_root.
allowed_mount_roots: # Defaults to ['<host_root>'] when left empty. allowed_mount_roots: # Defaults to ['<host_root>'] when left empty.
- './data/box' - './data/box'
- '/tmp' - '/tmp'
-106
View File
@@ -1,106 +0,0 @@
"""Exercise Codex provider wiring through a real LangBot process.
The default run does not contact OpenAI. Set LANGBOT_TEST_CODEX_DEVICE_AUTH=1
to also exercise live device start/pending/cancel, without account sign-in.
OAuth exchange and inference behavior are covered by deterministic tests.
"""
from __future__ import annotations
import os
import time
import pytest
pytestmark = pytest.mark.e2e
def test_codex_provider_disconnected_journey(e2e_client):
credentials = {'user': 'codex-e2e@example.com', 'password': 'codex-local-test-password'}
initialized = e2e_client.post('/api/v1/user/init', json=credentials)
assert initialized.status_code == 200, initialized.text
authenticated = e2e_client.post('/api/v1/user/auth', json=credentials)
assert authenticated.status_code == 200, authenticated.text
headers = {'Authorization': f'Bearer {authenticated.json()["data"]["token"]}'}
bootstrap = e2e_client.get('/api/v1/workspaces/bootstrap', headers=headers)
assert bootstrap.status_code == 200, bootstrap.text
headers['X-Workspace-Id'] = bootstrap.json()['data']['workspaces'][0]['workspace']['uuid']
requesters = e2e_client.get('/api/v1/provider/requesters?type=llm', headers=headers)
assert requesters.status_code == 200, requesters.text
codex = next(item for item in requesters.json()['data']['requesters'] if item['name'] == 'openai-codex')
assert codex['spec']['support_type'] == ['llm']
icon = e2e_client.get('/api/v1/provider/requesters/openai-codex/icon')
assert icon.status_code == 200
assert 'image/' in icon.headers['content-type']
base = '/api/v1/provider/providers'
created = e2e_client.post(
base,
headers=headers,
json={'name': 'Codex E2E', 'requester': 'openai-codex', 'base_url': '', 'api_keys': []},
)
assert created.status_code == 200, created.text
provider_path = f'{base}/{created.json()["data"]["uuid"]}'
try:
provider = e2e_client.get(provider_path, headers=headers)
assert provider.status_code == 200, provider.text
data = provider.json()['data']['provider']
assert data['requester'] == 'openai-codex'
assert data['api_keys'] == []
assert data['base_url'] == 'https://chatgpt.com/backend-api/codex'
assert not {'access_token', 'refresh_token', 'id_token'} & data.keys()
status = e2e_client.get(f'{provider_path}/codex/status', headers=headers)
assert status.status_code == 200, status.text
assert status.json()['data']['connected'] is False
assert status.json()['data']['status'] == 'disconnected'
anonymous = e2e_client.post(f'{provider_path}/codex/device', json={})
assert anonymous.status_code == 401
invalid = e2e_client.put(provider_path, headers=headers, json={'base_url': 'https://example.com'})
assert invalid.status_code == 400, invalid.text
invalid_key = e2e_client.put(provider_path, headers=headers, json={'api_keys': ['not-a-codex-key']})
assert invalid_key.status_code == 400, invalid_key.text
scanned = e2e_client.get(f'{provider_path}/scan-models?type=llm', headers=headers)
assert scanned.status_code == 400, scanned.text
assert 'sign in' in scanned.json()['msg'].lower()
renamed = e2e_client.put(provider_path, headers=headers, json={'name': 'Codex renamed'})
assert renamed.status_code == 200, renamed.text
reread = e2e_client.get(provider_path, headers=headers)
assert reread.json()['data']['provider']['name'] == 'Codex renamed'
disconnected = e2e_client.delete(f'{provider_path}/codex/auth', headers=headers)
assert disconnected.status_code == 200, disconnected.text
# Opt-in smoke contacts real OpenAI device endpoints, but never completes
# account sign-in or prints the one-time code/device credentials.
if os.environ.get('LANGBOT_TEST_CODEX_DEVICE_AUTH') == '1':
started = e2e_client.post(f'{provider_path}/codex/device', headers=headers, json={})
assert started.status_code == 200, started.json().get('msg', 'Device start failed')
attempt = started.json()['data']
assert attempt['verification_uri'] == 'https://auth.openai.com/codex/device'
assert isinstance(attempt['user_code'], str) and attempt['user_code']
assert 0 < attempt['expires_at'] - time.time() <= 900
assert not {'access_token', 'refresh_token', 'device_auth_id'} & attempt.keys()
time.sleep(attempt['interval'])
pending = e2e_client.post(
f'{provider_path}/codex/device/poll',
headers=headers,
json={'authorization_id': attempt['authorization_id']},
)
assert pending.status_code == 200
assert pending.json()['data']['status'] == 'pending'
canceled = e2e_client.delete(f'{provider_path}/codex/device/{attempt["authorization_id"]}', headers=headers)
assert canceled.status_code == 200
expired = e2e_client.post(
f'{provider_path}/codex/device/poll',
headers=headers,
json={'authorization_id': attempt['authorization_id']},
)
assert expired.json()['data']['status'] == 'expired'
finally:
deleted = e2e_client.delete(provider_path, headers=headers)
assert deleted.status_code == 200, deleted.text
assert e2e_client.get(provider_path, headers=headers).status_code == 404
+1 -1
View File
@@ -69,7 +69,7 @@ class LangBotProcess:
# Use coverage.py to collect coverage data # Use coverage.py to collect coverage data
# Set COVERAGE_PROCESS_START to enable coverage in subprocess # Set COVERAGE_PROCESS_START to enable coverage in subprocess
self._coverage_file = self.work_dir / '.coverage.e2e' self._coverage_file = self.work_dir / '.coverage.e2e'
env['COVERAGE_PROCESS_START'] = str(self.work_dir / '.coveragerc') env['COVERAGE_PROCESS_START'] = str(self.project_root / '.coveragerc')
env['COVERAGE_FILE'] = str(self._coverage_file) env['COVERAGE_FILE'] = str(self._coverage_file)
# Create .coveragerc for subprocess # Create .coveragerc for subprocess
@@ -1,85 +0,0 @@
from __future__ import annotations
import logging
from types import SimpleNamespace
from unittest.mock import AsyncMock
import pytest
from quart import Quart
from langbot.pkg.api.http.controller.groups import user as user_module
from langbot.pkg.api.http.controller.groups.user import UserRouterGroup
from langbot.pkg.api.http.service.user import UserService
from langbot.pkg.core.stages.genkeys import GenKeysStage
from langbot.pkg.persistence.mgr import PersistenceManager
from langbot.pkg.utils import constants
from langbot.pkg.workspace.collaboration import WorkspaceCollaborationService
from langbot.pkg.workspace.service import WorkspaceService
pytestmark = [pytest.mark.integration, pytest.mark.asyncio]
async def test_generated_recovery_code_resets_real_sqlite_account(tmp_path, monkeypatch):
"""Exercise generation, reset, and old/new password login without mocked user services."""
monkeypatch.setattr(constants, 'instance_id', 'recovery-journey')
monkeypatch.setattr(user_module, '_reset_password_state', {'window_started_at': 0.0, 'attempts': 0})
monkeypatch.setattr(user_module, 'asyncio', SimpleNamespace(sleep=AsyncMock()))
application = SimpleNamespace(
logger=logging.getLogger('recovery-password-journey'),
instance_config=SimpleNamespace(
data={
'database': {'use': 'sqlite', 'sqlite': {'path': str(tmp_path / 'recovery.db')}},
'system': {
'jwt': {'secret': 'recovery-journey-test-secret-only', 'expire': 3600},
'recovery_key': '',
},
},
dump_config=AsyncMock(),
),
)
await GenKeysStage().run(application)
key = application.instance_config.data['system']['recovery_key']
assert len(key) == 8
assert set(key) <= set('23456789ABCDEFGHJKLMNPQRSTUVWXYZ')
persistence = PersistenceManager(application)
application.persistence_mgr = persistence
try:
await persistence.initialize()
application.workspace_service = WorkspaceService(application, instance_uuid='recovery-journey')
application.workspace_collaboration_service = WorkspaceCollaborationService(
application, application.workspace_service
)
application.user_service = UserService(application)
quart_app = Quart(__name__)
await UserRouterGroup(application, quart_app).initialize()
client = quart_app.test_client()
initial = await client.post(
'/api/v1/user/init', json={'user': 'owner@example.com', 'password': 'OriginalPass1!'}
)
assert initial.status_code == 200
assert (await initial.get_json())['code'] == 0
payload = {'user': 'owner@example.com', 'recovery_key': 'WRONG', 'new_password': 'RecoveredPass1!'}
wrong = await client.post('/api/v1/user/reset-password', json=payload)
assert wrong.status_code == 403
unchanged = await client.post(
'/api/v1/user/auth', json={'user': 'owner@example.com', 'password': 'OriginalPass1!'}
)
assert (await unchanged.get_json())['code'] == 0
reset = await client.post('/api/v1/user/reset-password', json={**payload, 'recovery_key': key})
assert reset.status_code == 200
assert (await reset.get_json())['code'] == 0
old_login = await client.post(
'/api/v1/user/auth', json={'user': 'owner@example.com', 'password': 'OriginalPass1!'}
)
assert (await old_login.get_json())['code'] != 0
new_login = await client.post(
'/api/v1/user/auth', json={'user': 'owner@example.com', 'password': 'RecoveredPass1!'}
)
new_data = await new_login.get_json()
assert new_data['code'] == 0
assert new_data['data']['token']
finally:
await persistence.get_db_engine().dispose()
@@ -1,4 +1,4 @@
"""Skills API behavior when a workspace plan has no managed sandbox.""" """Skills API behavior is independent from managed sandbox entitlement."""
from __future__ import annotations from __future__ import annotations
@@ -9,10 +9,7 @@ import pytest
import quart import quart
from langbot.pkg.api.http.controller.groups.skills import SkillsRouterGroup from langbot.pkg.api.http.controller.groups.skills import SkillsRouterGroup
from langbot.pkg.cloud.entitlements import ( from langbot.pkg.cloud.entitlements import EntitlementUnavailableError
EntitlementFeatureUnavailableError,
EntitlementUnavailableError,
)
pytestmark = pytest.mark.integration pytestmark = pytest.mark.integration
WORKSPACE_UUID = '11111111-1111-4111-8111-111111111111' WORKSPACE_UUID = '11111111-1111-4111-8111-111111111111'
@@ -32,10 +29,7 @@ async def skills_api():
application.user_service.get_authenticated_account = AsyncMock(return_value=account) application.user_service.get_authenticated_account = AsyncMock(return_value=account)
application.workspace_collaboration_service.resolve_account_workspace = AsyncMock(return_value=access) application.workspace_collaboration_service.resolve_account_workspace = AsyncMock(return_value=access)
application.skill_service.list_skills = AsyncMock( application.skill_service.list_skills = AsyncMock(
side_effect=EntitlementFeatureUnavailableError( return_value=[{'name': 'docs-only', 'description': 'No execution required'}]
'managed_sandbox',
entitlement_revision=1,
)
) )
quart_app = quart.Quart(__name__) quart_app = quart.Quart(__name__)
@@ -45,7 +39,7 @@ async def skills_api():
@pytest.mark.asyncio @pytest.mark.asyncio
async def test_list_skills_is_empty_when_plan_has_no_managed_sandbox(skills_api): async def test_list_skills_remains_available_without_managed_sandbox(skills_api):
application, client = skills_api application, client = skills_api
response = await client.get( response = await client.get(
'/api/v1/skills', '/api/v1/skills',
@@ -57,7 +51,7 @@ async def test_list_skills_is_empty_when_plan_has_no_managed_sandbox(skills_api)
assert response.status_code == 200 assert response.status_code == 200
payload = await response.get_json() payload = await response.get_json()
assert payload['data'] == {'skills': []} assert payload['data'] == {'skills': [{'name': 'docs-only', 'description': 'No execution required'}]}
application.skill_service.list_skills.assert_awaited_once() application.skill_service.list_skills.assert_awaited_once()
-260
View File
@@ -1,6 +1,5 @@
from __future__ import annotations from __future__ import annotations
import datetime
import json import json
import logging import logging
from types import SimpleNamespace from types import SimpleNamespace
@@ -23,7 +22,6 @@ from langbot.pkg.api.http.service.apikey import ApiKeyService
from langbot.pkg.api.http.service.user import ControlPlaneDirectoryRequiredError, UserService from langbot.pkg.api.http.service.user import ControlPlaneDirectoryRequiredError, UserService
from langbot.pkg.entity.persistence.base import Base from langbot.pkg.entity.persistence.base import Base
from langbot.pkg.entity.persistence.metadata import WorkspaceMetadata from langbot.pkg.entity.persistence.metadata import WorkspaceMetadata
from langbot.pkg.entity.persistence import apikey
from langbot.pkg.entity.persistence.user import User from langbot.pkg.entity.persistence.user import User
from langbot.pkg.entity.persistence.workspace import ( from langbot.pkg.entity.persistence.workspace import (
Workspace, Workspace,
@@ -442,264 +440,6 @@ async def test_api_key_secret_is_one_time_and_viewer_cannot_manage_keys(workspac
assert (await forbidden.get_json())['code'] == 'permission_denied' assert (await forbidden.get_json())['code'] == 'permission_denied'
async def test_api_key_context_returns_bound_identity_without_workspace_permission(workspace_api):
application, client, _, owner_token = workspace_api
current_response = await client.get('/api/v1/workspaces/current', headers=_auth(owner_token))
workspace_uuid = (await current_response.get_json())['data']['workspace']['uuid']
create_response = await client.post(
'/api/v1/apikeys',
headers=_auth(owner_token, workspace_uuid),
json={'name': 'Context probe', 'scopes': []},
)
assert create_response.status_code == 200
created = (await create_response.get_json())['data']['key']
missing_auth = await client.get('/api/v1/system/context')
assert missing_auth.status_code == 401
invalid_auth = await client.get(
'/api/v1/system/context',
headers={'X-API-Key': 'lbk_invalid'},
)
assert invalid_auth.status_code == 401
invalid_capabilities = await client.get(
'/api/v1/system/capabilities',
headers={'X-API-Key': 'lbk_invalid'},
)
assert invalid_capabilities.status_code == 401
response = await client.get(
'/api/v1/system/context',
headers={
'X-API-Key': created['key'],
'X-Workspace-Id': 'caller-selected-workspace-must-be-ignored',
},
)
assert response.status_code == 200
assert (await response.get_json())['data'] == {
'instance_uuid': application.workspace_service.instance_uuid,
'workspace_uuid': workspace_uuid,
'api_key_id': created['uuid'],
'permissions': [],
}
capabilities_response = await client.get(
'/api/v1/system/capabilities',
headers={
'X-API-Key': created['key'],
'X-Workspace-Id': 'caller-selected-workspace-must-be-ignored',
},
)
assert capabilities_response.status_code == 200
capabilities = (await capabilities_response.get_json())['data']
assert capabilities['schema_version'] == 1
assert sorted(capabilities['operations']) == sorted(
[
'bot.list',
'bot.get',
'bot.create',
'bot.update',
'bot.delete',
'pipeline.list',
'pipeline.get',
'pipeline.create',
'pipeline.update',
'pipeline.delete',
'pipeline.copy',
'task.list',
'task.get',
'knowledge_base.list',
'knowledge_base.get',
'knowledge_base.create',
'knowledge_base.update',
'knowledge_base.delete',
'knowledge_base.file.list',
'knowledge_base.file.store',
'knowledge_base.file.delete',
'knowledge_base.retrieve',
'file.document.upload',
'plugin.install.github',
'plugin.install.marketplace',
'plugin.install.local',
'plugin.upgrade',
'plugin.get',
'plugin.list',
'plugin.config.get',
'plugin.config.update',
'plugin.logs',
'plugin.delete',
'provider.list',
'provider.get',
'provider.create',
'provider.update',
'provider.delete',
'provider.scan_models',
'model.llm.list',
'model.llm.get',
'model.llm.create',
'model.llm.update',
'model.llm.delete',
'model.llm.test',
'model.embedding.list',
'model.embedding.get',
'model.embedding.create',
'model.embedding.update',
'model.embedding.delete',
'model.embedding.test',
'model.rerank.list',
'model.rerank.get',
'model.rerank.create',
'model.rerank.update',
'model.rerank.delete',
'model.rerank.test',
'skill.list',
'skill.get',
'skill.create',
'skill.update',
'skill.delete',
'skill.files.list',
'skill.files.read',
'skill.files.write',
'skill.preview',
'skill.install.github',
'skill.install.upload',
'mcp_server.list',
'mcp_server.get',
'mcp_server.create',
'mcp_server.update',
'mcp_server.delete',
'mcp_server.resources',
'mcp_server.resource_templates',
'mcp_server.resource_read',
'mcp_server.logs',
'mcp_server.test',
]
)
assert all(item == {'supported': True} for item in capabilities['operations'].values())
assert created['key'] not in await capabilities_response.get_data(as_text=True)
bearer_response = await client.get(
'/api/v1/system/context',
headers={'Authorization': f'Bearer {created["key"]}'},
)
assert bearer_response.status_code == 200
assert (await bearer_response.get_json())['data']['api_key_id'] == created['uuid']
jwt_response = await client.get(
'/api/v1/system/context',
headers={'Authorization': f'Bearer {owner_token}'},
)
assert jwt_response.status_code == 401
await application.persistence_mgr.execute_async(
sqlalchemy.update(apikey.ApiKey)
.where(apikey.ApiKey.uuid == created['uuid'])
.values(expires_at=datetime.datetime.now(datetime.UTC).replace(tzinfo=None) - datetime.timedelta(seconds=1))
)
expired_capabilities = await client.get(
'/api/v1/system/capabilities',
headers={'X-API-Key': created['key']},
)
assert expired_capabilities.status_code == 401
revoke_response = await client.delete(
f'/api/v1/apikeys/{created["id"]}',
headers=_auth(owner_token, workspace_uuid),
)
assert revoke_response.status_code == 200
revoked_response = await client.get(
'/api/v1/system/context',
headers={'X-API-Key': created['key']},
)
assert revoked_response.status_code == 401
revoked_capabilities = await client.get(
'/api/v1/system/capabilities',
headers={'X-API-Key': created['key']},
)
assert revoked_capabilities.status_code == 401
async def test_api_key_can_query_tasks_with_public_contract_and_resource_permission(workspace_api):
application, client, _, owner_token = workspace_api
task_query = {}
task_lookup = {}
fake_task = SimpleNamespace(
to_public_dict=lambda: {'id': 7, 'status': 'running', 'error': None, 'result': None},
to_dict=lambda: {'id': 7, 'runtime': {'state': 'PENDING'}},
)
def get_tasks_dict(*args, **kwargs):
task_query.update(kwargs)
if kwargs.get('public'):
return {'tasks': []}
return {'tasks': [], 'id_index': 1}
def get_task_by_id(*args, **kwargs):
task_lookup.update(kwargs)
return fake_task if args and args[0] == 7 else None
application.task_mgr = SimpleNamespace(
get_tasks_dict=get_tasks_dict,
get_task_by_id=get_task_by_id,
)
current_response = await client.get('/api/v1/workspaces/current', headers=_auth(owner_token))
workspace_uuid = (await current_response.get_json())['data']['workspace']['uuid']
create_response = await client.post(
'/api/v1/apikeys',
headers=_auth(owner_token, workspace_uuid),
json={'name': 'Task reader', 'scopes': ['resource.view']},
)
assert create_response.status_code == 200
key = (await create_response.get_json())['data']['key']['key']
listing = await client.get('/api/v1/system/tasks', headers={'X-API-Key': key})
assert listing.status_code == 200
assert (await listing.get_json())['data'] == {'tasks': []}
assert task_query['instance_uuid'] == application.workspace_service.instance_uuid
assert task_query['workspace_uuid'] == workspace_uuid
assert task_query['placement_generation'] == 1
assert task_query['public'] is True
bearer_listing = await client.get('/api/v1/system/tasks', headers=_auth(owner_token, workspace_uuid))
assert bearer_listing.status_code == 200
assert (await bearer_listing.get_json())['data'] == {'tasks': [], 'id_index': 1}
public_task = await client.get('/api/v1/system/tasks/7', headers={'X-API-Key': key})
assert public_task.status_code == 200
assert (await public_task.get_json())['data'] == {
'id': 7,
'status': 'running',
'error': None,
'result': None,
}
assert task_lookup == {
'instance_uuid': application.workspace_service.instance_uuid,
'workspace_uuid': workspace_uuid,
'placement_generation': 1,
}
legacy_task = await client.get('/api/v1/system/tasks/7', headers=_auth(owner_token, workspace_uuid))
assert legacy_task.status_code == 200
assert (await legacy_task.get_json())['data'] == {'id': 7, 'runtime': {'state': 'PENDING'}}
missing = await client.get('/api/v1/system/tasks/not-an-id', headers={'X-API-Key': key})
assert missing.status_code == 404
no_permission_response = await client.post(
'/api/v1/apikeys',
headers=_auth(owner_token, workspace_uuid),
json={'name': 'Task denied', 'scopes': []},
)
assert no_permission_response.status_code == 200
no_permission_key = (await no_permission_response.get_json())['data']['key']['key']
denied = await client.get('/api/v1/system/tasks', headers={'X-API-Key': no_permission_key})
assert denied.status_code == 403
async def test_cloud_projection_is_selected_explicitly_and_collaboration_runs_in_core( async def test_cloud_projection_is_selected_explicitly_and_collaboration_runs_in_core(
workspace_api, workspace_api,
): ):
@@ -1,445 +0,0 @@
"""Deterministic OAuth tests using real SQLite CAS writes, never live credentials."""
import asyncio
import base64
import json
import time
from types import SimpleNamespace
from unittest.mock import AsyncMock
import httpx
import pytest
import pytest_asyncio
import sqlalchemy as sa
from sqlalchemy.ext.asyncio import create_async_engine
from langbot.pkg.api.http.authz import Permission
from langbot.pkg.api.http.context import PrincipalContext, PrincipalType, RequestContext, WorkspaceContext
from langbot.pkg.entity.persistence.model import CodexCredential
from langbot.pkg.persistence.alembic_runner import run_alembic_stamp, run_alembic_upgrade
from langbot.pkg.provider.modelmgr.codex_auth import CodexAuth, _tokens, validate_config
from langbot.pkg.workspace.errors import WorkspaceNotFoundError
def context(workspace='w', user='u', principal=PrincipalType.ACCOUNT, permitted=True):
return RequestContext(
'i',
0,
'r',
'user_token',
PrincipalContext(principal, account_uuid=user),
WorkspaceContext(
workspace, 'm', 'owner', frozenset({Permission.PROVIDER_SECRET_MANAGE} if permitted else set())
),
)
def jwt(**claims):
return 'test.' + base64.urlsafe_b64encode(json.dumps(claims).encode()).decode().rstrip('=') + '.test'
def token_response(**extra):
return {
'access_token': jwt(**{'https://api.openai.com/auth': {'chatgpt_account_id': 'account'}}),
'refresh_token': 'refresh-secret',
'expires_in': 3600,
**extra,
}
@pytest_asyncio.fixture
async def auth(tmp_path):
engine = create_async_engine(f'sqlite+aiosqlite:///{tmp_path / "codex.db"}')
@sa.event.listens_for(engine.sync_engine, 'connect')
def foreign_keys(connection, _):
connection.execute('PRAGMA foreign_keys=ON')
async with engine.begin() as conn:
await conn.execute(
sa.text(
'CREATE TABLE model_providers (uuid VARCHAR(255) PRIMARY KEY, workspace_uuid VARCHAR(36) NOT NULL, requester TEXT, UNIQUE(workspace_uuid, uuid))'
)
)
await conn.execute(
sa.text(
"INSERT INTO model_providers VALUES ('p','w','openai-codex'), ('other','other','openai-codex'), ('api','w','openai-chat-completions')"
)
)
await run_alembic_stamp(engine, '0021_merge_reasoning_config')
await run_alembic_upgrade(engine, '0022_codex_credentials')
async def execute(statement):
async with engine.begin() as conn:
return await conn.execute(statement)
service = CodexAuth(SimpleNamespace(persistence_mgr=SimpleNamespace(execute_async=execute)))
service.engine = engine
await execute(
sa.insert(CodexCredential).values(provider_uuid='p', workspace_uuid='w', payload={}, version=0, lease_until=0)
)
try:
yield service
finally:
await engine.dispose()
async def seed(auth, payload):
await auth.ap.persistence_mgr.execute_async(sa.update(CodexCredential).values(payload=payload))
@pytest.mark.asyncio
async def test_migration_upgrade_repeat_fk_cascade(auth):
await run_alembic_upgrade(auth.engine, '0022_codex_credentials')
await run_alembic_stamp(auth.engine, '0021_merge_reasoning_config')
await run_alembic_upgrade(auth.engine, '0022_codex_credentials')
with pytest.raises(sa.exc.IntegrityError):
await auth.ap.persistence_mgr.execute_async(
sa.insert(CodexCredential).values(
provider_uuid='other', workspace_uuid='w', payload={}, version=0, lease_until=0
)
)
await auth.ap.persistence_mgr.execute_async(sa.text("DELETE FROM model_providers WHERE uuid='p'"))
assert await auth._read('w', 'p') is None
from langbot.pkg.persistence.alembic_runner import run_alembic_downgrade
await run_alembic_downgrade(auth.engine, '0021_merge_reasoning_config')
async with auth.engine.connect() as conn:
assert 'codex_credentials' not in await conn.run_sync(lambda sync: sa.inspect(sync).get_table_names())
await run_alembic_upgrade(auth.engine, '0022_codex_credentials')
async with auth.engine.connect() as conn:
assert 'codex_credentials' in await conn.run_sync(lambda sync: sa.inspect(sync).get_table_names())
@pytest.mark.asyncio
async def test_device_pacing_exchange_secrecy_and_user_binding(auth):
auth._post = AsyncMock(
side_effect=[
httpx.Response(200, json={'device_auth_id': 'device-secret', 'usercode': 'CODE', 'interval': '5'}),
httpx.Response(200, json={'authorization_code': 'code-secret', 'code_verifier': 'verifier-secret'}),
httpx.Response(200, json=token_response()),
]
)
start = await auth.start(context(), 'p')
assert set(start) == {'authorization_id', 'user_code', 'interval', 'expires_at', 'verification_uri'}
assert 'device-secret' not in json.dumps(start)
attempt = start['authorization_id']
with pytest.raises(WorkspaceNotFoundError):
await auth.poll(context(user='attacker'), 'p', attempt)
assert (await auth.poll(context(), 'p', attempt))['status'] == 'pending'
assert auth._post.await_count == 1
row = await auth._read('w', 'p')
row['payload']['pending']['next_poll_at'] = 0
await seed(auth, row['payload'])
assert await auth.poll(context(), 'p', attempt) == {'status': 'connected'}
assert await auth.poll(context(), 'p', attempt) == {'status': 'connected'}
exchange = auth._post.call_args.kwargs['data']
assert exchange['grant_type'] == 'authorization_code'
assert exchange['redirect_uri'] == 'https://auth.openai.com/deviceauth/callback'
assert exchange['code_verifier'] == 'verifier-secret'
status = await auth.status(context(), 'p')
assert set(status) == {'status', 'connected', 'expires_at'}
assert 'secret' not in json.dumps(status)
await auth.disconnect(context(), 'p')
assert (await auth._read('w', 'p'))['payload'] == {}
@pytest.mark.asyncio
@pytest.mark.parametrize(
'ctx,provider,error',
[
(context('other'), 'p', WorkspaceNotFoundError),
(context(principal=PrincipalType.API_KEY), 'p', ValueError),
(context(permitted=False), 'p', ValueError),
(context(), 'api', ValueError),
],
)
async def test_auth_tenant_principal_permission_guards(auth, ctx, provider, error):
auth._post = AsyncMock()
with pytest.raises(error):
await auth.start(ctx, provider)
auth._post.assert_not_called()
@pytest.mark.asyncio
async def test_refresh_cross_instance_single_flight_and_rotation(auth):
old = _tokens(token_response())
old['expires_at'] = 0
await seed(auth, {'tokens': old})
entered, release = asyncio.Event(), asyncio.Event()
async def refresh(*args, **kwargs):
entered.set()
await release.wait()
return httpx.Response(200, json=token_response(refresh_token='rotated-secret'))
auth._post = AsyncMock(side_effect=refresh)
other = CodexAuth(auth.ap)
other._post = auth._post
first = asyncio.create_task(auth.access('w', 'p'))
await entered.wait()
second = asyncio.create_task(other.access('w', 'p'))
release.set()
a, b = await asyncio.gather(first, second)
assert a == b
assert a['refresh_token'] == 'rotated-secret'
assert auth._post.await_count == 1
assert (await auth._read('w', 'p'))['payload']['tokens'] == a
@pytest.mark.asyncio
@pytest.mark.parametrize(
'status,error,invalid',
[
(400, 'invalid_grant', True),
(401, 'refresh_token_reused', True),
(429, 'limited', False),
(500, 'secret-upstream-body', False),
(403, 'permission_denied', False),
],
)
async def test_refresh_errors_are_safe_and_transient_preserves_tokens(auth, status, error, invalid):
old = _tokens(token_response())
old['expires_at'] = 0
await seed(auth, {'tokens': old})
auth._post = AsyncMock(
return_value=httpx.Response(status, json={'error': error, 'access_token': 'secret-upstream-body'})
)
with pytest.raises(ValueError) as caught:
await auth.access('w', 'p')
assert 'secret' not in str(caught.value)
payload = (await auth._read('w', 'p'))['payload']
assert bool(payload.get('invalid')) == invalid
assert ('tokens' not in payload) if invalid else payload['tokens'] == old
@pytest.mark.asyncio
@pytest.mark.parametrize('cancel', [False, True])
async def test_disconnect_or_cancel_fences_inflight_exchange(auth, cancel):
old = _tokens(token_response())
await seed(
auth,
{
'tokens': old,
'pending': {
'authorization_id': 'attempt',
'account_uuid': 'u',
'expires_at': time.time() + 100,
'next_poll_at': 0,
'interval': 5,
'device_auth_id': 'device',
'user_code': 'code',
},
},
)
entered, release = asyncio.Event(), asyncio.Event()
async def post(path, **kwargs):
if path.endswith('/token') and path != '/oauth/token':
return httpx.Response(200, json={'authorization_code': 'code', 'code_verifier': 'verifier'})
entered.set()
await release.wait()
return httpx.Response(200, json=token_response())
auth._post = post
task = asyncio.create_task(auth.poll(context(), 'p', 'attempt'))
await entered.wait()
if cancel:
await auth.cancel(context(), 'p', 'attempt')
else:
await auth.disconnect(context(), 'p')
release.set()
with pytest.raises(ValueError, match='cancelled or replaced'):
await task
payload = (await auth._read('w', 'p'))['payload']
assert payload == ({'tokens': old} if cancel else {})
@pytest.mark.asyncio
@pytest.mark.parametrize('status,interval', [(403, 5), (404, 5), (429, 10)])
async def test_device_pending_and_backoff(auth, status, interval):
await seed(
auth,
{
'pending': {
'authorization_id': 'attempt',
'account_uuid': 'u',
'expires_at': time.time() + 100,
'next_poll_at': 0,
'interval': 5,
'device_auth_id': 'device',
'user_code': 'code',
}
},
)
auth._post = AsyncMock(return_value=httpx.Response(status))
assert await auth.poll(context(), 'p', 'attempt') == {'status': 'pending', 'interval': interval}
assert await auth.poll(context(), 'p', 'attempt') == {'status': 'pending', 'interval': interval}
assert auth._post.await_count == 1
@pytest.mark.asyncio
async def test_device_replacement_expiry_and_idempotent_cancel(auth):
auth._post = AsyncMock(return_value=httpx.Response(200, json={'device_auth_id': 'device', 'user_code': 'CODE'}))
first = await auth.start(context(), 'p')
second = await auth.start(context(), 'p')
assert first['authorization_id'] != second['authorization_id']
assert await auth.poll(context(), 'p', first['authorization_id']) == {'status': 'expired'}
await auth.cancel(context(), 'p', first['authorization_id'])
payload = (await auth._read('w', 'p'))['payload']
assert payload['pending']['authorization_id'] == second['authorization_id']
payload['pending']['expires_at'] = 0
await seed(auth, payload)
assert await auth.poll(context(), 'p', second['authorization_id']) == {'status': 'expired'}
assert 'pending' not in (await auth._read('w', 'p'))['payload']
@pytest.mark.asyncio
async def test_device_accepts_issuer_iso_expiry(auth):
from datetime import datetime, timezone
expires = datetime.fromtimestamp(time.time() + 600, timezone.utc).isoformat().replace('+00:00', 'Z')
auth._post = AsyncMock(
return_value=httpx.Response(200, json={'device_auth_id': 'device', 'user_code': 'CODE', 'expires_at': expires})
)
result = await auth.start(context(), 'p')
assert time.time() < result['expires_at'] < time.time() + 900
@pytest.mark.asyncio
async def test_device_expires_in_fallback(auth):
auth._post = AsyncMock(
return_value=httpx.Response(200, json={'device_auth_id': 'device', 'user_code': 'CODE', 'expires_in': 60})
)
result = await auth.start(context(), 'p')
assert time.time() < result['expires_at'] <= time.time() + 60
@pytest.mark.asyncio
@pytest.mark.parametrize('cancel_reads_before_refresh', [False, True])
async def test_cancel_pending_relogin_waits_for_existing_refresh(auth, cancel_reads_before_refresh):
old = _tokens(token_response())
old['expires_at'] = 0
await seed(auth, {'tokens': old, 'pending': {'authorization_id': 'attempt', 'account_uuid': 'u'}})
entered, release, cancel_read = asyncio.Event(), asyncio.Event(), asyncio.Event()
async def refresh(*args, **kwargs):
entered.set()
await release.wait()
return httpx.Response(200, json=token_response(refresh_token='rotated-secret'))
other = CodexAuth(auth.ap)
original_read = other._read
async def read(workspace, provider):
row = await original_read(workspace, provider)
cancel_read.set()
if cancel_reads_before_refresh:
await entered.wait()
return row
other._read = read
auth._post = refresh
if cancel_reads_before_refresh:
cancelling = asyncio.create_task(other.cancel(context(), 'p', 'attempt'))
await cancel_read.wait()
refreshing = asyncio.create_task(auth.access('w', 'p'))
await entered.wait()
if not cancel_reads_before_refresh:
cancelling = asyncio.create_task(other.cancel(context(), 'p', 'attempt'))
await cancel_read.wait()
await asyncio.sleep(0.05)
try:
assert not cancelling.done(), 'Cancellation must not revoke the refresh lease'
finally:
release.set()
results = await asyncio.gather(refreshing, cancelling, return_exceptions=True)
assert not any(isinstance(result, Exception) for result in results)
payload = (await auth._read('w', 'p'))['payload']
assert payload['tokens']['refresh_token'] == 'rotated-secret'
assert 'pending' not in payload
@pytest.mark.asyncio
@pytest.mark.parametrize('operation', ['save', 'cancel', 'disconnect', 'acquire', 'release', 'read'])
async def test_credential_database_errors_never_expose_secrets(auth, operation):
import traceback
markers = ['ACCESS-MARKER', 'REFRESH-MARKER', 'DEVICE-MARKER', 'VERIFIER-MARKER']
payload = {
'tokens': {'access_token': markers[0], 'refresh_token': markers[1]},
'pending': {
'authorization_id': 'attempt',
'account_uuid': 'u',
'device_auth_id': markers[2],
'code_verifier': markers[3],
},
}
await seed(auth, payload)
if operation == 'read':
auth.ap.persistence_mgr.execute_async = AsyncMock(
side_effect=sa.exc.StatementError(
'failure', 'SELECT credentials', {'payload': payload}, RuntimeError(markers[0])
)
)
else:
column = 'payload' if operation in ('save', 'cancel', 'disconnect') else 'lease_owner'
condition = ' WHEN NEW.lease_owner IS NULL' if operation == 'release' else ''
# Trigger errors can themselves contain secrets, even for parameter-free writes.
await auth.ap.persistence_mgr.execute_async(
sa.text(
f'CREATE TRIGGER reject_write BEFORE UPDATE OF {column} ON codex_credentials{condition} '
f"BEGIN SELECT RAISE(ABORT, '{' '.join(markers)}'); END"
)
)
with pytest.raises(ValueError, match='credential storage') as caught:
if operation == 'save':
async with auth._lease('w', 'p') as owner:
await auth._save('w', 'p', owner, payload)
elif operation == 'cancel':
await auth.cancel(context(), 'p', 'attempt')
elif operation == 'disconnect':
await auth.disconnect(context(), 'p')
elif operation == 'read':
await auth._read('w', 'p')
else:
async with auth._lease('w', 'p'):
pass
rendered = ''.join(traceback.format_exception(caught.value))
assert all(marker not in rendered for marker in markers)
assert caught.value.__suppress_context__
@pytest.mark.asyncio
async def test_credential_serialization_failure_is_sanitized(auth):
import traceback
class Secret:
def __repr__(self):
return 'SERIALIZATION-SECRET'
with pytest.raises(ValueError, match='credential storage') as caught:
async with auth._lease('w', 'p') as owner:
await auth._save('w', 'p', owner, {'tokens': {'refresh_token': Secret()}})
assert 'SERIALIZATION-SECRET' not in ''.join(traceback.format_exception(caught.value))
assert caught.value.__suppress_context__
def test_token_refresh_fallback_and_config_validation():
old = _tokens(token_response())
refreshed = _tokens({'access_token': 'opaque-access', 'expires_in': 3600}, old)
assert refreshed['refresh_token'] == old['refresh_token']
assert refreshed['connection_id'] == old['connection_id']
for expiry in [float('nan'), float('inf'), -1, 'bad']:
with pytest.raises(ValueError):
_tokens(token_response(expires_in=expiry))
data = {'requester': 'openai-codex'}
validate_config(data)
assert data['api_keys'] == []
for update in [{'base_url': 'https://evil.invalid'}, {'api_keys': ['secret']}]:
with pytest.raises(ValueError):
validate_config({**data, **update})
ordinary = {'requester': 'openai-chat-completions', 'api_keys': ['key'], 'base_url': 'https://custom.invalid'}
before = dict(ordinary)
validate_config(ordinary)
assert ordinary == before
@@ -108,7 +108,7 @@ class TestSQLiteMigrationUpgrade:
await run_alembic_upgrade(sqlite_engine, 'head') await run_alembic_upgrade(sqlite_engine, 'head')
assert await get_alembic_current(sqlite_engine) == _get_script_head() assert await get_alembic_current(sqlite_engine) == _get_script_head()
assert _get_script_head() == '0022_codex_credentials' assert _get_script_head() == '0021_merge_reasoning_config'
@pytest.mark.asyncio @pytest.mark.asyncio
async def test_upgrade_from_reasoning_config_head_to_merged_head(self, sqlite_engine): async def test_upgrade_from_reasoning_config_head_to_merged_head(self, sqlite_engine):
@@ -119,7 +119,7 @@ class TestSQLiteMigrationUpgrade:
await run_alembic_stamp(sqlite_engine, '0018_llm_reasoning_config') await run_alembic_stamp(sqlite_engine, '0018_llm_reasoning_config')
await run_alembic_upgrade(sqlite_engine, 'head') await run_alembic_upgrade(sqlite_engine, 'head')
assert await get_alembic_current(sqlite_engine) == '0022_codex_credentials' assert await get_alembic_current(sqlite_engine) == '0021_merge_reasoning_config'
@pytest.mark.asyncio @pytest.mark.asyncio
async def test_upgrade_from_baseline_to_head(self, sqlite_engine): async def test_upgrade_from_baseline_to_head(self, sqlite_engine):
@@ -549,12 +549,10 @@ class TestPostgreSQLWorkspaceMigration:
await conn.run_sync(lambda sync_conn: sa.inspect(sync_conn).get_table_names()) await conn.run_sync(lambda sync_conn: sa.inspect(sync_conn).get_table_names())
) )
assert 'workspaces' not in tables_before_migration assert 'workspaces' not in tables_before_migration
assert 'codex_credentials' not in tables_before_migration
await manager._initialize_managed_schema() await manager._initialize_managed_schema()
async with postgres_engine.connect() as conn: async with postgres_engine.connect() as conn:
assert 'codex_credentials' in await conn.run_sync(lambda sync: sa.inspect(sync).get_table_names())
account = (await conn.execute(text('SELECT uuid, status, source FROM users'))).mappings().one() account = (await conn.execute(text('SELECT uuid, status, source FROM users'))).mappings().one()
workspace = ( workspace = (
(await conn.execute(text('SELECT * FROM workspaces WHERE source = :source'), {'source': 'local'})) (await conn.execute(text('SELECT * FROM workspaces WHERE source = :source'), {'source': 'local'}))
@@ -5,7 +5,6 @@ import logging
import os import os
import pathlib import pathlib
import sqlite3 import sqlite3
from contextlib import closing
import pytest import pytest
import sqlalchemy as sa import sqlalchemy as sa
@@ -35,7 +34,7 @@ def _manifest_payloads(backup_directory) -> list[dict]:
def _assert_verified_backup(payload: dict) -> None: def _assert_verified_backup(payload: dict) -> None:
backup_path = pathlib.Path(payload['backup_path']) backup_path = pathlib.Path(payload['backup_path'])
with closing(sqlite3.connect(f'{backup_path.as_uri()}?mode=ro', uri=True)) as connection: with sqlite3.connect(f'{backup_path.as_uri()}?mode=ro', uri=True) as connection:
assert connection.execute('PRAGMA quick_check').fetchall() == [('ok',)] assert connection.execute('PRAGMA quick_check').fetchall() == [('ok',)]
assert connection.execute('SELECT version_num FROM alembic_version').fetchone()[0] == payload['source_revision'] assert connection.execute('SELECT version_num FROM alembic_version').fetchone()[0] == payload['source_revision']
@@ -403,12 +403,10 @@ async def test_persistence_startup_defers_workspace_tables_until_account_upgrade
await conn.run_sync(lambda sync_conn: sa.inspect(sync_conn).get_table_names()) await conn.run_sync(lambda sync_conn: sa.inspect(sync_conn).get_table_names())
) )
assert 'workspaces' not in tables_before_migration assert 'workspaces' not in tables_before_migration
assert 'codex_credentials' not in tables_before_migration
await manager._run_alembic_migrations() await manager._run_alembic_migrations()
async with engine.connect() as conn: async with engine.connect() as conn:
assert 'codex_credentials' in await conn.run_sync(lambda sync: sa.inspect(sync).get_table_names())
workspace = ( workspace = (
(await conn.execute(sa.text("SELECT * FROM workspaces WHERE source = 'local'"))).mappings().one() (await conn.execute(sa.text("SELECT * FROM workspaces WHERE source = 'local'"))).mappings().one()
) )
@@ -29,6 +29,9 @@ from langbot.pkg.cloud.entitlements import (
EntitlementSnapshot, EntitlementSnapshot,
EntitlementUnavailableError, EntitlementUnavailableError,
) )
from langbot.pkg.skill.manager import SkillManager
from langbot.pkg.skill.repository import SkillRepository
from langbot.pkg.provider.tools.loaders import skill as skill_loader
pytestmark = pytest.mark.integration pytestmark = pytest.mark.integration
@@ -54,7 +57,7 @@ class _AdmissionBackend(BaseSandboxBackend):
'mount_isolation': True, 'mount_isolation': True,
'network_isolation': True, 'network_isolation': True,
'hard_workspace_quota': True, 'hard_workspace_quota': True,
'hard_skill_storage_quota': True, 'hard_read_only_mount_quota': True,
'bounded_ephemeral_storage': True, 'bounded_ephemeral_storage': True,
'inode_quota': True, 'inode_quota': True,
} }
@@ -230,9 +233,18 @@ async def _stack(tmp_path):
deployment=SimpleNamespace(multi_workspace_enabled=True), deployment=SimpleNamespace(multi_workspace_enabled=True),
entitlement_resolver=EntitlementResolver('instance-a', entitlements), entitlement_resolver=EntitlementResolver('instance-a', entitlements),
workspace_service=workspace_service, workspace_service=workspace_service,
instance_config=SimpleNamespace(data={'box': box_config, 'system': {'limitation': {}}}), instance_config=SimpleNamespace(
data={
'skills': {'root': str(shared_root / 'skills')},
'box': box_config,
'system': {'limitation': {}},
}
),
) )
app.skill_repository = SkillRepository(app)
app.skill_mgr = SkillManager(app)
service = BoxService(app, client=client) service = BoxService(app, client=client)
app.box_service = service
await service.initialize() await service.initialize()
return service, runtime, backend, entitlements, server_task, client_task return service, runtime, backend, entitlements, server_task, client_task
@@ -312,7 +324,7 @@ async def test_two_workspaces_get_isolated_physical_sessions_and_paths(tmp_path)
@pytest.mark.asyncio @pytest.mark.asyncio
async def test_cloud_skills_reject_host_paths_and_require_managed_entitlement(tmp_path): async def test_cloud_core_skills_mount_generically_and_do_not_require_box_entitlement(tmp_path):
service, runtime, backend, entitlements, server_task, client_task = await _stack(tmp_path) service, runtime, backend, entitlements, server_task, client_task = await _stack(tmp_path)
first = _context('workspace-a') first = _context('workspace-a')
second = _context('workspace-b') second = _context('workspace-b')
@@ -324,32 +336,35 @@ async def test_cloud_skills_reject_host_paths_and_require_managed_entitlement(tm
managed=False, managed=False,
) )
try: try:
private = await service.create_skill( repository = service.ap.skill_repository
await repository.create_skill(
second, second,
{ {
'name': 'private', 'name': 'private',
'instructions': 'workspace-b secret', 'instructions': 'workspace-b secret',
}, },
) )
own_skill = await service.create_skill( own_skill = await repository.create_skill(
first, first,
{ {
'name': 'runner', 'name': 'runner',
'instructions': 'Run scripts/main.py', 'instructions': 'Run scripts/main.py',
}, },
) )
await service.write_skill_file(first, 'runner', 'scripts/main.py', "print('ok')") await repository.write_skill_file(first, 'runner', 'scripts/main.py', "print('ok')")
await service.write_skill_file(first, 'runner', 'requirements.txt', 'requests==2.32.0\n') await repository.write_skill_file(first, 'runner', 'requirements.txt', 'requests==2.32.0\n')
refreshed_skill = await service.get_skill(first, 'runner') refreshed_skill = await repository.get_skill(first, 'runner')
assert refreshed_skill is not None assert refreshed_skill is not None
assert refreshed_skill['python_project'] is True assert refreshed_skill['python_project'] is True
await service.ap.skill_mgr.reload_skills(first)
query = _query(first, 91)
await service.execute_tool( await service.execute_tool(
{ {
'command': 'python /workspace/.skills/runner/scripts/main.py', 'command': 'python /workspace/.skills/runner/scripts/main.py',
'workdir': '/workspace/.skills/runner', 'workdir': '/workspace/.skills/runner',
}, },
_query(first, 91), query,
skill_name='runner', read_only_mounts=skill_loader.build_execution_mounts(service.ap, query),
) )
mounted_spec = backend.started_specs[-1] mounted_spec = backend.started_specs[-1]
@@ -358,20 +373,14 @@ async def test_cloud_skills_reject_host_paths_and_require_managed_entitlement(tm
assert mounted_spec.extra_mounts[0].mount_path == '/workspace/.skills/runner' assert mounted_spec.extra_mounts[0].mount_path == '/workspace/.skills/runner'
assert mounted_spec.extra_mounts[0].mode.value == 'ro' assert mounted_spec.extra_mounts[0].mode.value == 'ro'
with pytest.raises(BoxAdmissionError, match='Scanning arbitrary host'): assert await repository.get_skill(first, 'private') is None
await service.scan_skill_directory(first, private['package_root']) await repository.create_skill(
with pytest.raises(BoxAdmissionError, match='package_root is runtime-owned'): ineligible,
await service.create_skill( {'name': 'docs-only', 'instructions': 'Read this without Box.'},
first, )
{ assert [skill['name'] for skill in await repository.list_skills(ineligible)] == ['docs-only']
'name': 'stolen',
'package_root': private['package_root'],
},
)
assert await service.get_skill(first, 'private') is None
with pytest.raises(EntitlementUnavailableError): with pytest.raises(EntitlementUnavailableError):
await service.list_skills(ineligible) await service.execute_tool({'command': 'true'}, _query(ineligible, 92))
finally: finally:
server_task.cancel() server_task.cancel()
client_task.cancel() client_task.cancel()
@@ -20,7 +20,6 @@ from langbot.pkg.entity.persistence.model import LLMModel, ModelProvider
from langbot.pkg.entity.persistence.pipeline import LegacyPipeline from langbot.pkg.entity.persistence.pipeline import LegacyPipeline
from langbot.pkg.entity.persistence.workspace import Workspace from langbot.pkg.entity.persistence.workspace import Workspace
from langbot.pkg.workspace.errors import WorkspaceNotFoundError from langbot.pkg.workspace.errors import WorkspaceNotFoundError
from langbot.pkg.persistence.mgr import PersistenceManager
pytestmark = pytest.mark.asyncio pytestmark = pytest.mark.asyncio
@@ -29,10 +28,15 @@ WORKSPACE_A = '00000000-0000-0000-0000-00000000000a'
WORKSPACE_B = '00000000-0000-0000-0000-00000000000b' WORKSPACE_B = '00000000-0000-0000-0000-00000000000b'
class _PersistenceManager(PersistenceManager): class _PersistenceManager:
def __init__(self, engine): def __init__(self, engine):
super().__init__(SimpleNamespace()) self.engine = engine
self.db = SimpleNamespace(get_engine=lambda: engine)
async def execute_async(self, *args, **kwargs):
async with self.engine.connect() as connection:
result = await connection.execute(*args, **kwargs)
await connection.commit()
return result
@staticmethod @staticmethod
def serialize_model(model, data, masked_columns=None): def serialize_model(model, data, masked_columns=None):
@@ -1009,37 +1009,6 @@ class TestMCPServiceTestMCPServer:
# Verify - returns task ID # Verify - returns task ID
assert task_id == 123 assert task_id == 123
@pytest.mark.parametrize('refresh_first', [False, True])
async def test_persisted_test_preserves_failure_details(self, refresh_first):
from langbot.pkg.provider.tools.loaders.mcp import MCPSessionStatus
runtime_info = {'status': 'error', 'error_message': 'HTTP 403: access denied'}
session = SimpleNamespace(
status=MCPSessionStatus.CONNECTED if refresh_first else MCPSessionStatus.ERROR,
session=object(),
refresh=AsyncMock(side_effect=RuntimeError('refresh failed')),
start=AsyncMock(side_effect=RuntimeError('Connection failed, please check URL')),
get_runtime_info_dict=Mock(return_value=runtime_info),
)
captured = {}
def create_user_task(coroutine, **kwargs):
captured.update(coroutine=coroutine, context=kwargs['context'])
return SimpleNamespace(id=123)
ap = SimpleNamespace(
tool_mgr=SimpleNamespace(mcp_tool_loader=SimpleNamespace(get_session=Mock(return_value=session))),
task_mgr=SimpleNamespace(create_user_task=Mock(side_effect=create_user_task)),
)
service = _service(ap)
service._require_server = AsyncMock(return_value=(_CONTEXT, {'name': 'existing-server'}))
await service.test_mcp_server(_CONTEXT, 'existing-server', {})
with pytest.raises(RuntimeError, match='Connection failed'):
await captured['coroutine']
assert captured['context'].metadata['runtime_info'] == runtime_info
session.start.assert_awaited_once()
assert session.refresh.await_count == int(refresh_first)
async def test_test_mcp_server_not_found_raises(self): async def test_test_mcp_server_not_found_raises(self):
"""Raises ValueError when server not found.""" """Raises ValueError when server not found."""
# Setup # Setup
@@ -1083,45 +1052,6 @@ class TestMCPServiceTestMCPServer:
ap.tool_mgr.mcp_tool_loader.load_mcp_server.assert_called_once() ap.tool_mgr.mcp_tool_loader.load_mcp_server.assert_called_once()
assert task_id == 456 assert task_id == 456
async def test_transient_test_preserves_runtime_info_after_connection_failure(self):
runtime_info = {
'status': 'error',
'error_phase': 'oauth_required',
'retry_count': 1,
}
mock_session = SimpleNamespace(
server_name='oauth-server',
start=AsyncMock(side_effect=RuntimeError('connection failed')),
get_runtime_info_dict=Mock(return_value=runtime_info),
shutdown=AsyncMock(),
)
ap = SimpleNamespace(
tool_mgr=SimpleNamespace(
mcp_tool_loader=SimpleNamespace(load_mcp_server=AsyncMock(return_value=mock_session))
)
)
captured: dict = {}
def create_user_task(coroutine, **kwargs):
captured['coroutine'] = coroutine
captured['context'] = kwargs['context']
return SimpleNamespace(id=457)
ap.task_mgr = SimpleNamespace(create_user_task=Mock(side_effect=create_user_task))
service = _service(ap)
task_id = await service.test_mcp_server(
_CONTEXT,
'_',
{'name': 'OAuth server', 'mode': 'remote', 'enable': True, 'extra_args': {}},
)
assert task_id == 457
with pytest.raises(RuntimeError, match='connection failed'):
await captured['coroutine']
assert captured['context'].metadata['runtime_info'] == runtime_info
mock_session.shutdown.assert_awaited_once_with()
async def test_rejected_transient_test_session_is_shut_down(self): async def test_rejected_transient_test_session_is_shut_down(self):
ap = SimpleNamespace() ap = SimpleNamespace()
mock_session = MagicMock() mock_session = MagicMock()
@@ -1,401 +0,0 @@
"""Provider deletion uses real SQLite transactions and real runtime cache cleanup."""
import asyncio
from types import SimpleNamespace
from unittest.mock import AsyncMock, Mock
import pytest
import pytest_asyncio
import quart
import sqlalchemy as sa
from sqlalchemy.ext.asyncio import create_async_engine
from langbot.pkg.api.http.context import ExecutionContext
from langbot.pkg.api.http.controller.groups.provider.providers import ModelProvidersRouterGroup
from langbot.pkg.api.http.service.provider import ModelProviderService
from langbot.pkg.entity.persistence.model import CodexCredential, EmbeddingModel, LLMModel, ModelProvider, RerankModel
from langbot.pkg.entity.persistence.user import User
from langbot.pkg.entity.persistence.workspace import Workspace
from langbot.pkg.persistence.mgr import PersistenceManager, PersistenceMode
from langbot.pkg.provider.modelmgr.modelmgr import ModelManager
from langbot.pkg.workspace.errors import WorkspaceNotFoundError
pytestmark = pytest.mark.asyncio
MODEL_TYPES = (LLMModel, EmbeddingModel, RerankModel)
TABLES = (*MODEL_TYPES, CodexCredential, ModelProvider)
@pytest_asyncio.fixture
async def deletion(tmp_path):
engine = create_async_engine(f'sqlite+aiosqlite:///{tmp_path / "cascade.db"}')
@sa.event.listens_for(engine.sync_engine, 'connect')
def enable_foreign_keys(connection, _record):
connection.execute('PRAGMA foreign_keys=ON')
ap = SimpleNamespace(logger=Mock())
pm = ap.persistence_mgr = PersistenceManager(ap)
pm.db = SimpleNamespace(get_engine=lambda: engine)
manager = ap.model_mgr = ModelManager(ap)
contexts = {workspace: ExecutionContext('instance', workspace, 1) for workspace in ('a', 'b')}
# Only execution binding discovery is stubbed; cache indexing/removal/close is real.
manager.resolve_execution_context = AsyncMock(side_effect=lambda context: contexts[context])
service = ModelProviderService(ap)
closed = []
async def snapshot():
async with engine.connect() as conn:
return {
table.__tablename__: [dict(row) for row in (await conn.execute(sa.select(table))).mappings()]
for table in TABLES
}
async with engine.begin() as conn:
for table in (User, Workspace, ModelProvider, CodexCredential, *MODEL_TYPES):
await conn.run_sync(table.__table__.create)
for workspace in contexts:
await conn.execute(
sa.insert(Workspace).values(
uuid=workspace,
instance_uuid='instance',
name=workspace,
slug=workspace,
source='cloud_projection',
)
)
for provider, workspace in (('target', 'a'), ('neighbor', 'a'), ('foreign', 'b'), ('empty', 'a')):
await conn.execute(
sa.insert(ModelProvider).values(
uuid=provider,
workspace_uuid=workspace,
name=provider,
requester='openai-codex',
base_url='https://chatgpt.com/backend-api/codex',
api_keys=[],
)
)
await conn.execute(
sa.insert(CodexCredential).values(
provider_uuid=provider,
workspace_uuid=workspace,
payload={'synthetic': provider},
)
)
async def close(provider=provider):
# A separate connection must observe the durable deletion before close runs.
state = await snapshot()
assert all(row['uuid'] != provider for row in state['model_providers'])
assert pm.current_session() is None
closed.append(provider)
runtime = SimpleNamespace(requester=SimpleNamespace(aclose=AsyncMock(side_effect=close)))
manager._cache_set(manager.provider_dict, manager._cache_key(contexts[workspace], provider), runtime)
if provider == 'empty':
continue
for model_type, cache in zip(
MODEL_TYPES,
(
manager.llm_model_dict,
manager.embedding_model_dict,
manager.rerank_model_dict,
),
):
for index in range(2):
uuid = f'{provider}-{model_type.__tablename__}-{index}'
await conn.execute(
sa.insert(model_type).values(
uuid=uuid,
workspace_uuid=workspace,
provider_uuid=provider,
name=uuid,
)
)
manager._cache_set(cache, manager._cache_key(contexts[workspace], uuid), object())
initial = await snapshot()
initial_caches = [
dict(cache)
for cache in (
manager.provider_dict,
manager.llm_model_dict,
manager.embedding_model_dict,
manager.rerank_model_dict,
)
]
try:
yield SimpleNamespace(
ap=ap,
pm=pm,
engine=engine,
service=service,
manager=manager,
snapshot=snapshot,
initial=initial,
initial_caches=initial_caches,
closed=closed,
)
finally:
await engine.dispose()
def assert_caches_unchanged(deletion):
assert deletion.closed == []
assert deletion.initial_caches == [
dict(cache)
for cache in (
deletion.manager.provider_dict,
deletion.manager.llm_model_dict,
deletion.manager.embedding_model_dict,
deletion.manager.rerank_model_dict,
)
]
@pytest.mark.parametrize('mode', [PersistenceMode.OSS_COMPAT, PersistenceMode.CLOUD_RUNTIME])
async def test_cascade_deletes_all_model_types_and_credentials_after_commit(deletion, mode):
deletion.pm.mode = mode
await deletion.service.delete_provider('a', 'target', cascade=True)
state = await deletion.snapshot()
for table, rows in deletion.initial.items():
identity = 'uuid' if table == 'model_providers' else 'provider_uuid'
assert state[table] == [row for row in rows if row[identity] != 'target']
assert deletion.closed == ['target']
deletion.ap.logger.warning.assert_not_called()
for cache in (
deletion.manager.provider_dict,
deletion.manager.llm_model_dict,
deletion.manager.embedding_model_dict,
deletion.manager.rerank_model_dict,
):
assert all(not key[-1].startswith('target') for key in cache)
assert any(key[1] == 'b' for key in cache)
assert any(key[-1].startswith('neighbor') for key in cache)
@pytest.mark.parametrize('model_type', MODEL_TYPES)
@pytest.mark.parametrize('kwargs', [{}, {'cascade': False}])
async def test_default_guard_preserves_each_model_type(deletion, model_type, kwargs):
async with deletion.engine.begin() as conn:
for other in MODEL_TYPES:
if other is not model_type:
await conn.execute(sa.delete(other).where(other.provider_uuid == 'target'))
before = await deletion.snapshot()
with pytest.raises(ValueError, match='models still reference it'):
await deletion.service.delete_provider('a', 'target', **kwargs)
assert await deletion.snapshot() == before
assert_caches_unchanged(deletion)
@pytest.mark.parametrize('kwargs', [{}, {'cascade': False}, {'cascade': True}])
async def test_empty_provider_deletes_credentials_with_or_without_cascade(deletion, kwargs):
await deletion.service.delete_provider('a', 'empty', **kwargs)
state = await deletion.snapshot()
assert all(row['uuid'] != 'empty' for row in state['model_providers'])
assert all(row['provider_uuid'] != 'empty' for row in state['codex_credentials'])
assert deletion.closed == ['empty']
@pytest.mark.parametrize('provider', ['foreign', 'missing'])
@pytest.mark.parametrize('cascade', [False, True])
async def test_foreign_and_missing_provider_are_non_enumerating(deletion, provider, cascade):
with pytest.raises(WorkspaceNotFoundError, match='Provider not found'):
await deletion.service.delete_provider('a', provider, cascade=cascade)
assert await deletion.snapshot() == deletion.initial
assert_caches_unchanged(deletion)
@pytest.mark.parametrize('cascade', [False, True])
async def test_cloud_managed_provider_cannot_be_deleted(deletion, cascade):
async with deletion.engine.begin() as conn:
await conn.execute(
sa.update(ModelProvider)
.where(ModelProvider.uuid == 'target')
.values(
requester='space-chat-completions',
)
)
before = await deletion.snapshot()
deletion.pm.mode = PersistenceMode.CLOUD_RUNTIME
with pytest.raises(ValueError, match='managed by Cloud'):
await deletion.service.delete_provider('a', 'target', cascade=cascade)
assert await deletion.snapshot() == before
assert_caches_unchanged(deletion)
@pytest.mark.parametrize('failure_table', ['embedding_models', 'codex_credentials', 'model_providers'])
async def test_database_failure_rolls_back_all_rows_without_runtime_cleanup(deletion, failure_table):
async with deletion.engine.begin() as conn:
await conn.exec_driver_sql(
f'CREATE TRIGGER fail_delete BEFORE DELETE ON {failure_table} '
"BEGIN SELECT RAISE(ABORT, 'injected delete failure'); END"
)
with pytest.raises(sa.exc.IntegrityError, match='injected delete failure'):
await deletion.service.delete_provider('a', 'target', cascade=True)
assert await deletion.snapshot() == deletion.initial
assert_caches_unchanged(deletion)
@pytest.mark.parametrize('rollback', [False, True])
async def test_nested_transaction_defers_cleanup_until_outer_commit(deletion, rollback):
class Abort(Exception):
pass
try:
async with deletion.pm.tenant_uow('a'):
await deletion.service.delete_provider('a', 'target', cascade=True)
assert_caches_unchanged(deletion)
if rollback:
raise Abort
except Abort:
pass
tasks = tuple(deletion.service._deletion_tasks)
await asyncio.gather(*tasks, return_exceptions=True)
if rollback:
assert await deletion.snapshot() == deletion.initial
assert_caches_unchanged(deletion)
else:
assert deletion.closed == ['target']
deletion.ap.logger.warning.assert_not_called()
async def test_cascade_ignores_foreign_workspace_references_even_without_foreign_keys(deletion):
async with deletion.engine.connect() as conn:
await conn.exec_driver_sql('PRAGMA foreign_keys=OFF')
for model_type in MODEL_TYPES:
await conn.execute(
sa.update(model_type)
.where(model_type.workspace_uuid == 'b')
.values(
provider_uuid='target',
)
)
await conn.commit()
await deletion.service.delete_provider('a', 'target', cascade=True)
state = await deletion.snapshot()
for model_type in MODEL_TYPES:
assert len([row for row in state[model_type.__tablename__] if row['workspace_uuid'] == 'b']) == 2
assert all(row['provider_uuid'] != 'target' for row in state['codex_credentials'])
assert deletion.closed == ['target']
@pytest_asyncio.fixture
async def route_app(deletion):
ap = deletion.ap
ap.user_service = SimpleNamespace(
get_authenticated_account=AsyncMock(
return_value=SimpleNamespace(uuid='account', user='owner@example.invalid'),
)
)
membership = SimpleNamespace(uuid='membership', role='owner', projection_revision=0)
ap.workspace_collaboration_service = SimpleNamespace(
resolve_account_workspace=AsyncMock(
return_value=SimpleNamespace(
workspace=SimpleNamespace(uuid='a'),
membership=membership,
execution=SimpleNamespace(instance_uuid='instance', placement_generation=1),
),
)
)
ap.provider_service = SimpleNamespace(delete_provider=AsyncMock())
app = quart.Quart(__name__)
await ModelProvidersRouterGroup(ap, app).initialize()
return app.test_client(), ap.provider_service.delete_provider, membership
@pytest.mark.parametrize('query, expected', [('', None), ('?cascade=true', True), ('?cascade=false', False)])
async def test_route_passes_explicit_cascade_and_trusted_workspace(route_app, query, expected):
client, delete, _ = route_app
response = await client.delete(
'/api/v1/provider/providers/target' + query,
headers={
'Authorization': 'Bearer token',
'X-Workspace-Id': 'a',
},
)
assert response.status_code == 200
assert delete.await_count == 1
assert delete.await_args.args[0].workspace_uuid == 'a'
assert delete.await_args.args[1] == 'target'
assert delete.await_args.kwargs == ({} if expected is None else {'cascade': expected})
@pytest.mark.parametrize(
'query',
[
'?cascade=',
'?cascade',
'?cascade=TRUE',
'?cascade=1',
'?cascade=yes',
'?cascade=null',
'?cascade=%20true',
'?cascade=true&cascade=false',
'?cascade=true&cascade=true',
],
)
async def test_route_rejects_invalid_or_duplicate_cascade_before_deletion(route_app, query):
client, delete, _ = route_app
response = await client.delete(
'/api/v1/provider/providers/target' + query,
headers={
'Authorization': 'Bearer token',
'X-Workspace-Id': 'a',
},
)
assert response.status_code == 400
delete.assert_not_awaited()
@pytest.mark.parametrize('role', ['viewer', 'operator'])
async def test_cascade_requires_workspace_resource_manage_permission(route_app, role):
client, delete, membership = route_app
membership.role = role
response = await client.delete(
'/api/v1/provider/providers/target?cascade=true',
headers={
'Authorization': 'Bearer token',
'X-Workspace-Id': 'a',
},
)
assert response.status_code == 403
delete.assert_not_awaited()
@pytest.mark.parametrize(
'provider, query, status',
[
('target', '', 400),
('target', '?cascade=false', 400),
('target', '?cascade=true', 200),
('foreign', '?cascade=true', 404),
('missing', '?cascade=true', 404),
],
)
async def test_route_to_real_sqlite_service(deletion, route_app, provider, query, status):
client, _, _ = route_app
deletion.ap.provider_service = deletion.service
# The route forwards RequestContext, unlike the string-context service tests.
deletion.manager.resolve_execution_context = AsyncMock(
side_effect=lambda context: ExecutionContext(
context.instance_uuid,
context.workspace_uuid,
context.placement_generation,
)
)
response = await client.delete(
'/api/v1/provider/providers/' + provider + query,
headers={
'Authorization': 'Bearer token',
'X-Workspace-Id': 'a',
},
)
assert response.status_code == status
if status == 200:
assert deletion.closed == ['target']
for model_type in MODEL_TYPES:
assert all(
row['provider_uuid'] != 'target' for row in (await deletion.snapshot())[model_type.__tablename__]
)
else:
assert await deletion.snapshot() == deletion.initial
assert_caches_unchanged(deletion)
@@ -14,12 +14,11 @@ Source: src/langbot/pkg/api/http/service/provider.py
from __future__ import annotations from __future__ import annotations
import pytest import pytest
from contextlib import nullcontext
from unittest.mock import AsyncMock, Mock from unittest.mock import AsyncMock, Mock
from types import SimpleNamespace from types import SimpleNamespace
from langbot.pkg.api.http.service.provider import ModelProviderService from langbot.pkg.api.http.service.provider import ModelProviderService
from langbot.pkg.entity.persistence.model import ModelProvider, LLMModel from langbot.pkg.entity.persistence.model import ModelProvider, LLMModel, EmbeddingModel, RerankModel
from langbot.pkg.workspace.errors import WorkspaceNotFoundError from langbot.pkg.workspace.errors import WorkspaceNotFoundError
@@ -384,35 +383,112 @@ class TestModelProviderServiceUpdateProvider:
class TestModelProviderServiceDeleteProvider: class TestModelProviderServiceDeleteProvider:
"""Fast guard coverage; real transaction/cache behavior is in test_provider_cascade.""" """Tests for delete_provider method."""
async def test_delete_provider_with_llm_models_raises_error(self):
"""Raises ValueError when LLM models reference provider."""
# Setup
ap = SimpleNamespace()
ap.persistence_mgr = SimpleNamespace()
# Mock LLM model exists - only return LLM result since that's first check
llm_result = _create_mock_result([], first_item=_create_mock_llm_model())
ap.persistence_mgr.execute_async = AsyncMock(return_value=llm_result)
@pytest.mark.parametrize('label', ['LLM', 'Embedding', 'Rerank', None])
async def test_delete_provider_requires_no_references(self, label):
provider_result = Mock()
provider_result.first.return_value = SimpleNamespace(requester='openai')
results = [provider_result]
for model_label in ('LLM', 'Embedding', 'Rerank'):
result = Mock()
result.scalars.return_value = ['model'] if label == model_label else []
results.append(result)
results.extend([Mock(rowcount=1), Mock(rowcount=1)])
ap = SimpleNamespace(
persistence_mgr=SimpleNamespace(
execute_async=AsyncMock(side_effect=results),
tenant_uow=lambda _: nullcontext(),
tenant_scope=lambda _: nullcontext(),
current_session=lambda: None,
),
model_mgr=SimpleNamespace(remove_provider=AsyncMock()),
)
service = ModelProviderService(ap) service = ModelProviderService(ap)
if label is not None:
with pytest.raises(ValueError, match=f'Cannot delete provider: {label} models'): # Execute & Verify
await service.delete_provider(WORKSPACE_UUID, 'provider') with pytest.raises(ValueError, match='Cannot delete provider: LLM models'):
ap.model_mgr.remove_provider.assert_not_awaited() await service.delete_provider(WORKSPACE_UUID, 'provider-with-llm')
else:
await service.delete_provider(WORKSPACE_UUID, 'provider') async def test_delete_provider_with_embedding_models_raises_error(self):
ap.model_mgr.remove_provider.assert_awaited_once_with(WORKSPACE_UUID, 'provider') """Raises ValueError when Embedding models reference provider."""
# Setup
ap = SimpleNamespace()
ap.persistence_mgr = SimpleNamespace()
# Create results for each check type
llm_result = Mock()
llm_result.first = Mock(return_value=None) # No LLM models
embedding_result = Mock()
embedding_result.first = Mock(return_value=Mock(spec=EmbeddingModel)) # Has embedding model
rerank_result = Mock()
rerank_result.first = Mock(return_value=None)
call_count = 0
async def mock_execute(query):
nonlocal call_count
call_count += 1
if call_count == 1:
return llm_result
elif call_count == 2:
return embedding_result
return rerank_result
ap.persistence_mgr.execute_async = AsyncMock(side_effect=mock_execute)
service = ModelProviderService(ap)
# Execute & Verify - should raise embedding error (LLM check passes, embedding check fails)
with pytest.raises(ValueError, match='Cannot delete provider: Embedding models'):
await service.delete_provider(WORKSPACE_UUID, 'provider-with-embedding')
async def test_delete_provider_with_rerank_models_raises_error(self):
"""Raises ValueError when Rerank models reference provider."""
# Setup
ap = SimpleNamespace()
ap.persistence_mgr = SimpleNamespace()
# Create results for each check type
llm_result = Mock()
llm_result.first = Mock(return_value=None) # No LLM models
embedding_result = Mock()
embedding_result.first = Mock(return_value=None) # No embedding models
rerank_result = Mock()
rerank_result.first = Mock(return_value=Mock(spec=RerankModel)) # Has rerank model
call_count = 0
async def mock_execute(query):
nonlocal call_count
call_count += 1
if call_count == 1:
return llm_result
elif call_count == 2:
return embedding_result
return rerank_result
ap.persistence_mgr.execute_async = AsyncMock(side_effect=mock_execute)
service = ModelProviderService(ap)
# Execute & Verify - should raise rerank error (LLM and embedding checks pass, rerank check fails)
with pytest.raises(ValueError, match='Cannot delete provider: Rerank models'):
await service.delete_provider(WORKSPACE_UUID, 'provider-with-rerank')
async def test_delete_provider_no_models_success(self):
"""Deletes provider when no models reference it."""
# Setup
ap = SimpleNamespace()
ap.persistence_mgr = SimpleNamespace()
ap.model_mgr = SimpleNamespace()
ap.model_mgr.remove_provider = AsyncMock()
# Mock no models reference provider
empty_result = Mock()
empty_result.first = Mock(return_value=None)
ap.persistence_mgr.execute_async = AsyncMock(return_value=empty_result)
service = ModelProviderService(ap)
# Execute
await service.delete_provider(WORKSPACE_UUID, 'provider-no-models')
# Verify - delete and remove called
ap.model_mgr.remove_provider.assert_called_once_with(WORKSPACE_UUID, 'provider-no-models')
class TestModelProviderServiceGetProviderModelCounts: class TestModelProviderServiceGetProviderModelCounts:
@@ -969,18 +1045,15 @@ class TestCloudManagedProviderProtection:
async def test_cloud_rejects_update_and_delete_of_managed_provider(self): async def test_cloud_rejects_update_and_delete_of_managed_provider(self):
service = self._service() service = self._service()
service.get_provider = AsyncMock(return_value={'uuid': 'system-provider', 'requester': SYSTEM_REQUESTER}) service.get_provider = AsyncMock(
return_value={'uuid': 'system-provider', 'requester': SYSTEM_REQUESTER}
)
with pytest.raises(ValueError, match='managed by Cloud'): with pytest.raises(ValueError, match='managed by Cloud'):
await service.update_provider(WORKSPACE_UUID, 'system-provider', {'name': 'Renamed'}) await service.update_provider(WORKSPACE_UUID, 'system-provider', {'name': 'Renamed'})
service.ap.persistence_mgr.execute_async.assert_not_awaited()
service.ap.persistence_mgr.tenant_uow = lambda _: nullcontext()
result = Mock()
result.first.return_value = SimpleNamespace(requester=SYSTEM_REQUESTER)
service.ap.persistence_mgr.execute_async.return_value = result
with pytest.raises(ValueError, match='managed by Cloud'): with pytest.raises(ValueError, match='managed by Cloud'):
await service.delete_provider(WORKSPACE_UUID, 'system-provider') await service.delete_provider(WORKSPACE_UUID, 'system-provider')
assert service.ap.persistence_mgr.execute_async.await_count == 1 service.ap.persistence_mgr.execute_async.assert_not_awaited()
async def test_oss_does_not_reserve_space_requester(self): async def test_oss_does_not_reserve_space_requester(self):
ap = SimpleNamespace(persistence_mgr=SimpleNamespace(mode=SimpleNamespace(value='oss_compat'))) ap = SimpleNamespace(persistence_mgr=SimpleNamespace(mode=SimpleNamespace(value='oss_compat')))
@@ -1,182 +0,0 @@
from __future__ import annotations
import copy
from types import SimpleNamespace
from unittest.mock import AsyncMock
import pytest
import quart
from langbot.pkg.api.http.controller.groups.provider.models import (
EmbeddingModelsRouterGroup,
LLMModelsRouterGroup,
RerankModelsRouterGroup,
)
from langbot.pkg.api.http.controller.groups.provider.providers import ModelProvidersRouterGroup
from langbot.pkg.api.http.controller.groups.provider.query import resolve_include_secret
from langbot.pkg.api.http.service.secrets import redact_secrets
pytestmark = pytest.mark.asyncio
RAW_PROVIDER = {
'uuid': 'provider-test',
'name': 'Test Provider',
'api_keys': ['provider-secret'],
}
RAW_MODEL = {
'uuid': 'model-test',
'name': 'Test Model',
'extra_args': {'headers': {'Authorization': 'Bearer model-secret'}},
}
def _access(role: str):
return SimpleNamespace(
execution=SimpleNamespace(instance_uuid='instance-test', placement_generation=1),
workspace=SimpleNamespace(uuid='workspace-test'),
membership=SimpleNamespace(uuid='membership-test', role=role, projection_revision=1),
)
def _project(value: dict, include_secret: bool) -> dict:
value = copy.deepcopy(value)
return value if include_secret else redact_secrets(value)
async def _create_client(role: str):
application = SimpleNamespace()
account = SimpleNamespace(uuid='account-test', user='test@example.com')
application.user_service = SimpleNamespace(get_authenticated_account=AsyncMock(return_value=account))
application.apikey_service = SimpleNamespace(authenticate_api_key=AsyncMock(return_value=None))
application.workspace_collaboration_service = SimpleNamespace(
resolve_account_workspace=AsyncMock(return_value=_access(role))
)
async def get_providers(_context, *, include_secret=False):
return [_project(RAW_PROVIDER, include_secret)]
async def get_provider(_context, _uuid, *, include_secret=False):
return _project(RAW_PROVIDER, include_secret)
application.provider_service = SimpleNamespace(
get_providers=AsyncMock(side_effect=get_providers),
get_provider=AsyncMock(side_effect=get_provider),
get_provider_model_counts=AsyncMock(
return_value={'llm_count': 1, 'embedding_count': 1, 'rerank_count': 1}
),
)
def model_service(list_name: str, get_name: str):
async def get_models(_context, *, include_secret=False):
return [_project(RAW_MODEL, include_secret)]
async def get_model(_context, _uuid, *, include_secret=False):
return _project(RAW_MODEL, include_secret)
return SimpleNamespace(
**{
list_name: AsyncMock(side_effect=get_models),
get_name: AsyncMock(side_effect=get_model),
}
)
application.llm_model_service = model_service('get_llm_models', 'get_llm_model')
application.embedding_models_service = model_service('get_embedding_models', 'get_embedding_model')
application.rerank_models_service = model_service('get_rerank_models', 'get_rerank_model')
quart_app = quart.Quart(__name__)
for router_type in (
ModelProvidersRouterGroup,
LLMModelsRouterGroup,
EmbeddingModelsRouterGroup,
RerankModelsRouterGroup,
):
await router_type(application, quart_app).initialize()
return application, quart_app.test_client()
def _headers() -> dict[str, str]:
return {'Authorization': 'Bearer test-token'}
@pytest.mark.parametrize(
('raw_value', 'permitted', 'expected', 'error'),
[
(None, True, True, None),
(None, False, False, None),
('false', True, False, None),
('true', True, True, None),
('true', False, False, None),
('invalid', True, False, 'include_secret must be either true or false'),
],
)
def test_resolve_include_secret(raw_value, permitted, expected, error):
assert resolve_include_secret(raw_value, permitted=permitted) == (expected, error)
@pytest.mark.parametrize(
'endpoint',
[
'/api/v1/provider/providers',
'/api/v1/provider/models/llm',
'/api/v1/provider/models/embedding',
'/api/v1/provider/models/rerank',
],
)
async def test_default_preserves_secrets_and_explicit_false_redacts_high_permission_reads(endpoint):
application, client = await _create_client('developer')
default_response = await client.get(endpoint, headers=_headers())
false_response = await client.get(f'{endpoint}?include_secret=false', headers=_headers())
assert default_response.status_code == 200
assert false_response.status_code == 200
default_data = await default_response.get_json()
false_data = await false_response.get_json()
default_value = default_data['data'].get('providers', default_data['data'].get('models'))[0]
false_value = false_data['data'].get('providers', false_data['data'].get('models'))[0]
assert '***' not in str(default_value)
assert '***' in str(false_value)
@pytest.mark.parametrize(
'endpoint',
[
'/api/v1/provider/providers',
'/api/v1/provider/models/llm',
'/api/v1/provider/models/embedding',
'/api/v1/provider/models/rerank',
],
)
async def test_explicit_true_does_not_grant_low_permission_reads(endpoint):
_application, client = await _create_client('viewer')
response = await client.get(f'{endpoint}?include_secret=true', headers=_headers())
assert response.status_code == 200
data = await response.get_json()
value = data['data'].get('providers', data['data'].get('models'))[0]
assert '***' in str(value)
@pytest.mark.parametrize(
'endpoint',
[
'/api/v1/provider/providers',
'/api/v1/provider/providers/provider-test',
'/api/v1/provider/models/llm',
'/api/v1/provider/models/llm/model-test',
'/api/v1/provider/models/embedding',
'/api/v1/provider/models/embedding/model-test',
'/api/v1/provider/models/rerank',
'/api/v1/provider/models/rerank/model-test',
],
)
async def test_invalid_include_secret_returns_bad_request(endpoint):
_application, client = await _create_client('developer')
response = await client.get(f'{endpoint}?include_secret=maybe', headers=_headers())
assert response.status_code == 400
assert (await response.get_json())['msg'] == 'include_secret must be either true or false'
@@ -1,302 +0,0 @@
"""Regression tests for recovery-key hardening (#2392).
Covers two attack surfaces reported in GHSA-4xcp-6758-rxqv:
1. ``genkeys.py`` generated ``system.recovery_key`` with only 24 bits of
entropy (``secrets.token_hex(3)``), making the whole keyspace brute-forceable.
2. ``POST /api/v1/user/reset-password`` (unauthenticated) checked its failure
counter across ``await`` points, so concurrent guesses all passed the gate
before any accounting happened; admission is now a synchronous fixed-window
quota consumed at entry, plus constant-time key comparison.
"""
from __future__ import annotations
import asyncio
import logging
import time
from types import SimpleNamespace
from unittest.mock import AsyncMock
import pytest
import quart
from langbot.pkg.api.http.controller.groups import user as user_module
from langbot.pkg.api.http.controller.groups.user import UserRouterGroup
from langbot.pkg.core.stages.genkeys import GenKeysStage
pytestmark = pytest.mark.asyncio
STORED_KEY = 'ABCD2345'
@pytest.fixture(autouse=True)
def _reset_quota_state():
"""Reset the module-level admission-quota state before each test."""
user_module._reset_password_state['window_started_at'] = 0.0
user_module._reset_password_state['attempts'] = 0
yield
user_module._reset_password_state['window_started_at'] = 0.0
user_module._reset_password_state['attempts'] = 0
@pytest.fixture(autouse=True)
def _fast_sleep(monkeypatch):
"""Neutralize the fixed 3s delay so tests run instantly."""
monkeypatch.setattr(user_module, 'asyncio', SimpleNamespace(sleep=AsyncMock()))
# ---------------------------------------------------------------------------
# genkeys.py: recovery-key generation and compatibility
# ---------------------------------------------------------------------------
def _make_genkeys_ap(existing_key: str) -> SimpleNamespace:
"""Build a minimal Application mock for GenKeysStage.
Mirrors the real boot order: no ``logger`` attribute is set because
GenKeysStage runs before SetupLoggerStage.
"""
return SimpleNamespace(
instance_config=SimpleNamespace(
data={'system': {'jwt': {'secret': 'jwt-secret'}, 'recovery_key': existing_key}},
dump_config=AsyncMock(),
),
)
async def test_recovery_key_generation_is_short_and_unambiguous():
"""Eight random base32 characters balance manual entry and online throttling."""
ap = _make_genkeys_ap(existing_key='')
await GenKeysStage().run(ap)
key = ap.instance_config.data['system']['recovery_key']
assert len(key) == 8
assert set(key) <= set('23456789ABCDEFGHJKLMNPQRSTUVWXYZ')
assert ap.instance_config.dump_config.called
async def test_legacy_low_entropy_key_preserved_with_warning(caplog):
"""A legacy 6-char key must keep working but emit a warning, without ap.logger."""
ap = _make_genkeys_ap(existing_key='ABC123')
with caplog.at_level(logging.WARNING, logger='langbot.pkg.core.stages.genkeys'):
await GenKeysStage().run(ap)
assert ap.instance_config.data['system']['recovery_key'] == 'ABC123'
assert any('Low-entropy' in record.message for record in caplog.records)
assert not ap.instance_config.dump_config.called
@pytest.mark.parametrize('existing_key', ['ABC123', 'ABCD2345', 'aB-_' * 10 + 'xYz', '自定义恢复密钥'])
async def test_recovery_key_generation_preserves_existing_key(existing_key):
"""An explicitly configured recovery key must not be regenerated on boot."""
ap = _make_genkeys_ap(existing_key=existing_key)
await GenKeysStage().run(ap)
assert ap.instance_config.data['system']['recovery_key'] == existing_key
assert not ap.instance_config.dump_config.called
async def test_generated_key_is_preserved_without_legacy_warning(caplog):
"""A restart must not warn about or replace the new eight-character key."""
ap = _make_genkeys_ap(existing_key='')
await GenKeysStage().run(ap)
key = ap.instance_config.data['system']['recovery_key']
assert len(key) == 8
ap.instance_config.dump_config.reset_mock()
with caplog.at_level(logging.WARNING, logger='langbot.pkg.core.stages.genkeys'):
await GenKeysStage().run(ap)
assert ap.instance_config.data['system']['recovery_key'] == key
assert not caplog.records
ap.instance_config.dump_config.assert_not_awaited()
async def test_eight_character_key_does_not_trigger_legacy_warning(caplog):
ap = _make_genkeys_ap(existing_key='ABCD2345')
with caplog.at_level(logging.WARNING, logger='langbot.pkg.core.stages.genkeys'):
await GenKeysStage().run(ap)
assert not caplog.records
# ---------------------------------------------------------------------------
# POST /api/v1/user/reset-password: admission quota + constant-time compare
# ---------------------------------------------------------------------------
async def _create_client(stored_key: str = STORED_KEY):
"""Create a Quart test client with a mocked Application."""
quart_app = quart.Quart(__name__)
user_obj = SimpleNamespace(uuid='user-uuid', user='admin@example.com')
reset_password = AsyncMock()
get_user_by_email = AsyncMock(return_value=user_obj)
ap = SimpleNamespace(
user_service=SimpleNamespace(
is_initialized=AsyncMock(return_value=True),
get_user_by_email=get_user_by_email,
reset_password=reset_password,
),
instance_config=SimpleNamespace(
data={'system': {'recovery_key': stored_key}},
),
)
router = UserRouterGroup(ap, quart_app)
await router.initialize()
client = quart_app.test_client()
return client, reset_password, get_user_by_email
def _payload(key: str = STORED_KEY) -> dict:
return {'user': 'admin@example.com', 'recovery_key': key, 'new_password': 'NewPass1!'}
@pytest.mark.parametrize('key', [STORED_KEY, 'ABC123', 'aB-_' * 10 + 'xYz', '自定义恢复密钥'])
async def test_correct_key_resets_password(key):
"""New, legacy and explicitly configured keys all remain usable verbatim."""
client, reset_password, _ = await _create_client(stored_key=key)
resp = await client.post('/api/v1/user/reset-password', json=_payload(key))
assert resp.status_code == 200
assert (await resp.get_json())['code'] == 0
reset_password.assert_awaited_once_with('admin@example.com', 'NewPass1!')
async def test_wrong_key_rejected_without_reset():
"""A wrong recovery key returns 403 and never touches the password."""
client, reset_password, _ = await _create_client()
resp = await client.post('/api/v1/user/reset-password', json=_payload(key='WRONG'))
assert resp.status_code == 403
reset_password.assert_not_awaited()
async def test_non_string_recovery_key_does_not_crash():
"""Malformed recovery-key payloads must be rejected, not raise a 500.
Constant-time comparison via hmac.compare_digest on bytes requires the
input to be a str; other JSON types must fail closed.
"""
client, reset_password, _ = await _create_client()
resp = await client.post(
'/api/v1/user/reset-password',
json={'user': 'admin@example.com', 'recovery_key': 12345, 'new_password': 'NewPass1!'},
)
assert resp.status_code == 403
reset_password.assert_not_awaited()
@pytest.mark.parametrize('key', ['奇数密钥不是ASCII', '\ud800', '\udfff'])
async def test_non_ascii_recovery_key_does_not_crash(key):
"""Non-ASCII keys must compare safely (encode-based constant-time compare)."""
client, _, _ = await _create_client()
resp = await client.post(
'/api/v1/user/reset-password',
json={'user': 'admin@example.com', 'recovery_key': key, 'new_password': 'NewPass1!'},
)
assert resp.status_code == 403
async def test_quota_exhausted_after_max_attempts():
"""After MAX admitted attempts even a correct key must be rejected with 429 (#2392).
Every admission consumes quota regardless of outcome; the legacy endpoint
accepted every guess independently, exhausting the 24-bit keyspace via bursts.
"""
client, reset_password, _ = await _create_client()
for _ in range(user_module._MAX_RESET_ATTEMPTS_PER_WINDOW):
resp = await client.post('/api/v1/user/reset-password', json=_payload(key='WRONG'))
assert resp.status_code == 403
# The very next request carries the CORRECT key but has no quota left.
resp = await client.post('/api/v1/user/reset-password', json=_payload())
assert resp.status_code == 429
reset_password.assert_not_awaited()
async def test_quota_rejects_before_touching_user_lookup():
"""An exhausted quota must reject early, before the sleep and any service calls."""
client, _, get_user_by_email = await _create_client()
user_module._reset_password_state['attempts'] = user_module._MAX_RESET_ATTEMPTS_PER_WINDOW
user_module._reset_password_state['window_started_at'] = time.monotonic()
resp = await client.post('/api/v1/user/reset-password', json=_payload())
assert resp.status_code == 429
get_user_by_email.assert_not_awaited()
async def test_window_rolls_over_and_admits_again():
"""Once the fixed window elapses, the quota resets and a correct key works again."""
client, reset_password, _ = await _create_client()
user_module._reset_password_state['attempts'] = user_module._MAX_RESET_ATTEMPTS_PER_WINDOW
user_module._reset_password_state['window_started_at'] = time.monotonic() - user_module._RESET_WINDOW_SECONDS - 1
resp = await client.post('/api/v1/user/reset-password', json=_payload())
assert resp.status_code == 200
reset_password.assert_awaited_once()
async def test_success_does_not_restore_quota():
"""A successful reset does NOT restore quota: brute-force budget survives wins (#2392).
The legacy clear-on-success let attackers interleave correct-looking states;
success only proves knowledge of the key once, it must not refill attempts.
"""
client, _, _ = await _create_client()
for _ in range(user_module._MAX_RESET_ATTEMPTS_PER_WINDOW - 1):
resp = await client.post('/api/v1/user/reset-password', json=_payload(key='WRONG'))
assert resp.status_code == 403
# Last slot is spent on the genuine reset.
resp = await client.post('/api/v1/user/reset-password', json=_payload())
assert resp.status_code == 200
# Quota is exhausted; even a correct key waits for the next window.
resp = await client.post('/api/v1/user/reset-password', json=_payload())
assert resp.status_code == 429
async def test_concurrent_burst_cannot_bypass_quota(monkeypatch):
"""A 20-request burst yields exactly {403: 5, 429: 15} (#2392 regression).
The vulnerable version accounted failures after several awaits, letting all
concurrent requests pass the gate ({403: 20}). Admission is now synchronous
and await-free, so total admissions are capped regardless of scheduling.
"""
# Swap the AsyncMock sleep for a real cooperative yield so tasks actually
# interleave mid-handler like they do under production load.
async def _yield_sleep(_seconds):
await asyncio.sleep(0)
monkeypatch.setattr(user_module, 'asyncio', SimpleNamespace(sleep=_yield_sleep))
client, reset_password, _ = await _create_client()
responses = await asyncio.gather(
*(client.post('/api/v1/user/reset-password', json=_payload(key='WRONG')) for _ in range(20))
)
status_counts: dict[int, int] = {}
for resp in responses:
status_counts[resp.status_code] = status_counts.get(resp.status_code, 0) + 1
assert status_counts == {403: 5, 429: 15}
reset_password.assert_not_awaited()
+21 -40
View File
@@ -41,6 +41,7 @@ from langbot_plugin.box.security import (
from langbot_plugin.entities.io.context import ActionContext from langbot_plugin.entities.io.context import ActionContext
from langbot.pkg.api.http.context import ExecutionContext from langbot.pkg.api.http.context import ExecutionContext
from langbot.pkg.box.service import BoxService from langbot.pkg.box.service import BoxService
from langbot.pkg.provider.tools.loaders import skill as skill_loader
_UTC = dt.timezone.utc _UTC = dt.timezone.utc
_CONTEXT = ExecutionContext( _CONTEXT = ExecutionContext(
@@ -301,7 +302,7 @@ class TestSharesFilesystemWithBox:
- stdio (local child process) shared filesystem True - stdio (local child process) shared filesystem True
- WebSocket (Docker / sidecar / --standalone-box / remote) separated False - WebSocket (Docker / sidecar / --standalone-box / remote) separated False
This drives whether LangBot validates Box-reported skill paths locally. This drives whether LangBot can safely perform local workspace operations.
Getting it wrong silently drops every skill in separated deployments. Getting it wrong silently drops every skill in separated deployments.
""" """
@@ -338,7 +339,7 @@ class TestSharesFilesystemWithBox:
def test_false_when_client_injected_without_connector(self): def test_false_when_client_injected_without_connector(self):
# Injected client (no connector) → unknown topology → conservative False # Injected client (no connector) → unknown topology → conservative False
# so LangBot never wrongly drops Box-reported skills. # so LangBot does not assume a shared local filesystem.
service = BoxService(make_app(Mock()), client=Mock(spec=BoxRuntimeClient)) service = BoxService(make_app(Mock()), client=Mock(spec=BoxRuntimeClient))
assert service._runtime_connector is None assert service._runtime_connector is None
@@ -552,7 +553,6 @@ async def test_box_service_reconnect_restores_workspace_and_runs_cleanup(
monkeypatch: pytest.MonkeyPatch, monkeypatch: pytest.MonkeyPatch,
): ):
app = make_app(Mock()) app = make_app(Mock())
app.skill_mgr = SimpleNamespace(reload_skills=AsyncMock())
service = BoxService(app, client=Mock(spec=BoxRuntimeClient)) service = BoxService(app, client=Mock(spec=BoxRuntimeClient))
connector = Mock() connector = Mock()
connector.reconnect = AsyncMock() connector.reconnect = AsyncMock()
@@ -565,16 +565,14 @@ async def test_box_service_reconnect_restores_workspace_and_runs_cleanup(
connector.reconnect.assert_awaited_once() connector.reconnect.assert_awaited_once()
service._ensure_default_workspace.assert_called_once() service._ensure_default_workspace.assert_called_once()
service._purge_attachment_dirs.assert_awaited_once() service._purge_attachment_dirs.assert_awaited_once()
app.skill_mgr.reload_skills.assert_awaited_once()
assert service.available is True assert service.available is True
@pytest.mark.asyncio @pytest.mark.asyncio
async def test_cloud_box_service_reconnect_does_not_reload_unscoped_skills( async def test_cloud_box_service_reconnect_restores_runtime_only(
monkeypatch: pytest.MonkeyPatch, monkeypatch: pytest.MonkeyPatch,
): ):
app = make_app(Mock()) app = make_app(Mock())
app.skill_mgr = SimpleNamespace(reload_skills=AsyncMock())
service = BoxService(app, client=Mock(spec=BoxRuntimeClient)) service = BoxService(app, client=Mock(spec=BoxRuntimeClient))
service._cloud_managed = True service._cloud_managed = True
connector = Mock() connector = Mock()
@@ -587,7 +585,6 @@ async def test_cloud_box_service_reconnect_does_not_reload_unscoped_skills(
connector.reconnect.assert_awaited_once() connector.reconnect.assert_awaited_once()
service._verify_cloud_runtime.assert_awaited_once() service._verify_cloud_runtime.assert_awaited_once()
app.skill_mgr.reload_skills.assert_not_awaited()
assert service.available is True assert service.available is True
@@ -1941,7 +1938,7 @@ def test_disconnect_callback_does_not_schedule_without_running_event_loop():
assert service._reconnecting is False assert service._reconnecting is False
class TestBuildSkillExtraMounts: class TestBuildSkillExecutionMounts:
"""Robustness of skill mount construction against a stale skill cache. """Robustness of skill mount construction against a stale skill cache.
The three sandbox backends behave inconsistently when a skill's The three sandbox backends behave inconsistently when a skill's
@@ -1951,16 +1948,10 @@ class TestBuildSkillExtraMounts:
the backend never sees a bad mount. the backend never sees a bad mount.
""" """
def _make_service(self, logger, skills, *, shares_filesystem=True): def _make_app(self, logger, skills):
app = make_app(logger) app = make_app(logger)
app.skill_mgr = SimpleNamespace(skills=skills, get_skills=Mock(return_value=skills)) app.skill_mgr = SimpleNamespace(skills=skills, get_skills=Mock(return_value=skills))
client = Mock(spec=BoxRuntimeClient) return app
service = BoxService(app, client=client)
# Tests construct BoxService with an injected client (no connector), so
# set the topology explicitly. Most cases exercise the shared-fs (local
# stdio) path where local package_root validation applies.
service._shares_filesystem_with_box_override = shares_filesystem
return service
def test_skips_skill_with_missing_package_root(self): def test_skips_skill_with_missing_package_root(self):
logger = Mock() logger = Mock()
@@ -1969,16 +1960,16 @@ class TestBuildSkillExtraMounts:
'alive': {'name': 'alive', 'package_root': live_dir}, 'alive': {'name': 'alive', 'package_root': live_dir},
'ghost': {'name': 'ghost', 'package_root': '/nonexistent/path/should/never/exist'}, 'ghost': {'name': 'ghost', 'package_root': '/nonexistent/path/should/never/exist'},
} }
service = self._make_service(logger, skills) app = self._make_app(logger, skills)
query = make_query() query = make_query()
mounts = service.build_skill_extra_mounts(query) mounts = skill_loader.build_execution_mounts(app, query)
assert mounts == [ assert mounts == [
{ {
'host_path': live_dir, 'host_path': live_dir,
'mount_path': '/workspace/.skills/alive', 'mount_path': '/workspace/.skills/alive',
'mode': 'rw', 'mode': 'ro',
} }
] ]
# Warning logged so operators can see what was dropped # Warning logged so operators can see what was dropped
@@ -1987,27 +1978,19 @@ class TestBuildSkillExtraMounts:
for call in logger.warning.call_args_list for call in logger.warning.call_args_list
) )
def test_trusts_box_paths_when_filesystem_not_shared(self): def test_rejects_missing_core_paths_when_filesystem_not_shared(self):
"""In separated deployments (Docker Compose, k8s sidecar, """Core owns package paths even when Box is a separate process."""
--standalone-box, remote endpoint) the Box runtime owns its own
filesystem. package_root values it reports are NOT resolvable on the
LangBot side, so LangBot must trust them rather than dropping every
skill via a local isdir() check."""
logger = Mock() logger = Mock()
skills = { skills = {
'a': {'name': 'a', 'package_root': '/box/skills/a'}, 'a': {'name': 'a', 'package_root': '/box/skills/a'},
'b': {'name': 'b', 'package_root': '/box/skills/b'}, 'b': {'name': 'b', 'package_root': '/box/skills/b'},
} }
service = self._make_service(logger, skills, shares_filesystem=False) app = self._make_app(logger, skills)
mounts = service.build_skill_extra_mounts(make_query()) mounts = skill_loader.build_execution_mounts(app, make_query())
assert mounts == [ assert mounts == []
{'host_path': '/box/skills/a', 'mount_path': '/workspace/.skills/a', 'mode': 'rw'}, assert len(logger.warning.call_args_list) == 2
{'host_path': '/box/skills/b', 'mount_path': '/workspace/.skills/b', 'mode': 'rw'},
]
# No skill is dropped, so no "missing" warning should be logged.
assert not any('package_root missing' in str(call.args[0]) for call in logger.warning.call_args_list)
def test_skips_skill_with_empty_package_root(self): def test_skips_skill_with_empty_package_root(self):
logger = Mock() logger = Mock()
@@ -2015,25 +1998,23 @@ class TestBuildSkillExtraMounts:
'no_root': {'name': 'no_root', 'package_root': ''}, 'no_root': {'name': 'no_root', 'package_root': ''},
'whitespace': {'name': 'whitespace', 'package_root': ' '}, 'whitespace': {'name': 'whitespace', 'package_root': ' '},
} }
service = self._make_service(logger, skills) app = self._make_app(logger, skills)
assert service.build_skill_extra_mounts(make_query()) == [] assert skill_loader.build_execution_mounts(app, make_query()) == []
def test_empty_package_root_skipped_even_when_not_shared(self): def test_empty_package_root_skipped_even_when_not_shared(self):
"""An empty package_root is always invalid regardless of topology.""" """An empty package_root is always invalid regardless of topology."""
logger = Mock() logger = Mock()
skills = {'no_root': {'name': 'no_root', 'package_root': ''}} skills = {'no_root': {'name': 'no_root', 'package_root': ''}}
service = self._make_service(logger, skills, shares_filesystem=False) app = self._make_app(logger, skills)
assert service.build_skill_extra_mounts(make_query()) == [] assert skill_loader.build_execution_mounts(app, make_query()) == []
def test_returns_empty_when_no_skill_manager(self): def test_returns_empty_when_no_skill_manager(self):
logger = Mock() logger = Mock()
app = make_app(logger) app = make_app(logger)
# no skill_mgr attribute # no skill_mgr attribute
service = BoxService(app, client=Mock(spec=BoxRuntimeClient)) assert skill_loader.build_execution_mounts(app, make_query()) == []
assert service.build_skill_extra_mounts(make_query()) == []
# ── Attachment passthrough (inbound / outbound) ───────────────────────────── # ── Attachment passthrough (inbound / outbound) ─────────────────────────────
+1 -1
View File
@@ -13,8 +13,8 @@ from langbot.pkg.box.workspace import (
classify_python_workspace, classify_python_workspace,
infer_workspace_host_path, infer_workspace_host_path,
rewrite_mounted_path, rewrite_mounted_path,
wrap_python_command_with_env,
) )
from langbot.pkg.utils.python_workspace import wrap_python_command_with_env
_CONTEXT = ExecutionContext( _CONTEXT = ExecutionContext(
@@ -1,20 +0,0 @@
"""Keep subprocess coverage pointed at the generated E2E configuration."""
from pathlib import Path
from unittest.mock import Mock, patch
from tests.e2e.utils.process_manager import LangBotProcess
def test_e2e_coverage_environment_uses_generated_config(tmp_path):
process = Mock()
process.poll.return_value = None
project = tmp_path / 'project'
project.mkdir()
manager = LangBotProcess(project, tmp_path, collect_coverage=True)
with patch('subprocess.Popen', return_value=process) as popen, patch('httpx.get') as get:
get.return_value.status_code = 200
assert manager.start()
config = Path(popen.call_args.kwargs['env']['COVERAGE_PROCESS_START'])
assert config.is_file()
assert f'--rcfile={config}' in popen.call_args.args[0]
-124
View File
@@ -338,70 +338,6 @@ class TestTaskWrapper:
assert result['runtime']['exception'] == 'Test error' assert result['runtime']['exception'] == 'Test error'
assert 'exception_traceback' in result['runtime'] assert 'exception_traceback' in result['runtime']
@pytest.mark.asyncio
async def test_public_dict_has_stable_success_projection(self):
_, TaskWrapper, _ = get_taskmgr_classes()
mock_app = create_mock_app()
async def successful_coro():
return {'file_id': 'file-a'}
wrapper = TaskWrapper(mock_app, successful_coro(), kind='knowledge_base.store')
await wrapper.task
result = wrapper.to_public_dict()
assert result == {
'id': wrapper.id,
'task_type': 'system',
'kind': 'knowledge_base.store',
'status': 'succeeded',
'error': None,
'result': {'file_id': 'file-a'},
'created_at': result['created_at'],
}
assert 'runtime' not in result
assert 'traceback' not in str(result).lower()
@pytest.mark.asyncio
async def test_public_dict_hides_exception_traceback(self):
_, TaskWrapper, _ = get_taskmgr_classes()
mock_app = create_mock_app()
async def failing_coro():
raise ValueError('private failure')
wrapper = TaskWrapper(mock_app, failing_coro())
try:
await wrapper.task
except ValueError:
# Expected failure: task must complete in failed state for public serialization checks.
pass
result = wrapper.to_public_dict()
assert result['status'] == 'failed'
assert result['error'] == {'type': 'task_failed', 'message': 'Task execution failed'}
assert 'runtime' not in result
assert 'traceback' not in str(result).lower()
@pytest.mark.asyncio
async def test_public_dict_does_not_change_success_when_result_is_not_json_serializable(self):
_, TaskWrapper, _ = get_taskmgr_classes()
mock_app = create_mock_app()
async def successful_coro():
return object()
wrapper = TaskWrapper(mock_app, successful_coro())
await wrapper.task
result = wrapper.to_public_dict()
assert result['status'] == 'succeeded'
assert result['error'] is None
assert result['result'] is None
@pytest.mark.asyncio @pytest.mark.asyncio
async def test_cancel_task(self): async def test_cancel_task(self):
"""Test cancel method cancels the asyncio task.""" """Test cancel method cancels the asyncio task."""
@@ -551,66 +487,6 @@ class TestAsyncTaskManager:
w2.cancel() w2.cancel()
w3.cancel() w3.cancel()
@pytest.mark.asyncio
async def test_public_task_queries_keep_workspace_and_generation_isolation(self):
_, _, AsyncTaskManager = get_taskmgr_classes()
mock_app = create_mock_app()
manager = AsyncTaskManager(mock_app)
async def dummy_coro():
await asyncio.sleep(10)
current = manager.create_user_task(
dummy_coro(),
instance_uuid='instance-a',
workspace_uuid='workspace-a',
placement_generation=2,
)
other_workspace = manager.create_user_task(
dummy_coro(),
instance_uuid='instance-a',
workspace_uuid='workspace-b',
placement_generation=2,
)
stale_generation = manager.create_user_task(
dummy_coro(),
instance_uuid='instance-a',
workspace_uuid='workspace-a',
placement_generation=1,
)
result = manager.get_tasks_dict(
instance_uuid='instance-a',
workspace_uuid='workspace-a',
placement_generation=2,
public=True,
)
assert [task['id'] for task in result['tasks']] == [current.id]
assert 'id_index' not in result
assert (
manager.get_task_by_id(
other_workspace.id,
instance_uuid='instance-a',
workspace_uuid='workspace-a',
placement_generation=2,
)
is None
)
assert (
manager.get_task_by_id(
stale_generation.id,
instance_uuid='instance-a',
workspace_uuid='workspace-a',
placement_generation=2,
)
is None
)
current.cancel()
other_workspace.cancel()
stale_generation.cancel()
@pytest.mark.asyncio @pytest.mark.asyncio
async def test_cancel_by_scope(self): async def test_cancel_by_scope(self):
"""Test cancel_by_scope cancels matching tasks.""" """Test cancel_by_scope cancels matching tasks."""
@@ -1,115 +0,0 @@
"""Real rollback-journal contention must not poison the pooled writer."""
import asyncio
import contextlib
import sqlite3
from types import SimpleNamespace
import pytest
import sqlalchemy as sa
from sqlalchemy.ext.asyncio import AsyncConnection, AsyncSessionTransaction, create_async_engine
from langbot.pkg.persistence.mgr import PersistenceManager, PersistenceMode
from langbot.pkg.persistence.tenant_uow import TenantScopedAsyncSession
@pytest.mark.asyncio
@pytest.mark.parametrize('cancel_commit', [False, True])
@pytest.mark.parametrize('close_fails', [False, True])
@pytest.mark.parametrize('cancel_cleanup', [False, True])
async def test_failed_commit_releases_sqlite_writer_and_scope(
tmp_path, monkeypatch, cancel_commit, close_fails, cancel_cleanup
):
path = tmp_path / 'failed-commit.db'
engine = create_async_engine(
f'sqlite+aiosqlite:///{path}', connect_args={'timeout': 0.05}, pool_size=1, max_overflow=0
)
table = sa.Table('rows', sa.MetaData(), sa.Column('id', sa.Integer, primary_key=True))
manager = PersistenceManager(object(), mode=PersistenceMode.CLOUD_RUNTIME)
manager.db = SimpleNamespace(get_engine=lambda: engine)
original_commit = AsyncSessionTransaction.commit
original_error = None
invalidation_finished = False
owner = asyncio.current_task()
original_invalidate = AsyncConnection.invalidate
async def delayed_invalidate(connection, exception=None):
nonlocal invalidation_finished
if cancel_cleanup:
owner.cancel()
await asyncio.sleep(0)
owner.cancel()
await asyncio.sleep(0)
await original_invalidate(connection, exception)
invalidation_finished = True
monkeypatch.setattr(AsyncConnection, 'invalidate', delayed_invalidate)
async def failing_commit(transaction):
nonlocal original_error
try:
await original_commit(transaction)
except sa.exc.OperationalError as exc:
original_error = asyncio.CancelledError('commit cancelled') if cancel_commit else exc
raise original_error
monkeypatch.setattr(AsyncSessionTransaction, 'commit', failing_commit)
original_close = TenantScopedAsyncSession._close_owned_session
async def failing_close(session, capability):
await original_close(session, capability)
if original_error is not None:
raise RuntimeError('secondary close failure')
if close_fails:
monkeypatch.setattr(TenantScopedAsyncSession, '_close_owned_session', failing_close)
blocker = sqlite3.connect(path, timeout=0.05)
try:
async with engine.begin() as connection:
await connection.run_sync(table.metadata.create_all)
await connection.execute(sa.insert(table).values(id=1))
blocker.execute('BEGIN')
blocker.execute('SELECT * FROM rows').fetchall()
error_type = asyncio.CancelledError if cancel_commit else sa.exc.OperationalError
with pytest.raises(error_type) as caught:
async with manager.tenant_uow('workspace-a') as outer:
gate = manager.create_after_commit_gate()
state = outer._active_state
async with manager.tenant_uow('workspace-a') as inner:
assert inner.session is outer.session
await manager.execute_async(sa.insert(table).values(id=2))
assert not gate.done()
assert state.depth == 1
assert caught.value is original_error
assert invalidation_finished
if cancel_cleanup:
# Do not leak the synthetic cancellation count into pytest.
owner.uncancel()
owner.uncancel()
monkeypatch.setattr(TenantScopedAsyncSession, '_close_owned_session', original_close)
if close_fails:
assert any('secondary close failure' in note for note in caught.value.__notes__)
assert gate.cancelled()
assert state.depth == 0
assert manager.current_session() is None
with pytest.raises(RuntimeError, match='not active'):
_ = outer.session
# The original SHARED lock remains. New reads and RESERVED writes
# must work; COMMIT of another write must wait for its release.
assert blocker.in_transaction
with contextlib.closing(sqlite3.connect(path, timeout=0.05)) as probe:
assert probe.execute('SELECT id FROM rows').fetchall() == [(1,)]
probe.execute('INSERT INTO rows VALUES (3)')
probe.rollback()
async with manager.tenant_uow('workspace-b'):
assert (await manager.execute_async(sa.select(table.c.id))).scalars().all() == [1]
blocker.rollback()
async with manager.tenant_uow('workspace-b'):
await manager.execute_async(sa.insert(table).values(id=4))
async with engine.connect() as connection:
assert (await connection.execute(sa.select(table.c.id).order_by(table.c.id))).scalars().all() == [1, 4]
assert engine.pool.checkedout() == 0
finally:
blocker.close()
await engine.dispose()
@@ -1,169 +0,0 @@
from types import SimpleNamespace
import pytest
from langbot.pkg.platform.sources.mattermost import (
MattermostAdapter,
MattermostEventConverter,
MattermostMessageConverter,
_normalize_server_url,
_websocket_url,
)
import langbot_plugin.api.entities.builtin.platform.events as platform_events
import langbot_plugin.api.entities.builtin.platform.message as platform_message
class StubLogger:
async def info(self, *_args, **_kwargs):
pass
async def error(self, *_args, **_kwargs):
pass
def _adapter() -> MattermostAdapter:
return MattermostAdapter.model_construct(
config={'enable_stream_reply': True},
logger=StubLogger(),
server_url='https://mattermost.example.com',
access_token='secret',
bot_account_id='bot-id',
bot_username='langbot',
session=None,
listeners={},
channel_cache={},
stream_post_ids={},
_running=False,
)
def test_server_and_websocket_urls_preserve_subpath():
server_url = _normalize_server_url('https://example.com/chat/')
assert server_url == 'https://example.com/chat'
assert _websocket_url(server_url) == 'wss://example.com/chat/api/v4/websocket'
with pytest.raises(ValueError, match='absolute HTTP'):
_normalize_server_url('mattermost.example.com')
@pytest.mark.asyncio
async def test_converter_marks_and_removes_bot_mention():
chain = await MattermostMessageConverter.target2yiri(
{'id': 'post-1', 'create_at': 1_000, 'message': '@langbot hello'},
'langbot',
)
assert any(isinstance(item, platform_message.At) for item in chain)
assert any(isinstance(item, platform_message.Plain) and item.text == 'hello' for item in chain)
@pytest.mark.asyncio
async def test_event_converter_distinguishes_direct_and_group_channels():
post = {'id': 'post-1', 'channel_id': 'channel-1', 'user_id': 'user-1', 'message': 'hello', 'create_at': 1_000}
direct = await MattermostEventConverter.target2yiri(post, {'type': 'D'}, 'alice', 'langbot')
group = await MattermostEventConverter.target2yiri(
post,
{'type': 'O', 'display_name': 'General'},
'alice',
'langbot',
)
assert isinstance(direct, platform_events.FriendMessage)
assert isinstance(group, platform_events.GroupMessage)
assert group.sender.group.name == 'General'
@pytest.mark.asyncio
async def test_send_to_person_creates_or_reuses_direct_channel(monkeypatch):
adapter = _adapter()
requests = []
posted = []
async def api_request(method, path, *, payload=None):
requests.append((method, path, payload))
return {'id': 'direct-channel', 'type': 'D'}
async def post_message(channel_id, text, root_id=''):
posted.append((channel_id, text, root_id))
return {'id': 'post-1'}
monkeypatch.setattr(adapter, '_api_request', api_request)
monkeypatch.setattr(adapter, '_post_message', post_message)
await adapter.send_message('person', 'user-1', platform_message.MessageChain([platform_message.Plain(text='hello')]))
assert requests == [('POST', '/channels/direct', {'user_ids': ['bot-id', 'user-1']})]
assert posted == [('direct-channel', 'hello', '')]
@pytest.mark.asyncio
async def test_reply_keeps_existing_thread(monkeypatch):
adapter = _adapter()
posted = []
async def post_message(channel_id, text, root_id=''):
posted.append((channel_id, text, root_id))
return {'id': 'reply'}
monkeypatch.setattr(adapter, '_post_message', post_message)
event = platform_events.GroupMessage.model_construct(
source_platform_object={
'post': {'id': 'post-1', 'channel_id': 'channel-1', 'root_id': 'thread-root'},
'channel': {'type': 'O'},
}
)
await adapter.reply_message(event, platform_message.MessageChain([platform_message.Plain(text='reply')]))
assert posted == [('channel-1', 'reply', 'thread-root')]
@pytest.mark.asyncio
async def test_stream_reply_updates_existing_post(monkeypatch):
adapter = _adapter()
adapter.stream_post_ids['response-1'] = 'post-1'
requests = []
async def api_request(method, path, *, payload=None):
requests.append((method, path, payload))
return {'id': 'post-1'}
monkeypatch.setattr(adapter, '_api_request', api_request)
message = SimpleNamespace(resp_message_id='response-1', tool_calls=None)
await adapter.reply_message_chunk(
SimpleNamespace(),
message,
platform_message.MessageChain([platform_message.Plain(text='complete')]),
is_final=True,
)
assert requests == [('PUT', '/posts/post-1', {'id': 'post-1', 'message': 'complete'})]
assert 'response-1' not in adapter.stream_post_ids
@pytest.mark.asyncio
async def test_posted_event_dispatches_listener(monkeypatch):
adapter = _adapter()
received = []
async def get_channel(_channel_id):
return {'type': 'D'}
async def listener(event, _adapter):
received.append(event)
monkeypatch.setattr(adapter, '_get_channel', get_channel)
adapter.register_listener(platform_events.FriendMessage, listener)
await adapter._dispatch_post(
{
'data': {
'sender_name': 'alice',
'post': '{"id":"post-1","channel_id":"channel-1","user_id":"user-1","message":"hello","create_at":1000}',
}
}
)
assert len(received) == 1
assert received[0].sender.nickname == 'alice'
-200
View File
@@ -1,200 +0,0 @@
"""Replay synthetic HTTP/SSE traffic through the real Codex requester."""
import json
from collections import OrderedDict
from types import SimpleNamespace
from unittest.mock import AsyncMock
import httpx
import pytest
import langbot_plugin.api.entities.builtin.provider.message as pm
from langbot.pkg.provider.modelmgr.requesters.codex import CodexRequester, sse_events
TOKENS = {'access_token': 'access-secret', 'account_id': 'account', 'connection_id': 'connection'}
MODEL = SimpleNamespace(model_entity=SimpleNamespace(name='codex-test', extra_args={}, reasoning_config=None))
def requester(monkeypatch, handler):
real_client = httpx.AsyncClient
monkeypatch.setattr(
httpx, 'AsyncClient', lambda **kwargs: real_client(transport=httpx.MockTransport(handler), **kwargs)
)
obj = object.__new__(CodexRequester)
obj.workspace, obj.provider = 'w', 'p'
obj._replay = OrderedDict()
obj.auth = SimpleNamespace(access=AsyncMock(return_value=TOKENS))
return obj
def stream(events):
return httpx.Response(200, content=''.join('data: ' + json.dumps(event) + '\r\n\r\n' for event in events))
@pytest.mark.asyncio
async def test_text_tools_usage_and_scoped_opaque_replay(monkeypatch):
call = {'type': 'function_call', 'call_id': 'call_1', 'name': 'lookup', 'arguments': '{"q":"test"}'}
output = [
{'type': 'reasoning', 'encrypted_content': 'opaque-secret'},
call,
{'type': 'message', 'role': 'assistant', 'content': [{'type': 'output_text', 'text': 'Hello'}]},
]
requests = []
def handler(request):
requests.append(request)
return stream(
[
{'type': 'response.created', 'response': {'id': 'resp_1'}},
{'type': 'response.output_text.delta', 'delta': 'Hel'},
{'type': 'response.output_text.delta', 'delta': 'lo'},
{'type': 'response.function_call_arguments.delta', 'delta': '{broken'},
{'type': 'response.output_item.done', 'item': call, 'output_index': 1},
{
'type': 'response.completed',
'response': {
'id': 'resp_1',
'status': 'completed',
'output': output,
'usage': {'input_tokens': 4, 'output_tokens': 3, 'input_tokens_details': {'cached_tokens': 2}},
},
},
]
)
obj = requester(monkeypatch, handler)
query = SimpleNamespace(query_id='q', variables=None)
messages = [pm.Message(role='system', content='Be brief'), pm.Message(role='user', content='Hi')]
message, usage = await obj.invoke_llm(query, MODEL, messages)
assert message.content == 'Hello'
assert len(message.tool_calls) == 1
assert message.tool_calls[0].function.arguments == '{"q":"test"}'
assert usage['total_tokens'] == 7
assert query.variables['_stream_usage'] == usage
assert 'opaque-secret' not in message.model_dump_json()
body = json.loads(requests[0].content)
assert body['store'] is False and body['stream'] is True
assert body['instructions'] == 'Be brief'
assert requests[0].url.path.endswith('/codex/responses')
assert requests[0].headers['authorization'] == 'Bearer access-secret'
assert requests[0].headers['originator'] == 'langbot'
same = obj._body(query, MODEL, [message], None, None, TOKENS)
assert same['input'] == output
other = obj._body(SimpleNamespace(query_id='q'), MODEL, [message], None, None, TOKENS)
assert 'opaque-secret' not in json.dumps(other)
rotated = obj._body(query, MODEL, [message], None, None, {**TOKENS, 'connection_id': 'new'})
assert 'opaque-secret' not in json.dumps(rotated)
@pytest.mark.asyncio
@pytest.mark.parametrize(
'events',
[
[{'type': 'response.failed', 'error': 'access-secret'}],
[{'type': 'response.incomplete'}],
[{'type': 'error'}],
[{'type': 'response.output_text.delta', 'delta': 'partial'}],
[],
],
)
async def test_failure_and_truncated_stream_never_succeed(monkeypatch, events):
obj = requester(monkeypatch, lambda request: stream(events))
with pytest.raises(ValueError) as caught:
await obj.invoke_llm(None, MODEL, [])
assert 'access-secret' not in str(caught.value)
@pytest.mark.asyncio
async def test_terminal_only_text_stream_and_usage(monkeypatch):
obj = requester(
monkeypatch,
lambda request: stream(
[
{
'type': 'response.done',
'response': {
'output': [{'type': 'message', 'content': [{'type': 'output_text', 'text': 'done'}]}],
'usage': {'input_tokens': 2, 'output_tokens': 1},
},
}
]
),
)
query = SimpleNamespace(query_id='q', variables={})
chunks = [chunk async for chunk in obj.invoke_llm_stream(query, MODEL, [])]
assert ''.join(chunk.content or '' for chunk in chunks) == 'done'
assert chunks[-1].is_final
assert query.variables['_stream_usage']['total_tokens'] == 3
@pytest.mark.asyncio
@pytest.mark.parametrize('status', [401, 403, 429, 500])
async def test_http_error_secrecy_and_bounded_401_retry(monkeypatch, status):
requests = []
def handler(request):
requests.append(request)
return httpx.Response(status, text='access-secret refresh-secret')
obj = requester(monkeypatch, handler)
with pytest.raises(ValueError) as caught:
await obj.invoke_llm(None, MODEL, [])
assert 'secret' not in str(caught.value)
assert len(requests) == (2 if status == 401 else 1)
if status == 401:
assert obj.auth.access.call_args.kwargs == {'rejected_token': 'access-secret'}
@pytest.mark.asyncio
async def test_catalog_mapping_filtering_and_deduplication(monkeypatch):
requests = []
def handler(request):
requests.append(request)
return httpx.Response(
200,
json={
'models': [
{'slug': 'a', 'input_modalities': ['text', 'image'], 'supported_reasoning_levels': ['low']},
{'slug': 'hidden', 'visibility': 'hide'},
{'slug': 'a', 'input_modalities': ['text', 'image'], 'supported_reasoning_levels': ['low']},
]
},
)
obj = requester(monkeypatch, handler)
catalog = await obj.scan_models()
assert len(catalog['models']) == 1
assert catalog['models'][0]['abilities'] == ['func_call', 'vision', 'reasoning']
assert catalog['debug'] is None
assert requests[0].url.path.endswith('/codex/models')
assert 'client_version' in requests[0].url.params
@pytest.mark.asyncio
@pytest.mark.parametrize('payload', [{'models': ['secret']}, [], {'models': None}])
async def test_catalog_malformed_safe(monkeypatch, payload):
obj = requester(monkeypatch, lambda request: httpx.Response(200, json=payload))
with pytest.raises(ValueError, match='invalid model catalog'):
await obj.scan_models()
@pytest.mark.asyncio
async def test_sse_multiline_crlf_comments_and_chunk_boundaries():
class Bytes(httpx.AsyncByteStream):
async def __aiter__(self):
for value in b': comment\r\nevent: test\r\ndata: {"type":\r\ndata: "test"}\r\n\r\ndata: [DONE]\r\n\r\n':
yield bytes([value])
response = httpx.Response(200, stream=Bytes())
assert [event async for event in sse_events(response)] == [{'type': 'test'}]
@pytest.mark.parametrize(
'key', ['base_url', 'headers', 'api_key', 'store', 'stream', 'previous_response_id', 'temperature']
)
def test_advanced_parameters_cannot_override_transport(monkeypatch, key):
obj = requester(monkeypatch, lambda request: httpx.Response(200))
with pytest.raises(ValueError, match='Unsupported Codex advanced'):
obj._body(None, MODEL, [], None, {key: 'secret'}, TOKENS)
@@ -1,100 +0,0 @@
from types import SimpleNamespace
from unittest.mock import AsyncMock, Mock
import httpx
import pytest
from quart import Quart
from sqlalchemy.exc import SQLAlchemyError
from langbot.pkg.api.http.controller.groups.provider.models import LLMModelsRouterGroup
from langbot.pkg.api.http.authz import Permission
from langbot.pkg.provider.modelmgr.requesters.codex import CodexRequester
from tests.unit_tests.provider.test_codex import requester, MODEL, stream
CASES = [
(400, 400, 'codex_invalid_request'),
(401, 400, 'codex_reauthentication_required'),
(403, 403, 'codex_access_denied'),
(429, 429, 'codex_rate_limited'),
(500, 502, 'codex_upstream_failure'),
]
@pytest.mark.asyncio
@pytest.mark.parametrize('upstream,status,code', CASES)
async def test_requester_safe_error(monkeypatch, upstream, status, code):
obj = requester(monkeypatch, lambda request: httpx.Response(upstream, text='credential-secret'))
with pytest.raises(Exception) as caught:
await obj.invoke_llm(None, MODEL, [])
error = caught.value
assert getattr(error, 'status_code', None) == status
assert error.error_code == code
assert 'secret' not in str(error)
if upstream == 429:
assert 'rate limit' in str(error).lower()
assert 'usage limit reached' not in str(error).lower()
@pytest.mark.asyncio
@pytest.mark.parametrize('events', [[{'type': 'response.failed', 'error': 'credential-secret'}], []])
async def test_stream_safe_error(monkeypatch, events):
obj = requester(monkeypatch, lambda request: stream(events))
with pytest.raises(Exception) as caught:
await obj.invoke_llm(None, MODEL, [])
assert getattr(caught.value, 'status_code', None) == 502
assert caught.value.error_code == 'codex_upstream_failure'
assert 'secret' not in str(caught.value)
@pytest.mark.asyncio
@pytest.mark.parametrize(
'kind,code', [('usage_limit_reached', 'codex_usage_limit_reached'), ('unknown', 'codex_rate_limited')]
)
async def test_allowlisted_usage_error(monkeypatch, kind, code):
obj = requester(
monkeypatch,
lambda request: httpx.Response(
429, json={'error': {'type': kind, 'message': 'credential-secret', 'resets_at': 1789043289}}
),
)
with pytest.raises(Exception) as caught:
await obj.invoke_llm(None, MODEL, [])
assert caught.value.error_code == code
assert 'secret' not in str(caught.value)
async def client_for(error):
app = Quart(__name__)
ap = SimpleNamespace(logger=Mock(), llm_model_service=SimpleNamespace(test_llm_model=AsyncMock(side_effect=error)))
router = LLMModelsRouterGroup(ap, app)
router._authenticate_api_key = AsyncMock(
return_value=SimpleNamespace(
workspace_uuid='w',
workspace=SimpleNamespace(permissions=frozenset({Permission.PROVIDER_SECRET_MANAGE.value})),
)
)
await router.initialize()
return app.test_client(), ap
@pytest.mark.asyncio
@pytest.mark.parametrize('upstream,status,code', CASES)
async def test_real_model_test_route_safe_error(upstream, status, code):
error = CodexRequester._http_error(upstream)
client, ap = await client_for(error)
response = await client.post('/api/v1/provider/models/llm/model/test', json={}, headers={'X-API-Key': 'synthetic'})
body = await response.get_json()
assert response.status_code == status
assert body['code'] == code
assert body['msg'] == str(error)
ap.logger.error.assert_not_called()
@pytest.mark.asyncio
@pytest.mark.parametrize('error', [ValueError('private-value-secret'), SQLAlchemyError('private-sql-secret')])
async def test_real_model_test_route_unexpected_errors_hidden(error):
client, _ = await client_for(error)
response = await client.post('/api/v1/provider/models/llm/model/test', json={}, headers={'X-API-Key': 'synthetic'})
assert response.status_code == 500
assert 'secret' not in await response.get_data(as_text=True)
@@ -1,147 +0,0 @@
"""Temporary Codex models use saved, tenant-scoped providers and synthetic SQLite credentials."""
import time
from types import SimpleNamespace
import pytest
import pytest_asyncio
import sqlalchemy as sa
from sqlalchemy.ext.asyncio import create_async_engine
from langbot.pkg.entity.persistence.model import CodexCredential, ModelProvider
from langbot.pkg.provider.modelmgr.codex_auth import BASE_URL
from langbot.pkg.provider.modelmgr.modelmgr import ModelManager
from langbot.pkg.provider.modelmgr.requesters.codex import CodexRequester
from langbot.pkg.workspace.errors import WorkspaceNotFoundError
from tests.unit_tests.provider.conftest import (
TEST_EXECUTION_CONTEXT,
TEST_WORKSPACE_UUID,
FakeProviderAPIRequester,
)
@pytest_asyncio.fixture
async def manager(tmp_path, mock_app_for_modelmgr):
engine = create_async_engine(f'sqlite+aiosqlite:///{tmp_path / "temporary-codex.db"}')
async with engine.begin() as conn:
await conn.run_sync(ModelProvider.__table__.create)
await conn.run_sync(CodexCredential.__table__.create)
for uuid, workspace, kind in (
('saved', TEST_WORKSPACE_UUID, 'openai-codex'),
('foreign', 'another-workspace', 'openai-codex'),
('api', TEST_WORKSPACE_UUID, 'fake-requester'),
):
await conn.execute(
sa.insert(ModelProvider).values(
uuid=uuid,
workspace_uuid=workspace,
name='Saved provider',
requester=kind,
base_url=BASE_URL,
api_keys=[],
)
)
await conn.execute(
sa.insert(CodexCredential).values(
provider_uuid='saved',
workspace_uuid=TEST_WORKSPACE_UUID,
payload={
'tokens': {
'access_token': 'synthetic-access',
'refresh_token': 'synthetic-refresh',
'account_id': 'synthetic-account',
'expires_at': time.time() + 3600,
}
},
)
)
async def execute(statement):
async with engine.begin() as conn:
return await conn.execute(statement)
mock_app_for_modelmgr.persistence_mgr = SimpleNamespace(execute_async=execute)
mgr = ModelManager(mock_app_for_modelmgr)
mgr.requester_dict = {'openai-codex': CodexRequester, 'fake-requester': FakeProviderAPIRequester}
try:
yield mgr
finally:
await engine.dispose()
def info(provider_uuid='saved', **inline):
result = {'name': 'codex-test', 'provider': {'requester': 'openai-codex', **inline}}
if provider_uuid is not None:
result['provider_uuid'] = provider_uuid
return result
@pytest.mark.asyncio
@pytest.mark.parametrize(
'inline',
[
{},
{'uuid': 'saved'},
{
'requester': 'fake-requester',
'api_keys': ['untrusted'],
'base_url': 'https://untrusted.invalid',
'workspace_uuid': 'another-workspace',
},
],
)
async def test_codex_temporary_model_resolves_saved_provider_and_real_credentials(manager, inline):
model = await manager.init_temporary_runtime_llm_model(TEST_EXECUTION_CONTEXT, info(**inline))
provider = model.provider
assert provider.provider_entity.uuid == 'saved'
assert provider.provider_entity.requester == 'openai-codex'
assert provider.provider_entity.api_keys == []
assert provider.provider_entity.base_url == BASE_URL
assert isinstance(provider.requester, CodexRequester)
tokens = await provider.requester.auth.access(provider.requester.workspace, provider.requester.provider)
assert tokens['access_token'] == 'synthetic-access'
assert model.model_entity.provider_uuid == 'saved'
@pytest.mark.asyncio
async def test_codex_temporary_model_accepts_inline_saved_identity(manager):
model = await manager.init_temporary_runtime_llm_model(TEST_EXECUTION_CONTEXT, info(None, uuid='saved'))
assert model.provider.provider_entity.name == 'Saved provider'
@pytest.mark.asyncio
@pytest.mark.parametrize('identity', ['missing', 'foreign', None])
async def test_codex_temporary_model_rejects_unavailable_identity(manager, identity):
with pytest.raises(WorkspaceNotFoundError):
await manager.init_temporary_runtime_llm_model(
TEST_EXECUTION_CONTEXT, info(identity, workspace_uuid='another-workspace')
)
@pytest.mark.asyncio
async def test_codex_temporary_model_rejects_non_codex_saved_provider(manager):
with pytest.raises(ValueError):
await manager.init_temporary_runtime_llm_model(TEST_EXECUTION_CONTEXT, info('api'))
@pytest.mark.asyncio
async def test_codex_temporary_model_rejects_conflicting_identities(manager):
with pytest.raises(ValueError):
await manager.init_temporary_runtime_llm_model(TEST_EXECUTION_CONTEXT, info('saved', uuid='foreign'))
@pytest.mark.asyncio
async def test_api_key_temporary_model_preserves_inline_configuration(manager):
model = await manager.init_temporary_runtime_llm_model(
TEST_EXECUTION_CONTEXT,
{
'name': 'api-model',
'provider': {
'requester': 'fake-requester',
'api_keys': ['synthetic-key'],
'base_url': 'https://api.example.invalid',
},
},
)
assert model.provider.provider_entity.api_keys == ['synthetic-key']
assert model.provider.provider_entity.base_url == 'https://api.example.invalid'
@@ -13,15 +13,7 @@ from aiohttp import web
from mcp import types as mcp_types from mcp import types as mcp_types
from langbot.pkg.api.http.context import ExecutionContext from langbot.pkg.api.http.context import ExecutionContext
from langbot.pkg.provider.tools.loaders.mcp import MCPSessionStatus, MCPToolCallTimeoutError, RuntimeMCPSession from langbot.pkg.provider.tools.loaders.mcp import MCPToolCallTimeoutError, RuntimeMCPSession
from langbot.pkg.provider.tools.loaders.mcp_stdio import MCPSessionErrorPhase
TEST_EXECUTION_CONTEXT = ExecutionContext(
instance_uuid='instance-a',
workspace_uuid='workspace-a',
placement_generation=1,
)
TEST_EXECUTION_CONTEXT = ExecutionContext( TEST_EXECUTION_CONTEXT = ExecutionContext(
@@ -32,9 +24,8 @@ TEST_EXECUTION_CONTEXT = ExecutionContext(
class _TransportProbe: class _TransportProbe:
def __init__(self, streamable_status: int | None, streamable_headers: dict[str, str] | None = None) -> None: def __init__(self, streamable_status: int | None) -> None:
self.streamable_status = streamable_status self.streamable_status = streamable_status
self.streamable_headers = streamable_headers or {}
self.streamable_posts = 0 self.streamable_posts = 0
self.streamable_messages: list[str] = [] self.streamable_messages: list[str] = []
self.sse_gets = 0 self.sse_gets = 0
@@ -102,7 +93,7 @@ class _TransportProbe:
} }
) )
return web.Response(status=202) return web.Response(status=202)
return web.Response(status=self.streamable_status, headers=self.streamable_headers) return web.Response(status=self.streamable_status)
self.sse_gets += 1 self.sse_gets += 1
response = web.StreamResponse( response = web.StreamResponse(
@@ -145,8 +136,8 @@ class _TransportProbe:
@asynccontextmanager @asynccontextmanager
async def _transport_server(streamable_status: int | None, streamable_headers: dict[str, str] | None = None): async def _transport_server(streamable_status: int | None):
probe = _TransportProbe(streamable_status, streamable_headers) probe = _TransportProbe(streamable_status)
application = web.Application() application = web.Application()
application.router.add_route('*', '/mcp', probe.handle_mcp_endpoint) application.router.add_route('*', '/mcp', probe.handle_mcp_endpoint)
application.router.add_post('/messages', probe.handle_sse_message) application.router.add_post('/messages', probe.handle_sse_message)
@@ -274,45 +265,6 @@ async def test_remote_transport_real_non_compatibility_error_does_not_fallback(s
await _close_session(session) await _close_session(session)
def test_remote_transport_extracts_oauth_resource_metadata_from_bearer_challenge():
request = httpx.Request('POST', 'https://mcp.example/mcp')
response = httpx.Response(
401,
headers={
'WWW-Authenticate': (
'Basic realm="MCP", Bearer resource_metadata="https://mcp.example/.well-known/oauth-protected-resource"'
)
},
request=request,
)
with pytest.raises(httpx.HTTPStatusError) as exc_info:
response.raise_for_status()
challenge = RuntimeMCPSession._extract_oauth_challenge(exc_info.value)
assert challenge is not None
assert challenge.resource_metadata_url == 'https://mcp.example/.well-known/oauth-protected-resource'
@pytest.mark.asyncio
async def test_remote_transport_oauth_challenge_sets_non_retryable_authorization_state():
headers = {
'WWW-Authenticate': 'Bearer resource_metadata="https://mcp.example/.well-known/oauth-protected-resource"'
}
async with _transport_server(401, headers) as (probe, url):
session = _session(url)
await session._lifecycle_loop_with_retry()
assert session.status == MCPSessionStatus.ERROR
assert session.error_phase == MCPSessionErrorPhase.OAUTH_REQUIRED
assert session.retry_count == 1
assert session._ready_event.is_set()
assert probe.streamable_posts == 1
assert probe.sse_gets == 0
@pytest.mark.asyncio @pytest.mark.asyncio
async def test_remote_transport_real_timeout_does_not_fallback(): async def test_remote_transport_real_timeout_does_not_fallback():
async with _transport_server(None) as (probe, url): async with _transport_server(None) as (probe, url):
@@ -361,25 +313,3 @@ async def test_remote_transport_external_cancellation_is_not_converted_to_sse_fa
finally: finally:
probe.release_streamable_request.set() probe.release_streamable_request.set()
await _close_session(session) await _close_session(session)
@pytest.mark.parametrize(
('error', 'expected'),
[
(httpx.ConnectError('secret host'), 'connection_unreachable'),
(httpx.ReadTimeout('secret URL'), 'connection_timeout'),
(TimeoutError('secret command'), 'connection_timeout'),
(RuntimeError('secret environment'), 'runtime_error'),
(
httpx.HTTPStatusError(
'secret response',
request=httpx.Request('POST', 'https://example.test/?token=secret'),
response=httpx.Response(403),
),
'http_403',
),
],
)
def test_public_error_category_does_not_expose_exception_details(error, expected):
grouped = ExceptionGroup('secret outer exception', [error])
assert RuntimeMCPSession._classify_public_error(grouped) == expected
+152 -115
View File
@@ -67,15 +67,10 @@ def _make_skill_data(
class TestSkillManagerCache: class TestSkillManagerCache:
"""The Box runtime is the only source of truth — SkillManager just holds """SkillManager caches the Core-owned SkillRepository catalog."""
an in-memory cache populated by ``reload_skills``. There is no local
filesystem reader anymore."""
def test_refresh_skill_from_disk_reports_cache_presence(self): def test_refresh_skill_from_disk_reports_cache_presence(self):
"""Box is the only source of truth for skill content. refresh_skill_from_disk """Disk mutations are reflected by an explicit repository reload."""
now just reports whether the skill is still in the in-memory cache
the actual content refresh is driven by SkillService awaiting
``reload_skills`` after every Box mutation."""
from langbot.pkg.skill.manager import SkillManager from langbot.pkg.skill.manager import SkillManager
ap = _make_ap() ap = _make_ap()
@@ -92,67 +87,26 @@ class TestSkillManagerCache:
assert mgr.refresh_skill_from_disk(_CONTEXT, '') is False assert mgr.refresh_skill_from_disk(_CONTEXT, '') is False
@pytest.mark.asyncio @pytest.mark.asyncio
async def test_reload_skills_drops_box_skills_with_missing_package_root(self): async def test_reload_skills_uses_repository_when_box_is_disabled(self):
"""When LangBot shares a filesystem with Box (local stdio mode) and Box
reports a skill whose package_root is gone from that shared filesystem,
the cache must drop it instead of keeping a stale entry that would later
produce a bad mount."""
from langbot.pkg.skill.manager import SkillManager from langbot.pkg.skill.manager import SkillManager
with tempfile.TemporaryDirectory() as live_dir: repository = SimpleNamespace(
ghost_dir = os.path.join(live_dir, '_does_not_exist')
box_service = SimpleNamespace(
available=True,
shares_filesystem_with_box=True,
list_skills=AsyncMock(
return_value=[
_make_skill_data(name='alive', package_root=live_dir),
_make_skill_data(name='ghost', package_root=ghost_dir),
]
),
)
ap = _make_ap()
ap.box_service = box_service
mgr = SkillManager(ap)
await mgr.reload_skills(_CONTEXT)
assert list(mgr.get_skills(_CONTEXT)) == ['alive']
# Warning fired with the dropped skill name so operators can see it.
warning_messages = [str(call.args[0]) for call in ap.logger.warning.call_args_list]
assert any('ghost' in msg and 'package_root missing' in msg for msg in warning_messages)
@pytest.mark.asyncio
async def test_reload_skills_trusts_box_paths_when_filesystem_not_shared(self):
"""In separated deployments (Docker Compose, k8s sidecar,
--standalone-box, remote endpoint) the package_root reported by Box
lives on the Box runtime's filesystem and is not resolvable on the
LangBot side. The cache must keep every Box-reported skill rather than
dropping them all via a local isdir() check."""
from langbot.pkg.skill.manager import SkillManager
box_service = SimpleNamespace(
available=True,
shares_filesystem_with_box=False,
list_skills=AsyncMock( list_skills=AsyncMock(
return_value=[ return_value=[
_make_skill_data(name='alpha', package_root='/box/skills/alpha'), _make_skill_data(name='alpha', package_root='/skills/alpha'),
_make_skill_data(name='beta', package_root='/box/skills/beta'), _make_skill_data(name='beta', package_root='/skills/beta'),
] ]
), ),
) )
ap = _make_ap() ap = _make_ap()
ap.box_service = box_service ap.box_service = SimpleNamespace(available=False, enabled=False)
ap.skill_repository = repository
mgr = SkillManager(ap) mgr = SkillManager(ap)
await mgr.reload_skills(_CONTEXT) await mgr.reload_skills(_CONTEXT)
assert sorted(mgr.get_skills(_CONTEXT)) == ['alpha', 'beta'] assert sorted(mgr.get_skills(_CONTEXT)) == ['alpha', 'beta']
# No skill dropped → no "package_root missing" warning. repository.list_skills.assert_awaited_once_with(_CONTEXT)
warning_messages = [str(call.args[0]) for call in ap.logger.warning.call_args_list]
assert not any('package_root missing' in msg for msg in warning_messages)
class TestSkillActivationHelper: class TestSkillActivationHelper:
@@ -322,7 +276,7 @@ class TestSkillPathHelpers:
class TestSkillToolLoader: class TestSkillToolLoader:
"""The skill tool surface is now just ``activate`` + ``register_skill``. """Skill activation and resources are independent from sandbox execution.
The legacy CRUD authoring tools (create/list/get/update/delete/ The legacy CRUD authoring tools (create/list/get/update/delete/
import_skill_from_directory/reload_skills) were removed; skill CRUD is import_skill_from_directory/reload_skills) were removed; skill CRUD is
@@ -338,8 +292,11 @@ class TestSkillToolLoader:
from langbot.pkg.provider.tools.loaders.skill import ACTIVATED_SKILLS_KEY from langbot.pkg.provider.tools.loaders.skill import ACTIVATED_SKILLS_KEY
skill = _make_skill_data(name='demo', package_root='/data/skills/demo', instructions='Step 1') skill = _make_skill_data(name='demo', package_root='/data/skills/demo', instructions='Step 1')
skill['revision'] = 'sha256:demo'
ap = _make_ap() ap = _make_ap()
ap.skill_mgr = _make_skill_manager({'demo': skill}) ap.skill_mgr = _make_skill_manager({'demo': skill})
ap.skill_repository = SimpleNamespace(get_skill=AsyncMock(return_value=skill))
ap.box_service = SimpleNamespace(is_workspace_sandbox_available=AsyncMock(return_value=False))
loader = SkillToolLoader(ap) loader = SkillToolLoader(ap)
query = _make_query() query = _make_query()
@@ -348,9 +305,13 @@ class TestSkillToolLoader:
assert result['activated'] is True assert result['activated'] is True
assert result['skill_name'] == 'demo' assert result['skill_name'] == 'demo'
assert result['mount_path'] == '/workspace/.skills/demo' assert result['mount_path'] is None
assert result['revision'] == 'sha256:demo'
assert result['capabilities']['resources_readable'] is True
assert result['capabilities']['execution_available'] is False
assert result['activated_skill_names'] == ['demo'] assert result['activated_skill_names'] == ['demo']
assert 'Step 1' in result['content'] assert 'Step 1' in result['content']
assert '<package-root>' not in result['content']
assert set(query.variables[ACTIVATED_SKILLS_KEY].keys()) == {'demo'} assert set(query.variables[ACTIVATED_SKILLS_KEY].keys()) == {'demo'}
@pytest.mark.asyncio @pytest.mark.asyncio
@@ -384,7 +345,11 @@ class TestSkillToolLoader:
os.makedirs(repo_dir) os.makedirs(repo_dir)
ap = _make_ap() ap = _make_ap()
ap.box_service = SimpleNamespace(default_workspace=tmpdir, available=True) ap.box_service = SimpleNamespace(
default_workspace=tmpdir,
available=True,
require_workspace_sandbox=AsyncMock(return_value=_CONTEXT),
)
ap.skill_service = SimpleNamespace( ap.skill_service = SimpleNamespace(
scan_directory_async=AsyncMock( scan_directory_async=AsyncMock(
return_value={ return_value={
@@ -394,7 +359,7 @@ class TestSkillToolLoader:
'instructions': 'Do work', 'instructions': 'Do work',
} }
), ),
create_skill=AsyncMock( import_skill_directory=AsyncMock(
return_value=_make_skill_data(name='cloned-skill', package_root=os.path.realpath(repo_dir)) return_value=_make_skill_data(name='cloned-skill', package_root=os.path.realpath(repo_dir))
), ),
) )
@@ -407,14 +372,14 @@ class TestSkillToolLoader:
) )
ap.skill_service.scan_directory_async.assert_awaited_once_with(_CONTEXT, os.path.realpath(repo_dir)) ap.skill_service.scan_directory_async.assert_awaited_once_with(_CONTEXT, os.path.realpath(repo_dir))
ap.skill_service.create_skill.assert_awaited_once_with( ap.skill_service.import_skill_directory.assert_awaited_once_with(
_CONTEXT, _CONTEXT,
os.path.realpath(repo_dir),
{ {
'name': 'cloned-skill', 'name': 'cloned-skill',
'display_name': 'Cloned Skill', 'display_name': 'Cloned Skill',
'description': 'Imported from clone', 'description': 'Imported from clone',
'instructions': 'Do work', 'instructions': 'Do work',
'package_root': os.path.realpath(repo_dir),
}, },
) )
assert result['registered'] is True assert result['registered'] is True
@@ -430,8 +395,15 @@ class TestSkillToolLoader:
with tempfile.TemporaryDirectory() as tmpdir: with tempfile.TemporaryDirectory() as tmpdir:
ap = _make_ap() ap = _make_ap()
ap.box_service = SimpleNamespace(default_workspace=tmpdir, available=True) ap.box_service = SimpleNamespace(
ap.skill_service = SimpleNamespace(scan_directory_async=AsyncMock(), create_skill=AsyncMock()) default_workspace=tmpdir,
available=True,
require_workspace_sandbox=AsyncMock(return_value=_CONTEXT),
)
ap.skill_service = SimpleNamespace(
scan_directory_async=AsyncMock(),
import_skill_directory=AsyncMock(),
)
loader = SkillToolLoader(ap) loader = SkillToolLoader(ap)
@@ -451,7 +423,11 @@ class TestSkillToolLoader:
with tempfile.TemporaryDirectory() as tmpdir: with tempfile.TemporaryDirectory() as tmpdir:
ap = _make_ap() # no skill_service attribute ap = _make_ap() # no skill_service attribute
ap.box_service = SimpleNamespace(default_workspace=tmpdir, available=True) ap.box_service = SimpleNamespace(
default_workspace=tmpdir,
available=True,
require_workspace_sandbox=AsyncMock(return_value=_CONTEXT),
)
loader = SkillToolLoader(ap) loader = SkillToolLoader(ap)
@@ -463,21 +439,22 @@ class TestSkillToolLoader:
) )
@pytest.mark.asyncio @pytest.mark.asyncio
async def test_tools_hidden_when_sandbox_backend_unavailable(self): async def test_read_only_tools_remain_when_sandbox_unavailable(self):
from langbot.pkg.provider.tools.loaders.skill_authoring import SkillToolLoader from langbot.pkg.provider.tools.loaders.skill_authoring import SkillToolLoader
ap = _make_ap() ap = _make_ap()
ap.skill_mgr = SimpleNamespace(skills={}) ap.skill_mgr = SimpleNamespace(skills={})
ap.box_service = SimpleNamespace( ap.skill_repository = SimpleNamespace()
available=True,
get_backend_status=AsyncMock(return_value={'backend': {'available': False}}),
)
loader = SkillToolLoader(ap) loader = SkillToolLoader(ap)
await loader.initialize() await loader.initialize()
assert await loader.get_tools() == [] assert sorted(tool.name for tool in await loader.get_tools(sandbox_available=False)) == [
assert await loader.has_tool('activate') is False 'activate',
'list_skill_resources',
'read_skill_resource',
]
assert await loader.has_tool('activate') is True
assert await loader.has_tool('register_skill') is False assert await loader.has_tool('register_skill') is False
@pytest.mark.asyncio @pytest.mark.asyncio
@@ -486,38 +463,89 @@ class TestSkillToolLoader:
ap = _make_ap() ap = _make_ap()
ap.skill_mgr = _make_skill_manager({'demo': _make_skill_data(name='demo')}) ap.skill_mgr = _make_skill_manager({'demo': _make_skill_data(name='demo')})
ap.box_service = SimpleNamespace( ap.skill_repository = SimpleNamespace()
available=True,
get_backend_status=AsyncMock(return_value={'backend': {'available': True}}),
)
loader = SkillToolLoader(ap) loader = SkillToolLoader(ap)
await loader.initialize() await loader.initialize()
tools = await loader.get_tools() tools = await loader.get_tools(sandbox_available=True)
assert sorted(tool.name for tool in tools) == ['activate', 'register_skill'] assert sorted(tool.name for tool in tools) == [
'activate',
'list_skill_resources',
'read_skill_resource',
'register_skill',
]
assert await loader.has_tool('activate') is True assert await loader.has_tool('activate') is True
assert await loader.has_tool('register_skill') is True assert await loader.has_tool('register_skill', sandbox_available=True) is True
@pytest.mark.asyncio @pytest.mark.asyncio
async def test_tools_reappear_after_box_backend_recovers(self): async def test_register_skill_appears_after_sandbox_recovers(self):
from langbot.pkg.provider.tools.loaders.skill_authoring import SkillToolLoader from langbot.pkg.provider.tools.loaders.skill_authoring import SkillToolLoader
ap = _make_ap() ap = _make_ap()
ap.skill_mgr = SimpleNamespace(skills={'demo': _make_skill_data(name='demo')}) ap.skill_mgr = SimpleNamespace(skills={'demo': _make_skill_data(name='demo')})
ap.box_service = SimpleNamespace( ap.skill_repository = SimpleNamespace()
available=False,
get_backend_status=AsyncMock(return_value={'backend': {'available': True}}),
)
loader = SkillToolLoader(ap) loader = SkillToolLoader(ap)
await loader.initialize() await loader.initialize()
assert await loader.get_tools() == [] assert 'register_skill' not in {tool.name for tool in await loader.get_tools(sandbox_available=False)}
assert 'register_skill' in {tool.name for tool in await loader.get_tools(sandbox_available=True)}
ap.box_service.available = True @pytest.mark.asyncio
async def test_resources_require_activation_and_use_pinned_revision(self):
from langbot.pkg.provider.tools.loaders.skill import register_activated_skill
from langbot.pkg.provider.tools.loaders.skill_authoring import SkillToolLoader
assert sorted(tool.name for tool in await loader.get_tools()) == ['activate', 'register_skill'] skill = _make_skill_data(name='demo', instructions='Read references')
skill['revision'] = 'sha256:demo'
ap = _make_ap()
ap.skill_mgr = _make_skill_manager({'demo': skill})
ap.skill_repository = SimpleNamespace(
list_skill_resources=AsyncMock(
return_value={'entries': [{'path': 'references/a.md'}], 'revision': 'sha256:demo'}
),
read_skill_resource=AsyncMock(
return_value={
'path': 'references/a.md',
'content': 'reference text',
'revision': 'sha256:demo',
'mime_type': 'text/markdown',
}
),
)
loader = SkillToolLoader(ap)
query = _make_query()
with pytest.raises(ValueError, match='must be activated'):
await loader.invoke_tool(
'read_skill_resource',
{'skill_name': 'demo', 'path': 'references/a.md'},
query,
)
register_activated_skill(query, skill)
listed = await loader.invoke_tool('list_skill_resources', {'skill_name': 'demo'}, query)
read = await loader.invoke_tool(
'read_skill_resource',
{'skill_name': 'demo', 'path': 'references/a.md', 'revision': 'sha256:demo'},
query,
)
assert listed['entries'][0]['path'] == 'references/a.md'
assert read['content'] == 'reference text'
ap.skill_repository.list_skill_resources.assert_awaited_once_with(
_CONTEXT,
'demo',
'.',
expected_revision='sha256:demo',
)
ap.skill_repository.read_skill_resource.assert_awaited_once_with(
_CONTEXT,
'demo',
'references/a.md',
expected_revision='sha256:demo',
)
class TestNativeToolLoaderSkillPaths: class TestNativeToolLoaderSkillPaths:
@@ -551,7 +579,7 @@ class TestNativeToolLoaderSkillPaths:
assert result['truncated'] is False assert result['truncated'] is False
@pytest.mark.asyncio @pytest.mark.asyncio
async def test_external_runtime_read_never_interprets_package_root_on_core_host(self): async def test_external_runtime_read_uses_core_skill_repository(self):
from langbot.pkg.provider.tools.loaders.native import NativeToolLoader from langbot.pkg.provider.tools.loaders.native import NativeToolLoader
from langbot.pkg.provider.tools.loaders.skill import PIPELINE_BOUND_SKILLS_KEY from langbot.pkg.provider.tools.loaders.skill import PIPELINE_BOUND_SKILLS_KEY
@@ -563,7 +591,9 @@ class TestNativeToolLoaderSkillPaths:
ap.box_service = SimpleNamespace( ap.box_service = SimpleNamespace(
available=True, available=True,
shares_filesystem_with_box=False, shares_filesystem_with_box=False,
read_skill_file=AsyncMock(return_value={'content': 'runtime-owned-content'}), )
ap.skill_repository = SimpleNamespace(
read_skill_file=AsyncMock(return_value={'content': 'repository-content'})
) )
ap.skill_mgr = _make_skill_manager({'demo': _make_skill_data(name='demo', package_root=tmpdir)}) ap.skill_mgr = _make_skill_manager({'demo': _make_skill_data(name='demo', package_root=tmpdir)})
loader = NativeToolLoader(ap) loader = NativeToolLoader(ap)
@@ -579,12 +609,12 @@ class TestNativeToolLoaderSkillPaths:
) )
assert result['ok'] is True assert result['ok'] is True
assert result['content'] == 'runtime-owned-content' assert result['content'] == 'repository-content'
assert 'core-host-secret' not in repr(result) assert 'core-host-secret' not in repr(result)
ap.box_service.read_skill_file.assert_awaited_once_with(_CONTEXT, 'demo', 'SKILL.md') ap.skill_repository.read_skill_file.assert_awaited_once_with(_CONTEXT, 'demo', 'SKILL.md')
@pytest.mark.asyncio @pytest.mark.asyncio
async def test_external_runtime_rejects_skill_host_fallback_without_protocol_capability(self): async def test_core_owned_skill_path_does_not_depend_on_runtime_topology(self):
from langbot.pkg.provider.tools.loaders.native import NativeToolLoader from langbot.pkg.provider.tools.loaders.native import NativeToolLoader
from langbot.pkg.provider.tools.loaders.skill import PIPELINE_BOUND_SKILLS_KEY from langbot.pkg.provider.tools.loaders.skill import PIPELINE_BOUND_SKILLS_KEY
@@ -604,18 +634,20 @@ class TestNativeToolLoaderSkillPaths:
variables={PIPELINE_BOUND_SKILLS_KEY: ['demo']}, variables={PIPELINE_BOUND_SKILLS_KEY: ['demo']},
) )
with pytest.raises(ValueError, match='owned by the Box Runtime'): result = await loader.invoke_tool(
await loader.invoke_tool( 'grep',
'grep', {
{ 'path': '/workspace/.skills/demo',
'path': '/workspace/.skills/demo', 'pattern': 'core-host-secret',
'pattern': 'core-host-secret', },
}, query,
query, )
)
assert result['ok'] is True
assert result['total'] == 1
@pytest.mark.asyncio @pytest.mark.asyncio
async def test_exec_in_activated_skill_mount_rewrites_command_and_refreshes(self): async def test_exec_in_activated_skill_mount_rewrites_command_without_mutating_skill(self):
from langbot.pkg.provider.tools.loaders.native import NativeToolLoader from langbot.pkg.provider.tools.loaders.native import NativeToolLoader
from langbot.pkg.provider.tools.loaders.skill import register_activated_skill from langbot.pkg.provider.tools.loaders.skill import register_activated_skill
@@ -626,11 +658,15 @@ class TestNativeToolLoaderSkillPaths:
default_workspace=tmpdir, default_workspace=tmpdir,
execute_tool=AsyncMock(return_value={'ok': True}), execute_tool=AsyncMock(return_value={'ok': True}),
) )
ap.skill_mgr = SimpleNamespace(refresh_skill_from_disk=Mock()) skill_data = _make_skill_data(name='demo', package_root=tmpdir)
ap.skill_mgr = _make_skill_manager(
{'demo': skill_data},
refresh_skill_from_disk=Mock(),
)
loader = NativeToolLoader(ap) loader = NativeToolLoader(ap)
query = _make_query(query_id='q1', launcher_type='person', launcher_id='123') query = _make_query(query_id='q1', launcher_type='person', launcher_id='123')
register_activated_skill(query, _make_skill_data(name='demo', package_root=tmpdir)) register_activated_skill(query, skill_data)
result = await loader.invoke_tool( result = await loader.invoke_tool(
'exec', 'exec',
@@ -645,8 +681,8 @@ class TestNativeToolLoaderSkillPaths:
tool_parameters = ap.box_service.execute_tool.await_args.args[0] tool_parameters = ap.box_service.execute_tool.await_args.args[0]
assert tool_parameters['command'] == 'python /workspace/.skills/demo/scripts/run.py' assert tool_parameters['command'] == 'python /workspace/.skills/demo/scripts/run.py'
assert tool_parameters['workdir'] == '/workspace/.skills/demo' assert tool_parameters['workdir'] == '/workspace/.skills/demo'
assert ap.box_service.execute_tool.await_args.kwargs['skill_name'] == 'demo' assert 'skill_name' not in ap.box_service.execute_tool.await_args.kwargs
ap.skill_mgr.refresh_skill_from_disk.assert_called_once_with(_CONTEXT, 'demo') ap.skill_mgr.refresh_skill_from_disk.assert_not_called()
@pytest.mark.asyncio @pytest.mark.asyncio
async def test_external_runtime_python_skill_uses_trusted_metadata_and_writable_env(self): async def test_external_runtime_python_skill_uses_trusted_metadata_and_writable_env(self):
@@ -659,17 +695,18 @@ class TestNativeToolLoaderSkillPaths:
shares_filesystem_with_box=False, shares_filesystem_with_box=False,
execute_tool=AsyncMock(return_value={'ok': True}), execute_tool=AsyncMock(return_value={'ok': True}),
) )
ap.skill_mgr = SimpleNamespace(refresh_skill_from_disk=Mock()) skill_data = _make_skill_data(
name='demo',
package_root='/box-runtime/skills/tenants/workspace/demo',
python_project=True,
)
ap.skill_mgr = _make_skill_manager(
{'demo': skill_data},
refresh_skill_from_disk=Mock(),
)
loader = NativeToolLoader(ap) loader = NativeToolLoader(ap)
query = _make_query(query_id='q-external', launcher_type='person', launcher_id='123') query = _make_query(query_id='q-external', launcher_type='person', launcher_id='123')
register_activated_skill( register_activated_skill(query, skill_data)
query,
_make_skill_data(
name='demo',
package_root='/box-runtime/skills/tenants/workspace/demo',
python_project=True,
),
)
result = await loader.invoke_tool( result = await loader.invoke_tool(
'exec', 'exec',
@@ -686,7 +723,7 @@ class TestNativeToolLoaderSkillPaths:
assert '_LB_VENV_DIR="/workspace/.skill-envs/demo/.venv"' in wrapped assert '_LB_VENV_DIR="/workspace/.skill-envs/demo/.venv"' in wrapped
assert 'root = "/workspace/.skills/demo"' in wrapped assert 'root = "/workspace/.skills/demo"' in wrapped
assert '/box-runtime/skills/tenants/workspace/demo' not in wrapped assert '/box-runtime/skills/tenants/workspace/demo' not in wrapped
assert ap.box_service.execute_tool.await_args.kwargs['skill_name'] == 'demo' assert 'skill_name' not in ap.box_service.execute_tool.await_args.kwargs
@pytest.mark.asyncio @pytest.mark.asyncio
async def test_write_requires_skill_activation(self): async def test_write_requires_skill_activation(self):
@@ -55,10 +55,20 @@ class StubLoader:
for tool in self._tools for tool in self._tools
] ]
async def has_tool(self, *args) -> bool: async def get_tool(self, name: str, **_kwargs):
return next((tool for tool in self._tools if tool.name == name), None)
async def has_tool(self, *args, **_kwargs) -> bool:
name = args[-1] name = args[-1]
return any(tool.name == name for tool in self._tools) return any(tool.name == name for tool in self._tools)
def recognizes_tool(self, name: str) -> bool:
return any(tool.name == name for tool in self._tools)
@staticmethod
def is_sandbox_tool(name: str) -> bool:
return name == 'register_skill'
async def invoke_tool(self, name: str, parameters: dict, query): async def invoke_tool(self, name: str, parameters: dict, query):
return self._invoke_result(name, parameters, query) if callable(self._invoke_result) else self._invoke_result return self._invoke_result(name, parameters, query) if callable(self._invoke_result) else self._invoke_result
@@ -145,7 +155,7 @@ async def test_tool_manager_routes_native_tool_calls():
@pytest.mark.asyncio @pytest.mark.asyncio
async def test_tool_manager_hides_sandbox_and_skill_tools_without_workspace_entitlement(): async def test_tool_manager_keeps_read_only_skill_tools_without_workspace_entitlement():
box_service = SimpleNamespace(is_workspace_sandbox_available=AsyncMock(return_value=False)) box_service = SimpleNamespace(is_workspace_sandbox_available=AsyncMock(return_value=False))
manager = ToolManager(SimpleNamespace(box_service=box_service)) manager = ToolManager(SimpleNamespace(box_service=box_service))
manager.native_tool_loader = StubLoader([make_tool('exec')]) manager.native_tool_loader = StubLoader([make_tool('exec')])
@@ -156,8 +166,8 @@ async def test_tool_manager_hides_sandbox_and_skill_tools_without_workspace_enti
tools = await manager.get_all_tools(_CONTEXT, include_skill_authoring=True) tools = await manager.get_all_tools(_CONTEXT, include_skill_authoring=True)
catalog = await manager.get_tool_catalog(_CONTEXT, include_skill_authoring=True) catalog = await manager.get_tool_catalog(_CONTEXT, include_skill_authoring=True)
assert [tool.name for tool in tools] == ['plugin_tool', 'mcp_tool'] assert [tool.name for tool in tools] == ['activate', 'plugin_tool', 'mcp_tool']
assert [item['name'] for item in catalog] == ['plugin_tool', 'mcp_tool'] assert [item['name'] for item in catalog] == ['activate', 'plugin_tool', 'mcp_tool']
assert box_service.is_workspace_sandbox_available.await_count == 2 assert box_service.is_workspace_sandbox_available.await_count == 2
@@ -176,9 +186,10 @@ async def test_tool_manager_rechecks_workspace_entitlement_before_native_invocat
query_uuid=None, query_uuid=None,
) )
with pytest.raises(Exception, match='exec'): result = await manager.execute_func_call('exec', {'command': 'pwd'}, query=query)
await manager.execute_func_call('exec', {'command': 'pwd'}, query=query)
assert result['code'] == 'sandbox_unavailable'
assert result['tool'] == 'exec'
box_service.is_workspace_sandbox_available.assert_awaited_once_with(_CONTEXT) box_service.is_workspace_sandbox_available.assert_awaited_once_with(_CONTEXT)
-61
View File
@@ -1,61 +0,0 @@
from __future__ import annotations
"""Regression test for the Matrix ``!relogin`` command (``_handle_relogin_command``).
The old code placed ``logout_cmd = ...`` on a line after an unconditional
``continue`` inside the ``if not bridge.login_command or not bridge.dm_room_id``
branch. Because the ``continue`` always fired, ``logout_cmd`` was never assigned
on the configured path, so any bridge with both ``login_command`` and
``dm_room_id`` raised ``UnboundLocalError``. The fix moves the assignment above
the ``if`` so it runs for configured bridges.
These tests replicate the loop's control flow with a minimal fake bridge so they
run without the Matrix SDK / langbot_plugin dependency.
"""
class FakeBridge:
def __init__(self, user_id: str, login_command: str, logout_command: str = '', dm_room_id: str | None = None):
self.user_id = user_id
self.login_command = login_command
self.logout_command = logout_command
self.dm_room_id = dm_room_id
def _relogin_commands(bridges: list[FakeBridge]) -> list[str]:
"""Return the logout commands the fixed loop would send for each bridge."""
commands: list[str] = []
for bridge in bridges:
if not bridge.login_command or not bridge.dm_room_id:
continue
# Use configured logout command, fallback to deriving from login command.
logout_cmd = bridge.logout_command or bridge.login_command.replace('login', 'logout')
commands.append(logout_cmd)
return commands
def test_configured_bridge_produces_logout_command_without_error() -> None:
bridges = [FakeBridge('@u:example.org', 'login', dm_room_id='!room:example.org')]
# Old code raised UnboundLocalError here; the fix must return the derived
# logout command (no configured logout_command -> derive from login).
assert _relogin_commands(bridges) == ['logout']
def test_configured_logout_command_is_used_verbatim() -> None:
bridges = [FakeBridge('@u:example.org', 'login', logout_command='leave', dm_room_id='!room:example.org')]
assert _relogin_commands(bridges) == ['leave']
def test_skipped_bridge_is_ignored() -> None:
# Missing dm_room_id -> skipped, no command emitted.
bridges = [FakeBridge('@u:example.org', 'login', dm_room_id=None)]
assert _relogin_commands(bridges) == []
def test_relogin_never_raises_for_mixed_configurations() -> None:
bridges = [
FakeBridge('@skip:example.org', '', dm_room_id='!room:example.org'), # no login_command
FakeBridge('@ok:example.org', 'login', dm_room_id='!room:example.org'),
FakeBridge('@skip2:example.org', 'login', dm_room_id=None), # no dm_room_id
]
assert _relogin_commands(bridges) == ['logout']
+17
View File
@@ -190,6 +190,7 @@ async def test_preproc_injects_skill_index_into_system_prompt():
preproc_module, entities_module = _import_preproc_modules() preproc_module, entities_module = _import_preproc_modules()
app = _make_app(skill_service=SimpleNamespace()) app = _make_app(skill_service=SimpleNamespace())
app.tool_mgr.get_all_tools.return_value = [SimpleNamespace(name='activate')]
addendum = '\n\nAvailable Skills:\n- demo (demo): Demo skill.\n\nCall activate ...' addendum = '\n\nAvailable Skills:\n- demo (demo): Demo skill.\n\nCall activate ...'
app.skill_mgr.build_skill_aware_prompt_addition = Mock(return_value=addendum) app.skill_mgr.build_skill_aware_prompt_addition = Mock(return_value=addendum)
@@ -204,6 +205,22 @@ async def test_preproc_injects_skill_index_into_system_prompt():
head = query.prompt.messages[0] head = query.prompt.messages[0]
assert head.role == 'system' assert head.role == 'system'
assert head.content.endswith(addendum) assert head.content.endswith(addendum)
assert query.variables['_skill_execution_available'] is False
@pytest.mark.asyncio
async def test_preproc_does_not_advertise_activation_when_tool_is_filtered_out():
preproc_module, entities_module = _import_preproc_modules()
app = _make_app(skill_service=SimpleNamespace())
addendum = '\n\nAvailable Skills:\n- demo (demo): Demo skill.\n\nCall activate ...'
app.skill_mgr.build_skill_aware_prompt_addition = Mock(return_value=addendum)
query = _make_query()
result = await stage_process_capture(preproc_module, app, query)
assert result.result_type == entities_module.ResultType.CONTINUE
assert addendum not in query.prompt.messages[0].content
@pytest.mark.asyncio @pytest.mark.asyncio
+209
View File
@@ -0,0 +1,209 @@
from types import SimpleNamespace
import pytest
from langbot.pkg.api.http.context import ExecutionContext
from langbot.pkg.skill.repository import SkillRepository, SkillRevisionMismatchError
_CONTEXT = ExecutionContext(
instance_uuid='instance-a',
workspace_uuid='workspace-a',
placement_generation=1,
)
async def _binding(workspace_uuid, *, expected_generation):
return SimpleNamespace(
instance_uuid=_CONTEXT.instance_uuid,
workspace_uuid=workspace_uuid,
placement_generation=expected_generation,
)
def _repository(tmp_path) -> SkillRepository:
app = SimpleNamespace(
workspace_service=SimpleNamespace(
get_execution_binding=_binding,
),
instance_config=SimpleNamespace(
data={
'skills': {'root': str(tmp_path / 'skill-store')},
'box': {
'enabled': False,
'local': {
'host_root': str(tmp_path / 'box'),
},
},
}
),
)
return SkillRepository(app)
def test_repository_prefers_standalone_skill_root(tmp_path):
repository = _repository(tmp_path)
assert repository._store.root == str((tmp_path / 'skill-store').resolve())
def test_repository_locks_are_workspace_scoped(tmp_path):
repository = _repository(tmp_path)
first = repository._workspace_lock('workspace-a')
assert repository._workspace_lock('workspace-a') is first
assert repository._workspace_lock('workspace-b') is not first
def test_repository_keeps_old_box_root_only_for_online_upgrade(tmp_path):
app = SimpleNamespace(
workspace_service=SimpleNamespace(get_execution_binding=_binding),
instance_config=SimpleNamespace(
data={
'box': {
'local': {
'host_root': str(tmp_path / 'box'),
'skills_root': 'legacy-skills',
}
}
}
),
)
repository = SkillRepository(app)
assert repository._store.root == str((tmp_path / 'box' / 'legacy-skills').resolve())
@pytest.mark.asyncio
async def test_repository_crud_and_reads_do_not_require_box(tmp_path):
repository = _repository(tmp_path)
await repository.create_skill(
_CONTEXT,
{
'name': 'docs-only',
'display_name': 'Docs only',
'description': 'Read-only guidance',
'instructions': 'Read references/guide.md.',
},
)
await repository.write_skill_file(
_CONTEXT,
'docs-only',
'references/guide.md',
'# Guide\n\nNo execution needed.',
)
skill = await repository.get_skill(_CONTEXT, 'docs-only', snapshot=True)
assert skill is not None
assert skill['revision'].startswith('stat-v1:')
assert [item['name'] for item in await repository.list_skills(_CONTEXT)] == ['docs-only']
listed = await repository.list_skill_resources(
_CONTEXT,
'docs-only',
'references',
expected_revision=skill['revision'],
)
assert listed['entries'][0]['path'] == 'references/guide.md'
assert listed['entries'][0]['mime_type'] == 'text/markdown'
resource = await repository.read_skill_resource(
_CONTEXT,
'docs-only',
'references/guide.md',
expected_revision=skill['revision'],
)
assert resource['content'].startswith('# Guide')
assert resource['revision'] == skill['revision']
@pytest.mark.asyncio
async def test_repository_rejects_traversal_and_stale_revision(tmp_path):
repository = _repository(tmp_path)
await repository.create_skill(
_CONTEXT,
{'name': 'safe', 'description': 'Safe', 'instructions': 'Use the reference.'},
)
await repository.write_skill_file(_CONTEXT, 'safe', 'reference.md', 'first')
skill = await repository.get_skill(_CONTEXT, 'safe', snapshot=True)
assert skill is not None
with pytest.raises(ValueError, match='stay within'):
await repository.read_skill_resource(_CONTEXT, 'safe', '../secret.txt')
await repository.write_skill_file(_CONTEXT, 'safe', 'reference.md', 'second')
with pytest.raises(SkillRevisionMismatchError, match='reactivate'):
await repository.read_skill_resource(
_CONTEXT,
'safe',
'reference.md',
expected_revision=skill['revision'],
)
@pytest.mark.asyncio
async def test_repository_scopes_skills_by_workspace(tmp_path):
repository = _repository(tmp_path)
other_context = ExecutionContext(
instance_uuid='instance-a',
workspace_uuid='workspace-b',
placement_generation=1,
)
await repository.create_skill(_CONTEXT, {'name': 'private', 'instructions': 'A'})
assert [skill['name'] for skill in await repository.list_skills(_CONTEXT)] == ['private']
assert await repository.list_skills(other_context) == []
@pytest.mark.asyncio
async def test_repository_rejects_stale_workspace_placement(tmp_path):
repository = _repository(tmp_path)
async def stale_binding(workspace_uuid, *, expected_generation):
return SimpleNamespace(
instance_uuid=_CONTEXT.instance_uuid,
workspace_uuid=workspace_uuid,
placement_generation=expected_generation + 1,
)
repository.ap.workspace_service.get_execution_binding = stale_binding
with pytest.raises(ValueError, match='stale Workspace placement'):
await repository.list_skills(_CONTEXT)
@pytest.mark.asyncio
async def test_repository_imports_only_from_the_fenced_workspace(tmp_path):
repository = _repository(tmp_path)
namespace = repository._namespace(_CONTEXT)
source = tmp_path / 'box' / 'default' / 'tenants' / namespace / 'draft'
source.mkdir(parents=True)
(source / 'SKILL.md').write_text(
'---\nname: draft\ndescription: Draft skill\n---\n\nFollow the guide.',
encoding='utf-8',
)
(source / 'guide.md').write_text('Imported resource', encoding='utf-8')
scanned = await repository.scan_skill_directory(_CONTEXT, str(source))
imported = await repository.import_skill_directory(
_CONTEXT,
str(source),
{
'name': scanned['name'],
'display_name': scanned['display_name'],
'description': scanned['description'],
'instructions': scanned['instructions'],
},
)
assert imported['name'] == 'draft'
resource = await repository.read_skill_file(_CONTEXT, 'draft', 'guide.md')
assert resource['content'] == 'Imported resource'
outside = tmp_path / 'outside'
outside.mkdir()
(outside / 'SKILL.md').write_text('Outside', encoding='utf-8')
with pytest.raises(ValueError, match='trusted source root'):
await repository.scan_skill_directory(_CONTEXT, str(outside))
+46 -75
View File
@@ -23,102 +23,73 @@ def _workspace_service():
) )
class TestRequireBoxForWrite: class TestSkillRepositoryBoundary:
"""Box is the only source of truth for skills — there is no local """Skill management and reads remain available without Box execution."""
filesystem fallback. Every write and (most) read methods refuse cleanly
when the Box runtime is disabled, unreachable, or simply not installed."""
def _ap_with_disabled_box(self): @staticmethod
def _ap_with_repository():
repository = SimpleNamespace(
list_skills=AsyncMock(return_value=[{'name': 'x', 'instructions': 'Do work'}]),
get_skill=AsyncMock(return_value={'name': 'x', 'instructions': 'Do work', 'revision': 'sha256:x'}),
create_skill=AsyncMock(return_value={'name': 'x', 'instructions': 'Do work'}),
update_skill=AsyncMock(return_value={'name': 'x', 'instructions': 'Updated'}),
delete_skill=AsyncMock(),
read_skill_file=AsyncMock(return_value={'path': 'a.txt', 'content': 'hello'}),
write_skill_file=AsyncMock(return_value={'path': 'a.txt'}),
)
return SimpleNamespace( return SimpleNamespace(
skill_mgr=SimpleNamespace(reload_skills=AsyncMock()), skill_mgr=SimpleNamespace(reload_skills=AsyncMock()),
workspace_service=_workspace_service(), workspace_service=_workspace_service(),
box_service=SimpleNamespace( box_service=SimpleNamespace(available=False, enabled=False),
available=False, skill_repository=repository,
enabled=False,
_connector_error='Box runtime is disabled in config (box.enabled = false)',
),
)
def _ap_with_failed_box(self):
return SimpleNamespace(
skill_mgr=SimpleNamespace(reload_skills=AsyncMock()),
workspace_service=_workspace_service(),
box_service=SimpleNamespace(
available=False,
enabled=True,
_connector_error='docker daemon not running',
),
) )
@pytest.mark.asyncio @pytest.mark.asyncio
async def test_create_skill_refused_when_box_disabled(self): async def test_list_and_read_work_when_box_disabled(self):
service = SkillService(self._ap_with_disabled_box()) ap = self._ap_with_repository()
with pytest.raises(ValueError, match='disabled in config'): service = SkillService(ap)
await service.create_skill(_CONTEXT, {'name': 'x'})
assert await service.list_skills(_CONTEXT) == [{'name': 'x', 'instructions': 'Do work'}]
assert await service.read_skill_file(_CONTEXT, 'x', 'a.txt') == {
'path': 'a.txt',
'content': 'hello',
}
ap.skill_repository.read_skill_file.assert_awaited_once_with(_CONTEXT, 'x', 'a.txt')
@pytest.mark.asyncio @pytest.mark.asyncio
async def test_create_skill_refused_when_box_failed(self): async def test_create_update_and_write_work_when_box_disabled(self):
service = SkillService(self._ap_with_failed_box()) ap = self._ap_with_repository()
with pytest.raises(ValueError, match='docker daemon not running'): service = SkillService(ap)
await service.create_skill(_CONTEXT, {'name': 'x'})
await service.create_skill(_CONTEXT, {'name': 'x'})
await service.update_skill(_CONTEXT, 'x', {'instructions': 'Updated'})
await service.write_skill_file(_CONTEXT, 'x', 'a.txt', 'hello')
ap.skill_repository.create_skill.assert_awaited_once_with(_CONTEXT, {'name': 'x'})
ap.skill_repository.update_skill.assert_awaited_once_with(_CONTEXT, 'x', {'instructions': 'Updated'})
ap.skill_repository.write_skill_file.assert_awaited_once_with(_CONTEXT, 'x', 'a.txt', 'hello')
@pytest.mark.asyncio @pytest.mark.asyncio
async def test_update_skill_refused_when_box_disabled(self): async def test_get_skill_returns_repository_revision(self):
service = SkillService(self._ap_with_disabled_box()) ap = self._ap_with_repository()
with pytest.raises(ValueError, match='Editing a skill requires the Box runtime'): service = SkillService(ap)
await service.update_skill(_CONTEXT, 'x', {})
skill = await service.get_skill(_CONTEXT, 'x')
assert skill['revision'] == 'sha256:x'
ap.skill_repository.get_skill.assert_awaited_once_with(_CONTEXT, 'x', snapshot=True)
@pytest.mark.asyncio @pytest.mark.asyncio
async def test_write_skill_file_refused_when_box_disabled(self): async def test_missing_repository_is_explicit(self):
service = SkillService(self._ap_with_disabled_box())
with pytest.raises(ValueError, match='Editing skill files requires the Box runtime'):
await service.write_skill_file(_CONTEXT, 'x', 'a.txt', 'hi')
@pytest.mark.asyncio
async def test_install_from_github_refused_when_box_disabled(self):
service = SkillService(self._ap_with_disabled_box())
with pytest.raises(ValueError, match='Installing a skill from GitHub'):
await service.install_from_github(
_CONTEXT,
{'owner': 'o', 'repo': 'r', 'asset_url': 'https://example/x.zip'},
)
@pytest.mark.asyncio
async def test_install_from_zip_upload_refused_when_box_disabled(self):
service = SkillService(self._ap_with_disabled_box())
with pytest.raises(ValueError, match='Installing a skill from upload'):
await service.install_from_zip_upload(
_CONTEXT,
file_bytes=b'',
filename='x.zip',
)
@pytest.mark.asyncio
async def test_create_skill_refused_when_box_service_missing_entirely(self):
"""No ap.box_service attribute at all (truly minimal setup):
Box is the only source of truth, so creation must still refuse."""
service = SkillService( service = SkillService(
SimpleNamespace( SimpleNamespace(
skill_mgr=SimpleNamespace(reload_skills=AsyncMock()), skill_mgr=SimpleNamespace(reload_skills=AsyncMock()),
workspace_service=_workspace_service(), workspace_service=_workspace_service(),
) )
) )
with pytest.raises(ValueError, match='not initialised'): with pytest.raises(ValueError, match='repository is not initialised'):
await service.create_skill(_CONTEXT, {'name': 'x'}) await service.create_skill(_CONTEXT, {'name': 'x'})
@pytest.mark.asyncio
async def test_list_skills_returns_empty_when_box_unavailable(self):
"""list_skills should render an empty surface (not crash) so the
skills page can show a banner instead of a broken state."""
service = SkillService(self._ap_with_disabled_box())
assert await service.list_skills(_CONTEXT) == []
@pytest.mark.asyncio
async def test_read_skill_file_refused_when_box_unavailable(self):
service = SkillService(self._ap_with_disabled_box())
with pytest.raises(ValueError, match='Reading a skill file'):
await service.read_skill_file(_CONTEXT, 'x', 'a.txt')
class TestGithubSkillArchiveLimits: class TestGithubSkillArchiveLimits:
@staticmethod @staticmethod
+1 -1
View File
@@ -17,7 +17,7 @@ export default defineConfig({
}, },
], ],
webServer: { webServer: {
command: 'corepack pnpm@8.9.2 exec vite --host 127.0.0.1 --port 4173', command: 'pnpm exec vite --host 127.0.0.1 --port 4173',
url: 'http://127.0.0.1:4173', url: 'http://127.0.0.1:4173',
reuseExistingServer: !process.env.CI, reuseExistingServer: !process.env.CI,
timeout: 120_000, timeout: 120_000,
@@ -486,7 +486,6 @@ export default function ModelsPanel({
// Get the provider info // Get the provider info
const provider = providers.find((p) => p.uuid === providerUuid); const provider = providers.find((p) => p.uuid === providerUuid);
const providerData = { const providerData = {
uuid: providerUuid,
requester: provider?.requester || '', requester: provider?.requester || '',
base_url: provider?.base_url || '', base_url: provider?.base_url || '',
api_keys: provider?.api_keys || [], api_keys: provider?.api_keys || [],
@@ -496,7 +495,7 @@ export default function ModelsPanel({
await httpClient.testLLMModel('_', { await httpClient.testLLMModel('_', {
uuid: '', uuid: '',
name, name,
provider_uuid: providerUuid, provider_uuid: '',
provider: providerData, provider: providerData,
abilities, abilities,
reasoning_config: reasoningConfig, reasoning_config: reasoningConfig,
@@ -506,7 +505,7 @@ export default function ModelsPanel({
await httpClient.testEmbeddingModel('_', { await httpClient.testEmbeddingModel('_', {
uuid: '', uuid: '',
name, name,
provider_uuid: providerUuid, provider_uuid: '',
provider: providerData, provider: providerData,
extra_args: extraArgsObj, extra_args: extraArgsObj,
} as never); } as never);
@@ -514,7 +513,7 @@ export default function ModelsPanel({
await httpClient.testRerankModel('_', { await httpClient.testRerankModel('_', {
uuid: '', uuid: '',
name, name,
provider_uuid: providerUuid, provider_uuid: '',
provider: providerData, provider: providerData,
extra_args: extraArgsObj, extra_args: extraArgsObj,
} as never); } as never);
@@ -537,29 +536,6 @@ export default function ModelsPanel({
expandedProviders.forEach((uuid) => loadProviderModels(uuid)); expandedProviders.forEach((uuid) => loadProviderModels(uuid));
} }
async function handleProviderDeleted(providerUuid: string) {
setProviders((prev) =>
prev.filter((provider) => provider.uuid !== providerUuid),
);
setProviderModels((prev) => {
const next = { ...prev };
delete next[providerUuid];
return next;
});
setExpandedProviders((prev) => {
const next = new Set(prev);
next.delete(providerUuid);
return next;
});
await Promise.all([
loadProviders(),
...Array.from(expandedProviders)
.filter((uuid) => uuid !== providerUuid)
.map((uuid) => loadProviderModels(uuid)),
]);
setProviderFormOpen(false);
}
function renderProviderCard( function renderProviderCard(
provider: ModelProvider, provider: ModelProvider,
isLangBotModels: boolean = false, isLangBotModels: boolean = false,
@@ -690,14 +666,8 @@ export default function ModelsPanel({
)} )}
</PanelBody> </PanelBody>
<Dialog <Dialog open={providerFormOpen} onOpenChange={setProviderFormOpen}>
open={providerFormOpen} <DialogContent className="w-full max-w-[calc(100%-2rem)] p-4 sm:max-w-[600px] sm:p-6">
onOpenChange={(open) => {
if (!open) handleFormClose();
else setProviderFormOpen(true);
}}
>
<DialogContent className="w-full max-w-[calc(100%-2rem)] max-h-[calc(100dvh-2rem)] overflow-y-auto p-4 sm:max-w-[600px] sm:p-6">
<DialogHeader> <DialogHeader>
<DialogTitle> <DialogTitle>
{editingProviderId {editingProviderId
@@ -705,15 +675,11 @@ export default function ModelsPanel({
: t('models.addProvider')} : t('models.addProvider')}
</DialogTitle> </DialogTitle>
</DialogHeader> </DialogHeader>
{providerFormOpen && ( <ProviderForm
<ProviderForm providerId={editingProviderId || undefined}
key={editingProviderId || 'new'} onFormSubmit={handleFormClose}
providerId={editingProviderId || undefined} onFormCancel={() => setProviderFormOpen(false)}
onFormSubmit={handleFormClose} />
onFormCancel={handleFormClose}
onProviderDeleted={canManage ? handleProviderDeleted : undefined}
/>
)}
</DialogContent> </DialogContent>
</Dialog> </Dialog>
</> </>
@@ -1,194 +0,0 @@
import { useEffect, useRef, useState } from 'react';
import { Check, Copy } from 'lucide-react';
import { toast } from 'sonner';
import { copyToClipboard } from '@/app/utils/clipboard';
import { useTranslation } from 'react-i18next';
import { Button } from '@/components/ui/button';
import type { useCodexLogin } from './useCodexLogin';
export default function CodexAccountSection({
login,
providerId,
}: {
login: ReturnType<typeof useCodexLogin>;
providerId?: string;
}) {
const { t } = useTranslation();
const [confirmDisconnect, setConfirmDisconnect] = useState(false);
const [copied, setCopied] = useState(false);
const [copyFailed, setCopyFailed] = useState(false);
const { phase, device } = login;
const copyGeneration = useRef(0);
const copyTimer = useRef<ReturnType<typeof setTimeout> | null>(null);
useEffect(() => {
const generation = copyGeneration;
setCopied(false);
setCopyFailed(false);
return () => {
generation.current++;
if (copyTimer.current) clearTimeout(copyTimer.current);
};
}, [providerId, device?.authorization_id, device?.user_code, phase]);
const handleCopy = async () => {
if (!device) return;
const generation = ++copyGeneration.current;
if (copyTimer.current) clearTimeout(copyTimer.current);
let ok = false;
try {
ok = await copyToClipboard(device.user_code);
} catch {
// Clipboard failures are recoverable; never log device codes.
}
if (generation !== copyGeneration.current) return;
setCopied(ok);
setCopyFailed(!ok);
if (ok) {
toast.success(t('common.copySuccess'));
copyTimer.current = setTimeout(() => setCopied(false), 2000);
} else {
toast.error(t('common.copyFailed'));
}
};
const waiting = ['starting', 'loading', 'canceling'].includes(phase);
return (
<section
data-testid="codex-account"
aria-label={t('models.codex.account')}
className="min-w-0 rounded-lg border p-3 space-y-3 text-sm"
>
<div>
<h3 className="font-medium">{t('models.codex.account')}</h3>
<p className="mt-1 text-muted-foreground">
{t('models.codex.description')}
</p>
</div>
<p
role={phase === 'error' ? 'alert' : 'status'}
aria-live="polite"
className={
phase === 'error' ? 'text-destructive' : 'text-muted-foreground'
}
>
{t(`models.codex.${phase}`)}
</p>
{device && phase === 'pending' && (
<div className="space-y-3">
<p className="text-muted-foreground">
{t('models.codex.instructions')}
</p>
<div className="flex flex-wrap items-center gap-2">
<code className="select-all break-all rounded border bg-muted px-3 py-2 text-base font-semibold tracking-wider">
{device.user_code}
</code>
<Button
type="button"
variant="outline"
size="sm"
onClick={handleCopy}
>
{copied ? (
<Check className="h-4 w-4" aria-hidden="true" />
) : (
<Copy className="h-4 w-4" aria-hidden="true" />
)}
{t(copied ? 'models.codex.copied' : 'models.codex.copyCode')}
</Button>
</div>
{copyFailed && (
<p role="status" className="text-muted-foreground">
{t('models.codex.copyManually')}
</p>
)}
<a
href={device.verification_uri}
target="_blank"
rel="noopener noreferrer"
className="inline-flex text-sm font-medium underline underline-offset-4"
>
{t('models.codex.continueAtOpenAI')}
</a>
<p className="text-xs text-muted-foreground">
{t('models.codex.expiresAt', {
time: new Date(device.expires_at * 1000).toLocaleTimeString(),
})}
</p>
{login.retrying && (
<p role="status" className="text-xs text-muted-foreground">
{t('models.codex.retrying')}
</p>
)}
<Button
type="button"
variant="outline"
size="sm"
onClick={() => providerId && void login.cancel(providerId)}
>
{t('models.codex.cancelSignIn')}
</Button>
</div>
)}
{providerId && !waiting && phase !== 'pending' && (
<div className="flex flex-wrap gap-2">
{phase !== 'connected' && (
<Button type="submit" size="sm" variant="outline">
{t(
phase === 'error' || phase === 'expired'
? 'models.codex.tryAgain'
: 'models.codex.signIn',
)}
</Button>
)}
{phase === 'connected' && (
<>
<Button
type="button"
size="sm"
variant="outline"
onClick={() => {
setConfirmDisconnect(false);
void login.start(providerId);
}}
>
{t('models.codex.reconnect')}
</Button>
<Button
type="button"
size="sm"
variant="ghost"
onClick={() => setConfirmDisconnect(true)}
>
{t('models.codex.disconnect')}
</Button>
</>
)}
</div>
)}
{confirmDisconnect && phase === 'connected' && (
<div className="space-y-2 border-t pt-3">
<p>{t('models.codex.disconnectConfirm')}</p>
<div className="flex flex-wrap gap-2">
<Button
type="button"
size="sm"
variant="destructive"
onClick={() => {
setConfirmDisconnect(false);
if (providerId) void login.disconnect(providerId);
}}
>
{t('models.codex.confirmDisconnect')}
</Button>
<Button
type="button"
size="sm"
variant="outline"
onClick={() => setConfirmDisconnect(false)}
>
{t('common.cancel')}
</Button>
</div>
</div>
)}
</section>
);
}
@@ -1,4 +1,4 @@
import { useEffect, useState, useRef } from 'react'; import { useEffect, useState, useRef, useCallback } from 'react';
import { httpClient } from '@/app/infra/http/HttpClient'; import { httpClient } from '@/app/infra/http/HttpClient';
import { zodResolver } from '@hookform/resolvers/zod'; import { zodResolver } from '@hookform/resolvers/zod';
@@ -16,31 +16,12 @@ import {
FormMessage, FormMessage,
} from '@/components/ui/form'; } from '@/components/ui/form';
import { Input } from '@/components/ui/input'; import { Input } from '@/components/ui/input';
import { LoadingSpinner } from '@/components/ui/loading-spinner';
import { DialogFooter } from '@/components/ui/dialog'; import { DialogFooter } from '@/components/ui/dialog';
import {
AlertDialog,
AlertDialogCancel,
AlertDialogContent,
AlertDialogDescription,
AlertDialogFooter,
AlertDialogHeader,
AlertDialogTitle,
} from '@/components/ui/alert-dialog';
import { LANGBOT_MODELS_PROVIDER_REQUESTER } from '../../types';
import {
Popover,
PopoverContent,
PopoverTrigger,
} from '@/components/ui/popover';
import { toast } from 'sonner'; import { toast } from 'sonner';
import { extractI18nObject } from '@/i18n/I18nProvider'; import { extractI18nObject } from '@/i18n/I18nProvider';
import { CustomApiError } from '@/app/infra/entities/common'; import { CustomApiError } from '@/app/infra/entities/common';
import { cn } from '@/lib/utils'; import { cn } from '@/lib/utils';
import { Check, ChevronDown, Search } from 'lucide-react'; import { Check, ChevronDown, Search } from 'lucide-react';
import { providerPayload } from './codexPolicy';
import { useCodexLogin } from './useCodexLogin';
import CodexAccountSection from './CodexAccountSection';
const getFormSchema = (t: (key: string) => string) => const getFormSchema = (t: (key: string) => string) =>
z.object({ z.object({
@@ -54,14 +35,12 @@ interface ProviderFormProps {
providerId?: string; providerId?: string;
onFormSubmit: (providerUuid: string) => void | Promise<void>; onFormSubmit: (providerUuid: string) => void | Promise<void>;
onFormCancel: () => void; onFormCancel: () => void;
onProviderDeleted?: (providerUuid: string) => void | Promise<void>;
} }
export default function ProviderForm({ export default function ProviderForm({
providerId, providerId,
onFormSubmit, onFormSubmit,
onFormCancel, onFormCancel,
onProviderDeleted,
}: ProviderFormProps) { }: ProviderFormProps) {
const { t } = useTranslation(); const { t } = useTranslation();
const formSchema = getFormSchema(t); const formSchema = getFormSchema(t);
@@ -75,31 +54,7 @@ export default function ProviderForm({
api_key: '', api_key: '',
}, },
}); });
const { reset } = form; const { setValue } = form;
const isCodex = form.watch('requester') === 'openai-codex';
const [savedProviderId, setSavedProviderId] = useState(providerId);
const savedId = useRef(providerId);
const submitting = useRef(false);
const deleting = useRef(false);
const [isDeleting, setIsDeleting] = useState(false);
const [deleteConfirmOpen, setDeleteConfirmOpen] = useState(false);
const [deleteError, setDeleteError] = useState('');
const [mutableProviderLoaded, setMutableProviderLoaded] = useState(false);
const [loadState, setLoadState] = useState<'loading' | 'ready' | 'error'>(
'loading',
);
const [loadAttempt, setLoadAttempt] = useState(0);
const mounted = useRef(true);
const login = useCodexLogin(isCodex, providerId);
const loginActive = ['starting', 'pending', 'canceling', 'loading'].includes(
login.phase,
);
useEffect(() => {
mounted.current = true;
return () => {
mounted.current = false;
};
}, []);
const [requesterList, setRequesterList] = useState< const [requesterList, setRequesterList] = useState<
{ {
@@ -113,59 +68,72 @@ export default function ProviderForm({
>([]); >([]);
const [searchQuery, setSearchQuery] = useState(''); const [searchQuery, setSearchQuery] = useState('');
const [isOpen, setIsOpen] = useState(false); const [isOpen, setIsOpen] = useState(false);
const dropdownRef = useRef<HTMLDivElement>(null);
const searchInputRef = useRef<HTMLInputElement>(null); const searchInputRef = useRef<HTMLInputElement>(null);
useEffect(() => { const loadRequesters = useCallback(async () => {
// Ignore both success and failure from a closed form or superseded attempt. const resp = await httpClient.getProviderRequesters();
let canceled = false; setRequesterList(
setLoadState('loading'); resp.requesters
setMutableProviderLoaded(false); .filter((item) => item.name !== 'space-chat-completions')
.map((item) => ({
label: extractI18nObject(item.label),
value: item.name,
category: item.spec.provider_category || 'manufacturer',
defaultUrl:
item.spec.config
.find((c) => c.name === 'base_url')
?.default?.toString() || '',
description: extractI18nObject(item.description),
alias: item.spec.alias || '',
})),
);
}, []);
const loadProvider = useCallback(
async (id: string) => {
const resp = await httpClient.getModelProvider(id);
const provider = resp.provider;
setValue('name', provider.name);
setValue('requester', provider.requester);
setValue('base_url', provider.base_url);
setValue('api_key', provider.api_keys?.[0] || '');
},
[setValue],
);
useEffect(() => {
async function init() { async function init() {
try { await loadRequesters();
const [requesters, detail] = await Promise.all([ if (providerId) {
httpClient.getProviderRequesters(), await loadProvider(providerId);
providerId ? httpClient.getModelProvider(providerId) : null,
]);
if (canceled) return;
setRequesterList(
requesters.requesters
.filter((item) => item.name !== LANGBOT_MODELS_PROVIDER_REQUESTER)
.map((item) => ({
label: extractI18nObject(item.label),
value: item.name,
category: item.spec.provider_category || 'manufacturer',
defaultUrl:
item.spec.config
.find((c) => c.name === 'base_url')
?.default?.toString() || '',
description: extractI18nObject(item.description),
alias: item.spec.alias || '',
})),
);
if (detail) {
const provider = detail.provider;
reset({
name: provider.name,
requester: provider.requester,
base_url: provider.base_url,
api_key: provider.api_keys?.[0] || '',
});
setMutableProviderLoaded(
provider.uuid === providerId &&
provider.requester !== LANGBOT_MODELS_PROVIDER_REQUESTER,
);
}
setLoadState('ready');
} catch {
if (!canceled) setLoadState('error');
} }
} }
void init(); init();
return () => { }, [providerId, loadProvider, loadRequesters]);
canceled = true;
}; // Close dropdown when clicking outside
}, [providerId, reset, loadAttempt]); useEffect(() => {
function handleClickOutside(event: MouseEvent) {
if (
dropdownRef.current &&
!dropdownRef.current.contains(event.target as Node)
) {
setIsOpen(false);
setSearchQuery('');
}
}
document.addEventListener('mousedown', handleClickOutside);
return () => document.removeEventListener('mousedown', handleClickOutside);
}, []);
// Focus search input when dropdown opens
useEffect(() => {
if (isOpen && searchInputRef.current) {
searchInputRef.current.focus();
}
}, [isOpen]);
// Filter requesters based on search query // Filter requesters based on search query
const filteredRequesters = requesterList.filter( const filteredRequesters = requesterList.filter(
@@ -195,105 +163,29 @@ export default function ProviderForm({
}; };
async function handleFormSubmit(values: z.infer<typeof formSchema>) { async function handleFormSubmit(values: z.infer<typeof formSchema>) {
if ( const data = {
loadState !== 'ready' || name: values.name,
submitting.current || requester: values.requester,
deleting.current || base_url: values.base_url,
(isCodex && loginActive) api_keys: values.api_key ? [values.api_key] : [],
) };
return;
submitting.current = true;
const data = providerPayload(values);
try { try {
if (savedId.current) { let savedProviderUuid = providerId;
await httpClient.updateModelProvider(savedId.current, data); if (providerId) {
await httpClient.updateModelProvider(providerId, data);
toast.success(t('models.providerSaved'));
} else { } else {
const response = await httpClient.createModelProvider(data); const response = await httpClient.createModelProvider(data);
savedId.current = response.uuid; savedProviderUuid = response.uuid;
if (mounted.current) setSavedProviderId(response.uuid); toast.success(t('models.providerCreated'));
}
if (!mounted.current) return;
if (isCodex && login.phase !== 'connected') {
await login.start(savedId.current);
} else {
toast.success(t('models.providerSaved'));
await onFormSubmit(savedId.current);
} }
await onFormSubmit(savedProviderUuid as string);
} catch (err) { } catch (err) {
if (mounted.current) toast.error(t('models.providerSaveError') + (err as CustomApiError).msg);
toast.error(
t('models.providerSaveError') + (err as CustomApiError).msg,
);
} finally {
submitting.current = false;
} }
} }
async function handleDelete() {
if (
loadState !== 'ready' ||
!providerId ||
!mutableProviderLoaded ||
!onProviderDeleted ||
deleting.current ||
submitting.current ||
(isCodex && loginActive)
)
return;
deleting.current = true;
setIsDeleting(true);
setDeleteError('');
try {
await httpClient.deleteModelProvider(providerId, true);
} catch (err) {
const detail =
(err as CustomApiError | null)?.msg ||
(err instanceof Error ? err.message : '');
setDeleteError(t('models.providerDeleteError') + detail);
deleting.current = false;
setIsDeleting(false);
return;
}
toast.success(t('models.providerDeleted'));
await onProviderDeleted(providerId);
}
if (loadState !== 'ready') {
return (
<>
{loadState === 'loading' ? (
<div
role="status"
aria-label={t('common.loading')}
className="flex justify-center py-8"
>
<LoadingSpinner text={t('common.loading')} />
</div>
) : (
<p role="alert" className="py-8 text-sm text-destructive">
{t('models.loadError')}
</p>
)}
<DialogFooter>
{loadState === 'error' && (
<Button
type="button"
onClick={() => {
setLoadState('loading');
setLoadAttempt((attempt) => attempt + 1);
}}
>
{t('common.retry')}
</Button>
)}
<Button type="button" variant="outline" onClick={onFormCancel}>
{t('common.cancel')}
</Button>
</DialogFooter>
</>
);
}
return ( return (
<Form {...form}> <Form {...form}>
<form <form
@@ -310,12 +202,7 @@ export default function ProviderForm({
<span className="text-red-500">*</span> <span className="text-red-500">*</span>
</FormLabel> </FormLabel>
<FormControl> <FormControl>
<Input <Input {...field} />
{...field}
disabled={
form.formState.isSubmitting || (isCodex && loginActive)
}
/>
</FormControl> </FormControl>
<FormMessage /> <FormMessage />
</FormItem> </FormItem>
@@ -335,65 +222,45 @@ export default function ProviderForm({
{t('models.requester')} {t('models.requester')}
<span className="text-red-500">*</span> <span className="text-red-500">*</span>
</FormLabel> </FormLabel>
<Popover <div ref={dropdownRef} className="relative">
open={isOpen}
onOpenChange={(open) => {
setIsOpen(open);
if (!open) setSearchQuery('');
}}
>
{/* Trigger button */} {/* Trigger button */}
<PopoverTrigger asChild> <button
<button type="button"
type="button" onClick={() => setIsOpen(!isOpen)}
disabled={ className={cn(
form.formState.isSubmitting || 'flex h-10 w-full items-center justify-between rounded-md border border-input bg-background px-3 py-2 text-sm ring-offset-background placeholder:text-muted-foreground focus:outline-none focus:ring-2 focus:ring-ring focus:ring-offset-2 disabled:cursor-not-allowed disabled:opacity-50',
(isCodex && (!!savedProviderId || loginActive)) isOpen && 'ring-2 ring-ring ring-offset-2',
} )}
aria-expanded={isOpen} >
{selectedRequester ? (
<div className="flex items-center gap-2">
<img
src={httpClient.getProviderRequesterIconURL(
selectedRequester.value,
)}
alt={selectedRequester.label}
className="h-5 w-5 rounded"
/>
<span>{selectedRequester.label}</span>
</div>
) : (
<span className="text-muted-foreground">
{t('models.selectRequester')}
</span>
)}
<ChevronDown
className={cn( className={cn(
'flex h-10 w-full items-center justify-between rounded-md border border-input bg-background px-3 py-2 text-sm ring-offset-background placeholder:text-muted-foreground focus:outline-none focus:ring-2 focus:ring-ring focus:ring-offset-2 disabled:cursor-not-allowed disabled:opacity-50', 'h-4 w-4 opacity-50 transition-transform',
isOpen && 'ring-2 ring-ring ring-offset-2', isOpen && 'rotate-180',
)} )}
> />
{selectedRequester ? ( </button>
<div className="flex items-center gap-2">
<img
src={httpClient.getProviderRequesterIconURL(
selectedRequester.value,
)}
alt={selectedRequester.label}
className="h-5 w-5 rounded"
/>
<span>{selectedRequester.label}</span>
</div>
) : (
<span className="text-muted-foreground">
{t('models.selectRequester')}
</span>
)}
<ChevronDown
className={cn(
'h-4 w-4 opacity-50 transition-transform',
isOpen && 'rotate-180',
)}
/>
</button>
</PopoverTrigger>
{/* Unmount on close so an exiting layer cannot eat Dialog Escape. */} {/* Dropdown */}
{isOpen && ( {isOpen && (
<PopoverContent <div className="absolute z-50 mt-1 w-full rounded-md border bg-popover text-popover-foreground shadow-md animate-in fade-in-0 zoom-in-95">
align="start"
collisionPadding={8}
className="flex max-h-[var(--radix-popover-content-available-height)] w-[var(--radix-popover-trigger-width)] max-w-[calc(100vw-16px)] flex-col overflow-hidden p-0"
onOpenAutoFocus={(event) => {
event.preventDefault();
searchInputRef.current?.focus();
}}
>
{/* Search input */} {/* Search input */}
<div className="flex shrink-0 items-center border-b px-3"> <div className="flex items-center border-b px-3">
<Search className="mr-2 h-4 w-4 shrink-0 opacity-50" /> <Search className="mr-2 h-4 w-4 shrink-0 opacity-50" />
<input <input
ref={searchInputRef} ref={searchInputRef}
@@ -408,13 +275,7 @@ export default function ProviderForm({
</div> </div>
{/* Options list */} {/* Options list */}
<div <div className="max-h-[300px] overflow-y-auto p-1">
className="min-h-0 max-h-[300px] overflow-y-auto overscroll-contain p-1"
// The dialog's document-level scroll lock treats this portal as outside.
// Keep native list scrolling without forwarding gestures to that lock.
onWheel={(event) => event.stopPropagation()}
onTouchMove={(event) => event.stopPropagation()}
>
{Object.entries(groupedRequesters).map( {Object.entries(groupedRequesters).map(
([category, items]) => { ([category, items]) => {
if (items.length === 0) return null; if (items.length === 0) return null;
@@ -427,11 +288,6 @@ export default function ProviderForm({
<button <button
key={r.value} key={r.value}
type="button" type="button"
disabled={
!!providerId &&
r.value === 'openai-codex' &&
!isCodex
}
onClick={() => { onClick={() => {
field.onChange(r.value); field.onChange(r.value);
const req = requesterList.find( const req = requesterList.find(
@@ -481,9 +337,9 @@ export default function ProviderForm({
</div> </div>
)} )}
</div> </div>
</PopoverContent> </div>
)} )}
</Popover> </div>
<FormMessage /> <FormMessage />
{selectedRequester?.description && ( {selectedRequester?.description && (
<p className="text-sm text-muted-foreground"> <p className="text-sm text-muted-foreground">
@@ -495,131 +351,40 @@ export default function ProviderForm({
}} }}
/> />
{isCodex ? ( <FormField
<CodexAccountSection login={login} providerId={savedProviderId} /> control={form.control}
) : ( name="base_url"
<> render={({ field }) => (
<FormField <FormItem>
control={form.control} <FormLabel>{t('models.requestURL')}</FormLabel>
name="base_url" <FormControl>
render={({ field }) => ( <Input {...field} />
<FormItem> </FormControl>
<FormLabel>{t('models.requestURL')}</FormLabel> <FormMessage />
<FormControl> </FormItem>
<Input
{...field}
disabled={
form.formState.isSubmitting || (isCodex && loginActive)
}
/>
</FormControl>
<FormMessage />
</FormItem>
)}
/>
<FormField
control={form.control}
name="api_key"
render={({ field }) => (
<FormItem>
<FormLabel>{t('models.apiKey')}</FormLabel>
<FormControl>
<Input {...field} type="password" />
</FormControl>
<FormMessage />
</FormItem>
)}
/>
</>
)}
<DialogFooter className="flex-row flex-wrap items-start justify-between sm:justify-between">
{providerId && mutableProviderLoaded && onProviderDeleted && (
<Button
type="button"
variant="destructive"
disabled={
isDeleting ||
form.formState.isSubmitting ||
(isCodex && loginActive)
}
onClick={() => {
setDeleteError('');
setDeleteConfirmOpen(true);
}}
>
{t('common.delete')}
</Button>
)} )}
<div className="ml-auto flex flex-col gap-2 sm:flex-row"> />
{(!isCodex || !savedProviderId || login.phase === 'connected') && (
<Button <FormField
type="submit" control={form.control}
disabled={ name="api_key"
isDeleting || render={({ field }) => (
form.formState.isSubmitting || <FormItem>
(isCodex && loginActive) <FormLabel>{t('models.apiKey')}</FormLabel>
} <FormControl>
> <Input {...field} type="password" />
{isCodex </FormControl>
? t( <FormMessage />
login.phase === 'connected' </FormItem>
? 'models.codex.done' )}
: 'models.codex.saveAndSignIn', />
)
: t('common.save')} <DialogFooter>
</Button> <Button type="submit">{t('common.save')}</Button>
)} <Button type="button" variant="outline" onClick={onFormCancel}>
<Button {t('common.cancel')}
type="button" </Button>
variant="outline"
disabled={isDeleting}
onClick={onFormCancel}
>
{t('common.cancel')}
</Button>
</div>
</DialogFooter> </DialogFooter>
<AlertDialog
open={deleteConfirmOpen}
onOpenChange={(open) => {
if (!deleting.current) setDeleteConfirmOpen(open);
}}
>
{deleteConfirmOpen && (
<AlertDialogContent className="max-w-[calc(100%-2rem)] max-h-[calc(100dvh-2rem)] overflow-y-auto sm:max-w-lg">
<AlertDialogHeader>
<AlertDialogTitle>{t('common.delete')}</AlertDialogTitle>
<AlertDialogDescription>
{t('models.deleteProviderCascadeConfirmation')}
</AlertDialogDescription>
</AlertDialogHeader>
{deleteError && (
<p
role="alert"
className="text-sm text-destructive break-words"
>
{deleteError}
</p>
)}
<AlertDialogFooter>
<AlertDialogCancel disabled={isDeleting}>
{t('common.cancel')}
</AlertDialogCancel>
<Button
type="button"
variant="destructive"
disabled={isDeleting}
aria-busy={isDeleting}
onClick={handleDelete}
>
{t('common.delete')}
</Button>
</AlertDialogFooter>
</AlertDialogContent>
)}
</AlertDialog>
</form> </form>
</Form> </Form>
); );
@@ -1,26 +0,0 @@
/** Subscription credentials are server-owned, never API-key form values. */
export function providerPayload(values: {
name: string;
requester: string;
base_url: string;
api_key?: string;
}) {
const subscription = values.requester === 'openai-codex';
return {
name: values.name,
requester: values.requester,
base_url: subscription
? 'https://chatgpt.com/backend-api/codex'
: values.base_url,
api_keys: subscription ? [] : values.api_key ? [values.api_key] : [],
};
}
export function pollDelay(interval: number, failures = 0): number {
const seconds = Number.isFinite(interval) && interval > 0 ? interval : 5;
return Math.max(seconds, Math.min(60, seconds * 2 ** failures)) * 1000;
}
export function isCodexVerificationUri(uri: string): boolean {
return uri === 'https://auth.openai.com/codex/device';
}
@@ -1,203 +0,0 @@
import { useCallback, useEffect, useRef, useState } from 'react';
import { httpClient } from '@/app/infra/http/HttpClient';
import type { CodexDeviceAuthorization } from '@/app/infra/entities/codex';
import { isCodexVerificationUri, pollDelay } from './codexPolicy';
type Phase =
| 'disconnected'
| 'loading'
| 'starting'
| 'pending'
| 'connected'
| 'expired'
| 'error'
| 'canceling';
/** One in-memory authorization, sequential polls, and stale-response fencing. */
export function useCodexLogin(enabled: boolean, providerId?: string) {
const [phase, setPhase] = useState<Phase>('disconnected');
const [device, setDevice] = useState<CodexDeviceAuthorization | null>(null);
const [retrying, setRetrying] = useState(false);
const generation = useRef(0);
const busy = useRef(false);
const attempt = useRef<{ uuid: string; authorizationId: string } | null>(
null,
);
const timer = useRef<ReturnType<typeof setTimeout> | undefined>(undefined);
const deadline = useRef<ReturnType<typeof setTimeout> | undefined>(undefined);
const request = useRef<AbortController | null>(null);
const stop = useCallback(() => {
generation.current++;
clearTimeout(timer.current);
clearTimeout(deadline.current);
request.current?.abort();
busy.current = false;
const pending = attempt.current;
attempt.current = null;
return pending;
}, []);
const clearPending = useCallback(async () => {
const pending = stop();
if (pending)
await httpClient.cancelCodexDeviceLogin(
pending.uuid,
pending.authorizationId,
);
}, [stop]);
const loadStatus = useCallback(async (uuid: string) => {
const version = generation.current;
request.current = new AbortController();
setPhase('loading');
try {
const status = await httpClient.getCodexAuthStatus(
uuid,
request.current.signal,
);
if (version === generation.current) setPhase(status.status);
} catch {
if (version === generation.current) setPhase('error');
}
}, []);
useEffect(() => {
setDevice(null);
setPhase('disconnected');
if (enabled && providerId) void loadStatus(providerId);
return () => {
// Device creation is deliberately not aborted: its late response must be
// canceled server-side even if this form has already unmounted.
void clearPending().catch(() => {});
};
}, [enabled, providerId, loadStatus, clearPending]);
async function start(uuid: string) {
if (busy.current) return;
const old = stop();
busy.current = true;
const version = generation.current;
setPhase('starting');
setDevice(null);
setRetrying(false);
try {
if (old)
await httpClient.cancelCodexDeviceLogin(old.uuid, old.authorizationId);
if (version !== generation.current) return;
const authorization = await httpClient.startCodexDeviceLogin(uuid);
if (version !== generation.current) {
await httpClient.cancelCodexDeviceLogin(
uuid,
authorization.authorization_id,
);
return;
}
attempt.current = {
uuid,
authorizationId: authorization.authorization_id,
};
if (
!isCodexVerificationUri(authorization.verification_uri) ||
!Number.isFinite(authorization.expires_at)
) {
await clearPending();
setPhase('error');
return;
}
setDevice(authorization);
setPhase('pending');
let interval = authorization.interval;
let failures = 0;
request.current = new AbortController();
const signal = request.current.signal;
const expire = () => {
if (version !== generation.current) return;
void clearPending().catch(() => {});
setDevice(null);
setPhase('expired');
};
deadline.current = setTimeout(
expire,
Math.max(0, authorization.expires_at * 1000 - Date.now()),
);
const poll = async () => {
if (version !== generation.current) return;
if (Date.now() >= authorization.expires_at * 1000) {
expire();
return;
}
try {
const result = await httpClient.pollCodexDeviceLogin(
uuid,
authorization.authorization_id,
signal,
);
if (version !== generation.current) return;
if (result.status !== 'pending') {
attempt.current = null;
stop();
setDevice(null);
setPhase(result.status);
return;
}
interval = result.interval ?? interval;
failures = 0;
setRetrying(false);
} catch (error) {
if (version !== generation.current) return;
const code = (error as { code?: number }).code;
if (
(code === -1 || (code !== undefined && code >= 500)) &&
failures < 3
) {
failures++;
setRetrying(true);
} else {
void clearPending().catch(() => {});
setDevice(null);
setPhase('error');
return;
}
}
timer.current = setTimeout(poll, pollDelay(interval, failures));
};
timer.current = setTimeout(poll, pollDelay(interval));
} catch {
if (version === generation.current) {
busy.current = false;
setPhase('error');
}
}
}
async function cancel(uuid: string) {
setPhase('canceling');
setDevice(null);
const pending = clearPending();
const version = generation.current;
try {
await pending;
if (version === generation.current) await loadStatus(uuid);
} catch {
if (version === generation.current) setPhase('error');
}
}
async function disconnect(uuid: string) {
if (busy.current) return;
busy.current = true;
setPhase('loading');
const version = generation.current;
try {
await httpClient.disconnectCodex(uuid);
if (version === generation.current) await loadStatus(uuid);
} catch {
if (version === generation.current) setPhase('error');
} finally {
busy.current = false;
}
}
return { phase, device, retrying, start, cancel, disconnect, loadStatus };
}
@@ -8,14 +8,7 @@ import React, {
} from 'react'; } from 'react';
import { useTranslation } from 'react-i18next'; import { useTranslation } from 'react-i18next';
import type { TFunction } from 'i18next'; import type { TFunction } from 'i18next';
import { import { Braces, Loader2, Trash2, Wrench, XCircle } from 'lucide-react';
Braces,
Loader2,
ShieldAlert,
Trash2,
Wrench,
XCircle,
} from 'lucide-react';
import { Resolver, useForm } from 'react-hook-form'; import { Resolver, useForm } from 'react-hook-form';
import { zodResolver } from '@hookform/resolvers/zod'; import { zodResolver } from '@hookform/resolvers/zod';
import { z } from 'zod'; import { z } from 'zod';
@@ -108,7 +101,7 @@ function StatusDisplay({
<div className="space-y-1"> <div className="space-y-1">
<div className="flex items-center gap-2 text-red-600"> <div className="flex items-center gap-2 text-red-600">
<XCircle className="size-5" /> <XCircle className="size-5" />
<span className="font-medium">{t('mcp.connectionFailedStatus')}</span> <span className="font-medium">{t('mcp.connectionFailed')}</span>
</div> </div>
<div className="pl-7 text-sm text-red-500 space-y-0.5"> <div className="pl-7 text-sm text-red-500 space-y-0.5">
<div> <div>
@@ -124,41 +117,15 @@ function StatusDisplay({
); );
} }
if (runtimeInfo.error_phase === 'oauth_required') {
return (
<div className="space-y-1">
<div className="flex items-center gap-2 text-amber-700 dark:text-amber-400">
<ShieldAlert className="size-5" />
<span className="font-medium">
{t('mcp.oauthAuthorizationRequired')}
</span>
</div>
<div className="pl-7 text-sm text-muted-foreground">
{t('mcp.oauthAuthorizationRequiredSuggestion')}
</div>
</div>
);
}
const httpStatus = runtimeInfo.error_code?.match(/^http_(\d{3})$/)?.[1];
const errorDetail =
runtimeInfo.error_code === 'connection_unreachable'
? t('mcp.connectionUnreachable')
: runtimeInfo.error_code === 'connection_timeout'
? t('mcp.connectionTimeout')
: httpStatus
? t('mcp.connectionHttpError', { status: httpStatus })
: runtimeInfo.error_message || t('mcp.unknownError');
return ( return (
<div className="space-y-1"> <div className="space-y-1">
<div className="flex items-center gap-2 text-red-600"> <div className="flex items-center gap-2 text-red-600">
<XCircle className="size-5" /> <XCircle className="size-5" />
<span className="font-medium">{t('mcp.connectionFailedStatus')}</span> <span className="font-medium">{t('mcp.connectionFailed')}</span>
</div> </div>
{errorDetail && ( {runtimeInfo.error_message && (
<div className="pl-7 whitespace-pre-wrap break-words text-sm text-muted-foreground"> <div className="pl-7 text-sm text-red-500">
{errorDetail} {runtimeInfo.error_message}
</div> </div>
)} )}
</div> </div>
@@ -868,31 +835,15 @@ const MCPForm = forwardRef<MCPFormHandle, MCPFormProps>(function MCPForm(
async function testMcp() { async function testMcp() {
setMcpTesting(true); setMcpTesting(true);
const showConnectionFailure = (
message: string,
info?: MCPServerRuntimeInfo,
) => {
toast.error(t('mcp.connectionFailedStatus'));
setRuntimeInfo({
tool_count: 0,
tools: [],
resource_count: 0,
resources: [],
...info,
status: MCPSessionStatus.ERROR,
error_message: info?.error_message || message,
});
};
try { try {
const mode = form.getValues('mode'); const mode = form.getValues('mode');
if (mode === 'stdio' && !mcpStdioEnabled) { if (mode === 'stdio' && !mcpStdioEnabled) {
showConnectionFailure(t('mcp.stdioDisabledByPolicy')); toast.error(t('mcp.stdioDisabledByPolicy'));
setMcpTesting(false); setMcpTesting(false);
return; return;
} }
if (mode === 'stdio' && !boxAvailable) { if (mode === 'stdio' && !boxAvailable) {
showConnectionFailure(t('mcp.stdioBlockedByBoxToast')); toast.error(t('mcp.stdioBlockedByBoxToast'));
setMcpTesting(false); setMcpTesting(false);
return; return;
} }
@@ -963,9 +914,15 @@ const MCPForm = forwardRef<MCPFormHandle, MCPFormProps>(function MCPForm(
if (taskResp.runtime.exception) { if (taskResp.runtime.exception) {
const errorMsg = const errorMsg =
taskResp.runtime.exception || t('mcp.unknownError'); taskResp.runtime.exception || t('mcp.unknownError');
const runtimeInfoFromTest = taskResp.task_context?.metadata toast.error(`${t('mcp.testError')}: ${errorMsg}`);
?.runtime_info as MCPServerRuntimeInfo | undefined; setRuntimeInfo({
showConnectionFailure(errorMsg, runtimeInfoFromTest); status: MCPSessionStatus.ERROR,
error_message: errorMsg,
tool_count: 0,
tools: [],
resource_count: 0,
resources: [],
});
if (shouldTestPersistedServer) { if (shouldTestPersistedServer) {
await onPersistedTestComplete?.(serverName); await onPersistedTestComplete?.(serverName);
} }
@@ -992,19 +949,14 @@ const MCPForm = forwardRef<MCPFormHandle, MCPFormProps>(function MCPForm(
clearInterval(interval); clearInterval(interval);
setMcpTesting(false); setMcpTesting(false);
const errorMsg = const errorMsg =
(err as CustomApiError).msg || (err as CustomApiError).msg || t('mcp.getTaskFailed');
(err as Error).message || toast.error(`${t('mcp.testError')}: ${errorMsg}`);
t('mcp.getTaskFailed');
showConnectionFailure(errorMsg);
} }
}, 1000); }, 1000);
} catch (err) { } catch (err) {
setMcpTesting(false); setMcpTesting(false);
const errorMsg = const errorMsg = (err as Error).message || t('mcp.unknownError');
(err as CustomApiError).msg || toast.error(`${t('mcp.testError')}: ${errorMsg}`);
(err as Error).message ||
t('mcp.unknownError');
showConnectionFailure(errorMsg);
} }
} }
@@ -24,8 +24,6 @@ import {
import { Plugin } from '@/app/infra/entities/plugin'; import { Plugin } from '@/app/infra/entities/plugin';
import { MCPServer, Skill } from '@/app/infra/entities/api'; import { MCPServer, Skill } from '@/app/infra/entities/api';
import PluginComponentList from '@/app/home/plugins/components/plugin-installed/PluginComponentList'; import PluginComponentList from '@/app/home/plugins/components/plugin-installed/PluginComponentList';
import { BoxUnavailableNotice } from '@/app/home/components/BoxUnavailableNotice';
import { useBoxStatus } from '@/app/infra/hooks/useBoxStatus';
function InfoTooltip({ label }: { label: string }) { function InfoTooltip({ label }: { label: string }) {
return ( return (
@@ -52,11 +50,6 @@ export default function PipelineExtension({
pipelineId: string; pipelineId: string;
}) { }) {
const { t } = useTranslation(); const { t } = useTranslation();
const {
available: boxAvailable,
hint: boxHint,
reason: boxReason,
} = useBoxStatus();
const [loading, setLoading] = useState(true); const [loading, setLoading] = useState(true);
const [enableAllPlugins, setEnableAllPlugins] = useState(true); const [enableAllPlugins, setEnableAllPlugins] = useState(true);
const [enableAllMCPServers, setEnableAllMCPServers] = useState(true); const [enableAllMCPServers, setEnableAllMCPServers] = useState(true);
@@ -558,13 +551,9 @@ export default function PipelineExtension({
id="enable-all-skills" id="enable-all-skills"
checked={enableAllSkills} checked={enableAllSkills}
onCheckedChange={handleToggleEnableAllSkills} onCheckedChange={handleToggleEnableAllSkills}
disabled={!boxAvailable}
/> />
</div> </div>
</div> </div>
{!boxAvailable && (
<BoxUnavailableNotice hint={boxHint} reason={boxReason} />
)}
<div className="space-y-2"> <div className="space-y-2">
{enableAllSkills ? ( {enableAllSkills ? (
<div className="flex h-32 items-center justify-center rounded-lg border-2 border-dashed border-border bg-muted/30"> <div className="flex h-32 items-center justify-center rounded-lg border-2 border-dashed border-border bg-muted/30">
@@ -602,7 +591,6 @@ export default function PipelineExtension({
variant="ghost" variant="ghost"
size="icon" size="icon"
onClick={() => handleRemoveSkill(skill.name)} onClick={() => handleRemoveSkill(skill.name)}
disabled={!boxAvailable}
> >
<X className="h-4 w-4" /> <X className="h-4 w-4" />
</Button> </Button>
@@ -616,7 +604,7 @@ export default function PipelineExtension({
onClick={handleOpenSkillDialog} onClick={handleOpenSkillDialog}
variant="outline" variant="outline"
className="w-full" className="w-full"
disabled={enableAllSkills || !boxAvailable} disabled={enableAllSkills}
> >
<Plus className="mr-2 h-4 w-4" /> <Plus className="mr-2 h-4 w-4" />
{t('pipelines.extensions.addSkill')} {t('pipelines.extensions.addSkill')}
+2 -27
View File
@@ -21,8 +21,6 @@ import {
import { useSidebarData } from '@/app/home/components/home-sidebar/SidebarDataContext'; import { useSidebarData } from '@/app/home/components/home-sidebar/SidebarDataContext';
import { httpClient } from '@/app/infra/http/HttpClient'; import { httpClient } from '@/app/infra/http/HttpClient';
import SkillForm from '@/app/home/skills/components/skill-form/SkillForm'; import SkillForm from '@/app/home/skills/components/skill-form/SkillForm';
import { BoxUnavailableNotice } from '@/app/home/components/BoxUnavailableNotice';
import { useBoxStatus } from '@/app/infra/hooks/useBoxStatus';
import { Sparkles, Trash2 } from 'lucide-react'; import { Sparkles, Trash2 } from 'lucide-react';
import { useCurrentWorkspace } from '@/app/infra/http'; import { useCurrentWorkspace } from '@/app/infra/http';
@@ -36,12 +34,6 @@ export default function SkillDetailContent({ id }: { id: string }) {
const { refreshSkills, skills, setDetailEntityName } = useSidebarData(); const { refreshSkills, skills, setDetailEntityName } = useSidebarData();
const [showDeleteConfirm, setShowDeleteConfirm] = useState(false); const [showDeleteConfirm, setShowDeleteConfirm] = useState(false);
const skill = skills.find((item) => item.id === id); const skill = skills.find((item) => item.id === id);
const {
available: boxAvailable,
hint: boxHint,
reason: boxReason,
} = useBoxStatus();
useEffect(() => { useEffect(() => {
if (isCreateMode) { if (isCreateMode) {
setDetailEntityName(t('skills.createSkill')); setDetailEntityName(t('skills.createSkill'));
@@ -93,23 +85,12 @@ export default function SkillDetailContent({ id }: { id: string }) {
</div> </div>
</div> </div>
{canManage && ( {canManage && (
<Button <Button type="submit" form="skill-form" className="shrink-0">
type="submit"
form="skill-form"
className="shrink-0"
disabled={!boxAvailable}
>
{t('common.save')} {t('common.save')}
</Button> </Button>
)} )}
</div> </div>
{!boxAvailable && (
<div className="pb-4 shrink-0">
<BoxUnavailableNotice hint={boxHint} reason={boxReason} />
</div>
)}
<div className="min-h-0 flex-1"> <div className="min-h-0 flex-1">
<fieldset className="contents" disabled={!canManage}> <fieldset className="contents" disabled={!canManage}>
<SkillForm <SkillForm
@@ -184,18 +165,12 @@ export default function SkillDetailContent({ id }: { id: string }) {
type="submit" type="submit"
form="skill-form" form="skill-form"
className="shrink-0" className="shrink-0"
disabled={!boxAvailable} disabled={!canManage}
> >
{t('common.save')} {t('common.save')}
</Button> </Button>
</div> </div>
{!boxAvailable && (
<div className="pb-4 shrink-0">
<BoxUnavailableNotice hint={boxHint} reason={boxReason} />
</div>
)}
<div className="min-h-0 flex-1"> <div className="min-h-0 flex-1">
<fieldset className="contents" disabled={!canManage}> <fieldset className="contents" disabled={!canManage}>
<SkillForm <SkillForm
+1 -18
View File
@@ -5,8 +5,6 @@ import { Button } from '@/components/ui/button';
import SkillDetailContent from '@/app/home/skills/SkillDetailContent'; import SkillDetailContent from '@/app/home/skills/SkillDetailContent';
import SkillForm from '@/app/home/skills/components/skill-form/SkillForm'; import SkillForm from '@/app/home/skills/components/skill-form/SkillForm';
import { useSidebarData } from '@/app/home/components/home-sidebar/SidebarDataContext'; import { useSidebarData } from '@/app/home/components/home-sidebar/SidebarDataContext';
import { BoxUnavailableNotice } from '@/app/home/components/BoxUnavailableNotice';
import { useBoxStatus } from '@/app/infra/hooks/useBoxStatus';
import { useCurrentWorkspace } from '@/app/infra/http'; import { useCurrentWorkspace } from '@/app/infra/http';
export default function SkillsPage() { export default function SkillsPage() {
@@ -21,12 +19,6 @@ export default function SkillsPage() {
const { refreshSkills } = useSidebarData(); const { refreshSkills } = useSidebarData();
const isCreateView = actionParam === 'create'; const isCreateView = actionParam === 'create';
const {
available: boxAvailable,
hint: boxHint,
reason: boxReason,
} = useBoxStatus();
useEffect(() => { useEffect(() => {
if (!detailId && !isCreateView) { if (!detailId && !isCreateView) {
navigate('/home/add-extension', { replace: true }); navigate('/home/add-extension', { replace: true });
@@ -65,20 +57,11 @@ export default function SkillsPage() {
<Button variant="outline" onClick={handleCancel}> <Button variant="outline" onClick={handleCancel}>
{t('common.cancel')} {t('common.cancel')}
</Button> </Button>
<Button <Button type="submit" form="skill-form" disabled={!canManage}>
type="submit"
form="skill-form"
disabled={!boxAvailable || !canManage}
>
{t('common.save')} {t('common.save')}
</Button> </Button>
</div> </div>
</div> </div>
{!boxAvailable && (
<div className="pb-4 shrink-0">
<BoxUnavailableNotice hint={boxHint} reason={boxReason} />
</div>
)}
<div className="min-h-0 flex-1"> <div className="min-h-0 flex-1">
<fieldset className="contents" disabled={!canManage}> <fieldset className="contents" disabled={!canManage}>
<SkillForm <SkillForm
-1
View File
@@ -586,7 +586,6 @@ export enum MCPSessionStatus {
} }
export interface MCPServerRuntimeInfo { export interface MCPServerRuntimeInfo {
error_code?: string;
status: MCPSessionStatus; status: MCPSessionStatus;
error_message?: string; error_message?: string;
/** Stage at which the session failed. Frontends key off this to render /** Stage at which the session failed. Frontends key off this to render
-19
View File
@@ -1,19 +0,0 @@
/** Public device-login responses only. OAuth credentials stay on the server. */
export interface CodexAuthStatus {
status: 'connected' | 'disconnected' | 'expired';
connected: boolean;
expires_at: number | null;
}
export interface CodexDeviceAuthorization {
authorization_id: string;
user_code: string;
verification_uri: string;
interval: number;
expires_at: number;
}
export interface CodexDevicePoll {
status: 'pending' | 'connected' | 'expired';
interval?: number;
}

Some files were not shown because too many files have changed in this diff Show More