Compare commits

..

7 Commits

256 changed files with 4879 additions and 3291 deletions
+4 -4
View File
@@ -53,7 +53,7 @@ LangBot/
│ │ ├── platform/ # IM adapters and runtime bot manager
│ │ ├── pipeline/ # Message routing and pipeline stages
│ │ ├── provider/ # Model providers and Host-owned tools
│ │ ├── agent/ # Agent/AgentRunner orchestration and run state
│ │ ├── agent/ # Agent/Runner orchestration and run state
│ │ ├── plugin/ # LangBot-side Plugin Runtime connector/handler
│ │ ├── box/ # LangBot-side Box service/connector
│ │ ├── skill/ # Skill metadata/activation integration
@@ -81,7 +81,7 @@ Platform adapter
→ Controller
→ RuntimePipeline
→ PipelineStage chain
AgentRunner orchestrator / ToolManager / PluginRuntimeConnector / BoxService
→ Runner orchestrator / ToolManager / PluginRuntimeConnector / BoxService
→ response via adapter
```
@@ -108,7 +108,7 @@ Inbound platform messages enter through adapter-specific SDK callbacks. The comm
3. `MessageAggregator` batches/normalizes messages before adding a `Query` to `QueryPool`.
4. `Controller` in `pkg/pipeline/controller.py` selects queries subject to global pipeline concurrency and per-session concurrency.
5. `RuntimePipeline` in `pkg/pipeline/pipelinemgr.py` runs configured pipeline stages using a responsibility-chain style executor that supports generator stages.
6. The chat stage emits plugin events and projects the current query into the AgentRunner Host orchestrator. The selected plugin AgentRunner returns streaming or final results while the Host owns authorization, tools, telemetry, and conversation history.
6. The chat stage emits plugin events and projects the current query into the Runner Host orchestrator. The selected plugin Runner returns streaming or final results while the Host owns authorization, tools, telemetry, and conversation history.
7. Output stages send text, cards, chunks, files, or error notices back through the original platform adapter.
Pipeline components are registered by decorators and package import side effects. When adding a new stage, loader, runner, or adapter, check the corresponding preregistration mechanism instead of inventing a second registry.
@@ -142,7 +142,7 @@ Pipelines are configuration-driven. Prefer adding a stage or extending an existi
Agent orchestration lives under `pkg/agent/`; model providers and tools live under `pkg/provider/`.
- `modelmgr/` manages configured model providers and requesters.
- `pkg/agent/runner/` discovers plugin AgentRunner components, resolves bindings, constructs run-scoped context/resources, and records execution state.
- `pkg/agent/runner/` discovers plugin Runner components, resolves bindings, constructs run-scoped context/resources, and records execution state.
- `tools/toolmgr.py` aggregates tools from native tools, plugin tools, external MCP servers, and skill-authoring tools.
- `tools/loaders/mcp.py` is the MCP client side: external MCP servers that LangBot connects to for agent tools.
- RAG lives across `pkg/rag/`, `pkg/vector/`, model services, and plugin KnowledgeEngine actions.
@@ -1,8 +1,8 @@
# Agent-owned Context 协议设计
本文档描述插件化 AgentRunner 场景下的上下文边界**设计理由**。结论先行:LangBot 不应成为最终 agentic context manager;它提供 context substrateAgentRunner 或其背后的 runtime 自己决定如何管理历史、压缩、召回和 KV cache。
本文档描述插件化 Runner 场景下的上下文边界**设计理由**。结论先行:LangBot 不应成为最终 agentic context manager;它提供 context substrateRunner 或其背后的 runtime 自己决定如何管理历史、压缩、召回和 KV cache。
> 涉及的数据结构(`AgentRunContext`、`ContextAccess`、`AgentRunAPIProxy` 等)唯一定义在 [PROTOCOL_V1.md](./PROTOCOL_V1.md)。本文只讲语义和约束,不重抄 schema。
> 涉及的数据结构(`RunnerContext`、`ContextAccess`、`RunnerAPIProxy` 等)唯一定义在 [PROTOCOL_V1.md](./PROTOCOL_V1.md)。本文只讲语义和约束,不重抄 schema。
## 1. 设计原则
@@ -18,7 +18,7 @@
### 1.2 Host 不定义通用历史窗口
历史窗口策略不是 AgentRunner 协议或 Query entry adapter 的核心概念。Host 只提供 history pull API、cursor、hard cap 和权限边界;runner 自己决定是否读取、读取多少、如何截断和压缩。
历史窗口策略不是 Runner 协议或 Query entry adapter 的核心概念。Host 只提供 history pull API、cursor、hard cap 和权限边界;runner 自己决定是否读取、读取多少、如何截断和压缩。
正确的问题不是"LangBot 每轮裁几轮历史给 agent",而是:
@@ -33,13 +33,13 @@
- `EventLog`: Host 保存原始事件、工具调用、投递结果、错误和系统事件。
- `Transcript`: Host 从 EventLog 投影出的对话视图,用于 UI、审计和按需历史读取。
- `Working context`: Agent 本轮实际送进模型或 runtime 的上下文,由 AgentRunner 决定。
- `Working context`: Agent 本轮实际送进模型或 runtime 的上下文,由 Runner 决定。
LangBot 不提供 host-side inline history window。简单 runner 如果需要历史窗口,应在 runner 内部通过 Host history API 拉取并裁剪。
## 2. Event 到来时传什么
默认 `AgentRunContext`PROTOCOL_V1 §5.2)应尽量小且稳定。默认规则:
默认 `RunnerContext`PROTOCOL_V1 §5.2)应尽量小且稳定。默认规则:
- Host MUST NOT inline full history by default.
- Host SHOULD inline only current event / input and context handles.
@@ -57,7 +57,7 @@ LangBot 不提供 host-side inline history window。简单 runner 如果需要
### 2.3 不提供 Host Inline History Window
`AgentRunContext` 不包含 `bootstrap` 字段。Host 不下发历史窗口,也不通过 Pipeline 配置决定窗口大小。runner 若需要类似 `recent_tail` 的策略,应在自己的 manifest/config schema 中声明参数,并在 runner 内部通过 history API 读取、裁剪和压缩。Host 只负责权限、分页、hard cap 和事实源。
`RunnerContext` 不包含 `bootstrap` 字段。Host 不下发历史窗口,也不通过 Pipeline 配置决定窗口大小。runner 若需要类似 `recent_tail` 的策略,应在自己的 manifest/config schema 中声明参数,并在 runner 内部通过 history API 读取、裁剪和压缩。Host 只负责权限、分页、hard cap 和事实源。
## 3. ContextAccess 的作用
@@ -65,7 +65,7 @@ LangBot 不提供 host-side inline history window。简单 runner 如果需要
## 4. Agent 如何获取更多上下文
所有 API 都走 `AgentRunAPIProxy`PROTOCOL_V1 §8),由 host 用 `run_id` 校验。
所有 API 都走 `RunnerAPIProxy`PROTOCOL_V1 §8),由 host 用 `run_id` 校验。
外部 harness 不能直接访问 LangBot 资源。无论是 history、event、state、model、tool、knowledge base,还是 LangBot skills,都必须通过 SDK runtime 转发到 Host API,并由 Host 按 active `run_id`、runner identity、binding resource policy 和 caller plugin identity 校验。当前运行文件进入授权 sandbox/workspace 后,再由 runner 用 read/write/exec 类工具按需访问。harness 自己的 native tools 只属于 harness 执行环境,不能绕过 SDK runtime 访问 LangBot 内部资源。
@@ -109,7 +109,7 @@ Claude Code、Codex、Kimi Code 这类 runtime 通常已有自己的 session、
- `agent-context.json`:结构化 JSON,包含 `run_id``event``actor``subject``input``delivery``resources``context``state``runtime`
- `LANGBOT_CONTEXT.md`:人类可读摘要。
- `resources`:只包含本次 run 授权后的资源句柄和能力摘要,不暴露 Host 内部私有对象、secret 或资源内容。
- `skills`LangBot skills 不是直接投影给 harness native tool loop 的文件能力,而是**一组被授权的 tool**。发现走 `list_skills`(或 `langbot_list_assets` 增加 skills 一类),激活/注册走 `activate` / `register_skill`,包内操作走 native exec/read/write,统一通过 `ctx.resources.tools``AgentRunAPIProxy` 或 SDK-owned MCP bridge 暴露。Host 不向 prompt 注入 skill 索引(无 progressive-disclosure 注入);harness 通过调用发现工具主动查询 skill 清单。`agent-context.json``skills` 字段仅作发现工具的数据来源与可选 `suggested_skill_prompt` 的输入。
- `skills`LangBot skills 不是直接投影给 harness native tool loop 的文件能力,而是**一组被授权的 tool**。发现走 `list_skills`(或 `langbot_list_assets` 增加 skills 一类),激活/注册走 `activate` / `register_skill`,包内操作走 native exec/read/write,统一通过 `ctx.resources.tools``RunnerAPIProxy` 或 SDK-owned MCP bridge 暴露。Host 不向 prompt 注入 skill 索引(无 progressive-disclosure 注入);harness 通过调用发现工具主动查询 skill 清单。`agent-context.json``skills` 字段仅作发现工具的数据来源与可选 `suggested_skill_prompt` 的输入。
- `MCP config`:只投影 per-run、scoped 的 SDK-owned bridge 或外部 MCP 连接配置;LangBot 资源访问必须回到 SDK runtime / Host API,不允许 harness 通过自带 MCP/native tool 直接读 Host 内部资源。
- `state pointers`:外部 session id、working directory、checkpoint 等小型 JSON 状态通过 Host state API 保存。
@@ -134,7 +134,7 @@ Host 只给当前事件、当前输入和 context handles。Runner 是否能拉
稳定 session key 的用途是隔离外部 runtime 的 resume/cache/state,不是改变 PROTOCOL_V1 §13 定义的 Agent 复用和 dispatch 边界。只有当某个外部 harness 的同一 native session 不支持并发 turn 时,runner 或 future runtime control plane 才应按 external session key 做 turn-level 串行化。
对长期运行的 external harness / daemon,推荐运行形态是 reader 与 writer 分离:一个 session reader 独占读取 stdout/SSE/native event stream,并把 native event 转成 `AgentRunResult` 或 task progress;用户输入只作为 turn write 进入该 session。当前一次性 CLI subprocess runner 可以继续在单次 `run(ctx)` 内同步收集 stdout,但后续改成长连接时不应让多个 request 同时读取同一 native stream。
对长期运行的 external harness / daemon,推荐运行形态是 reader 与 writer 分离:一个 session reader 独占读取 stdout/SSE/native event stream,并把 native event 转成 `RunnerResult` 或 task progress;用户输入只作为 turn write 进入该 session。当前一次性 CLI subprocess runner 可以继续在单次 `run(ctx)` 内同步收集 stdout,但后续改成长连接时不应让多个 request 同时读取同一 native stream。
## 7. Host guardrail
@@ -1,4 +1,4 @@
# Agent Runner QA 指南
# Runner QA 指南
本文档是 agent-runner 插件化下一轮测试的唯一 QA 入口。它合并并取代旧的 Phase 1 验收矩阵与 2026-05-18 / 2026-05-29 两份本地 QA 报告。
@@ -6,7 +6,7 @@
## 1. 测试边界
当前主线验证的是 AgentRunner Protocol v1
当前主线验证的是 Runner Protocol v1
```text
event -> binding -> runner.run(ctx) -> result stream
@@ -137,7 +137,7 @@ bin/lbs case list
通过条件:
- 用户可见回复正常。
- 后端日志显示走 `AgentRunOrchestrator` / `RUN_AGENT`
- 后端日志显示走 `AgentRunOrchestrator` / `RUN_RUNNER`
- 不走旧内置 local-agent 主执行分支。
- conversation transcript 写入用户消息和助手消息。
@@ -170,7 +170,7 @@ Smoke 前应优先保留一层轻量单测或 fixture 测试:session 创建/
步骤:
1. 确认目标 harness(例如 ACP daemon、Claude Code 或 Codex)在对应机器上可执行且已登录。
2. 绑定目标 runner,例如 `plugin:langbot-team/ACPAgentRunner/default``plugin:langbot-team/ClaudeCodeAgent/default``plugin:langbot-team/CodexAgent/default`
2. 绑定目标 runner,例如 `plugin:langbot-team/ACPRunner/default``plugin:langbot-team/ClaudeCodeAgent/default``plugin:langbot-team/CodexAgent/default`
3. 配置 runner 必要字段,例如 remote target、workspace、provider、startup timeout、reuse session 等。
4. 在 Debug Chat 执行一次确定性真实 smoke。
5. 检查 LangBot MCP gateway、`run_id` 回填和 host-owned state。
@@ -5,13 +5,13 @@
> 数据结构唯一定义在 [PROTOCOL_V1.md](./PROTOCOL_V1.md)runner 可见)与 [HOST_SDK_INFRASTRUCTURE.md](./HOST_SDK_INFRASTRUCTURE.md)Host 内部模型);本文只讲 EBA 语义,不重抄 schema。
> 与当前 runner 外化分支、后续 Agent Platform / Runtime Control Plane 的边界见 [EXTENSION_SCOPE_MATRIX.md](./EXTENSION_SCOPE_MATRIX.md)。
本文描述当前事件如何进入 LangBot、如何在平级的 Pipeline / Agent 之间路由,以及 Agent 如何复用插件化 AgentRunner。路由逻辑由 `pkg/platform/botmgr.py::RuntimeBot` 承担;文中的 EventRouter 表示职责,不代表独立进程或同名类。
本文描述当前事件如何进入 LangBot、如何在平级的 Pipeline / Agent 之间路由,以及 Agent 如何复用插件化 Runner。路由逻辑由 `pkg/platform/botmgr.py::RuntimeBot` 承担;文中的 EventRouter 表示职责,不代表独立进程或同名类。
## 1. 设计目标
- 消息、撤回、入群、好友申请、定时任务、API 调用都能抽象为 host event。
- EventRouter 可以根据 event type、bot、workspace、conversation、actor、subject 选择一个 Pipeline 或 Agent 处理器。
- Pipeline 目标执行完整消息 Stage 链;Agent 目标通过统一 orchestrator 调用 AgentRunner。
- Pipeline 目标执行完整消息 Stage 链;Agent 目标通过统一 orchestrator 调用 Runner。
- 非消息事件不伪造成用户文本消息。
- 平台动作通过已授权的语义工具执行;结构化交互通过 `action.requested` 中的 `interaction.requested` 白名单执行。
@@ -44,7 +44,7 @@
- 入口事件用 `AgentEventEnvelope`HOST_SDK §4.1)承载;顶层字段使用 LangBot 稳定协议名,平台原始事件名和原始 payload 放 `metadata` / `raw_ref`
- EBA 持久路由通过 `event_pattern``filters``target_type``target_uuid` 选择处理器。只有 `target_type=agent`,或 Pipeline AI Stage 需要调用 runner 时,才进一步解析 `AgentBinding`HOST_SDK §4.2)。
EBA 每个事件只选择一个有效处理器;AgentRunner 调用的基数、Agent 复用和 fan-out 边界以 PROTOCOL_V1 §13 为准。
EBA 每个事件只选择一个有效处理器;Runner 调用的基数、Agent 复用和 fan-out 边界以 PROTOCOL_V1 §13 为准。
路由 scope 示例:workspace 全局、bot 级、platform channel 级、conversation / group / thread 级、user / actor 级。Pipeline 是 `message.*` 场景的一等处理器,适合需要预处理、AI、后处理、扩展和输出控制的消息链路;Agent 是 runner 驱动的一等处理器,可处理其声明支持的消息与非消息事件。二者都不会被转换成对方。
@@ -59,12 +59,12 @@ Platform Adapter canonical event
-> RuntimeBot match saved event_bindings and resolve one Processor target
-> target_type=pipeline: MessageAggregator -> QueryPool -> Pipeline stages
-> target_type=agent: resolve AgentBinding -> AgentRunOrchestrator
-> AgentRunContextBuilder -> PluginRuntimeConnector.run_agent()
-> AgentRunResult stream
-> RunnerContextBuilder -> PluginRuntimeConnector.run_runner()
-> RunnerResult stream
-> Host result delivery / authorized platform tool
```
约束:Pipeline 和 Agent 是 EventRouter 的平级目标;Pipeline 仅接受消息事件,Agent 受其事件能力声明约束。任何 AgentRunner 调用都必须复用现有 orchestrator,不能为 EBA 单独实现另一套 plugin runner 协议;非消息事件不能绕过 resource authorizationdelivery 和 platform action 走统一权限模型;外部 harness runner 也通过同一套 envelope/binding/context/result 协议接入。observer / fan-out / parallel arbitration 的额外语义仍按 PROTOCOL_V1 §13 处理。
约束:Pipeline 和 Agent 是 EventRouter 的平级目标;Pipeline 仅接受消息事件,Agent 受其事件能力声明约束。任何 Runner 调用都必须复用现有 orchestrator,不能为 EBA 单独实现另一套 plugin runner 协议;非消息事件不能绕过 resource authorizationdelivery 和 platform action 走统一权限模型;外部 harness runner 也通过同一套 envelope/binding/context/result 协议接入。observer / fan-out / parallel arbitration 的额外语义仍按 PROTOCOL_V1 §13 处理。
## 6. 平台动作执行
@@ -84,13 +84,13 @@ Platform Adapter canonical event
## 7. 与 Context 协议的关系
EBA 事件进入 AgentRunner 时仍遵循 [AGENT_CONTEXT_PROTOCOL.md](./AGENT_CONTEXT_PROTOCOL.md)inline 当前事件、大 payload 用 raw/staged file ref、不默认 inline 完整 history、agent 按需通过 API 拉取、Host 保留 EventLog 和权限 guardrail。非消息事件可以被投影进 Transcript,但不能强制伪装为 user messageAgentRunner 根据 event type 自己决定是否纳入模型上下文。
EBA 事件进入 Runner 时仍遵循 [AGENT_CONTEXT_PROTOCOL.md](./AGENT_CONTEXT_PROTOCOL.md)inline 当前事件、大 payload 用 raw/staged file ref、不默认 inline 完整 history、agent 按需通过 API 拉取、Host 保留 EventLog 和权限 guardrail。非消息事件可以被投影进 Transcript,但不能强制伪装为 user messageRunner 根据 event type 自己决定是否纳入模型上下文。
## 8. 当前集成状态
当前分支已完成 EventRouter、Pipeline / Agent 平级处理器路由、Bot
`event_bindings` 持久化与 WebUI、AgentBinding 投影、路由 dry-run、合成测试事件、
运行状态和真实 OneBot 非消息事件到 Agent 的闭环。Pipeline 消息链和独立 Agent
均复用同一个 AgentRunner orchestrator / context / result 协议。
均复用同一个 Runner orchestrator / context / result 协议。
平台动作授权和结构化交互已实现,但真实平台/provider 验收不等同于单测通过。SDK 的 `platform_tools` 分类发现于 2026-09-05 检视时仍是未提交工作区改动。剩余发布工作和历史验证边界见 [STATUS.md](./STATUS.md)。通用订阅、Scheduler、Workflow 和多 Agent 串并联仍未作为产品交付。
@@ -96,4 +96,4 @@ The full regression also found Windows portability problems in the skills toolin
2. **Bounded stream pressure can fail long responses.** Two recovery runs emitted thinking and then `Streaming action consumer is too slow; response buffer full`; later recovery and six-tool runs succeeded. The error originates in the SDK's existing 128-frame response queue and is preserved in the UI. It is not a missing-tool-result or 30-second HTTP timeout. Full end-to-end flow control under sustained overload remains open; queue limits were not removed to conceal the failure.
3. Mock results verify debug behavior, not real platform credentials, permissions, delivery, or every adapter/event Cartesian combination. Already launched external tool termination timing was not independently measured. A development-branch push does not mean these remaining production release gates passed.
For reproduction, use the retained test Agent, the Mock examples in [AGENT_RUNNER_QA_GUIDE.md](./AGENT_RUNNER_QA_GUIDE.md), and the original welcome Agent. Local diagnostic traces and fixture artifacts are excluded from Git; they contain model responses and local runtime state. This report contains outcomes rather than raw provider reasoning or secrets.
For reproduction, use the retained test Agent, the Mock examples in [RUNNER_QA_GUIDE.md](./RUNNER_QA_GUIDE.md), and the original welcome Agent. Local diagnostic traces and fixture artifacts are excluded from Git; they contain model responses and local runtime state. This report contains outcomes rather than raw provider reasoning or secrets.
@@ -1,6 +1,6 @@
# AgentRunner 与产品扩展边界
# Runner 与产品扩展边界
更新:2026-09-05,适用于 `dev/4.11.x`。EBA、独立 Agent、Bot 事件绑定和处理器 UI 已与 AgentRunner 插件化合并。当前状态和测试证据以 [STATUS.md](./STATUS.md) 为准;runner 可见 schema 与调度基数以 [PROTOCOL_V1.md](./PROTOCOL_V1.md) 为准。
更新:2026-09-05,适用于 `dev/4.11.x`。EBA、独立 Agent、Bot 事件绑定和处理器 UI 已与 Runner 插件化合并。当前状态和测试证据以 [STATUS.md](./STATUS.md) 为准;runner 可见 schema 与调度基数以 [PROTOCOL_V1.md](./PROTOCOL_V1.md) 为准。
## 当前职责
@@ -9,7 +9,7 @@
| LangBot 产品层 | 独立 Agent CRUD、Pipeline、处理器工作台、Bot 事件绑定、Runner 安装与调试 | Agent 与 Pipeline 各自持久化;聚合列表不转换实体 |
| 平台层 | 适配器事件转换、能力声明、observer 广播、路由匹配、平台 API 与回复 | 路由逻辑在 RuntimeBot;不为每个入口重建 runner 协议 |
| Host Agent 底座 | envelope/binding 投影、统一编排、资源授权、run session、EventLog/Transcript/State、run/result ledger | SDK 不持有 Host 私有 Query 或数据库 |
| SDK / Plugin Runtime | typed contract、AgentRunner 组件和脚手架、proxy、MCP bridge、结果流转发、installation worker 管理 | 具体 Agent 执行策略由 Runner 插件承担 |
| SDK / Plugin Runtime | typed contract、Runner 组件和脚手架、proxy、MCP bridge、结果流转发、installation worker 管理 | 具体 Agent 执行策略由 Runner 插件承担 |
| Box Runtime | 沙盒会话、文件、托管进程、Skill、资源限制与作用域 | 不等于外部 harness 的通用托管承诺;存储统计不等于硬配额 |
## 已有能力与后续扩展
@@ -2,24 +2,24 @@
本文档描述 LangBot 作为 agent host 的内部能力与分层架构,以及 Host 内部模型。
- SDK ↔ Host 的协议数据结构(`AgentRunContext``AgentRunnerManifest``AgentRunResult``AgentRunAPIProxy` 等)的**唯一定义在** [PROTOCOL_V1.md](./PROTOCOL_V1.md);本文只引用,不重抄。
- 测试执行入口和 smoke 记录见 [AGENT_RUNNER_QA_GUIDE.md](./AGENT_RUNNER_QA_GUIDE.md);安全发布门槛见 [SECURITY_HARDENING.md](./SECURITY_HARDENING.md)。
- 本文定义的 Host 内部模型(`AgentEventEnvelope``AgentBinding``AgentRunnerDescriptor`)不属于 SDK 协议字段。
- SDK ↔ Host 的协议数据结构(`RunnerContext``RunnerManifest``RunnerResult``RunnerAPIProxy` 等)的**唯一定义在** [PROTOCOL_V1.md](./PROTOCOL_V1.md);本文只引用,不重抄。
- 测试执行入口和 smoke 记录见 [RUNNER_QA_GUIDE.md](./RUNNER_QA_GUIDE.md);安全发布门槛见 [SECURITY_HARDENING.md](./SECURITY_HARDENING.md)。
- 本文定义的 Host 内部模型(`AgentEventEnvelope``AgentBinding``RunnerDescriptor`)不属于 SDK 协议字段。
## 1. 目标
LangBot 要转为 agent host,而不是内置 runner 容器:
- 接收 IM、WebUI、API 和当前 RuntimeBot 事件路由产生的事件。
- 接收 EBA 选中的 Agent 处理器,并根据事件、bot、workspace、scope 解析 AgentRunner binding。
- 发现、校验和调用插件提供的 AgentRunner。
- 接收 EBA 选中的 Agent 处理器,并根据事件、bot、workspace、scope 解析 Runner binding。
- 发现、校验和调用插件提供的 Runner。
- 为每次 run 提供受限资源、状态、存储、上下文引用和生命周期控制。
- 接收 AgentRunner 返回的事件流,投递到 IM、WebUI 或其他 output surface。
- 接收 Runner 返回的事件流,投递到 IM、WebUI 或其他 output surface。
## 2. 非目标
- 不定义 Pipeline 的 Stage 编排语义;Pipeline 是 EBA 的同级处理器,其 AI Stage 只在需要 runner 时接入本 Host 边界。
- 不要求所有 AgentRunner 依赖 LangBot 的上下文管理。
- 不要求所有 Runner 依赖 LangBot 的上下文管理。
- 不要求官方 local-agent 的旧行为反向塑造 host 协议。
- 不在 host 中实现通用 agentic prompt assembler。
- 不强制 runner 使用 LangBot state / storage;只提供可选、受控的寄宿能力。
@@ -41,17 +41,17 @@ RuntimeBot event_bindings -> one Processor target
|
v
AgentRunOrchestrator
|-- AgentRunnerRegistry
|-- RunnerRegistry
|-- AgentResourceBuilder
|-- AgentContextBuilder
|-- AgentRunSessionRegistry
|-- PersistentStateStore / EventLogStore / TranscriptStore
|-- Sandbox / workspace file tools
v
Plugin Runtime / AgentRunner
Plugin Runtime / Runner
|
v
AgentRunResult stream
RunnerResult stream
|
v
Delivery / Renderer / Platform API
@@ -91,7 +91,7 @@ class AgentEventEnvelope(BaseModel):
### 4.2 AgentConfig 与 AgentBinding
`AgentConfig` 是 Host 内部的一次 AgentRunner 调用配置投影(不暴露给 SDK)。独立 Agent 从自己的持久配置生成它;Pipeline 只在 AI Stage 调用 runner 时,由 Query entry adapter 从该 Stage 的当前配置生成它。两种来源随后都由 BindingResolver 结合事件和 scope 解析为 `AgentBinding`。Pipeline 本身不是 `AgentConfig`,该调用投影也不会创建或更新持久 Agent。
`AgentConfig` 是 Host 内部的一次 Runner 调用配置投影(不暴露给 SDK)。独立 Agent 从自己的持久配置生成它;Pipeline 只在 AI Stage 调用 runner 时,由 Query entry adapter 从该 Stage 的当前配置生成它。两种来源随后都由 BindingResolver 结合事件和 scope 解析为 `AgentBinding`。Pipeline 本身不是 `AgentConfig`,该调用投影也不会创建或更新持久 Agent。
```python
class AgentConfig(BaseModel):
@@ -106,7 +106,7 @@ class AgentConfig(BaseModel):
metadata: dict[str, Any] = {}
```
`AgentBinding` 是"什么事件调用哪个 AgentRunner、带什么 Agent 配置"的 Host 内部运行投影(不暴露给 SDK)。它是 EventRouter / 当前 QueryEntryAdapter 在一次运行前解析出的有效绑定。
`AgentBinding` 是"什么事件调用哪个 Runner、带什么 Agent 配置"的 Host 内部运行投影(不暴露给 SDK)。它是 EventRouter / 当前 QueryEntryAdapter 在一次运行前解析出的有效绑定。
```python
class AgentBinding(BaseModel):
@@ -131,12 +131,12 @@ BindingResolver 的基数、fan-out 和冲突处理约束见 PROTOCOL_V1 §13
→ runner_config、extension preference → resource_policy、output settings →
delivery_policy,但 Pipeline 仍执行并拥有完整 Stage/config 语义。该适配不会把 Pipeline 持久化为 Agent;独立 Agent 由用户自行新增和绑定。
### 4.3 AgentRunnerRegistry
### 4.3 RunnerRegistry
Registry 收集 runner descriptor(来自插件 runtime、开发期本地插件):
```python
class AgentRunnerDescriptor(BaseModel):
class RunnerDescriptor(BaseModel):
id: str
source: Literal["plugin"]
label: I18nObject
@@ -144,17 +144,17 @@ class AgentRunnerDescriptor(BaseModel):
plugin_author: str
plugin_name: str
runner_name: str
capabilities: AgentRunnerCapabilities # 见 PROTOCOL_V1 §4.3
permissions: AgentRunnerPermissions # 见 PROTOCOL_V1 §4.4
capabilities: RunnerCapabilities # 见 PROTOCOL_V1 §4.3
permissions: RunnerPermissions # 见 PROTOCOL_V1 §4.4
config_schema: list[DynamicFormItemSchema]
plugin_version: str | None = None
raw_manifest: dict[str, Any] = {}
```
职责:调用 `plugin_connector.list_agent_runners()` 拉取 runner、校验 typed `AgentRunnerManifest`、输出 descriptor、缓存 discovery 结果并提供 `refresh()`。单个插件 manifest 失败只记 warning,不影响其它 runner。`plugin:author/name/runner` 是稳定 id 格式;插件实例边界见 PROTOCOL_V1 §13。
职责:调用 `plugin_connector.list_runners()` 拉取 runner、校验 typed `RunnerManifest`、输出 descriptor、缓存 discovery 结果并提供 `refresh()`。单个插件 manifest 失败只记 warning,不影响其它 runner。`plugin:author/name/runner` 是稳定 id 格式;插件实例边界见 PROTOCOL_V1 §13。
Host 内置 runner / adapter 不能作为 `AgentRunnerDescriptor.source` 绕过插件
runtime、`run_id``ctx.resources``AgentRunAPIProxy` 权限链。若需要
Host 内置 runner / adapter 不能作为 `RunnerDescriptor.source` 绕过插件
runtime、`run_id``ctx.resources``RunnerAPIProxy` 权限链。若需要
开发期调试 adapter,应放在 Host 内部测试入口,不进入可选 runner 列表。
刷新触发点:插件安装/卸载/升级/重启后;Pipeline metadata 请求时发现缓存为空;可选 TTL(优先保证正确性)。
@@ -182,15 +182,15 @@ run(event, binding)
```text
QueryEntryAdapter / EventRouter
-> AgentRunOrchestrator.run(event, binding)
-> AgentRunnerRegistry.resolve(runner_id)
-> RunnerRegistry.resolve(runner_id)
-> AgentResourceBuilder.freeze_snapshot(binding, event)
-> AgentRunSessionRegistry.register(run_id, runner_id, snapshot)
-> AgentContextBuilder.build(event, binding, snapshot)
-> PluginRuntimeConnector.run_agent(ctx)
-> AgentRunAPIProxy action
-> PluginRuntimeConnector.run_runner(ctx)
-> RunnerAPIProxy action
-> validate active run session + caller identity + snapshot
-> Host API / Store
<- AgentRunResult stream
<- RunnerResult stream
-> apply state.updated to PersistentStateStore
-> write message.completed to Transcript
-> keep current-run files and large tool outputs in sandbox/workspace
@@ -237,11 +237,11 @@ LangBot 可提供 host-owned state 让 runner 寄宿状态(conversation / acto
- `Transcript`: 从 EventLog 投影出的对话视图,用于 UI、审计和按需历史读取。
- `Sandbox / workspace files`: 当前 run 的上传文件、平台附件、工具大结果和临时产物。Host 负责 staging 与授权边界,runner 通过 read/write/exec 类工具按需访问。
三类数据与 working context 的边界、读取约束见 [AGENT_CONTEXT_PROTOCOL.md](./AGENT_CONTEXT_PROTOCOL.md)。AgentRunner 可读取这些能力,但不被迫使用 LangBot 作为唯一记忆系统。
三类数据与 working context 的边界、读取约束见 [AGENT_CONTEXT_PROTOCOL.md](./AGENT_CONTEXT_PROTOCOL.md)。Runner 可读取这些能力,但不被迫使用 LangBot 作为唯一记忆系统。
### 4.8 External harness resource projection
Claude Code、Codex、Kimi Code 等外部 harness runner 可能不直接调用 LangBot 的 model/tool loop,而是把 LangBot 事件和授权资源句柄投影到自己的 harness 执行。Host 侧仍保持统一边界:Host 负责构造 event-first context、资源授权、state/storage、EventLog/Transcript、sandbox/workspace 文件边界和审计;Host 或 binding policy 决定哪些 MCP bridge、skill-backed tool、sandbox path、history/state 句柄可投影给 runnerrunner plugin 把 scoped projection 转成目标 harness 可消费形式;所有 LangBot 资源访问必须经 SDK runtime / `AgentRunAPIProxy` / SDK-owned MCP bridge 转发并接受 Host 校验;外部 harness 负责自己的 native session、tool loop、压缩、权限模式和 resume,但不能用 native tools 绕过 Host 授权。
Claude Code、Codex、Kimi Code 等外部 harness runner 可能不直接调用 LangBot 的 model/tool loop,而是把 LangBot 事件和授权资源句柄投影到自己的 harness 执行。Host 侧仍保持统一边界:Host 负责构造 event-first context、资源授权、state/storage、EventLog/Transcript、sandbox/workspace 文件边界和审计;Host 或 binding policy 决定哪些 MCP bridge、skill-backed tool、sandbox path、history/state 句柄可投影给 runnerrunner plugin 把 scoped projection 转成目标 harness 可消费形式;所有 LangBot 资源访问必须经 SDK runtime / `RunnerAPIProxy` / SDK-owned MCP bridge 转发并接受 Host 校验;外部 harness 负责自己的 native session、tool loop、压缩、权限模式和 resume,但不能用 native tools 绕过 Host 授权。
投影的具体形态(context 文件、resource handles、LangBot MCP gateway、state pointers)见 AGENT_CONTEXT_PROTOCOL §4.5;当前 code-agent harness runner 形态见 OFFICIAL_RUNNER_PLUGINS §7。发布级隔离要求见 SECURITY_HARDENING。
@@ -250,17 +250,17 @@ Claude Code、Codex、Kimi Code 等外部 harness runner 可能不直接调用 L
SDK 组件入口如下;所有数据结构定义见 PROTOCOL_V1。
```python
class AgentRunner(BaseComponent):
__kind__ = "AgentRunner"
class Runner(BaseComponent):
__kind__ = "Runner"
@classmethod
def get_config_schema(cls) -> list[dict]: ...
async def run(self, ctx: AgentRunContext) -> AsyncGenerator[AgentRunResult, None]: ...
# ctx: PROTOCOL_V1 §5.2 ; AgentRunResult: PROTOCOL_V1 §7
async def run(self, ctx: RunnerContext) -> AsyncGenerator[RunnerResult, None]: ...
# ctx: PROTOCOL_V1 §5.2 ; RunnerResult: PROTOCOL_V1 §7
```
- Manifest / capabilities / effective accessPROTOCOL_V1 §4。Capabilities 来自组件 manifest 的 `spec.capabilities`,不是 SDK 基类 classmethod。
- `AgentRunContext`PROTOCOL_V1 §5.2。`messages` / `bootstrap` 不是协议字段。
- `AgentRunResult`PROTOCOL_V1 §7。
- `AgentRunAPIProxy`PROTOCOL_V1 §8,是 runner 访问 host 能力的唯一入口,所有请求带 `run_id`
- `RunnerContext`PROTOCOL_V1 §5.2。`messages` / `bootstrap` 不是协议字段。
- `RunnerResult`PROTOCOL_V1 §7。
- `RunnerAPIProxy`PROTOCOL_V1 §8,是 runner 访问 host 能力的唯一入口,所有请求带 `run_id`
@@ -1,12 +1,12 @@
# 官方 AgentRunner 插件迁移计划
# 官方 Runner 插件迁移计划
本文档描述内置 `RequestRunner` 迁出 LangBot 后,官方 runner 插件如何组织、迁移和验收。它是 [HOST_SDK_INFRASTRUCTURE.md](./HOST_SDK_INFRASTRUCTURE.md) 和 [AGENT_CONTEXT_PROTOCOL.md](./AGENT_CONTEXT_PROTOCOL.md) 的下游落地计划,不是 LangBot 宿主协议的设计前提。QA 入口和 smoke 记录见 [AGENT_RUNNER_QA_GUIDE.md](./AGENT_RUNNER_QA_GUIDE.md)。
本文档描述内置 `RequestRunner` 迁出 LangBot 后,官方 runner 插件如何组织、迁移和验收。它是 [HOST_SDK_INFRASTRUCTURE.md](./HOST_SDK_INFRASTRUCTURE.md) 和 [AGENT_CONTEXT_PROTOCOL.md](./AGENT_CONTEXT_PROTOCOL.md) 的下游落地计划,不是 LangBot 宿主协议的设计前提。QA 入口和 smoke 记录见 [RUNNER_QA_GUIDE.md](./RUNNER_QA_GUIDE.md)。
官方 `local-agent` 可以外移,也可以重写。设计重点不是保留旧内置 runner 的内部结构,而是验证一个依附 LangBot host 基础设施的官方 agent 能否完整工作。同时,LangBot host 协议必须服务 Claude Code SDK、Codex、Pi Agent SDK、外部 Agent 平台等自管 context/runtime 的 runner,不能被官方插件的实现细节绑死。
## 1. 仓库组织
官方 runner 插件与 LangBot 主仓库、SDK 仓库以不同节奏迭代:LangBot 主仓库只维护宿主协议和调度,SDK 仓库维护 AgentRunner 组件和 runtime 协议,官方 runner 插件承载业务 runner 的具体实现和第三方平台适配。
官方 runner 插件与 LangBot 主仓库、SDK 仓库以不同节奏迭代:LangBot 主仓库只维护宿主协议和调度,SDK 仓库维护 Runner 组件和 runtime 协议,官方 runner 插件承载业务 runner 的具体实现和第三方平台适配。
当前推荐"官方插件可独立发布,必要时共享 SDK helper"。开发期采用本地多目录布局:
@@ -14,7 +14,7 @@
langbot-app/
langbot-local-agent/ # plugin:langbot-team/LocalAgent/default
manifest.yaml
components/agent_runner/default.{yaml,py}
components/runner/default.{yaml,py}
langbot-agent-runner/ # 外部服务 runner 仓库
acp-agent-runner/ claude-code-agent/ codex-agent/ dify-agent/ n8n-agent/ ...
```
@@ -29,7 +29,7 @@ langbot-app/
| `dify-service-api` | `langbot-team/DifyAgent` | `plugin:langbot-team/DifyAgent/default` |
| `n8n-service-api` | `langbot-team/N8nAgent` | `plugin:langbot-team/N8nAgent/default` |
| `coze-api` | `langbot-team/CozeAgent` | `plugin:langbot-team/CozeAgent/default` |
| - | `langbot-team/ACPAgentRunner` | `plugin:langbot-team/ACPAgentRunner/default` |
| - | `langbot-team/ACPRunner` | `plugin:langbot-team/ACPRunner/default` |
| - | `langbot-team/ClaudeCodeAgent` | `plugin:langbot-team/ClaudeCodeAgent/default` |
| - | `langbot-team/CodexAgent` | `plugin:langbot-team/CodexAgent/default` |
| `dashscope-app-api` | `langbot-team/DashScopeAgent` | `plugin:langbot-team/DashScopeAgent/default` |
@@ -48,23 +48,23 @@ langbot-app/
## 4. 每个官方插件的组件要求
每个插件至少包含一个 `AgentRunner` 组件,manifest 示例:
每个插件至少包含一个 `Runner` 组件,manifest 示例:
```yaml
apiVersion: langbot/v1
kind: AgentRunner
kind: Runner
metadata:
name: default
label: { en_US: Dify Agent, zh_Hans: Dify Agent }
description:
en_US: Run a Dify application as a LangBot AgentRunner.
zh_Hans: 将 Dify 应用作为 LangBot AgentRunner 运行。
en_US: Run a Dify application as a LangBot Runner.
zh_Hans: 将 Dify 应用作为 LangBot Runner 运行。
spec:
config: []
capabilities: # 字段语义见 PROTOCOL_V1 §4.3
streaming: true
execution:
python: { path: ./main.py, attr: DefaultAgentRunner }
python: { path: ./main.py, attr: DefaultRunner }
```
## 5. local-agent 插件方向
@@ -76,18 +76,18 @@ execution:
责任边界与 Host API 消费方式见 AGENT_CONTEXT_PROTOCOL §8。关键约束:
-`ctx.config` 读取静态绑定 `prompt`**不**读取 `ctx.adapter.extra["prompt"]`;不消费 Query entry adapter 生成的历史窗口。
- 通过 `AgentRunAPIProxy.history` 拉取 transcript,而不是依赖 host 每轮强塞历史窗口。
- 通过 `RunnerAPIProxy.history` 拉取 transcript,而不是依赖 host 每轮强塞历史窗口。
- `ctx.input.contents` 保留图片/文件等多模态内容;RAG 只替换/插入文本部分,不丢图片/文件。
- 不能绕过 `ctx.resources` 调用未授权模型、工具或知识库。
- manifest 声明功能能力、LangBot 资源 permissions 和配置表单;实际授权来自 manifest permissions 与 binding resource policy、runner config、`ctx.context.available_apis` 和 Host run session snapshot 的交集。
### 5.1 Native Execution / Skills 后续接入
本阶段不把 sandbox/skills 做成 AgentRunner 协议字段。后续 sandbox/skills 分支合并后,命令执行、文件操作、skill、MCP managed process 应先由 Host / sandbox 封装成 scoped tools,再通过 `ctx.resources.tools` 和 SDK runtime 转发暴露给 runner。这让 local-agent 只消费授权后的 Host 基础设施,而不是直接持有宿主机执行能力。
本阶段不把 sandbox/skills 做成 Runner 协议字段。后续 sandbox/skills 分支合并后,命令执行、文件操作、skill、MCP managed process 应先由 Host / sandbox 封装成 scoped tools,再通过 `ctx.resources.tools` 和 SDK runtime 转发暴露给 runner。这让 local-agent 只消费授权后的 Host 基础设施,而不是直接持有宿主机执行能力。
## 6. 外部 runner 插件要求
外部平台 runner 迁移遵循:旧配置字段尽量保持同名便于 migration 复制;输出统一转换为 `AgentRunResult`;外部 API timeout 从 runner config 读取;平台 conversation id 存 plugin storage 或 context runtime state,不依赖 LangBot 内置 conversation uuid 私有结构;流式按平台能力声明,没有流式就只发 `message.completed`
外部平台 runner 迁移遵循:旧配置字段尽量保持同名便于 migration 复制;输出统一转换为 `RunnerResult`;外部 API timeout 从 runner config 读取;平台 conversation id 存 plugin storage 或 context runtime state,不依赖 LangBot 内置 conversation uuid 私有结构;流式按平台能力声明,没有流式就只发 `message.completed`
### 6.1 Code-agent harness runner
@@ -95,7 +95,7 @@ Claude Code、Codex、Kimi Code 这类 runner 不一定通过 LangBot 的模型/
本文件只补充官方 runner 的实现要求:输入来自 `ctx.event` / `ctx.input`,不依赖 Pipeline 私有 `Query`;外部 session id / workspace / checkpoint 写入 Host state 或 plugin storage;插件实例边界见 PROTOCOL_V1 §13CLI / subprocess runner 必须处理 timeout、取消、空输出、非零退出和 stderr 映射。
实现结构应把 provider-native output 解析与 LangBot result stream 组装分开:Claude stream-json、Codex JSONL、Kimi / OpenCode 事件等只在 runner adapter 内解析,输出统一归一为 `AgentRunResult``message.completed` / `message.delta``state.updated``run.completed` / `run.failed`)。文件和工具大结果留在当前 run 的 sandbox/workspace,通过消息 metadata、attachment ref 或 path 指向。未知 native event 不应导致 run 崩溃;应记录诊断 metadata 或 warning。新增 harness 时优先补 native fixture -> `AgentRunResult` 的转换测试,再接 WebUI smoke。
实现结构应把 provider-native output 解析与 LangBot result stream 组装分开:Claude stream-json、Codex JSONL、Kimi / OpenCode 事件等只在 runner adapter 内解析,输出统一归一为 `RunnerResult``message.completed` / `message.delta``state.updated``run.completed` / `run.failed`)。文件和工具大结果留在当前 run 的 sandbox/workspace,通过消息 metadata、attachment ref 或 path 指向。未知 native event 不应导致 run 崩溃;应记录诊断 metadata 或 warning。新增 harness 时优先补 native fixture -> `RunnerResult` 的转换测试,再接 WebUI smoke。
并发约束应按外部 session 粒度表达,而不是按 Agent / runner id / 插件实例表达;Agent 复用和全局锁边界见 PROTOCOL_V1 §13。若 runner 使用 `external.session_id` / `thread_id` resume 到同一 native session,且该 harness 不支持并发 turnrunner 应按稳定 external session key 串行写入;一次性 subprocess runner 可以只在单次 `run(ctx)` 内处理,长连接/daemon runner 则应采用 reader 独占 native stream、turn writer 串行写入的结构。
@@ -111,11 +111,11 @@ Claude Code、Codex、Kimi Code 这类 runner 不一定通过 LangBot 的模型/
## 7. Code-agent harness runner 当前形态
外部 code-agent harness 由直接 runner 插件承接,例如 `acp-agent-runner``claude-code-agent``codex-agent`,每个 runner 负责把目标 harness 的 native session、workspace、MCP bridge 和输出事件转换为统一 `AgentRunResult`。本地 smoke 验收入口与记录见 [AGENT_RUNNER_QA_GUIDE.md](./AGENT_RUNNER_QA_GUIDE.md)。
外部 code-agent harness 由直接 runner 插件承接,例如 `acp-agent-runner``claude-code-agent``codex-agent`,每个 runner 负责把目标 harness 的 native session、workspace、MCP bridge 和输出事件转换为统一 `RunnerResult`。本地 smoke 验收入口与记录见 [RUNNER_QA_GUIDE.md](./RUNNER_QA_GUIDE.md)。
当前形态:
- Runner ID 示例:`plugin:langbot-team/ACPAgentRunner/default``plugin:langbot-team/ClaudeCodeAgent/default``plugin:langbot-team/CodexAgent/default`
- Runner ID 示例:`plugin:langbot-team/ACPRunner/default``plugin:langbot-team/ClaudeCodeAgent/default``plugin:langbot-team/CodexAgent/default`
- Runner 可通过 ACP、远端 daemon、本机 subprocess 或外部 HTTP API 调用 harnessharness 的安装、登录态、workspace 和 provider-native 权限由该运行环境负责。
- Runner 会把当前 LangBot `run_id`、可访问资源摘要和 gateway 使用规则注入本次消息;harness 通过 gateway 回填 `run_id` 后访问 LangBot 资产。
- 外部 session id / workspace / checkpoint 写回 Host state 或 plugin storage,后续轮次可复用目标 harness 会话。
@@ -126,11 +126,11 @@ Claude Code、Codex、Kimi Code 这类 runner 不一定通过 LangBot 的模型/
## 8. 发布和安装策略
最终 LangBot 安装/升级时需保证官方 runner 插件可用,可选方案:首次启动检测缺失并提示安装,或由用户从 marketplace 安装。当前分支未发布,因此不保留历史 Pipeline Agent 配置兼容、旧内置 runner fallback,也不把旧 Pipeline 内的 Agent 配置迁移成独立 Agent。4.x 只读取 `ai.runner.id``ai.runner_config[id]`;升级后由用户选择或安装需要的 AgentRunner。
最终 LangBot 安装/升级时需保证官方 runner 插件可用,可选方案:首次启动检测缺失并提示安装,或由用户从 marketplace 安装。当前分支未发布,因此不保留历史 Pipeline Agent 配置兼容、旧内置 runner fallback,也不把旧 Pipeline 内的 Agent 配置迁移成独立 Agent。4.x 只读取 `ai.runner.id``ai.runner_config[id]`;升级后由用户选择或安装需要的 Runner。
## 9. 验收标准
- 每个目标 runner 都有对应官方 AgentRunner 插件和稳定 runner id;当前配置只使用 `ai.runner.id` + `ai.runner_config[id]`
- 每个目标 runner 都有对应官方 Runner 插件和稳定 runner id;当前配置只使用 `ai.runner.id` + `ai.runner_config[id]`
- LangBot 主聊天路径不再通过 `RequestRunner` 执行业务 runner。
- 官方插件测试覆盖非流式、流式、错误、timeout、配置缺失。
- `local-agent` 能完成模型 fallback、tool calling、知识库检索、多模态输入、静态绑定 prompt 消费、history API 拉取、rerank。
@@ -1,6 +1,6 @@
# Agent 工具权限
Agent 配置页展示同一次运行中可能投射给 AgentRunner 的完整工具目录:
Agent 配置页展示同一次运行中可能投射给 Runner 的完整工具目录:
- 事件级工具由 Agent 选择的事件范围自动启用。
- `allowed_platform_tools` 管理需要 Agent 自行指定目标的平台级动作。
@@ -12,7 +12,7 @@ Host 会按当前 Workspace 实时解析工具来源。未安装的插件、未
Agent 不直接持有平台适配器,也不能调用任意原始平台接口。每次运行时,Host 根据当前
事件自动加入兼容的事件级工具,并加入 `allowed_platform_tools` 中选择的平台级工具,
再与 AgentRunner 权限、当前适配器声明的 API、当前事件能够安全绑定的目标取交集,
再与 Runner 权限、当前适配器声明的 API、当前事件能够安全绑定的目标取交集,
得到 `ctx.resources.tools``tool_type=platform` 的最终工具集合。
## 两类工具
@@ -38,16 +38,16 @@ Agent 不直接持有平台适配器,也不能调用任意原始平台接口
current event type ── compatible event tools
Agent.allowed_platform_tools ── selected platform tools
├─ AgentRunner capability tool_calling is enabled
├─ AgentRunner manifest permissions.tools contains call
├─ Runner capability tool_calling is enabled
├─ Runner manifest permissions.tools contains call
├─ current adapter.get_supported_apis()
└─ current event type and frozen target are compatible
ctx.resources.tools[tool_type=platform]
├─ Local Agent: AgentRunAPIProxy.call_tool
└─ External AgentRunner: langbot_list_assets / langbot_get_tool_detail /
├─ Local Agent: RunnerAPIProxy.call_tool
└─ External Runner: langbot_list_assets / langbot_get_tool_detail /
langbot_call_tool (MCP Asset Gateway)
@@ -57,7 +57,7 @@ Host revalidates run_id, runner plugin identity, operation and frozen source
current bot adapter semantic API
```
本地和外部 AgentRunner 因此使用同一个工具名、参数 Schema 和 Host 授权快照。外部
本地和外部 Runner 因此使用同一个工具名、参数 Schema 和 Host 授权快照。外部
平台不会获得适配器对象或长期凭据;MCP 网关中的 run token 和 Host 中的 run session
都只对应当前运行。
+41 -41
View File
@@ -1,8 +1,8 @@
# LangBot AgentRunner Protocol v1
# LangBot Runner Protocol v1
本文档是 LangBot Host 与插件 SDK / Runtime / AgentRunner 之间协议合同的**唯一规范来源(single source of truth**。
本文档是 LangBot Host 与插件 SDK / Runtime / Runner 之间协议合同的**唯一规范来源(single source of truth**。
- 本文件描述当前 Protocol v1 稳定合同,不混入验收流水。当前实现状态见 [STATUS.md](./STATUS.md),测试执行入口见 [AGENT_RUNNER_QA_GUIDE.md](./AGENT_RUNNER_QA_GUIDE.md),安全发布门槛见 [SECURITY_HARDENING.md](./SECURITY_HARDENING.md)。
- 本文件描述当前 Protocol v1 稳定合同,不混入验收流水。当前实现状态见 [STATUS.md](./STATUS.md),测试执行入口见 [RUNNER_QA_GUIDE.md](./RUNNER_QA_GUIDE.md),安全发布门槛见 [SECURITY_HARDENING.md](./SECURITY_HARDENING.md)。
- 本文件之外的任何文档**不得重新定义这里的数据结构**,只能引用,例如"见 PROTOCOL_V1 §4.2"。
- Host 内部模型(`AgentEventEnvelope``AgentBinding`、Descriptor、各 Store)不属于 SDK 协议,定义在 [HOST_SDK_INFRASTRUCTURE.md](./HOST_SDK_INFRASTRUCTURE.md)。
@@ -10,15 +10,15 @@
Protocol v1 只解决四件事:
- LangBot 如何发现插件提供的 AgentRunner。
- LangBot 如何把一次事件调用封装成 `AgentRunContext`
- AgentRunner 如何以事件流形式返回运行结果。
- AgentRunner 如何通过受限 API 访问 LangBot host 能力。
- LangBot 如何发现插件提供的 Runner。
- LangBot 如何把一次事件调用封装成 `RunnerContext`
- Runner 如何以事件流形式返回运行结果。
- Runner 如何通过受限 API 访问 LangBot host 能力。
Protocol v1 **不定义**
- LangBot 内部如何持久化 `AgentBinding`(见 HOST_SDK)。
- AgentRunner 内部如何组装 prompt、压缩历史、管理 memory(见 [AGENT_CONTEXT_PROTOCOL.md](./AGENT_CONTEXT_PROTOCOL.md))。
- Runner 内部如何组装 prompt、压缩历史、管理 memory(见 [AGENT_CONTEXT_PROTOCOL.md](./AGENT_CONTEXT_PROTOCOL.md))。
- 官方 runner 的具体实现(见 [OFFICIAL_RUNNER_PLUGINS.md](./OFFICIAL_RUNNER_PLUGINS.md))。
- Pipeline 的长期配置模型。
- 发布级安全 hardening 的完整实现(见 [SECURITY_HARDENING.md](./SECURITY_HARDENING.md))。
@@ -29,8 +29,8 @@ Protocol v1 **不定义**
| --- | --- |
| LangBot Host | 事件入口、绑定解析、权限、资源、存储、生命周期、结果投递。 |
| Plugin Runtime | 加载插件,响应 Host 的 runner discovery 和 run 调用。 |
| AgentRunner | 插件提供的 agent 执行组件。 |
| AgentRunAPIProxy | AgentRunner 访问 Host 能力的受限 API。 |
| Runner | 插件提供的 agent 执行组件。 |
| RunnerAPIProxy | Runner 访问 Host 能力的受限 API。 |
| AgentBinding | Host 内部的事件到 runner 绑定配置,不直接暴露给 SDK(见 HOST_SDK §4.2)。 |
产品层同时保留 Pipeline 与独立 `Agent`:现有 Pipeline 不迁移为 Agent
@@ -39,11 +39,11 @@ Protocol v1 **不定义**
`ctx.config``ctx.resources``ctx.context``ctx.delivery`。SDK 不需要知道
Agent / binding 的持久化形态。
外部 harness runnerClaude Code、Codex、Kimi Code 等)也是 `AgentRunner`:它们消费 event-first `AgentRunContext`、返回 `AgentRunResult`,并通过 Host 授权的 state/storage API 保存跨轮次指针;当前运行文件和工具大结果进入 sandbox/workspace。它们内部可以继续使用自己的 session、tool loop、MCP、上下文压缩和权限模型。
外部 harness runnerClaude Code、Codex、Kimi Code 等)也是 `Runner`:它们消费 event-first `RunnerContext`、返回 `RunnerResult`,并通过 Host 授权的 state/storage API 保存跨轮次指针;当前运行文件和工具大结果进入 sandbox/workspace。它们内部可以继续使用自己的 session、tool loop、MCP、上下文压缩和权限模型。
## 3. 协议演进
当前 AgentRunner 合同不暴露显式 `protocol_version` 字段。协议演进先按字段级兼容规则处理:
当前 Runner 合同不暴露显式 `protocol_version` 字段。协议演进先按字段级兼容规则处理:
- 新增可选字段保持向后兼容。
- 删除字段或改变既有字段语义,需要在 SDK 发布前完成;发布后应走新的显式兼容方案。
@@ -52,35 +52,35 @@ Agent / binding 的持久化形态。
## 4. Discovery 协议
### 4.1 LIST_AGENT_RUNNERS
### 4.1 LIST_RUNNERS
Host 调用 Plugin Runtime 获取当前插件暴露的 runner 列表,请求无额外 payload。返回:
```python
class ListAgentRunnersResponse(BaseModel):
runners: list[AgentRunnerDiscovery]
class ListRunnersResponse(BaseModel):
runners: list[RunnerDiscovery]
class AgentRunnerDiscovery(BaseModel):
class RunnerDiscovery(BaseModel):
plugin_author: str
plugin_name: str
runner_name: str
manifest: AgentRunnerManifest
manifest: RunnerManifest
```
`manifest` 是 SDK typed `AgentRunnerManifest`,由 Runtime 从插件组件 manifest 解析并校验后返回。`plugin_author` / `plugin_name` / `runner_name` 保留为 transport 寻址字段;Host 以它们生成稳定 runner id,并把 `manifest.id` 校验为 `plugin:author/name/runner`。单个 runner manifest 解析失败时 Runtime/Host 记录 warning 并跳过该 runner,不影响同一插件或其它插件的 runner discovery。
`manifest` 是 SDK typed `RunnerManifest`,由 Runtime 从插件组件 manifest 解析并校验后返回。`plugin_author` / `plugin_name` / `runner_name` 保留为 transport 寻址字段;Host 以它们生成稳定 runner id,并把 `manifest.id` 校验为 `plugin:author/name/runner`。单个 runner manifest 解析失败时 Runtime/Host 记录 warning 并跳过该 runner,不影响同一插件或其它插件的 runner discovery。
### 4.2 AgentRunnerManifest
### 4.2 RunnerManifest
这里的 manifest 指 Runtime 返回给 Host 的 typed runner manifest
```python
class AgentRunnerManifest(BaseModel):
class RunnerManifest(BaseModel):
id: str
name: str
label: I18nObject
description: I18nObject | None = None
capabilities: AgentRunnerCapabilities = AgentRunnerCapabilities()
permissions: AgentRunnerPermissions = AgentRunnerPermissions()
capabilities: RunnerCapabilities = RunnerCapabilities()
permissions: RunnerPermissions = RunnerPermissions()
config_schema: list[DynamicFormItemSchema] = []
metadata: dict[str, Any] = {}
```
@@ -95,7 +95,7 @@ class AgentRunnerManifest(BaseModel):
### 4.3 Capabilities
```python
class AgentRunnerCapabilities(BaseModel):
class RunnerCapabilities(BaseModel):
streaming: bool = False
tool_calling: bool = False
knowledge_retrieval: bool = False
@@ -122,7 +122,7 @@ Capabilities 字段全部是 `bool`,未知 key 禁止进入 typed manifest。
### 4.4 Permissions 与 Effective Access
```python
class AgentRunnerPermissions(BaseModel):
class RunnerPermissions(BaseModel):
models: list[Literal["invoke", "stream", "rerank"]] = []
tools: list[Literal["detail", "call"]] = []
knowledge_bases: list[Literal["list", "retrieve"]] = []
@@ -151,7 +151,7 @@ effective_access = manifest.permissions ∩ binding.resource_policy ∩ current
1. `AgentResourceBuilder` 先用 manifest permissions 与 binding resource policy / runner config 求交,生成 `ctx.resources`
2. `AgentContextBuilder` 用 manifest permissions 与 binding state/storage policy 求交,生成 `ctx.context.available_apis`
3. `AgentRunSessionRegistry` 冻结 run-scoped resources 与 available APIs。
4. Runtime handler / `AgentRunAPIProxy` 按 active `run_id`、runner identity、caller plugin identity、resource id、scope、payload size、rate limit 和 deadline 校验每次调用。
4. Runtime handler / `RunnerAPIProxy` 按 active `run_id`、runner identity、caller plugin identity、resource id、scope、payload size、rate limit 和 deadline 校验每次调用。
反承诺:manifest permissions **只约束 LangBot 持有的资源访问**。它不承诺限制外部 harness 的 native shell、文件系统、CLI、MCP、网络或本机权限;这些能力由 operator/runtime/sandbox 另行约束,见 HOST_SDK §4.8 与 SECURITY_HARDENING。
@@ -167,7 +167,7 @@ context 边界的设计理由见 [AGENT_CONTEXT_PROTOCOL.md](./AGENT_CONTEXT_PRO
## 5. Run 协议
### 5.1 RUN_AGENT
### 5.1 RUN_RUNNER
Host 调用 Runtime
@@ -175,17 +175,17 @@ Host 调用 Runtime
class AgentRunRequest(BaseModel):
runner_id: str
runner_name: str
context: AgentRunContext
context: RunnerContext
```
Runtime 返回 `AgentRunResult` 异步流。底层 transport 可继续用 `plugin_author` / `plugin_name` / `runner_name` 定位组件,但协议语义以 `runner_id``context` 为准。
Runtime 返回 `RunnerResult` 异步流。底层 transport 可继续用 `plugin_author` / `plugin_name` / `runner_name` 定位组件,但协议语义以 `runner_id``context` 为准。
### 5.2 AgentRunContext
### 5.2 RunnerContext
这是 SDK 看到的**唯一权威 context 定义**。
```python
class AgentRunContext(BaseModel):
class RunnerContext(BaseModel):
run_id: str
trigger: AgentTrigger
event: AgentEventContext
@@ -361,7 +361,7 @@ class InteractionDeliveryCapabilities(BaseModel):
max_fields: int | None = None
```
Runner 使用 `AgentRunResult.interaction_requested()` 生成
Runner 使用 `RunnerResult.interaction_requested()` 生成
`action.requested(action="interaction.requested")`。Host 只能把请求投递到当前 run 冻结的
delivery targetRunner 不得通过 `target` 改写 bot、conversation 或用户。Host 为请求保存
`interaction_id -> processor/binding/conversation/expiry` 关联;平台 callback 必须先经过签名、
@@ -455,11 +455,11 @@ class AgentResources(BaseModel):
`skills` 是本次 run 中 pipeline-visible 的 skill facts`skill_name``display_name``description`)。**skill 通过统一 tool 形式消费,不是独立资源类别**:发现走 `list_skills` tool(或 `langbot_list_assets` 增加 skills 一类),激活走 `activate`,操作走 native exec/read/write。Host **不**把 skill 索引注入 system prompt,也不做 progressive-disclosure 注入;LLM 通过调用发现工具主动查询 skill 清单。Host **可选**在 ctx 提供预渲染的 `suggested_skill_prompt`(首轮延迟优化,runner 可忽略 / override),但它不是访问前提。`skills` 字段本身仅作为发现工具的数据来源与该可选预渲染的输入。
资源列表是本次 run 的授权结果。History / Event / State / Storage 访问通过 `ctx.context.available_apis` 和 Host 侧 run session 校验控制,不作为可枚举 resource list 暴露。Runner 只能通过 `AgentRunAPIProxy` 访问这些能力。当前事件的文件和工具大结果优先进入授权 sandbox/workspace,由 runner 通过 read/write/exec 类工具按需读取。
资源列表是本次 run 的授权结果。History / Event / State / Storage 访问通过 `ctx.context.available_apis` 和 Host 侧 run session 校验控制,不作为可枚举 resource list 暴露。Runner 只能通过 `RunnerAPIProxy` 访问这些能力。当前事件的文件和工具大结果优先进入授权 sandbox/workspace,由 runner 通过 read/write/exec 类工具按需读取。
## 7. Result Stream
### 7.1 AgentRunResult envelope
### 7.1 RunnerResult envelope
```python
JSONValue = str | int | float | bool | None | list["JSONValue"] | dict[str, "JSONValue"]
@@ -475,9 +475,9 @@ ResultType = Literal[
"run.failed",
]
class AgentRunResult(BaseModel):
class RunnerResult(BaseModel):
run_id: str
type: AgentRunResultType | str
type: RunnerResultType | str
data: dict[str, Any] = {}
usage: LLMTokenUsage | None = None
sequence: int | None = None
@@ -568,7 +568,7 @@ Host 必须校验 `state.updated` 的 scope、key、value 大小和 JSON 可序
{ "type": "action.requested", "data": { "action": "interaction.requested", "payload": { "interaction_id": "form_1", "kind": "choice", "title": "Approve?", "actions": [{"id": "approve", "label": "Approve", "style": "primary"}], "fallback_text": "Reply approve or reject." } } }
```
## 8. AgentRunAPIProxy
## 8. RunnerAPIProxy
所有 proxy action 必须携带 `run_id`。Host 必须校验:active run session 存在、caller plugin identity 匹配、resource 在本次 `ctx.resources` 中授权、scope 不越界、payload size / rate limit / deadline 合法。
@@ -777,7 +777,7 @@ Protocol v1 的安全边界在 Host
- 大 payload 不应塞进 result event;当前 run 的文件和工具大结果应进入授权 sandbox/workspace,由 read/write/exec 类工具按需访问。
- Host 必须记录 run_id、runner_id、action、resource、scope、result。
Host 不负责业务编排:不拼接全量历史、不替 runner 做 prompt assembly、不内置 agent memory / tool loop / 上下文压缩策略。这些由官方或第三方 AgentRunner 插件实现。
Host 不负责业务编排:不拼接全量历史、不替 runner 做 prompt assembly、不内置 agent memory / tool loop / 上下文压缩策略。这些由官方或第三方 Runner 插件实现。
外部 harness runner 的边界统一见 HOST_SDK §4.8。简言之:harness native permission mode、allowed/disallowed tools、shell/MCP 权限只是额外执行约束,不能替代 Host 对 LangBot 资源的授权。
@@ -786,7 +786,7 @@ Host 不负责业务编排:不拼接全量历史、不替 runner 做 prompt as
## 12. Pipeline AI Stage Adapter 边界
Pipeline 与 Agent 是 EBA 中平级的处理器:Pipeline 处理消息事件并执行完整
Stage 链,Agent 处理其声明支持的消息或非消息事件。本协议只约束 AgentRunner
Stage 链,Agent 处理其声明支持的消息或非消息事件。本协议只约束 Runner
调用,因此 Pipeline 仅在 AI Stage 调用 runner 时进入 Query entry adapter
该适配不会把 Pipeline 变成 Agent,也不会创建或更新持久 Agent。adapter 负责:
@@ -804,12 +804,12 @@ Stage 链,Agent 处理其声明支持的消息或非消息事件。本协议
## 13. 已确认约束
- EBA 路由层是 `one event -> one Processor target (Pipeline | Agent)`;同一 bot / channel 可以让不同事件绑定不同类型的处理器。
- 进入 AgentRunner Protocol 后,调用基数是 `one AgentBinding -> one run_id -> one runner`。这既适用于独立 Agent,也适用于 Pipeline AI Stage 的单次 runner 调用。
- 进入 Runner Protocol 后,调用基数是 `one AgentBinding -> one run_id -> one runner`。这既适用于独立 Agent,也适用于 Pipeline AI Stage 的单次 runner 调用。
- 一个 Agent 可以被多个 bot / channel 复用。如果 Agent 分支出现多个匹配 bindingBindingResolver 必须按明确规则选出一个或拒绝配置,不应默认 fan-out。
- observer agent、多 runner fan-out、并行裁决、result 合并等能力需要单独设计 delivery、state、platform action 和 audit 语义,不属于当前 v1 契约。
- `AgentRunnerDescriptor.source` 只允许 `plugin`Host 内置 adapter 不能作为 runner source 绕过插件/runtime/proxy 权限链。
- `RunnerDescriptor.source` 只允许 `plugin`Host 内置 adapter 不能作为 runner source 绕过插件/runtime/proxy 权限链。
- `ctx.resources` 与 proxy action 校验必须来自同一个 run authorization snapshotruntime handler 不应重新执行资源裁剪。
- v1 不要求 Agent、AgentRunner 插件实例或 runner id 全局串行。多个 bot / channel 可复用同一个 Agent;并发隔离依赖 `run_id`、binding、conversation / thread scope 和 Host authorization snapshot。
- v1 不要求 Agent、Runner 插件实例或 runner id 全局串行。多个 bot / channel 可复用同一个 Agent;并发隔离依赖 `run_id`、binding、conversation / thread scope 和 Host authorization snapshot。
- 外部 harness runner 当前是 MVP / dev path,证明协议可接入,不代表发布级安全边界或 Docker 生产可用性完成。
## 14. 开放问题
+18 -18
View File
@@ -1,23 +1,23 @@
# Agent Runner 插件化文档入口
# Runner 插件化文档入口
本文档是 agent-runner 插件化工作的路由页。具体设计拆到独立文档中维护,避免把 LangBot 宿主架构、SDK 协议、上下文管理、EBA 接入边界和官方 runner 迁移混在同一份 README 里。
## 背景与问题
旧 runner 路径主要围绕 Pipeline / Query 和 `pkg/provider/runners` 内置实现展开,扩展外部 agent runtime 时容易把 runner 选择、上下文裁剪、资源授权和消息投递绑在同一条聊天链路里。这个分支要把 LangBot 收敛成 Agent Host:Host 负责事件、绑定、授权、事实源和结果投递;AgentRunner 作为插件或外部 harness 消费统一协议并自主管理 prompt / history / memory。
旧 runner 路径主要围绕 Pipeline / Query 和 `pkg/provider/runners` 内置实现展开,扩展外部 agent runtime 时容易把 runner 选择、上下文裁剪、资源授权和消息投递绑在同一条聊天链路里。这个分支要把 LangBot 收敛成 Agent Host:Host 负责事件、绑定、授权、事实源和结果投递;Runner 作为插件或外部 harness 消费统一协议并自主管理 prompt / history / memory。
## 文档维护原则(单一事实源)
- **协议数据结构(schema)唯一定义在 [PROTOCOL_V1.md](./PROTOCOL_V1.md)。** 其他文档不得重抄 schema,只能引用,例如"见 PROTOCOL_V1 §4.2"。
- 当前实现状态、spec 差距与 runner 验收状态归 [STATUS.md](./STATUS.md);测试执行入口归 [AGENT_RUNNER_QA_GUIDE.md](./AGENT_RUNNER_QA_GUIDE.md),安全发布门槛归 [SECURITY_HARDENING.md](./SECURITY_HARDENING.md)。
- 当前实现状态、spec 差距与 runner 验收状态归 [STATUS.md](./STATUS.md);测试执行入口归 [RUNNER_QA_GUIDE.md](./RUNNER_QA_GUIDE.md),安全发布门槛归 [SECURITY_HARDENING.md](./SECURITY_HARDENING.md)。
- Host 内部模型(`AgentEventEnvelope``AgentBinding`、Descriptor、各 Store)定义在 [HOST_SDK_INFRASTRUCTURE.md](./HOST_SDK_INFRASTRUCTURE.md),不属于 SDK 协议。
- 其余专题文档只讲"为什么/边界/怎么用",避免重复叙述。
## 本分支目标
**本分支目标:AgentRunner 外化 / 插件化基础设施**
**本分支目标:Runner 外化 / 插件化基础设施**
本分支只做 LangBot 作为 Agent Host 的基础能力建设,让现有 Pipeline 与用户新建的独立 `Agent` 都能调用插件化 AgentRunner;不负责把两者做持久化迁移:
本分支只做 LangBot 作为 Agent Host 的基础能力建设,让现有 Pipeline 与用户新建的独立 `Agent` 都能调用插件化 Runner;不负责把两者做持久化迁移:
- LangBot 与 SDK 的稳定协议合同(Protocol v1
- Host-side `AgentEventEnvelope` / `AgentBinding` 模型
@@ -30,7 +30,7 @@
## 当前已集成与后续扩展
截至 2026-09-05`dev/4.11.x` 已合并 EBA 与 AgentRunner 插件化。下面按当前代码划分实现边界:
截至 2026-09-05`dev/4.11.x` 已合并 EBA 与 Runner 插件化。下面按当前代码划分实现边界:
- **已实现的平台事件路由**`RuntimeBot` 负责事件转换后的 observer 广播、`event_bindings` 匹配和 Pipeline / Agent / discard 单目标分派;EventRouter 是逻辑职责,不是另一个独立服务。
- **已实现的持久化与 UI**:独立 Agent、Bot 事件绑定、处理器工作台、Runner 市场安装、事件范围与工具权限、路由诊断和调试入口。
@@ -48,22 +48,22 @@
调度基数、Agent 复用、插件实例无状态、Pipeline adapter 和 fan-out 边界的规范来源是 [PROTOCOL_V1.md](./PROTOCOL_V1.md) §13;README 不复写这些约束。
## Pipeline 与 AgentRunner 的关系
## Pipeline 与 Runner 的关系
**Pipeline 与 Agent 是 EBA 中平级的处理器;`QueryEntryAdapter` 只适配 Pipeline 内部的 AgentRunner 调用。**
**Pipeline 与 Agent 是 EBA 中平级的处理器;`QueryEntryAdapter` 只适配 Pipeline 内部的 Runner 调用。**
EBA 先根据 `target_type` 选择 Pipeline 或 Agent。Pipeline 目标执行完整 Stage 链;当 Pipeline 的 AI Stage 调用 runner 时,`run_from_query()``QueryEntryAdapter``Query` 转换为 `AgentEventEnvelope` + `AgentBinding`,再委托到统一的 `run(event, binding, ...)`。Agent 目标则直接从自己的持久配置构造 binding。两条路径可以复用同一套 AgentRunner Host capabilities,但 Pipeline 本身不会被投影或持久化为 Agent。
EBA 先根据 `target_type` 选择 Pipeline 或 Agent。Pipeline 目标执行完整 Stage 链;当 Pipeline 的 AI Stage 调用 runner 时,`run_from_query()``QueryEntryAdapter``Query` 转换为 `AgentEventEnvelope` + `AgentBinding`,再委托到统一的 `run(event, binding, ...)`。Agent 目标则直接从自己的持久配置构造 binding。两条路径可以复用同一套 Runner Host capabilities,但 Pipeline 本身不会被投影或持久化为 Agent。
下一轮测试路径、状态定义和 smoke 记录见 [AGENT_RUNNER_QA_GUIDE.md](./AGENT_RUNNER_QA_GUIDE.md)。
下一轮测试路径、状态定义和 smoke 记录见 [RUNNER_QA_GUIDE.md](./RUNNER_QA_GUIDE.md)。
## 术语表
| 术语 | 含义 |
| --- | --- |
| Protocol v1 | Host 调用 AgentRunner 的 runner 可见合同:discovery、`AgentRunContext`、result stream、Host pull API 和错误模型。 |
| Protocol v1 | Host 调用 Runner 的 runner 可见合同:discovery、`RunnerContext`、result stream、Host pull API 和错误模型。 |
| Processor | EBA 的处理器上位概念;当前平级类型为 Pipeline 与 Agent。 |
| Agent | 目标产品层配置对象,保存 runner id、runner config 和资源/状态/投递策略;不等于插件实例。 |
| AgentConfig | Host 内部的单次 AgentRunner 调用配置投影,可由 Pipeline AI Stage 或持久 Agent 生成;投影本身不会创建 Agent。 |
| AgentConfig | Host 内部的单次 Runner 调用配置投影,可由 Pipeline AI Stage 或持久 Agent 生成;投影本身不会创建 Agent。 |
| AgentBinding / binding | Host 在一次事件运行前解析出的有效绑定,决定调用哪个 runner 以及带什么策略。 |
| envelope | Host 内部事件封装,即 `AgentEventEnvelope`runner 看到的是由它投影出的 `ctx.event`。 |
| descriptor / manifest | runner discovery 的能力和配置描述;manifest 来自插件,descriptor 是 Host 校验后的注册表视图。 |
@@ -76,18 +76,18 @@ EBA 先根据 `target_type` 选择 Pipeline 或 Agent。Pipeline 目标执行完
| 文档 | 关注点 |
| --- | --- |
| [PROTOCOL_V1.md](./PROTOCOL_V1.md) | **🔒 唯一 schema 事实源**。LangBot Host 与 SDK / Runtime / AgentRunner 的协议合同:版本协商、discovery、run context、result stream、proxy actions、错误和 adapter 边界。 |
| [PROTOCOL_V1.md](./PROTOCOL_V1.md) | **🔒 唯一 schema 事实源**。LangBot Host 与 SDK / Runtime / Runner 的协议合同:版本协商、discovery、run context、result stream、proxy actions、错误和 adapter 边界。 |
| [HOST_SDK_INFRASTRUCTURE.md](./HOST_SDK_INFRASTRUCTURE.md) | LangBot 宿主能力与分层架构、Host 内部模型(`AgentEventEnvelope` / `AgentBinding` / Descriptor / 各 Store)、runner 发现、绑定、资源授权、状态、存储、生命周期和调用链。 |
| [AGENT_CONTEXT_PROTOCOL.md](./AGENT_CONTEXT_PROTOCOL.md) | Agent-owned context 方向:事件到来时 LangBot 传什么,agent 如何按需拉取更多历史 / state、如何访问 sandbox/workspace 文件,以及如何支持 KV cache 友好的上下文管理。 |
| [EXTENSION_SCOPE_MATRIX.md](./EXTENSION_SCOPE_MATRIX.md) | AgentRunner 外化与外部 EBA / Agent Platform / Runtime Control Plane 的扩展边界矩阵,说明哪些是本分支底座、哪些由外部分支接入。 |
| [EXTENSION_SCOPE_MATRIX.md](./EXTENSION_SCOPE_MATRIX.md) | Runner 外化与外部 EBA / Agent Platform / Runtime Control Plane 的扩展边界矩阵,说明哪些是本分支底座、哪些由外部分支接入。 |
| [EVENT_BASED_AGENT.md](./EVENT_BASED_AGENT.md) | 已集成的事件路由、处理器分派、平台工具和结构化交互边界。 |
| [eba-productization-release.md](./eba-productization-release.md) | EBA 适配器与 AgentRunner 插件化合并后的产品化 / 发布计划,说明非技术用户快速上手差距、Bot 与处理器边界、未来 Solution 分发标的,以及多 namespace SaaS 支持要求。 |
| [eba-productization-release.md](./eba-productization-release.md) | EBA 适配器与 Runner 插件化合并后的产品化 / 发布计划,说明非技术用户快速上手差距、Bot 与处理器边界、未来 Solution 分发标的,以及多 namespace SaaS 支持要求。 |
| [RUNTIME_CONTROL_PLANE_V2.md](./RUNTIME_CONTROL_PLANE_V2.md) | Agent Platform v2 / runtime 管控面决策:`AgentRun` / `AgentRunEvent` / run control 已作为 Host 事实源落地,最小 runtime heartbeat/claim lease 已落地;完整 runtime registry / daemon 管控仍是后续可选阶段。 |
| [OFFICIAL_RUNNER_PLUGINS.md](./OFFICIAL_RUNNER_PLUGINS.md) | 官方 runner 插件迁移,包括 local-agent 和外部 runner。它是下游落地计划,不是 LangBot 基础能力设计的前置约束。 |
| [RUN_STEERING_AND_CHECKPOINT.md](./RUN_STEERING_AND_CHECKPOINT.md) | 运行中消息注入(steering / follow-up)与压缩摘要持久化(compaction checkpoint)的设计与落地状态记录;schema 仍以 PROTOCOL_V1 为准。 |
| [STATUS.md](./STATUS.md) | 当前实现状态、spec 与实现已知差距、runner 验收状态和历史高价值记录。 |
| [PLATFORM_ACTION_TOOLS.md](./PLATFORM_ACTION_TOOLS.md) | 当前事件工具、平台动作和普通工具的配置、投射及执行授权。 |
| [AGENT_RUNNER_QA_GUIDE.md](./AGENT_RUNNER_QA_GUIDE.md) | Agent Runner QA 指南:保留最高价值测试路径,指导 agent 开展下一轮 WebUI / runner smoke 验证。 |
| [RUNNER_QA_GUIDE.md](./RUNNER_QA_GUIDE.md) | Runner QA 指南:保留最高价值测试路径,指导 agent 开展下一轮 WebUI / runner smoke 验证。 |
| [SECURITY_HARDENING.md](./SECURITY_HARDENING.md) | 安全发布级 hardening 的后续发布门槛:路径隔离、权限边界、secret、资源配额、MCP / skill 投影和审计。 |
## 工作拆分
@@ -103,7 +103,7 @@ EBA 先根据 `target_type` 选择 Pipeline 或 Agent。Pipeline 目标执行完
- resource authorization 与 `run_id` 级权限校验
- host-owned state / storage / event log / transcript 能力
- sandbox/workspace 文件 staging 与 read/write/exec 能力
- SDK `AgentRunner``AgentRunContext``AgentRunResult``AgentRunAPIProxy`
- SDK `Runner``RunnerContext``RunnerResult``RunnerAPIProxy`
协议合同详见 [PROTOCOL_V1.md](./PROTOCOL_V1.md)。
@@ -141,7 +141,7 @@ EBA dispatch 的基数和 fan-out 边界仍以 PROTOCOL_V1 §13 为准;新增
### 5. Runtime Control Plane v2Foundation Partial
当前 AgentRunner v1 主线仍以 `event -> binding -> runner.run(ctx) -> result stream` 为 runner 可见合同。Host 侧已经新增持久 `AgentRun` / `AgentRunEvent`、result persistence、cancel/finalize/query 等通用 run control primitives,并提供受权限保护的最小 runtime register/heartbeat/list、claim/renew/release 和 reconcile 原语。
当前 Runner v1 主线仍以 `event -> binding -> runner.run(ctx) -> result stream` 为 runner 可见合同。Host 侧已经新增持久 `AgentRun` / `AgentRunEvent`、result persistence、cancel/finalize/query 等通用 run control primitives,并提供受权限保护的最小 runtime register/heartbeat/list、claim/renew/release 和 reconcile 原语。
在这些 Host 能力之上,可以构建独立 agent 管控面插件;插件负责 UI、策略和编排体验,runtime/task 的事实源仍由 Host 持有。完整 daemon supervisor、任务唤醒/长轮询/WebSocket、跨 Host 分布式锁、provider 登录态诊断和产品化业务队列仍是后续工作。
@@ -1,10 +1,10 @@
# Agent Platform / Runtime Control Plane Decision Note
本文档记录 AgentRunner 插件化之后,LangBot 如何继续演进成 Agent Platform 基础设施层。这里讨论的是 Host capability layer,不是 `AgentRunner Protocol v2`,也不是把某个具体 Agent Platform 产品写进 LangBot core。
本文档记录 Runner 插件化之后,LangBot 如何继续演进成 Agent Platform 基础设施层。这里讨论的是 Host capability layer,不是 `Runner Protocol v2`,也不是把某个具体 Agent Platform 产品写进 LangBot core。
> 本文是当前决策版。协议数据结构仍以 [PROTOCOL_V1.md](./PROTOCOL_V1.md) 为准;测试执行入口见 [AGENT_RUNNER_QA_GUIDE.md](./AGENT_RUNNER_QA_GUIDE.md);扩展边界见 [EXTENSION_SCOPE_MATRIX.md](./EXTENSION_SCOPE_MATRIX.md)。
> 本文是当前决策版。协议数据结构仍以 [PROTOCOL_V1.md](./PROTOCOL_V1.md) 为准;测试执行入口见 [RUNNER_QA_GUIDE.md](./RUNNER_QA_GUIDE.md);扩展边界见 [EXTENSION_SCOPE_MATRIX.md](./EXTENSION_SCOPE_MATRIX.md)。
>
> 实现状态说明:本文描述的是 Runtime Control Plane v2 的目标能力和分阶段落地建议。当前 AgentRunner 插件化主线已经具备 event-first context、run-scoped authorization、EventLog / Transcript / State / sandbox 文件等 Host capability,并已落地持久 `AgentRun` / `AgentRunEvent` ledger、run control actions、最小 runtime heartbeat/claim lease 和 admin reconcile 原语。完整 Agent Platform 产品形态、daemon supervisor、runtime wakeup channel 和分布式 runtime 管控仍未完成。当前实现状态以 [STATUS.md](./STATUS.md) 为准。
> 实现状态说明:本文描述的是 Runtime Control Plane v2 的目标能力和分阶段落地建议。当前 Runner 插件化主线已经具备 event-first context、run-scoped authorization、EventLog / Transcript / State / sandbox 文件等 Host capability,并已落地持久 `AgentRun` / `AgentRunEvent` ledger、run control actions、最小 runtime heartbeat/claim lease 和 admin reconcile 原语。完整 Agent Platform 产品形态、daemon supervisor、runtime wakeup channel 和分布式 runtime 管控仍未完成。当前实现状态以 [STATUS.md](./STATUS.md) 为准。
## 1. 当前决策
@@ -14,7 +14,7 @@ LangBot 后续定位应更像 **Agent Host / infrastructure provider / transfer
- **Agent Platform 产品形态做成插件**。插件负责 agent 管理、策略、业务队列、UI、编排、多 agent 协作和产品体验。
- **Agent Platform 所需的基础事实源做进 Host**。当前 Host 已保存 event、state、transcript、sandbox 文件边界、active run 权限快照、持久 run/result ledger、审计关联和通用控制状态。
- **最小 runtime registry / heartbeat / claim lease 已作为 Host 原语落地,但不等于完整 daemon worker 管控**。远程 harness / daemon 的进程托管、wakeup channel、provider 登录态诊断和分布式调度仍可以先由 AgentRunner 插件和 SDK remote layer 自己维护。
- **最小 runtime registry / heartbeat / claim lease 已作为 Host 原语落地,但不等于完整 daemon worker 管控**。远程 harness / daemon 的进程托管、wakeup channel、provider 登录态诊断和分布式调度仍可以先由 Runner 插件和 SDK remote layer 自己维护。
- **不把业务调度写进 Host**。Host 提供通用 run/result/control primitivesPlatform 插件决定哪些事件触发哪些 agent、如何排队、如何分配、是否 fan-out。
推荐分层:
@@ -30,16 +30,16 @@ Agent Platform plugin
Agent management UI / project-task model / event routing policy
Business queue / multi-agent orchestration / runtime selection policy
AgentRunner plugin / external harness runtime
Runner plugin / external harness runtime
Connects ACP / remote daemon / local subprocess / HTTP API
Executes and converts provider-native events to AgentRunResult
Executes and converts provider-native events to RunnerResult
```
## 2. Platform 与非 Platform 的区别
当前 LangBot 已经具备 Agent Host 的核心特征:
- 抹平不同 AgentRunner。
- 抹平不同 Runner。
- 从 IM / Pipeline 入口触发 runner。
- 有 event-first context 方向。
- 有 Host-owned EventLog / Transcript / State 和 sandbox/workspace 文件边界。
@@ -116,7 +116,7 @@ metadata
### 3.3 RunEvent / RunResult
RunEvent 是一次 run 过程中产生的结果事件流,对应 runner 返回的 `AgentRunResult`。它不同于 EBA/EventLog 的输入事件:
RunEvent 是一次 run 过程中产生的结果事件流,对应 runner 返回的 `RunnerResult`。它不同于 EBA/EventLog 的输入事件:
```text
message.delta
@@ -151,8 +151,8 @@ Runtime / daemon 表示执行位置或执行能力,例如某台机器上的 Cl
当前决策:
- Host 不在第一阶段维护完整 runtime registry。
- AgentRunner 插件可以通过 SDK remote layer 与 daemon 保持连接、心跳和执行通道。
- 外部 harness / agent 不应直接访问 LangBot Host 或数据库。访问 LangBot 资源必须通过 daemon / AgentRunner plugin / SDK runtime / `AgentRunAPIProxy` / scoped MCP bridge,并接受 run-scoped authorization 校验。
- Runner 插件可以通过 SDK remote layer 与 daemon 保持连接、心跳和执行通道。
- 外部 harness / agent 不应直接访问 LangBot Host 或数据库。访问 LangBot 资源必须通过 daemon / Runner plugin / SDK runtime / `RunnerAPIProxy` / scoped MCP bridge,并接受 run-scoped authorization 校验。
- 如果后续多个插件都需要共享 runtime 状态,再把薄的 `RuntimeLease` / registry 下沉为 Host 通用能力。
## 4. Host 应新增的最小能力
@@ -222,7 +222,7 @@ metadata_json
- append 必须幂等,支持远程 daemon / plugin 重试。
- 未知 result type 可保存但 Host 只对已知类型执行副作用。
- 大 payload 仍应进入 sandbox/workspace,不直接塞入 result event。
- `usage_json` 保存 `AgentRunResult.usage` 原样结构;缺失表示 unknown,不等于 0。
- `usage_json` 保存 `RunnerResult.usage` 原样结构;缺失表示 unknown,不等于 0。
### 4.3 Run Control API
@@ -258,7 +258,7 @@ event -> binding -> context -> runner invocation -> result normalization
需要补齐:
- run 开始时创建 `AgentRun`
- 每个 `AgentRunResult` 进入 `AgentRunEvent`
- 每个 `RunnerResult` 进入 `AgentRunEvent`
- `run.completed` / 正常 generator 结束时标记 completed。
- `run.failed` / exception / timeout 标记 failed 或 timeout。
- terminal result 携带 usage 时,写入 `AgentRunEvent.usage_json` 并汇总到 `AgentRun.usage_json`
@@ -266,7 +266,7 @@ event -> binding -> context -> runner invocation -> result normalization
### 4.5 Usage / Cost Accounting
SDK 侧 `AgentRunResult` 已提供可选 `usage` 字段,用于把不同 runner / external harness / provider-native event 的 token usage 归一到同一个 run result envelope。
SDK 侧 `RunnerResult` 已提供可选 `usage` 字段,用于把不同 runner / external harness / provider-native event 的 token usage 归一到同一个 run result envelope。
语义:
@@ -309,7 +309,7 @@ RunCreateRequest / RunCreateResult
RunAppendResultRequest
```
这些是 Host control primitives,不替代 `AgentRunContext` / `AgentRunResult`
这些是 Host control primitives,不替代 `RunnerContext` / `RunnerResult`
### 5.2 Proxy Methods
@@ -327,17 +327,17 @@ finalize_run(run_id, status, error=None)
访问边界:
- 普通 AgentRunner 在同步 `run(ctx)` 内不一定需要直接调用这些 APIHost orchestrator 可自动记录。
- 普通 Runner 在同步 `run(ctx)` 内不一定需要直接调用这些 APIHost orchestrator 可自动记录。
- Platform 插件可以创建/查询/取消 run。
- AgentRunner 插件或 daemon bridge 可以 append/finalize 自己负责的 run。
- Runner 插件或 daemon bridge 可以 append/finalize 自己负责的 run。
- 外部 harness 仍不能直接调用 Host;必须经 SDK runtime / proxy / bridge。
### 5.3 Plugin-Daemon Heartbeat
远程 daemon 的初始心跳可以是 SDK / AgentRunner plugin 私有能力:
远程 daemon 的初始心跳可以是 SDK / Runner plugin 私有能力:
```text
daemon <-> AgentRunner plugin / SDK remote layer <-> LangBot plugin runtime <-> Host
daemon <-> Runner plugin / SDK remote layer <-> LangBot plugin runtime <-> Host
```
Host 第一阶段只需要知道:
@@ -365,7 +365,7 @@ Platform 插件不应负责:
- 在 Host Run Ledger 落地后,私有保存通用 run/result 事实源。
- 绕过 Host 直接写 transcript/state 或越权访问 sandbox/workspace 文件。
- 让外部 harness 直接访问 LangBot DB 或 Host 内部资源。
- 把某个业务队列语义强塞进 AgentRunner Protocol v1。
- 把某个业务队列语义强塞进 Runner Protocol v1。
## 7. 与 EBA 的关系
@@ -395,18 +395,18 @@ EventGateway
这两条路径最终应共享 Host run/result/state 事实源和 sandbox/workspace 文件边界。当前阶段可共享的是 event/transcript/state、sandbox 文件和同步执行链路;持久 run/result ledger 需要 Runtime Control Plane v2 Phase 1 补齐。区别在于是否有 Platform 插件参与产品化调度和业务队列。
## 8. 与 AgentRunner Protocol v1 的关系
## 8. 与 Runner Protocol v1 的关系
本设计不改变 v1 的 runner 可见合同:
```text
AgentRunContext -> AgentRunner.run(ctx) -> AgentRunResult stream
RunnerContext -> Runner.run(ctx) -> RunnerResult stream
```
必须保持:
- `AgentRunContext` 不塞入 daemon/worker/pod 细节。
- `AgentRunResult` 仍是 runner 输出的统一事件流。
- `RunnerContext` 不塞入 daemon/worker/pod 细节。
- `RunnerResult` 仍是 runner 输出的统一事件流。
- 普通 runner 不需要知道 task queue / runtime registry。
- 远程 harness 可以自管 session、tool loop、MCP、上下文压缩,但访问 LangBot 资源必须通过 SDK proxy / bridge。
- Runtime-managed execution 是 placement / transport 选择,不是普通 runner 协议的强制概念。
@@ -422,7 +422,7 @@ AgentRunContext -> AgentRunner.run(ctx) -> AgentRunResult stream
- `AgentRun` 表。
- `AgentRunEvent` 表。
- Orchestrator 自动创建/更新 run。
- Journal 持久化每个 `AgentRunResult`
- Journal 持久化每个 `RunnerResult`
- Run 查询和事件分页 API。
- SDK entities + proxy 方法。
@@ -517,9 +517,9 @@ Tests: 40+ 个文件
- Host 不写业务调度策略,但要保存通用状态、结果、权限和审计。
- EBA event 不是 queuequeue 是执行生命周期问题。
- 业务 queue 可以先在 Platform 插件里;执行 queue 只有在复用需求明确后再下沉 Host。
- Daemon registry 不应污染 AgentRunner Protocol v1。
- Daemon registry 不应污染 Runner Protocol v1。
- 外部 harness 不直接访问 LangBot Host 或 DB。
- 所有 LangBot 资源访问必须走 SDK runtime / `AgentRunAPIProxy` / scoped MCP bridge。
- 所有 LangBot 资源访问必须走 SDK runtime / `RunnerAPIProxy` / scoped MCP bridge。
- Docker / remote / local subprocess 只是 runtime placement,不是 runner 协议差异。
## 11. 非目标
@@ -528,8 +528,8 @@ Tests: 40+ 个文件
- 完整 Multica 式 runtime registry。
- Host 内置项目管理、任务板、agent team、workflow 产品逻辑。
- 把 daemon heartbeat / worker liveness 放进 `AgentRunContext`
- 把业务 queue 定义为 AgentRunner Protocol 字段。
- 把 daemon heartbeat / worker liveness 放进 `RunnerContext`
- 把业务 queue 定义为 Runner Protocol 字段。
- 让 Platform 插件私有保存 run/result 事实源。
- 让外部 agent/harness 直连 Host 内部资源。
@@ -16,7 +16,7 @@ local-agent 已移植 Pi 的事件生命周期、并行工具语义、hook 扩
### 1.1 问题
IM 场景下用户在 agent 运行中追加消息非常常见(补充信息、纠正方向、"算了别查了")。
EBA 先按事件选择一个 Pipeline 或 Agent 处理器;进入 AgentRunner 后,当前调用链是 `one AgentBinding -> one run_id -> one runner`
EBA 先按事件选择一个 Pipeline 或 Agent 处理器;进入 Runner 后,当前调用链是 `one AgentBinding -> one run_id -> one runner`
PROTOCOL_V1 §13):同会话的新消息要么等待当前 run 结束后触发新 run,
要么并发触发独立 run。两种行为都无法把新消息送进**正在执行的 tool loop**
用户体验是"agent 自顾自跑完过期任务,然后才看到新消息"。
@@ -59,7 +59,7 @@ pi-agent-core 区分两个队列,注入时机都在 turn 边界,不打断进
已落地的协议面(最终定义归 PROTOCOL_V1):
1. `ContextAccess.available_apis` 增加 steering pull 能力位。
2. `AgentRunAPIProxy` 增加 steering 拉取 action:默认 `mode=all`Host 保序返回全部
2. `RunnerAPIProxy` 增加 steering 拉取 action:默认 `mode=all`Host 保序返回全部
pending 输入;`one-at-a-time` 仅作为 runner 主动节流选项。
3. dispatch 层的"认领"规则:`message.received` 可被同 conversation 的 active run
吸收,原事件写 EventLog / Transcriptdispatch 行为写入 EventLog metadata。
@@ -1,4 +1,4 @@
# Agent Runner Security Boundary
# Runner Security Boundary
本文档记录 agent-runner 插件化后的安全边界和最小护栏。
@@ -6,7 +6,7 @@
**当前结论:不采用高强度监管模型。**
LangBot 的目标不是托管一个强隔离、不可信 code runner 平台。AgentRunner 插件,尤其是 ACP / Claude Code / Codex / OpenCode / Kimi Code 这类外部 harness,默认视为 **operator-owned execution**:用户或部署者显式配置并承担其文件系统、进程、网络、workspace、provider 登录态和 native tool 风险。
LangBot 的目标不是托管一个强隔离、不可信 code runner 平台。Runner 插件,尤其是 ACP / Claude Code / Codex / OpenCode / Kimi Code 这类外部 harness,默认视为 **operator-owned execution**:用户或部署者显式配置并承担其文件系统、进程、网络、workspace、provider 登录态和 native tool 风险。
LangBot 需要负责的是保护 **LangBot 自己持有的资源**,包括模型、知识库、LangBot tools、history、event、state、plugin/workspace storage、sandbox/workspace 文件访问等。只要这些资源访问是 run-scoped、permission-scoped、可校验、可诊断的,当前阶段即可接受。
@@ -171,7 +171,7 @@ LangBot 需要提供基本可控性:
截至 2026-06-15,已有实现覆盖:
- SDK typed AgentRunner manifest、capabilities、permissions。
- SDK typed Runner manifest、capabilities、permissions。
- Host resource builder 按 manifest permissions 和 binding policy 生成 `ctx.resources`
- Active run session snapshot 和 `caller_plugin_identity` 校验。
- History / event / state / tool / knowledge runtime action 的 run-scoped 校验。
@@ -204,7 +204,7 @@ LangBot 需要提供基本可控性:
可以对外说明:
> AgentRunner 插件通过 run-scoped authorization 和 scoped MCP gateway 保护 LangBot 持有资源。外部 code harness 的执行环境由用户或部署平台负责隔离;LangBot 当前不提供 managed sandbox。
> Runner 插件通过 run-scoped authorization 和 scoped MCP gateway 保护 LangBot 持有资源。外部 code harness 的执行环境由用户或部署平台负责隔离;LangBot 当前不提供 managed sandbox。
不能对外说明:
+9 -9
View File
@@ -1,19 +1,19 @@
# AgentRunner Pluginization Status
# Runner Pluginization Status
本文档是 `docs/agent-runner-pluginization/` 的状态事实源。协议 schema 仍以 [PROTOCOL_V1.md](./PROTOCOL_V1.md) 为准;测试步骤以 [AGENT_RUNNER_QA_GUIDE.md](./AGENT_RUNNER_QA_GUIDE.md) 为准;安全发布门槛以 [SECURITY_HARDENING.md](./SECURITY_HARDENING.md) 为准。
本文档是 `docs/agent-runner-pluginization/` 的状态事实源。协议 schema 仍以 [PROTOCOL_V1.md](./PROTOCOL_V1.md) 为准;测试步骤以 [RUNNER_QA_GUIDE.md](./RUNNER_QA_GUIDE.md) 为准;安全发布门槛以 [SECURITY_HARDENING.md](./SECURITY_HARDENING.md) 为准。
状态快照日期:2026-09-05。代码基线为 LangBot `dev/4.11.x` / `a4d36aa2d` 和 SDK `dev/4.11.x` / `f1da058`;本文是本地检视快照,不代表远端最新状态或正式发布批准。
## 当前版本与验收边界
- Core 声明版本 `4.11.0``pyproject.toml` / `uv.lock` 仍依赖 `langbot-plugin==0.5.3`SDK 源码声明版本 `0.5.5`。本机 Core 实际导入旁边 SDK 的可编辑源码,不能用这个组合的测试通过证明 registry 安装可复现。
- 检视时 SDK 有四个未提交文件:`api/agent_tools/asset_gateway.py``api/agent_tools/external_tools.py``api/entities/builtin/agent_runner/resources.py`(均位于 `src/langbot_plugin/`),以及 `tests/api/test_agent_tools_mcp_bridge.py`。这些改动增加 `platform_tools` 分类发现、可选 schema 和 gateway 引导。下面的 SDK 测试包含这些工作区改动;正式配套版本尚需冻结。
- 检视时 SDK 有四个未提交文件:`api/agent_tools/asset_gateway.py``api/agent_tools/external_tools.py``api/entities/builtin/runner/resources.py`(均位于 `src/langbot_plugin/`),以及 `tests/api/test_agent_tools_mcp_bridge.py`。这些改动增加 `platform_tools` 分类发现、可选 schema 和 gateway 引导。下面的 SDK 测试包含这些工作区改动;正式配套版本尚需冻结。
- 本轮仅重新执行下列定向测试与 TypeScript 检查,没有重跑全量 backend、真实平台、provider、浏览器 E2E 或 Cloud 部署门禁。后文旧日期的成功记录仍是历史证据。
| 2026-09-05 验证 | 结果 |
| --- | --- |
| Core`tests/unit_tests/agent``tests/unit_tests/api/service/test_agent_service.py``tests/unit_tests/platform/test_routing_rules.py``tests/unit_tests/api/service/test_maintenance_service.py` | 561 passed74 warnings |
| SDK`tests/api/entities/builtin/agent_runner``tests/api/proxies``tests/api/test_agent_tools_mcp_bridge.py``tests/runtime/plugin/test_mgr_agent_runner.py``tests/runtime/plugin/test_dependency_environment.py``tests/runtime/plugin/test_restart_coordinator.py` | 362 passed10 warnings |
| SDK`tests/api/entities/builtin/runner``tests/api/proxies``tests/api/test_agent_tools_mcp_bridge.py``tests/runtime/plugin/test_mgr_runner.py``tests/runtime/plugin/test_dependency_environment.py``tests/runtime/plugin/test_restart_coordinator.py` | 362 passed10 warnings |
| Web`pnpm exec tsc --noEmit` | pass |
| Web`pnpm test:unit` | 62 passed2 failed |
@@ -31,7 +31,7 @@
| 领域 | 状态 | 说明 |
| --- | --- | --- |
| SDK manifest schema | Done | `AgentRunnerManifest` 包含 typed `capabilities` / `permissions`;未知 capability / permission key 禁止进入 typed model。 |
| SDK manifest schema | Done | `RunnerManifest` 包含 typed `capabilities` / `permissions`;未知 capability / permission key 禁止进入 typed model。 |
| Runner discovery | Done | Runtime 返回 typed manifestHost registry 校验单个 runner,失败 warning + skip,不影响其它 runner。 |
| Host resource authorization | Done | `ctx.resources``ctx.context.available_apis` 由 manifest permissions 与 binding policy / run scope 求交后生成。 |
| Run authorization snapshot | Done | active run session 冻结 run-scoped resources 与 available APIsruntime handler 按 snapshot 校验 pull API。 |
@@ -61,8 +61,8 @@
| Runner | 状态 | 最近证据 |
| --- | --- | --- |
| `plugin:langbot-team/LocalAgent/default` | Unit-pass; Marketplace UI pass; Debug Chat E2E pass | 2026-07-12 隔离 first-run 实例从真实 AgentRunner catalog 安装 `langbot-team/LocalAgent` 0.1.0Host 注册 `plugin:langbot-team/LocalAgent/default`,Wizard 自动选中并解锁后续操作。2026-07-15 `2026-07-15-08-44-10-770-08-00-sandbox-skill-authoring-edit-existing-e2e` 使用真实 `gpt-5.5` 完成 Skill 创建、注册、同 Query 激活、已激活包编辑与脚本执行;三阶段 UI、浏览器诊断和结构化文件系统检查全部通过,每阶段恰好新增一个 Bot 气泡,p95 14.6 秒、错误率 0。 |
| `plugin:langbot-team/ACPAgentRunner/default` | Unit-pass; Debug Chat E2E pass | 2026-07-15 从本地 0.1.4 发布包安装并注册 PascalCase runnerremote-ssh Claude ACP 通过反向隧道调用 run-scoped `langbot_get_current_event`,97.8 秒返回可见结果;Host 将增量 delta 和 `message.completed` 聚合为一个完整 Bot 气泡。 |
| `plugin:langbot-team/LocalAgent/default` | Unit-pass; Marketplace UI pass; Debug Chat E2E pass | 2026-07-12 隔离 first-run 实例从真实 Runner catalog 安装 `langbot-team/LocalAgent` 0.1.0Host 注册 `plugin:langbot-team/LocalAgent/default`,Wizard 自动选中并解锁后续操作。2026-07-15 `2026-07-15-08-44-10-770-08-00-sandbox-skill-authoring-edit-existing-e2e` 使用真实 `gpt-5.5` 完成 Skill 创建、注册、同 Query 激活、已激活包编辑与脚本执行;三阶段 UI、浏览器诊断和结构化文件系统检查全部通过,每阶段恰好新增一个 Bot 气泡,p95 14.6 秒、错误率 0。 |
| `plugin:langbot-team/ACPRunner/default` | Unit-pass; Debug Chat E2E pass | 2026-07-15 从本地 0.1.4 发布包安装并注册 PascalCase runnerremote-ssh Claude ACP 通过反向隧道调用 run-scoped `langbot_get_current_event`,97.8 秒返回可见结果;Host 将增量 delta 和 `message.completed` 聚合为一个完整 Bot 气泡。 |
| `plugin:langbot-team/ClaudeCodeAgent/default` / `plugin:langbot-team/CodexAgent/default` | Unit-pass; E2E pending | 通过 runner 仓库单测覆盖 session、run_id 注入和 LangBot MCP gateway;真实 harness E2E 取决于对应运行环境、CLI/daemon 可用性和 provider 登录态。 |
| Dify | Human-input unit-pass; credential E2E pending | `langbot-agent-runner/dify-agent` 已实现 `workflow_paused`、原子字段/确认交互、plugin-storage continuation、Dify submit/events 恢复与再次暂停;真实 Dify 凭据 E2E 待执行。 |
| n8n / Coze / DashScope / Langflow / Tbox / DeerFlow / WeKnora | Unit-pass; credential smoke optional | 2026-06-13 plugin layout / parser tests 通过;真实服务凭据 smoke 非每轮必跑。 |
@@ -71,9 +71,9 @@
| 范围 | 状态 | 最近证据 |
| --- | --- | --- |
| LangBot Runtime Control Plane v2 foundation | Unit-pass; EBA release gate 5/5 pass; AgentRunner preflight pass | 2026-07-12 `eba-functional-20260712-release-gate-rerun` 通过 Quick Start 场景筛选、隔离实例 Runner Marketplace 安装、Runner 健康状态、事件路由 dry-run / 合成派发,以及真实 OneBot `group.member_joined` → Agent → `send_group_msg` 链路。2026-07-15 AgentRunner release preflight 16 项通过、0 warningfixture contract、5 类 behavior matrix、ledger schema / async DB readiness / 100-run stress / 120-run 8-worker contention / claim-lease-auth concurrency、SDK runtime chaos 探针全部通过。 |
| LangBot Runtime Control Plane v2 foundation | Unit-pass; EBA release gate 5/5 pass; Runner preflight pass | 2026-07-12 `eba-functional-20260712-release-gate-rerun` 通过 Quick Start 场景筛选、隔离实例 Runner Marketplace 安装、Runner 健康状态、事件路由 dry-run / 合成派发,以及真实 OneBot `group.member_joined` → Agent → `send_group_msg` 链路。2026-07-15 Runner release preflight 16 项通过、0 warningfixture contract、5 类 behavior matrix、ledger schema / async DB readiness / 100-run stress / 120-run 8-worker contention / claim-lease-auth concurrency、SDK runtime chaos 探针全部通过。 |
| Host Skill / native tool integration | Unit-pass; WebUI E2E pass | 2026-07-15 provider / native / Skill / monitoring 定向测试 67 项通过,Pipeline / Chat / Wrapper 定向测试 61 项通过,Skills CLI 105 项通过;真实 Debug Chat 验证 `register_skill` 后同 Query `activate` 成功,监控工具调用不再把 SQL 行误取为字符串,结构化 JSON 文件检查不依赖格式空格,非流式多阶段 runner 结果只生成一个最终 Bot 气泡。 |
| SDK AgentRunner control entities / proxy | Unit-pass | 2026-06-23 SDK `tests/api/entities/builtin/agent_runner``tests/api/proxies``tests/api/test_agent_tools_mcp_bridge.py``tests/runtime/plugin/test_mgr_agent_runner.py``tests/runtime/test_pull_api_handlers.py``tests/runtime/io/handlers/test_plugin_handler.py`、EBA event entities 和 message tests 通过,覆盖 typed entities、AgentRunAPIProxy、MCP bridge、runtime manager 与 pull API handlers。 |
| SDK Runner control entities / proxy | Unit-pass | 2026-06-23 SDK `tests/api/entities/builtin/runner``tests/api/proxies``tests/api/test_agent_tools_mcp_bridge.py``tests/runtime/plugin/test_mgr_runner.py``tests/runtime/test_pull_api_handlers.py``tests/runtime/io/handlers/test_plugin_handler.py`、EBA event entities 和 message tests 通过,覆盖 typed entities、RunnerAPIProxy、MCP bridge、runtime manager 与 pull API handlers。 |
## 历史高价值记录
@@ -7,7 +7,7 @@
- **Bot**:平台连接、凭据和事件路由入口。用户在机器人上决定“发生什么时,使用哪个处理器”。
- **Processor**:可复用处理逻辑的上位概念,当前类型为 Agent 与 Pipeline。
- **Pipeline**:保留完整 Stage 链的消息处理器,提供预处理、AI、后处理、扩展和输出控制。
- **Agent**:独立配置对象,选择 AgentRunner 插件并配置事件范围、运行器与工具权限,可被多个 Bot 引用。
- **Agent**:独立配置对象,选择 Runner 插件并配置事件范围、运行器与工具权限,可被多个 Bot 引用。
- **Workflow**:后续编排方向,当前尚无完整执行产品。
- **Solution**:后续分发单元,包含处理器、路由模板、依赖、变量和文档。
+8 -8
View File
@@ -1,6 +1,6 @@
# Event Based Agents 架构设计总览
> Product revision (2026-09-07): [Event processors and Pipeline plugin compatibility](./09-event-processors.md) defines an explicitly bound EventProcessor alongside Pipeline and Agent. It supersedes the automatic EBA observer product model below; the new component and UI are planned, not yet implemented.
> Product revision (2026-09-07): [Event processors and Pipeline plugin compatibility](./09-event-processors.md) defines an explicitly bound Runner alongside Pipeline and Agent. It supersedes the automatic EBA observer product model below; the new component and UI are planned, not yet implemented.
> 当前状态(2026-09-05):平台事件、Bot `event_bindings`、独立 Agent、Pipeline / Agent 平级路由及 WebUI 已集成到 `dev/4.11.x`。实现入口为 `pkg/platform/botmgr.py::RuntimeBot` 与 `pkg/agent/runner/`。下文“当前架构的局限性”“现有架构”描述改造前背景;EventBus / EventRouter 图表示职责划分,不表示存在同名独立服务。当前实现和验收以 [STATUS.md](../agent-runner-pluginization/STATUS.md) 为准,平台动作使用[授权工具](../agent-runner-pluginization/PLATFORM_ACTION_TOOLS.md)。
@@ -52,9 +52,9 @@ MessageAggregator (消息聚合)
QueryPool → Controller → Pipeline (固定阶段链)
│ │
│ ▼
AgentRunner Host orchestrator
│ Runner Host orchestrator
│ ▼
│ plugin AgentRunner
│ plugin Runner
adapter.reply_message() / adapter.send_message()
@@ -83,7 +83,7 @@ EventBus (统一事件总线)
EventRouter (读取 Bot 的 event_bindings)
├─→ Pipeline target — 完整 Stage 链,仅消息事件
├─→ Agent target — 独立 Agent,经插件 AgentRunner 执行
├─→ Agent target — 独立 Agent,经插件 Runner 执行
└─→ discard — 明确丢弃
@@ -150,9 +150,9 @@ pkg/platform/adapters/
### 3.4 事件响应目标与观察者
Pipeline 与 Agent 是长期并存、场景不同的同级处理器。Pipeline 保留完整 Stage 链,面向消息处理;Agent 是独立配置对象,选择一个已安装的插件 AgentRunner,并可声明消息或非消息事件能力。Bot 的 `event_bindings` 只负责把事件绑定到既有 Pipeline、独立 Agent 或 `discard`
Pipeline 与 Agent 是长期并存、场景不同的同级处理器。Pipeline 保留完整 Stage 链,面向消息处理;Agent 是独立配置对象,选择一个已安装的插件 Runner,并可声明消息或非消息事件能力。Bot 的 `event_bindings` 只负责把事件绑定到既有 Pipeline、独立 Agent 或 `discard`
插件 EventListener 是观察者:事件先广播给有权限的监听器,随后路由器再选择一个响应目标。Webhook、Dify、n8n 等外部执行方式若需要作为响应者,应由对应 AgentRunner 插件表达,而不是增加另一套 Host Handler 主链。
插件 EventListener 是观察者:事件先广播给有权限的监听器,随后路由器再选择一个响应目标。Webhook、Dify、n8n 等外部执行方式若需要作为响应者,应由对应 Runner 插件表达,而不是增加另一套 Host Handler 主链。
现有 Pipeline 不会被转换为 AgentPipeline 内的 runner 配置也不会复制到独立 Agent。用户需要 Agent 时自行创建并绑定。
@@ -174,7 +174,7 @@ Pipeline 与 Agent 是长期并存、场景不同的同级处理器。Pipeline
| 2 | 适配器特有 API | 统一抽象 + `call_platform_api` 透传 | 通用 API 覆盖大部分场景,透传机制保证灵活性,避免每个适配器导出独立的类型化 API 包 |
| 3 | 向后兼容策略 | 兼容层适配 | 保留旧事件类型和 API 作为新系统的 alias/wrapper,现有插件无需修改 |
| 4 | 处理器配置存储 | Bot 表使用 `event_bindings`,目标引用原始 Pipeline 或独立 Agent UUID | 路由关系不复制处理器配置,Pipeline/Agent 各自保持事实源 |
| 5 | Agent 处理器定位 | 独立 Agent + 插件 AgentRunner | Host 不再内置具体 runner;不同 AgentRunner 通过统一协议接入 |
| 5 | Agent 处理器定位 | 独立 Agent + 插件 Runner | Host 不再内置具体 runner;不同 Runner 通过统一协议接入 |
| 6 | 事件命名方式 | 命名空间式(`message.received` | 清晰的分类层级,便于通配匹配(`message.*`),与 WebUI 配置天然对应 |
## 5. 文档索引
@@ -194,7 +194,7 @@ Pipeline 与 Agent 是长期并存、场景不同的同级处理器。Pipeline
| 仓库 | 改动范围 |
|------|----------|
| **langbot-plugin-sdk** | 事件定义、实体模型、API 接口、适配器基类、通信协议扩展 |
| **LangBot**(后端) | 适配器实现、事件路由引擎、Bot/Agent 实体、AgentRunner Host 编排 |
| **LangBot**(后端) | 适配器实现、事件路由引擎、Bot/Agent 实体、Runner Host 编排 |
| **LangBot**(前端) | Bot 事件处理器编排面板 |
| **langbot-wiki** | 新架构文档、插件开发指南更新、适配器开发指南 |
| **langbot-plugin-demo** | 示例更新(使用新事件和 API) |
+1 -1
View File
@@ -499,7 +499,7 @@ class PlatformSpecificEvent(Event):
8. 目标处理事件
│ Pipeline → 进入完整 Pipeline 流水线(仅消息事件)
│ Agent → Host 编排已安装的插件 AgentRunner
│ Agent → Host 编排已安装的插件 Runner
│ discard → 不产生响应
9. 处理器执行完毕,可能通过 Host 授权 API 执行响应动作
+10 -10
View File
@@ -1,6 +1,6 @@
# 事件路由与编排
> Implementation update (2026-09-08): the EventListener observer-broadcast proposal below is superseded by [Event processors](09-event-processors.md). Legacy EventListener hooks run only inside Pipeline. New EBA handlers use explicitly created and bound EventProcessor instances, a third peer processor type alongside Agent and Pipeline.
> Implementation update (2026-09-08): the EventListener observer-broadcast proposal below is superseded by [Event processors](09-event-processors.md). Legacy EventListener hooks run only inside Pipeline. New EBA handlers use explicitly created and bound Runner instances, a third peer processor type alongside Agent and Pipeline.
> 状态:当前实施模型(2026-07-12)。本文以 Pipeline / Agent 平级并存为准,不再保留早期 `pipeline / agent / webhook / plugin` 四种 Handler 草案。
@@ -16,10 +16,10 @@ Pipeline 与 Agent 是平级处理器:
| 处理器 | 配置事实源 | 执行路径 | 事件范围 |
| --- | --- | --- | --- |
| Pipeline | Pipeline 表与完整 Stage 配置 | MessageAggregator -> QueryPool -> RuntimePipeline | 消息事件,首版为 `message.received` |
| Agent | Agent 表中的 runner 与 runner config | AgentRunner Host orchestrator -> plugin AgentRunner | Agent/Runner 声明支持的消息或非消息事件 |
| Agent | Agent 表中的 runner 与 runner config | Runner Host orchestrator -> plugin Runner | Agent/Runner 声明支持的消息或非消息事件 |
| discard | 无处理器配置 | 明确结束路由 | 任意事件 |
插件 EventListener 不是第三种响应目标。Webhook、Dify、n8n、Coze 等外部系统需要响应事件时,由对应 AgentRunner 插件承接。
插件 EventListener 不是第三种响应目标。Webhook、Dify、n8n、Coze 等外部系统需要响应事件时,由对应 Runner 插件承接。
## 2. 数据模型
@@ -34,7 +34,7 @@ class Agent(Base):
description: str
emoji: str
kind: str # 固定为 "agent"
component_ref: str # AgentRunner id
component_ref: str # Runner id
config: dict # runner + runner_config
supported_event_patterns: list[str]
```
@@ -57,7 +57,7 @@ class Agent(Base):
}
```
Runner id 来自已安装插件的 AgentRunner manifest。Host 不维护 LocalAgent、Dify 或其他具体实现的内置分支。
Runner id 来自已安装插件的 Runner manifest。Host 不维护 LocalAgent、Dify 或其他具体实现的内置分支。
### 2.2 EventBinding
@@ -129,20 +129,20 @@ Platform adapter
-> authorized Plugin EventListener observers
-> EventRouter
-> Pipeline target -> full Pipeline stage chain
-> Agent target -> AgentRunner Host orchestrator
-> Agent target -> Runner Host orchestrator
-> discard -> stop
-> Host delivery/platform API
```
### 4.1 Pipeline target
消息事件按原有方式构造 Query,经 MessageAggregator、QueryPool 和完整 Pipeline Stage 链执行。Pipeline 可以继续使用 AgentRunner 作为 AI stage 的实现,但 Pipeline 本身不会因此变成 Agent。
消息事件按原有方式构造 Query,经 MessageAggregator、QueryPool 和完整 Pipeline Stage 链执行。Pipeline 可以继续使用 Runner 作为 AI stage 的实现,但 Pipeline 本身不会因此变成 Agent。
### 4.2 Agent target
Host 读取独立 Agent 的 Runner id/config,构造 event-first context、run-scoped resources 与 delivery policy,再调用插件 AgentRunner。Runner 输出由 Host 统一归一化、记录和投递。
Host 读取独立 Agent 的 Runner id/config,构造 event-first context、run-scoped resources 与 delivery policy,再调用插件 Runner。Runner 输出由 Host 统一归一化、记录和投递。
AgentRunner 可通过 SDK/Python `AgentRunAPIProxy.call_tool` 或 SDK-owned scoped MCP bridge 回调 Host 能力。两条路径都映射到 `PluginToRuntimeAction.CALL_TOOL`,使用相同的 run authorization、Host execution Query、ToolManager 和 Box session 规则。Box session 是 Host canonical scope 的固定长度安全哈希;同一平台会话稳定、不同 scope 隔离、缺少 identity 时 fail closedRunner 不配置 sandbox scope。
Runner 可通过 SDK/Python `RunnerAPIProxy.call_tool` 或 SDK-owned scoped MCP bridge 回调 Host 能力。两条路径都映射到 `PluginToRuntimeAction.CALL_TOOL`,使用相同的 run authorization、Host execution Query、ToolManager 和 Box session 规则。Box session 是 Host canonical scope 的固定长度安全哈希;同一平台会话稳定、不同 scope 隔离、缺少 identity 时 fail closedRunner 不配置 sandbox scope。
### 4.3 Observer side effects
@@ -153,7 +153,7 @@ AgentRunner 可通过 SDK/Python `AgentRunAPIProxy.call_tool` 或 SDK-owned scop
1. Pipeline 与 Agent 保留各自的持久化、编辑和执行语义。
2. 处理器聚合页面可以统一展示二者,但不会创建第三份处理器记录。
3. 旧 Pipeline 仍是 Pipeline;其 runner config 不迁移、不复制为独立 Agent。
4. 需要 Agent 的用户新建 Agent、选择已安装 AgentRunner,再建立 event binding。
4. 需要 Agent 的用户新建 Agent、选择已安装 Runner,再建立 event binding。
5. 一个 Bot 可按不同事件同时绑定 Pipeline 与 Agent。
## 6. WebUI 约束
+1 -1
View File
@@ -1,6 +1,6 @@
# 插件 SDK 改造
> Implementation update (2026-09-08): the EventListener observer-broadcast proposal below is superseded by [Event processors](09-event-processors.md). Legacy EventListener hooks run only inside Pipeline. New EBA handlers use explicitly created and bound EventProcessor instances, a third peer processor type alongside Agent and Pipeline.
> Implementation update (2026-09-08): the EventListener observer-broadcast proposal below is superseded by [Event processors](09-event-processors.md). Legacy EventListener hooks run only inside Pipeline. New EBA handlers use explicitly created and bound Runner instances, a third peer processor type alongside Agent and Pipeline.
## 1. 概述
+11 -11
View File
@@ -1,6 +1,6 @@
# EBA 分阶段实施计划
> Implementation update (2026-09-08): the EventListener observer-broadcast proposal below is superseded by [Event processors](09-event-processors.md). Legacy EventListener hooks run only inside Pipeline. New EBA handlers use explicitly created and bound EventProcessor instances, a third peer processor type alongside Agent and Pipeline.
> Implementation update (2026-09-08): the EventListener observer-broadcast proposal below is superseded by [Event processors](09-event-processors.md). Legacy EventListener hooks run only inside Pipeline. New EBA handlers use explicitly created and bound Runner instances, a third peer processor type alongside Agent and Pipeline.
> 更新:2026-09-05。P0P4 的主要实现已落入 `dev/4.11.x`,P5 仍需按当前版本验收;下文工作项用于维护实现边界,不表示全部待开发。文件名沿用早期设计,但这里的“迁移”仅指代码架构逐步接入 EBA,不代表 LangBot 3.x 数据库或配置升级。当前提交、定向测试及发布缺口见 [STATUS.md](../agent-runner-pluginization/STATUS.md)。
@@ -11,15 +11,15 @@ EBA 跨越 SDK、平台适配器、LangBot Host、WebUI 与插件生态,按可
- LangBot 4.x 不支持从 3.x 数据库或配置升级;不保留 legacy migration chain、旧 JSON 模板或旧 Runner 字段读取。
- Pipeline 与 Agent 平级且长期并存,分别保留持久化模型与执行链。
- 现有 Pipeline 不迁移为 AgentPipeline 内的 runner config 不复制到 Agent。
- 用户需要 Agent 时新建独立 Agent并选择已安装的 AgentRunner。
- 用户需要 Agent 时新建独立 Agent并选择已安装的 Runner。
- Host 不按 LocalAgent id 做运行时、Box 或 WebUI 特判。
- AgentRunner 的 SDK/Python 与 scoped MCP bridge 回调共享 Host 授权与事件 session 规则。
- Runner 的 SDK/Python 与 scoped MCP bridge 回调共享 Host 授权与事件 session 规则。
## 2. 阶段总览
| 阶段 | 目标 | 主要仓库 | 完成条件 |
| --- | --- | --- | --- |
| P0 | SDK 事件、能力与 AgentRunner 协议 | `langbot-plugin-sdk` | typed entities、manifest、proxy、runtime action 通过测试 |
| P0 | SDK 事件、能力与 Runner 协议 | `langbot-plugin-sdk` | typed entities、manifest、proxy、runtime action 通过测试 |
| P1 | 平台适配器 EBA 化 | LangBot + SDK | 事件转换、能力声明、通用/透传 API 通过 adapter checklist |
| P2 | Host 观察者与响应者路由 | LangBot backend | observer 广播 + Pipeline/Agent/discard 单目标仲裁可运行 |
| P3 | 独立 Agent 与 Runner 注册 | LangBot backend + plugins | Agent CRUD、registry、run authorization、delivery 可运行 |
@@ -32,8 +32,8 @@ EBA 跨越 SDK、平台适配器、LangBot Host、WebUI 与插件生态,按可
- 定义规范化平台事件、actor/subject/conversation/delivery context。
- 定义 adapter `supported_events``supported_apis` 与平台透传 API。
- 定义 AgentRunner manifest、run context/result、resource handles 和 pull/callback API。
- 提供 `AgentRunAPIProxy` 与 SDK-owned scoped MCP bridge。
- 定义 Runner manifest、run context/result、resource handles 和 pull/callback API。
- 提供 `RunnerAPIProxy` 与 SDK-owned scoped MCP bridge。
- 保持协议传输与权限校验可测试,不把 Host 私有 Query 对象暴露给插件。
### 验收
@@ -76,7 +76,7 @@ adapter event
- Plugin EventListener 是 observer,不作为 priority fallback。
- Pipeline 只处理消息事件并复用完整 Stage 链。
- Agent 使用独立 Agent 配置和 AgentRunner Host orchestrator。
- Agent 使用独立 Agent 配置和 Runner Host orchestrator。
- edit/reaction 等事件的 observer 副作用能力按事件和 adapter 能力过滤。
- dry-run 与合成派发必须使用同一匹配器,避免 UI 预览与真实路由漂移。
@@ -86,13 +86,13 @@ adapter event
- 同一事件最多一个响应目标,但 observer 仍能收到事件。
- Pipeline 与 Agent 可以在同一个 Bot 的不同 binding 中同时生效。
## 6. P3:独立 Agent 与 AgentRunner
## 6. P3:独立 Agent 与 Runner
### 工作项
- `agents` 只保存 Agent;Pipeline 继续使用自己的表和 API。
- Agent config 使用 `runner.id``runner_config[runner_id]`
- registry 只展示已安装、有效的插件 AgentRunner。
- registry 只展示已安装、有效的插件 Runner。
- Host 构造 run-scoped resources、state、delivery 与 event log/transcript。
- SDK/Python `call_tool` 和 scoped MCP bridge 都回到同一个 Host ToolManager。
- Box session 由 Host 将 instance/workspace/bot/adapter/target/thread scope 规范化并哈希为固定长度 `lb-box-<sha256>`;同 scope 稳定、不同 scope 隔离、缺少 identity 时 fail closed。
@@ -117,7 +117,7 @@ adapter event
### 验收
- 页面不出现 LocalAgent 专属 banner、变量隐藏或 Box/Pipeline 注入逻辑。
- 空 Runner 市场状态给出可安装 AgentRunner 的正常路径。
- 空 Runner 市场状态给出可安装 Runner 的正常路径。
- Pipeline Debug Chat/Monitoring 与 Agent 运行日志分别可用。
## 8. P5:发布门禁
@@ -125,7 +125,7 @@ adapter event
### 自动化
- LangBot backend unit/integration tests 与 Ruff。
- SDK AgentRunner/proxy/MCP bridge tests。
- SDK Runner/proxy/MCP bridge tests。
- Web lint/build 与关键 Playwright cases。
- `skills/bin/lbs validate``skills/bin/lbs index --check`
- LocalAgent 与其他官方 Runner plugin package/test gate。
@@ -1,6 +1,6 @@
# Agent 与 Pipeline 统一编排(产品最终形态)
> Implementation update (2026-09-08): the EventListener observer-broadcast proposal below is superseded by [Event processors](09-event-processors.md). Legacy EventListener hooks run only inside Pipeline. New EBA handlers use explicitly created and bound EventProcessor instances, a third peer processor type alongside Agent and Pipeline.
> Implementation update (2026-09-08): the EventListener observer-broadcast proposal below is superseded by [Event processors](09-event-processors.md). Legacy EventListener hooks run only inside Pipeline. New EBA handlers use explicitly created and bound Runner instances, a third peer processor type alongside Agent and Pipeline.
> **状态**:历史方向稿(2026-06-12);2026-09-05 标记归档用途。本文的示意 schema、5.0 发布火车、SDK 0.5.0aX 配套与多租户“预留”描述不再作为实施合同。当前 4.11 产品形态见 [08-agent-page-and-event-orchestration.md](./08-agent-page-and-event-orchestration.md),协议见 [PROTOCOL_V1.md](../agent-runner-pluginization/PROTOCOL_V1.md),已完成与剩余事项见 [STATUS.md](../agent-runner-pluginization/STATUS.md)。保留正文仅用于解释早期设计取舍。
>
@@ -20,7 +20,7 @@
EventRouter(事件 → 处理器绑定)
├─→ 选中的处理器(响应者,单一仲裁)
│ ├─ Pipeline:保留现有实体和执行链,仅处理消息事件
│ └─ Agent:用户新建并选择 AgentRunner 插件,可接本地、低代码或外部 runtime
│ └─ Agent:用户新建并选择 Runner 插件,可接本地、低代码或外部 runtime
└─→ 插件 EventListener(观察者,N 个广播,可 prevent_default
```
@@ -51,7 +51,7 @@ EventRouter(事件 → 处理器绑定)
04 文档中的 pipeline / agent / webhook / plugin 四种 handler_type,本质上都是"对事件作出响应的逻辑",差别只在编写和部署方式。产品层统一展示和绑定这些处理器,但不会把既有 Pipeline 持久化为 Agent
- **产品**:用户只需理解"给 Bot 的事件绑定处理器",处理器可以是 Pipeline 或 Agent
- **工程**:路由层按 `target_type` 分发到 Pipeline 或 AgentAgent 的扩展集中到 AgentRunner 抽象;
- **工程**:路由层按 `target_type` 分发到 Pipeline 或 AgentAgent 的扩展集中到 Runner 抽象;
- **生态**:Agent 成为市场上可分发、可复用的一等公民。
### 2.2 收编映射
@@ -59,7 +59,7 @@ EventRouter(事件 → 处理器绑定)
| 原 handler_type04 文档) | 收编后 |
|---------------------------|--------|
| `pipeline` | 保留 Pipeline 实体;binding 使用 `target_type=pipeline` 和原 `pipeline_uuid`,进程内直接复用 MessageAggregator → QueryPool → Pipeline 机制 |
| `agent`RequestRunner | 用户新建独立 Agent,并选择对应 AgentRunner 插件;不读取或复制旧 Pipeline 内嵌 runner 配置 |
| `agent`RequestRunner | 用户新建独立 Agent,并选择对应 Runner 插件;不读取或复制旧 Pipeline 内嵌 runner 配置 |
| `webhook` | 外部 Agent 的一种:事件 POST 出去、响应解析为动作(保留 04 §5.4 的请求/响应格式) |
| `plugin`EventListener 分发) | **不收编**——角色不同,见 §2.3 |
@@ -76,7 +76,7 @@ EventRouter(事件 → 处理器绑定)
### 3.1 独立 Agent 与现有 Pipeline
Agent 与 Pipeline 都是一等处理器。用户创建 Agent、选择已安装的 AgentRunner,再把适合的事件绑定到 AgentPipeline 继续保存在 Pipeline 表中,以完整 Stage 链处理消息事件。两者可在同一处理器列表中以不同 `kind` 展示和选择;这种聚合展示不会创建额外记录,也不会在两种模型之间复制配置。
Agent 与 Pipeline 都是一等处理器。用户创建 Agent、选择已安装的 Runner,再把适合的事件绑定到 AgentPipeline 继续保存在 Pipeline 表中,以完整 Stage 链处理消息事件。两者可在同一处理器列表中以不同 `kind` 展示和选择;这种聚合展示不会创建额外记录,也不会在两种模型之间复制配置。
```python
class Agent(Base):
@@ -84,7 +84,7 @@ class Agent(Base):
uuid: str # 主键
name: str
kind: str # 固定为 "agent"Pipeline 使用自己的持久模型
component_ref: str # AgentRunner id,例如 plugin:<author>/<plugin>/<runner>
component_ref: str # Runner id,例如 plugin:<author>/<plugin>/<runner>
config: dict # JSON — runner id、runner config 与资源/状态/投递策略
# 多租户预留:归属主体字段(tenant/workspace),首版可空
```
@@ -156,7 +156,7 @@ class AgentChunk:
```
**流式**:复用 SDK 通信协议既有的 `chunk_status: continue/end` 机制,`handle()` 的每次 yield 对应一个 chunk。
**Pipeline 与 Agent 分流**Pipeline target 继续走 LangBot 进程内的 Pipeline 执行链;独立 Agent 经 AgentRunner 插件 runtime 分发。路由层通过 binding 的 `target_type` 明确区分二者。
**Pipeline 与 Agent 分流**Pipeline target 继续走 LangBot 进程内的 Pipeline 执行链;独立 Agent 经 Runner 插件 runtime 分发。路由层通过 binding 的 `target_type` 明确区分二者。
### 4.3 执行语义与可靠性
@@ -173,7 +173,7 @@ class AgentChunk:
| 版本 | 内容 | 备注 |
|------|------|------|
| 4.11(可选) | 现状成果:12 个 EBA 适配器、插件全事件订阅、`call_platform_api` | 对用户不可见的管道工程 + 插件新能力,不动产品概念 |
| **5.0** | 产品形态首发:EventRouter + event→处理器绑定 + WebUI 编排 + 旧 Bot 路由迁移 + 独立 Agent / AgentRunner 插件 + SDK Agent 组件契约(可标 experimental | `use_pipeline_uuid` 仅改写为指向原 Pipeline 的 binding,不生成 Agent;配 SDK 0.5.0 正式版;走 beta 周期 |
| **5.0** | 产品形态首发:EventRouter + event→处理器绑定 + WebUI 编排 + 旧 Bot 路由迁移 + 独立 Agent / Runner 插件 + SDK Agent 组件契约(可标 experimental | `use_pipeline_uuid` 仅改写为指向原 Pipeline 的 binding,不生成 Agent;配 SDK 0.5.0 正式版;走 beta 周期 |
| 5.x | 工作流 Agent(工作流引擎线挂入)、Agent 市场生态、剩余适配器(satori 等)、Agent 插件化收尾 | 验证开放注册机制 |
| 多租户 | 独立评估:仅数据隔离 → 5.x 部署选项;伴随权限/计费/产品定位变化 → 6.0 | 前置条件是 §4.3 的归属主体预留已落实 |
@@ -1,6 +1,6 @@
# 处理器页面与事件编排产品设计
> Implementation update (2026-09-08): the EventListener observer-broadcast proposal below is superseded by [Event processors](09-event-processors.md). Legacy EventListener hooks run only inside Pipeline. New EBA handlers use explicitly created and bound EventProcessor instances, a third peer processor type alongside Agent and Pipeline.
> Implementation update (2026-09-08): the EventListener observer-broadcast proposal below is superseded by [Event processors](09-event-processors.md). Legacy EventListener hooks run only inside Pipeline. New EBA handlers use explicitly created and bound Runner instances, a third peer processor type alongside Agent and Pipeline.
> 状态:当前实现说明(2026-09-05),对应 `dev/4.11.x`。P0P3 已集成;发布验收见 [STATUS.md](../agent-runner-pluginization/STATUS.md)。
>
@@ -12,7 +12,7 @@ LangBot 的处理逻辑分成两种同级形态:
| 形态 | 定位 | 可处理事件 | 典型用户 |
| --- | --- | --- | --- |
| Agent | runner 驱动的事件优先处理器,承载 AgentRunner / 外部 runner | `message.*``group.*``friend.*``bot.*``feedback.*``platform.*` 等声明范围 | 需要直接处理多类平台事件或接入外部 agent runtime 的用户 |
| Agent | runner 驱动的事件优先处理器,承载 Runner / 外部 runner | `message.*``group.*``friend.*``bot.*``feedback.*``platform.*` 等声明范围 | 需要直接处理多类平台事件或接入外部 agent runtime 的用户 |
| Pipeline | 可视化、可控、可组合的消息处理流水线,执行完整 Stage 链 | 仅 `message.*`,首版等价于 `message.received` | 需要预处理、AI、后处理、扩展和输出控制的消息场景 |
处理器页面负责统一管理这两种处理单元:
@@ -38,7 +38,7 @@ LangBot 的处理逻辑分成两种同级形态:
- Pipeline:沿用原 Pipeline 配置页,包括 AI、触发、安全、输出、扩展、Debug、Monitoring
- Agent:基础信息由详情入口编辑,主配置分为运行器、运行器配置、事件与工具;事件范围、自动事件工具、平台级动作和普通工具白名单在同一配置流程内维护。
处理器详情复用 `ProcessorDetailWorkbench`Agent 与 Pipeline 保留各自的配置、调试和日志语义。`AgentRunnerSelect` 提供已安装 Runner 和市场安装入口,安装状态可恢复;Runner 配置来自动态 metadata,不按 LocalAgent id 定制 Host 表单。调试事件选择位于输入区域;Bot 的平台事件调试位于机器人配置中,路由 dry-run 只解释匹配结果。
处理器详情复用 `ProcessorDetailWorkbench`Agent 与 Pipeline 保留各自的配置、调试和日志语义。`RunnerSelect` 提供已安装 Runner 和市场安装入口,安装状态可恢复;Runner 配置来自动态 metadata,不按 LocalAgent id 定制 Host 表单。调试事件选择位于输入区域;Bot 的平台事件调试位于机器人配置中,路由 dry-run 只解释匹配结果。
`/home/pipelines` 继续提供 Pipeline 直接编辑路径;共享处理器入口当前使用 `/home/agents`。URL 是实现路径,不代表 Agent 包含 Pipeline。
@@ -174,8 +174,8 @@ Bot 使用 `event_bindings` JSON 字段持久化路由。当前未引入独立
- EBA 事件先广播插件 observer。
- 然后按 `event_bindings` 的事件模式、filters、priority 和顺序选择一个处理器。
- Pipeline 目标通过 MessageAggregator 进入完整 Pipeline Stage 链;Agent 目标直接进入 AgentRunner 链路。
- 非消息事件只选择声明支持该事件的 Agent,不调用 PipelineAgentRunner 输出有平台 reply target 时会投递回平台。
- Pipeline 目标通过 MessageAggregator 进入完整 Pipeline Stage 链;Agent 目标直接进入 Runner 链路。
- 非消息事件只选择声明支持该事件的 Agent,不调用 PipelineRunner 输出有平台 reply target 时会投递回平台。
## 7. 不做的事
+47 -179
View File
@@ -1,30 +1,18 @@
# Event processors and Pipeline plugin compatibility
Status: implemented in the 4.11 development branches of LangBot and the Plugin SDK,
2026-09-08. The Host uv configuration pins the matching SDK commit. Deploy both
revisions together; older SDK releases do not contain this component. Switch the
development source pin to a published SDK release before a stable PyPI release.
This design supersedes the automatic EBA EventListener observer broadcast in
the earlier EBA documents. Existing Pipeline plugin behavior remains supported.
# Runner components and Pipeline plugin compatibility
## Product boundary
Three processor types appear together in the Processors area:
| Product | Implementation | Event entry |
| --- | --- | --- |
| Pipeline | Pipeline stages and an agent-capable Runner | Received messages |
| Agent | A Runner with `spec.usages: [agent]` | Configured events |
| Plugin processor | A Runner with `spec.usages: [event]` | Events declared in `spec.events` |
| Product | Implementation | Event entry | Flow ownership |
| --- | --- | --- | --- |
| Pipeline | Existing Pipeline stages and configuration | Received messages | Pipeline stages, including legacy plugin hooks |
| Agent | A configured plugin AgentRunner | Supported EBA events | The selected runner |
| Event processor | A configured plugin EventProcessor | Supported EBA events | Plugin Python handlers |
Use **Event processor** as the product label and **EventProcessor** as the SDK
component name. The localized description should explain that the plugin defines
the processing logic. It must not suggest an LLM, prompt, or visual workflow is
required.
An installed component is a reusable implementation. A processor instance is a
user-created configuration of that component. A Bot event binding selects an
instance, not an installed plugin package directly.
Runner is the only component for these execution styles. `spec.usages` can contain
both `agent` and `event`; these are selection capabilities, not mutually exclusive
execution modes. An installed component is reusable code. Users create processor
instances, select a Runner and configure it, then bind Bot events to the instance.
Installation alone never subscribes a component to incoming events.
## Legacy EventListener contract
@@ -57,168 +45,48 @@ information must be distinguished from fields that were dropped during conversio
Direct Agent and Event processor execution must not synthesize Pipeline lifecycle
hooks. Those hooks describe actual Pipeline stages.
## EventProcessor SDK contract
## SDK and runtime
Introduce a distinct component kind instead of changing what an existing
EventListener manifest means. One package may contain both kinds; only the legacy
EventListener participates in Pipeline hook dispatch.
`lbp comp Runner` generates `components/runner`. Every component uses
`plugin:author/plugin/name` as its identity. Names are unique within a plugin.
The component can override `async run(ctx)` and yield RunnerResult objects, or
register typed platform callbacks through `@self.handler(EventClass)` in
`initialize()`. Default run dispatches an exact handler, falling back to EBAEvent.
A custom run can delegate to this dispatch with `await super().run(ctx)`.
Retain the familiar authoring shape:
Both styles share RunnerContext, invocation-bound ctx.api, logs, replies, deadlines,
cancellation, worker isolation and the run ledger. ctx.event is the envelope;
ctx.platform_event is the typed platform payload. Each invocation owns its context;
never put the current context or run ID on a shared component or plugin instance.
```python
from langbot_plugin.api.definition.components.event_processor import EventProcessor, EventProcessorContext
from langbot_plugin.api.entities.builtin.platform.events import MemberJoinedEvent
class WelcomeProcessor(EventProcessor):
async def initialize(self):
await super().initialize()
The runtime emits completion on normal return unless the Runner already emitted a
terminal result. Exceptions fail the run and retain preceding results. Cancelling
the result stream cancels execution. There is no implicit retry or hidden model
loop. Returned text and logs do not send platform messages: replies are explicit
ctx.reply / ctx.reply_stream actions. Pipeline retains its configured output stage.
@self.handler(MemberJoinedEvent)
async def on_join(ctx: EventProcessorContext):
await ctx.reply(f"Hello, {ctx.event.member.nickname}")
```
`self.plugin` continues to expose ordinary plugin APIs. ctx.api carries run-scoped
resource grants and records tool actions. Workspace and installation authorization
remain Host-enforced. Run identity and API operation scope are separate concepts.
Handlers receive typed EBA events directly. Do not maintain a second, incomplete
mapping into plugin-only EBA wrapper classes. Preserve complete public event
fields; compact log previews must not become the execution payload. Include the
generic platform-specific event contract for adapter-specific events.
## Selection and observability
The context belongs to one invocation and exposes the event, processor/run
identifiers, instance configuration, logging, and authorized Host APIs. It has no
fabricated Pipeline Query. Reuse Host run tracking, deadlines, installation
authority, platform capabilities, and delivery records where appropriate.
Both product selectors discover the same Runner catalog and filter by usage.
Validate usage again before execution. Event-capable Runners must declare events;
users can route a subset, but cannot expand the manifest capability. Unconfigured
instances expose no event subscriptions. Workspace ownership, plugin scope and
instance identity are checked for routing, execution, cancellation and run reads.
A handler returning normally completes its invocation. There is no implicit LLM
loop, automatic second processor, or hidden retry of side effects. An exception
marks the run failed and retains the associated log. New processing handlers do
not use prevent_default to control another processor; routing has already chosen
the current processor. The legacy methods keep their existing Pipeline meaning.
Plugin processor details keep event debugging on the left and configuration/logs
on the right. Component settings use the existing schema form. Logs and action
results remain distinct from actual platform delivery; debug delivery is Mock.
Agent-native interactions stay on the Agent product path; typed handlers consume
platform events. Legacy Pipeline lifecycle hooks remain on the Pipeline path.
## Activation and routing
## Validation
The activation sequence is explicit:
1. Install a plugin containing an EventProcessor component.
2. Create an Event processor in the Processors area.
3. Open its detail page, select a plugin component, and save its configuration.
4. Bind a Bot event to that processor instance in the existing event routing UI.
Installation and processor creation alone do not subscribe to Bot events.
The component declares the event types it handles; Bot bindings select the subset
of supported events to deliver. One package may supply multiple components, and
multiple instances may use the same component with independent configuration.
Extend the existing single-target route arbitration with `event_processor`.
There is no automatic EBA broadcast to installed EventListeners. Keep Pipeline hook dispatch inside the Pipeline path. Existing observer
plugins must explicitly adopt the new component and be bound by the user; do not
create subscriptions during migration.
An unconfigured instance has no supported events and cannot execute. Validate
component availability, event compatibility, Workspace ownership, and instance
identity when configuring the instance and again at invocation. A disabled or
unavailable plugin leaves the instance visible with an actionable unavailable
status. It must not silently fall back to Agent or Pipeline.
## Compact UI
Creation adds a third type next to Agent and Pipeline and asks only for basic
instance information. Select the plugin component in the detail-page header.
Show component-defined configuration in the right pane, with Configuration and
Logs tabs. Keep unsaved values when switching tabs, and open Logs after a debug
run finishes. The component selector remains in the page header.
If no component is installed, show a relevant plugin installation entry point;
installing still does not create a binding.
The detail page keeps event debugging on the left while the right pane switches
between configuration and logs. A compact run list shows event type, time, status and known
processing duration. Selecting a row shows that run's identity, input, logs,
actions and outcome below. There is no shared timeline between unrelated runs.
The additive `created_at_ms`, `started_at_ms`, and `finished_at_ms` fields retain
Host lifecycle precision for elapsed-time display.
Keep payloads and error details collapsed until expanded. Distinguish attempted
delivery from confirmed delivery and display the actual destination.
Place component identity, availability, bindings, and configuration in a compact
secondary area. Do not add a prompt editor, model selector, or flow designer.
The plugin implements the processing flow in code.
## Delivery sequence and acceptance
1. Repair and regression-test Pipeline EventContext handling and legacy payload
conversion independently of the new processor feature.
2. Add the SDK component, context, manifest/scaffolding, and explicit invocation
contract; verify registration, event coverage, and process isolation.
3. Add Host instance management, event routing, execution tracking, and matching
HTTP/MCP/skill surfaces. Turn off automatic EBA observer dispatch in this step.
4. Add creation, binding, availability, logs, and delivery trace UI with i18n.
5. Exercise a real packaged plugin through installation, explicit instance
creation, Bot binding, invocation, logging, and reply delivery.
Acceptance must prove that installation alone invokes no handlers; one matching
binding invokes exactly the chosen component; instance configuration and run
history remain separate; all declared EBA events retain their fields; unavailable
components fail visibly; and legacy plugins keep the documented Pipeline hook
order and behavior. Unit tests alone do not establish a successful live plugin
installation or platform delivery.
## Implemented transport and APIs
`lbp comp EventProcessor` scaffolds a component in `components/event_processor`.
Its manifest uses `kind: EventProcessor` and `spec.events`, for example
`[group.member_joined]`. `spec.config` defines instance parameters. A component
that calls `ctx.reply()` declares `spec.permissions.tools: [detail, call]`.
References use `event_processor:author/plugin/component`, separate from
`plugin:author/plugin/runner`. Both kinds share the existing run transport,
installation authorization, deadlines and run ledger. The trusted Host selects
the component kind; the worker invokes only that exact kind and name.
There is no model invocation in the EventProcessor base class.
`EventProcessorContext` provides `event`, `run_id`, `config`, `api`, `log()` and
`reply()`. `api` is the existing run-scoped Host proxy. Use `ctx.config` for instance
parameters; plugin installation configuration remains separate. Handlers may
register the `EBAEvent` base class as a catch-all. An exact typed handler takes
precedence over that fallback. Multiple handlers for the same type run in their
registration order, within one invocation.
HTTP instance management uses `/api/v1/agents` with `kind: event_processor`.
Metadata at `/api/v1/agents/_/metadata` lists installed `event_processors`.
Creation accepts `component_ref` and `parameters`; the Host derives the supported
event patterns from the component. Bot bindings use `target_type: event_processor`
and the created instance UUID as `target_id`.
- `GET /api/v1/agents/{id}/runs?before_id=...` lists this instance's runs.
- `GET /api/v1/agents/{id}/runs/{run_id}/events?after_sequence=...` pages its logs
and action results. A run from another instance or Workspace is rejected.
- The corresponding MCP tools are `get_processor_metadata`, `list_processor_runs`
and `get_processor_run_events`, alongside processor CRUD.
- `/api/v1/agents/{id}/debug` accepts a full typed EBA event in `data`. Platform
actions use Mock; other authorized tools retain their configured behavior.
The detail page polls run updates, keeps payload details collapsed and separates
logs from platform delivery. Completed handlers produce no synthetic reply text.
## Verification (2026-09-08)
- SDK API, scaffolding and Plugin Runtime suites: 684 passed.
- Host runner, service, controller, MCP and adapter regression suites: 971 passed.
- Pipeline and registry regression suites: 243 passed, one environment-dependent skip.
- Frontend unit suite: 74 passed; TypeScript and changed-file lint checks passed.
- Real packaged-plugin tests cover installation, component-kind separation,
invocation, mock platform delivery, instance isolation and persisted logs.
- Authenticated Edge testing created an instance and a loopback OneBot bot, saved
a member-join binding, injected one native notice and received exactly one
`send_group_msg` response. The detail page displayed the completed run,
localized action name, destination and returned message ID. No external IM
account or live model was involved.
- Browser regression covers expanding long payloads, reaching the last log and
pagination without duplication. Eight unrelated existing browser failures were
reproduced against the pre-change commit; the full suite is not green.
- Repository-wide lint also retains the pre-existing duplicate `send_image_msg`
in the WeCom customer-service library and existing formatting failures outside
this change. The i18n check has the same pre-existing diagnostics as its baseline.
Native Agent interaction-resumption is not enabled for EventProcessor bindings;
its input contract is the platform EBA event collection, not a synthetic Agent
continuation. Plugins should handle platform events through their typed handlers.
SDK tests cover both execution styles, event matrices, concurrent contexts,
termination, cancellation and permissions. Packaged CLI tests generate, build and
execute the published component. Core tests cover usage-filtered discovery, event
routing, Workspace authorization and real plugin-runtime transport. RunnerDemo
provides multi-step actions, configuration isolation and controlled failures.
+7 -7
View File
@@ -13,7 +13,7 @@
┌──────────────────────────────────────────────────────────────────┐
│ LangBot 主进程 │
│ │
AgentRunner ──> SDK call_tool / scoped MCP bridge │
│ Runner ──> SDK call_tool / scoped MCP bridge │
│ │ │ │
│ └────────────────> ToolManager ──> NativeToolLoader │
│ │ │ │ │
@@ -87,8 +87,8 @@
**核心设计原则**:
- Box Runtime 作为独立进程运行,通过 Action RPC 与 LangBot 主进程通信,两者复用 SDK 的 IO 层(Handler → Connection → Controller
- 一个 session_id 对应一个容器/沙箱实例。同一 session 内可并存多条 mount 与多个 managed process
- AgentRunner 无权指定 session scope。SDK/Python `call_tool` 与 scoped MCP bridge 都发出同一个 `PluginToRuntimeAction.CALL_TOOL`,最终由 Host 的 ToolManager 执行,并使用当前 run 保存的同一个 execution Query
- Box 内托管的 stdio MCP server 使用独立的长期 `mcp-shared` session;它不是 AgentRunner 本次事件的 sandbox session(详见 [box-session-scope.md](./box-session-scope.md)
- Runner 无权指定 session scope。SDK/Python `call_tool` 与 scoped MCP bridge 都发出同一个 `PluginToRuntimeAction.CALL_TOOL`,最终由 Host 的 ToolManager 执行,并使用当前 run 保存的同一个 execution Query
- Box 内托管的 stdio MCP server 使用独立的长期 `mcp-shared` session;它不是 Runner 本次事件的 sandbox session(详见 [box-session-scope.md](./box-session-scope.md)
---
@@ -140,7 +140,7 @@ BoxService
**输出截断**: 默认 4000 字符上限,保留前 60% + 后 40%,中间插入 `[...truncated...]`
**Session 所有权**: `resolve_box_session_id(query)` 只接受 Host 已确定的私有 scope 或 Query launcher/session identity,并输出 `lb-box-` + 64 位小写 SHA-256 十六进制摘要(固定 71 个 ASCII 字符)。哈希输入是 canonical JSON,包含 instance、workspace、bot、platform adapter、target type/id 与 thread;原始用户、群组、conversation 或 event id 不会出现在 Box session id 中。相同 Host scope 稳定复用,不同 target/thread/workspace/bot/adapter/instance 相互隔离;缺少可用 identity 时 fail closed。Pipeline、Agent 或 AgentRunner 配置都不能覆盖该规则。
**Session 所有权**: `resolve_box_session_id(query)` 只接受 Host 已确定的私有 scope 或 Query launcher/session identity,并输出 `lb-box-` + 64 位小写 SHA-256 十六进制摘要(固定 71 个 ASCII 字符)。哈希输入是 canonical JSON,包含 instance、workspace、bot、platform adapter、target type/id 与 thread;原始用户、群组、conversation 或 event id 不会出现在 Box session id 中。相同 Host scope 稳定复用,不同 target/thread/workspace/bot/adapter/instance 相互隔离;缺少可用 identity 时 fail closed。Pipeline、Agent 或 Runner 配置都不能覆盖该规则。
**Skill 挂载合并**: `execute_tool()` 调用时,`build_skill_extra_mounts(query)` 会把当前 pipeline-bound 的所有 skill 的 `package_root` 作为 `extra_mounts` 加入 BoxSpec,挂在 `/workspace/.skills/<name>`。LLM 通过 `activate` 工具显式激活某个 skill 后,工具调用才允许引用这个 skill 的虚拟路径。
@@ -421,7 +421,7 @@ ToolManager.initialize()
3. 若 skill 是 Python 项目(有 `requirements.txt``pyproject.toml`),命令会被 venv bootstrap 包裹(在 skill 挂载点内创建 `.venv`
4. 调用 `box_service.execute_tool()` → 走 Host 从当前事件生成的 session_id 与已组装好的 `extra_mounts`**不再为每 skill 起独立 session**
AgentRunner 可以直接通过 SDK/Python `AgentRunAPIProxy.call_tool` 调用这些工具,也可以让外部 harness 通过 SDK-owned scoped MCP bridge 回调。两条入口都发送 `PluginToRuntimeAction.CALL_TOOL`,共享同一个 run authorization、Host session 中保存的 execution Query、ToolManager 与 `resolve_box_session_id(query)` 规则;Runner 不能提交自定义 Box session id。Pipeline run 保存原 Query;纯 EBA run 由 Host 构造 `pipeline_config=None``pipeline_uuid=None` 的最小 Query。
Runner 可以直接通过 SDK/Python `RunnerAPIProxy.call_tool` 调用这些工具,也可以让外部 harness 通过 SDK-owned scoped MCP bridge 回调。两条入口都发送 `PluginToRuntimeAction.CALL_TOOL`,共享同一个 run authorization、Host session 中保存的 execution Query、ToolManager 与 `resolve_box_session_id(query)` 规则;Runner 不能提交自定义 Box session id。Pipeline run 保存原 Query;纯 EBA run 由 Host 构造 `pipeline_config=None``pipeline_uuid=None` 的最小 Query。
### 4.3 MCP-in-Box (`mcp_stdio.py`, 354 行)
@@ -442,7 +442,7 @@ initialize()
每条 MCP server 是同一 session 中的一个 managed process,独立的 `process_id`、独立 attach URL,互不阻塞。
这里的 `mcp-shared` 只承载 LangBot 管理的 stdio MCP server 进程。AgentRunner 的 scoped MCP bridge 是回调 Host 工具的协议入口,不会把事件运行的 exec/read/write 改到 `mcp-shared`
这里的 `mcp-shared` 只承载 LangBot 管理的 stdio MCP server 进程。Runner 的 scoped MCP bridge 是回调 Host 工具的协议入口,不会把事件运行的 exec/read/write 改到 `mcp-shared`
---
@@ -582,7 +582,7 @@ volumes:
### Session scope
Pipeline 与 AgentRunner 配置不再暴露 sandbox session 模板。Host 将当前平台会话/事件 scope 规范化后哈希成固定长度的 `lb-box-<sha256>`;相同 scope 稳定复用,不同 scope 隔离,缺少 identity 时拒绝执行。SDK/Python 与 scoped MCP bridge 的工具调用遵守同一规则。详见 [box-session-scope.md](./box-session-scope.md)。
Pipeline 与 Runner 配置不再暴露 sandbox session 模板。Host 将当前平台会话/事件 scope 规范化后哈希成固定长度的 `lb-box-<sha256>`;相同 scope 稳定复用,不同 scope 隔离,缺少 identity 时拒绝执行。SDK/Python 与 scoped MCP bridge 的工具调用遵守同一规则。详见 [box-session-scope.md](./box-session-scope.md)。
### REST API
+11 -11
View File
@@ -7,7 +7,7 @@
## 1. Decision
The LangBot Host owns the Box session used by an event run. A Pipeline, Agent,
or AgentRunner cannot choose a global, per-user, per-conversation, or per-query
or Runner cannot choose a global, per-user, per-conversation, or per-query
sandbox mode.
`BoxService.resolve_box_session_id(query)` always returns this shape:
@@ -81,7 +81,7 @@ Host scope or launcher/session identity is also rejected. There is no
## 3. Host execution Query
AgentRunner callbacks need a Host-owned Query view because model/tool loaders
Runner callbacks need a Host-owned Query view because model/tool loaders
already consume that type. The Query is internal and is never exposed as a
Runner-controlled object.
@@ -96,11 +96,11 @@ Runner-controlled object.
This gives Pipeline and pure EBA execution the same Host tool path without
inventing a fake Pipeline for an independent Agent.
## 4. AgentRunner callback paths
## 4. Runner callback paths
AgentRunner implementations may use either callback transport:
Runner implementations may use either callback transport:
1. SDK/Python runners call `AgentRunAPIProxy.call_tool`.
1. SDK/Python runners call `RunnerAPIProxy.call_tool`.
2. External harnesses call the SDK-owned scoped MCP bridge.
Both transports emit the same `PluginToRuntimeAction.CALL_TOOL`. The Host then
@@ -108,8 +108,8 @@ validates the same run authorization, restores the same execution Query, and
dispatches to the same ToolManager and BoxService.
```text
AgentRunner
+-- AgentRunAPIProxy.call_tool --------+
Runner
+-- RunnerAPIProxy.call_tool --------+
| |
+-- SDK-owned scoped MCP bridge -------+--> PluginToRuntimeAction.CALL_TOOL
--> run authorization
@@ -119,7 +119,7 @@ AgentRunner
--> lb-box-<sha256>
```
An AgentRunner is not required to use MCP. Local Python runners can use the SDK
An Runner is not required to use MCP. Local Python runners can use the SDK
directly; code-agent harnesses can use the bridge. The transports do not define
different authorization or sandbox semantics.
@@ -143,16 +143,16 @@ This is separate from the scoped MCP bridge above:
| Path | Purpose | Session rule |
| --- | --- | --- |
| AgentRunner scoped MCP bridge | Call authorized Host tools for one active run | Host-owned `lb-box-<sha256>` from the run execution Query |
| Runner scoped MCP bridge | Call authorized Host tools for one active run | Host-owned `lb-box-<sha256>` from the run execution Query |
| MCP-in-Box stdio server | Keep configured MCP server processes running | Dedicated persistent `mcp-shared` session |
Calling a sandbox tool through the AgentRunner bridge never redirects the run
Calling a sandbox tool through the Runner bridge never redirects the run
workspace into `mcp-shared`. Conversely, an MCP server's managed-process
lifecycle does not inherit the current event scope.
## 7. Configuration and compatibility
There is no Box session scope field in Pipeline metadata, AgentRunner config,
There is no Box session scope field in Pipeline metadata, Runner config,
or the public Pipeline/Runner API. Operators configure the Box subsystem itself
(`box.enabled`, backend/runtime settings, profiles, mount allowlists, quotas,
and workspace roots), not per-Runner session templates.
+1 -1
View File
@@ -62,7 +62,7 @@
| Backend selection | 良好 | 显式 backend 优先级、local 探测顺序、配置变更触发 reselect |
| MCP Box 集成 | 良好 | config model、路径重写、payload、shared-session 多 process |
| Native tool loader | 良好 | 6 工具(exec/read/write/edit/glob/grep)、路径穿越拦截 |
| AgentRunner 工具入口 | 良好 | SDK proxy 与 MCP bridge 都映射到 `PluginToRuntimeAction.CALL_TOOL`Host action 测试覆盖 run-scoped execution Query 与纯 EBA native exec |
| Runner 工具入口 | 良好 | SDK proxy 与 MCP bridge 都映射到 `PluginToRuntimeAction.CALL_TOOL`Host action 测试覆盖 run-scoped execution Query 与纯 EBA native exec |
| Skill 系统 | 良好 | 加载、Tool Call 激活、marker、路径解析、authoring CRUD、HTTP service |
---
+6 -6
View File
@@ -3,24 +3,24 @@
> 更新日期: 2026-06-29
> 分支: `mcp_resources`
> PR: langbot-app/LangBot#2215
> 主题: MCP Resources 在 LangBot 中的产品价值、AgentRunner 集成方式与后续架构方向
> 主题: MCP Resources 在 LangBot 中的产品价值、Runner 集成方式与后续架构方向
## 结论
PR #2215 对 LangBot 有明确价值:它补齐了 MCP 协议中 Resources 这一重要能力,让 MCP server 不再只暴露 tools,也可以暴露文档、代码片段、配置、日志、图片等上下文资源。管理端可以发现和预览资源,Agent 也可以通过当前实现按需列出和读取资源。
但当前 AgentRunner 层的接入方式更接近一个可用的第一阶段方案,而不是最终架构。现在 MCP Resources 被包装成两个 synthetic tools
但当前 Runner 层的接入方式更接近一个可用的第一阶段方案,而不是最终架构。现在 MCP Resources 被包装成两个 synthetic tools
- `langbot_mcp_list_resources`
- `langbot_mcp_read_resource`
这让模型可以通过 function calling 主动探索资源,落地成本低,也复用了已有 `ToolManager` / `LocalAgentRunner` 的工具调用链路。不过从 MCP 规范和主流实现来看,Resources 更适合作为一种一等上下文来源,而不是长期隐藏在工具列表里。
这让模型可以通过 function calling 主动探索资源,落地成本低,也复用了已有 `ToolManager` / `LocalRunner` 的工具调用链路。不过从 MCP 规范和主流实现来看,Resources 更适合作为一种一等上下文来源,而不是长期隐藏在工具列表里。
建议保留当前 synthetic tools 作为探索能力,同时把后续主线设计调整为:MCP Resources 是 pipeline / conversation / message 级别可选择、可固定、可审计的上下文输入。
## 当前实现判断
当前 AgentRunner 集成路径如下:
当前 Runner 集成路径如下:
```text
Pipeline 绑定 MCP server
@@ -28,7 +28,7 @@ Pipeline 绑定 MCP server
-> Preproc 为 local-agent 加载工具
-> ToolManager.get_all_tools()
-> MCPLoader 注入 synthetic resource tools
-> LocalAgentRunner 将工具 schema 传给模型
-> LocalRunner 将工具 schema 传给模型
-> 模型发起 list/read tool call
-> ToolManager.execute_func_call()
-> MCPLoader 调 MCP session.list_resources/read_resource
@@ -178,7 +178,7 @@ LangBot 后续应支持模板发现、参数填写、实例化和绑定。否则
- 支持 resource templates。
- 支持资源订阅更新。
- 支持 chunk、summary、RAG 化接入。
- 为 DifyAgentRunner、LocalAgentRunner 等不同 runner 定义统一资源上下文接口。
- 为 DifyRunner、LocalRunner 等不同 runner 定义统一资源上下文接口。
## 最终建议
@@ -68,7 +68,7 @@ Anthropic、Google 和 LiteLLM 的官方文档域名在本次环境中被浏览
- `LLMModel.extra_args` 是 JSON 字段,Web 端已有通用高级参数编辑器。
- `LiteLLMRequester` 会按“模型级 `extra_args`,再调用级 `extra_args`”的顺序合并参数。
- LiteLLM 已统一处理多个 Provider 的 `reasoning_effort``thinking` 和返回的 `reasoning_content`
- `LocalAgentRunner` 的非流式、流式、工具调用和 fallback 路径都经过 `RuntimeProvider.invoke_llm*()`
- `LocalRunner` 的非流式、流式、工具调用和 fallback 路径都经过 `RuntimeProvider.invoke_llm*()`
- `remove-think` 已能控制 `<think>` 或独立 reasoning 内容是否进入展示文本。
- Gemini 工具调用所需的 `provider_specific_fields` / thought signature 已有保留逻辑和单元测试。
+1 -1
View File
@@ -232,4 +232,4 @@ line-ending = "auto"
[tool.uv.sources]
# Development contract: update to the matching SDK release before publishing.
langbot-plugin = { git = "https://github.com/langbot-app/langbot-plugin-sdk", rev = "ca0671b81db5b2ed937d01158a3982de3fd8500f" }
langbot-plugin = { git = "https://github.com/langbot-app/langbot-plugin-sdk", rev = "92a9e03fa9c791f4ed30cc3f5f0602c13b800d28" }
@@ -99,7 +99,7 @@ LangBot 是异步且集成度高的系统,有些问题不会直接表现为页
```text
Action list_plugins call timed out
Action list_agent_runners call timed out
Action list_runners call timed out
Action invoke_llm_stream call timed out
```
+3 -3
View File
@@ -61,7 +61,7 @@ bin/lbs fixture check
```
`env doctor` 会检查 URL、路径、代理变量等。代理变量是可选项;只有大小写代理变量互相冲突时才会报错。失败不一定代表仓库坏了,通常说明本地 LangBot 没启动、代理不一致或浏览器 profile 不存在。
`fixture check` 会检查仓库内测试 fixture 是否存在,例如 MCP stdio server、RAG 文档、多模态图片、qa-plugin-smoke 包和 QA AgentRunner 包。它也会校验 `.lbpkg` 是 zip 包,并检查 QA AgentRunner fixture 的入口文件未漂移。
`fixture check` 会检查仓库内测试 fixture 是否存在,例如 MCP stdio server、RAG 文档、多模态图片、qa-plugin-smoke 包和 QA Runner 包。它也会校验 `.lbpkg` 是 zip 包,并检查 QA Runner fixture 的入口文件未漂移。
4. 查看已有测试 case
@@ -344,8 +344,8 @@ npx playwright install chromium
脚本会尝试通过 `LANGBOT_PIPELINE_NAME` 从 Pipelines 页面进入目标 pipeline。两者都没有时,
该自动化会返回 `blocked`,不会伪造通过。
Runner 专用 case 不应复用通用 pipeline 变量。Local Agent、Codex AgentRunner 和
Claude Code AgentRunner 这类 case 会通过 `automation_pipeline_url_env` /
Runner 专用 case 不应复用通用 pipeline 变量。Local Agent、Codex Runner 和
Claude Code Runner 这类 case 会通过 `automation_pipeline_url_env` /
`automation_pipeline_name_env` 映射到 case-specific env,例如
`LANGBOT_LOCAL_AGENT_PIPELINE_URL`。这些 case 如果缺少专用变量,会返回 `blocked`
不会退回到 `LANGBOT_PIPELINE_URL`,避免跑错 pipeline 后产生假阳性。
+195 -164
View File
@@ -1,5 +1,5 @@
#!/usr/bin/env python3
"""Audit one persisted AgentRunner run without exposing authorization secrets."""
"""Audit one persisted Runner run without exposing authorization secrets."""
from __future__ import annotations
@@ -25,29 +25,29 @@ from agent_run_ledger_policy import (
def database_url(repo: pathlib.Path) -> str:
config = yaml.safe_load((repo / "data/config.yaml").read_text(encoding="utf-8")) or {}
database = config.get("database", {})
kind = database.get("use", "sqlite")
if kind == "sqlite":
path = pathlib.Path(database.get("sqlite", {}).get("path", "data/langbot.db"))
config = yaml.safe_load((repo / 'data/config.yaml').read_text(encoding='utf-8')) or {}
database = config.get('database', {})
kind = database.get('use', 'sqlite')
if kind == 'sqlite':
path = pathlib.Path(database.get('sqlite', {}).get('path', 'data/langbot.db'))
if not path.is_absolute():
path = repo / path
return f"sqlite+aiosqlite:///{path}"
if kind in {"postgres", "postgresql"}:
values = database.get("postgresql", {})
user = urllib.parse.quote_plus(str(values.get("user", "postgres")))
password = urllib.parse.quote_plus(str(values.get("password", "postgres")))
host = values.get("host", "127.0.0.1")
port = values.get("port", 5432)
name = values.get("database", "postgres")
return f"postgresql+asyncpg://{user}:{password}@{host}:{port}/{name}"
raise RuntimeError(f"Unsupported database backend: {kind}")
return f'sqlite+aiosqlite:///{path}'
if kind in {'postgres', 'postgresql'}:
values = database.get('postgresql', {})
user = urllib.parse.quote_plus(str(values.get('user', 'postgres')))
password = urllib.parse.quote_plus(str(values.get('password', 'postgres')))
host = values.get('host', '127.0.0.1')
port = values.get('port', 5432)
name = values.get('database', 'postgres')
return f'postgresql+asyncpg://{user}:{password}@{host}:{port}/{name}'
raise RuntimeError(f'Unsupported database backend: {kind}')
def parse_created_after(value: str | None) -> datetime.datetime | None:
if not value:
return None
parsed = datetime.datetime.fromisoformat(value.replace("Z", "+00:00"))
parsed = datetime.datetime.fromisoformat(value.replace('Z', '+00:00'))
if parsed.tzinfo is not None:
parsed = parsed.astimezone(datetime.timezone.utc).replace(tzinfo=None)
return parsed
@@ -55,19 +55,19 @@ def parse_created_after(value: str | None) -> datetime.datetime | None:
def event_matches_tool_call(data_json: str | None, tool_name: str, parameters: dict | None) -> bool:
try:
data = json.loads(data_json or "{}")
data = json.loads(data_json or '{}')
except (TypeError, ValueError):
return False
if not isinstance(data, dict) or data.get("tool_name") != tool_name:
if not isinstance(data, dict) or data.get('tool_name') != tool_name:
return False
return parameters is None or data.get("parameters") == parameters
return parameters is None or data.get('parameters') == parameters
def collect_result_texts(value: object) -> list[str]:
texts: list[str] = []
if isinstance(value, dict):
for key, item in value.items():
if key == "text" and isinstance(item, str):
if key == 'text' and isinstance(item, str):
texts.append(item)
else:
texts.extend(collect_result_texts(item))
@@ -85,7 +85,7 @@ async def audit(
expected_tool_name: str | None = None,
expected_parameters: dict | None = None,
expected_result_text: str | None = None,
tool_authorization_mode: str = "strict",
tool_authorization_mode: str = 'strict',
) -> dict:
engine = create_async_engine(database_url(repo))
failures: list[dict] = []
@@ -93,82 +93,106 @@ async def audit(
try:
async with engine.connect() as connection:
if run_id:
run_row = (await connection.execute(
sqlalchemy.text("SELECT * FROM agent_run WHERE run_id = :run_id"),
{"run_id": run_id},
)).mappings().first()
run_row = (
(
await connection.execute(
sqlalchemy.text('SELECT * FROM agent_run WHERE run_id = :run_id'),
{'run_id': run_id},
)
)
.mappings()
.first()
)
elif expected_tool_name:
query = "SELECT * FROM agent_run"
query = 'SELECT * FROM agent_run'
params = {}
if created_after is not None:
query += " WHERE created_at >= :created_after"
params["created_after"] = created_after
query += " ORDER BY id DESC LIMIT 100"
query += ' WHERE created_at >= :created_after'
params['created_after'] = created_after
query += ' ORDER BY id DESC LIMIT 100'
candidates = (await connection.execute(sqlalchemy.text(query), params)).mappings().all()
run_row = None
for candidate in candidates:
started_rows = (await connection.execute(
sqlalchemy.text(
"SELECT data_json FROM agent_run_event "
"WHERE run_id = :run_id AND type = 'tool.call.started' ORDER BY sequence"
),
{"run_id": str(candidate["run_id"])},
)).mappings().all()
started_rows = (
(
await connection.execute(
sqlalchemy.text(
'SELECT data_json FROM agent_run_event '
"WHERE run_id = :run_id AND type = 'tool.call.started' ORDER BY sequence"
),
{'run_id': str(candidate['run_id'])},
)
)
.mappings()
.all()
)
if any(
event_matches_tool_call(row.get("data_json"), expected_tool_name, expected_parameters)
event_matches_tool_call(row.get('data_json'), expected_tool_name, expected_parameters)
for row in started_rows
):
run_row = candidate
break
else:
run_row = (await connection.execute(
sqlalchemy.text("SELECT * FROM agent_run ORDER BY id DESC LIMIT 1")
)).mappings().first()
run_row = (
(await connection.execute(sqlalchemy.text('SELECT * FROM agent_run ORDER BY id DESC LIMIT 1')))
.mappings()
.first()
)
if run_row is None:
status = "fail" if expected_tool_name else "env_issue"
status = 'fail' if expected_tool_name else 'env_issue'
return {
"status": status,
"reason": "No AgentRunner run contains the expected tool call." if expected_tool_name else "No matching AgentRunner run exists.",
"failures": [{"kind": "expected_tool_call_missing"}] if expected_tool_name else [],
"warnings": [],
'status': status,
'reason': 'No Runner run contains the expected tool call.'
if expected_tool_name
else 'No matching Runner run exists.',
'failures': [{'kind': 'expected_tool_call_missing'}] if expected_tool_name else [],
'warnings': [],
}
selected_run_id = str(run_row["run_id"])
event_rows = (await connection.execute(
sqlalchemy.text("SELECT sequence, type, data_json, metadata_json FROM agent_run_event WHERE run_id = :run_id ORDER BY sequence"),
{"run_id": selected_run_id},
)).mappings().all()
selected_run_id = str(run_row['run_id'])
event_rows = (
(
await connection.execute(
sqlalchemy.text(
'SELECT sequence, type, data_json, metadata_json FROM agent_run_event WHERE run_id = :run_id ORDER BY sequence'
),
{'run_id': selected_run_id},
)
)
.mappings()
.all()
)
finally:
await engine.dispose()
authorization = load_ledger_json(
run_row.get("authorization_json"),
field="agent_run.authorization_json",
run_row.get('authorization_json'),
field='agent_run.authorization_json',
failures=failures,
)
tools = authorization.get("resources", {}).get("tools", []) if isinstance(authorization, dict) else []
tools = authorization.get('resources', {}).get('tools', []) if isinstance(authorization, dict) else []
allowed_tools: dict[str, dict] = {}
incomplete_tool_metadata: list[dict] = []
for tool in tools if isinstance(tools, list) else []:
if not isinstance(tool, dict):
incomplete_tool_metadata.append({"tool_name": "", "missing": ["tool object"]})
incomplete_tool_metadata.append({'tool_name': '', 'missing': ['tool object']})
continue
name = str(tool.get("tool_name", ""))
name = str(tool.get('tool_name', ''))
missing = []
if not name:
missing.append("tool_name")
if not str(tool.get("description", "")).strip():
missing.append("description")
if not isinstance(tool.get("parameters"), dict):
missing.append("parameters")
if not (tool.get("source") or tool.get("tool_type") or tool.get("source_id")):
missing.append("owner")
missing.append('tool_name')
if not str(tool.get('description', '')).strip():
missing.append('description')
if not isinstance(tool.get('parameters'), dict):
missing.append('parameters')
if not (tool.get('source') or tool.get('tool_type') or tool.get('source_id')):
missing.append('owner')
if missing:
incomplete_tool_metadata.append({"tool_name": name, "missing": missing})
incomplete_tool_metadata.append({'tool_name': name, 'missing': missing})
if name:
allowed_tools[name] = tool
if incomplete_tool_metadata:
failures.append({"kind": "incomplete_tool_metadata", "tools": incomplete_tool_metadata})
failures.append({'kind': 'incomplete_tool_metadata', 'tools': incomplete_tool_metadata})
starts: dict[str, list[dict]] = {}
completions: dict[str, list[dict]] = {}
@@ -178,7 +202,7 @@ async def audit(
invalid_tool_argument_errors: list[dict] = []
successful_tool_completion_sequences: list[int] = []
forbidden_pattern = re.compile(
r"invalid json(?! arguments)|unauthori[sz]ed|permission denied|forbidden|timed?\s*out|timeout",
r'invalid json(?! arguments)|unauthori[sz]ed|permission denied|forbidden|timed?\s*out|timeout',
re.I,
)
@@ -189,55 +213,53 @@ async def audit(
return collected
for key, item in value.items():
normalized = str(key).lower()
if normalized in {"error", "code", "status", "reason", "error_message"} and item is not None and item != "":
if normalized in {'error', 'code', 'status', 'reason', 'error_message'} and item is not None and item != '':
collected.append(str(item))
if isinstance(item, dict):
collected.extend(error_surface(item))
return collected
for row in event_rows:
event_type = str(row["type"])
event_type = str(row['type'])
event_types.append(event_type)
before = len(failures)
data = load_ledger_json(
row.get("data_json"),
field=f"agent_run_event[{row['sequence']}].data_json",
row.get('data_json'),
field=f'agent_run_event[{row["sequence"]}].data_json',
failures=failures,
)
invalid_event_json += int(len(failures) > before)
if not isinstance(data, dict):
failures.append({"kind": "invalid_event_payload", "sequence": row["sequence"], "type": event_type})
failures.append({'kind': 'invalid_event_payload', 'sequence': row['sequence'], 'type': event_type})
continue
if event_type in {"tool.call.started", "tool.call.completed"}:
call_id = str(data.get("tool_call_id", ""))
item = {"sequence": row["sequence"], "tool_name": str(data.get("tool_name", "")), "data": data}
if event_type in {'tool.call.started', 'tool.call.completed'}:
call_id = str(data.get('tool_call_id', ''))
item = {'sequence': row['sequence'], 'tool_name': str(data.get('tool_name', '')), 'data': data}
if not call_id:
failures.append({"kind": "missing_tool_call_id", "sequence": row["sequence"], "type": event_type})
elif event_type == "tool.call.started":
failures.append({'kind': 'missing_tool_call_id', 'sequence': row['sequence'], 'type': event_type})
elif event_type == 'tool.call.started':
starts.setdefault(call_id, []).append(item)
else:
completions.setdefault(call_id, []).append(item)
if not data.get("error") and data.get("result") is not None:
successful_tool_completion_sequences.append(row["sequence"])
diagnostic_text = "\n".join(error_surface(data))
if event_type == "run.failed":
diagnostic_text += "\n" + json.dumps(data, ensure_ascii=True)
if not data.get('error') and data.get('result') is not None:
successful_tool_completion_sequences.append(row['sequence'])
diagnostic_text = '\n'.join(error_surface(data))
if event_type == 'run.failed':
diagnostic_text += '\n' + json.dumps(data, ensure_ascii=True)
match = forbidden_pattern.search(diagnostic_text)
if match:
suspicious_errors.append({"sequence": row["sequence"], "type": event_type, "signal": match.group(0)})
elif event_type == "tool.call.completed":
suspicious_errors.append({'sequence': row['sequence'], 'type': event_type, 'signal': match.group(0)})
elif event_type == 'tool.call.completed':
signal = invalid_tool_argument_error_signal(diagnostic_text)
if signal:
invalid_tool_argument_errors.append(
{"sequence": row["sequence"], "type": event_type, "signal": signal}
)
invalid_tool_argument_errors.append({'sequence': row['sequence'], 'type': event_type, 'signal': signal})
if run_row["status"] != "completed":
failures.append({"kind": "run_status", "actual": run_row["status"], "expected": "completed"})
if "run.completed" not in event_types:
failures.append({"kind": "missing_run_completed_event"})
if "run.failed" in event_types:
failures.append({"kind": "run_failed_event"})
if run_row['status'] != 'completed':
failures.append({'kind': 'run_status', 'actual': run_row['status'], 'expected': 'completed'})
if 'run.completed' not in event_types:
failures.append({'kind': 'missing_run_completed_event'})
if 'run.failed' in event_types:
failures.append({'kind': 'run_failed_event'})
all_call_ids = sorted(set(starts) | set(completions))
unauthorized_calls = []
@@ -245,14 +267,21 @@ async def audit(
started = starts.get(call_id, [])
completed = completions.get(call_id, [])
if len(started) != 1 or len(completed) != 1:
failures.append({"kind": "tool_call_pairing", "tool_call_id": call_id, "started": len(started), "completed": len(completed)})
failures.append(
{
'kind': 'tool_call_pairing',
'tool_call_id': call_id,
'started': len(started),
'completed': len(completed),
}
)
continue
if started[0]["tool_name"] != completed[0]["tool_name"]:
failures.append({"kind": "tool_name_mismatch", "tool_call_id": call_id})
if started[0]["sequence"] >= completed[0]["sequence"]:
failures.append({"kind": "tool_call_order", "tool_call_id": call_id})
if started[0]["tool_name"] not in allowed_tools:
unauthorized_calls.append({"tool_call_id": call_id, "tool_name": started[0]["tool_name"]})
if started[0]['tool_name'] != completed[0]['tool_name']:
failures.append({'kind': 'tool_name_mismatch', 'tool_call_id': call_id})
if started[0]['sequence'] >= completed[0]['sequence']:
failures.append({'kind': 'tool_call_order', 'tool_call_id': call_id})
if started[0]['tool_name'] not in allowed_tools:
unauthorized_calls.append({'tool_call_id': call_id, 'tool_name': started[0]['tool_name']})
authorization_failures, authorization_warnings = classify_tool_authorization(
unauthorized_calls,
authorization_mode=tool_authorization_mode,
@@ -263,20 +292,18 @@ async def audit(
invalid_tool_argument_errors,
successful_tool_completion_sequences=successful_tool_completion_sequences,
run_completed=(
run_row["status"] == "completed"
and "run.completed" in event_types
and "run.failed" not in event_types
run_row['status'] == 'completed' and 'run.completed' in event_types and 'run.failed' not in event_types
),
)
if unrecovered_argument_errors:
suspicious_errors.extend(unrecovered_argument_errors)
warnings.extend(recovered_argument_warnings)
if suspicious_errors:
failures.append({"kind": "forbidden_error_signals", "events": suspicious_errors})
failures.append({'kind': 'forbidden_error_signals', 'events': suspicious_errors})
if not event_rows:
failures.append({"kind": "missing_run_events"})
failures.append({'kind': 'missing_run_events'})
if not tools:
warnings.append({"kind": "no_authorized_tools", "reason": "The run authorization snapshot exposes no tools."})
warnings.append({'kind': 'no_authorized_tools', 'reason': 'The run authorization snapshot exposes no tools.'})
expected_call_summary = None
if expected_tool_name:
@@ -284,97 +311,101 @@ async def audit(
item
for items in starts.values()
for item in items
if item["tool_name"] == expected_tool_name
and (expected_parameters is None or item["data"].get("parameters") == expected_parameters)
if item['tool_name'] == expected_tool_name
and (expected_parameters is None or item['data'].get('parameters') == expected_parameters)
]
if len(matching_starts) != 1:
failures.append({"kind": "expected_tool_call_count", "actual": len(matching_starts), "expected": 1})
failures.append({'kind': 'expected_tool_call_count', 'actual': len(matching_starts), 'expected': 1})
matching_completions = []
for started in matching_starts:
call_id = str(started["data"].get("tool_call_id", ""))
call_id = str(started['data'].get('tool_call_id', ''))
matching_completions.extend(completions.get(call_id, []))
result_text_match = expected_result_text is None or any(
expected_result_text in collect_result_texts(completed["data"].get("result"))
expected_result_text in collect_result_texts(completed['data'].get('result'))
for completed in matching_completions
)
if expected_result_text is not None and not result_text_match:
failures.append({"kind": "expected_tool_result_text_missing"})
failures.append({'kind': 'expected_tool_result_text_missing'})
expected_call_summary = {
"tool_name": expected_tool_name,
"parameters_match_required": expected_parameters is not None,
"matched_started_count": len(matching_starts),
"matched_completed_count": len(matching_completions),
"result_text_match_required": expected_result_text is not None,
"result_text_match": result_text_match,
'tool_name': expected_tool_name,
'parameters_match_required': expected_parameters is not None,
'matched_started_count': len(matching_starts),
'matched_completed_count': len(matching_completions),
'result_text_match_required': expected_result_text is not None,
'result_text_match': result_text_match,
}
metrics = {
"event_count": len(event_rows),
"tool_call_started": sum(len(items) for items in starts.values()),
"tool_call_completed": sum(len(items) for items in completions.values()),
"tool_call_ids": len(all_call_ids),
"authorized_tool_count": len(allowed_tools),
"tool_authorization_mode": tool_authorization_mode,
"runner_native_tool_call_count": len(unauthorized_calls) if tool_authorization_mode == "runner-native" else 0,
"invalid_event_json": invalid_event_json,
"suspicious_error_count": len(suspicious_errors),
"recovered_tool_argument_error_count": len(recovered_argument_warnings),
'event_count': len(event_rows),
'tool_call_started': sum(len(items) for items in starts.values()),
'tool_call_completed': sum(len(items) for items in completions.values()),
'tool_call_ids': len(all_call_ids),
'authorized_tool_count': len(allowed_tools),
'tool_authorization_mode': tool_authorization_mode,
'runner_native_tool_call_count': len(unauthorized_calls) if tool_authorization_mode == 'runner-native' else 0,
'invalid_event_json': invalid_event_json,
'suspicious_error_count': len(suspicious_errors),
'recovered_tool_argument_error_count': len(recovered_argument_warnings),
}
return {
"status": "pass" if not failures else "fail",
"reason": "Agent run ledger audit passed." if not failures else f"Agent run ledger audit found {len(failures)} invariant failure(s).",
"run": {
"run_id": selected_run_id,
"runner_id": run_row["runner_id"],
"status": run_row["status"],
"created_at": str(run_row["created_at"]),
"finished_at": str(run_row["finished_at"]),
'status': 'pass' if not failures else 'fail',
'reason': 'Agent run ledger audit passed.'
if not failures
else f'Agent run ledger audit found {len(failures)} invariant failure(s).',
'run': {
'run_id': selected_run_id,
'runner_id': run_row['runner_id'],
'status': run_row['status'],
'created_at': str(run_row['created_at']),
'finished_at': str(run_row['finished_at']),
},
"metrics": metrics,
"expected_tool_call": expected_call_summary,
"failures": failures,
"warnings": warnings,
'metrics': metrics,
'expected_tool_call': expected_call_summary,
'failures': failures,
'warnings': warnings,
}
def main() -> int:
parser = argparse.ArgumentParser()
parser.add_argument("--repo", required=True)
parser.add_argument("--run-id")
parser.add_argument("--created-after")
parser.add_argument("--expected-tool-name")
parser.add_argument("--expected-parameters-json")
parser.add_argument("--expected-result-text")
parser.add_argument('--repo', required=True)
parser.add_argument('--run-id')
parser.add_argument('--created-after')
parser.add_argument('--expected-tool-name')
parser.add_argument('--expected-parameters-json')
parser.add_argument('--expected-result-text')
parser.add_argument(
"--tool-authorization-mode",
choices=("strict", "runner-native"),
default="strict",
'--tool-authorization-mode',
choices=('strict', 'runner-native'),
default='strict',
)
parser.add_argument("--output", required=True)
parser.add_argument('--output', required=True)
args = parser.parse_args()
try:
expected_parameters = None
if args.expected_parameters_json:
expected_parameters = json.loads(args.expected_parameters_json)
if not isinstance(expected_parameters, dict):
raise ValueError("--expected-parameters-json must decode to an object")
raise ValueError('--expected-parameters-json must decode to an object')
if (expected_parameters is not None or args.expected_result_text) and not args.expected_tool_name:
raise ValueError("--expected-tool-name is required with expected parameters or result text")
report = asyncio.run(audit(
pathlib.Path(args.repo).resolve(),
args.run_id,
created_after=parse_created_after(args.created_after),
expected_tool_name=args.expected_tool_name,
expected_parameters=expected_parameters,
expected_result_text=args.expected_result_text,
tool_authorization_mode=args.tool_authorization_mode,
))
raise ValueError('--expected-tool-name is required with expected parameters or result text')
report = asyncio.run(
audit(
pathlib.Path(args.repo).resolve(),
args.run_id,
created_after=parse_created_after(args.created_after),
expected_tool_name=args.expected_tool_name,
expected_parameters=expected_parameters,
expected_result_text=args.expected_result_text,
tool_authorization_mode=args.tool_authorization_mode,
)
)
except Exception as exc: # noqa: BLE001 - probe must classify environment failures
report = {"status": "env_issue", "reason": str(exc), "failures": [], "warnings": []}
pathlib.Path(args.output).write_text(json.dumps(report, indent=2) + "\n", encoding="utf-8")
report = {'status': 'env_issue', 'reason': str(exc), 'failures': [], 'warnings': []}
pathlib.Path(args.output).write_text(json.dumps(report, indent=2) + '\n', encoding='utf-8')
print(json.dumps(report))
return 0 if report["status"] == "pass" else 2 if report["status"] == "env_issue" else 1
return 0 if report['status'] == 'pass' else 2 if report['status'] == 'env_issue' else 1
if __name__ == "__main__":
if __name__ == '__main__':
sys.exit(main())
@@ -143,7 +143,7 @@ try {
}
if (!runner?.name) {
result.status = "blocked";
throw new Error("No registered AgentRunner is available for the UI check.");
throw new Error("No registered Runner is available for the UI check.");
}
const runnerConfigStage = runnerTab.stages.find(
@@ -155,7 +155,7 @@ try {
body: {
kind: "agent",
name: `Runner Health ${paths.runId.slice(-40)}`,
description: "Temporary AgentRunner health visibility fixture",
description: "Temporary Runner health visibility fixture",
emoji: "H",
component_ref: runner.name,
config: {
@@ -229,7 +229,7 @@ try {
}
result.status = "pass";
result.reason =
"Agent Runner settings visibly distinguished a registered runner from a stale binding.";
"Runner settings visibly distinguished a registered runner from a stale binding.";
} catch (error) {
if (!["blocked", "env_issue"].includes(result.status)) result.status = "fail";
result.reason = result.reason || error.message;
@@ -24,7 +24,10 @@ function loadEnvDefaults(path) {
if (sep === -1) continue;
const key = line.slice(0, sep).trim();
if (env[key]) continue;
env[key] = line.slice(sep + 1).trim().replace(/^["']|["']$/g, "");
env[key] = line
.slice(sep + 1)
.trim()
.replace(/^["']|["']$/g, "");
}
}
@@ -46,12 +49,16 @@ function redactMessage(text) {
return String(text ?? "")
.replace(/\bbearer\s+[A-Za-z0-9._~+/=-]{8,}/gi, "Bearer [redacted]")
.replace(/\bsk-[A-Za-z0-9_-]{6,}\b/g, "[redacted]")
.replace(/(api[_-]?key|authorization|credential|jwt|oauth|password|secret|token)\s*[:=]\s*["']?[^"',\s]+/gi, "$1=[redacted]");
.replace(
/(api[_-]?key|authorization|credential|jwt|oauth|password|secret|token)\s*[:=]\s*["']?[^"',\s]+/gi,
"$1=[redacted]",
);
}
function isEnvironmentError(message) {
return /Playwright is not installed|LANGBOT_FRONTEND_URL|LANGBOT_BACKEND_URL|ERR_CONNECTION_REFUSED|ECONNREFUSED|net::ERR_|fetch failed|timed out/i
.test(message);
return /Playwright is not installed|LANGBOT_FRONTEND_URL|LANGBOT_BACKEND_URL|ERR_CONNECTION_REFUSED|ECONNREFUSED|net::ERR_|fetch failed|timed out/i.test(
message,
);
}
loadEnvDefaults("skills/.env");
@@ -80,9 +87,15 @@ const targets = [
},
{
id: "acp-agent-runner",
expected_runner_id: "plugin:langbot-team/ACPAgentRunner/default",
pipeline_url: firstEnv("LANGBOT_ACP_AGENT_RUNNER_PIPELINE_URL", "LANGBOT_AGENT_RUNNER_PIPELINE_URL"),
pipeline_name: firstEnv("LANGBOT_ACP_AGENT_RUNNER_PIPELINE_NAME", "LANGBOT_AGENT_RUNNER_PIPELINE_NAME"),
expected_runner_id: "plugin:langbot-team/ACPRunner/default",
pipeline_url: firstEnv(
"LANGBOT_ACP_RUNNER_PIPELINE_URL",
"LANGBOT_RUNNER_PIPELINE_URL",
),
pipeline_name: firstEnv(
"LANGBOT_ACP_RUNNER_PIPELINE_NAME",
"LANGBOT_RUNNER_PIPELINE_NAME",
),
require_func_call_model: false,
require_vision_model: false,
},
@@ -111,20 +124,29 @@ const result = {
automation_result_json: paths.automationResultJson,
result_json: paths.resultJson,
},
evidence_collected: ["ui", "screenshot", "console", "network", "api_diagnostic"],
evidence_collected: [
"ui",
"screenshot",
"console",
"network",
"api_diagnostic",
],
};
async function run() {
if (!backendUrl || !frontendUrl) {
result.status = "env_issue";
result.reason = "LANGBOT_FRONTEND_URL and LANGBOT_BACKEND_URL must be configured.";
result.reason =
"LANGBOT_FRONTEND_URL and LANGBOT_BACKEND_URL must be configured.";
return;
}
browser = await createBrowser(paths);
const { page } = browser;
await page.goto(frontendUrl, { waitUntil: "domcontentloaded" });
await page.waitForLoadState("networkidle", { timeout: 10_000 }).catch(() => {});
await page
.waitForLoadState("networkidle", { timeout: 10_000 })
.catch(() => {});
const workspace = await ensureBrowserWorkspace(page, backendUrl);
if (workspace.status !== "pass") {
result.status = workspace.status;
@@ -132,309 +154,428 @@ async function run() {
return;
}
const diagnostic = await page.evaluate(async ({ backendUrl, targets, testModels }) => {
const blockers = [];
const envIssues = [];
const warnings = [];
const checks = [];
const diagnostic = await page.evaluate(
async ({ backendUrl, targets, testModels }) => {
const blockers = [];
const envIssues = [];
const warnings = [];
const checks = [];
const addCheck = (name, status, detail = {}) => {
checks.push({ name, status, ...detail });
if (status === "blocked") blockers.push({ name, ...detail });
if (status === "env_issue") envIssues.push({ name, ...detail });
};
const safeMessage = (value) => String(value ?? "")
.replace(/\bbearer\s+[A-Za-z0-9._~+/=-]{8,}/gi, "Bearer [redacted]")
.replace(/\bsk-[A-Za-z0-9_-]{6,}\b/g, "[redacted]")
.replace(/(api[_-]?key|authorization|credential|jwt|oauth|password|secret|token)\s*[:=]\s*["']?[^"',\s]+/gi, "$1=[redacted]");
const token = localStorage.getItem("token");
if (!token) {
addCheck("browser-auth", "blocked", { reason: "Browser profile has no localStorage token." });
return { authenticated: false, blockers, env_issues: envIssues, warnings, checks };
}
const headers = {
Authorization: `Bearer ${token}`,
"Content-Type": "application/json",
"X-Workspace-Id": localStorage.getItem("langbot_active_workspace_uuid") || "",
};
const getJson = async (path) => {
const response = await fetch(`${backendUrl}${path}`, { headers });
return {
status: response.status,
json: await response.json().catch(() => ({})),
const addCheck = (name, status, detail = {}) => {
checks.push({ name, status, ...detail });
if (status === "blocked") blockers.push({ name, ...detail });
if (status === "env_issue") envIssues.push({ name, ...detail });
};
};
const postJson = async (path, body) => {
const response = await fetch(`${backendUrl}${path}`, {
method: "POST",
headers,
body: JSON.stringify(body),
});
return {
status: response.status,
json: await response.json().catch(() => ({})),
};
};
const safeMessage = (value) =>
String(value ?? "")
.replace(/\bbearer\s+[A-Za-z0-9._~+/=-]{8,}/gi, "Bearer [redacted]")
.replace(/\bsk-[A-Za-z0-9_-]{6,}\b/g, "[redacted]")
.replace(
/(api[_-]?key|authorization|credential|jwt|oauth|password|secret|token)\s*[:=]\s*["']?[^"',\s]+/gi,
"$1=[redacted]",
);
const tokenCheck = await getJson("/api/v1/user/check-token");
addCheck(
"browser-auth",
tokenCheck.status < 400 && (tokenCheck.json.code ?? 0) === 0 ? "pass" : "blocked",
{ http_status: tokenCheck.status, code: tokenCheck.json.code ?? null, reason: safeMessage(tokenCheck.json.msg || "") },
);
const systemInfo = await getJson("/api/v1/system/info");
addCheck(
"backend-system-info",
systemInfo.status < 400 ? "pass" : "env_issue",
{
http_status: systemInfo.status,
version: systemInfo.json.data?.version || systemInfo.json.data?.system?.version || "",
},
);
const pluginSystem = await getJson("/api/v1/system/status/plugin-system");
addCheck(
"plugin-system",
pluginSystem.status < 400 && (pluginSystem.json.code ?? 0) === 0 ? "pass" : "env_issue",
{
http_status: pluginSystem.status,
code: pluginSystem.json.code ?? null,
status: pluginSystem.json.data?.status || pluginSystem.json.data?.state || "",
reason: safeMessage(pluginSystem.json.msg || ""),
},
);
const boxStatus = await getJson("/api/v1/box/status");
addCheck(
"box-runtime",
boxStatus.status < 400 && (boxStatus.json.code ?? 0) === 0 ? "pass" : "env_issue",
{
http_status: boxStatus.status,
code: boxStatus.json.code ?? null,
status: boxStatus.json.data?.status || "",
backend: boxStatus.json.data?.backend || "",
reason: safeMessage(boxStatus.json.msg || ""),
},
);
const plugins = await getJson("/api/v1/plugins");
const installedPluginIds = (plugins.json.data?.plugins || [])
.map((plugin) => {
const metadata = plugin.manifest?.manifest?.metadata || plugin.manifest?.metadata || plugin.metadata || {};
return metadata.author && metadata.name ? `${metadata.author}/${metadata.name}` : "";
})
.filter(Boolean);
const requiredPlugins = ["langbot-team/LocalAgent", "langbot-team/ACPAgentRunner", "qa/plugin-smoke"];
const pluginPresence = Object.fromEntries(requiredPlugins.map((id) => [id, installedPluginIds.includes(id)]));
for (const [id, present] of Object.entries(pluginPresence)) {
addCheck(`plugin:${id}`, present ? "pass" : "blocked", { plugin_id: id, reason: present ? "" : "Required plugin is not listed by /api/v1/plugins." });
}
const tools = await getJson("/api/v1/tools");
const toolNames = (tools.json.data?.tools || [])
.map((tool) => tool.name || tool.tool_name || tool.function?.name || "")
.filter(Boolean)
.sort();
addCheck(
"tool:qa_plugin_echo",
toolNames.includes("qa_plugin_echo") ? "pass" : "blocked",
{ reason: toolNames.includes("qa_plugin_echo") ? "" : "qa-plugin-smoke tool qa_plugin_echo is not exposed through /api/v1/tools." },
);
if (!toolNames.includes("qa_mcp_echo")) {
warnings.push({
name: "tool:qa_mcp_echo",
reason: "qa_mcp_echo is not currently exposed. This is acceptable before mcp-stdio-register, but mcp-stdio-tool-call must run after registration.",
});
}
const modelResponse = await getJson("/api/v1/provider/models/llm");
const models = (modelResponse.json.data?.models || []).map((model) => ({
uuid: model.uuid,
name: model.name,
abilities: Array.isArray(model.abilities) ? model.abilities : [],
provider_uuid: model.provider_uuid || model.provider?.uuid || "",
provider_name: model.provider_name || model.provider?.name || "",
requester: model.requester || model.provider?.requester || "",
}));
addCheck(
"llm-model-list",
modelResponse.status < 400 && (modelResponse.json.code ?? 0) === 0 ? "pass" : "env_issue",
{ http_status: modelResponse.status, model_count: models.length, reason: safeMessage(modelResponse.json.msg || "") },
);
const modelById = new Map(models.map((model) => [model.uuid, model]));
const pipelineList = await getJson("/api/v1/pipelines");
const pipelines = pipelineList.json.data?.pipelines || [];
addCheck(
"pipeline-list",
pipelineList.status < 400 && (pipelineList.json.code ?? 0) === 0 ? "pass" : "blocked",
{ http_status: pipelineList.status, pipeline_count: pipelines.length, reason: safeMessage(pipelineList.json.msg || "") },
);
const resolvedPipelines = [];
const modelTested = new Set();
for (const target of targets) {
let pipelineId = "";
let matchedBy = "";
if (target.pipeline_url) {
try {
pipelineId = new URL(target.pipeline_url).searchParams.get("id") || "";
matchedBy = pipelineId ? "url" : "";
} catch {
pipelineId = "";
}
}
if (!pipelineId && target.pipeline_name) {
const match = pipelines.find((pipeline) => pipeline.name === target.pipeline_name);
if (match) {
pipelineId = match.uuid;
matchedBy = "name";
}
}
if (!pipelineId) {
addCheck(`pipeline:${target.id}`, "blocked", {
target: target.id,
reason: "Required pipeline env is missing or could not resolve to a pipeline id.",
const token = localStorage.getItem("token");
if (!token) {
addCheck("browser-auth", "blocked", {
reason: "Browser profile has no localStorage token.",
});
continue;
return {
authenticated: false,
blockers,
env_issues: envIssues,
warnings,
checks,
};
}
const response = await getJson(`/api/v1/pipelines/${encodeURIComponent(pipelineId)}`);
const pipeline = response.json.data?.pipeline;
if (response.status >= 400 || !pipeline) {
addCheck(`pipeline:${target.id}`, "blocked", {
target: target.id,
pipeline_id: pipelineId,
http_status: response.status,
reason: safeMessage(response.json.msg || "Could not load pipeline."),
});
continue;
}
const config = pipeline.config || {};
const aiConfig = config.ai && typeof config.ai === "object" ? config.ai : {};
const runner = aiConfig.runner && typeof aiConfig.runner === "object" ? aiConfig.runner : {};
const runnerId = runner.id || "";
const runnerConfigs = aiConfig.runner_config && typeof aiConfig.runner_config === "object" ? aiConfig.runner_config : {};
const runnerConfig = runnerConfigs[runnerId] && typeof runnerConfigs[runnerId] === "object" ? runnerConfigs[runnerId] : {};
const pipelineSummary = {
target: target.id,
pipeline_id: pipelineId,
pipeline_name: pipeline.name,
matched_by: matchedBy,
runner_id: runnerId,
expected_runner_id: target.expected_runner_id,
runner_config_keys: Object.keys(runnerConfig).sort(),
const headers = {
Authorization: `Bearer ${token}`,
"Content-Type": "application/json",
"X-Workspace-Id":
localStorage.getItem("langbot_active_workspace_uuid") || "",
};
const getJson = async (path) => {
const response = await fetch(`${backendUrl}${path}`, { headers });
return {
status: response.status,
json: await response.json().catch(() => ({})),
};
};
const postJson = async (path, body) => {
const response = await fetch(`${backendUrl}${path}`, {
method: "POST",
headers,
body: JSON.stringify(body),
});
return {
status: response.status,
json: await response.json().catch(() => ({})),
};
};
resolvedPipelines.push(pipelineSummary);
const tokenCheck = await getJson("/api/v1/user/check-token");
addCheck(
`pipeline:${target.id}:runner`,
runnerId === target.expected_runner_id ? "pass" : "blocked",
"browser-auth",
tokenCheck.status < 400 && (tokenCheck.json.code ?? 0) === 0
? "pass"
: "blocked",
{
...pipelineSummary,
reason: runnerId === target.expected_runner_id ? "" : `Expected ${target.expected_runner_id}, got ${runnerId || "<missing>"}.`,
http_status: tokenCheck.status,
code: tokenCheck.json.code ?? null,
reason: safeMessage(tokenCheck.json.msg || ""),
},
);
if (target.require_func_call_model || target.require_vision_model || (testModels && target.id === "local-agent")) {
const modelConfig = runnerConfig.model;
const primaryModelId = typeof modelConfig === "string"
? modelConfig
: modelConfig && typeof modelConfig === "object"
? modelConfig.primary || ""
const systemInfo = await getJson("/api/v1/system/info");
addCheck(
"backend-system-info",
systemInfo.status < 400 ? "pass" : "env_issue",
{
http_status: systemInfo.status,
version:
systemInfo.json.data?.version ||
systemInfo.json.data?.system?.version ||
"",
},
);
const pluginSystem = await getJson("/api/v1/system/status/plugin-system");
addCheck(
"plugin-system",
pluginSystem.status < 400 && (pluginSystem.json.code ?? 0) === 0
? "pass"
: "env_issue",
{
http_status: pluginSystem.status,
code: pluginSystem.json.code ?? null,
status:
pluginSystem.json.data?.status ||
pluginSystem.json.data?.state ||
"",
reason: safeMessage(pluginSystem.json.msg || ""),
},
);
const boxStatus = await getJson("/api/v1/box/status");
addCheck(
"box-runtime",
boxStatus.status < 400 && (boxStatus.json.code ?? 0) === 0
? "pass"
: "env_issue",
{
http_status: boxStatus.status,
code: boxStatus.json.code ?? null,
status: boxStatus.json.data?.status || "",
backend: boxStatus.json.data?.backend || "",
reason: safeMessage(boxStatus.json.msg || ""),
},
);
const plugins = await getJson("/api/v1/plugins");
const installedPluginIds = (plugins.json.data?.plugins || [])
.map((plugin) => {
const metadata =
plugin.manifest?.manifest?.metadata ||
plugin.manifest?.metadata ||
plugin.metadata ||
{};
return metadata.author && metadata.name
? `${metadata.author}/${metadata.name}`
: "";
if (!primaryModelId) {
addCheck(`pipeline:${target.id}:primary-model`, "blocked", {
...pipelineSummary,
reason: "Local-agent runner config has no primary model.",
});
continue;
}
const model = modelById.get(primaryModelId);
if (!model) {
addCheck(`pipeline:${target.id}:primary-model`, "blocked", {
...pipelineSummary,
model_uuid: primaryModelId,
reason: "Primary model is not listed by /api/v1/provider/models/llm.",
});
continue;
}
addCheck(`pipeline:${target.id}:primary-model`, "pass", {
...pipelineSummary,
model: {
uuid: model.uuid,
name: model.name,
abilities: model.abilities,
provider_name: model.provider_name,
requester: model.requester,
},
})
.filter(Boolean);
const requiredPlugins = [
"langbot-team/LocalAgent",
"langbot-team/ACPRunner",
"qa/plugin-smoke",
];
const pluginPresence = Object.fromEntries(
requiredPlugins.map((id) => [id, installedPluginIds.includes(id)]),
);
for (const [id, present] of Object.entries(pluginPresence)) {
addCheck(`plugin:${id}`, present ? "pass" : "blocked", {
plugin_id: id,
reason: present
? ""
: "Required plugin is not listed by /api/v1/plugins.",
});
if (target.require_func_call_model) {
addCheck(
`pipeline:${target.id}:func-call-model`,
model.abilities.includes("func_call") ? "pass" : "env_issue",
{
model_uuid: model.uuid,
model_name: model.name,
abilities: model.abilities,
reason: model.abilities.includes("func_call") ? "" : "Release gate includes tool-call cases; the local-agent primary model must advertise func_call.",
},
);
}
const tools = await getJson("/api/v1/tools");
const toolNames = (tools.json.data?.tools || [])
.map((tool) => tool.name || tool.tool_name || tool.function?.name || "")
.filter(Boolean)
.sort();
addCheck(
"tool:qa_plugin_echo",
toolNames.includes("qa_plugin_echo") ? "pass" : "blocked",
{
reason: toolNames.includes("qa_plugin_echo")
? ""
: "qa-plugin-smoke tool qa_plugin_echo is not exposed through /api/v1/tools.",
},
);
if (!toolNames.includes("qa_mcp_echo")) {
warnings.push({
name: "tool:qa_mcp_echo",
reason:
"qa_mcp_echo is not currently exposed. This is acceptable before mcp-stdio-register, but mcp-stdio-tool-call must run after registration.",
});
}
const modelResponse = await getJson("/api/v1/provider/models/llm");
const models = (modelResponse.json.data?.models || []).map((model) => ({
uuid: model.uuid,
name: model.name,
abilities: Array.isArray(model.abilities) ? model.abilities : [],
provider_uuid: model.provider_uuid || model.provider?.uuid || "",
provider_name: model.provider_name || model.provider?.name || "",
requester: model.requester || model.provider?.requester || "",
}));
addCheck(
"llm-model-list",
modelResponse.status < 400 && (modelResponse.json.code ?? 0) === 0
? "pass"
: "env_issue",
{
http_status: modelResponse.status,
model_count: models.length,
reason: safeMessage(modelResponse.json.msg || ""),
},
);
const modelById = new Map(models.map((model) => [model.uuid, model]));
const pipelineList = await getJson("/api/v1/pipelines");
const pipelines = pipelineList.json.data?.pipelines || [];
addCheck(
"pipeline-list",
pipelineList.status < 400 && (pipelineList.json.code ?? 0) === 0
? "pass"
: "blocked",
{
http_status: pipelineList.status,
pipeline_count: pipelines.length,
reason: safeMessage(pipelineList.json.msg || ""),
},
);
const resolvedPipelines = [];
const modelTested = new Set();
for (const target of targets) {
let pipelineId = "";
let matchedBy = "";
if (target.pipeline_url) {
try {
pipelineId =
new URL(target.pipeline_url).searchParams.get("id") || "";
matchedBy = pipelineId ? "url" : "";
} catch {
pipelineId = "";
}
}
if (target.require_vision_model) {
addCheck(
`pipeline:${target.id}:vision-model`,
model.abilities.includes("vision") ? "pass" : "env_issue",
{
model_uuid: model.uuid,
model_name: model.name,
abilities: model.abilities,
reason: model.abilities.includes("vision") ? "" : "Release gate includes multimodal cases; the local-agent primary model must advertise vision.",
},
if (!pipelineId && target.pipeline_name) {
const match = pipelines.find(
(pipeline) => pipeline.name === target.pipeline_name,
);
if (match) {
pipelineId = match.uuid;
matchedBy = "name";
}
}
if (testModels && !modelTested.has(model.uuid)) {
modelTested.add(model.uuid);
const modelTest = await postJson(`/api/v1/provider/models/llm/${encodeURIComponent(model.uuid)}/test`, { extra_args: {} });
const passed = modelTest.status < 400 && (modelTest.json.code ?? 0) === 0;
addCheck(
`model-test:${model.name}`,
passed ? "pass" : "env_issue",
{
model_uuid: model.uuid,
model_name: model.name,
http_status: modelTest.status,
code: modelTest.json.code ?? null,
reason: passed ? "" : safeMessage(modelTest.json.msg || modelTest.json.message || "Model test failed."),
if (!pipelineId) {
addCheck(`pipeline:${target.id}`, "blocked", {
target: target.id,
reason:
"Required pipeline env is missing or could not resolve to a pipeline id.",
});
continue;
}
const response = await getJson(
`/api/v1/pipelines/${encodeURIComponent(pipelineId)}`,
);
const pipeline = response.json.data?.pipeline;
if (response.status >= 400 || !pipeline) {
addCheck(`pipeline:${target.id}`, "blocked", {
target: target.id,
pipeline_id: pipelineId,
http_status: response.status,
reason: safeMessage(
response.json.msg || "Could not load pipeline.",
),
});
continue;
}
const config = pipeline.config || {};
const aiConfig =
config.ai && typeof config.ai === "object" ? config.ai : {};
const runner =
aiConfig.runner && typeof aiConfig.runner === "object"
? aiConfig.runner
: {};
const runnerId = runner.id || "";
const runnerConfigs =
aiConfig.runner_config && typeof aiConfig.runner_config === "object"
? aiConfig.runner_config
: {};
const runnerConfig =
runnerConfigs[runnerId] && typeof runnerConfigs[runnerId] === "object"
? runnerConfigs[runnerId]
: {};
const pipelineSummary = {
target: target.id,
pipeline_id: pipelineId,
pipeline_name: pipeline.name,
matched_by: matchedBy,
runner_id: runnerId,
expected_runner_id: target.expected_runner_id,
runner_config_keys: Object.keys(runnerConfig).sort(),
};
resolvedPipelines.push(pipelineSummary);
addCheck(
`pipeline:${target.id}:runner`,
runnerId === target.expected_runner_id ? "pass" : "blocked",
{
...pipelineSummary,
reason:
runnerId === target.expected_runner_id
? ""
: `Expected ${target.expected_runner_id}, got ${runnerId || "<missing>"}.`,
},
);
if (
target.require_func_call_model ||
target.require_vision_model ||
(testModels && target.id === "local-agent")
) {
const modelConfig = runnerConfig.model;
const primaryModelId =
typeof modelConfig === "string"
? modelConfig
: modelConfig && typeof modelConfig === "object"
? modelConfig.primary || ""
: "";
if (!primaryModelId) {
addCheck(`pipeline:${target.id}:primary-model`, "blocked", {
...pipelineSummary,
reason: "Local-agent runner config has no primary model.",
});
continue;
}
const model = modelById.get(primaryModelId);
if (!model) {
addCheck(`pipeline:${target.id}:primary-model`, "blocked", {
...pipelineSummary,
model_uuid: primaryModelId,
reason:
"Primary model is not listed by /api/v1/provider/models/llm.",
});
continue;
}
addCheck(`pipeline:${target.id}:primary-model`, "pass", {
...pipelineSummary,
model: {
uuid: model.uuid,
name: model.name,
abilities: model.abilities,
provider_name: model.provider_name,
requester: model.requester,
},
);
});
if (target.require_func_call_model) {
addCheck(
`pipeline:${target.id}:func-call-model`,
model.abilities.includes("func_call") ? "pass" : "env_issue",
{
model_uuid: model.uuid,
model_name: model.name,
abilities: model.abilities,
reason: model.abilities.includes("func_call")
? ""
: "Release gate includes tool-call cases; the local-agent primary model must advertise func_call.",
},
);
}
if (target.require_vision_model) {
addCheck(
`pipeline:${target.id}:vision-model`,
model.abilities.includes("vision") ? "pass" : "env_issue",
{
model_uuid: model.uuid,
model_name: model.name,
abilities: model.abilities,
reason: model.abilities.includes("vision")
? ""
: "Release gate includes multimodal cases; the local-agent primary model must advertise vision.",
},
);
}
if (testModels && !modelTested.has(model.uuid)) {
modelTested.add(model.uuid);
const modelTest = await postJson(
`/api/v1/provider/models/llm/${encodeURIComponent(model.uuid)}/test`,
{ extra_args: {} },
);
const passed =
modelTest.status < 400 && (modelTest.json.code ?? 0) === 0;
addCheck(
`model-test:${model.name}`,
passed ? "pass" : "env_issue",
{
model_uuid: model.uuid,
model_name: model.name,
http_status: modelTest.status,
code: modelTest.json.code ?? null,
reason: passed
? ""
: safeMessage(
modelTest.json.msg ||
modelTest.json.message ||
"Model test failed.",
),
},
);
}
}
}
}
return {
authenticated: true,
blockers,
env_issues: envIssues,
warnings,
checks,
resolved_pipelines: resolvedPipelines,
tools: {
required: ["qa_plugin_echo"],
optional_before_register: ["qa_mcp_echo"],
present: toolNames.filter((name) => ["qa_plugin_echo", "qa_mcp_echo"].includes(name)),
},
models,
};
}, { backendUrl, targets, testModels });
return {
authenticated: true,
blockers,
env_issues: envIssues,
warnings,
checks,
resolved_pipelines: resolvedPipelines,
tools: {
required: ["qa_plugin_echo"],
optional_before_register: ["qa_mcp_echo"],
present: toolNames.filter((name) =>
["qa_plugin_echo", "qa_mcp_echo"].includes(name),
),
},
models,
};
},
{ backendUrl, targets, testModels },
);
diagnostic.blockers = (diagnostic.blockers || []).map((item) => ({ ...item, reason: redactMessage(item.reason || "") }));
diagnostic.env_issues = (diagnostic.env_issues || []).map((item) => ({ ...item, reason: redactMessage(item.reason || "") }));
await writeFile(diagnosticPath, `${JSON.stringify(diagnostic, null, 2)}\n`, "utf8");
diagnostic.blockers = (diagnostic.blockers || []).map((item) => ({
...item,
reason: redactMessage(item.reason || ""),
}));
diagnostic.env_issues = (diagnostic.env_issues || []).map((item) => ({
...item,
reason: redactMessage(item.reason || ""),
}));
await writeFile(
diagnosticPath,
`${JSON.stringify(diagnostic, null, 2)}\n`,
"utf8",
);
await safeScreenshot(page, paths.screenshot);
const blockers = diagnostic.blockers || [];
@@ -447,31 +588,49 @@ async function run() {
result.reason = `Preflight environment issue: ${envIssues.map((item) => item.name).join(", ")}`;
} else {
result.status = "pass";
result.reason = "Release gate preflight passed: auth, plugin runtime, required pipelines, runner ids, tools, and local-agent model checks are ready.";
result.reason =
"Release gate preflight passed: auth, plugin runtime, required pipelines, runner ids, tools, and local-agent model checks are ready.";
}
result.check_count = Array.isArray(diagnostic.checks) ? diagnostic.checks.length : 0;
result.warning_count = Array.isArray(diagnostic.warnings) ? diagnostic.warnings.length : 0;
result.check_count = Array.isArray(diagnostic.checks)
? diagnostic.checks.length
: 0;
result.warning_count = Array.isArray(diagnostic.warnings)
? diagnostic.warnings.length
: 0;
}
try {
await run();
} catch (error) {
const message = redactMessage(error instanceof Error ? error.message : String(error));
const message = redactMessage(
error instanceof Error ? error.message : String(error),
);
result.status = isEnvironmentError(message) ? "env_issue" : "fail";
result.reason = message;
await writeFile(diagnosticPath, `${JSON.stringify({
authenticated: false,
blockers: [],
env_issues: result.status === "env_issue" ? [{ name: "preflight-runtime", reason: message }] : [],
warnings: [],
checks: [
await writeFile(
diagnosticPath,
`${JSON.stringify(
{
name: "preflight-runtime",
status: result.status,
reason: message,
authenticated: false,
blockers: [],
env_issues:
result.status === "env_issue"
? [{ name: "preflight-runtime", reason: message }]
: [],
warnings: [],
checks: [
{
name: "preflight-runtime",
status: result.status,
reason: message,
},
],
},
],
}, null, 2)}\n`, "utf8").catch(() => {});
null,
2,
)}\n`,
"utf8",
).catch(() => {});
} finally {
if (browser) await browser.close().catch(() => {});
const finishedAt = new Date();
+14 -15
View File
@@ -1,4 +1,4 @@
"""Policy helpers for classifying AgentRunner ledger error signals."""
"""Policy helpers for classifying Runner ledger error signals."""
from __future__ import annotations
@@ -7,7 +7,7 @@ import re
_INVALID_TOOL_ARGUMENT_PATTERN = re.compile(
r"invalid json arguments|\b\d+\s+validation errors?\s+for\s+[A-Za-z_][A-Za-z0-9_]*Args\b",
r'invalid json arguments|\b\d+\s+validation errors?\s+for\s+[A-Za-z_][A-Za-z0-9_]*Args\b',
re.IGNORECASE,
)
@@ -19,14 +19,14 @@ def load_ledger_json(value: str | None, *, field: str, failures: list[dict]) ->
try:
return json.loads(value)
except (TypeError, ValueError) as exc:
failures.append({"kind": "invalid_json", "field": field, "reason": str(exc)})
failures.append({'kind': 'invalid_json', 'field': field, 'reason': str(exc)})
return {}
def invalid_tool_argument_error_signal(value: str) -> str:
"""Return the persisted signal for malformed model-supplied tool arguments."""
match = _INVALID_TOOL_ARGUMENT_PATTERN.search(value)
return match.group(0) if match else ""
return match.group(0) if match else ''
def classify_invalid_tool_argument_errors(
@@ -40,15 +40,14 @@ def classify_invalid_tool_argument_errors(
warnings: list[dict] = []
for event in events:
recovered = run_completed and any(
sequence > event["sequence"]
for sequence in successful_tool_completion_sequences
sequence > event['sequence'] for sequence in successful_tool_completion_sequences
)
if recovered:
warnings.append(
{
"kind": "recovered_tool_argument_error",
"event": event,
"reason": "The model continued with a later successful tool call and the run completed.",
'kind': 'recovered_tool_argument_error',
'event': event,
'reason': 'The model continued with a later successful tool call and the run completed.',
}
)
else:
@@ -64,15 +63,15 @@ def classify_tool_authorization(
"""Classify tool names absent from the Host authorization snapshot."""
if not calls:
return [], []
if authorization_mode == "runner-native":
if authorization_mode == 'runner-native':
return [], [
{
"kind": "runner_native_tool_calls",
"calls": calls,
"reason": (
"External runner tool telemetry is not a LangBot Host tool call; "
'kind': 'runner_native_tool_calls',
'calls': calls,
'reason': (
'External runner tool telemetry is not a LangBot Host tool call; '
"the runner's own permission system governs it."
),
}
]
return [{"kind": "unauthorized_tool_calls", "calls": calls}], []
return [{'kind': 'unauthorized_tool_calls', 'calls': calls}], []
@@ -12,7 +12,7 @@ import {
writeResult,
} from "./lib/langbot-e2e.mjs";
const RUNNER_ID = "plugin:langbot-team/ACPAgentRunner/default";
const RUNNER_ID = "plugin:langbot-team/ACPRunner/default";
const DEFAULT_PIPELINE_NAME = "Agent QA ACP Claude Debug Chat";
const DEFAULT_LOCAL_PASSWORD = "LangBotE2ELocalPass!2026";
const caseId = "ensure-acp-agent-runner-pipeline";
@@ -24,13 +24,18 @@ await ensureEvidence(paths);
const writeEnv = process.argv.includes("--write-env");
const frontendUrl = env.LANGBOT_FRONTEND_URL || "";
const backendUrl = env.LANGBOT_BACKEND_URL || "";
const pipelineName = env.LANGBOT_E2E_CREATE_PIPELINE_NAME || env.LANGBOT_ACP_AGENT_RUNNER_PIPELINE_NAME || DEFAULT_PIPELINE_NAME;
const sshTarget = env.LANGBOT_ACP_AGENT_RUNNER_SSH_TARGET || "yhh@101.34.71.12";
const sshConnectTimeout = env.LANGBOT_ACP_AGENT_RUNNER_SSH_CONNECT_TIMEOUT || "8";
const sshPort = env.LANGBOT_ACP_AGENT_RUNNER_SSH_PORT || "22";
const sshIdentityFile = env.LANGBOT_ACP_AGENT_RUNNER_SSH_IDENTITY_FILE || "";
const sshExtraOptions = env.LANGBOT_ACP_AGENT_RUNNER_SSH_EXTRA_OPTIONS || "";
const remoteWorkspace = env.LANGBOT_ACP_AGENT_RUNNER_REMOTE_WORKSPACE || "/home/yhh/langbot-e2e/acp-workspace";
const pipelineName =
env.LANGBOT_E2E_CREATE_PIPELINE_NAME ||
env.LANGBOT_ACP_RUNNER_PIPELINE_NAME ||
DEFAULT_PIPELINE_NAME;
const sshTarget = env.LANGBOT_ACP_RUNNER_SSH_TARGET || "yhh@101.34.71.12";
const sshConnectTimeout = env.LANGBOT_ACP_RUNNER_SSH_CONNECT_TIMEOUT || "8";
const sshPort = env.LANGBOT_ACP_RUNNER_SSH_PORT || "22";
const sshIdentityFile = env.LANGBOT_ACP_RUNNER_SSH_IDENTITY_FILE || "";
const sshExtraOptions = env.LANGBOT_ACP_RUNNER_SSH_EXTRA_OPTIONS || "";
const remoteWorkspace =
env.LANGBOT_ACP_RUNNER_REMOTE_WORKSPACE ||
"/home/yhh/langbot-e2e/acp-workspace";
const envLocalPath = resolve("skills/.env.local");
const result = {
@@ -64,7 +69,9 @@ try {
const user = env.LANGBOT_E2E_LOGIN_USER || "";
const password = env.LANGBOT_E2E_LOGIN_PASSWORD || DEFAULT_LOCAL_PASSWORD;
if (!user) {
throw new Error("LANGBOT_E2E_LOGIN_USER is required so this setup can create/update the pipeline via backend API.");
throw new Error(
"LANGBOT_E2E_LOGIN_USER is required so this setup can create/update the pipeline via backend API.",
);
}
const auth = await resetAndAuthLocalUser({ backendUrl, user, password });
@@ -116,13 +123,13 @@ try {
if (writeEnv && result.pipeline_id) {
await upsertEnvLocal(envLocalPath, {
LANGBOT_E2E_LOGIN_USER: user,
LANGBOT_ACP_AGENT_RUNNER_SSH_TARGET: sshTarget,
LANGBOT_ACP_AGENT_RUNNER_SSH_PORT: sshPort,
LANGBOT_ACP_AGENT_RUNNER_SSH_IDENTITY_FILE: sshIdentityFile,
LANGBOT_ACP_AGENT_RUNNER_SSH_EXTRA_OPTIONS: sshExtraOptions,
LANGBOT_ACP_AGENT_RUNNER_REMOTE_WORKSPACE: remoteWorkspace,
LANGBOT_ACP_AGENT_RUNNER_PIPELINE_URL: result.pipeline_url,
LANGBOT_ACP_AGENT_RUNNER_PIPELINE_NAME: result.pipeline_name || pipelineName,
LANGBOT_ACP_RUNNER_SSH_TARGET: sshTarget,
LANGBOT_ACP_RUNNER_SSH_PORT: sshPort,
LANGBOT_ACP_RUNNER_SSH_IDENTITY_FILE: sshIdentityFile,
LANGBOT_ACP_RUNNER_SSH_EXTRA_OPTIONS: sshExtraOptions,
LANGBOT_ACP_RUNNER_REMOTE_WORKSPACE: remoteWorkspace,
LANGBOT_ACP_RUNNER_PIPELINE_URL: result.pipeline_url,
LANGBOT_ACP_RUNNER_PIPELINE_NAME: result.pipeline_name || pipelineName,
});
result.wrote_env = true;
}
@@ -133,10 +140,20 @@ try {
console.log(JSON.stringify(result, null, 2));
}
process.exit(result.status === "pass" ? 0 : result.status === "env_issue" ? 2 : 1);
process.exit(
result.status === "pass" ? 0 : result.status === "env_issue" ? 2 : 1,
);
async function ensurePipeline({ backendUrl, token, pipelineName, runnerId, runnerConfig }) {
const pipelineList = await apiJson(backendUrl, "/api/v1/pipelines", { token });
async function ensurePipeline({
backendUrl,
token,
pipelineName,
runnerId,
runnerConfig,
}) {
const pipelineList = await apiJson(backendUrl, "/api/v1/pipelines", {
token,
});
if (isApiFailure(pipelineList)) {
return {
status: "fail",
@@ -155,7 +172,8 @@ async function ensurePipeline({ backendUrl, token, pipelineName, runnerId, runne
token,
body: {
name: pipelineName,
description: "Local QA pipeline for real ACP Claude AgentRunner Debug Chat smoke tests.",
description:
"Local QA pipeline for real ACP Claude Runner Debug Chat smoke tests.",
emoji: "QA",
},
});
@@ -167,7 +185,11 @@ async function ensurePipeline({ backendUrl, token, pipelineName, runnerId, runne
};
}
const pipelineId = createdResponse.json.data?.uuid || "";
const loaded = await apiJson(backendUrl, `/api/v1/pipelines/${encodeURIComponent(pipelineId)}`, { token });
const loaded = await apiJson(
backendUrl,
`/api/v1/pipelines/${encodeURIComponent(pipelineId)}`,
{ token },
);
pipeline = loaded.json.data?.pipeline || null;
created = true;
}
@@ -179,7 +201,11 @@ async function ensurePipeline({ backendUrl, token, pipelineName, runnerId, runne
};
}
const loaded = await apiJson(backendUrl, `/api/v1/pipelines/${encodeURIComponent(pipeline.uuid)}`, { token });
const loaded = await apiJson(
backendUrl,
`/api/v1/pipelines/${encodeURIComponent(pipeline.uuid)}`,
{ token },
);
if (isApiFailure(loaded) || !loaded.json.data?.pipeline) {
return {
status: "fail",
@@ -190,9 +216,15 @@ async function ensurePipeline({ backendUrl, token, pipelineName, runnerId, runne
}
pipeline = loaded.json.data.pipeline;
const config = pipeline.config && typeof pipeline.config === "object" ? pipeline.config : {};
const config =
pipeline.config && typeof pipeline.config === "object"
? pipeline.config
: {};
const ai = config.ai && typeof config.ai === "object" ? config.ai : {};
const runnerConfigs = ai.runner_config && typeof ai.runner_config === "object" ? ai.runner_config : {};
const runnerConfigs =
ai.runner_config && typeof ai.runner_config === "object"
? ai.runner_config
: {};
const updatedConfig = {
...config,
ai: {
@@ -209,16 +241,21 @@ async function ensurePipeline({ backendUrl, token, pipelineName, runnerId, runne
},
};
const updateResponse = await apiJson(backendUrl, `/api/v1/pipelines/${encodeURIComponent(pipeline.uuid)}`, {
method: "PUT",
token,
body: {
name: pipelineName,
description: "Local QA pipeline for real ACP Claude AgentRunner Debug Chat smoke tests.",
emoji: "QA",
config: updatedConfig,
const updateResponse = await apiJson(
backendUrl,
`/api/v1/pipelines/${encodeURIComponent(pipeline.uuid)}`,
{
method: "PUT",
token,
body: {
name: pipelineName,
description:
"Local QA pipeline for real ACP Claude Runner Debug Chat smoke tests.",
emoji: "QA",
config: updatedConfig,
},
},
});
);
if (isApiFailure(updateResponse)) {
return {
status: "fail",
@@ -230,7 +267,9 @@ async function ensurePipeline({ backendUrl, token, pipelineName, runnerId, runne
return {
status: "pass",
reason: created ? "ACP AgentRunner pipeline created and configured." : "ACP AgentRunner pipeline updated.",
reason: created
? "ACP Runner pipeline created and configured."
: "ACP Runner pipeline updated.",
pipeline_id: pipeline.uuid,
pipeline_name: pipelineName,
created,
@@ -239,7 +278,12 @@ async function ensurePipeline({ backendUrl, token, pipelineName, runnerId, runne
}
function isApiFailure(response) {
return response.status >= 400 || (response.json && response.json.code !== undefined && response.json.code !== 0);
return (
response.status >= 400 ||
(response.json &&
response.json.code !== undefined &&
response.json.code !== 0)
);
}
async function upsertEnvLocal(path, values) {
@@ -102,14 +102,16 @@ try {
backend_token_check: auth.check,
};
const pluginSetup = await ensureLocalAgentRunner({
const pluginSetup = await ensureLocalRunner({
backendUrl,
token: auth.token,
});
result.plugin_setup = pluginSetup;
if (pluginSetup.status !== "pass") {
result.status = pluginSetup.status === "env_issue" ? "env_issue" : "fail";
throw new Error(pluginSetup.reason || "Failed to prepare the LocalAgent runner plugin.");
throw new Error(
pluginSetup.reason || "Failed to prepare the LocalAgent runner plugin.",
);
}
const wizard = await skipWizard({ backendUrl, token: auth.token });
@@ -205,7 +207,7 @@ async function skipWizard({ backendUrl, token }) {
};
}
async function ensureLocalAgentRunner({ backendUrl, token }) {
async function ensureLocalRunner({ backendUrl, token }) {
const [author, name] = RUNNER_ID.replace(/^plugin:/, "").split("/");
const existingRunnerIds = await listRunnerIds(backendUrl, token);
if (existingRunnerIds.includes(RUNNER_ID)) {
@@ -264,10 +266,9 @@ async function ensureLocalAgentRunner({ backendUrl, token }) {
};
}
const spaceUrl = String(env.LANGBOT_SPACE_URL || "https://space.langbot.app").replace(
/\/$/,
"",
);
const spaceUrl = String(
env.LANGBOT_SPACE_URL || "https://space.langbot.app",
).replace(/\/$/, "");
let detailResponse;
try {
detailResponse = await fetch(
@@ -376,7 +377,8 @@ async function waitForRunnerRegistration({
}) {
const deadline = Date.now() + timeoutMs;
while (Date.now() < deadline) {
if ((await listRunnerIds(backendUrl, token)).includes(runnerId)) return true;
if ((await listRunnerIds(backendUrl, token)).includes(runnerId))
return true;
await sleep(1000);
}
return false;
@@ -501,8 +503,7 @@ async function ensureLocalAgentPipeline({
token,
body: {
name: pipelineName,
description:
"Local QA pipeline for AgentRunner Debug Chat smoke tests.",
description: "Local QA pipeline for Runner Debug Chat smoke tests.",
emoji: "QA",
},
});
@@ -640,8 +641,7 @@ async function ensureLocalAgentPipeline({
token,
body: {
name: pipelineName,
description:
"Local QA pipeline for AgentRunner Debug Chat smoke tests.",
description: "Local QA pipeline for Runner Debug Chat smoke tests.",
emoji: "QA",
config: updatedConfig,
},
@@ -24,7 +24,10 @@ await ensureEvidence(paths);
const writeEnv = process.argv.includes("--write-env");
const frontendUrl = env.LANGBOT_FRONTEND_URL || "";
const backendUrl = env.LANGBOT_BACKEND_URL || "";
const pipelineName = env.LANGBOT_E2E_CREATE_PIPELINE_NAME || env.LANGBOT_QA_AGENT_RUNNER_PIPELINE_NAME || DEFAULT_PIPELINE_NAME;
const pipelineName =
env.LANGBOT_E2E_CREATE_PIPELINE_NAME ||
env.LANGBOT_QA_RUNNER_PIPELINE_NAME ||
DEFAULT_PIPELINE_NAME;
const envLocalPath = resolve("skills/.env.local");
const result = {
@@ -55,7 +58,9 @@ try {
const user = env.LANGBOT_E2E_LOGIN_USER || "";
const password = env.LANGBOT_E2E_LOGIN_PASSWORD || DEFAULT_LOCAL_PASSWORD;
if (!user) {
throw new Error("LANGBOT_E2E_LOGIN_USER is required so this setup can create/update the pipeline via backend API.");
throw new Error(
"LANGBOT_E2E_LOGIN_USER is required so this setup can create/update the pipeline via backend API.",
);
}
const auth = await resetAndAuthLocalUser({ backendUrl, user, password });
@@ -80,8 +85,8 @@ try {
if (writeEnv && result.pipeline_id) {
await upsertEnvLocal(envLocalPath, {
LANGBOT_E2E_LOGIN_USER: user,
LANGBOT_QA_AGENT_RUNNER_PIPELINE_URL: result.pipeline_url,
LANGBOT_QA_AGENT_RUNNER_PIPELINE_NAME: result.pipeline_name || pipelineName,
LANGBOT_QA_RUNNER_PIPELINE_URL: result.pipeline_url,
LANGBOT_QA_RUNNER_PIPELINE_NAME: result.pipeline_name || pipelineName,
});
result.wrote_env = true;
}
@@ -92,10 +97,20 @@ try {
console.log(JSON.stringify(result, null, 2));
}
process.exit(result.status === "pass" ? 0 : result.status === "env_issue" ? 2 : 1);
process.exit(
result.status === "pass" ? 0 : result.status === "env_issue" ? 2 : 1,
);
async function ensurePipeline({ backendUrl, token, pipelineName, runnerId, runnerConfig }) {
const pipelineList = await apiJson(backendUrl, "/api/v1/pipelines", { token });
async function ensurePipeline({
backendUrl,
token,
pipelineName,
runnerId,
runnerConfig,
}) {
const pipelineList = await apiJson(backendUrl, "/api/v1/pipelines", {
token,
});
if (isApiFailure(pipelineList)) {
return {
status: "fail",
@@ -114,7 +129,8 @@ async function ensurePipeline({ backendUrl, token, pipelineName, runnerId, runne
token,
body: {
name: pipelineName,
description: "Local QA pipeline for deterministic QA AgentRunner Debug Chat smoke tests.",
description:
"Local QA pipeline for deterministic QA Runner Debug Chat smoke tests.",
emoji: "QA",
},
});
@@ -126,7 +142,11 @@ async function ensurePipeline({ backendUrl, token, pipelineName, runnerId, runne
};
}
const pipelineId = createdResponse.json.data?.uuid || "";
const loaded = await apiJson(backendUrl, `/api/v1/pipelines/${encodeURIComponent(pipelineId)}`, { token });
const loaded = await apiJson(
backendUrl,
`/api/v1/pipelines/${encodeURIComponent(pipelineId)}`,
{ token },
);
pipeline = loaded.json.data?.pipeline || null;
created = true;
}
@@ -138,7 +158,11 @@ async function ensurePipeline({ backendUrl, token, pipelineName, runnerId, runne
};
}
const loaded = await apiJson(backendUrl, `/api/v1/pipelines/${encodeURIComponent(pipeline.uuid)}`, { token });
const loaded = await apiJson(
backendUrl,
`/api/v1/pipelines/${encodeURIComponent(pipeline.uuid)}`,
{ token },
);
if (isApiFailure(loaded) || !loaded.json.data?.pipeline) {
return {
status: "fail",
@@ -149,9 +173,15 @@ async function ensurePipeline({ backendUrl, token, pipelineName, runnerId, runne
}
pipeline = loaded.json.data.pipeline;
const config = pipeline.config && typeof pipeline.config === "object" ? pipeline.config : {};
const config =
pipeline.config && typeof pipeline.config === "object"
? pipeline.config
: {};
const ai = config.ai && typeof config.ai === "object" ? config.ai : {};
const runnerConfigs = ai.runner_config && typeof ai.runner_config === "object" ? ai.runner_config : {};
const runnerConfigs =
ai.runner_config && typeof ai.runner_config === "object"
? ai.runner_config
: {};
const updatedConfig = {
...config,
ai: {
@@ -168,16 +198,21 @@ async function ensurePipeline({ backendUrl, token, pipelineName, runnerId, runne
},
};
const updateResponse = await apiJson(backendUrl, `/api/v1/pipelines/${encodeURIComponent(pipeline.uuid)}`, {
method: "PUT",
token,
body: {
name: pipelineName,
description: "Local QA pipeline for deterministic QA AgentRunner Debug Chat smoke tests.",
emoji: "QA",
config: updatedConfig,
const updateResponse = await apiJson(
backendUrl,
`/api/v1/pipelines/${encodeURIComponent(pipeline.uuid)}`,
{
method: "PUT",
token,
body: {
name: pipelineName,
description:
"Local QA pipeline for deterministic QA Runner Debug Chat smoke tests.",
emoji: "QA",
config: updatedConfig,
},
},
});
);
if (isApiFailure(updateResponse)) {
return {
status: "fail",
@@ -189,7 +224,9 @@ async function ensurePipeline({ backendUrl, token, pipelineName, runnerId, runne
return {
status: "pass",
reason: created ? "QA AgentRunner pipeline created and configured." : "QA AgentRunner pipeline updated.",
reason: created
? "QA Runner pipeline created and configured."
: "QA Runner pipeline updated.",
pipeline_id: pipeline.uuid,
pipeline_name: pipelineName,
created,
@@ -198,7 +235,12 @@ async function ensurePipeline({ backendUrl, token, pipelineName, runnerId, runne
}
function isApiFailure(response) {
return response.status >= 400 || (response.json && response.json.code !== undefined && response.json.code !== 0);
return (
response.status >= 400 ||
(response.json &&
response.json.code !== undefined &&
response.json.code !== 0)
);
}
async function upsertEnvLocal(path, values) {
@@ -20,7 +20,10 @@ await loadEnvFiles();
const paths = evidencePaths(caseId);
await ensureEvidence(paths);
const mobileScreenshot = paths.screenshot.replace(/\.png$/, "-mobile.png");
const installedScreenshot = paths.screenshot.replace(/\.png$/, "-installed.png");
const installedScreenshot = paths.screenshot.replace(
/\.png$/,
"-installed.png",
);
const startedAt = new Date();
let frontendUrl = "";
@@ -84,7 +87,7 @@ try {
}
try {
const payload = request.postDataJSON();
if (payload?.component_filter === "AgentRunner") {
if (payload?.component_filter === "Runner") {
result.marketplace_request = {
endpoint: new URL(request.url()).pathname,
component_filter: payload.component_filter,
@@ -98,14 +101,17 @@ try {
});
page.on("response", async (response) => {
const pathname = new URL(response.url()).pathname;
if (!/\/api\/v1\/marketplace\/(extensions|plugins)\/search$/.test(pathname)) {
if (
!/\/api\/v1\/marketplace\/(extensions|plugins)\/search$/.test(pathname)
) {
return;
}
try {
const payload = await response.json();
const entries = payload?.data?.extensions || payload?.data?.plugins || [];
const localAgent = entries.find(
(entry) => `${entry.author}/${entry.name}` === "langbot-team/LocalAgent",
(entry) =>
`${entry.author}/${entry.name}` === "langbot-team/LocalAgent",
);
result.marketplace_response = {
endpoint: pathname,
@@ -228,7 +234,7 @@ try {
});
await browseLink.waitFor();
const href = await browseLink.getAttribute("href");
if (href !== "/home/extensions?type=plugin&component=AgentRunner") {
if (href !== "/home/extensions?type=plugin&component=Runner") {
throw new Error(`Unexpected Runner marketplace URL: ${href}`);
}
const nextButton = page.getByRole("button", {
@@ -238,9 +244,7 @@ try {
throw new Error("Wizard allowed continuing without an installed Runner.");
}
if (!result.marketplace_request) {
throw new Error(
"Wizard did not request the AgentRunner Marketplace catalog.",
);
throw new Error("Wizard did not request the Runner Marketplace catalog.");
}
if (
!result.marketplace_response?.local_agent_present ||
@@ -287,10 +291,14 @@ try {
.then(() => "failed"),
]);
if (installOutcome === "failed") {
throw new Error("LocalAgent installation failed before Runner registration.");
throw new Error(
"LocalAgent installation failed before Runner registration.",
);
}
if (await nextButton.isDisabled()) {
throw new Error("Create & Deploy remained disabled after LocalAgent installation.");
throw new Error(
"Create & Deploy remained disabled after LocalAgent installation.",
);
}
const [installedPluginsResponse, installedMetadataResponse] =
@@ -298,8 +306,7 @@ try {
apiJson(backendUrl, "/api/v1/plugins", { token }),
apiJson(backendUrl, "/api/v1/pipelines/_/metadata", { token }),
]);
const postInstallPlugins =
installedPluginsResponse.json.data?.plugins || [];
const postInstallPlugins = installedPluginsResponse.json.data?.plugins || [];
const installedRunnerStage = installedMetadataResponse.json.data?.configs
?.find((config) => config.name === "ai")
?.stages?.find((stage) => stage.name === "runner");
@@ -338,7 +345,7 @@ try {
}
result.status = "pass";
result.reason =
"A clean first-run instance discovered LocalAgent in the AgentRunner catalog, installed and registered it, selected it, and enabled Create & Deploy.";
"A clean first-run instance discovered LocalAgent in the Runner catalog, installed and registered it, selected it, and enabled Create & Deploy.";
} catch (error) {
if (!["blocked", "env_issue"].includes(result.status)) result.status = "fail";
result.reason = result.reason || error.message;
@@ -42,26 +42,83 @@ const result = {
};
const repositories = [
{ id: "langbot", directory: "LangBot", envKey: "LANGBOT_REPO", manifest: false },
{ id: "plugin-sdk", directory: "langbot-plugin-sdk", envKey: "LANGBOT_PLUGIN_SDK_REPO", manifest: false },
{ id: "agent-runner", directory: "langbot-agent-runner", envKey: "LANGBOT_AGENT_RUNNER_REPO", manifest: false },
{ id: "local-agent", directory: "langbot-local-agent", envKey: "LANGBOT_LOCAL_AGENT_REPO", identity: "langbot-team/LocalAgent" },
{ id: "control-plane", directory: "langbot-agent-control-plane", envKey: "LANGBOT_AGENT_CONTROL_PLANE_REPO", identity: "langbot/agent-control-plane" },
{ id: "longterm-memory", directory: "langbot-longterm-memory", envKey: "LANGBOT_LONGTERM_MEMORY_REPO", identity: "langbot-team/LongTermMemory" },
{ id: "parser", directory: "langbot-parser", envKey: "LANGBOT_PARSER_PLUGIN_REPO", identity: "langbot-team/GeneralParsers" },
{ id: "rag", directory: "langbot-rag", envKey: "LANGBOT_RAG_PLUGIN_REPO", identity: "langbot-team/LangRAG" },
{ id: "skill-authoring", directory: "langbot-skill-authoring", envKey: "LANGBOT_SKILL_AUTHORING_REPO", identity: "huanghuoguoguo/skill-authoring" },
{
id: "langbot",
directory: "LangBot",
envKey: "LANGBOT_REPO",
manifest: false,
},
{
id: "plugin-sdk",
directory: "langbot-plugin-sdk",
envKey: "LANGBOT_PLUGIN_SDK_REPO",
manifest: false,
},
{
id: "agent-runner",
directory: "langbot-agent-runner",
envKey: "LANGBOT_RUNNER_REPO",
manifest: false,
},
{
id: "local-agent",
directory: "langbot-local-agent",
envKey: "LANGBOT_LOCAL_AGENT_REPO",
identity: "langbot-team/LocalAgent",
},
{
id: "control-plane",
directory: "langbot-agent-control-plane",
envKey: "LANGBOT_AGENT_CONTROL_PLANE_REPO",
identity: "langbot/agent-control-plane",
},
{
id: "longterm-memory",
directory: "langbot-longterm-memory",
envKey: "LANGBOT_LONGTERM_MEMORY_REPO",
identity: "langbot-team/LongTermMemory",
},
{
id: "parser",
directory: "langbot-parser",
envKey: "LANGBOT_PARSER_PLUGIN_REPO",
identity: "langbot-team/GeneralParsers",
},
{
id: "rag",
directory: "langbot-rag",
envKey: "LANGBOT_RAG_PLUGIN_REPO",
identity: "langbot-team/LangRAG",
},
{
id: "skill-authoring",
directory: "langbot-skill-authoring",
envKey: "LANGBOT_SKILL_AUTHORING_REPO",
identity: "huanghuoguoguo/skill-authoring",
},
];
function run(command, args, options = {}) {
return new Promise((resolvePromise) => {
const child = spawn(command, args, { ...options, encoding: "utf8", stdio: ["ignore", "pipe", "pipe"] });
const child = spawn(command, args, {
...options,
encoding: "utf8",
stdio: ["ignore", "pipe", "pipe"],
});
let stdout = "";
let stderr = "";
child.stdout.on("data", (chunk) => { stdout += chunk; });
child.stderr.on("data", (chunk) => { stderr += chunk; });
child.on("error", (error) => resolvePromise({ status: null, stdout, stderr, error }));
child.on("close", (status) => resolvePromise({ status, stdout, stderr, error: null }));
child.stdout.on("data", (chunk) => {
stdout += chunk;
});
child.stderr.on("data", (chunk) => {
stderr += chunk;
});
child.on("error", (error) =>
resolvePromise({ status: null, stdout, stderr, error }),
);
child.on("close", (status) =>
resolvePromise({ status, stdout, stderr, error: null }),
);
});
}
@@ -71,31 +128,50 @@ function addCheck(name, status, detail = {}) {
try {
const langbotRepo = await resolveLangBotRepo();
const workspaceRoot = resolve(env.LANGBOT_WORKSPACE_ROOT || dirname(langbotRepo));
const workspaceRoot = resolve(
env.LANGBOT_WORKSPACE_ROOT || dirname(langbotRepo),
);
const resolved = {};
for (const repository of repositories) {
const path = resolve(env[repository.envKey] || (repository.id === "langbot" ? langbotRepo : join(workspaceRoot, repository.directory)));
const path = resolve(
env[repository.envKey] ||
(repository.id === "langbot"
? langbotRepo
: join(workspaceRoot, repository.directory)),
);
resolved[repository.id] = path;
try {
await access(join(path, ".git"));
} catch {
addCheck(`repo:${repository.id}`, "fail", { path, reason: "Git checkout is missing." });
addCheck(`repo:${repository.id}`, "fail", {
path,
reason: "Git checkout is missing.",
});
continue;
}
const branch = await run("git", ["branch", "--show-current"], { cwd: path });
const branch = await run("git", ["branch", "--show-current"], {
cwd: path,
});
const branchName = branch.stdout.trim();
const compatible = branch.status === 0 && /^(?:main|dev\/4\.11\.x)$/.test(branchName);
const compatible =
branch.status === 0 && /^(?:main|dev\/4\.11\.x)$/.test(branchName);
addCheck(`repo:${repository.id}`, compatible ? "pass" : "fail", {
path,
branch: branchName,
reason: compatible ? "" : "Expected main or dev/4.11.x compatibility branch.",
reason: compatible
? ""
: "Expected main or dev/4.11.x compatibility branch.",
});
const dirty = await run("git", ["status", "--short"], { cwd: path });
if (dirty.stdout.trim()) {
result.warnings.push({ name: `dirty:${repository.id}`, path, entries: dirty.stdout.trim().split(/\r?\n/).length });
result.warnings.push({
name: `dirty:${repository.id}`,
path,
entries: dirty.stdout.trim().split(/\r?\n/).length,
});
}
if (repository.identity) {
@@ -105,13 +181,20 @@ try {
const author = manifest.match(/^\s{2}author:\s*([^\s#]+)/m)?.[1] || "";
const name = manifest.match(/^\s{2}name:\s*([^\s#]+)/m)?.[1] || "";
const identity = `${author}/${name}`;
addCheck(`manifest:${repository.id}`, identity === repository.identity ? "pass" : "fail", {
path: manifestPath,
identity,
expected_identity: repository.identity,
});
addCheck(
`manifest:${repository.id}`,
identity === repository.identity ? "pass" : "fail",
{
path: manifestPath,
identity,
expected_identity: repository.identity,
},
);
} catch (error) {
addCheck(`manifest:${repository.id}`, "fail", { path: manifestPath, reason: error.message });
addCheck(`manifest:${repository.id}`, "fail", {
path: manifestPath,
reason: error.message,
});
}
}
}
@@ -121,41 +204,69 @@ try {
await access(python);
addCheck("langbot-venv", "pass", { python });
} catch {
addCheck("langbot-venv", "fail", { python, reason: "LangBot virtualenv Python is missing." });
addCheck("langbot-venv", "fail", {
python,
reason: "LangBot virtualenv Python is missing.",
});
}
const sdkSrc = join(resolved["plugin-sdk"], "src");
const importProbe = await run(python, ["-c", [
"import json, pathlib, langbot_plugin",
"from langbot_plugin.api.entities.builtin.agent_runner.input import AgentInput",
"from langbot_plugin.api.entities.builtin.agent_runner.result import AgentRunResult",
"print(json.dumps({'path': str(pathlib.Path(langbot_plugin.__file__).resolve()), 'entities': [AgentInput.__name__, AgentRunResult.__name__]}))",
].join("; ")], {
cwd: resolved.langbot,
env: { ...env, PYTHONPATH: [sdkSrc, env.PYTHONPATH].filter(Boolean).join(delimiter) },
});
const importProbe = await run(
python,
[
"-c",
[
"import json, pathlib, langbot_plugin",
"from langbot_plugin.api.entities.builtin.runner.input import AgentInput",
"from langbot_plugin.api.entities.builtin.runner.result import RunnerResult",
"print(json.dumps({'path': str(pathlib.Path(langbot_plugin.__file__).resolve()), 'entities': [AgentInput.__name__, RunnerResult.__name__]}))",
].join("; "),
],
{
cwd: resolved.langbot,
env: {
...env,
PYTHONPATH: [sdkSrc, env.PYTHONPATH].filter(Boolean).join(delimiter),
},
},
);
let importDetail = {};
try { importDetail = JSON.parse(importProbe.stdout.trim()); } catch { importDetail = { stderr: importProbe.stderr.trim() }; }
const localSdkLoaded = importProbe.status === 0 && resolve(importDetail.path || "").startsWith(resolve(sdkSrc));
try {
importDetail = JSON.parse(importProbe.stdout.trim());
} catch {
importDetail = { stderr: importProbe.stderr.trim() };
}
const localSdkLoaded =
importProbe.status === 0 &&
resolve(importDetail.path || "").startsWith(resolve(sdkSrc));
addCheck("local-sdk-import", localSdkLoaded ? "pass" : "fail", {
expected_root: resolve(sdkSrc),
...importDetail,
reason: localSdkLoaded ? "" : "langbot_plugin did not load from the workspace SDK source tree.",
reason: localSdkLoaded
? ""
: "langbot_plugin did not load from the workspace SDK source tree.",
});
const failures = result.checks.filter((check) => check.status === "fail");
result.status = failures.length === 0 ? "pass" : "fail";
result.reason = failures.length === 0
? `Workspace compatibility preflight passed with ${result.warnings.length} non-blocking dirty-worktree warning(s).`
: `Workspace compatibility preflight found ${failures.length} blocking check(s).`;
result.reason =
failures.length === 0
? `Workspace compatibility preflight passed with ${result.warnings.length} non-blocking dirty-worktree warning(s).`
: `Workspace compatibility preflight found ${failures.length} blocking check(s).`;
} catch (error) {
result.status = /missing|ENOENT|not found/i.test(error.message) ? "env_issue" : "fail";
result.status = /missing|ENOENT|not found/i.test(error.message)
? "env_issue"
: "fail";
result.reason = error.message;
} finally {
const finishedAt = new Date();
result.finished_at = finishedAt.toISOString();
result.finished_at_local = localIsoWithOffset(finishedAt);
await writeFile(detailsPath, `${JSON.stringify({ checks: result.checks, warnings: result.warnings }, null, 2)}\n`, "utf8");
await writeFile(
detailsPath,
`${JSON.stringify({ checks: result.checks, warnings: result.warnings }, null, 2)}\n`,
"utf8",
);
await writeResult(paths, result);
console.log(JSON.stringify(result, null, 2));
}
+24 -24
View File
@@ -210,7 +210,7 @@
"case_summaries": [
{
"id": "acp-agent-runner-debug-chat",
"title": "ACP AgentRunner can answer through Debug Chat using real remote Claude",
"title": "ACP Runner can answer through Debug Chat using real remote Claude",
"mode": "agent-browser",
"area": "pipeline",
"type": "regression",
@@ -229,8 +229,8 @@
"node:scripts/e2e/ensure-acp-agent-runner-pipeline.mjs --write-env"
],
"setup_provides_env": [
"LANGBOT_ACP_AGENT_RUNNER_PIPELINE_URL",
"LANGBOT_ACP_AGENT_RUNNER_PIPELINE_NAME"
"LANGBOT_ACP_RUNNER_PIPELINE_URL",
"LANGBOT_ACP_RUNNER_PIPELINE_NAME"
],
"evidence_required": [
"ui",
@@ -240,7 +240,7 @@
},
{
"id": "agent-run-ledger-audit",
"title": "Persisted AgentRunner run ledger passes end-to-end invariants",
"title": "Persisted Runner run ledger passes end-to-end invariants",
"mode": "probe",
"area": "agent",
"type": "regression",
@@ -264,7 +264,7 @@
},
{
"id": "agent-runner-async-db-readiness",
"title": "AgentRunner async DB readiness probe",
"title": "Runner async DB readiness probe",
"mode": "probe",
"area": "release",
"type": "smoke",
@@ -286,7 +286,7 @@
},
{
"id": "agent-runner-behavior-matrix",
"title": "AgentRunner deterministic behavior matrix probe",
"title": "Runner deterministic behavior matrix probe",
"mode": "probe",
"area": "release",
"type": "regression",
@@ -308,7 +308,7 @@
},
{
"id": "agent-runner-fixture-contract",
"title": "QA AgentRunner fixture contract probe",
"title": "QA Runner fixture contract probe",
"mode": "probe",
"area": "release",
"type": "regression",
@@ -355,7 +355,7 @@
},
{
"id": "agent-runner-ledger-concurrency",
"title": "AgentRunner run ledger concurrency and auth pytest probe",
"title": "Runner run ledger concurrency and auth pytest probe",
"mode": "probe",
"area": "release",
"type": "regression",
@@ -378,7 +378,7 @@
},
{
"id": "agent-runner-ledger-contention",
"title": "AgentRunner ledger SQLite contention probe",
"title": "Runner ledger SQLite contention probe",
"mode": "probe",
"area": "release",
"type": "regression",
@@ -401,7 +401,7 @@
},
{
"id": "agent-runner-ledger-invariants",
"title": "AgentRunner ledger schema and status invariants probe",
"title": "Runner ledger schema and status invariants probe",
"mode": "probe",
"area": "release",
"type": "regression",
@@ -423,7 +423,7 @@
},
{
"id": "agent-runner-ledger-stress",
"title": "AgentRunner ledger lightweight stress probe",
"title": "Runner ledger lightweight stress probe",
"mode": "probe",
"area": "release",
"type": "regression",
@@ -445,7 +445,7 @@
},
{
"id": "agent-runner-live-install",
"title": "QA AgentRunner package installs and registers in LangBot",
"title": "QA Runner package installs and registers in LangBot",
"mode": "probe",
"area": "release",
"type": "regression",
@@ -468,7 +468,7 @@
},
{
"id": "agent-runner-qa-debug-chat",
"title": "QA AgentRunner returns deterministic output through Debug Chat",
"title": "QA Runner returns deterministic output through Debug Chat",
"mode": "agent-browser",
"area": "pipeline",
"type": "regression",
@@ -487,8 +487,8 @@
"node:scripts/e2e/ensure-qa-agent-runner-pipeline.mjs --write-env"
],
"setup_provides_env": [
"LANGBOT_QA_AGENT_RUNNER_PIPELINE_URL",
"LANGBOT_QA_AGENT_RUNNER_PIPELINE_NAME"
"LANGBOT_QA_RUNNER_PIPELINE_URL",
"LANGBOT_QA_RUNNER_PIPELINE_NAME"
],
"evidence_required": [
"ui",
@@ -526,7 +526,7 @@
},
{
"id": "agent-runner-runtime-chaos",
"title": "AgentRunner SDK runtime chaos pytest probe",
"title": "Runner SDK runtime chaos pytest probe",
"mode": "probe",
"area": "release",
"type": "regression",
@@ -657,7 +657,7 @@
},
{
"id": "dify-agent-debug-chat",
"title": "Dify AgentRunner returns a response through Pipeline Debug Chat",
"title": "Dify Runner returns a response through Pipeline Debug Chat",
"mode": "agent-browser",
"area": "pipeline",
"type": "provider",
@@ -1937,7 +1937,7 @@
},
{
"id": "wizard-runner-marketplace-catalog",
"title": "Quick Start installs a published AgentRunner on a clean instance",
"title": "Quick Start installs a published Runner on a clean instance",
"mode": "agent-browser",
"area": "wizard",
"type": "feature",
@@ -2289,7 +2289,7 @@
{
"id": "langbot-workspace-release-gate",
"title": "LangBot workspace top-down release gate",
"description": "Broad release gate combining deterministic repository contracts with representative browser workflows, plugin pages, RAG/parser, EBA, external AgentRunner, and one complex LocalAgent task.",
"description": "Broad release gate combining deterministic repository contracts with representative browser workflows, plugin pages, RAG/parser, EBA, external Runner, and one complex LocalAgent task.",
"type": "release_gate",
"priority": "p0",
"tags": [
@@ -2352,7 +2352,7 @@
"fixtures": [
{
"id": "qa-agent-runner-behaviors",
"title": "Deterministic AgentRunner behavior matrix",
"title": "Deterministic Runner behavior matrix",
"kind": "json",
"path": "fixtures/agent-runner/qa-runner-behaviors.json",
"related_cases": [
@@ -2363,7 +2363,7 @@
},
{
"id": "qa-agent-runner-source",
"title": "QA deterministic AgentRunner fixture source",
"title": "QA deterministic Runner fixture source",
"kind": "plugin_source",
"path": "fixtures/plugins/qa-agent-runner/manifest.yaml",
"related_cases": [
@@ -2375,7 +2375,7 @@
},
{
"id": "qa-agent-runner-package",
"title": "QA deterministic AgentRunner prebuilt package",
"title": "QA deterministic Runner prebuilt package",
"kind": "plugin_package",
"path": "fixtures/plugins/qa-agent-runner/dist/qa-agent-runner-0.1.0.lbpkg",
"related_cases": [
@@ -2486,7 +2486,7 @@
"troubleshooting_summaries": [
{
"id": "agent-runner-actor-context-fields",
"title": "AgentRunner reads old actor.type and actor.id fields",
"title": "Runner reads old actor.type and actor.id fields",
"category": "product",
"related_cases": [
"dify-agent-debug-chat",
@@ -2504,7 +2504,7 @@
},
{
"id": "ambiguous-runner-default-label",
"title": "AgentRunner selector shows multiple Default or 默认 options",
"title": "Runner selector shows multiple Default or 默认 options",
"category": "product",
"related_cases": [
"dify-agent-debug-chat",
+7 -7
View File
@@ -37,10 +37,10 @@ LANGBOT_NO_PROXY=localhost,127.0.0.1,::1
# LANGBOT_PIPELINE_NAME=Generic QA Pipeline
# LANGBOT_LOCAL_AGENT_PIPELINE_URL=http://127.0.0.1:3000/home/pipelines?id=<local-agent-pipeline-uuid>
# LANGBOT_LOCAL_AGENT_PIPELINE_NAME=Local Agent QA Pipeline
# LANGBOT_ACP_AGENT_RUNNER_PIPELINE_URL=http://127.0.0.1:3000/home/pipelines?id=<acp-agent-runner-pipeline-uuid>
# LANGBOT_ACP_AGENT_RUNNER_PIPELINE_NAME=ACP AgentRunner QA Pipeline
# LANGBOT_ACP_AGENT_RUNNER_SSH_TARGET=yhh@101.34.71.12
# LANGBOT_ACP_AGENT_RUNNER_SSH_PORT=22
# LANGBOT_ACP_AGENT_RUNNER_SSH_IDENTITY_FILE=
# LANGBOT_ACP_AGENT_RUNNER_SSH_EXTRA_OPTIONS=
# LANGBOT_ACP_AGENT_RUNNER_REMOTE_WORKSPACE=/home/yhh/langbot-e2e/acp-workspace
# LANGBOT_ACP_RUNNER_PIPELINE_URL=http://127.0.0.1:3000/home/pipelines?id=<acp-agent-runner-pipeline-uuid>
# LANGBOT_ACP_RUNNER_PIPELINE_NAME=ACP Runner QA Pipeline
# LANGBOT_ACP_RUNNER_SSH_TARGET=yhh@101.34.71.12
# LANGBOT_ACP_RUNNER_SSH_PORT=22
# LANGBOT_ACP_RUNNER_SSH_IDENTITY_FILE=
# LANGBOT_ACP_RUNNER_SSH_EXTRA_OPTIONS=
# LANGBOT_ACP_RUNNER_REMOTE_WORKSPACE=/home/yhh/langbot-e2e/acp-workspace
+1 -1
View File
@@ -66,7 +66,7 @@ The tools wrap the LangBot service layer. Current tools (v1):
| `list_bots` / `get_bot` / `create_bot` / `update_bot` / `delete_bot` | Manage messaging-platform bots (secrets redacted on read) |
| `list_bot_event_route_statuses` | Inspect bot event-route runtime status |
| `list_processors` / `get_processor` / `create_processor` / `update_processor` / `delete_processor` | Manage the peer Agent, Pipeline and Event processor types |
| `get_processor_metadata` | Discover installed EventProcessor components, schemas and supported event patterns. |
| `get_processor_metadata` | Discover installed event-capable Runner components, schemas and supported event patterns. |
| `list_processor_runs` / `get_processor_run_events` | Read one Event processor instance run history and logs; paginate with `before_id` / `after_sequence`. |
| `debug_agent` | Execute a synthetic Agent event (`processor_uuid`, `payload`); requires `runtime.operate`. Returns final text and up to 1000 execution events (thinking, text, tool arguments/results). Platform tools use Mock; other configured tools execute normally. Optional `payload.mock`: `errors`/`results` keyed by platform tool name, `unsupported_apis` lists unavailable platform APIs. |
| `list_pipelines` / `get_pipeline` / `create_pipeline` / `update_pipeline` / `delete_pipeline` | Manage pipelines |
+5 -5
View File
@@ -11,13 +11,13 @@ Use this skill when an agent needs to verify LangBot behavior through the WebUI
- **General WebUI testing**: read `references/web-ui-testing.md`.
- **Pipeline Debug Chat**: read `references/pipeline-debug-chat.md`.
- **Dify AgentRunner**: read `references/dify-agent-runner.md`.
- **Dify Runner**: read `references/dify-agent-runner.md`.
- **Model provider setup or test button**: read `references/model-provider-testing.md`.
- **Plugin install/runtime/tool/page smoke**: read `references/plugin-e2e-smoke.md`.
- **Local Agent Runner**: read `references/local-agent-runner.md`.
- **Local Agent Runner path coverage**: read `references/local-agent-runner-coverage.md`.
- **Diff-aware AgentRunner QA after code changes**: read `references/agent-runner-qa-workflow.md`.
- **Agent Runner release gate**: read `references/agent-runner-release-gate.md`.
- **Local Runner**: read `references/local-agent-runner.md`.
- **Local Runner path coverage**: read `references/local-agent-runner-coverage.md`.
- **Diff-aware Runner QA after code changes**: read `references/agent-runner-qa-workflow.md`.
- **Runner release gate**: read `references/agent-runner-release-gate.md`.
- **Sandbox-backed skill authoring**: read `references/sandbox-skill-authoring.md`.
- **LangRAG knowledge bases**: read `references/langrag-knowledge-base.md`.
- **MCP stdio tool testing**: read `references/mcp-stdio-testing.md`.
@@ -1,5 +1,5 @@
id: acp-agent-runner-debug-chat
title: "ACP AgentRunner can answer through Debug Chat using real remote Claude"
title: "ACP Runner can answer through Debug Chat using real remote Claude"
mode: agent-browser
area: pipeline
type: regression
@@ -19,42 +19,42 @@ env:
- LANGBOT_FRONTEND_URL
- LANGBOT_BACKEND_URL
env_any:
- LANGBOT_ACP_AGENT_RUNNER_PIPELINE_URL|LANGBOT_ACP_AGENT_RUNNER_PIPELINE_NAME
- LANGBOT_ACP_RUNNER_PIPELINE_URL|LANGBOT_ACP_RUNNER_PIPELINE_NAME
automation: scripts/e2e/pipeline-debug-chat.mjs
automation_env:
- LANGBOT_FRONTEND_URL
- LANGBOT_BACKEND_URL
- LANGBOT_BROWSER_PROFILE
- LANGBOT_CHROMIUM_EXECUTABLE
- LANGBOT_ACP_AGENT_RUNNER_PIPELINE_URL
- LANGBOT_ACP_AGENT_RUNNER_PIPELINE_NAME
- LANGBOT_ACP_RUNNER_PIPELINE_URL
- LANGBOT_ACP_RUNNER_PIPELINE_NAME
- LANGBOT_E2E_PROMPT
- LANGBOT_E2E_EXPECTED_TEXT
- LANGBOT_E2E_EXPECTED_RUNNER_ID
- LANGBOT_E2E_RESPONSE_TIMEOUT_MS
automation_pipeline_url_env: LANGBOT_ACP_AGENT_RUNNER_PIPELINE_URL
automation_pipeline_name_env: LANGBOT_ACP_AGENT_RUNNER_PIPELINE_NAME
automation_expected_runner_id: "plugin:langbot-team/ACPAgentRunner/default"
automation_prompt: "Do not launch any background agent, subagent, task, or worker. In this current ACP session, directly call the MCP tool named langbot_get_current_event exactly once and wait for its result. After it returns, reply exactly ACP_AGENT_RUNNER_E2E_OK with no other text."
automation_expected_text: "ACP_AGENT_RUNNER_E2E_OK"
automation_pipeline_url_env: LANGBOT_ACP_RUNNER_PIPELINE_URL
automation_pipeline_name_env: LANGBOT_ACP_RUNNER_PIPELINE_NAME
automation_expected_runner_id: "plugin:langbot-team/ACPRunner/default"
automation_prompt: "Do not launch any background agent, subagent, task, or worker. In this current ACP session, directly call the MCP tool named langbot_get_current_event exactly once and wait for its result. After it returns, reply exactly ACP_RUNNER_E2E_OK with no other text."
automation_expected_text: "ACP_RUNNER_E2E_OK"
automation_response_timeout_ms: "300000"
setup_automation:
- "node:scripts/e2e/ensure-acp-agent-runner-pipeline.mjs --write-env"
setup_provides_env:
- LANGBOT_ACP_AGENT_RUNNER_PIPELINE_URL
- LANGBOT_ACP_AGENT_RUNNER_PIPELINE_NAME
- LANGBOT_ACP_RUNNER_PIPELINE_URL
- LANGBOT_ACP_RUNNER_PIPELINE_NAME
preconditions:
- "The remote machine has a working Claude Code login and can run npx -y @agentclientprotocol/claude-agent-acp."
- "LangBot can non-interactively SSH to the remote machine; the runner opens the MCP reverse tunnel automatically."
steps:
- "Open LANGBOT_FRONTEND_URL."
- "Open the ACP AgentRunner QA pipeline."
- "Confirm the pipeline AI runner is plugin:langbot-team/ACPAgentRunner/default."
- "Open the ACP Runner QA pipeline."
- "Confirm the pipeline AI runner is plugin:langbot-team/ACPRunner/default."
- "Open Debug Chat."
- "Ask the real remote Claude ACP agent to call langbot_get_current_event and return ACP_AGENT_RUNNER_E2E_OK exactly."
- "Ask the real remote Claude ACP agent to call langbot_get_current_event and return ACP_RUNNER_E2E_OK exactly."
checks:
- "UI: Debug Chat shows the user prompt."
- "UI: Debug Chat shows a Bot response containing ACP_AGENT_RUNNER_E2E_OK."
- "UI: Debug Chat shows a Bot response containing ACP_RUNNER_E2E_OK."
- "Logs: Backend logs include Processing request from person_websocket and Streaming completed for this run."
- "Logs: No acp runner request error appears for this run."
- "Console: No unexpected frontend errors appear during Debug Chat."
@@ -66,13 +66,13 @@ diagnostics:
- "Use scripts/e2e/ensure-acp-agent-runner-pipeline.mjs --write-env to create/update the pipeline."
- "For remote Claude on 101, verify ssh yhh@101.34.71.12 can run without password prompts; no separate ssh -R process is required."
success_patterns:
- "ACP_AGENT_RUNNER_E2E_OK"
- "ACP_RUNNER_E2E_OK"
- "Processing request from person_websocket"
- "Streaming completed"
failure_patterns:
- "acp.command_not_found"
- "acp.process_exited"
- "Agent runner plugin:langbot-team/ACPAgentRunner/default execution failed"
- "Agent runner plugin:langbot-team/ACPRunner/default execution failed"
troubleshooting:
- backend-not-listening
- plugin-runtime-timeout
@@ -1,5 +1,5 @@
id: agent-run-ledger-audit
title: "Persisted AgentRunner run ledger passes end-to-end invariants"
title: "Persisted Runner run ledger passes end-to-end invariants"
mode: probe
area: agent
type: regression
@@ -15,7 +15,7 @@ skills:
- langbot-testing
automation: scripts/e2e/agent-run-ledger-audit.mjs
steps:
- "Set LANGBOT_AGENT_RUN_ID to audit a specific run, or leave it unset to audit the latest persisted AgentRunner run."
- "Set LANGBOT_AGENT_RUN_ID to audit a specific run, or leave it unset to audit the latest persisted Runner run."
- "For an external runner's own CLI tools, set LANGBOT_AGENT_TOOL_AUTHORIZATION_MODE=runner-native; keep the default strict mode for Host tool calls."
- "Read the active LangBot database configuration and inspect the selected run and its ordered events."
- "Verify completed terminal state, run.completed, paired tool.call.started/completed events, stable tool names, and monotonic ordering."
@@ -1,5 +1,5 @@
id: agent-runner-async-db-readiness
title: "AgentRunner async DB readiness probe"
title: "Runner async DB readiness probe"
mode: probe
area: release
type: smoke
@@ -1,5 +1,5 @@
id: agent-runner-behavior-matrix
title: "AgentRunner deterministic behavior matrix probe"
title: "Runner deterministic behavior matrix probe"
mode: probe
area: release
type: regression
@@ -1,5 +1,5 @@
id: agent-runner-fixture-contract
title: "QA AgentRunner fixture contract probe"
title: "QA Runner fixture contract probe"
mode: probe
area: release
type: regression
@@ -17,19 +17,19 @@ env:
automation: skills/langbot-testing/probes/agent-runner-fixture-contract.mjs
steps:
- "Run `rtk bin/lbs test run agent-runner-fixture-contract --dry-run` first; remove `--dry-run` after checking the planned evidence directory."
- "Automation imports the QA AgentRunner fixture source and executes normal, streaming, and controlled-failure paths with SDK entities."
- "Automation imports the QA Runner fixture source and executes normal, streaming, and controlled-failure paths with SDK entities."
checks:
- "automation-result.json status is pass."
- "probe-stdout.log contains QA_AGENT_RUNNER_FIXTURE_CONTRACT_OK."
- "Normal input returns QA_AGENT_RUNNER_OK:<input>."
- "probe-stdout.log contains QA_RUNNER_FIXTURE_CONTRACT_OK."
- "Normal input returns QA_RUNNER_OK:<input>."
- "Streaming input emits message.delta chunks and completes."
- "Failure input returns QA_AGENT_RUNNER_CONTROLLED_FAILURE."
- "Failure input returns QA_RUNNER_CONTROLLED_FAILURE."
evidence_required:
- filesystem
diagnostics:
- "This validates the deterministic fixture source contract. It does not prove the plugin package is installed in a live LangBot instance."
success_patterns:
- "QA_AGENT_RUNNER_FIXTURE_CONTRACT_OK"
- "QA_RUNNER_FIXTURE_CONTRACT_OK"
failure_patterns:
- "AssertionError"
- "fixture contract exited"
@@ -25,7 +25,7 @@ automation_env:
- LANGBOT_BROWSER_PROFILE
- LANGBOT_CHROMIUM_EXECUTABLE
preconditions:
- "The plugin runtime is enabled and connected, with at least one AgentRunner registered."
- "The plugin runtime is enabled and connected, with at least one Runner registered."
- "The target is a local test instance where a temporary Agent may be created and deleted."
steps:
- "Read the live plugin runtime status and select a registered runner from Agent metadata."
@@ -1,5 +1,5 @@
id: agent-runner-ledger-concurrency
title: "AgentRunner run ledger concurrency and auth pytest probe"
title: "Runner run ledger concurrency and auth pytest probe"
mode: probe
area: release
type: regression
@@ -1,5 +1,5 @@
id: agent-runner-ledger-contention
title: "AgentRunner ledger SQLite contention probe"
title: "Runner ledger SQLite contention probe"
mode: probe
area: release
type: regression
@@ -1,5 +1,5 @@
id: agent-runner-ledger-invariants
title: "AgentRunner ledger schema and status invariants probe"
title: "Runner ledger schema and status invariants probe"
mode: probe
area: release
type: regression
@@ -1,5 +1,5 @@
id: agent-runner-ledger-stress
title: "AgentRunner ledger lightweight stress probe"
title: "Runner ledger lightweight stress probe"
mode: probe
area: release
type: regression
@@ -1,5 +1,5 @@
id: agent-runner-live-install
title: "QA AgentRunner package installs and registers in LangBot"
title: "QA Runner package installs and registers in LangBot"
mode: probe
area: release
type: regression
@@ -25,7 +25,7 @@ automation_expected_tool: ""
automation_expected_runner_id: "plugin:qa/agent-runner/default"
steps:
- "Run `rtk bin/lbs test run agent-runner-live-install --dry-run` first; remove `--dry-run` only after readiness points at a local test LangBot instance."
- "Automation authenticates the local test user, uploads the QA AgentRunner .lbpkg package, waits for the install task, and reads pipeline metadata."
- "Automation authenticates the local test user, uploads the QA Runner .lbpkg package, waits for the install task, and reads pipeline metadata."
checks:
- "automation-result.json status is pass."
- "/api/v1/plugins lists qa/agent-runner after install."
@@ -1,5 +1,5 @@
id: agent-runner-qa-debug-chat
title: "QA AgentRunner returns deterministic output through Debug Chat"
title: "QA Runner returns deterministic output through Debug Chat"
mode: agent-browser
area: pipeline
type: regression
@@ -17,21 +17,21 @@ skills:
env:
- LANGBOT_FRONTEND_URL
- LANGBOT_BACKEND_URL
- LANGBOT_QA_AGENT_RUNNER_PIPELINE_URL
- LANGBOT_QA_AGENT_RUNNER_PIPELINE_NAME
- LANGBOT_QA_RUNNER_PIPELINE_URL
- LANGBOT_QA_RUNNER_PIPELINE_NAME
automation: scripts/e2e/pipeline-debug-chat.mjs
automation_env:
- LANGBOT_FRONTEND_URL
- LANGBOT_BACKEND_URL
- LANGBOT_BROWSER_PROFILE
- LANGBOT_CHROMIUM_EXECUTABLE
- LANGBOT_QA_AGENT_RUNNER_PIPELINE_URL
- LANGBOT_QA_AGENT_RUNNER_PIPELINE_NAME
automation_pipeline_url_env: LANGBOT_QA_AGENT_RUNNER_PIPELINE_URL
automation_pipeline_name_env: LANGBOT_QA_AGENT_RUNNER_PIPELINE_NAME
- LANGBOT_QA_RUNNER_PIPELINE_URL
- LANGBOT_QA_RUNNER_PIPELINE_NAME
automation_pipeline_url_env: LANGBOT_QA_RUNNER_PIPELINE_URL
automation_pipeline_name_env: LANGBOT_QA_RUNNER_PIPELINE_NAME
automation_expected_runner_id: "plugin:qa/agent-runner/default"
automation_prompt: "hello-live"
automation_expected_text: "QA_AGENT_RUNNER_OK:hello-live"
automation_expected_text: "QA_RUNNER_OK:hello-live"
automation_response_timeout_ms: "120000"
automation_debug_chat_response_p95_ms: "120000"
automation_reset_debug_chat: "1"
@@ -39,17 +39,17 @@ setup_automation:
- "case:agent-runner-live-install"
- "node:scripts/e2e/ensure-qa-agent-runner-pipeline.mjs --write-env"
setup_provides_env:
- LANGBOT_QA_AGENT_RUNNER_PIPELINE_URL
- LANGBOT_QA_AGENT_RUNNER_PIPELINE_NAME
- LANGBOT_QA_RUNNER_PIPELINE_URL
- LANGBOT_QA_RUNNER_PIPELINE_NAME
steps:
- "Open LANGBOT_FRONTEND_URL."
- "Open the pipeline from LANGBOT_QA_AGENT_RUNNER_PIPELINE_URL or LANGBOT_QA_AGENT_RUNNER_PIPELINE_NAME."
- "Open the pipeline from LANGBOT_QA_RUNNER_PIPELINE_URL or LANGBOT_QA_RUNNER_PIPELINE_NAME."
- "Confirm the pipeline AI runner is plugin:qa/agent-runner/default."
- "Open Debug Chat."
- "Send: hello-live."
checks:
- "UI: The user message appears in Debug Chat."
- "UI: A Bot message appears and contains QA_AGENT_RUNNER_OK:hello-live."
- "UI: A Bot message appears and contains QA_RUNNER_OK:hello-live."
- "API diagnostic: pipeline config uses plugin:qa/agent-runner/default."
- "Console: No unexpected frontend runtime errors appear during the send/receive path."
evidence_required:
@@ -62,7 +62,7 @@ diagnostics:
- "This is the deterministic live execution proof that sits after fixture contract and live install."
- "If the runner id mismatch is reported, rerun ensure-qa-agent-runner-pipeline.mjs --write-env."
success_patterns:
- "QA_AGENT_RUNNER_OK:hello-live"
- "QA_RUNNER_OK:hello-live"
failure_patterns:
- "plugin:qa/agent-runner/default execution failed"
- "Action invoke_llm_stream call timed out"
@@ -19,7 +19,7 @@ env:
- LANGBOT_BACKEND_URL
env_any:
- LANGBOT_LOCAL_AGENT_PIPELINE_URL|LANGBOT_LOCAL_AGENT_PIPELINE_NAME
- LANGBOT_ACP_AGENT_RUNNER_PIPELINE_URL|LANGBOT_ACP_AGENT_RUNNER_PIPELINE_NAME
- LANGBOT_ACP_RUNNER_PIPELINE_URL|LANGBOT_ACP_RUNNER_PIPELINE_NAME
automation: scripts/e2e/agent-runner-release-preflight.mjs
automation_env:
- LANGBOT_FRONTEND_URL
@@ -28,24 +28,24 @@ automation_env:
- LANGBOT_CHROMIUM_EXECUTABLE
automation_env_any:
- LANGBOT_LOCAL_AGENT_PIPELINE_URL|LANGBOT_LOCAL_AGENT_PIPELINE_NAME
- LANGBOT_ACP_AGENT_RUNNER_PIPELINE_URL|LANGBOT_ACP_AGENT_RUNNER_PIPELINE_NAME
- LANGBOT_ACP_RUNNER_PIPELINE_URL|LANGBOT_ACP_RUNNER_PIPELINE_NAME
preconditions:
- "LANGBOT_LOCAL_AGENT_PIPELINE_URL or LANGBOT_LOCAL_AGENT_PIPELINE_NAME points to the local-agent release pipeline."
- "LANGBOT_ACP_AGENT_RUNNER_PIPELINE_URL or LANGBOT_ACP_AGENT_RUNNER_PIPELINE_NAME points to the ACP AgentRunner release pipeline."
- "LANGBOT_ACP_RUNNER_PIPELINE_URL or LANGBOT_ACP_RUNNER_PIPELINE_NAME points to the ACP Runner release pipeline."
- "The active browser profile is authenticated for the same LangBot backend."
- "By default the preflight performs a cheap model test for the local-agent primary model; set LANGBOT_PREFLIGHT_TEST_MODELS=0 only when deliberately classifying model credentials outside this run."
steps:
- "Open LANGBOT_FRONTEND_URL with the configured browser profile."
- "Use the browser token to call LangBot backend readiness APIs without printing token values."
- "Check plugin runtime status, Box status, required runner plugins, qa-plugin-smoke, and qa_plugin_echo."
- "Resolve the local-agent and ACP AgentRunner QA pipelines from their case-specific env vars."
- "Resolve the local-agent and ACP Runner QA pipelines from their case-specific env vars."
- "Assert each pipeline uses the expected runner id."
- "Assert the external runner pipeline uses the expected runner id."
- "Assert the local-agent primary model advertises func_call and vision for the full release gate."
- "Run the local-agent primary model test endpoint unless LANGBOT_PREFLIGHT_TEST_MODELS=0."
checks:
- "API diagnostic: api-diagnostic.json has no blockers and no env_issues."
- "API diagnostic: required pipelines resolve to plugin:langbot-team/LocalAgent/default and plugin:langbot-team/ACPAgentRunner/default."
- "API diagnostic: required pipelines resolve to plugin:langbot-team/LocalAgent/default and plugin:langbot-team/ACPRunner/default."
- "API diagnostic: qa_plugin_echo is exposed by /api/v1/tools."
- "API diagnostic: local-agent model check catches invalid credentials or missing func_call/vision before release E2E starts."
- "Secret safety: token values, api keys, and provider secrets are not printed."
@@ -1,5 +1,5 @@
id: agent-runner-runtime-chaos
title: "AgentRunner SDK runtime chaos pytest probe"
title: "Runner SDK runtime chaos pytest probe"
mode: probe
area: release
type: regression
@@ -19,10 +19,10 @@ automation: skills/langbot-testing/probes/agent-runner-runtime-chaos.mjs
steps:
- "Run `rtk bin/lbs test run agent-runner-runtime-chaos --dry-run` first; remove `--dry-run` after checking the SDK repo target and evidence directory."
- "Automation resolves LANGBOT_PLUGIN_SDK_REPO, defaulting to ../../langbot-plugin-sdk when the env var is unset."
- "Automation runs the existing SDK pytest files tests/runtime/plugin/test_mgr_agent_runner.py and tests/runtime/test_pull_api_handlers.py."
- "Automation runs the existing SDK pytest files tests/runtime/plugin/test_mgr_runner.py and tests/runtime/test_pull_api_handlers.py."
checks:
- "automation-result.json status is pass."
- "pytest exit status is 0 for the existing AgentRunner runtime and pull API handler tests."
- "pytest exit status is 0 for the existing Runner runtime and pull API handler tests."
- "pytest-stdout.log and pytest-stderr.log are written under LBS_EVIDENCE_DIR."
evidence_required:
- filesystem
@@ -45,7 +45,7 @@ steps:
- "Select exactly one Box child whose parent is main.py running from LANGBOT_REPO; abort on zero or multiple matches."
- "Send SIGTERM to that Box child and poll process, Box status, MCP runtime info, and global tools for up to 30 seconds."
- "Without running MCP setup or registration, reset Debug Chat and call qa_mcp_echo with a unique per-run value through the browser."
- "Audit the matching AgentRunner ledger run and require the exact qa_mcp_echo arguments plus the complete tool result text."
- "Audit the matching Runner ledger run and require the exact qa_mcp_echo arguments plus the complete tool result text."
checks:
- "The old Box PID exits and a new Box PID appears under the same LangBot parent."
- "Box returns available=true with at least one active session and managed process."
@@ -1,5 +1,5 @@
id: dify-agent-debug-chat
title: "Dify AgentRunner returns a response through Pipeline Debug Chat"
title: "Dify Runner returns a response through Pipeline Debug Chat"
mode: agent-browser
area: pipeline
type: provider
@@ -18,8 +18,8 @@ skills:
env:
- LANGBOT_FRONTEND_URL
- LANGBOT_BACKEND_URL
- LANGBOT_ACP_AGENT_RUNNER_PIPELINE_URL
- LANGBOT_ACP_AGENT_RUNNER_PIPELINE_NAME
- LANGBOT_ACP_RUNNER_PIPELINE_URL
- LANGBOT_ACP_RUNNER_PIPELINE_NAME
preconditions:
- "An external-harness runner pipeline (e.g. ACP remote claude-code) is configured with langbot-assets-enabled=true so the LangBot MCP gateway is exposed to the harness."
- "The remote harness (claude-code) is reachable and responsive (claude -p returns within the runner timeout)."
@@ -29,10 +29,10 @@ automation_env:
- LANGBOT_FRONTEND_URL
- LANGBOT_BROWSER_PROFILE
- LANGBOT_CHROMIUM_EXECUTABLE
- LANGBOT_ACP_AGENT_RUNNER_PIPELINE_URL
- LANGBOT_ACP_AGENT_RUNNER_PIPELINE_NAME
automation_pipeline_url_env: LANGBOT_ACP_AGENT_RUNNER_PIPELINE_URL
automation_pipeline_name_env: LANGBOT_ACP_AGENT_RUNNER_PIPELINE_NAME
- LANGBOT_ACP_RUNNER_PIPELINE_URL
- LANGBOT_ACP_RUNNER_PIPELINE_NAME
automation_pipeline_url_env: LANGBOT_ACP_RUNNER_PIPELINE_URL
automation_pipeline_name_env: LANGBOT_ACP_RUNNER_PIPELINE_NAME
automation_prompt: "You have LangBot tools available via an MCP server (tools prefixed langbot_). Call langbot_list_assets with asset_types = [\"skills\",\"tools\"]. Then reply with one single line: the literal token PROBEDONE, a space, the number of skills you found, a space, and the number of tools you found."
automation_expected_text: "PROBEDONE"
automation_response_timeout_ms: "540000"
@@ -1,5 +1,5 @@
id: wizard-runner-marketplace-catalog
title: "Quick Start installs a published AgentRunner on a clean instance"
title: "Quick Start installs a published Runner on a clean instance"
mode: agent-browser
area: wizard
type: feature
@@ -27,19 +27,19 @@ preconditions:
steps:
- "Start an isolated first-run instance and confirm zero installed plugins and zero registered runners."
- "Resume Quick Start at the AI Engine step with a temporary disabled Bot."
- "Confirm the browser requests Marketplace plugins with component_filter=AgentRunner."
- "Confirm the browser requests Marketplace plugins with component_filter=Runner."
- "Confirm langbot-team/LocalAgent is published with an installable version and the Runner Extensions link is correct."
- "Install LocalAgent and wait for plugin initialization and AgentRunner registration."
- "Install LocalAgent and wait for plugin initialization and Runner registration."
- "Confirm Create & Deploy is disabled before installation and enabled after LocalAgent is selected."
- "Verify the layout at desktop and mobile widths."
checks:
- "API: The instance has zero installed plugins and zero registered runners."
- "API: The instance wizard status is none."
- "Network: Marketplace search uses component_filter=AgentRunner and type_filter=plugin."
- "Network: Marketplace search uses component_filter=Runner and type_filter=plugin."
- "UI: The AI Engine step displays the published langbot-team/LocalAgent card."
- "Marketplace: LocalAgent includes latest_version so installation can proceed."
- "Runtime: LocalAgent installs and registers plugin:langbot-team/LocalAgent/default."
- "UI: Browse Runner Extensions links to the AgentRunner-filtered market."
- "UI: Browse Runner Extensions links to the Runner-filtered market."
- "UI: Create & Deploy transitions from disabled to enabled only after Runner selection."
- "Console: No unexpected frontend errors appear during the flow."
- "Cleanup: Wizard progress and the temporary Bot are removed."
@@ -18,7 +18,7 @@ steps:
- "Discover the active LangBot checkout and sibling workspace repositories, with LANGBOT_WORKSPACE_ROOT and repository-specific env overrides available for non-default layouts."
- "Verify every checkout is on main or dev/4.11.x and record dirty worktrees as warnings only."
- "Validate plugin manifest identities for LocalAgent, Control Plane, LongTermMemory, GeneralParsers, LangRAG, and Skill Authoring."
- "Use the LangBot virtualenv to import langbot_plugin and AgentRunner Protocol entities from the workspace SDK source tree."
- "Use the LangBot virtualenv to import langbot_plugin and Runner Protocol entities from the workspace SDK source tree."
checks:
- "workspace-preflight.json contains no failed checks."
- "The langbot_plugin import path is inside the discovered langbot-plugin-sdk/src directory."
@@ -16,7 +16,7 @@ skills:
automation: scripts/e2e/workspace-repository-contracts.mjs
steps:
- "Resolve the active LangBot virtualenv and workspace SDK source path."
- "Run tests independently for AgentRunner, Control Plane, LongTermMemory, Parser, RAG, Skill Authoring, LocalAgent, LangBot Agent/Provider, the skills CLI, and SDK runtime contracts."
- "Run tests independently for Runner, Control Plane, LongTermMemory, Parser, RAG, Skill Authoring, LocalAgent, LangBot Agent/Provider, the skills CLI, and SDK runtime contracts."
- "Run SDK packaging blackbox separately so an isolated build dependency network failure is classified as env_issue without masking product test failures."
- "Write per-repository stdout, stderr, status, and duration under repository-contracts/."
checks:
@@ -1,7 +1,7 @@
[
{
"id": "qa-agent-runner-behaviors",
"title": "Deterministic AgentRunner behavior matrix",
"title": "Deterministic Runner behavior matrix",
"kind": "json",
"path": "fixtures/agent-runner/qa-runner-behaviors.json",
"related_cases": [
@@ -13,7 +13,7 @@
},
{
"id": "qa-agent-runner-source",
"title": "QA deterministic AgentRunner fixture source",
"title": "QA deterministic Runner fixture source",
"kind": "plugin_source",
"path": "fixtures/plugins/qa-agent-runner/manifest.yaml",
"related_cases": [
@@ -22,11 +22,11 @@
"agent-runner-live-install",
"agent-runner-qa-debug-chat"
],
"checks": ["exists", "qa_agent_runner_source"]
"checks": ["exists", "qa_runner_source"]
},
{
"id": "qa-agent-runner-package",
"title": "QA deterministic AgentRunner prebuilt package",
"title": "QA deterministic Runner prebuilt package",
"kind": "plugin_package",
"path": "fixtures/plugins/qa-agent-runner/dist/qa-agent-runner-0.1.0.lbpkg",
"related_cases": [
@@ -1,6 +1,6 @@
# QA AgentRunner Fixture
# QA Runner Fixture
Deterministic AgentRunner plugin source used by `langbot-skills` probes and future browser release-gate cases.
Deterministic Runner plugin source used by `langbot-skills` probes and future browser release-gate cases.
Runner id after installation should be:
@@ -10,6 +10,6 @@ plugin:qa/agent-runner/default
Expected behavior:
- normal input returns `QA_AGENT_RUNNER_OK:<input>`
- normal input returns `QA_RUNNER_OK:<input>`
- input containing `stream` emits streaming chunks then completes
- input containing `fail` returns `QA_AGENT_RUNNER_CONTROLLED_FAILURE`
- input containing `fail` returns `QA_RUNNER_CONTROLLED_FAILURE`
@@ -1,4 +1,4 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 64 64" role="img" aria-label="QA AgentRunner icon">
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 64 64" role="img" aria-label="QA Runner icon">
<rect width="64" height="64" rx="12" fill="#111827"/>
<path d="M16 20h32v22H35l-7 8v-8H16z" fill="#22c55e"/>
<path d="M24 30h16" stroke="#111827" stroke-width="4" stroke-linecap="round"/>

Before

Width:  |  Height:  |  Size: 306 B

After

Width:  |  Height:  |  Size: 301 B

@@ -1,39 +0,0 @@
from __future__ import annotations
import typing
from langbot_plugin.api.definition.components.agent_runner.runner import AgentRunner
from langbot_plugin.api.entities.builtin.agent_runner import AgentRunContext, AgentRunResult
from langbot_plugin.api.entities.builtin.provider.message import Message, MessageChunk
class DefaultAgentRunner(AgentRunner):
async def run(
self,
ctx: AgentRunContext,
) -> typing.AsyncGenerator[AgentRunResult, None]:
text = (ctx.input.to_text() or "").strip()
if "fail" in text.lower():
yield AgentRunResult.run_failed(
ctx.run_id,
error="QA_AGENT_RUNNER_CONTROLLED_FAILURE",
code="qa.controlled_failure",
retryable=False,
)
return
content = f"QA_AGENT_RUNNER_OK:{text or 'empty'}"
if "stream" in text.lower():
for chunk in ("QA_", "AGENT_", f"RUNNER_OK:{text}"):
yield AgentRunResult.message_delta(
ctx.run_id,
MessageChunk(role="assistant", content=chunk),
)
yield AgentRunResult.run_completed(ctx.run_id, finish_reason="stop")
return
yield AgentRunResult.run_completed(
ctx.run_id,
Message(role="assistant", content=content),
finish_reason="stop",
)
@@ -0,0 +1,39 @@
from __future__ import annotations
import typing
from langbot_plugin.api.definition.components.runner.runner import Runner
from langbot_plugin.api.entities.builtin.runner import RunnerContext, RunnerResult
from langbot_plugin.api.entities.builtin.provider.message import Message, MessageChunk
class DefaultRunner(Runner):
async def run(
self,
ctx: RunnerContext,
) -> typing.AsyncGenerator[RunnerResult, None]:
text = (ctx.input.to_text() or '').strip()
if 'fail' in text.lower():
yield RunnerResult.run_failed(
ctx.run_id,
error='QA_RUNNER_CONTROLLED_FAILURE',
code='qa.controlled_failure',
retryable=False,
)
return
content = f'QA_RUNNER_OK:{text or "empty"}'
if 'stream' in text.lower():
for chunk in ('QA_', 'AGENT_', f'RUNNER_OK:{text}'):
yield RunnerResult.message_delta(
ctx.run_id,
MessageChunk(role='assistant', content=chunk),
)
yield RunnerResult.run_completed(ctx.run_id, finish_reason='stop')
return
yield RunnerResult.run_completed(
ctx.run_id,
Message(role='assistant', content=content),
finish_reason='stop',
)
@@ -1,5 +1,5 @@
apiVersion: langbot/v1
kind: AgentRunner
kind: Runner
metadata:
name: default
label:
@@ -27,4 +27,4 @@ spec:
execution:
python:
path: default.py
attr: DefaultAgentRunner
attr: DefaultRunner
@@ -3,6 +3,6 @@ from __future__ import annotations
from langbot_plugin.api.definition.plugin import BasePlugin
class QAAgentRunnerPlugin(BasePlugin):
class QARunnerPlugin(BasePlugin):
async def initialize(self) -> None:
self.ready_marker = "qa-agent-runner-ready"
self.ready_marker = 'qa-agent-runner-ready'
@@ -6,20 +6,20 @@ metadata:
repository: https://example.invalid/langbot/qa-agent-runner
version: 0.1.0
description:
en_US: Deterministic AgentRunner fixture for LangBot QA.
zh_Hans: LangBot QA 使用的确定性 AgentRunner 夹具。
en_US: Deterministic Runner fixture for LangBot QA.
zh_Hans: LangBot QA 使用的确定性 Runner 夹具。
label:
en_US: QA AgentRunner
zh_Hans: QA AgentRunner
en_US: QA Runner
zh_Hans: QA Runner
icon: assets/icon.svg
spec:
config: []
components:
AgentRunner:
Runner:
fromDirs:
- path: components/agent_runner/
- path: components/runner/
maxDepth: 1
execution:
python:
path: main.py
attr: QAAgentRunnerPlugin
attr: QARunnerPlugin
@@ -7,7 +7,11 @@ import { delimiter, join, resolve } from "node:path";
import { env } from "node:process";
function timestampSlug(date = new Date()) {
return date.toISOString().replace(/\.\d{3}Z$/, "Z").replace(/[^0-9A-Za-z]+/g, "-").replace(/^-|-$/g, "");
return date
.toISOString()
.replace(/\.\d{3}Z$/, "Z")
.replace(/[^0-9A-Za-z]+/g, "-")
.replace(/^-|-$/g, "");
}
function localIsoWithOffset(date = new Date()) {
@@ -51,7 +55,14 @@ function run(command, timeoutMs, childEnv) {
});
child.on("error", (error) => {
clearTimeout(timeout);
resolveDone({ stdout, stderr, error, timedOut, status: null, signal: null });
resolveDone({
stdout,
stderr,
error,
timedOut,
status: null,
signal: null,
});
});
child.on("close", (status, signal) => {
clearTimeout(timeout);
@@ -66,7 +77,7 @@ import json
import sys
from pathlib import Path
from langbot.pkg.agent.runner.descriptor import AgentRunnerDescriptor
from langbot.pkg.agent.runner.descriptor import RunnerDescriptor
from langbot.pkg.agent.runner.errors import RunnerExecutionError, RunnerProtocolError
from langbot.pkg.agent.runner.result_normalizer import AgentResultNormalizer
@@ -80,10 +91,10 @@ class App:
logger = Logger()
def descriptor():
return AgentRunnerDescriptor(
return RunnerDescriptor(
id='plugin:qa/agent-runner/default',
source='plugin',
label={'en_US': 'QA AgentRunner'},
label={'en_US': 'QA Runner'},
plugin_author='qa',
plugin_name='agent-runner',
runner_name='default',
@@ -139,18 +150,26 @@ async function main() {
const root = resolve(env.LBS_ROOT || process.cwd());
const caseId = "agent-runner-behavior-matrix";
const runId = env.LBS_RUN_ID || `${timestampSlug()}-${caseId}`;
const evidenceDir = resolve(env.LBS_EVIDENCE_DIR || join(root, "reports", "evidence", runId));
const evidenceDir = resolve(
env.LBS_EVIDENCE_DIR || join(root, "reports", "evidence", runId),
);
await mkdir(evidenceDir, { recursive: true });
const startedAt = new Date();
const langbotRepo = resolve(root, env.LANGBOT_REPO || "..");
const sdkRepo = resolve(root, env.LANGBOT_PLUGIN_SDK_REPO || "../../langbot-plugin-sdk");
const sdkRepo = resolve(
root,
env.LANGBOT_PLUGIN_SDK_REPO || "../../langbot-plugin-sdk",
);
const sdkSrc = resolve(sdkRepo, "src");
const fixturePath = resolve(root, "skills/langbot-testing/fixtures/agent-runner/qa-runner-behaviors.json");
const fixturePath = resolve(
root,
"skills/langbot-testing/fixtures/agent-runner/qa-runner-behaviors.json",
);
const stdoutLog = join(evidenceDir, "probe-stdout.log");
const stderrLog = join(evidenceDir, "probe-stderr.log");
const automationResultJson = join(evidenceDir, "automation-result.json");
const resultJson = join(evidenceDir, "result.json");
const timeoutMs = Number(env.LANGBOT_AGENT_RUNNER_PROBE_TIMEOUT_MS || "30000");
const timeoutMs = Number(env.LANGBOT_RUNNER_PROBE_TIMEOUT_MS || "30000");
const command = {
executable: "rtk",
args: [resolve(langbotRepo, ".venv/bin/python"), "-c", script, fixturePath],
@@ -175,7 +194,12 @@ async function main() {
timeout_ms: timeoutMs,
exit_status: null,
signal: null,
evidence: { stdout_log: stdoutLog, stderr_log: stderrLog, automation_result_json: automationResultJson, result_json: resultJson },
evidence: {
stdout_log: stdoutLog,
stderr_log: stderrLog,
automation_result_json: automationResultJson,
result_json: resultJson,
},
evidence_collected: ["filesystem"],
};
try {
@@ -185,7 +209,9 @@ async function main() {
} else {
const proc = await run(command, timeoutMs, {
...process.env,
PYTHONPATH: [sdkSrc, process.env.PYTHONPATH].filter(Boolean).join(delimiter),
PYTHONPATH: [sdkSrc, process.env.PYTHONPATH]
.filter(Boolean)
.join(delimiter),
UV_CACHE_DIR: env.UV_CACHE_DIR || join(evidenceDir, ".uv-cache"),
});
await writeFile(stdoutLog, proc.stdout, "utf8");
@@ -198,7 +224,10 @@ async function main() {
} else if (proc.timedOut) {
result.status = "fail";
result.reason = `behavior matrix timed out after ${timeoutMs}ms`;
} else if (proc.status === 0 && proc.stdout.includes("QA_RUNNER_BEHAVIOR_MATRIX_OK")) {
} else if (
proc.status === 0 &&
proc.stdout.includes("QA_RUNNER_BEHAVIOR_MATRIX_OK")
) {
result.status = "pass";
result.reason = "behavior matrix passed";
} else {
@@ -219,7 +248,9 @@ async function main() {
await writeFile(resultJson, resultText, "utf8");
console.log(JSON.stringify(result, null, 2));
}
process.exit(result.status === "pass" ? 0 : result.status === "env_issue" ? 2 : 1);
process.exit(
result.status === "pass" ? 0 : result.status === "env_issue" ? 2 : 1,
);
}
await main();
@@ -7,7 +7,11 @@ import { delimiter, join, resolve } from "node:path";
import { env } from "node:process";
function timestampSlug(date = new Date()) {
return date.toISOString().replace(/\.\d{3}Z$/, "Z").replace(/[^0-9A-Za-z]+/g, "-").replace(/^-|-$/g, "");
return date
.toISOString()
.replace(/\.\d{3}Z$/, "Z")
.replace(/[^0-9A-Za-z]+/g, "-")
.replace(/^-|-$/g, "");
}
function localIsoWithOffset(date = new Date()) {
@@ -51,7 +55,14 @@ function run(command, timeoutMs, childEnv) {
});
child.on("error", (error) => {
clearTimeout(timeout);
resolveDone({ stdout, stderr, error, timedOut, status: null, signal: null });
resolveDone({
stdout,
stderr,
error,
timedOut,
status: null,
signal: null,
});
});
child.on("close", (status, signal) => {
clearTimeout(timeout);
@@ -66,27 +77,27 @@ import importlib.util
import sys
from pathlib import Path
from langbot_plugin.api.entities.builtin.agent_runner.context import AgentRunContext
from langbot_plugin.api.entities.builtin.agent_runner.delivery import DeliveryContext
from langbot_plugin.api.entities.builtin.agent_runner.event import AgentEventContext
from langbot_plugin.api.entities.builtin.agent_runner.input import AgentInput
from langbot_plugin.api.entities.builtin.agent_runner.resources import AgentResources
from langbot_plugin.api.entities.builtin.agent_runner.runtime import AgentRuntimeContext
from langbot_plugin.api.entities.builtin.agent_runner.trigger import AgentTrigger
from langbot_plugin.api.entities.builtin.runner.context import RunnerContext
from langbot_plugin.api.entities.builtin.runner.delivery import DeliveryContext
from langbot_plugin.api.entities.builtin.runner.event import AgentEventContext
from langbot_plugin.api.entities.builtin.runner.input import AgentInput
from langbot_plugin.api.entities.builtin.runner.resources import AgentResources
from langbot_plugin.api.entities.builtin.runner.runtime import AgentRuntimeContext
from langbot_plugin.api.entities.builtin.runner.trigger import AgentTrigger
fixture = Path(sys.argv[1])
runner_py = fixture / "components" / "agent_runner" / "default.py"
runner_py = fixture / "components" / "runner" / "default.py"
manifest = fixture / "manifest.yaml"
runner_yaml = fixture / "components" / "agent_runner" / "default.yaml"
runner_yaml = fixture / "components" / "runner" / "default.yaml"
assert manifest.exists(), manifest
assert runner_yaml.exists(), runner_yaml
spec = importlib.util.spec_from_file_location("qa_agent_runner_fixture", runner_py)
spec = importlib.util.spec_from_file_location("qa_runner_fixture", runner_py)
module = importlib.util.module_from_spec(spec)
assert spec and spec.loader
spec.loader.exec_module(module)
def context(run_id, text):
return AgentRunContext(
return RunnerContext(
run_id=run_id,
trigger=AgentTrigger(type="message.received", source="webui"),
event=AgentEventContext(event_id=f"evt-{run_id}", event_type="message.received", source="webui"),
@@ -97,7 +108,7 @@ def context(run_id, text):
)
async def collect(text):
runner = module.DefaultAgentRunner()
runner = module.DefaultRunner()
results = []
async for result in runner.run(context(f"run-{len(text)}", text)):
results.append(result)
@@ -107,17 +118,17 @@ async def main():
normal = await collect("hello")
assert len(normal) == 1, normal
assert normal[0].type.value == "run.completed"
assert normal[0].data["message"]["content"] == "QA_AGENT_RUNNER_OK:hello"
assert normal[0].data["message"]["content"] == "QA_RUNNER_OK:hello"
stream = await collect("stream hello")
assert [item.type.value for item in stream] == ["message.delta", "message.delta", "message.delta", "run.completed"]
assert "".join(item.data["chunk"]["content"] for item in stream[:3]) == "QA_AGENT_RUNNER_OK:stream hello"
assert "".join(item.data["chunk"]["content"] for item in stream[:3]) == "QA_RUNNER_OK:stream hello"
failed = await collect("please fail")
assert len(failed) == 1
assert failed[0].type.value == "run.failed"
assert failed[0].data["error"] == "QA_AGENT_RUNNER_CONTROLLED_FAILURE"
print("QA_AGENT_RUNNER_FIXTURE_CONTRACT_OK")
assert failed[0].data["error"] == "QA_RUNNER_CONTROLLED_FAILURE"
print("QA_RUNNER_FIXTURE_CONTRACT_OK")
asyncio.run(main())
`;
@@ -126,18 +137,30 @@ async function main() {
const root = resolve(env.LBS_ROOT || process.cwd());
const caseId = "agent-runner-fixture-contract";
const runId = env.LBS_RUN_ID || `${timestampSlug()}-${caseId}`;
const evidenceDir = resolve(env.LBS_EVIDENCE_DIR || join(root, "reports", "evidence", runId));
const evidenceDir = resolve(
env.LBS_EVIDENCE_DIR || join(root, "reports", "evidence", runId),
);
await mkdir(evidenceDir, { recursive: true });
const startedAt = new Date();
const sdkRepo = resolve(root, env.LANGBOT_PLUGIN_SDK_REPO || "../../langbot-plugin-sdk");
const sdkRepo = resolve(
root,
env.LANGBOT_PLUGIN_SDK_REPO || "../../langbot-plugin-sdk",
);
const sdkSrc = resolve(sdkRepo, "src");
const fixturePath = resolve(root, "skills/langbot-testing/fixtures/plugins/qa-agent-runner");
const fixturePath = resolve(
root,
"skills/langbot-testing/fixtures/plugins/qa-agent-runner",
);
const stdoutLog = join(evidenceDir, "probe-stdout.log");
const stderrLog = join(evidenceDir, "probe-stderr.log");
const automationResultJson = join(evidenceDir, "automation-result.json");
const resultJson = join(evidenceDir, "result.json");
const timeoutMs = Number(env.LANGBOT_AGENT_RUNNER_PROBE_TIMEOUT_MS || "30000");
const command = { executable: "rtk", args: ["uv", "run", "--no-sync", "python", "-c", script, fixturePath], cwd: sdkRepo };
const timeoutMs = Number(env.LANGBOT_RUNNER_PROBE_TIMEOUT_MS || "30000");
const command = {
executable: "rtk",
args: ["uv", "run", "--no-sync", "python", "-c", script, fixturePath],
cwd: sdkRepo,
};
const result = {
source: "automation",
probe: "agent-runner-fixture-contract",
@@ -156,7 +179,12 @@ async function main() {
timeout_ms: timeoutMs,
exit_status: null,
signal: null,
evidence: { stdout_log: stdoutLog, stderr_log: stderrLog, automation_result_json: automationResultJson, result_json: resultJson },
evidence: {
stdout_log: stdoutLog,
stderr_log: stderrLog,
automation_result_json: automationResultJson,
result_json: resultJson,
},
evidence_collected: ["filesystem"],
};
try {
@@ -166,7 +194,9 @@ async function main() {
} else {
const proc = await run(command, timeoutMs, {
...process.env,
PYTHONPATH: [sdkSrc, process.env.PYTHONPATH].filter(Boolean).join(delimiter),
PYTHONPATH: [sdkSrc, process.env.PYTHONPATH]
.filter(Boolean)
.join(delimiter),
UV_CACHE_DIR: env.UV_CACHE_DIR || join(evidenceDir, ".uv-cache"),
});
await writeFile(stdoutLog, proc.stdout, "utf8");
@@ -179,9 +209,12 @@ async function main() {
} else if (proc.timedOut) {
result.status = "fail";
result.reason = `fixture contract probe timed out after ${timeoutMs}ms`;
} else if (proc.status === 0 && proc.stdout.includes("QA_AGENT_RUNNER_FIXTURE_CONTRACT_OK")) {
} else if (
proc.status === 0 &&
proc.stdout.includes("QA_RUNNER_FIXTURE_CONTRACT_OK")
) {
result.status = "pass";
result.reason = "QA AgentRunner fixture contract passed";
result.reason = "QA Runner fixture contract passed";
} else {
result.status = "fail";
result.reason = `fixture contract exited with status ${proc.status}`;
@@ -200,7 +233,9 @@ async function main() {
await writeFile(resultJson, resultText, "utf8");
console.log(JSON.stringify(result, null, 2));
}
process.exit(result.status === "pass" ? 0 : result.status === "env_issue" ? 2 : 1);
process.exit(
result.status === "pass" ? 0 : result.status === "env_issue" ? 2 : 1,
);
}
await main();
@@ -8,7 +8,8 @@ await runPytestProbe({
defaultRepo: "..",
pythonPathEnvKeys: ["LANGBOT_PLUGIN_SDK_REPO"],
defaultPythonPaths: ["../../langbot-plugin-sdk/src"],
description: "LangBot AgentRunner run ledger claim, lease, authorization, and runtime-admin pytest probe.",
description:
"LangBot Runner run ledger claim, lease, authorization, and runtime-admin pytest probe.",
testTargets: [
"tests/unit_tests/agent/test_run_ledger_store.py::test_create_queued_run_claim_renew_release",
"tests/unit_tests/agent/test_run_ledger_store.py::test_expired_claim_can_be_reclaimed",
@@ -7,7 +7,11 @@ import { delimiter, join, resolve } from "node:path";
import { env } from "node:process";
function timestampSlug(date = new Date()) {
return date.toISOString().replace(/\.\d{3}Z$/, "Z").replace(/[^0-9A-Za-z]+/g, "-").replace(/^-|-$/g, "");
return date
.toISOString()
.replace(/\.\d{3}Z$/, "Z")
.replace(/[^0-9A-Za-z]+/g, "-")
.replace(/^-|-$/g, "");
}
function localIsoWithOffset(date = new Date()) {
@@ -51,7 +55,14 @@ function run(command, timeoutMs, childEnv) {
});
child.on("error", (error) => {
clearTimeout(timeout);
resolveDone({ stdout, stderr, error, timedOut, status: null, signal: null });
resolveDone({
stdout,
stderr,
error,
timedOut,
status: null,
signal: null,
});
});
child.on("close", (status, signal) => {
clearTimeout(timeout);
@@ -149,18 +160,23 @@ async function main() {
const root = resolve(env.LBS_ROOT || process.cwd());
const caseId = "agent-runner-ledger-contention";
const runId = env.LBS_RUN_ID || `${timestampSlug()}-${caseId}`;
const evidenceDir = resolve(env.LBS_EVIDENCE_DIR || join(root, "reports", "evidence", runId));
const evidenceDir = resolve(
env.LBS_EVIDENCE_DIR || join(root, "reports", "evidence", runId),
);
await mkdir(evidenceDir, { recursive: true });
const startedAt = new Date();
const langbotRepo = resolve(root, env.LANGBOT_REPO || "..");
const sdkRepo = resolve(root, env.LANGBOT_PLUGIN_SDK_REPO || "../../langbot-plugin-sdk");
const sdkRepo = resolve(
root,
env.LANGBOT_PLUGIN_SDK_REPO || "../../langbot-plugin-sdk",
);
const sdkSrc = resolve(sdkRepo, "src");
const dbPath = join(evidenceDir, "ledger-contention.sqlite3");
const stdoutLog = join(evidenceDir, "probe-stdout.log");
const stderrLog = join(evidenceDir, "probe-stderr.log");
const automationResultJson = join(evidenceDir, "automation-result.json");
const resultJson = join(evidenceDir, "result.json");
const timeoutMs = Number(env.LANGBOT_AGENT_RUNNER_PROBE_TIMEOUT_MS || "30000");
const timeoutMs = Number(env.LANGBOT_RUNNER_PROBE_TIMEOUT_MS || "30000");
const command = {
executable: "rtk",
args: [resolve(langbotRepo, ".venv/bin/python"), "-c", script, dbPath],
@@ -185,7 +201,13 @@ async function main() {
timeout_ms: timeoutMs,
exit_status: null,
signal: null,
evidence: { stdout_log: stdoutLog, stderr_log: stderrLog, database: dbPath, automation_result_json: automationResultJson, result_json: resultJson },
evidence: {
stdout_log: stdoutLog,
stderr_log: stderrLog,
database: dbPath,
automation_result_json: automationResultJson,
result_json: resultJson,
},
evidence_collected: ["filesystem"],
};
try {
@@ -195,7 +217,9 @@ async function main() {
} else {
const proc = await run(command, timeoutMs, {
...process.env,
PYTHONPATH: [sdkSrc, process.env.PYTHONPATH].filter(Boolean).join(delimiter),
PYTHONPATH: [sdkSrc, process.env.PYTHONPATH]
.filter(Boolean)
.join(delimiter),
UV_CACHE_DIR: env.UV_CACHE_DIR || join(evidenceDir, ".uv-cache"),
});
await writeFile(stdoutLog, proc.stdout, "utf8");
@@ -208,7 +232,10 @@ async function main() {
} else if (proc.timedOut) {
result.status = "fail";
result.reason = `ledger contention timed out after ${timeoutMs}ms`;
} else if (proc.status === 0 && proc.stdout.includes("LEDGER_CONTENTION_OK")) {
} else if (
proc.status === 0 &&
proc.stdout.includes("LEDGER_CONTENTION_OK")
) {
result.status = "pass";
result.reason = "ledger contention probe passed";
} else {
@@ -229,7 +256,9 @@ async function main() {
await writeFile(resultJson, resultText, "utf8");
console.log(JSON.stringify(result, null, 2));
}
process.exit(result.status === "pass" ? 0 : result.status === "env_issue" ? 2 : 1);
process.exit(
result.status === "pass" ? 0 : result.status === "env_issue" ? 2 : 1,
);
}
await main();
@@ -7,7 +7,11 @@ import { delimiter, join, resolve } from "node:path";
import { env } from "node:process";
function timestampSlug(date = new Date()) {
return date.toISOString().replace(/\.\d{3}Z$/, "Z").replace(/[^0-9A-Za-z]+/g, "-").replace(/^-|-$/g, "");
return date
.toISOString()
.replace(/\.\d{3}Z$/, "Z")
.replace(/[^0-9A-Za-z]+/g, "-")
.replace(/^-|-$/g, "");
}
function localIsoWithOffset(date = new Date()) {
@@ -55,7 +59,14 @@ function runProcess(command, timeoutMs, childEnv) {
});
child.on("error", (error) => {
clearTimeout(timeout);
resolveDone({ stdout, stderr, error, timedOut, status: null, signal: null });
resolveDone({
stdout,
stderr,
error,
timedOut,
status: null,
signal: null,
});
});
child.on("close", (status, signal) => {
clearTimeout(timeout);
@@ -124,11 +135,16 @@ async function main() {
const root = resolve(env.LBS_ROOT || process.cwd());
const caseId = "agent-runner-ledger-invariants";
const runId = env.LBS_RUN_ID || `${timestampSlug()}-${caseId}`;
const evidenceDir = resolve(env.LBS_EVIDENCE_DIR || join(root, "reports", "evidence", runId));
const evidenceDir = resolve(
env.LBS_EVIDENCE_DIR || join(root, "reports", "evidence", runId),
);
await mkdir(evidenceDir, { recursive: true });
const startedAt = new Date();
const langbotRepo = resolveFromRoot(root, env.LANGBOT_REPO || "..");
const sdkRepo = resolveFromRoot(root, env.LANGBOT_PLUGIN_SDK_REPO || "../../langbot-plugin-sdk");
const sdkRepo = resolveFromRoot(
root,
env.LANGBOT_PLUGIN_SDK_REPO || "../../langbot-plugin-sdk",
);
const sdkSrc = resolve(sdkRepo, "src");
const stdoutLog = join(evidenceDir, "probe-stdout.log");
const stderrLog = join(evidenceDir, "probe-stderr.log");
@@ -139,7 +155,7 @@ async function main() {
args: [resolve(langbotRepo, ".venv/bin/python"), "-c", probeScript],
cwd: langbotRepo,
};
const timeoutMs = Number(env.LANGBOT_AGENT_RUNNER_PROBE_TIMEOUT_MS || "30000");
const timeoutMs = Number(env.LANGBOT_RUNNER_PROBE_TIMEOUT_MS || "30000");
const result = {
source: "automation",
probe: "python-sync",
@@ -174,7 +190,9 @@ async function main() {
} else {
const childEnv = {
...process.env,
PYTHONPATH: [sdkSrc, process.env.PYTHONPATH].filter(Boolean).join(delimiter),
PYTHONPATH: [sdkSrc, process.env.PYTHONPATH]
.filter(Boolean)
.join(delimiter),
UV_CACHE_DIR: env.UV_CACHE_DIR || join(evidenceDir, ".uv-cache"),
};
await mkdir(childEnv.UV_CACHE_DIR, { recursive: true });
@@ -210,7 +228,9 @@ async function main() {
await writeFile(resultJson, resultText, "utf8");
console.log(JSON.stringify(result, null, 2));
}
process.exit(result.status === "pass" ? 0 : result.status === "env_issue" ? 2 : 1);
process.exit(
result.status === "pass" ? 0 : result.status === "env_issue" ? 2 : 1,
);
}
await main();
@@ -7,7 +7,11 @@ import { delimiter, join, resolve } from "node:path";
import { env } from "node:process";
function timestampSlug(date = new Date()) {
return date.toISOString().replace(/\.\d{3}Z$/, "Z").replace(/[^0-9A-Za-z]+/g, "-").replace(/^-|-$/g, "");
return date
.toISOString()
.replace(/\.\d{3}Z$/, "Z")
.replace(/[^0-9A-Za-z]+/g, "-")
.replace(/^-|-$/g, "");
}
function localIsoWithOffset(date = new Date()) {
@@ -51,7 +55,14 @@ function run(command, timeoutMs, childEnv) {
});
child.on("error", (error) => {
clearTimeout(timeout);
resolveDone({ stdout, stderr, error, timedOut, status: null, signal: null });
resolveDone({
stdout,
stderr,
error,
timedOut,
status: null,
signal: null,
});
});
child.on("close", (status, signal) => {
clearTimeout(timeout);
@@ -116,17 +127,22 @@ async function main() {
const root = resolve(env.LBS_ROOT || process.cwd());
const caseId = "agent-runner-ledger-stress";
const runId = env.LBS_RUN_ID || `${timestampSlug()}-${caseId}`;
const evidenceDir = resolve(env.LBS_EVIDENCE_DIR || join(root, "reports", "evidence", runId));
const evidenceDir = resolve(
env.LBS_EVIDENCE_DIR || join(root, "reports", "evidence", runId),
);
await mkdir(evidenceDir, { recursive: true });
const startedAt = new Date();
const langbotRepo = resolve(root, env.LANGBOT_REPO || "..");
const sdkRepo = resolve(root, env.LANGBOT_PLUGIN_SDK_REPO || "../../langbot-plugin-sdk");
const sdkRepo = resolve(
root,
env.LANGBOT_PLUGIN_SDK_REPO || "../../langbot-plugin-sdk",
);
const sdkSrc = resolve(sdkRepo, "src");
const stdoutLog = join(evidenceDir, "probe-stdout.log");
const stderrLog = join(evidenceDir, "probe-stderr.log");
const automationResultJson = join(evidenceDir, "automation-result.json");
const resultJson = join(evidenceDir, "result.json");
const timeoutMs = Number(env.LANGBOT_AGENT_RUNNER_PROBE_TIMEOUT_MS || "30000");
const timeoutMs = Number(env.LANGBOT_RUNNER_PROBE_TIMEOUT_MS || "30000");
const command = {
executable: "rtk",
args: [resolve(langbotRepo, ".venv/bin/python"), "-c", script],
@@ -150,7 +166,12 @@ async function main() {
timeout_ms: timeoutMs,
exit_status: null,
signal: null,
evidence: { stdout_log: stdoutLog, stderr_log: stderrLog, automation_result_json: automationResultJson, result_json: resultJson },
evidence: {
stdout_log: stdoutLog,
stderr_log: stderrLog,
automation_result_json: automationResultJson,
result_json: resultJson,
},
evidence_collected: ["filesystem"],
};
try {
@@ -160,7 +181,9 @@ async function main() {
} else {
const proc = await run(command, timeoutMs, {
...process.env,
PYTHONPATH: [sdkSrc, process.env.PYTHONPATH].filter(Boolean).join(delimiter),
PYTHONPATH: [sdkSrc, process.env.PYTHONPATH]
.filter(Boolean)
.join(delimiter),
UV_CACHE_DIR: env.UV_CACHE_DIR || join(evidenceDir, ".uv-cache"),
});
await writeFile(stdoutLog, proc.stdout, "utf8");
@@ -173,7 +196,10 @@ async function main() {
} else if (proc.timedOut) {
result.status = "fail";
result.reason = `ledger stress timed out after ${timeoutMs}ms`;
} else if (proc.status === 0 && proc.stdout.includes("LEDGER_STRESS_OK")) {
} else if (
proc.status === 0 &&
proc.stdout.includes("LEDGER_STRESS_OK")
) {
result.status = "pass";
result.reason = "ledger stress probe passed";
} else {
@@ -194,7 +220,9 @@ async function main() {
await writeFile(resultJson, resultText, "utf8");
console.log(JSON.stringify(result, null, 2));
}
process.exit(result.status === "pass" ? 0 : result.status === "env_issue" ? 2 : 1);
process.exit(
result.status === "pass" ? 0 : result.status === "env_issue" ? 2 : 1,
);
}
await main();
@@ -6,9 +6,10 @@ await runPytestProbe({
caseId: "agent-runner-runtime-chaos",
repoEnvKey: "LANGBOT_PLUGIN_SDK_REPO",
defaultRepo: "../../langbot-plugin-sdk",
description: "LangBot plugin SDK AgentRunner runtime failure, timeout, forwarding, and pull API pytest probe.",
description:
"LangBot plugin SDK Runner runtime failure, timeout, forwarding, and pull API pytest probe.",
testTargets: [
"tests/runtime/plugin/test_mgr_agent_runner.py",
"tests/runtime/plugin/test_mgr_runner.py",
"tests/runtime/test_pull_api_handlers.py",
],
});
@@ -5,7 +5,10 @@ import { basename, delimiter, join, resolve } from "node:path";
import { env } from "node:process";
function loadEnvDefaults(root) {
for (const path of [join(root, "skills/.env"), join(root, "skills/.env.local")]) {
for (const path of [
join(root, "skills/.env"),
join(root, "skills/.env.local"),
]) {
if (!existsSync(path)) continue;
for (const rawLine of readFileSync(path, "utf8").split(/\r?\n/)) {
const line = rawLine.trim();
@@ -14,13 +17,20 @@ function loadEnvDefaults(root) {
if (sep === -1) continue;
const key = line.slice(0, sep).trim();
if (env[key]) continue;
env[key] = line.slice(sep + 1).trim().replace(/^["']|["']$/g, "");
env[key] = line
.slice(sep + 1)
.trim()
.replace(/^["']|["']$/g, "");
}
}
}
function timestampSlug(date = new Date()) {
return date.toISOString().replace(/\.\d{3}Z$/, "Z").replace(/[^0-9A-Za-z]+/g, "-").replace(/^-|-$/g, "");
return date
.toISOString()
.replace(/\.\d{3}Z$/, "Z")
.replace(/[^0-9A-Za-z]+/g, "-")
.replace(/^-|-$/g, "");
}
function localIsoWithOffset(date = new Date()) {
@@ -88,7 +98,14 @@ async function runProcess(command, timeoutMs, childEnv) {
});
child.on("error", (error) => {
clearTimeout(timeout);
resolveDone({ stdout, stderr, error, timedOut, status: null, signal: null });
resolveDone({
stdout,
stderr,
error,
timedOut,
status: null,
signal: null,
});
});
child.on("close", (status, signal) => {
clearTimeout(timeout);
@@ -109,10 +126,14 @@ export async function runPytestProbe({
}) {
const root = resolve(env.LBS_ROOT || process.cwd());
loadEnvDefaults(root);
const resolvedTimeoutMs = Number(timeoutMs || env.LANGBOT_AGENT_RUNNER_PROBE_TIMEOUT_MS || "180000");
const resolvedTimeoutMs = Number(
timeoutMs || env.LANGBOT_RUNNER_PROBE_TIMEOUT_MS || "180000",
);
const runId = env.LBS_RUN_ID || `${timestampSlug()}-${caseId}`;
const evidenceDir = resolve(env.LBS_EVIDENCE_DIR || join(root, "reports", "evidence", runId));
const evidenceDir = resolve(
env.LBS_EVIDENCE_DIR || join(root, "reports", "evidence", runId),
);
await mkdir(evidenceDir, { recursive: true });
const uvCacheDir = env.UV_CACHE_DIR || join(evidenceDir, ".uv-cache");
await mkdir(uvCacheDir, { recursive: true });
@@ -171,14 +192,21 @@ export async function runPytestProbe({
result.status = "env_issue";
result.reason = `${repoEnvKey || "repo"} did not resolve to an existing directory: ${repoPath}`;
} else {
const missingTargets = testTargets.filter((target) => !existsSync(join(repoPath, target.split("::")[0])));
const missingTargets = testTargets.filter(
(target) => !existsSync(join(repoPath, target.split("::")[0])),
);
if (missingTargets.length > 0) {
result.status = "env_issue";
result.reason = `pytest target file(s) not found in ${basename(repoPath)}: ${missingTargets.join(", ")}`;
} else {
const childEnv = { ...process.env, UV_CACHE_DIR: uvCacheDir };
if (pythonPaths.length > 0) {
childEnv.PYTHONPATH = [pythonPaths.join(delimiter), childEnv.PYTHONPATH].filter(Boolean).join(delimiter);
childEnv.PYTHONPATH = [
pythonPaths.join(delimiter),
childEnv.PYTHONPATH,
]
.filter(Boolean)
.join(delimiter);
}
const proc = await runProcess(command, resolvedTimeoutMs, childEnv);
result.exit_status = proc.status;
@@ -195,7 +223,11 @@ export async function runPytestProbe({
} else if (proc.status === 0) {
result.status = "pass";
result.reason = `pytest passed for ${testTargets.join(", ")}.`;
} else if (/command not found|no such file or directory|executable file not found/i.test(`${proc.stdout}\n${proc.stderr}`)) {
} else if (
/command not found|no such file or directory|executable file not found/i.test(
`${proc.stdout}\n${proc.stderr}`,
)
) {
result.status = "env_issue";
result.reason = `pytest command could not run in ${repoPath}. See ${stdoutLog} and ${stderrLog}.`;
} else {
@@ -1,6 +1,6 @@
# AgentRunner QA Workflow
# Runner QA Workflow
Use this workflow when an agent finishes AgentRunner-related code and enters a
Use this workflow when an agent finishes Runner-related code and enters a
test phase.
## Order
@@ -25,7 +25,7 @@ test phase.
backend is available and installing the QA fixture is acceptable.
- `rtk bin/lbs test run agent-runner-qa-debug-chat --dry-run` when WebUI live
execution needs deterministic coverage without a model provider. This
case runs its setup automation first: install the QA AgentRunner fixture,
case runs its setup automation first: install the QA Runner fixture,
create/update the QA pipeline, write the case-specific pipeline env, then
execute Debug Chat.
- `rtk bin/lbs test run agent-runner-ledger-invariants --dry-run`
@@ -51,8 +51,8 @@ only to review or adjust the generated list.
| --- | --- | --- |
| `LangBot/src/langbot/pkg/agent/runner/*`, `tests/unit_tests/agent/test_result_normalizer.py`, protocol/result/context/resource builders | `rtk bin/lbs test run agent-runner-fixture-contract --dry-run`; `rtk bin/lbs test run agent-runner-behavior-matrix --dry-run`; targeted LangBot unit tests for touched files | Result shape, user-visible runner output, or Debug Chat delivery changed: add `pipeline-debug-chat` or `local-agent-basic-debug-chat`. |
| `LangBot/src/langbot/pkg/entity/persistence/agent_run.py`, `run_journal.py`, run ledger store/API/auth tests, claim/lease/status code | `rtk bin/lbs test run agent-runner-ledger-invariants --dry-run`; `rtk bin/lbs test run agent-runner-ledger-stress --dry-run`; `rtk bin/lbs test run agent-runner-ledger-contention --dry-run`; `rtk bin/lbs test run agent-runner-async-db-readiness --dry-run` before `rtk bin/lbs test run agent-runner-ledger-concurrency --dry-run` | Debug Chat run lifecycle, resume, or visible completion changed: add `local-agent-basic-debug-chat`. |
| `langbot-plugin-sdk/src/langbot_plugin/api/entities/builtin/agent_runner/*`, `api/proxies/agent_run_api.py`, runtime pull handlers, plugin manager/runtime IO | `rtk bin/lbs test run agent-runner-runtime-chaos --dry-run`; `rtk bin/lbs test run agent-runner-behavior-matrix --dry-run`; targeted SDK pytest | Runtime delivery or tool-call surface changed: add `agent-runner-release-preflight`, then `local-agent-basic-debug-chat`. |
| `langbot-agent-runner/*/components/agent_runner/*`, external runner daemon/client code, ACP/Codex/Claude runner command wrappers | Repo-local targeted tests; `rtk bin/lbs test run agent-runner-runtime-chaos --dry-run`; `rtk bin/lbs test run agent-runner-release-preflight --dry-run` | ACP or external coding runner behavior changed: add `acp-agent-runner-debug-chat`. |
| `langbot-plugin-sdk/src/langbot_plugin/api/entities/builtin/runner/*`, `api/proxies/agent_run_api.py`, runtime pull handlers, plugin manager/runtime IO | `rtk bin/lbs test run agent-runner-runtime-chaos --dry-run`; `rtk bin/lbs test run agent-runner-behavior-matrix --dry-run`; targeted SDK pytest | Runtime delivery or tool-call surface changed: add `agent-runner-release-preflight`, then `local-agent-basic-debug-chat`. |
| `langbot-agent-runner/*/components/runner/*`, external runner daemon/client code, ACP/Codex/Claude runner command wrappers | Repo-local targeted tests; `rtk bin/lbs test run agent-runner-runtime-chaos --dry-run`; `rtk bin/lbs test run agent-runner-release-preflight --dry-run` | ACP or external coding runner behavior changed: add `acp-agent-runner-debug-chat`. |
| Prompt preprocessing, effective prompt, pipeline AI config, runner binding/default runner migration | `rtk bin/lbs test run agent-runner-behavior-matrix --dry-run`; targeted LangBot pipeline/agent tests | The runner reads host-provided prompt or saved runner config: add `local-agent-effective-prompt-debug-chat`. |
| Context window, transcript, history/event state, compaction, checkpoint/steering | `rtk bin/lbs test run agent-runner-behavior-matrix --dry-run`; targeted LangBot agent state/context tests | Multi-turn memory, compaction, or steering behavior changed: add `local-agent-context-compaction-debug-chat` and, for steering-specific changes, `local-agent-steering-debug-chat`. |
| Plugin tool authorization, host tool listing, MCP tool bridge, function-call conversion | `rtk bin/lbs test run agent-runner-behavior-matrix --dry-run`; targeted plugin/MCP/tool tests | Tool execution is user-visible: add `local-agent-plugin-tool-call-debug-chat`; for MCP-specific changes add `mcp-stdio-register` then `mcp-stdio-tool-call`. |
@@ -1,4 +1,4 @@
# Agent Runner Release Gate
# Runner Release Gate
Use this reference when judging whether runner externalization is release-ready. The goal is not to enumerate every possible prompt. The gate covers product abilities and trust boundaries with deterministic normal-path cases, then leaves rare negative branches to unit and contract tests.
@@ -35,7 +35,7 @@ For a quick early blocker check, run:
rtk bin/lbs test run agent-runner-release-preflight --dry-run
```
For the code-level AgentRunner probes, run:
For the code-level Runner probes, run:
```bash
rtk bin/lbs test run agent-runner-behavior-matrix --dry-run
@@ -70,7 +70,7 @@ API integration gate, not a Debug Chat execution proof.
`agent-runner-qa-debug-chat` is the deterministic live execution proof. It uses
a pipeline created by `scripts/e2e/ensure-qa-agent-runner-pipeline.mjs` and
expects Debug Chat to return `QA_AGENT_RUNNER_OK:<input>` through
expects Debug Chat to return `QA_RUNNER_OK:<input>` through
`plugin:qa/agent-runner/default`.
`agent-runner-ledger-invariants` is the fast Host ledger probe. It uses
@@ -97,7 +97,7 @@ If it times out before any test result and a direct `aiosqlite.connect()` script
also hangs, classify the run with troubleshooting id
`aiosqlite-connect-hangs` instead of treating it as a browser E2E failure.
`agent-runner-runtime-chaos` runs SDK AgentRunner runtime and pull API handler
`agent-runner-runtime-chaos` runs SDK Runner runtime and pull API handler
tests from `LANGBOT_PLUGIN_SDK_REPO` or `../langbot-plugin-sdk`.
Each probe writes `automation-result.json` and probe logs under
`LBS_EVIDENCE_DIR`.
@@ -108,9 +108,9 @@ Each probe writes `automation-result.json` and probe logs under
| --- | --- | --- |
| Authenticated WebUI session | `webui-login-state`, `agent-runner-release-preflight` | The browser profile can operate the same backend that later cases use. |
| Generic Pipeline Debug Chat | `pipeline-debug-chat` | The WebUI Debug Chat path itself works before runner-specific failures are diagnosed. |
| Deterministic QA runner install | `agent-runner-live-install` | A local `.lbpkg` AgentRunner package can install and register a runner. |
| Deterministic QA runner install | `agent-runner-live-install` | A local `.lbpkg` Runner package can install and register a runner. |
| Deterministic QA runner Debug Chat | `agent-runner-qa-debug-chat` | The installed QA runner executes through WebUI Debug Chat without a model provider. |
| Required runner plugins | `agent-runner-release-preflight` | `langbot-team/LocalAgent` and `langbot-team/ACPAgentRunner` are visible to the host. |
| Required runner plugins | `agent-runner-release-preflight` | `langbot-team/LocalAgent` and `langbot-team/ACPRunner` are visible to the host. |
| Required QA plugin tools | `plugin-e2e-smoke`, `agent-runner-release-preflight`, `qa-plugin-smoke-live-install` | The deterministic `qa_plugin_echo` and `qa_plugin_fail` tools are exposed before tool-loop and tool-error cases start. |
| Knowledge base fixture | `langrag-kb-retrieve`, `local-agent-rag-debug-chat` | LangRAG data is queryable and the runner inserts retrieved context. |
| Effective prompt bridge | `local-agent-effective-prompt-debug-chat` | Host prompt preprocessing reaches the runner. |
@@ -149,7 +149,7 @@ rtk uv run pytest -q
# langbot-plugin-sdk
rtk uv run pytest -q
# langbot-skills saved AgentRunner probes
# langbot-skills saved Runner probes
rtk bin/lbs test run agent-runner-behavior-matrix --dry-run
rtk bin/lbs test run agent-runner-ledger-invariants --dry-run
rtk bin/lbs test run agent-runner-ledger-stress --dry-run
@@ -1,4 +1,4 @@
# Dify AgentRunner
# Dify Runner
Use this reference when validating `langbot-team/DifyAgent` through LangBot WebUI.
@@ -1,4 +1,4 @@
# Local Agent Runner Coverage
# Local Runner Coverage
Use this matrix when judging whether the external `langbot-team/LocalAgent` plugin still behaves like the old built-in local-agent runner.
@@ -10,7 +10,7 @@ The QA target is end-to-end behavior. UI cases prove the host, SDK, plugin runti
- `LangBot/src/langbot/pkg/agent/runner/pipeline_adapter.py` adapts Pipeline-only fields into `ctx.adapter.extra.prompt`, `ctx.adapter.extra.params`, and optional `ctx.bootstrap.messages`.
- `LangBot/src/langbot/pkg/agent/runner/resource_builder.py` authorizes models, fallback models, rerank models, tools, and knowledge bases for the current run.
- `LangBot/src/langbot/pkg/plugin/handler.py` validates run-scoped model/tool/rerank access and calls the host model provider or tool manager with the current query.
- `langbot-local-agent/components/agent_runner/default.py` selects streaming or non-streaming execution, retrieves RAG context, builds messages, invokes models with fallback, and runs tool loops.
- `langbot-local-agent/components/runner/default.py` selects streaming or non-streaming execution, retrieves RAG context, builds messages, invokes models with fallback, and runs tool loops.
- `langbot-local-agent/pkg/messages.py` prefers the host effective prompt from `ctx.adapter.extra.prompt`, uses `ctx.bootstrap.messages` only as a small bootstrap window, and preserves structured/multimodal input while inserting RAG context.
TODO: Treat `ctx.adapter.extra.prompt` as a temporary Pipeline bridge for old
@@ -1,10 +1,10 @@
# Local Agent Runner
# Local Runner
Use this reference when validating the pluginized `langbot-team/LocalAgent` runner through the WebUI.
The goal is behavior parity with the old built-in local-agent runner. The code does not need to be identical, but the visible behavior should match: effective prompt, current input, history, model selection and fallback, tool calling, knowledge retrieval, multimodal input, streaming and non-streaming output all have to reach the runner through the host and SDK.
For path-by-path coverage, read [Local Agent Runner Coverage](local-agent-runner-coverage.md).
For path-by-path coverage, read [Local Runner Coverage](local-agent-runner-coverage.md).
## Main Surface
@@ -35,7 +35,7 @@ Measure user experience and internal composition separately:
- WebUI load and interaction latency.
- Debug Chat send-to-first-visible-token and send-to-completion latency.
- Pipeline, RAG, plugin runtime, MCP, AgentRunner, and persistence segment
- Pipeline, RAG, plugin runtime, MCP, Runner, and persistence segment
latency.
- Queue wait time, concurrency, throughput, timeout rate, and p95/p99 latency.
- Startup, plugin install, knowledge-base ingestion, migration, and recovery
@@ -33,7 +33,7 @@ Both external runners receive the same host-generated gateway `AgentMCPServerCon
This is a **runner-plugin transport detail, not a host all-tool-branch issue** — proven by **both** runners discovering skills end-to-end with the unmodified branch (see cases below).
> **Correction (2026-06-22).** An earlier revision of this doc claimed acp was "blocked" on remote-ssh and *required* `langbot-assets-gateway-public-url`, based on a run that returned `PROBEDONE 0 0` / timeout. That was an **environment artifact, not an acp defect**: a duplicate backend instance (a second checkout `LangBot-master/` whose box runtime contended for the same `--ws-control-port 5410`) plus a wedged plugin runtime (host `emit_event` / `list_agent_runners` action calls timing out with `ActionCallTimeoutError`). Re-run on a clean single-instance runtime, **acp passes via the reverse tunnel with no `public-url`** (`PROBEDONE 1 17`, 824s).
> **Correction (2026-06-22).** An earlier revision of this doc claimed acp was "blocked" on remote-ssh and *required* `langbot-assets-gateway-public-url`, based on a run that returned `PROBEDONE 0 0` / timeout. That was an **environment artifact, not an acp defect**: a duplicate backend instance (a second checkout `LangBot-master/` whose box runtime contended for the same `--ws-control-port 5410`) plus a wedged plugin runtime (host `emit_event` / `list_runners` action calls timing out with `ActionCallTimeoutError`). Re-run on a clean single-instance runtime, **acp passes via the reverse tunnel with no `public-url`** (`PROBEDONE 1 17`, 824s).
- **Lifecycle**: discover → activate → operate (native exec under the activated mount path) → register.
- **Backend**: docker · nsjail · e2b.
@@ -12,7 +12,7 @@ Date: 2026-05-16
### Symptom
The WebUI can send a Debug Chat message, but the bot response is missing or says `Agent runner temporarily unavailable`. Backend logs may include `Action list_plugins call timed out`, `Action list_agent_runners call timed out`, or `Action invoke_llm_stream call timed out`.
The WebUI can send a Debug Chat message, but the bot response is missing or says `Agent runner temporarily unavailable`. Backend logs may include `Action list_plugins call timed out`, `Action list_runners call timed out`, or `Action invoke_llm_stream call timed out`.
### Likely Cause
@@ -78,7 +78,7 @@ Structured entry: `../troubleshooting/marketplace-network-flaky.yaml`
Marketplace icon/tag/recommendation requests can fail while plugin cards are already visible. Retry first, and use backend component endpoints only to confirm installation results.
## agent-runner-actor-context-fields: AgentRunner reads old actor fields
## agent-runner-actor-context-fields: Runner reads old actor fields
Structured entry: `../troubleshooting/agent-runner-actor-context-fields.yaml`
@@ -1,6 +1,6 @@
# Workspace Release Testing
Use the workspace gates when changes span LangBot core, the plugin SDK, AgentRunner, or multiple first-party plugins.
Use the workspace gates when changes span LangBot core, the plugin SDK, Runner, or multiple first-party plugins.
## Cost Ladder
@@ -1,6 +1,6 @@
id: langbot-workspace-release-gate
title: "LangBot workspace top-down release gate"
description: "Broad release gate combining deterministic repository contracts with representative browser workflows, plugin pages, RAG/parser, EBA, external AgentRunner, and one complex LocalAgent task."
description: "Broad release gate combining deterministic repository contracts with representative browser workflows, plugin pages, RAG/parser, EBA, external Runner, and one complex LocalAgent task."
type: release_gate
priority: p0
tags:
@@ -1,9 +1,9 @@
id: agent-runner-actor-context-fields
title: "AgentRunner reads old actor.type and actor.id fields"
title: "Runner reads old actor.type and actor.id fields"
date: 2026-05-17
symptoms:
- "Pipeline Debug Chat shows Agent runner execution failed."
- "Backend logs show an AttributeError from an AgentRunner plugin."
- "Backend logs show an AttributeError from an Runner plugin."
patterns:
- "AttributeError: 'ActorContext' object has no attribute 'type'"
- "AttributeError: 'ActorContext' object has no attribute 'id'"
@@ -15,8 +15,8 @@ fix_steps:
- "Update runner code to read actor.actor_type and actor.actor_id."
- "Keep getattr fallback to type/id only if compatibility with older host data is required."
- "Restart LangBot or the plugin runtime so the updated plugin code is loaded."
- "Add a regression test that builds AgentRunContext with ActorContext(actor_type=..., actor_id=...)."
verification: "Run dify-agent-debug-chat or another AgentRunner Debug Chat and confirm the assistant/bot message contains the expected sentinel while backend logs show Streaming completed."
- "Add a regression test that builds RunnerContext with ActorContext(actor_type=..., actor_id=...)."
verification: "Run dify-agent-debug-chat or another Runner Debug Chat and confirm the assistant/bot message contains the expected sentinel while backend logs show Streaming completed."
related_cases:
- dify-agent-debug-chat
- pipeline-debug-chat
@@ -2,7 +2,7 @@ id: aiosqlite-connect-hangs
title: "aiosqlite connect hangs before ledger pytest starts"
category: env_issue
symptoms:
- "AgentRunner ledger pytest probe times out after collecting tests but before reporting a test result."
- "Runner ledger pytest probe times out after collecting tests but before reporting a test result."
- "pytest stdout stops at a line like tests/unit_tests/agent/test_run_ledger_store.py."
- "A direct aiosqlite.connect(':memory:') script prints its first line and then hangs."
patterns:
@@ -1,5 +1,5 @@
id: ambiguous-runner-default-label
title: "AgentRunner selector shows multiple Default or 默认 options"
title: "Runner selector shows multiple Default or 默认 options"
date: 2026-05-17
symptoms:
- "The Pipeline AI runner selector shows multiple options named Default or 默认."
@@ -9,7 +9,7 @@ patterns:
- "label.zh_Hans: 默认"
- "label.en_US: Default"
likely_causes:
- "AgentRunner component ids are commonly named default, but the user-facing metadata.label was also left generic."
- "Runner component ids are commonly named default, but the user-facing metadata.label was also left generic."
- "The frontend displays metadata.label as the primary option label."
fix_steps:
- "Keep metadata.name as default if the plugin component id is intended to remain stable."
@@ -8,7 +8,7 @@ symptoms:
- "Knowledge sidebar or plugin sidebar loading may hang or time out."
patterns:
- "Action list_plugins call timed out"
- "Action list_agent_runners call timed out"
- "Action list_runners call timed out"
- "Action invoke_llm_stream call timed out"
- "All models failed during streaming setup"
- "Failed to fetch plugins for sidebar"
+65 -34
View File
@@ -4,17 +4,22 @@ import { loadFixtureItems } from "../fixtures.ts";
import { dirname, join } from "node:path";
import { existsSync, readFileSync } from "node:fs";
function fixtureRows(root: string, skill: string | undefined): ReturnType<typeof loadFixtureItems> {
function fixtureRows(
root: string,
skill: string | undefined,
): ReturnType<typeof loadFixtureItems> {
return loadFixtureItems(root, skill);
}
function qaAgentRunnerSourceFindings(item: ReturnType<typeof loadFixtureItems>["items"][number]) {
if (!item.checks.includes("qa_agent_runner_source") || !item.exists) return [];
function qaRunnerSourceFindings(
item: ReturnType<typeof loadFixtureItems>["items"][number],
) {
if (!item.checks.includes("qa_runner_source") || !item.exists) return [];
const root = dirname(item.absolute_path);
const required = [
"main.py",
"components/agent_runner/default.yaml",
"components/agent_runner/default.py",
"components/runner/default.yaml",
"components/runner/default.py",
"assets/icon.svg",
];
const missing = required
@@ -28,15 +33,21 @@ function qaAgentRunnerSourceFindings(item: ReturnType<typeof loadFixtureItems>["
if (missing.length > 0) return missing;
const manifest = readFileSync(item.absolute_path, "utf8");
const runnerYaml = readFileSync(join(root, "components/agent_runner/default.yaml"), "utf8");
const runnerPy = readFileSync(join(root, "components/agent_runner/default.py"), "utf8");
const runnerYaml = readFileSync(
join(root, "components/runner/default.yaml"),
"utf8",
);
const runnerPy = readFileSync(
join(root, "components/runner/default.py"),
"utf8",
);
const requiredText = [
[manifest, "AgentRunner", "manifest.yaml"],
[manifest, "QAAgentRunnerPlugin", "manifest.yaml"],
[runnerYaml, "kind: AgentRunner", "components/agent_runner/default.yaml"],
[runnerYaml, "DefaultAgentRunner", "components/agent_runner/default.yaml"],
[runnerPy, "QA_AGENT_RUNNER_OK", "components/agent_runner/default.py"],
[runnerPy, "QA_AGENT_RUNNER_CONTROLLED_FAILURE", "components/agent_runner/default.py"],
[manifest, "Runner", "manifest.yaml"],
[manifest, "QARunnerPlugin", "manifest.yaml"],
[runnerYaml, "kind: Runner", "components/runner/default.yaml"],
[runnerYaml, "DefaultRunner", "components/runner/default.yaml"],
[runnerPy, "QA_RUNNER_OK", "components/runner/default.py"],
[runnerPy, "QA_RUNNER_CONTROLLED_FAILURE", "components/runner/default.py"],
];
return requiredText
.filter(([text, needle]) => !text.includes(needle))
@@ -49,16 +60,22 @@ function qaAgentRunnerSourceFindings(item: ReturnType<typeof loadFixtureItems>["
}));
}
function zipPackageFindings(item: ReturnType<typeof loadFixtureItems>["items"][number]) {
function zipPackageFindings(
item: ReturnType<typeof loadFixtureItems>["items"][number],
) {
if (!item.checks.includes("zip_package") || !item.exists) return [];
const header = readFileSync(item.absolute_path).subarray(0, 4).toString("binary");
const header = readFileSync(item.absolute_path)
.subarray(0, 4)
.toString("binary");
if (header === "PK\u0003\u0004" || header === "PK\u0005\u0006") return [];
return [{
severity: "fail",
kind: "fixture_check_invalid_zip",
id: item.id,
path: item.path,
}];
return [
{
severity: "fail",
kind: "fixture_check_invalid_zip",
id: item.id,
path: item.path,
},
];
}
export function commandFixtureList(ctx: CommandContext): number {
@@ -72,14 +89,16 @@ export function commandFixtureList(ctx: CommandContext): number {
}
for (const item of result.items) {
console.log([
item.skill,
item.id,
item.kind,
item.exists ? "present" : "missing",
item.path,
item.title,
].join("\t"));
console.log(
[
item.skill,
item.id,
item.kind,
item.exists ? "present" : "missing",
item.path,
item.title,
].join("\t"),
);
}
for (const error of result.errors) console.error(`ERROR: ${error}`);
return result.errors.length > 0 ? 1 : 0;
@@ -90,7 +109,11 @@ export function commandFixtureCheck(ctx: CommandContext): number {
const skill = positional[0];
const result = fixtureRows(ctx.root, skill);
const findings = [
...result.errors.map((error) => ({ severity: "fail", kind: "invalid_manifest", detail: error })),
...result.errors.map((error) => ({
severity: "fail",
kind: "invalid_manifest",
detail: error,
})),
...result.items
.filter((item) => !item.exists)
.map((item) => ({
@@ -100,11 +123,13 @@ export function commandFixtureCheck(ctx: CommandContext): number {
path: item.path,
absolute_path: item.absolute_path,
})),
...result.items.flatMap(qaAgentRunnerSourceFindings),
...result.items.flatMap(qaRunnerSourceFindings),
...result.items.flatMap(zipPackageFindings),
];
const report = {
status: findings.some((finding) => finding.severity === "fail") ? "fail" : "pass",
status: findings.some((finding) => finding.severity === "fail")
? "fail"
: "pass",
fixture_count: result.items.length,
findings,
fixtures: result.items,
@@ -120,12 +145,18 @@ export function commandFixtureCheck(ctx: CommandContext): number {
console.log("");
console.log("## Fixtures");
for (const item of result.items) {
console.log(`- ${item.id}: ${item.exists ? "present" : "missing"} (${item.path})`);
console.log(
`- ${item.id}: ${item.exists ? "present" : "missing"} (${item.path})`,
);
}
console.log("");
console.log("## Findings");
if (findings.length === 0) console.log("- None.");
else for (const finding of findings) console.log(`- [${finding.severity}] ${finding.kind}: ${"detail" in finding ? finding.detail : finding.id}`);
else
for (const finding of findings)
console.log(
`- [${finding.severity}] ${finding.kind}: ${"detail" in finding ? finding.detail : finding.id}`,
);
}
return report.status === "pass" ? 0 : 1;
File diff suppressed because it is too large Load Diff
+151 -71
View File
@@ -51,12 +51,22 @@ import { commandValidate } from "../src/commands/validate.ts";
import { commandIndex } from "../src/commands/skill.ts";
import { loadEnv, parseFrontmatter } from "../src/fs.ts";
test('frontmatter preserves metadata and body with LF and CRLF checkouts', () => {
for (const newline of ['\n', '\r\n']) {
const source = ['---', 'name: example', 'description: "Example skill"', '---', '# Body', ''].join(newline);
test("frontmatter preserves metadata and body with LF and CRLF checkouts", () => {
for (const newline of ["\n", "\r\n"]) {
const source = [
"---",
"name: example",
'description: "Example skill"',
"---",
"# Body",
"",
].join(newline);
const parsed = parseFrontmatter(source);
assert.deepEqual(parsed.meta, { name: 'example', description: 'Example skill' });
assert.equal(parsed.body, '# Body' + newline);
assert.deepEqual(parsed.meta, {
name: "example",
description: "Example skill",
});
assert.equal(parsed.body, "# Body" + newline);
}
});
import { repoRoot } from "../src/cli.ts";
@@ -123,19 +133,25 @@ test("clickFirstVisible waits for a later visible DOM match", async () => {
let clickedIndex = -1;
const emptyLocator = {
count: async () => 0,
nth: () => { throw new Error("empty locator has no children"); },
nth: () => {
throw new Error("empty locator has no children");
},
};
const textLocator = {
count: async () => 2,
nth: (index: number) => ({
isVisible: async () => index === 1 && pollCount >= 1,
click: async () => { clickedIndex = index; },
click: async () => {
clickedIndex = index;
},
}),
};
const page = {
getByRole: () => emptyLocator,
getByText: () => textLocator,
waitForTimeout: async () => { pollCount += 1; },
waitForTimeout: async () => {
pollCount += 1;
},
};
const clicked = await clickFirstVisible(page, ["Debug Chat"], 1_000);
@@ -349,7 +365,9 @@ test("apiJson bootstraps and sends the selected Workspace for scoped APIs", asyn
return new Response(
JSON.stringify({
code: 0,
data: { workspaces: [{ workspace: { uuid: "workspace-api-test" } }] },
data: {
workspaces: [{ workspace: { uuid: "workspace-api-test" } }],
},
}),
{ status: 200 },
);
@@ -359,15 +377,16 @@ test("apiJson bootstraps and sends the selected Workspace for scoped APIs", asyn
});
}) as typeof fetch;
const response = await apiJson(
"http://127.0.0.1:5300",
"/api/v1/tools",
{ token: "workspace-api-token" },
);
const response = await apiJson("http://127.0.0.1:5300", "/api/v1/tools", {
token: "workspace-api-token",
});
assert.equal(response.status, 200);
assert.equal(requests.length, 2);
assert.equal(requests[0].url, "http://127.0.0.1:5300/api/v1/workspaces/bootstrap");
assert.equal(
requests[0].url,
"http://127.0.0.1:5300/api/v1/workspaces/bootstrap",
);
assert.equal(requests[0].headers["X-Workspace-Id"], undefined);
assert.equal(requests[1].headers["X-Workspace-Id"], "workspace-api-test");
} finally {
@@ -623,9 +642,7 @@ test("index includes case summaries for agent discovery", () => {
}) =>
item.id === "agent-runner-qa-debug-chat" &&
item.setup_automation.includes("case:agent-runner-live-install") &&
item.setup_provides_env.includes(
"LANGBOT_QA_AGENT_RUNNER_PIPELINE_URL",
),
item.setup_provides_env.includes("LANGBOT_QA_RUNNER_PIPELINE_URL"),
),
);
assert.ok(
@@ -1908,12 +1925,12 @@ test("fixture check reports missing manifest paths", () => {
}
});
test("fixture check verifies QA AgentRunner source shape", () => {
test("fixture check verifies QA Runner source shape", () => {
const tmp = mkdtempSync(join(tmpdir(), "lbs-fixture-check-"));
try {
const skillDir = join(tmp, "skills", "langbot-testing");
const fixtureDir = join(skillDir, "fixtures", "plugins", "qa-agent-runner");
mkdirSync(join(fixtureDir, "components", "agent_runner"), {
mkdirSync(join(fixtureDir, "components", "runner"), {
recursive: true,
});
writeFileSync(
@@ -1925,15 +1942,15 @@ test("fixture check verifies QA AgentRunner source shape", () => {
JSON.stringify([
{
id: "qa-agent-runner-source",
title: "QA AgentRunner",
title: "QA Runner",
path: "fixtures/plugins/qa-agent-runner/manifest.yaml",
checks: ["exists", "qa_agent_runner_source"],
checks: ["exists", "qa_runner_source"],
},
]),
);
writeFileSync(
join(fixtureDir, "manifest.yaml"),
"spec:\n components:\n AgentRunner: {}\nexecution:\n python:\n attr: QAAgentRunnerPlugin\n",
"spec:\n components:\n Runner: {}\nexecution:\n python:\n attr: QARunnerPlugin\n",
);
const result = capture(() =>
@@ -1949,7 +1966,7 @@ test("fixture check verifies QA AgentRunner source shape", () => {
report.findings.some(
(finding: { kind?: string; path?: string }) =>
finding.kind === "fixture_check_missing_file" &&
finding.path?.endsWith("components/agent_runner/default.py"),
finding.path?.endsWith("components/runner/default.py"),
),
);
} finally {
@@ -1957,7 +1974,7 @@ test("fixture check verifies QA AgentRunner source shape", () => {
}
});
test("fixture check accepts complete QA AgentRunner source shape", () => {
test("fixture check accepts complete QA Runner source shape", () => {
const result = capture(() =>
commandFixtureCheck(ctx(["fixture", "check", "langbot-testing", "--json"])),
);
@@ -1967,7 +1984,7 @@ test("fixture check accepts complete QA AgentRunner source shape", () => {
report.fixtures.some(
(item: { id: string; checks: string[] }) =>
item.id === "qa-agent-runner-source" &&
item.checks.includes("qa_agent_runner_source"),
item.checks.includes("qa_runner_source"),
),
);
});
@@ -2091,9 +2108,17 @@ test("debug chat classifier distinguishes new failure signals from old history",
test("debug chat outcome wait can stop on a new failure signal", () => {
const baselines = [{ signal: "runner.timeout", count: 1 }];
assert.equal(hasDebugChatOutcome("old runner.timeout", "EXPECTED", 1, baselines), false);
assert.equal(
hasDebugChatOutcome("old runner.timeout\nnew runner.timeout", "EXPECTED", 1, baselines),
hasDebugChatOutcome("old runner.timeout", "EXPECTED", 1, baselines),
false,
);
assert.equal(
hasDebugChatOutcome(
"old runner.timeout\nnew runner.timeout",
"EXPECTED",
1,
baselines,
),
true,
);
assert.equal(hasDebugChatOutcome("EXPECTED", "EXPECTED", 1, baselines), true);
@@ -2215,7 +2240,13 @@ test("debug chat classifier rejects a matching assistant message that is not fin
});
test("debug chat classifier accepts formatted responses containing every required fragment", () => {
const expectedTexts = ["MULTITOOL_COMBO_FINAL", "passcode-6718", "rag-7421", "tool-a", "tool-b"];
const expectedTexts = [
"MULTITOOL_COMBO_FINAL",
"passcode-6718",
"rag-7421",
"tool-a",
"tool-b",
];
const result = classifyDebugChatResult({
beforeText: "",
afterText: "Bot response with formatted details",
@@ -2225,11 +2256,14 @@ test("debug chat classifier accepts formatted responses containing every require
latestExpectedLeaf: "MULTITOOL_COMBO_FINAL",
latestFailureLeaf: "",
beforeMessages: [],
afterMessages: [{
role: "assistant",
text: "MULTITOOL_COMBO_FINAL\n- passcode-6718\n- rag-7421\n- tool-a\n- tool-b",
}],
latestAssistantText: "MULTITOOL_COMBO_FINAL\n- passcode-6718\n- rag-7421\n- tool-a\n- tool-b",
afterMessages: [
{
role: "assistant",
text: "MULTITOOL_COMBO_FINAL\n- passcode-6718\n- rag-7421\n- tool-a\n- tool-b",
},
],
latestAssistantText:
"MULTITOOL_COMBO_FINAL\n- passcode-6718\n- rag-7421\n- tool-a\n- tool-b",
});
assert.equal(result.status, "pass");
@@ -2246,7 +2280,9 @@ test("debug chat classifier rejects formatted responses missing a required fragm
latestExpectedLeaf: "MULTITOOL_COMBO_FINAL",
latestFailureLeaf: "",
beforeMessages: [],
afterMessages: [{ role: "assistant", text: "MULTITOOL_COMBO_FINAL\n- tool-a" }],
afterMessages: [
{ role: "assistant", text: "MULTITOOL_COMBO_FINAL\n- tool-a" },
],
latestAssistantText: "MULTITOOL_COMBO_FINAL\n- tool-a",
});
@@ -2608,7 +2644,7 @@ test("generic pipeline readiness accepts either URL or name target", () => {
}
});
test("test recommend maps AgentRunner ledger changes to focused probes", () => {
test("test recommend maps Runner ledger changes to focused probes", () => {
const result = capture(() =>
commandTestRecommend(
ctx([
@@ -2642,14 +2678,14 @@ test("test recommend maps AgentRunner ledger changes to focused probes", () => {
);
});
test("test recommend maps AgentRunner result changes to fixture contract", () => {
test("test recommend maps Runner result changes to fixture contract", () => {
const result = capture(() =>
commandTestRecommend(
ctx([
"test",
"recommend",
"--file",
"langbot-plugin-sdk/src/langbot_plugin/api/entities/builtin/agent_runner/result.py",
"langbot-plugin-sdk/src/langbot_plugin/api/entities/builtin/runner/result.py",
"--json",
]),
),
@@ -2662,14 +2698,14 @@ test("test recommend maps AgentRunner result changes to fixture contract", () =>
assert.ok(!ids.includes("agent-runner-ledger-invariants"));
});
test("test recommend maps QA AgentRunner fixture changes to live install", () => {
test("test recommend maps QA Runner fixture changes to live install", () => {
const result = capture(() =>
commandTestRecommend(
ctx([
"test",
"recommend",
"--file",
"langbot-skills/skills/langbot-testing/fixtures/plugins/qa-agent-runner/components/agent_runner/default.py",
"langbot-skills/skills/langbot-testing/fixtures/plugins/qa-agent-runner/components/runner/default.py",
"--json",
]),
),
@@ -2705,7 +2741,7 @@ test("test recommend keeps git status paths intact", () => {
const originalRepos = {
LANGBOT_REPO: process.env.LANGBOT_REPO,
LANGBOT_PLUGIN_SDK_REPO: process.env.LANGBOT_PLUGIN_SDK_REPO,
LANGBOT_AGENT_RUNNER_REPO: process.env.LANGBOT_AGENT_RUNNER_REPO,
LANGBOT_RUNNER_REPO: process.env.LANGBOT_RUNNER_REPO,
LANGBOT_LOCAL_AGENT_REPO: process.env.LANGBOT_LOCAL_AGENT_REPO,
};
try {
@@ -2752,7 +2788,7 @@ test("test recommend keeps git status paths intact", () => {
process.env.LANGBOT_REPO = repo;
process.env.LANGBOT_PLUGIN_SDK_REPO = join(tmp, "missing-sdk");
process.env.LANGBOT_AGENT_RUNNER_REPO = join(tmp, "missing-runner");
process.env.LANGBOT_RUNNER_REPO = join(tmp, "missing-runner");
process.env.LANGBOT_LOCAL_AGENT_REPO = join(tmp, "missing-local");
const result = capture(() =>
commandTestRecommend({ root, args: ["test", "recommend", "--json"] }),
@@ -3763,10 +3799,20 @@ test("fake provider can inject faults for only the selected model", async () =>
});
assert.equal(fakeProviderMessage(fallback).content, "OK");
const state = await fetch(`${rootUrl}/__qa/config`).then((response) => response.json());
const state = await fetch(`${rootUrl}/__qa/config`).then((response) =>
response.json(),
);
assert.deepEqual(
state.recent_requests.map((request: { model: string; status: string }) => [request.model, request.status]),
[["qa-primary", "http_fault"], ["qa-fallback", "ok"]],
state.recent_requests.map(
(request: { model: string; status: string }) => [
request.model,
request.status,
],
),
[
["qa-primary", "http_fault"],
["qa-fallback", "ok"],
],
);
} finally {
await provider.stop();
@@ -3776,38 +3822,64 @@ test("fake provider can inject faults for only the selected model", async () =>
test("local-agent model failure cases expose fallback fault controls", () => {
const beforeFirstChunk = capture(() =>
commandTestRun(
ctx(["test", "run", "local-agent-model-fallback-before-first-chunk-debug-chat", "--dry-run", "--json"]),
ctx([
"test",
"run",
"local-agent-model-fallback-before-first-chunk-debug-chat",
"--dry-run",
"--json",
]),
),
);
assert.equal(beforeFirstChunk.code, 0);
const fallbackRun = JSON.parse(beforeFirstChunk.output);
assert.equal(fallbackRun.automation.env_defaults.LANGBOT_FAKE_PROVIDER_MODEL_NAME, "qa-fallback-primary");
assert.equal(
fallbackRun.automation.env_defaults.LANGBOT_FAKE_PROVIDER_FALLBACK_MODEL_NAMES,
fallbackRun.automation.env_defaults.LANGBOT_FAKE_PROVIDER_MODEL_NAME,
"qa-fallback-primary",
);
assert.equal(
fallbackRun.automation.env_defaults
.LANGBOT_FAKE_PROVIDER_FALLBACK_MODEL_NAMES,
"qa-fallback-secondary",
);
assert.equal(fallbackRun.automation.env_defaults.LANGBOT_FAKE_PROVIDER_FAIL_MODELS, "qa-fallback-primary");
assert.equal(
fallbackRun.automation.env_defaults.LANGBOT_FAKE_PROVIDER_FAIL_MODELS,
"qa-fallback-primary",
);
const postCommit = capture(() =>
commandTestRun(
ctx(["test", "run", "local-agent-streaming-post-commit-failure-debug-chat", "--dry-run", "--json"]),
ctx([
"test",
"run",
"local-agent-streaming-post-commit-failure-debug-chat",
"--dry-run",
"--json",
]),
),
);
assert.equal(postCommit.code, 0);
const postCommitRun = JSON.parse(postCommit.output);
assert.equal(
postCommitRun.automation.env_defaults.LANGBOT_FAKE_PROVIDER_FAIL_AFTER_FIRST_CHUNK_MODELS,
postCommitRun.automation.env_defaults
.LANGBOT_FAKE_PROVIDER_FAIL_AFTER_FIRST_CHUNK_MODELS,
"qa-post-commit-primary",
);
assert.equal(
postCommitRun.automation.env_defaults.LANGBOT_FAKE_PROVIDER_FAIL_AFTER_FIRST_CHUNK_DELAY_MS,
postCommitRun.automation.env_defaults
.LANGBOT_FAKE_PROVIDER_FAIL_AFTER_FIRST_CHUNK_DELAY_MS,
"1000",
);
assert.equal(
postCommitRun.automation.env_defaults.LANGBOT_FAKE_PROVIDER_FAIL_AFTER_FIRST_CHUNK_MODE,
postCommitRun.automation.env_defaults
.LANGBOT_FAKE_PROVIDER_FAIL_AFTER_FIRST_CHUNK_MODE,
"error_event",
);
assert.equal(postCommitRun.automation.env_defaults.LANGBOT_DEBUG_CHAT_LOAD_REQUIRE_SUCCESS, "false");
assert.equal(
postCommitRun.automation.env_defaults
.LANGBOT_DEBUG_CHAT_LOAD_REQUIRE_SUCCESS,
"false",
);
});
test("fake provider requires the effective system prompt before returning its sentinel", async () => {
@@ -3854,10 +3926,13 @@ test("fake provider preserves a unique qa_mcp_echo probe value across the tool l
const initial = fakeProviderMessage(
await requestFakeProvider(provider, {
tools: [tool],
messages: [{
role: "user",
content: "Call qa_mcp_echo with exactly this text: box-recovery-unique-42. Return only the tool result.",
}],
messages: [
{
role: "user",
content:
"Call qa_mcp_echo with exactly this text: box-recovery-unique-42. Return only the tool result.",
},
],
}),
);
assert.equal(initial.tool_calls?.[0]?.function?.name, "qa_mcp_echo");
@@ -3872,10 +3947,15 @@ test("fake provider preserves a unique qa_mcp_echo probe value across the tool l
messages: [
{
role: "user",
content: "Call qa_mcp_echo with exactly this text: box-recovery-unique-42. Return only the tool result.",
content:
"Call qa_mcp_echo with exactly this text: box-recovery-unique-42. Return only the tool result.",
},
initial,
{ role: "tool", tool_call_id: initial.tool_calls?.[0]?.id, content: "qa_mcp_echo:box-recovery-unique-42" },
{
role: "tool",
tool_call_id: initial.tool_calls?.[0]?.id,
content: "qa_mcp_echo:box-recovery-unique-42",
},
],
}),
);
@@ -4057,7 +4137,7 @@ test("generic pipeline automation can still use the shared pipeline env", () =>
);
});
test("AgentRunner live install case exposes package automation defaults", () => {
test("Runner live install case exposes package automation defaults", () => {
const result = capture(() =>
commandTestRun(
ctx(["test", "run", "agent-runner-live-install", "--dry-run", "--json"]),
@@ -4106,7 +4186,7 @@ test("QA plugin live install checks the fixture package before installed state",
}
});
test("AgentRunner QA Debug Chat case uses dedicated pipeline env", () => {
test("Runner QA Debug Chat case uses dedicated pipeline env", () => {
const result = capture(() =>
commandTestRun(
ctx(["test", "run", "agent-runner-qa-debug-chat", "--dry-run", "--json"]),
@@ -4135,12 +4215,12 @@ test("AgentRunner QA Debug Chat case uses dedicated pipeline env", () => {
run.automation.env_aliases.some(
(alias: { target: string; source: string }) =>
alias.target === "LANGBOT_E2E_PIPELINE_URL" &&
alias.source === "LANGBOT_QA_AGENT_RUNNER_PIPELINE_URL",
alias.source === "LANGBOT_QA_RUNNER_PIPELINE_URL",
),
);
});
test("AgentRunner QA Debug Chat setup automation removes manual readiness", () => {
test("Runner QA Debug Chat setup automation removes manual readiness", () => {
withEnv(
{
LANGBOT_BROWSER_PROFILE: "/tmp/langbot-test-profile",
@@ -4156,8 +4236,8 @@ test("AgentRunner QA Debug Chat setup automation removes manual readiness", () =
const plan = JSON.parse(planResult.output);
assert.equal(plan.manual_readiness.status, "not_required");
assert.deepEqual(plan.setup_provides_env, [
"LANGBOT_QA_AGENT_RUNNER_PIPELINE_URL",
"LANGBOT_QA_AGENT_RUNNER_PIPELINE_NAME",
"LANGBOT_QA_RUNNER_PIPELINE_URL",
"LANGBOT_QA_RUNNER_PIPELINE_NAME",
]);
assert.equal(plan.automation_readiness.status, "ready");
@@ -4177,7 +4257,7 @@ test("AgentRunner QA Debug Chat setup automation removes manual readiness", () =
);
});
test("ACP AgentRunner Debug Chat case setups the ACP pipeline env", () => {
test("ACP Runner Debug Chat case setups the ACP pipeline env", () => {
const result = capture(() =>
commandTestRun(
ctx([
@@ -4199,7 +4279,7 @@ test("ACP AgentRunner Debug Chat case setups the ACP pipeline env", () => {
run.automation.env_aliases.some(
(alias: { target: string; source: string }) =>
alias.target === "LANGBOT_E2E_PIPELINE_URL" &&
alias.source === "LANGBOT_ACP_AGENT_RUNNER_PIPELINE_URL",
alias.source === "LANGBOT_ACP_RUNNER_PIPELINE_URL",
),
);
@@ -4211,8 +4291,8 @@ test("ACP AgentRunner Debug Chat case setups the ACP pipeline env", () => {
assert.equal(planResult.code, 0);
const plan = JSON.parse(planResult.output);
assert.deepEqual(plan.setup_provides_env, [
"LANGBOT_ACP_AGENT_RUNNER_PIPELINE_URL",
"LANGBOT_ACP_AGENT_RUNNER_PIPELINE_NAME",
"LANGBOT_ACP_RUNNER_PIPELINE_URL",
"LANGBOT_ACP_RUNNER_PIPELINE_NAME",
]);
assert.ok(
!plan.preconditions.some((item: string) =>
@@ -4885,7 +4965,7 @@ test("test report classifies provider quota tracebacks as env_issue", () => {
logPath,
[
"[05-21 10:31:00.000] chat.py (2) - [ERROR] : Request Failed: Traceback (most recent call last):",
" File \"provider.py\", line 1, in invoke",
' File "provider.py", line 1, in invoke',
"openai.PermissionDeniedError: insufficient user quota",
"[05-21 10:31:01.000] pipeline.py (3) - [ERROR] : runner.llm_error All models failed during streaming setup: insufficient user quota",
].join("\n"),
+1
View File
@@ -17,6 +17,7 @@ from quart import Quart, request, Response, jsonify
from langbot.libs.wecom_ai_bot_api import wecombotevent
from langbot.libs.wecom_ai_bot_api.WXBizMsgCrypt3 import WXBizMsgCrypt
if TYPE_CHECKING:
from langbot.pkg.platform.logger import EventLogger
from langbot.pkg.utils import httpclient
+8 -8
View File
@@ -2,35 +2,35 @@
from __future__ import annotations
from .runner.descriptor import AgentRunnerDescriptor
from .runner.descriptor import RunnerDescriptor
from .runner.id import parse_runner_id, format_runner_id, RunnerIdParts, is_plugin_runner_id
from .runner.errors import (
AgentRunnerError,
RunnerError,
RunnerNotFoundError,
RunnerNotAuthorizedError,
RunnerProtocolError,
RunnerExecutionError,
)
from .runner.registry import AgentRunnerRegistry
from .runner.context_builder import AgentRunContextBuilder
from .runner.registry import RunnerRegistry
from .runner.context_builder import RunnerContextBuilder
from .runner.resource_builder import AgentResourceBuilder
from .runner.result_normalizer import AgentResultNormalizer
from .runner.orchestrator import AgentRunOrchestrator
from .runner.config_resolver import RunnerConfigResolver
__all__ = [
'AgentRunnerDescriptor',
'RunnerDescriptor',
'parse_runner_id',
'format_runner_id',
'is_plugin_runner_id',
'RunnerIdParts',
'AgentRunnerError',
'RunnerError',
'RunnerNotFoundError',
'RunnerNotAuthorizedError',
'RunnerProtocolError',
'RunnerExecutionError',
'AgentRunnerRegistry',
'AgentRunContextBuilder',
'RunnerRegistry',
'RunnerContextBuilder',
'AgentResourceBuilder',
'AgentResultNormalizer',
'AgentRunOrchestrator',

Some files were not shown because too many files have changed in this diff Show More