Compare commits

..

93 Commits

Author SHA1 Message Date
dadachann 4cf25be974 test(web): align smoke fixtures with current processor UI 2026-09-11 04:51:12 +00:00
dadachann 3b047ab080 test(persistence): align standalone migration fixtures with current models 2026-09-11 04:43:38 +00:00
dadachann ce3f10a3ea fix(ci): format sources and resolve current migration head 2026-09-11 04:32:31 +00:00
dadachann 6189b06dfc fix(runner): align SDK pin and workspace-aware integration fixtures 2026-09-11 04:16:55 +00:00
RockChinQ 24ddcfe13e fix(i18n): localize Runner component labels 2026-09-11 00:03:09 +08:00
RockChinQ d93e44d82d feat(runner): authorize contextual platform APIs 2026-09-11 00:02:44 +08:00
RockChinQ f24a7c9bb2 feat(runner): unify plugin execution across agents and event processors 2026-09-10 18:04:38 +08:00
RockChinQ 8903a40c41 fix(runtime): preserve explicit replies and report bot configuration errors 2026-09-10 17:21:22 +08:00
RockChinQ b7a04f7a24 fix(bots): streamline detail layout and event listening 2026-09-10 15:29:52 +08:00
RockChinQ 15fa0f50e3 fix(ui): distinguish EventProcessor with a trigger icon 2026-09-10 15:18:04 +08:00
RockChinQ d1c79a35e8 fix(4.11): align omni adapters and EventProcessor marketplace support 2026-09-10 15:06:57 +08:00
RockChinQ 5ba25c39af feat(processors): refine plugin processor configuration and presentation 2026-09-09 15:37:12 +08:00
RockChinQ ea3e32c904 feat(processors): streamline event debugging and run inspection 2026-09-08 02:30:59 +08:00
RockChinQ 237fa6545d feat(processors): add explicitly bound plugin event processors 2026-09-08 00:43:35 +08:00
RockChinQ 812eb09ee4 fix(pipeline): apply returned plugin event contexts 2026-09-07 23:51:34 +08:00
RockChinQ 5d0657b1db feat(agent-debug): edit event data with compact forms 2026-09-06 01:57:47 +08:00
RockChinQ 2a46325232 fix(agent-debug): simplify and localize tool call traces 2026-09-06 00:33:32 +08:00
RockChinQ 75f117ae50 fix(provider): preserve explicit streaming finish reasons 2026-09-06 00:33:20 +08:00
RockChinQ f5d6a3b348 fix(agent): tolerate intent annotations on no-argument platform tools 2026-09-05 17:15:20 +08:00
Hyu 8f0a55a1f4 fix(agent-debug): stream execution traces with platform mocks and coverage 2026-09-05 15:34:29 +08:00
Hyu a4d36aa2db feat(agent): add event-aware tool permissions 2026-09-04 15:37:41 +08:00
Hyu 7c64756203 fix(ui): improve agent debug and model test feedback 2026-09-03 14:51:01 +08:00
Hyu be0214cf4d feat(storage): group usage by application and runtimes 2026-09-03 00:11:05 +08:00
Hyu 380bdfeca6 fix(web): contain nested scrolling without blocking forms 2026-09-01 17:29:59 +08:00
Hyu a87c814dae feat(web): improve marketplace install workflows 2026-09-01 16:43:50 +08:00
Hyu 982d660236 fix(web): harden agent runner marketplace flows 2026-09-01 12:46:16 +08:00
Hyu e5e62c8fe9 fix(plugin): keep status responsive during installation 2026-09-01 12:46:16 +08:00
Hyu 91e09af76a fix(web): complete pluginized agent onboarding flows 2026-09-01 10:41:29 +08:00
Hyu 7e51044a87 Merge fix/box-reconnect-skill-context into dev/4.11.x 2026-08-31 19:18:34 +08:00
Hyu b4980ab536 fix(box): reload workspace-scoped skills after reconnect 2026-08-31 19:18:19 +08:00
Hyu fde04e64f6 Merge remote-tracking branch 'origin/master' into dev/4.11.x
# Conflicts:
#	src/langbot/pkg/api/http/controller/groups/pipelines/pipelines.py
#	src/langbot/pkg/api/http/service/bot.py
#	src/langbot/pkg/provider/runners/localagent.py
#	src/langbot/templates/metadata/pipeline/ai.yaml
#	tests/unit_tests/api/service/test_bot_service.py
#	tests/unit_tests/provider/runners/test_difysvapi_runner.py
#	tests/unit_tests/utils/test_safe_regex.py
#	web/src/app/infra/entities/adapter-categories.ts
#	web/src/app/wizard/page.tsx
#	web/src/i18n/locales/en-US.ts
#	web/src/i18n/locales/ja-JP.ts
#	web/src/i18n/locales/zh-Hans.ts
#	web/tests/e2e/plugin-page-auth.spec.ts
2026-08-31 17:17:47 +08:00
RockChinQ 1b7ae791b3 fix(plugin): return not found after removal (#2483)
Co-authored-by: Hyu <chenhyu@proton.me>
2026-08-31 13:40:03 +08:00
Neos e69a80f5e9 Windows 兼容性修复 + Dify API 适配 + 企微 WS 加载优化 (#2470)
* fix: Windows fsync requires write access to file descriptor

- _fsync_file: os.O_RDONLY -> os.O_RDWR (Windows _commit() requires write access)
- _fsync_directory: tolerate OSError on fsync and os.open (chmod 0o700 blocks access on Windows)

* fix: Dify SSE empty data lines and upload response format mismatch

- Skip malformed/flushed SSE data lines (JSONDecodeError)
- Accept HTTP 200 besides 201 for file upload response
- Unwrap data wrapper in upload response (resp.get('data', resp))

* feat: WeCom WS mode sends empty initial stream frame for loading spinner

Send an empty reply_stream frame immediately after stream session
creation so the WeCom client shows its built-in loading indicator
while the pipeline processes the message (e.g. RAG retrieval).

* fix(compat): harden Windows, Dify, and WeCom changes

* fix(migration): clean interrupted SQLite temp files

---------

Co-authored-by: Hyu <chenhyu@proton.me>
2026-08-29 22:34:10 +08:00
Hyu bafdaf0033 fix(monitoring): restore SQLite token statistics (#2479)
* fix(monitoring): restore SQLite token statistics

Allow the monitored SQLite strftime bucket expression through the tenant SQL guard and preserve structured backend errors in the token dashboard. Add persistence and frontend regressions.\n\nVerified-by: independent-review

* test(runtime): accept reconcile timeout in capacity stub

Keep the PostgreSQL capacity probe aligned with the runtime handler contract and assert the bounded reconcile timeout.

---------

Co-authored-by: dadachann <185672915+dadachann@users.noreply.github.com>
2026-08-28 11:41:42 +08:00
RockChinQ aeff8d7e30 fix(i18n): complete wizard locale keys 2026-08-28 01:37:45 +08:00
Dongchuan Fu be3734ffda feat(wizard): rework agent onboarding flow (#2471)
* feat(wizard): rework agent onboarding flow

* fix(web): support LAN development access

* fix(wizard): parse ranked model selection entries

* feat(wizard): add inbound bot verification

* feat(wizard): add floating page bot verification

* fix(wizard): repair HTTP bot inbound test setup

* feat(wizard): streamline custom model onboarding

* feat(wizard): label page bot test preview

* style(space): apply ruff formatting

* fix(wizard): polish AI engine onboarding

* fix(wizard): clarify local account message test

* feat(wizard): animate AI engine transitions

* fix(wizard): align AI engine setup headers

---------

Co-authored-by: langbot-dev <langbot@users.noreply.github.com>
Co-authored-by: RockChinQ <rockchinq@gmail.com>
2026-08-28 01:30:30 +08:00
leonoxo 855ae2bdba fix(line): map LINE mentions to At elements so the at-bot rule works (#2478)
The LINE adapter passed text through as a single Plain component,
ignoring the mention payload (mentions[].index/length/isSelf) that the
Line Messaging API includes in the webhook. As a result:

- At(target=bot_account_id) never appeared in the message chain, so the
  'at-bot' group respond rule silently dropped every @bot mention.
- The bot only replied when the message happened to match the prefix
  rule (e.g. starting with 'ai').

Now LINEMessageConverter reads message.message.mention and builds the
chain per mention position:

- Bot mention (isSelf) -> At(target=bot_account_id) so AtBotRule matches
  the same way as other adapters (dingtalk/lark etc.).
- Other mentions -> At(target=<line user id>, display=<mention text>).
  At.__str__ already prepends '@', so the display text carries no
  double '@' and the rendered text (prefix/regexp rules, quotes,
  session context) is byte-identical to before.
- Missing/out-of-bounds mentions are skipped defensively.

target2yiri becomes an instance method (like wechatpad/aiocqhttp) so
the converters can hold bot_account_id; LINEAdapter passes it in from
its own config.
2026-08-27 18:38:27 +08:00
fishzjp b66db86bff fix(provider): stringify MCP tool results for OpenAI-compatible APIs (#2476)
execute_func_call returns list[ContentElement] for MCP tools, but the
runner assigned that list directly to the tool-message content. The
OpenAI chat-completions spec requires tool-message content to be a
string, so OpenAI-compatible endpoints return HTTP 500 when the raw
list is sent.

Serialize the list to a string before building the tool message, using
ContentElement.__str__ which returns the text payload for text elements
and a human-readable placeholder for images and files. Fixes #2457.
2026-08-27 18:19:04 +08:00
fishzjp 95b8736e93 fix(provider): tolerate trimmed image parts in litellm message conversion (#2475)
SessionManager clears image_base64 on past turns to save memory, and
exclude_none serialization drops the hollowed field entirely, so a
replayed history part can arrive as {'type': 'image_base64'} with no
payload. The converter accessed the missing key unconditionally and
raised KeyError on every turn after an image was sent.

Prefer the base64 payload when present, fall back to an image_url that
survived on the same element, and drop hollow parts otherwise (same
strategy as the existing file-part handling). Fixes #2469.
2026-08-27 18:07:43 +08:00
Yang cabde423a1 Fix wecomcs open_kfid msgid (#2449)
* Update wecomcs.py

fix bug wecomcs send_message open_kfid
event.receiver_id  is open_kfid

* Update wecomcs.py

fix bug msgid exceeds the 32-byte limit

* test(wecomcs): cover bounded message IDs and images

---------

Co-authored-by: dadachann <185672915+dadachann@users.noreply.github.com>
2026-08-27 17:30:49 +08:00
RockChinQ 4b888ad390 copy(agents): explain the pipeline message flow 2026-08-27 16:03:56 +08:00
RockChinQ 7d9bdf0562 copy(agents): simplify pipeline type description 2026-08-27 15:59:28 +08:00
RockChinQ 5e144ac7c8 fix(i18n): localize pipeline processor type 2026-08-27 15:57:31 +08:00
RockChinQ 21ffaf9a14 refactor(agents): move debug event picker to input area 2026-08-27 15:52:59 +08:00
RockChinQ f3c1887e92 refactor(pipelines): clarify log view switching 2026-08-27 15:49:55 +08:00
RockChinQ 7671aefed7 fix(agents): improve diagram contrast in dark mode 2026-08-27 15:32:35 +08:00
RockChinQ e6f6789e8b fix(agents): make pipeline flow animation seamless 2026-08-27 15:28:43 +08:00
RockChinQ 05fe93e108 style(agents): animate processor diagram connections 2026-08-27 15:18:34 +08:00
RockChinQ e20a73c83a style(agents): distinguish agent capabilities from events 2026-08-27 15:14:53 +08:00
RockChinQ fde31d590e style(agents): clarify agent event direction 2026-08-27 15:11:13 +08:00
RockChinQ af03dbe958 style(agents): align pipeline diagram with agent 2026-08-27 15:08:33 +08:00
RockChinQ bd7aeab53d copy(agents): clarify processor use cases 2026-08-27 15:03:43 +08:00
RockChinQ 91f484e0ce copy(agents): expand processor type descriptions 2026-08-27 15:02:02 +08:00
RockChinQ a9b24e59ec refactor(agents): use shadcn processor type toggle 2026-08-27 14:55:59 +08:00
RockChinQ 139494cd63 style(agents): apply LangBot colors to processor diagrams 2026-08-27 14:49:04 +08:00
RockChinQ b4d9659ffe refactor(agents): remove decorative agent icon 2026-08-27 14:44:51 +08:00
RockChinQ 3f0ca13d6d refactor(agents): simplify processor diagrams 2026-08-27 14:43:32 +08:00
RockChinQ 6b40c2cf66 refactor(agents): polish processor creation layout 2026-08-27 14:39:56 +08:00
RockChinQ 847bfc8df5 feat(agents): improve processor creation choice 2026-08-27 14:31:37 +08:00
RockChinQ 823feac7e0 fix(bots): clarify adapter test prerequisites 2026-08-26 23:14:42 +08:00
RockChinQ a7badf6258 fix(bots): move adapter debugger to configuration 2026-08-26 23:07:02 +08:00
RockChinQ 3d692fa8db feat(bots): add platform event debugger 2026-08-26 19:46:59 +08:00
RockChinQ 7990d36c78 fix(bots): align route event selector 2026-08-26 19:32:39 +08:00
RockChinQ 600a173918 refactor(bots): remove route execution test 2026-08-26 14:28:30 +08:00
RockChinQ 8b63cc0281 fix(events): show raw codes in test selectors 2026-08-26 14:12:52 +08:00
RockChinQ 4787799cd8 fix(agents): align debug event selection 2026-08-26 14:06:09 +08:00
RockChinQ 47f5515fa9 fix(pipelines): retain debug chat image history 2026-08-26 00:32:03 +08:00
RockChinQ 94fd3d274c refactor(pipelines): simplify connection status copy 2026-08-26 00:24:37 +08:00
RockChinQ 8380bfc7d4 refactor(pipelines): soften chat session selection 2026-08-26 00:22:22 +08:00
RockChinQ a936734efa fix(pipelines): move chat reset action to footer 2026-08-26 00:20:16 +08:00
RockChinQ 0c5656d0d0 refactor(pipelines): streamline debug chat controls 2026-08-26 00:18:46 +08:00
RockChinQ 532d1b73d2 refactor(bots): group event route creation menu 2026-08-26 00:10:51 +08:00
RockChinQ aaeb9ad178 fix(bots): restore event route drag sorting 2026-08-26 00:08:09 +08:00
RockChinQ d28b385a9f refactor(events): group event selection options 2026-08-25 23:59:52 +08:00
ciri667 08307790e5 fix(cntfilter): allow legacy sensitive-word lists over 64 patterns (#2467)
* fix(cntfilter): allow legacy sensitive-word lists over 64 patterns

Legacy sensitive-words.json files shipped ~70 rules. After v4.10.7,
BanWordFilter treated the 64-pattern safe_regex cap as a hard failure
and blocked every message. Raise the cap only on the sensitive-word
path, keep the 50ms CPU budget, and truncate oversized lists with a
one-time warning.

Fixes #2443

* fix(cntfilter): reject oversized sensitive-word lists

---------

Co-authored-by: dadachann <185672915+dadachann@users.noreply.github.com>
2026-08-25 23:53:41 +08:00
RockChinQ 4d7a333802 feat(agent): add visual event range picker 2026-08-25 23:51:24 +08:00
RockChinQ 114612a984 refactor(agent): move delete action to header 2026-08-25 23:34:35 +08:00
RockChinQ 2aceefce47 test(agent): remove obsolete enabled fixture 2026-08-25 23:28:37 +08:00
RockChinQ 80f1790e1d refactor(agent): remove enabled state 2026-08-25 23:27:45 +08:00
RockChinQ 22d9053bf1 feat(web): streamline management and knowledge settings 2026-08-25 23:02:33 +08:00
RockChinQ db2a9155f8 feat(web): edit entity details from page titles 2026-08-25 22:31:23 +08:00
RockChinQ 69ca7e21cd fix(bots): explain unavailable route tests 2026-08-25 22:03:57 +08:00
RockChinQ 792d961d28 fix(web): clarify route test guidance 2026-08-25 21:53:06 +08:00
RockChinQ a8eb265c11 fix(web): clarify route test warning 2026-08-25 21:31:02 +08:00
RockChinQ e62f8a957a fix(web): simplify route test dialog 2026-08-25 21:27:33 +08:00
RockChinQ ad6b8b3209 fix(web): restore bot card spacing 2026-08-25 21:21:36 +08:00
RockChinQ 68620c4572 fix(bots): simplify event routing status 2026-08-25 21:18:01 +08:00
leonoxo 777fe1f20b fix(line): use stable source id for session identity (#2398)
LINEEventConverter.target2yiri() built Friend.id/Group.id from
event.message.id, which is unique per message. Every incoming message
therefore mapped to a new session key, so LINE users and groups lost
conversation context on every turn.

Use event.source.user_id/group_id/room_id instead, matching the stable
identifiers other adapters (e.g. Telegram) use for session identity.
Falls back to the group/room id when user_id is absent, per LINE's
documented behavior for some group/room members.
2026-08-25 12:30:29 +08:00
Hyu f0ee57c1e0 style(email): use generic LangBot invitation branding (#2466)
Co-authored-by: Junyan Qin <rockchinq@gmail.com>
2026-08-24 14:53:59 +08:00
Hyu a45e27e76e style(cloud): redesign workspace invitation email (#2464)
* style(cloud): redesign workspace invitation email

* fix(email): harden Outlook spacing and text contrast

---------

Co-authored-by: Junyan Qin <rockchinq@gmail.com>
2026-08-24 14:14:56 +08:00
Hyu 536fcdf29f fix(web): restore plugin page SDK loading (#2435)
Co-authored-by: dadachann <185672915+dadachann@users.noreply.github.com>
2026-08-24 12:40:50 +08:00
QuasarRyan c87548c0b9 feat(qqofficial): add markdown reply rendering (#2459) 2026-08-24 11:59:23 +08:00
QuasarRyan 79634772da fix(qqofficial): send complete stream snapshots (#2458) 2026-08-24 11:49:22 +08:00
521 changed files with 29190 additions and 8459 deletions
+4 -4
View File
@@ -53,7 +53,7 @@ LangBot/
│ │ ├── platform/ # IM adapters and runtime bot manager
│ │ ├── pipeline/ # Message routing and pipeline stages
│ │ ├── provider/ # Model providers and Host-owned tools
│ │ ├── agent/ # Agent/AgentRunner orchestration and run state
│ │ ├── agent/ # Agent/Runner orchestration and run state
│ │ ├── plugin/ # LangBot-side Plugin Runtime connector/handler
│ │ ├── box/ # LangBot-side Box service/connector
│ │ ├── skill/ # Skill metadata/activation integration
@@ -81,7 +81,7 @@ Platform adapter
→ Controller
→ RuntimePipeline
→ PipelineStage chain
AgentRunner orchestrator / ToolManager / PluginRuntimeConnector / BoxService
→ Runner orchestrator / ToolManager / PluginRuntimeConnector / BoxService
→ response via adapter
```
@@ -108,7 +108,7 @@ Inbound platform messages enter through adapter-specific SDK callbacks. The comm
3. `MessageAggregator` batches/normalizes messages before adding a `Query` to `QueryPool`.
4. `Controller` in `pkg/pipeline/controller.py` selects queries subject to global pipeline concurrency and per-session concurrency.
5. `RuntimePipeline` in `pkg/pipeline/pipelinemgr.py` runs configured pipeline stages using a responsibility-chain style executor that supports generator stages.
6. The chat stage emits plugin events and projects the current query into the AgentRunner Host orchestrator. The selected plugin AgentRunner returns streaming or final results while the Host owns authorization, tools, telemetry, and conversation history.
6. The chat stage emits plugin events and projects the current query into the Runner Host orchestrator. The selected plugin Runner returns streaming or final results while the Host owns authorization, tools, telemetry, and conversation history.
7. Output stages send text, cards, chunks, files, or error notices back through the original platform adapter.
Pipeline components are registered by decorators and package import side effects. When adding a new stage, loader, runner, or adapter, check the corresponding preregistration mechanism instead of inventing a second registry.
@@ -142,7 +142,7 @@ Pipelines are configuration-driven. Prefer adding a stage or extending an existi
Agent orchestration lives under `pkg/agent/`; model providers and tools live under `pkg/provider/`.
- `modelmgr/` manages configured model providers and requesters.
- `pkg/agent/runner/` discovers plugin AgentRunner components, resolves bindings, constructs run-scoped context/resources, and records execution state.
- `pkg/agent/runner/` discovers plugin Runner components, resolves bindings, constructs run-scoped context/resources, and records execution state.
- `tools/toolmgr.py` aggregates tools from native tools, plugin tools, external MCP servers, and skill-authoring tools.
- `tools/loaders/mcp.py` is the MCP client side: external MCP servers that LangBot connects to for agent tools.
- RAG lives across `pkg/rag/`, `pkg/vector/`, model services, and plugin KnowledgeEngine actions.
@@ -1,8 +1,8 @@
# Agent-owned Context 协议设计
本文档描述插件化 AgentRunner 场景下的上下文边界**设计理由**。结论先行:LangBot 不应成为最终 agentic context manager;它提供 context substrateAgentRunner 或其背后的 runtime 自己决定如何管理历史、压缩、召回和 KV cache。
本文档描述插件化 Runner 场景下的上下文边界**设计理由**。结论先行:LangBot 不应成为最终 agentic context manager;它提供 context substrateRunner 或其背后的 runtime 自己决定如何管理历史、压缩、召回和 KV cache。
> 涉及的数据结构(`AgentRunContext`、`ContextAccess`、`AgentRunAPIProxy` 等)唯一定义在 [PROTOCOL_V1.md](./PROTOCOL_V1.md)。本文只讲语义和约束,不重抄 schema。
> 涉及的数据结构(`RunnerContext`、`ContextAccess`、`RunnerAPIProxy` 等)唯一定义在 [PROTOCOL_V1.md](./PROTOCOL_V1.md)。本文只讲语义和约束,不重抄 schema。
## 1. 设计原则
@@ -18,7 +18,7 @@
### 1.2 Host 不定义通用历史窗口
历史窗口策略不是 AgentRunner 协议或 Query entry adapter 的核心概念。Host 只提供 history pull API、cursor、hard cap 和权限边界;runner 自己决定是否读取、读取多少、如何截断和压缩。
历史窗口策略不是 Runner 协议或 Query entry adapter 的核心概念。Host 只提供 history pull API、cursor、hard cap 和权限边界;runner 自己决定是否读取、读取多少、如何截断和压缩。
正确的问题不是"LangBot 每轮裁几轮历史给 agent",而是:
@@ -33,13 +33,13 @@
- `EventLog`: Host 保存原始事件、工具调用、投递结果、错误和系统事件。
- `Transcript`: Host 从 EventLog 投影出的对话视图,用于 UI、审计和按需历史读取。
- `Working context`: Agent 本轮实际送进模型或 runtime 的上下文,由 AgentRunner 决定。
- `Working context`: Agent 本轮实际送进模型或 runtime 的上下文,由 Runner 决定。
LangBot 不提供 host-side inline history window。简单 runner 如果需要历史窗口,应在 runner 内部通过 Host history API 拉取并裁剪。
## 2. Event 到来时传什么
默认 `AgentRunContext`PROTOCOL_V1 §5.2)应尽量小且稳定。默认规则:
默认 `RunnerContext`PROTOCOL_V1 §5.2)应尽量小且稳定。默认规则:
- Host MUST NOT inline full history by default.
- Host SHOULD inline only current event / input and context handles.
@@ -57,7 +57,7 @@ LangBot 不提供 host-side inline history window。简单 runner 如果需要
### 2.3 不提供 Host Inline History Window
`AgentRunContext` 不包含 `bootstrap` 字段。Host 不下发历史窗口,也不通过 Pipeline 配置决定窗口大小。runner 若需要类似 `recent_tail` 的策略,应在自己的 manifest/config schema 中声明参数,并在 runner 内部通过 history API 读取、裁剪和压缩。Host 只负责权限、分页、hard cap 和事实源。
`RunnerContext` 不包含 `bootstrap` 字段。Host 不下发历史窗口,也不通过 Pipeline 配置决定窗口大小。runner 若需要类似 `recent_tail` 的策略,应在自己的 manifest/config schema 中声明参数,并在 runner 内部通过 history API 读取、裁剪和压缩。Host 只负责权限、分页、hard cap 和事实源。
## 3. ContextAccess 的作用
@@ -65,7 +65,7 @@ LangBot 不提供 host-side inline history window。简单 runner 如果需要
## 4. Agent 如何获取更多上下文
所有 API 都走 `AgentRunAPIProxy`PROTOCOL_V1 §8),由 host 用 `run_id` 校验。
所有 API 都走 `RunnerAPIProxy`PROTOCOL_V1 §8),由 host 用 `run_id` 校验。
外部 harness 不能直接访问 LangBot 资源。无论是 history、event、state、model、tool、knowledge base,还是 LangBot skills,都必须通过 SDK runtime 转发到 Host API,并由 Host 按 active `run_id`、runner identity、binding resource policy 和 caller plugin identity 校验。当前运行文件进入授权 sandbox/workspace 后,再由 runner 用 read/write/exec 类工具按需访问。harness 自己的 native tools 只属于 harness 执行环境,不能绕过 SDK runtime 访问 LangBot 内部资源。
@@ -109,7 +109,7 @@ Claude Code、Codex、Kimi Code 这类 runtime 通常已有自己的 session、
- `agent-context.json`:结构化 JSON,包含 `run_id``event``actor``subject``input``delivery``resources``context``state``runtime`
- `LANGBOT_CONTEXT.md`:人类可读摘要。
- `resources`:只包含本次 run 授权后的资源句柄和能力摘要,不暴露 Host 内部私有对象、secret 或资源内容。
- `skills`LangBot skills 不是直接投影给 harness native tool loop 的文件能力,而是**一组被授权的 tool**。发现走 `list_skills`(或 `langbot_list_assets` 增加 skills 一类),激活/注册走 `activate` / `register_skill`,包内操作走 native exec/read/write,统一通过 `ctx.resources.tools``AgentRunAPIProxy` 或 SDK-owned MCP bridge 暴露。Host 不向 prompt 注入 skill 索引(无 progressive-disclosure 注入);harness 通过调用发现工具主动查询 skill 清单。`agent-context.json``skills` 字段仅作发现工具的数据来源与可选 `suggested_skill_prompt` 的输入。
- `skills`LangBot skills 不是直接投影给 harness native tool loop 的文件能力,而是**一组被授权的 tool**。发现走 `list_skills`(或 `langbot_list_assets` 增加 skills 一类),激活/注册走 `activate` / `register_skill`,包内操作走 native exec/read/write,统一通过 `ctx.resources.tools``RunnerAPIProxy` 或 SDK-owned MCP bridge 暴露。Host 不向 prompt 注入 skill 索引(无 progressive-disclosure 注入);harness 通过调用发现工具主动查询 skill 清单。`agent-context.json``skills` 字段仅作发现工具的数据来源与可选 `suggested_skill_prompt` 的输入。
- `MCP config`:只投影 per-run、scoped 的 SDK-owned bridge 或外部 MCP 连接配置;LangBot 资源访问必须回到 SDK runtime / Host API,不允许 harness 通过自带 MCP/native tool 直接读 Host 内部资源。
- `state pointers`:外部 session id、working directory、checkpoint 等小型 JSON 状态通过 Host state API 保存。
@@ -134,7 +134,7 @@ Host 只给当前事件、当前输入和 context handles。Runner 是否能拉
稳定 session key 的用途是隔离外部 runtime 的 resume/cache/state,不是改变 PROTOCOL_V1 §13 定义的 Agent 复用和 dispatch 边界。只有当某个外部 harness 的同一 native session 不支持并发 turn 时,runner 或 future runtime control plane 才应按 external session key 做 turn-level 串行化。
对长期运行的 external harness / daemon,推荐运行形态是 reader 与 writer 分离:一个 session reader 独占读取 stdout/SSE/native event stream,并把 native event 转成 `AgentRunResult` 或 task progress;用户输入只作为 turn write 进入该 session。当前一次性 CLI subprocess runner 可以继续在单次 `run(ctx)` 内同步收集 stdout,但后续改成长连接时不应让多个 request 同时读取同一 native stream。
对长期运行的 external harness / daemon,推荐运行形态是 reader 与 writer 分离:一个 session reader 独占读取 stdout/SSE/native event stream,并把 native event 转成 `RunnerResult` 或 task progress;用户输入只作为 turn write 进入该 session。当前一次性 CLI subprocess runner 可以继续在单次 `run(ctx)` 内同步收集 stdout,但后续改成长连接时不应让多个 request 同时读取同一 native stream。
## 7. Host guardrail
@@ -1,4 +1,4 @@
# Agent Runner QA 指南
# Runner QA 指南
本文档是 agent-runner 插件化下一轮测试的唯一 QA 入口。它合并并取代旧的 Phase 1 验收矩阵与 2026-05-18 / 2026-05-29 两份本地 QA 报告。
@@ -6,7 +6,7 @@
## 1. 测试边界
当前主线验证的是 AgentRunner Protocol v1
当前主线验证的是 Runner Protocol v1
```text
event -> binding -> runner.run(ctx) -> result stream
@@ -22,8 +22,8 @@ event -> binding -> runner.run(ctx) -> result stream
本指南不验证:
- Runtime Control Plane v2
- EventGateway / EventRouter 完整落地由外部 EBA 分支联调;本指南只验证本分支 Host 底座
- 完整外部 harness daemon 管控和分布式业务队列;已实现的 run ledger / heartbeat / claim 原语仍应执行定向回归
- 尚未实现的通用事件订阅和定时自动化;已集成的 Bot 路由、独立 Agent 和 Pipeline 必须纳入当前产品验收
- 发布级 path isolation、secret filtering、MCP allowlist、资源配额和 workspace cleanup。
- 所有外部服务 runner 的真实凭据联调。
@@ -44,6 +44,16 @@ event -> binding -> runner.run(ctx) -> result stream
## 3. 执行顺序
2026-09-05 更新:先记录 Core/SDK/Runner 提交、发行版元数据与实际 import 路径,区分 editable 工作区和正式包安装。当前定向测试及前端未通过断言见 [STATUS.md](./STATUS.md)。本指南的步骤不是已执行记录。
本轮发布验收至少覆盖:
- 空白实例从市场安装 Runner,创建 Agent 与 Pipeline,并绑定到 Bot。
- 同一 Bot 的消息事件走 Pipeline,非消息事件走独立 Agent;dry-run 与保存后的路由结果一致。
- `event_*` 使用冻结目标;未授权 `platform_*`、额外参数和失效机器人调用被拒绝;SDK/Python 与 MCP gateway 均走 Host 工具授权。
- `interaction.requested` 回调恢复原处理器,重复/过期/跨作用域提交被拒绝。
- 分别记录合成事件、mock provider、真实平台和真实 provider 的结果,不能相互替代。
推荐按以下顺序执行,前一层失败时不要继续扩大测试面:
1. Host / SDK / runner 单测。
@@ -127,7 +137,7 @@ bin/lbs case list
通过条件:
- 用户可见回复正常。
- 后端日志显示走 `AgentRunOrchestrator` / `RUN_AGENT`
- 后端日志显示走 `AgentRunOrchestrator` / `RUN_RUNNER`
- 不走旧内置 local-agent 主执行分支。
- conversation transcript 写入用户消息和助手消息。
@@ -160,7 +170,7 @@ Smoke 前应优先保留一层轻量单测或 fixture 测试:session 创建/
步骤:
1. 确认目标 harness(例如 ACP daemon、Claude Code 或 Codex)在对应机器上可执行且已登录。
2. 绑定目标 runner,例如 `plugin:langbot-team/ACPAgentRunner/default``plugin:langbot-team/ClaudeCodeAgent/default``plugin:langbot-team/CodexAgent/default`
2. 绑定目标 runner,例如 `plugin:langbot-team/ACPRunner/default``plugin:langbot-team/ClaudeCodeAgent/default``plugin:langbot-team/CodexAgent/default`
3. 配置 runner 必要字段,例如 remote target、workspace、provider、startup timeout、reuse session 等。
4. 在 Debug Chat 执行一次确定性真实 smoke。
5. 检查 LangBot MCP gateway、`run_id` 回填和 host-owned state。
@@ -225,3 +235,25 @@ Dify、n8n、Coze、DashScope、Langflow、Tbox 等外部服务 runner 不作为
## 10. 历史高价值记录
历史高价值记录与当前 runner 验收状态见 [STATUS.md](./STATUS.md)。本指南只保留可重复执行的测试步骤和证据要求。
### Event debug execution trace
The Agent workbench uses `POST /api/v1/agents/{uuid}/debug/stream` (NDJSON, `runtime.operate`). Verify incremental text and provider-returned reasoning, tool call arguments/results in execution order, automatic scrolling, and preservation of partial output on errors. Thinking is only shown when returned by the runner. Disconnecting cancels the debug task. The existing `/debug` endpoint and MCP `debug_agent` return final text and a bounded `execution_events` snapshot; they are not live transports.
Platform tools now use Host-owned mock adapters in synthetic debug runs. The model really invokes the authorized tool, validates its parameters, and receives a result; the Host does not call a live platform adapter. Event tools retain their frozen targets and platform tools retain explicit targets. Native, plugin and MCP tools continue to execute normally. A successful mock platform result is completion of that action in the debug run; runners must communicate that context to the model to avoid repeated attempts at real delivery.
The optional `mock` payload field (also editable under **Mock 场景(JSON**) supports:
```json
{
"errors": {"event_reply": "Simulated permission denied"},
"results": {"event_get_actor": {"id": "user-42", "nickname": "Fixture User"}},
"unsupported_apis": ["delete_message"]
}
```
`errors` and `results` cannot both override the same tool. Unknown tools/APIs and malformed options are rejected before model execution. Default read fixtures follow SDK `User`, `UserGroup`, `UserGroupMember` and `MessageReceivedEvent` structures; list reads return a synthetic fixture list (override with `[]` to test emptiness). `unsupported_apis` participates in the normal capability intersection, so unsupported tools are not offered to the model. Event payloads should include the intended group/member/request IDs; the built-in presets supply examples.
Verify a welcome event shows `event_reply`, the expected text and frozen target, and **模拟执行成功 · Mock**. Plain text alone must not count as a successful platform action. A simulated failure must show **模拟执行失败 · Mock**. Also verify **停止调试** retains the partial trace and allows the next run, and that entering a new draft while streaming does not erase it on completion.
Full event/Mock regression evidence: [2026-09-05 follow-up](./EVENT_DEBUG_FULL_QA_2026-09-05.md).
@@ -1,19 +1,19 @@
# Event Based Agent 接入设计
> 本文记录 EBA 如何接入当前 AgentRunner Protocol v1 / Host 底座。EventGateway、EventRouter、Event subscription/notification 由外部 EBA 分支实现并联调;本分支只保留 event-first 入口和 envelope/binding models
> 更新:2026-09-05。EBA 平台事件、Bot 路由和独立 Agent 已集成到 `dev/4.11.x`。通用事件订阅、通知与定时自动化仍是后续扩展
>
> 数据结构唯一定义在 [PROTOCOL_V1.md](./PROTOCOL_V1.md)runner 可见)与 [HOST_SDK_INFRASTRUCTURE.md](./HOST_SDK_INFRASTRUCTURE.md)Host 内部模型);本文只讲 EBA 语义,不重抄 schema。
> 与当前 runner 外化分支、后续 Agent Platform / Runtime Control Plane 的边界见 [EXTENSION_SCOPE_MATRIX.md](./EXTENSION_SCOPE_MATRIX.md)。
本文描述 EBA 接入时,事件如何进入 LangBot、如何在平级的 Pipeline / Agent 处理器之间路由,以及 Agent 分支如何复用插件化 AgentRunner 基础设施。本分支不实现完整 EventBus / EventRouter / Platform API;这些能力正在外部 EBA 分支联调。这里的目标是把处理器路由与 runner 协议边界说清楚
本文描述当前事件如何进入 LangBot、如何在平级的 Pipeline / Agent 之间路由,以及 Agent 如何复用插件化 Runner。路由逻辑由 `pkg/platform/botmgr.py::RuntimeBot` 承担;文中的 EventRouter 表示职责,不代表独立进程或同名类
## 1. 设计目标
- 消息、撤回、入群、好友申请、定时任务、API 调用都能抽象为 host event。
- EventRouter 可以根据 event type、bot、workspace、conversation、actor、subject 选择一个 Pipeline 或 Agent 处理器。
- Pipeline 目标执行完整消息 Stage 链;Agent 目标通过统一 orchestrator 调用 AgentRunner。
- Pipeline 目标执行完整消息 Stage 链;Agent 目标通过统一 orchestrator 调用 Runner。
- 非消息事件不伪造成用户文本消息。
- 平台动作执行通过显式 capability / permission / result type 预留,不混入普通文本回复
- 平台动作通过已授权的语义工具执行;结构化交互通过 `action.requested` 中的 `interaction.requested` 白名单执行
## 2. 事件不是消息
@@ -22,7 +22,7 @@
| event_type | actor | subject | input |
| --- | --- | --- | --- |
| `message.received` | 发消息的人 | 当前消息 | 文本、图片、文件等 |
| `message.recalled` | 撤回操作者,未知时为系统 | 被撤回消息 | 通常为空 |
| `message.deleted` | 撤回操作者,未知时为系统 | 被删除消息 | 通常为空 |
| `group.member_joined` | 新成员或邀请人 | 群/成员关系 | 通常为空 |
| `friend.request_received` | 申请人 | 好友申请 | 验证消息或申请理由 |
| `schedule.triggered` | 系统 | 定时任务 | 任务 payload |
@@ -30,10 +30,10 @@
## 3. 稳定事件名
先保留的稳定事件名(作为插件协议的一部分保持稳定):
当前平台事件名示例(定时任务与 API 事件示例仅表示未来入口;实际能力以 SDK 实体和适配器声明为准):
- `message.received`
- `message.recalled`
- `message.deleted`
- `group.member_joined`
- `friend.request_received`
@@ -44,7 +44,7 @@
- 入口事件用 `AgentEventEnvelope`HOST_SDK §4.1)承载;顶层字段使用 LangBot 稳定协议名,平台原始事件名和原始 payload 放 `metadata` / `raw_ref`
- EBA 持久路由通过 `event_pattern``filters``target_type``target_uuid` 选择处理器。只有 `target_type=agent`,或 Pipeline AI Stage 需要调用 runner 时,才进一步解析 `AgentBinding`HOST_SDK §4.2)。
EBA 每个事件只选择一个有效处理器;AgentRunner 调用的基数、Agent 复用和 fan-out 边界以 PROTOCOL_V1 §13 为准。
EBA 每个事件只选择一个有效处理器;Runner 调用的基数、Agent 复用和 fan-out 边界以 PROTOCOL_V1 §13 为准。
路由 scope 示例:workspace 全局、bot 级、platform channel 级、conversation / group / thread 级、user / actor 级。Pipeline 是 `message.*` 场景的一等处理器,适合需要预处理、AI、后处理、扩展和输出控制的消息链路;Agent 是 runner 驱动的一等处理器,可处理其声明支持的消息与非消息事件。二者都不会被转换成对方。
@@ -53,33 +53,30 @@ Event Source 可包括:`platform_adapter`(飞书、QQ、微信、Telegram
## 5. EventRouter 调用链
```text
Platform Adapter / WebUI / API
-> Event Gateway normalize payload
-> EventLog append raw event
-> EventRouter resolve one Processor target
Platform Adapter canonical event
-> RuntimeBot record adapter event
-> Plugin EventListener observer broadcast
-> RuntimeBot match saved event_bindings and resolve one Processor target
-> target_type=pipeline: MessageAggregator -> QueryPool -> Pipeline stages
-> target_type=agent: resolve AgentBinding -> AgentRunOrchestrator
-> AgentRunContextBuilder -> PluginRuntimeConnector.run_agent()
-> AgentRunResult stream
-> DeliveryController render / platform action
-> RunnerContextBuilder -> PluginRuntimeConnector.run_runner()
-> RunnerResult stream
-> Host result delivery / authorized platform tool
```
约束:Pipeline 和 Agent 是 EventRouter 的平级目标;Pipeline 仅接受消息事件,Agent 受其事件能力声明约束。任何 AgentRunner 调用都必须复用现有 orchestrator,不能为 EBA 单独实现另一套 plugin runner 协议;非消息事件不能绕过 resource authorizationdelivery 和 platform action 走统一权限模型;外部 harness runner 也通过同一套 envelope/binding/context/result 协议接入。observer / fan-out / parallel arbitration 的额外语义仍按 PROTOCOL_V1 §13 处理。
约束:Pipeline 和 Agent 是 EventRouter 的平级目标;Pipeline 仅接受消息事件,Agent 受其事件能力声明约束。任何 Runner 调用都必须复用现有 orchestrator,不能为 EBA 单独实现另一套 plugin runner 协议;非消息事件不能绕过 resource authorizationdelivery 和 platform action 走统一权限模型;外部 harness runner 也通过同一套 envelope/binding/context/result 协议接入。observer / fan-out / parallel arbitration 的额外语义仍按 PROTOCOL_V1 §13 处理。
## 6. 平台动作执行
EBA 后 `action.requested`PROTOCOL_V1 §7.3,当前仅 telemetry 不执行)将用于请求 host 执行平台动作
平台动作走统一工具入口,详见 [PLATFORM_ACTION_TOOLS.md](./PLATFORM_ACTION_TOOLS.md)
```json
{ "type": "action.requested",
"data": { "action": "friend.request.accept",
"target": {"platform": "wechat", "request_id": "..."},
"payload": {"reason": "policy matched"} } }
```
- `event_*` 的目标由 Host 从当前事件冻结;Agent 只填写动作参数。
- `platform_*` 允许填写目标,必须在 `allowed_platform_tools` 中显式选择。
- 最终资源与 Runner 权限、适配器能力及事件目标求交,执行时再校验运行身份和当前机器人。
- SDK/Python `call_tool` 和 scoped MCP gateway 使用同一 Host 授权;原始 `call_platform_api` 不作为 Agent 工具开放。
- `action.requested` 只执行白名单 `interaction.requested`,用于持久化交互和回调恢复;其它 action 仍是 telemetry,不能用于执行好友审核等平台动作。
Host 必须校验:binding / platform action policy 是否授权该 action、actor / bot / workspace 是否允许、是否需要人工审批,以及当前 run session / caller identity 是否匹配。EBA 还可能预留 `delivery.requested`(请求投递到某 surface)。
Delivery 方面,event 不一定回复到当前聊天窗口:消息事件通常带 reply target;系统事件可能没有默认 reply target,需要 runner 返回 `action.requested` 或由 binding 的 delivery policy 决定投递位置(`DeliveryContext` 见 PROTOCOL_V1 §5.7)。
事件可能没有默认 reply target;Host 不为缺少目标的事件猜测投递对象。Runner 只能使用当前授权允许的工具和投递能力(`DeliveryContext` 见 PROTOCOL_V1 §5.7)。
当前 Host 会把 adapter 声明的通用 API 投影到
`DeliveryContext.platform_capabilities.supported_apis`,并据此设置
`supports_edit` / `supports_reaction`。该投影只供 runner 选择输出形态,不构成
@@ -87,15 +84,13 @@ Delivery 方面,event 不一定回复到当前聊天窗口:消息事件通
## 7. 与 Context 协议的关系
EBA 事件进入 AgentRunner 时仍遵循 [AGENT_CONTEXT_PROTOCOL.md](./AGENT_CONTEXT_PROTOCOL.md)inline 当前事件、大 payload 用 raw/staged file ref、不默认 inline 完整 history、agent 按需通过 API 拉取、Host 保留 EventLog 和权限 guardrail。非消息事件可以被投影进 Transcript,但不能强制伪装为 user messageAgentRunner 根据 event type 自己决定是否纳入模型上下文。
EBA 事件进入 Runner 时仍遵循 [AGENT_CONTEXT_PROTOCOL.md](./AGENT_CONTEXT_PROTOCOL.md)inline 当前事件、大 payload 用 raw/staged file ref、不默认 inline 完整 history、agent 按需通过 API 拉取、Host 保留 EventLog 和权限 guardrail。非消息事件可以被投影进 Transcript,但不能强制伪装为 user messageRunner 根据 event type 自己决定是否纳入模型上下文。
## 8. 当前集成状态
当前分支已完成 EventRouter、Pipeline / Agent 平级处理器路由、Bot
`event_bindings` 持久化与 WebUI、AgentBinding 投影、路由 dry-run、合成测试事件、
运行状态和真实 OneBot 非消息事件到 Agent 的闭环。Pipeline 消息链和独立 Agent
均复用同一个 AgentRunner orchestrator / context / result 协议。
均复用同一个 Runner orchestrator / context / result 协议。
尚未落地的是 platform action permission model 和 `action.requested` 执行器;在显式
action allowlist、binding policy、adapter capability 和审批模型完成前,该 result 仍只
记录 telemetry,不执行平台副作用。
平台动作授权和结构化交互已实现,但真实平台/provider 验收不等同于单测通过。SDK 的 `platform_tools` 分类发现于 2026-09-05 检视时仍是未提交工作区改动。剩余发布工作和历史验证边界见 [STATUS.md](./STATUS.md)。通用订阅、Scheduler、Workflow 和多 Agent 串并联仍未作为产品交付。
@@ -0,0 +1,74 @@
# Event debug end-to-end regression — 2026-09-05
Environment: Windows, Vite :3000, Core :5300, standalone Plugin Runtime :5400/:5401, standalone Box :5410, Docker backend. Tests use browser controls in Edge and an isolated unbound Agent `E2E Debug 0905` (`49e52ae0-6a99-46ce-975e-70164b2786ec`). Local plugin maintenance uses the authenticated localhost management API because Edge file upload is disabled.
## Coverage inventory
- Message input: empty input, plain text, multi-turn context, draft edits during streaming.
- Structured events: group join/leave/ban, friend request/add, feedback, bot state, message edit/delete/reaction, platform and custom events.
- Payload validation: invalid JSON, non-object JSON, Unicode and nested fields.
- Execution trace: intermediate/final text, returned thinking, tool arguments/results, tool failure, no-output failure, completion without duplication.
- Lifecycle: save-and-run, provider failure and recovery, long output/autoscroll, switch Agent during run, reconnect.
- Visual checks: scroll containment, long tool JSON, error card, narrow viewport.
## Findings and fixes
1. Synthetic debug Query omitted public Workspace fields. Native tools failed during skill mount lookup. Project trusted ExecutionContext fields onto synthetic queries; real Docker exec returned `E2E_TOOL_OK` after repair.
2. LocalAgent context assembly omitted structured event data. Add a bounded user-role event facts message before current input, preserving ordinary empty-data message behavior. Browser join/leave/friend/feedback returned unique JSON probes after installing the patched local plugin.
3. A model failure before visible output left an empty Agent card. Hide output entries with no visible execution steps.
4. A completed message request cleared a newly edited draft. Clear only the submitted input value; the browser retained `NEXT_DRAFT_SHOULD_REMAIN`.
5. Tool transport completion was shown as success even with `result.ok=false`. Treat explicit failed tool results as failure; real exec exit 7/stderr is preserved.
6. Native file tools resolved a Core host path before selecting remote Box execution. Choose Box first for remote/no-host-root deployments, validate virtual path boundaries, and cover all five file tools.
7. This local legacy config had no `box.local.host_root`, so Docker had no writable `/workspace` mount. Add an explicit local development root and keep the read-only root filesystem enabled.
8. LocalAgent repeats previous turns in cumulative chunks after tools. Strip the already displayed prefix for those chunks; real thought text now appears once across write/read.
9. Remote grep generated Python `include = null` when no filter was supplied. Serialize it as a Python literal and test optional/quoted values.
## External configuration observations
`gpt-4.1-mini` under LangBot Models returned no available channel; NewAPI rockchin returned invalid token. These are reported by the trace; tests continued with working `claude-opus-4-8` and `deepseek-v4-flash` models. The test Agent now uses the latter. No credentials were changed.
## Results
### Browser outcomes
All 15 preset event types and one named custom event were executed through the UI. These exercise synthetic debug event dispatch, not delivery from real messaging platforms.
| Events | Observed outcome |
| --- | --- |
| `message.received` | Plain text marker, multi-turn tool execution and recovery succeeded. |
| `message.edited`, `message.deleted`, `message.reaction` | Returned `EDIT_10`, `DELETE_11`, `REACTION_12`. |
| `group.member_joined`, `group.member_left`, `group.member_banned` | Returned Unicode member data/`JOIN_42`, `LEFT_7`, `BANNED_20`. |
| `friend.request_received`, `friend.added` | Returned `FRIEND_8`, `ADDED_19`. |
| `bot.muted`, `bot.unmuted`, `bot.invited_to_group`, `bot.removed_from_group` | Returned `MUTED_13`, `UNMUTED_16`, `INVITED_17`, `REMOVED_18`. |
| `feedback.received` | Returned `FEEDBACK_9` and numeric rating from JSON. |
| `platform.specific`, `custom.e2e_probe` | Read nested JSON; returned `PLATFORM_14` and custom event type/`CUSTOM_15`. |
- Empty message, malformed JSON and non-object JSON did not start execution. Unicode, nested arrays/objects and an empty event object were exercised. Empty custom event name validation was not conclusively verified.
- Real Docker `exec`, `write`, `read`, `edit`, `glob`, and `grep` all succeeded after fixes. Files were limited to `/workspace/e2e-debug-0905`. Final grep without `include` returned `probe.txt`, line 1, `E2E_EDIT_OK`, total 1.
- Nonzero exec exit preserved exit code 7/stderr and displayed failure. `sleep 3` with `timeout_sec=1` returned `timed_out`, `ok=false`, about 1037 ms, and displayed failure with the actual timeout message.
- Provider-returned thinking, intermediate text, tool arguments/results and final text appeared in order. A repeated prior thought appeared once after the cumulative-prefix fix. Models that do not return thinking are not expected to display it.
- Two browser tabs ran independent debug sessions without mixed transcripts. While `exec sleep 30` showed running, switching Agents cleared the old transcript/running state. Returning and sending a new request produced `AFTER_CANCEL_OK`. This verifies UI cancellation/recovery; termination timing of the already launched container command was not independently measured.
- Core restart and standalone Plugin Runtime reconnection recovered successfully. The development services remain running.
- At the 1280 x 800 test viewport, the final transcript measured height 315, scroll height 5203, distance from bottom 0, and client/scroll widths both 333: automatic scrolling and horizontal containment passed. Screenshot inspected; viewport override reset afterward.
### Automated verification
- Core: 130 passed, 24 skipped across orchestrator, execution context, debug controller/service, native tools and skill tools. Skips are POSIX secure-host-filesystem cases unavailable on Windows; they still require Linux verification. Remote Box routing is covered on both capability branches.
- LocalAgent full suite: 185 passed with `PYTHONUTF8=1`.
- Frontend trace reducer: 5 passed; TypeScript `tsc --noEmit` passed.
- Ruff checks on changed Python implementation/tests and Git whitespace checks passed. Existing Pydantic deprecation warnings remain.
The isolated unbound Agent `E2E Debug 0905` is retained for reproduction. No real platform messages were sent. This run does not claim coverage of every provider, model fallback policy, Linux host file operations, or live platform adapter ingress.
## Follow-up: mock platform actions
The original run did not verify platform action tools: the synthetic envelope had no supported platform APIs or reply target. This was a coverage gap. Debug now supplies mock adapter capabilities and a synthetic reply target while retaining tool selection, event compatibility and parameter validation. Event targets remain frozen by the Host. Only platform operations are simulated; native tools still execute normally.
Verified in the browser on the user's `localagent test` Agent (`0dc5d7d3-07b2-4c4c-bb2b-a4f8e40e3a76`) without changing its welcome system prompt: `group.member_joined` caused a real `event_reply` tool call with `text: HelloDebug User`. Its mock result contained `api: send_message`, `target_type: group`, `target_id: debug-group`, `mock: true`, and `delivery: simulated`. The UI labels this as mock execution rather than text output or real delivery, and shows the tool count after completion.
Mock platform responses are deterministic fixtures, not evidence of real adapter support or delivery. Read operations return synthetic information; list operations return empty fixture lists. Service/platform-tool regressions: 44 passed, including mock reply, explicit-target send, identity/group lookup, request rejection, validation and assertions that the real bot manager is never accessed. TypeScript and Ruff checks passed.
The same browser run subsequently exposed repeated `event_reply` calls after successful mock results, and one invalid `event_get_actor` call with an unexpected `_call` parameter. The invalid call was rejected and shown as failure. The run was cancelled by switching Agents. Therefore this verifies the mock call/result path, but does not establish that this configured model completes the welcome workflow exactly once. At this checkpoint the cause had not been determined.
For the subsequent investigation, updated SDK-shaped Mock fixtures (including non-empty default lists), cancellation fix, complete event matrix, Linux verification and remaining provider/stream-pressure limits, see [the full follow-up report](./EVENT_DEBUG_FULL_QA_2026-09-05.md). Its results supersede the checkpoint counts and fixture description above.
@@ -0,0 +1,99 @@
# Event debug full QA and release verification — 2026-09-05
## Scope and inventory
This follow-up investigates repeated welcome actions, verifies the debug surface with standalone Box/Plugin Runtime, and checks the changes before committing/pushing Core, SDK, and LocalAgent. Real messaging-platform effects must remain isolated from debug runs.
| Area | Required checks | Evidence |
| --- | --- | --- |
| Root cause | Follow-up messages/tool IDs preserved; direct model replay before/after mock guidance; original welcome prompt | Sanitized local traces, browser tool count |
| Events | All 15 built-in event types plus custom; nested/Unicode data and identity/target mapping | Browser results and parameterized service tests |
| Platform actions | Every catalog tool; frozen and explicit targets; permission/event/API intersection | Parameterized host boundary tests, browser reply/query/moderation/request cases |
| Mock scenarios | Success, error, query fixture, unsupported API; malformed options rejected | Browser and service tests |
| Input | Empty message/name, invalid/non-object JSON, unsupported event, actor/subject validation | Browser and request tests |
| Trace | Thinking/text/tool ordering, final snapshot deduplication, status/error correctness, no-call summary | Browser and reducer tests |
| Lifecycle | Cancel retains partial record; rerun; tabs/Agent switching; provider failure/fallback; reconnect | Browser and transport/runner tests |
| Native tools | Exec/file operations, nonzero exit, timeout and path escape | Browser and native-tool tests |
| Layout | Long output/parameters, automatic bottom scroll, narrow view, KB card spacing | DOM geometry and screenshots |
| Release | Python/SDK suites, frontend build/lint, whitespace checks, change review, commit and push | Commands, counts and remote SHAs |
## Investigation
The real follow-up request contained the assistant tool call and successful tool result with matching IDs. No loss occurred at the LocalAgent → SDK → Host model boundary. Direct replay to the configured provider, bypassing the runner, reproduced `event_reply` after success. Adding explicit mock completion semantics stopped further calls in two direct replays. The previous LocalAgent system context omitted the debug/mock semantics although the tool result said no real platform operation occurred.
The configured upstream `claude-opus-4-8` also returned an unsolicited CLI identity statement in direct replay. This originates in the provider response, not the debug renderer. It is separate from preserving tool results; it must not be presented as normal LangBot-generated status.
## Final results
### Browser event matrix
Executed through the real Edge WebUI, against Core `:5300`, Vite `:3000`, standalone Plugin Runtime `:5400/:5401`, and Docker-backed standalone Box `:5410`. The unbound `E2E Debug 0905` Agent uses `deepseek-v4-flash`; the user's original Agent and welcome prompt remain unchanged and use `claude-opus-4-8`.
| Event | Observed result in this pass |
| --- | --- |
| `bot.invited_to_group` | Actor/group queries and `event_respond_group_invite(approve=true)` succeeded; request ID frozen to `debug-group-request`. |
| `bot.muted`, `bot.removed_from_group` | One `event_get_group` call each, correct group target. |
| `bot.unmuted` | One actor query; correct user target. |
| `feedback.received` | One mock reply, `FEEDBACK_MOCK_OK`, person target. |
| `friend.added` | One mock reply, `FRIEND_ADDED_OK`, person target. |
| `friend.request_received` | Acceptance and separate explicit rejection exercised; rejection preserved `approve=false`, Unicode remark and frozen request ID. |
| `group.member_banned` | One member lookup, SDK-compatible nested `user` and `group_id`. |
| `group.member_left` | One group lookup, SDK-compatible `id`/`name`. |
| `group.member_joined` | Welcome reply, configured failure, query fixture, unsupported API, and mute/unmute/kick scenarios. Original-model repeat caveat below. |
| `message.deleted` | One actor lookup, SDK-compatible `id`/`nickname`. |
| `message.edited` | One simulated deletion; frozen group, chat and message IDs. |
| `message.reaction` | One reply, `REACTION_OK`, and correct `👍` event data. |
| `message.received` | Real six-tool file/exec chain, plain-text recovery, cancellation, draft retention, nonzero exit and timeout. |
| `platform.specific` | Preserved nested arrays, booleans, null and `测试🙂`; no tools called. |
| `custom.event`, `custom.e2e` | Empty object and named custom event with `CUSTOM_中文🙂`; no tools called. |
The first invitation/default friend-request runs submitted the preset text; they are recorded as default-behavior cases, not as explicit one-call/rejection tests. The rejection was separately rerun with the actual submitted instruction verified. Browser automation reads the controlled input after filling before treating a scenario as submitted.
### Mock and execution behavior
- Platform Mock runs the actual model/tool selection, authorization, parameter validation and frozen-target resolution; only the adapter boundary is simulated. Tests assert that mock execution never accesses the real bot manager. Native tools retain real Box behavior.
- The platform catalog contains 24 tools. Parameterized tests exercise success and configured failure for all 24, plus permission/event/API filtering, explicit/frozen targets, SDK fixture shapes and invalid options. This is full catalog contract coverage, not 24 separate browser clicks or proof of live adapter support.
- `errors.event_reply = "E2E permission denied"` produced one failed call and the **模拟执行失败 · Mock** status. The model reported the actual error without retrying in this scenario.
- `results.event_get_actor` returned the configured `fixture-user-77` / `测试用户🙂`; the model used these values instead of the original event identity. Default query results now serialize SDK platform models; default lists contain one synthetic entry and can be overridden with `[]`.
- Disabling `send_message` removed `event_reply` from the model's available tools. It returned `REPLY_UNAVAILABLE`, with the explicit no-tool-call summary. Clearing Mock options restored action availability; the three moderation calls succeeded with the correct group/member IDs.
- Mock `[]`, unknown tool names, whitespace-only message/custom name, malformed event JSON and event JSON `[]` were rejected. Browser transcript counts did not increase for client-side validation failures. Backend tests additionally cover falsey/non-object actor, subject and data values, conflicting outcomes and invalid API names.
- Thinking returned by the provider, intermediate text, tool parameters, results and final text remain distinct and ordered. Missing thinking is not fabricated. Cumulative final snapshots and prior tool-turn prefixes are deduplicated without hiding actual repeated tool calls.
### Lifecycle, native tools and layout
- A browser cancellation regression exposed a stuck **停止调试** button: the `finally` block skipped resetting state for aborted requests. It now clears the matching controller and resets running state even after user cancellation. Retest retained partial thinking/tool arguments, marked the unfinished call **未返回结果**, showed the cancellation notice and restored **运行测试**.
- Two subsequent recovery requests encountered the bounded-stream error below. A later request in the same tab/session returned `42` successfully; further real tool runs also completed. Cancellation and recovery are therefore verified, but immediate model success after cancellation is not guaranteed.
- Editing `NEXT_DRAFT_FINAL_0905` while a `sleep 5` run was still awaiting its final response preserved the draft after completion.
- Real `write → read → edit → glob → grep → exec` succeeded. `final.txt` changed from `FINAL_WRITE` to `FINAL_EDIT`; grep without `include` found one match; exec returned `FINAL_EXEC` with exit code 0. Writes stayed under `/workspace/e2e-debug-0905`.
- `printf E2E_ERROR >&2; exit 7` preserved stderr and exit code 7 and displayed failure. `sleep 3` with `timeout_sec=1` returned `timed_out` in 1025 ms and displayed failure. Neither was retried.
- Separate tabs retained separate transcripts during concurrent runs. Agent switching reset the debug surface. File workspace sharing remains intentional; transcript isolation does not mean separate Box filesystems.
- The final transcript measured 674 px high with 16,630 px scroll height, distance from bottom 0, and equal client/scroll widths of 468 px. Visual inspection confirmed contained parameters/results and accessible input controls. The earlier pass also verified 1280 × 800; the final attempted viewport override did not change this tab, so that attempt is not counted as an additional narrow-screen result. Temporary overrides were reset.
- Knowledge-base card geometry: retrieval card bottom 1413 px, danger card top 1437 px, giving the expected **24 px** gap. Both adjacent card gaps measured 24 px.
- Core health check returned `ok`; Plugin Runtime was connected and all development service ports remained listening. Previous restart/reconnect evidence is in the earlier report.
### Automated checks
| Suite | Result |
| --- | --- |
| Core agent unit directory plus debug service/controller, native/skill tools and model conversion | 659 passed, 24 Windows-only skips |
| Final changed platform/debug service/controller checks | 133 passed |
| Linux Docker: native tools, skill tools and all platform tools | 150 passed, 0 skipped; includes the 24 POSIX cases skipped on Windows |
| SDK API suite plus runtime I/O handler | 411 passed |
| LocalAgent full suite (`PYTHONUTF8=1`) | 187 passed |
| Frontend unit suite | 69 passed |
| Skills CLI suite | 122 passed |
| Frontend TypeScript/Vite build and changed-file ESLint | Passed; existing large-bundle advisory remains |
| Changed Python Ruff; Git whitespace checks | Passed |
| Skills index generation, validate and index consistency | Passed |
Counts overlap where a final focused/Linux run repeats an earlier suite; do not add them as unique tests. Existing Pydantic deprecation warnings remain. This is the relevant subsystem regression set, not the entire repository/integration matrix. Provider fallback has deterministic LocalAgent tests; live fallback after a partially emitted response is deliberately unsupported and was not claimed as a successful browser fallback.
The full regression also found Windows portability problems in the skills tooling: LF-only frontmatter parsing, `/cases/` detection and native path separators in generated references. These are fixed and covered by the CLI suite. Two frontend source-contract expectations were stale after the existing three-tab workbench/layout changes; they were updated to the actual product structure.
### Remaining limits and reproduction
1. **Original provider is still nondeterministic.** With the latest LocalAgent guidance, one rerun made exactly one successful `event_reply`; another made repeated successful replies before stopping. The trace faithfully records each call. Earlier invalid `_call` arguments were rejected. Direct provider replay proved that repeated calls can originate upstream even with a valid matching tool result. Prompt guidance reduces ambiguity but does not establish exactly-once actions. No silent deduplication or fabricated success was added.
2. **Bounded stream pressure can fail long responses.** Two recovery runs emitted thinking and then `Streaming action consumer is too slow; response buffer full`; later recovery and six-tool runs succeeded. The error originates in the SDK's existing 128-frame response queue and is preserved in the UI. It is not a missing-tool-result or 30-second HTTP timeout. Full end-to-end flow control under sustained overload remains open; queue limits were not removed to conceal the failure.
3. Mock results verify debug behavior, not real platform credentials, permissions, delivery, or every adapter/event Cartesian combination. Already launched external tool termination timing was not independently measured. A development-branch push does not mean these remaining production release gates passed.
For reproduction, use the retained test Agent, the Mock examples in [RUNNER_QA_GUIDE.md](./RUNNER_QA_GUIDE.md), and the original welcome Agent. Local diagnostic traces and fixture artifacts are excluded from Git; they contain model responses and local runtime state. This report contains outcomes rather than raw provider reasoning or secrets.
@@ -1,51 +1,41 @@
# AgentRunner 外化扩展边界矩阵
# Runner 与产品扩展边界
本文用于回答一个问题:本分支只做 AgentRunner 外化时,哪些能力已经作为扩展底座完成,哪些由外部 EBA / Agent Platform / Runtime Control Plane 分支接入,后续分支接入时应该走哪个扩展点
更新:2026-09-05,适用于 `dev/4.11.x`。EBA、独立 Agent、Bot 事件绑定和处理器 UI 已与 Runner 插件化合并。当前状态和测试证据以 [STATUS.md](./STATUS.md) 为准;runner 可见 schema 与调度基数以 [PROTOCOL_V1.md](./PROTOCOL_V1.md) 为准
结论:本分支不实现完整 Agent Platform,也不实现完整 EBA。EBA 完整事件网关与事件路由由外部 EBA 分支联调。本分支必须把 runner 外化的 Host / SDK 边界做干净,让外部分支只需要接入持久模型、事件路由或 runtime task,而不需要重写 `AgentRunner Protocol v1`
## 当前职责
调度基数、Agent 复用、插件实例无状态、Pipeline adapter 和 fan-out 边界的单一事实源是 [PROTOCOL_V1.md](./PROTOCOL_V1.md) §13;本矩阵只说明后续能力应该接入哪个扩展点。
## 1. 分支边界
| 范围 | 本分支职责 | 不在本分支做 |
| 层 | 已实现职责 | 边界 |
| --- | --- | --- |
| AgentRunner Protocol v1 | 定义 Host 调用 runner 的稳定合同:discovery、`AgentRunContext`、result stream、Host pull API、错误和权限边界。 | 不定义 Agent Platform 的产品数据库模型;不定义 runtime task queue。 |
| Host runner 外化底座 | 提供 `AgentEventEnvelope``AgentBinding` 运行投影、`run(event, binding)`、resource authorization、run-scoped session、EventLog / Transcript / State / sandbox 文件边界。 | 不实现 EventGateway、scheduler、integration provider、Agent 管控面 UI。 |
| Pipeline 的 AgentRunner 接入 | Pipeline 作为一等消息处理器执行完整 Stage 链;仅在 AI Stage 调用 runner 时,`QueryEntryAdapter` 把当前 Query/config 投影成 event + binding。 | 不把整个 Pipeline 当成临时 Agent;不复制 Pipeline 配置来自动创建 Agent。 |
| 官方 runner 插件 | 作为协议消费者验证 local-agent / 外部 harness runner 能接入 Host 基础设施。 | 不让官方 runner 的内部实现反向决定 Host / SDK 协议形态。 |
| LangBot 产品层 | 独立 Agent CRUD、Pipeline、处理器工作台、Bot 事件绑定、Runner 安装与调试 | Agent 与 Pipeline 各自持久化;聚合列表不转换实体 |
| 平台层 | 适配器事件转换、能力声明、observer 广播、路由匹配、平台 API 与回复 | 路由逻辑在 RuntimeBot;不为每个入口重建 runner 协议 |
| Host Agent 底座 | envelope/binding 投影、统一编排、资源授权、run session、EventLog/Transcript/State、run/result ledger | SDK 不持有 Host 私有 Query 或数据库 |
| SDK / Plugin Runtime | typed contract、Runner 组件和脚手架、proxy、MCP bridge、结果流转发、installation worker 管理 | 具体 Agent 执行策略由 Runner 插件承担 |
| Box Runtime | 沙盒会话、文件、托管进程、Skill、资源限制与作用域 | 不等于外部 harness 的通用托管承诺;存储统计不等于硬配额 |
## 2. 扩展矩阵
## 已有能力与后续扩展
| 能力 | 当前分支状态 | 后续归属 | 后续接入方式 | 禁止事项 |
| --- | --- | --- | --- | --- |
| Product `Agent` | 已有 `agents` 产品表 / API 和运行期 `AgentConfig` / `AgentBinding` 投影;完整 binding persistence / EventRouter / UI 闭环仍未完成。 | Agent Platform / binding persistence UI。 | 持久 Agent 保存 runner id、runner config、resource/state/delivery policy;运行前投影为 `AgentBinding`。 | 不把持久 Agent schema 加进 SDK 协议;插件实例边界见 PROTOCOL_V1 §13。 |
| Agent 处理器调用 runner | 已有单次运行前的 `AgentBinding` 解析投影;AgentRunner 调度语义见 PROTOCOL_V1 §13。 | EBA / Agent Platform。 | EventRouter 先选中 Agent 处理器,再根据 bot、channel、workspace、conversation、event type 解析有效 `AgentBinding`。Pipeline 目标走独立 Stage 链。 | 不用 `AgentBinding` 取代 EBA 的 Pipeline / Agent 处理器选择;不在本矩阵重定义 fan-out / observer 语义。 |
| Agent session / run | 已有持久 `AgentRun` / `AgentRunEvent` ledger 和 active `AgentRunSessionRegistry`;还没有独立 `AgentSession` / task 产品模型。 | Agent Platform / Runtime Control Plane。 | 如需要可新增 `AgentSession` / task 表,但执行仍回到 `run(event, binding)` 或 runtime-managed 等价入口。 | 不把持久 session 字段塞进 `AgentRunContext` 顶层;不要求所有 runner 长期持有 LangBot session |
| EventLog / Transcript / Sandbox files | 已完成 Host-owned store、history pull API 和 sandbox 文件边界;runner 不直接写 DB。 | 本分支持续维护底座;Agent Platform 可复用。 | 外部 EBA、scheduler、integration、runtime task 都写同一套 EventLog / Transcript;当前 run 文件通过 sandbox/workspace staging 共享。 | 不让 runner / sandbox 直接访问 Host DB;不把大 payload 内联进 prompt。 |
| Host-owned state / storage | 已有 state snapshot、`state.updated` 处理和 State APIstorage 作为授权能力保留。 | 本分支持续维护底座;Runtime / Platform 可复用。 | 外部 session id、working directory、checkpoint 等小 JSON 用 state;当前 run 大对象用 sandbox/workspace 文件。 | 不把跨轮次状态存在插件实例内;不绕过 run-scoped authorization。 |
| EventGateway / EventRouter | 本分支只提供 event-first envelope 和 `run(event, binding)` 入口。 | EBA 分支(联调中)。 | EventGateway 规范化平台/WebUI/API/scheduler 事件;EventRouter 解析一个 binding;调用现有 orchestrator。 | 不为 EBA 新增另一套 runner 调用协议;不把非消息事件伪装成 user message。 |
| Scheduler / Automation | 不实现。文档中只把 `scheduler` 作为 future event source。 | EBA / Agent Platform。 | 定时任务触发 `schedule.triggered` host event,复用 EventGateway -> EventRouter -> `run(event, binding)`。 | 不直接调用某个 runner 插件;不绕过 EventLog / authorization。 |
| Integration provider | 不实现。IM platform adapter 仍是当前平台接入系统。 | EBA / Agent Platform。 | OAuth/webhook/outbound provider 应先转成 canonical host event 或 platform action,再交给 AgentRunner。 | 不把 Linear/Slack/GitHub 等 provider 私有 payload 扩散到 runner 协议顶层。 |
| Platform action / delivery | `action.requested` 已预留但当前仅 telemetry,不执行。`DeliveryContext` 只作为上下文/策略投影。 | EBA / platform action executor。 | 后续 executor 校验 runner capability、binding policy、actor/bot/workspace 权限和审批后执行。 | 不让 runner 直接调用平台 adapter 私有 API;不把平台动作伪装成文本回复副作用。 |
| Runtime registry / worker / task queue | 已落地 Host-owned `AgentRun` / `AgentRunEvent`、run control primitives、最小 runtime registry / heartbeat / claim lease;当前官方外部 harness 仍通过 ACP、远端 daemon、本机 subprocess 或外部 HTTP API runner 调用目标运行环境,不在本分支维护完整通用 worker 队列。 | Runtime Control Plane v2。 | 后续可在现有 Host 事实源上补 queued run producer、daemon wakeup、claim execution loop、progress/audit 和运维诊断。 | 不把 heartbeat/task/warm pool 放进 Protocol v1;不让管理插件拥有 runtime/task 事实源。 |
| Warm pool / reconcile / diagnose | 不实现。 | Runtime Control Plane v2 / deployment layer。 | 作为 task/runtime 的运维能力,围绕 Host-owned runtime/task/audit 表实现 | 不把 runtime 运维语义写进普通 runner 协议;不把 pod/task 细节泄漏给普通 runner。 |
| Agent memory | 不实现通用长期记忆产品层;提供 history/state/storage 和 sandbox 文件基础能力。 | Agent Platform 或具体 runner/plugin。 | 平台 memory 可通过 Host storage/state 或独立产品表实现,runner 通过授权 API 拉取。 | 不在 Host core 内置通用 agentic memory 策略;不默认把 memory 全量 inline 到 context |
| External harness native session | ACP / Claude Code / Codex 等 runner 支持 external session id state handoff 和 LangBot resource projection。 | 官方 runner 后续增强;Runtime Control Plane v2 可接管执行。 | 外部 harness 调用继续走 `runner.run(ctx)`;如后续引入长连接/daemon 模式,按 external session key 串行 turnreader 独占 native stream。 | 不把具体 provider native wire 变成 LangBot 协议;全局锁边界见 PROTOCOL_V1 §13。 |
| 能力 | 当前状态 | 后续工作与接入点 |
| --- | --- | --- |
| Agent / binding | Agent 表、API、配置 UI、Bot event_bindings 已存在;AgentBinding 是运行投影 | 新产品模型复用现有投影,不把 Pipeline 持久化成 Agent |
| 事件路由 | observer 广播后按 pattern/filter/priority 选择一个 Pipeline、Agent 或 discard | 通用订阅、通知和其他事件源仍需单独设计 |
| 平台动作 | event_* 冻结目标,platform_* 显式授权;通过 Host 工具调用 | 新动作先定义语义、schema 和授权,不开放任意原始 action |
| 结构化交互 | interaction.requested 白名单、持久回调关联、TTL/作用域/幂等、原处理器恢复 | 补真实 provider/platform 验收;其它 action.requested 仍仅 telemetry |
| Run / runtime | 持久 AgentRun/AgentRunEvent、取消/结果/终态、heartbeat/claim/reconcile 原语 | 业务队列、任务生产、唤醒、跨 Host 执行和运维产品面 |
| Plugin worker | 独立安装进程、依赖环境、supervisor、退避及重启协调器 | 最终部署故障注入、出站网络策略及硬存储配额 |
| External harness | 通过 Runner 消费协议、按 run 访问 Host 资源 | 通用 daemon supervisor、登录态诊断、分布式调度;不要与 Plugin worker 混淆 |
| History / state / storage | Host 事实源、按需读取、state/checkpoint、sandbox 文件能力 | EventLog/Transcript 的定时 retention 接入和完整文件生命周期 |
| Scheduler / Automation | 仅保留可扩展的事件入口 | 用户定时任务必须走事件、授权和运行记录链路,不直调插件绕过 Host |
| Workflow / 多 Agent | 尚无完整产品实现 | 先定义串并联、失败恢复、投递与状态冲突语义 |
| Solution | 尚无导出/导入实现 | 处理器、路由模板、依赖、变量与文档;不导出凭据或已安装 UUID |
| 长期 memory 产品 | 提供 history/state/storage 基础 | 由 Runner 或后续产品定义召回策略,不把全量 memory 默认塞入 context |
| Cloud | 作用域与运行时隔离底座已合入;OSS 为单 Workspace 多成员 | 生产激活独立通过网络、硬配额、事务代次切换和部署验收 |
## 3. 后续分支接入规则
平台动作详见 [PLATFORM_ACTION_TOOLS.md](./PLATFORM_ACTION_TOOLS.md),控制面规划详见 [RUNTIME_CONTROL_PLANE_V2.md](./RUNTIME_CONTROL_PLANE_V2.md)Cloud 门禁见 [剩余验证清单](../multi-tenant/cloud-v2-pending-verification.md)。
外部 EBA、Agent Platform 或 Runtime Control Plane 分支接入时,默认遵守以下规则:
## 扩展规则
- 新入口只生产或解析 Host 内部模型:`AgentEventEnvelope`、持久 Agent 投影出的 `AgentBinding`、以及必要的 delivery/resource/state policy
- runner 调用仍走 `AgentRunOrchestrator.run(event, binding)`,除非 Runtime Control Plane 明确引入 runtime-managed 执行模式;即便如此,runner 可见合同仍应保持 Protocol v1
- Host-owned facts 继续写入 EventLog / Transcript / State,当前 run 文件继续走 sandbox/workspace;产品层可以新增更高阶视图,但不能替代这些事实源
- 新能力如果需要持久化,优先加 Host-owned 表或 service;不要把事实源藏在插件 storage 或 runner subprocess 内
- 新 result type 可以按 Protocol v1 的演进规则增加;不能用入口 adapter 私有字段绕过 schema
- 任何 fan-out、observer agent、parallel arbitration、platform action execution 都必须单独定义 delivery、state conflict、approval 和 audit 语义。
## 4. 与 Agent Platform 产品层的关系
这里的 Agent Platform 指面向 agent 产品层的实体拆分:`Agent` 描述可配置 agent`Session` / `SessionMessage` 描述会话事实,`Automation` 描述自动触发,`IntegrationBinding` 描述外部集成连接,`Memory` 描述长期记忆,`WarmTask` 描述预热/后台任务。这些拆分对 LangBot 后续产品层有参考价值,但不能直接搬进本分支。
LangBot 当前分支的对应目标是更底层的:把 IM/WebUI/API 等入口统一投影到 Host event,把 Agent / binding 配置统一投影到 runner binding,把 runner 能力统一收束到 Protocol v1。完整 Agent Platform 可以在这个底座之上构建,而不应反过来污染本分支的 runner 外化边界。
- 新入口构造 Host event 和有效 binding,继续调用统一 orchestratorPipeline AI Stage 使用 QueryEntryAdapter
- Host 保持 run/result、授权、事件、状态和历史的事实源;插件负责自己的执行策略及 provider 私有 continuation
- 新增业务表、调度或 UI 不要求修改 runner 可见协议;需要协议扩展时先更新 canonical spec,再同步 SDK、Runtime、模板与测试
- fan-out、并行仲裁和自动重试必须明确副作用、幂等、状态冲突和审计语义,不能通过多个隐式回复者实现
- 外部 harness 自带 shell、文件系统和网络权限由部署环境负责;manifest permissions 约束的是 LangBot 持有的资源
@@ -2,70 +2,70 @@
本文档描述 LangBot 作为 agent host 的内部能力与分层架构,以及 Host 内部模型。
- SDK ↔ Host 的协议数据结构(`AgentRunContext``AgentRunnerManifest``AgentRunResult``AgentRunAPIProxy` 等)的**唯一定义在** [PROTOCOL_V1.md](./PROTOCOL_V1.md);本文只引用,不重抄。
- 测试执行入口和 smoke 记录见 [AGENT_RUNNER_QA_GUIDE.md](./AGENT_RUNNER_QA_GUIDE.md);安全发布门槛见 [SECURITY_HARDENING.md](./SECURITY_HARDENING.md)。
- 本文定义的 Host 内部模型(`AgentEventEnvelope``AgentBinding``AgentRunnerDescriptor`)不属于 SDK 协议字段。
- SDK ↔ Host 的协议数据结构(`RunnerContext``RunnerManifest``RunnerResult``RunnerAPIProxy` 等)的**唯一定义在** [PROTOCOL_V1.md](./PROTOCOL_V1.md);本文只引用,不重抄。
- 测试执行入口和 smoke 记录见 [RUNNER_QA_GUIDE.md](./RUNNER_QA_GUIDE.md);安全发布门槛见 [SECURITY_HARDENING.md](./SECURITY_HARDENING.md)。
- 本文定义的 Host 内部模型(`AgentEventEnvelope``AgentBinding``RunnerDescriptor`)不属于 SDK 协议字段。
## 1. 目标
LangBot 要转为 agent host,而不是内置 runner 容器:
- 接收 IM、WebUI、API 和外部 EBA 分支 EventRouter 产生的事件。
- 接收 EBA 选中的 Agent 处理器,并根据事件、bot、workspace、scope 解析 AgentRunner binding。
- 发现、校验和调用插件提供的 AgentRunner。
- 接收 IM、WebUI、API 和当前 RuntimeBot 事件路由产生的事件。
- 接收 EBA 选中的 Agent 处理器,并根据事件、bot、workspace、scope 解析 Runner binding。
- 发现、校验和调用插件提供的 Runner。
- 为每次 run 提供受限资源、状态、存储、上下文引用和生命周期控制。
- 接收 AgentRunner 返回的事件流,投递到 IM、WebUI 或其他 output surface。
- 接收 Runner 返回的事件流,投递到 IM、WebUI 或其他 output surface。
## 2. 非目标
- 不定义 Pipeline 的 Stage 编排语义;Pipeline 是 EBA 的同级处理器,其 AI Stage 只在需要 runner 时接入本 Host 边界。
- 不要求所有 AgentRunner 依赖 LangBot 的上下文管理。
- 不要求所有 Runner 依赖 LangBot 的上下文管理。
- 不要求官方 local-agent 的旧行为反向塑造 host 协议。
- 不在 host 中实现通用 agentic prompt assembler。
- 不强制 runner 使用 LangBot state / storage;只提供可选、受控的寄宿能力。
-实现 EventGateway / EventRouter:它们由外部 EBA 分支提供并联调。本分支只定义 host-side envelope/binding models 和 `run(event, binding)` 入口
-在 runner 底座重复实现平台路由;当前分支的 RuntimeBot 已承担 observer 广播和 Bot event_bindings 匹配,runner 层消费其 envelope/binding 投影
## 3. 分层架构
```text
IM / WebUI / API / EventRouter (external EBA branch)
IM platform event
|
v
Event Gateway (external EBA branch)
RuntimeBot event normalization / observer broadcast
|
v
EventRouter -> one Processor target
RuntimeBot event_bindings -> one Processor target
|-- target_type=pipeline -> Pipeline Stage chain
|
`-- target_type=agent -> AgentBindingResolver
|
v
AgentRunOrchestrator
|-- AgentRunnerRegistry
|-- RunnerRegistry
|-- AgentResourceBuilder
|-- AgentContextBuilder
|-- AgentRunSessionRegistry
|-- PersistentStateStore / EventLogStore / TranscriptStore
|-- Sandbox / workspace file tools
v
Plugin Runtime / AgentRunner
Plugin Runtime / Runner
|
v
AgentRunResult stream
RunnerResult stream
|
v
Delivery / Renderer / Platform API
```
Pipeline 与 Agent 是 EventRouter 的平级处理器目标。本文只定义 AgentRunner Host 边界:Agent 目标直接解析 `AgentBinding`Pipeline 目标执行自己的完整 Stage 链,仅在 AI Stage 调用 runner 时通过 Query entry adapter 构造一次性 `AgentConfig` / `AgentBinding`。该 runner 调用投影不改变 Pipeline 的一等处理器地位,也不会把 Pipeline 持久化 Agent。AgentRunner 的单绑定调度、Agent 复用、插件实例无状态和 fan-out 边界以 [PROTOCOL_V1.md](./PROTOCOL_V1.md) §13 为准。EventGateway / EventRouter 由外部 EBA 分支实现并联调。
Pipeline 与 Agent 是事件路由的平级目标。Agent 直接解析 `AgentBinding`Pipeline 执行完整 Stage 链,仅在 AI Stage 调用 runner 时通过 QueryEntryAdapter 构造一次性运行投影,不会持久化 Agent。WebUI/API 调试按服务入口构造调试事件或 Query,再使用同一编排器;不要求绕行真实 Bot 回调。调度基数、Agent 复用和 fan-out 边界以 [PROTOCOL_V1.md](./PROTOCOL_V1.md) §13 为准。
## 4. LangBot 侧能力
### 4.1 Event Gateway / EventRouterExternal EBA Branch Integration Point
### 4.1 事件入口与路由(已集成
> EventGateway / EventRouter 由外部 EBA 分支实现并联调,不在本分支范围。本分支只保留 event-first 入口和 envelope/binding models
> 2026-09-05:平台转换、Bot event_bindings、独立 Agent 和配置 UI 已集成。源码入口为 `pkg/platform/botmgr.py`、`pkg/api/http/service/agent.py` 和 `pkg/agent/runner/`。通用订阅与定时自动化仍是后续能力
Event Gateway 将把入口统一成 host eventIM 平台消息、WebUI debug chat、API 触发、后续非消息事件),输出稳定的 `AgentEventEnvelope`Host 内部模型)
各入口将消息或非消息输入投影为 `AgentEventEnvelope`。以下为概念字段摘要,精确字段及默认值以 `pkg/agent/runner/host_models.py` 为准
```python
class AgentEventEnvelope(BaseModel):
@@ -91,7 +91,7 @@ class AgentEventEnvelope(BaseModel):
### 4.2 AgentConfig 与 AgentBinding
`AgentConfig` 是 Host 内部的一次 AgentRunner 调用配置投影(不暴露给 SDK)。独立 Agent 从自己的持久配置生成它;Pipeline 只在 AI Stage 调用 runner 时,由 Query entry adapter 从该 Stage 的当前配置生成它。两种来源随后都由 BindingResolver 结合事件和 scope 解析为 `AgentBinding`。Pipeline 本身不是 `AgentConfig`,该调用投影也不会创建或更新持久 Agent。
`AgentConfig` 是 Host 内部的一次 Runner 调用配置投影(不暴露给 SDK)。独立 Agent 从自己的持久配置生成它;Pipeline 只在 AI Stage 调用 runner 时,由 Query entry adapter 从该 Stage 的当前配置生成它。两种来源随后都由 BindingResolver 结合事件和 scope 解析为 `AgentBinding`。Pipeline 本身不是 `AgentConfig`,该调用投影也不会创建或更新持久 Agent。
```python
class AgentConfig(BaseModel):
@@ -106,7 +106,7 @@ class AgentConfig(BaseModel):
metadata: dict[str, Any] = {}
```
`AgentBinding` 是"什么事件调用哪个 AgentRunner、带什么 Agent 配置"的 Host 内部运行投影(不暴露给 SDK)。它是 EventRouter / 当前 QueryEntryAdapter 在一次运行前解析出的有效绑定。
`AgentBinding` 是"什么事件调用哪个 Runner、带什么 Agent 配置"的 Host 内部运行投影(不暴露给 SDK)。它是 EventRouter / 当前 QueryEntryAdapter 在一次运行前解析出的有效绑定。
```python
class AgentBinding(BaseModel):
@@ -131,12 +131,12 @@ BindingResolver 的基数、fan-out 和冲突处理约束见 PROTOCOL_V1 §13
→ runner_config、extension preference → resource_policy、output settings →
delivery_policy,但 Pipeline 仍执行并拥有完整 Stage/config 语义。该适配不会把 Pipeline 持久化为 Agent;独立 Agent 由用户自行新增和绑定。
### 4.3 AgentRunnerRegistry
### 4.3 RunnerRegistry
Registry 收集 runner descriptor(来自插件 runtime、开发期本地插件):
```python
class AgentRunnerDescriptor(BaseModel):
class RunnerDescriptor(BaseModel):
id: str
source: Literal["plugin"]
label: I18nObject
@@ -144,17 +144,17 @@ class AgentRunnerDescriptor(BaseModel):
plugin_author: str
plugin_name: str
runner_name: str
capabilities: AgentRunnerCapabilities # 见 PROTOCOL_V1 §4.3
permissions: AgentRunnerPermissions # 见 PROTOCOL_V1 §4.4
capabilities: RunnerCapabilities # 见 PROTOCOL_V1 §4.3
permissions: RunnerPermissions # 见 PROTOCOL_V1 §4.4
config_schema: list[DynamicFormItemSchema]
plugin_version: str | None = None
raw_manifest: dict[str, Any] = {}
```
职责:调用 `plugin_connector.list_agent_runners()` 拉取 runner、校验 typed `AgentRunnerManifest`、输出 descriptor、缓存 discovery 结果并提供 `refresh()`。单个插件 manifest 失败只记 warning,不影响其它 runner。`plugin:author/name/runner` 是稳定 id 格式;插件实例边界见 PROTOCOL_V1 §13。
职责:调用 `plugin_connector.list_runners()` 拉取 runner、校验 typed `RunnerManifest`、输出 descriptor、缓存 discovery 结果并提供 `refresh()`。单个插件 manifest 失败只记 warning,不影响其它 runner。`plugin:author/name/runner` 是稳定 id 格式;插件实例边界见 PROTOCOL_V1 §13。
Host 内置 runner / adapter 不能作为 `AgentRunnerDescriptor.source` 绕过插件
runtime、`run_id``ctx.resources``AgentRunAPIProxy` 权限链。若需要
Host 内置 runner / adapter 不能作为 `RunnerDescriptor.source` 绕过插件
runtime、`run_id``ctx.resources``RunnerAPIProxy` 权限链。若需要
开发期调试 adapter,应放在 Host 内部测试入口,不进入可选 runner 列表。
刷新触发点:插件安装/卸载/升级/重启后;Pipeline metadata 请求时发现缓存为空;可选 TTL(优先保证正确性)。
@@ -182,15 +182,15 @@ run(event, binding)
```text
QueryEntryAdapter / EventRouter
-> AgentRunOrchestrator.run(event, binding)
-> AgentRunnerRegistry.resolve(runner_id)
-> RunnerRegistry.resolve(runner_id)
-> AgentResourceBuilder.freeze_snapshot(binding, event)
-> AgentRunSessionRegistry.register(run_id, runner_id, snapshot)
-> AgentContextBuilder.build(event, binding, snapshot)
-> PluginRuntimeConnector.run_agent(ctx)
-> AgentRunAPIProxy action
-> PluginRuntimeConnector.run_runner(ctx)
-> RunnerAPIProxy action
-> validate active run session + caller identity + snapshot
-> Host API / Store
<- AgentRunResult stream
<- RunnerResult stream
-> apply state.updated to PersistentStateStore
-> write message.completed to Transcript
-> keep current-run files and large tool outputs in sandbox/workspace
@@ -221,7 +221,7 @@ SDK 侧本地校验只用于开发体验,host 侧 run authorization snapshot
资源裁剪应通用,不写死 local-agent。selector 与资源的映射示例:`model-fallback-selector` → primary/fallback LLM、`llm-model-selector` → LLM、`rerank-model-selector` → rerank 模型、`knowledge-base-multi-selector` → 知识库;新增 selector 时在 resource builder 中统一扩展。
构造 `ctx.resources.tools` 时,Host 一次塞齐每个工具的完整 schema`ToolResource.parameters`),runner 不需再逐个 `get_tool_detail` 拉取,减少 N 次往返
构造 `ctx.resources.tools` 时,Host 尽可能一次提供完整 schema`ToolResource.parameters`),减少逐个 detail 查询。Runner 仍需兼容 `parameters=None` 并按需调用 `get_tool_detail`。平台动作以 `tool_type=platform` 使用同一资源面,授权与冻结目标见 [PLATFORM_ACTION_TOOLS.md](./PLATFORM_ACTION_TOOLS.md)
执行/文件/skill/MCP 等能力的接入方向:先由 Host / sandbox 封装成普通 scoped tool,再通过 `ctx.resources.tools` 和 SDK runtime 转发进入 runnerrunner 不应识别或硬编码执行环境 provider。外部 harness 的 native tools 不能直接访问 LangBot 资源。skill 的整个生命周期都走统一 tool:发现走 `list_skills` / `langbot_list_assets`,激活/注册走 `activate` / `register_skill`,包内操作走 native exec/read/write——runner 不需要独立的 skill 渲染或门控。
@@ -237,11 +237,11 @@ LangBot 可提供 host-owned state 让 runner 寄宿状态(conversation / acto
- `Transcript`: 从 EventLog 投影出的对话视图,用于 UI、审计和按需历史读取。
- `Sandbox / workspace files`: 当前 run 的上传文件、平台附件、工具大结果和临时产物。Host 负责 staging 与授权边界,runner 通过 read/write/exec 类工具按需访问。
三类数据与 working context 的边界、读取约束见 [AGENT_CONTEXT_PROTOCOL.md](./AGENT_CONTEXT_PROTOCOL.md)。AgentRunner 可读取这些能力,但不被迫使用 LangBot 作为唯一记忆系统。
三类数据与 working context 的边界、读取约束见 [AGENT_CONTEXT_PROTOCOL.md](./AGENT_CONTEXT_PROTOCOL.md)。Runner 可读取这些能力,但不被迫使用 LangBot 作为唯一记忆系统。
### 4.8 External harness resource projection
Claude Code、Codex、Kimi Code 等外部 harness runner 可能不直接调用 LangBot 的 model/tool loop,而是把 LangBot 事件和授权资源句柄投影到自己的 harness 执行。Host 侧仍保持统一边界:Host 负责构造 event-first context、资源授权、state/storage、EventLog/Transcript、sandbox/workspace 文件边界和审计;Host 或 binding policy 决定哪些 MCP bridge、skill-backed tool、sandbox path、history/state 句柄可投影给 runnerrunner plugin 把 scoped projection 转成目标 harness 可消费形式;所有 LangBot 资源访问必须经 SDK runtime / `AgentRunAPIProxy` / SDK-owned MCP bridge 转发并接受 Host 校验;外部 harness 负责自己的 native session、tool loop、压缩、权限模式和 resume,但不能用 native tools 绕过 Host 授权。
Claude Code、Codex、Kimi Code 等外部 harness runner 可能不直接调用 LangBot 的 model/tool loop,而是把 LangBot 事件和授权资源句柄投影到自己的 harness 执行。Host 侧仍保持统一边界:Host 负责构造 event-first context、资源授权、state/storage、EventLog/Transcript、sandbox/workspace 文件边界和审计;Host 或 binding policy 决定哪些 MCP bridge、skill-backed tool、sandbox path、history/state 句柄可投影给 runnerrunner plugin 把 scoped projection 转成目标 harness 可消费形式;所有 LangBot 资源访问必须经 SDK runtime / `RunnerAPIProxy` / SDK-owned MCP bridge 转发并接受 Host 校验;外部 harness 负责自己的 native session、tool loop、压缩、权限模式和 resume,但不能用 native tools 绕过 Host 授权。
投影的具体形态(context 文件、resource handles、LangBot MCP gateway、state pointers)见 AGENT_CONTEXT_PROTOCOL §4.5;当前 code-agent harness runner 形态见 OFFICIAL_RUNNER_PLUGINS §7。发布级隔离要求见 SECURITY_HARDENING。
@@ -250,17 +250,17 @@ Claude Code、Codex、Kimi Code 等外部 harness runner 可能不直接调用 L
SDK 组件入口如下;所有数据结构定义见 PROTOCOL_V1。
```python
class AgentRunner(BaseComponent):
__kind__ = "AgentRunner"
class Runner(BaseComponent):
__kind__ = "Runner"
@classmethod
def get_config_schema(cls) -> list[dict]: ...
async def run(self, ctx: AgentRunContext) -> AsyncGenerator[AgentRunResult, None]: ...
# ctx: PROTOCOL_V1 §5.2 ; AgentRunResult: PROTOCOL_V1 §7
async def run(self, ctx: RunnerContext) -> AsyncGenerator[RunnerResult, None]: ...
# ctx: PROTOCOL_V1 §5.2 ; RunnerResult: PROTOCOL_V1 §7
```
- Manifest / capabilities / effective accessPROTOCOL_V1 §4。Capabilities 来自组件 manifest 的 `spec.capabilities`,不是 SDK 基类 classmethod。
- `AgentRunContext`PROTOCOL_V1 §5.2。`messages` / `bootstrap` 不是协议字段。
- `AgentRunResult`PROTOCOL_V1 §7。
- `AgentRunAPIProxy`PROTOCOL_V1 §8,是 runner 访问 host 能力的唯一入口,所有请求带 `run_id`
- `RunnerContext`PROTOCOL_V1 §5.2。`messages` / `bootstrap` 不是协议字段。
- `RunnerResult`PROTOCOL_V1 §7。
- `RunnerAPIProxy`PROTOCOL_V1 §8,是 runner 访问 host 能力的唯一入口,所有请求带 `run_id`
@@ -1,12 +1,12 @@
# 官方 AgentRunner 插件迁移计划
# 官方 Runner 插件迁移计划
本文档描述内置 `RequestRunner` 迁出 LangBot 后,官方 runner 插件如何组织、迁移和验收。它是 [HOST_SDK_INFRASTRUCTURE.md](./HOST_SDK_INFRASTRUCTURE.md) 和 [AGENT_CONTEXT_PROTOCOL.md](./AGENT_CONTEXT_PROTOCOL.md) 的下游落地计划,不是 LangBot 宿主协议的设计前提。QA 入口和 smoke 记录见 [AGENT_RUNNER_QA_GUIDE.md](./AGENT_RUNNER_QA_GUIDE.md)。
本文档描述内置 `RequestRunner` 迁出 LangBot 后,官方 runner 插件如何组织、迁移和验收。它是 [HOST_SDK_INFRASTRUCTURE.md](./HOST_SDK_INFRASTRUCTURE.md) 和 [AGENT_CONTEXT_PROTOCOL.md](./AGENT_CONTEXT_PROTOCOL.md) 的下游落地计划,不是 LangBot 宿主协议的设计前提。QA 入口和 smoke 记录见 [RUNNER_QA_GUIDE.md](./RUNNER_QA_GUIDE.md)。
官方 `local-agent` 可以外移,也可以重写。设计重点不是保留旧内置 runner 的内部结构,而是验证一个依附 LangBot host 基础设施的官方 agent 能否完整工作。同时,LangBot host 协议必须服务 Claude Code SDK、Codex、Pi Agent SDK、外部 Agent 平台等自管 context/runtime 的 runner,不能被官方插件的实现细节绑死。
## 1. 仓库组织
官方 runner 插件与 LangBot 主仓库、SDK 仓库以不同节奏迭代:LangBot 主仓库只维护宿主协议和调度,SDK 仓库维护 AgentRunner 组件和 runtime 协议,官方 runner 插件承载业务 runner 的具体实现和第三方平台适配。
官方 runner 插件与 LangBot 主仓库、SDK 仓库以不同节奏迭代:LangBot 主仓库只维护宿主协议和调度,SDK 仓库维护 Runner 组件和 runtime 协议,官方 runner 插件承载业务 runner 的具体实现和第三方平台适配。
当前推荐"官方插件可独立发布,必要时共享 SDK helper"。开发期采用本地多目录布局:
@@ -14,7 +14,7 @@
langbot-app/
langbot-local-agent/ # plugin:langbot-team/LocalAgent/default
manifest.yaml
components/agent_runner/default.{yaml,py}
components/runner/default.{yaml,py}
langbot-agent-runner/ # 外部服务 runner 仓库
acp-agent-runner/ claude-code-agent/ codex-agent/ dify-agent/ n8n-agent/ ...
```
@@ -29,7 +29,7 @@ langbot-app/
| `dify-service-api` | `langbot-team/DifyAgent` | `plugin:langbot-team/DifyAgent/default` |
| `n8n-service-api` | `langbot-team/N8nAgent` | `plugin:langbot-team/N8nAgent/default` |
| `coze-api` | `langbot-team/CozeAgent` | `plugin:langbot-team/CozeAgent/default` |
| - | `langbot-team/ACPAgentRunner` | `plugin:langbot-team/ACPAgentRunner/default` |
| - | `langbot-team/ACPRunner` | `plugin:langbot-team/ACPRunner/default` |
| - | `langbot-team/ClaudeCodeAgent` | `plugin:langbot-team/ClaudeCodeAgent/default` |
| - | `langbot-team/CodexAgent` | `plugin:langbot-team/CodexAgent/default` |
| `dashscope-app-api` | `langbot-team/DashScopeAgent` | `plugin:langbot-team/DashScopeAgent/default` |
@@ -48,23 +48,23 @@ langbot-app/
## 4. 每个官方插件的组件要求
每个插件至少包含一个 `AgentRunner` 组件,manifest 示例:
每个插件至少包含一个 `Runner` 组件,manifest 示例:
```yaml
apiVersion: langbot/v1
kind: AgentRunner
kind: Runner
metadata:
name: default
label: { en_US: Dify Agent, zh_Hans: Dify Agent }
description:
en_US: Run a Dify application as a LangBot AgentRunner.
zh_Hans: 将 Dify 应用作为 LangBot AgentRunner 运行。
en_US: Run a Dify application as a LangBot Runner.
zh_Hans: 将 Dify 应用作为 LangBot Runner 运行。
spec:
config: []
capabilities: # 字段语义见 PROTOCOL_V1 §4.3
streaming: true
execution:
python: { path: ./main.py, attr: DefaultAgentRunner }
python: { path: ./main.py, attr: DefaultRunner }
```
## 5. local-agent 插件方向
@@ -76,18 +76,18 @@ execution:
责任边界与 Host API 消费方式见 AGENT_CONTEXT_PROTOCOL §8。关键约束:
- 从 `ctx.config` 读取静态绑定 `prompt`**不**读取 `ctx.adapter.extra["prompt"]`;不消费 Query entry adapter 生成的历史窗口。
- 通过 `AgentRunAPIProxy.history` 拉取 transcript,而不是依赖 host 每轮强塞历史窗口。
- 通过 `RunnerAPIProxy.history` 拉取 transcript,而不是依赖 host 每轮强塞历史窗口。
- `ctx.input.contents` 保留图片/文件等多模态内容;RAG 只替换/插入文本部分,不丢图片/文件。
- 不能绕过 `ctx.resources` 调用未授权模型、工具或知识库。
- manifest 声明功能能力、LangBot 资源 permissions 和配置表单;实际授权来自 manifest permissions 与 binding resource policy、runner config、`ctx.context.available_apis` 和 Host run session snapshot 的交集。
### 5.1 Native Execution / Skills 后续接入
本阶段不把 sandbox/skills 做成 AgentRunner 协议字段。后续 sandbox/skills 分支合并后,命令执行、文件操作、skill、MCP managed process 应先由 Host / sandbox 封装成 scoped tools,再通过 `ctx.resources.tools` 和 SDK runtime 转发暴露给 runner。这让 local-agent 只消费授权后的 Host 基础设施,而不是直接持有宿主机执行能力。
本阶段不把 sandbox/skills 做成 Runner 协议字段。后续 sandbox/skills 分支合并后,命令执行、文件操作、skill、MCP managed process 应先由 Host / sandbox 封装成 scoped tools,再通过 `ctx.resources.tools` 和 SDK runtime 转发暴露给 runner。这让 local-agent 只消费授权后的 Host 基础设施,而不是直接持有宿主机执行能力。
## 6. 外部 runner 插件要求
外部平台 runner 迁移遵循:旧配置字段尽量保持同名便于 migration 复制;输出统一转换为 `AgentRunResult`;外部 API timeout 从 runner config 读取;平台 conversation id 存 plugin storage 或 context runtime state,不依赖 LangBot 内置 conversation uuid 私有结构;流式按平台能力声明,没有流式就只发 `message.completed`
外部平台 runner 迁移遵循:旧配置字段尽量保持同名便于 migration 复制;输出统一转换为 `RunnerResult`;外部 API timeout 从 runner config 读取;平台 conversation id 存 plugin storage 或 context runtime state,不依赖 LangBot 内置 conversation uuid 私有结构;流式按平台能力声明,没有流式就只发 `message.completed`
### 6.1 Code-agent harness runner
@@ -95,7 +95,7 @@ Claude Code、Codex、Kimi Code 这类 runner 不一定通过 LangBot 的模型/
本文件只补充官方 runner 的实现要求:输入来自 `ctx.event` / `ctx.input`,不依赖 Pipeline 私有 `Query`;外部 session id / workspace / checkpoint 写入 Host state 或 plugin storage;插件实例边界见 PROTOCOL_V1 §13CLI / subprocess runner 必须处理 timeout、取消、空输出、非零退出和 stderr 映射。
实现结构应把 provider-native output 解析与 LangBot result stream 组装分开:Claude stream-json、Codex JSONL、Kimi / OpenCode 事件等只在 runner adapter 内解析,输出统一归一为 `AgentRunResult``message.completed` / `message.delta``state.updated``run.completed` / `run.failed`)。文件和工具大结果留在当前 run 的 sandbox/workspace,通过消息 metadata、attachment ref 或 path 指向。未知 native event 不应导致 run 崩溃;应记录诊断 metadata 或 warning。新增 harness 时优先补 native fixture -> `AgentRunResult` 的转换测试,再接 WebUI smoke。
实现结构应把 provider-native output 解析与 LangBot result stream 组装分开:Claude stream-json、Codex JSONL、Kimi / OpenCode 事件等只在 runner adapter 内解析,输出统一归一为 `RunnerResult``message.completed` / `message.delta``state.updated``run.completed` / `run.failed`)。文件和工具大结果留在当前 run 的 sandbox/workspace,通过消息 metadata、attachment ref 或 path 指向。未知 native event 不应导致 run 崩溃;应记录诊断 metadata 或 warning。新增 harness 时优先补 native fixture -> `RunnerResult` 的转换测试,再接 WebUI smoke。
并发约束应按外部 session 粒度表达,而不是按 Agent / runner id / 插件实例表达;Agent 复用和全局锁边界见 PROTOCOL_V1 §13。若 runner 使用 `external.session_id` / `thread_id` resume 到同一 native session,且该 harness 不支持并发 turnrunner 应按稳定 external session key 串行写入;一次性 subprocess runner 可以只在单次 `run(ctx)` 内处理,长连接/daemon runner 则应采用 reader 独占 native stream、turn writer 串行写入的结构。
@@ -111,11 +111,11 @@ Claude Code、Codex、Kimi Code 这类 runner 不一定通过 LangBot 的模型/
## 7. Code-agent harness runner 当前形态
外部 code-agent harness 由直接 runner 插件承接,例如 `acp-agent-runner``claude-code-agent``codex-agent`,每个 runner 负责把目标 harness 的 native session、workspace、MCP bridge 和输出事件转换为统一 `AgentRunResult`。本地 smoke 验收入口与记录见 [AGENT_RUNNER_QA_GUIDE.md](./AGENT_RUNNER_QA_GUIDE.md)。
外部 code-agent harness 由直接 runner 插件承接,例如 `acp-agent-runner``claude-code-agent``codex-agent`,每个 runner 负责把目标 harness 的 native session、workspace、MCP bridge 和输出事件转换为统一 `RunnerResult`。本地 smoke 验收入口与记录见 [RUNNER_QA_GUIDE.md](./RUNNER_QA_GUIDE.md)。
当前形态:
- Runner ID 示例:`plugin:langbot-team/ACPAgentRunner/default``plugin:langbot-team/ClaudeCodeAgent/default``plugin:langbot-team/CodexAgent/default`
- Runner ID 示例:`plugin:langbot-team/ACPRunner/default``plugin:langbot-team/ClaudeCodeAgent/default``plugin:langbot-team/CodexAgent/default`
- Runner 可通过 ACP、远端 daemon、本机 subprocess 或外部 HTTP API 调用 harnessharness 的安装、登录态、workspace 和 provider-native 权限由该运行环境负责。
- Runner 会把当前 LangBot `run_id`、可访问资源摘要和 gateway 使用规则注入本次消息;harness 通过 gateway 回填 `run_id` 后访问 LangBot 资产。
- 外部 session id / workspace / checkpoint 写回 Host state 或 plugin storage,后续轮次可复用目标 harness 会话。
@@ -126,11 +126,11 @@ Claude Code、Codex、Kimi Code 这类 runner 不一定通过 LangBot 的模型/
## 8. 发布和安装策略
最终 LangBot 安装/升级时需保证官方 runner 插件可用,可选方案:首次启动检测缺失并提示安装,或由用户从 marketplace 安装。当前分支未发布,因此不保留历史 Pipeline Agent 配置兼容、旧内置 runner fallback,也不把旧 Pipeline 内的 Agent 配置迁移成独立 Agent。4.x 只读取 `ai.runner.id``ai.runner_config[id]`;升级后由用户选择或安装需要的 AgentRunner。
最终 LangBot 安装/升级时需保证官方 runner 插件可用,可选方案:首次启动检测缺失并提示安装,或由用户从 marketplace 安装。当前分支未发布,因此不保留历史 Pipeline Agent 配置兼容、旧内置 runner fallback,也不把旧 Pipeline 内的 Agent 配置迁移成独立 Agent。4.x 只读取 `ai.runner.id``ai.runner_config[id]`;升级后由用户选择或安装需要的 Runner。
## 9. 验收标准
- 每个目标 runner 都有对应官方 AgentRunner 插件和稳定 runner id;当前配置只使用 `ai.runner.id` + `ai.runner_config[id]`
- 每个目标 runner 都有对应官方 Runner 插件和稳定 runner id;当前配置只使用 `ai.runner.id` + `ai.runner_config[id]`
- LangBot 主聊天路径不再通过 `RequestRunner` 执行业务 runner。
- 官方插件测试覆盖非流式、流式、错误、timeout、配置缺失。
- `local-agent` 能完成模型 fallback、tool calling、知识库检索、多模态输入、静态绑定 prompt 消费、history API 拉取、rerank。
@@ -0,0 +1,75 @@
# Agent 工具权限
Agent 配置页展示同一次运行中可能投射给 Runner 的完整工具目录:
- 事件级工具由 Agent 选择的事件范围自动启用。
- `allowed_platform_tools` 管理需要 Agent 自行指定目标的平台级动作。
- `allowed_tools` 管理沙盒内置工具、MCP 工具、插件工具和技能工具。
Host 会按当前 Workspace 实时解析工具来源。未安装的插件、未连接的 MCP、不可用的 Box
沙盒以及名称存在歧义的工具不会进入可选目录。旧 Agent 若尚未保存 `allowed_tools`,继续
沿用运行器原有的工具策略;一旦在配置页保存,就转为明确的顶层白名单。
Agent 不直接持有平台适配器,也不能调用任意原始平台接口。每次运行时,Host 根据当前
事件自动加入兼容的事件级工具,并加入 `allowed_platform_tools` 中选择的平台级工具,
再与 Runner 权限、当前适配器声明的 API、当前事件能够安全绑定的目标取交集,
得到 `ctx.resources.tools``tool_type=platform` 的最终工具集合。
## 两类工具
- 事件级工具以 `event_` 开头。用户、群组、消息或请求标识由 Host 从当前事件冻结,
Agent 只能填写回复文本、审核结果、禁言时长等动作参数。
- 平台级工具以 `platform_` 开头。Agent 可以填写目标用户、群组或消息标识,因此权限
更宽,配置页将其与事件级工具分开展示。
当前事件级工具包括:回复当前会话、删除当前消息、查询事件发起者或相关群组/成员、
禁言/解除禁言/移出相关成员、同意或拒绝好友请求、同意或拒绝入群邀请。
当前平台级工具包括:发送/查询/删除消息,查询群组、群列表、群成员,修改群名称,
禁言/解除禁言/移出成员、退出群组,以及查询用户和好友列表。
`call_platform_api` 不在 Agent 工具目录中。平台私有透传接口必须先在 Host 中定义为
具有固定名称、JSON Schema、风险级别和授权规则的语义工具,不能让 Agent 自行传入
原始 action 名称。
## 运行时投射
```text
current event type ── compatible event tools
Agent.allowed_platform_tools ── selected platform tools
├─ Runner capability tool_calling is enabled
├─ Runner manifest permissions.tools contains call
├─ current adapter.get_supported_apis()
└─ current event type and frozen target are compatible
ctx.resources.tools[tool_type=platform]
├─ Local Agent: RunnerAPIProxy.call_tool
└─ External Runner: langbot_list_assets / langbot_get_tool_detail /
langbot_call_tool (MCP Asset Gateway)
Host revalidates run_id, runner plugin identity, operation and frozen source
current bot adapter semantic API
```
本地和外部 Runner 因此使用同一个工具名、参数 Schema 和 Host 授权快照。外部
平台不会获得适配器对象或长期凭据;MCP 网关中的 run token 和 Host 中的 run session
都只对应当前运行。
## 失败语义
- Agent 未选择当前事件:不会触发运行,也不会生成事件级工具。
- 平台级工具未在 Agent 配置中选择:不进入运行资源。
- Runner 未启用 `tool_calling` 或没有 `tools.call` 权限:所有平台动作均不可用。
- 当前适配器不声明对应 API:该工具记入 `platform_capabilities.unavailable_tools`,不投射。
- 事件类型不匹配或缺少可冻结目标:事件级工具不投射。
- 调用期间机器人下线或适配器能力变化:Host 拒绝执行并返回具体错误。
- 参数包含 Schema 之外的字段:Host 拒绝执行。
这些规则保证配置白名单不是唯一防线;真正的执行授权始终由单次运行快照和执行时检查
共同决定。
+42 -42
View File
@@ -1,8 +1,8 @@
# LangBot AgentRunner Protocol v1
# LangBot Runner Protocol v1
本文档是 LangBot Host 与插件 SDK / Runtime / AgentRunner 之间协议合同的**唯一规范来源(single source of truth**。
本文档是 LangBot Host 与插件 SDK / Runtime / Runner 之间协议合同的**唯一规范来源(single source of truth**。
- 本文件描述当前 Protocol v1 稳定合同,不混入验收流水。当前实现状态见 [STATUS.md](./STATUS.md),测试执行入口见 [AGENT_RUNNER_QA_GUIDE.md](./AGENT_RUNNER_QA_GUIDE.md),安全发布门槛见 [SECURITY_HARDENING.md](./SECURITY_HARDENING.md)。
- 本文件描述当前 Protocol v1 稳定合同,不混入验收流水。当前实现状态见 [STATUS.md](./STATUS.md),测试执行入口见 [RUNNER_QA_GUIDE.md](./RUNNER_QA_GUIDE.md),安全发布门槛见 [SECURITY_HARDENING.md](./SECURITY_HARDENING.md)。
- 本文件之外的任何文档**不得重新定义这里的数据结构**,只能引用,例如"见 PROTOCOL_V1 §4.2"。
- Host 内部模型(`AgentEventEnvelope``AgentBinding`、Descriptor、各 Store)不属于 SDK 协议,定义在 [HOST_SDK_INFRASTRUCTURE.md](./HOST_SDK_INFRASTRUCTURE.md)。
@@ -10,15 +10,15 @@
Protocol v1 只解决四件事:
- LangBot 如何发现插件提供的 AgentRunner。
- LangBot 如何把一次事件调用封装成 `AgentRunContext`
- AgentRunner 如何以事件流形式返回运行结果。
- AgentRunner 如何通过受限 API 访问 LangBot host 能力。
- LangBot 如何发现插件提供的 Runner。
- LangBot 如何把一次事件调用封装成 `RunnerContext`
- Runner 如何以事件流形式返回运行结果。
- Runner 如何通过受限 API 访问 LangBot host 能力。
Protocol v1 **不定义**
- LangBot 内部如何持久化 `AgentBinding`(见 HOST_SDK)。
- AgentRunner 内部如何组装 prompt、压缩历史、管理 memory(见 [AGENT_CONTEXT_PROTOCOL.md](./AGENT_CONTEXT_PROTOCOL.md))。
- Runner 内部如何组装 prompt、压缩历史、管理 memory(见 [AGENT_CONTEXT_PROTOCOL.md](./AGENT_CONTEXT_PROTOCOL.md))。
- 官方 runner 的具体实现(见 [OFFICIAL_RUNNER_PLUGINS.md](./OFFICIAL_RUNNER_PLUGINS.md))。
- Pipeline 的长期配置模型。
- 发布级安全 hardening 的完整实现(见 [SECURITY_HARDENING.md](./SECURITY_HARDENING.md))。
@@ -29,8 +29,8 @@ Protocol v1 **不定义**
| --- | --- |
| LangBot Host | 事件入口、绑定解析、权限、资源、存储、生命周期、结果投递。 |
| Plugin Runtime | 加载插件,响应 Host 的 runner discovery 和 run 调用。 |
| AgentRunner | 插件提供的 agent 执行组件。 |
| AgentRunAPIProxy | AgentRunner 访问 Host 能力的受限 API。 |
| Runner | 插件提供的 agent 执行组件。 |
| RunnerAPIProxy | Runner 访问 Host 能力的受限 API。 |
| AgentBinding | Host 内部的事件到 runner 绑定配置,不直接暴露给 SDK(见 HOST_SDK §4.2)。 |
产品层同时保留 Pipeline 与独立 `Agent`:现有 Pipeline 不迁移为 Agent
@@ -39,11 +39,11 @@ Protocol v1 **不定义**
`ctx.config``ctx.resources``ctx.context``ctx.delivery`。SDK 不需要知道
Agent / binding 的持久化形态。
外部 harness runnerClaude Code、Codex、Kimi Code 等)也是 `AgentRunner`:它们消费 event-first `AgentRunContext`、返回 `AgentRunResult`,并通过 Host 授权的 state/storage API 保存跨轮次指针;当前运行文件和工具大结果进入 sandbox/workspace。它们内部可以继续使用自己的 session、tool loop、MCP、上下文压缩和权限模型。
外部 harness runnerClaude Code、Codex、Kimi Code 等)也是 `Runner`:它们消费 event-first `RunnerContext`、返回 `RunnerResult`,并通过 Host 授权的 state/storage API 保存跨轮次指针;当前运行文件和工具大结果进入 sandbox/workspace。它们内部可以继续使用自己的 session、tool loop、MCP、上下文压缩和权限模型。
## 3. 协议演进
当前 AgentRunner 合同不暴露显式 `protocol_version` 字段。协议演进先按字段级兼容规则处理:
当前 Runner 合同不暴露显式 `protocol_version` 字段。协议演进先按字段级兼容规则处理:
- 新增可选字段保持向后兼容。
- 删除字段或改变既有字段语义,需要在 SDK 发布前完成;发布后应走新的显式兼容方案。
@@ -52,35 +52,35 @@ Agent / binding 的持久化形态。
## 4. Discovery 协议
### 4.1 LIST_AGENT_RUNNERS
### 4.1 LIST_RUNNERS
Host 调用 Plugin Runtime 获取当前插件暴露的 runner 列表,请求无额外 payload。返回:
```python
class ListAgentRunnersResponse(BaseModel):
runners: list[AgentRunnerDiscovery]
class ListRunnersResponse(BaseModel):
runners: list[RunnerDiscovery]
class AgentRunnerDiscovery(BaseModel):
class RunnerDiscovery(BaseModel):
plugin_author: str
plugin_name: str
runner_name: str
manifest: AgentRunnerManifest
manifest: RunnerManifest
```
`manifest` 是 SDK typed `AgentRunnerManifest`,由 Runtime 从插件组件 manifest 解析并校验后返回。`plugin_author` / `plugin_name` / `runner_name` 保留为 transport 寻址字段;Host 以它们生成稳定 runner id,并把 `manifest.id` 校验为 `plugin:author/name/runner`。单个 runner manifest 解析失败时 Runtime/Host 记录 warning 并跳过该 runner,不影响同一插件或其它插件的 runner discovery。
`manifest` 是 SDK typed `RunnerManifest`,由 Runtime 从插件组件 manifest 解析并校验后返回。`plugin_author` / `plugin_name` / `runner_name` 保留为 transport 寻址字段;Host 以它们生成稳定 runner id,并把 `manifest.id` 校验为 `plugin:author/name/runner`。单个 runner manifest 解析失败时 Runtime/Host 记录 warning 并跳过该 runner,不影响同一插件或其它插件的 runner discovery。
### 4.2 AgentRunnerManifest
### 4.2 RunnerManifest
这里的 manifest 指 Runtime 返回给 Host 的 typed runner manifest
```python
class AgentRunnerManifest(BaseModel):
class RunnerManifest(BaseModel):
id: str
name: str
label: I18nObject
description: I18nObject | None = None
capabilities: AgentRunnerCapabilities = AgentRunnerCapabilities()
permissions: AgentRunnerPermissions = AgentRunnerPermissions()
capabilities: RunnerCapabilities = RunnerCapabilities()
permissions: RunnerPermissions = RunnerPermissions()
config_schema: list[DynamicFormItemSchema] = []
metadata: dict[str, Any] = {}
```
@@ -95,7 +95,7 @@ class AgentRunnerManifest(BaseModel):
### 4.3 Capabilities
```python
class AgentRunnerCapabilities(BaseModel):
class RunnerCapabilities(BaseModel):
streaming: bool = False
tool_calling: bool = False
knowledge_retrieval: bool = False
@@ -122,7 +122,7 @@ Capabilities 字段全部是 `bool`,未知 key 禁止进入 typed manifest。
### 4.4 Permissions 与 Effective Access
```python
class AgentRunnerPermissions(BaseModel):
class RunnerPermissions(BaseModel):
models: list[Literal["invoke", "stream", "rerank"]] = []
tools: list[Literal["detail", "call"]] = []
knowledge_bases: list[Literal["list", "retrieve"]] = []
@@ -138,7 +138,7 @@ class AgentRunnerPermissions(BaseModel):
通用交互投递是当前唯一允许执行的 `action.requested` 白名单动作。Runner 必须同时声明
`capabilities.interactions=true``permissions.interactions=["request"]`Host 还必须将其与
当前 binding delivery policy、run authorization snapshot 和 adapter delivery capability 求交。
其它平台动作仍不属于当前 permissionsHost 收到后只记录 telemetry,不得执行
其它 `action.requested` 动作仍只记录 telemetry,不得作为任意平台动作执行器。平台语义动作已经通过 `ctx.resources.tools``tool_type="platform"` 的工具提供:Runner 需具备 `tool_calling` capability 和 `permissions.tools``call` 操作,Host 再与 Agent 工具策略、适配器能力和当前事件目标求交。`event_*` 的目标由 Host 冻结,`platform_*` 需显式选择;调用仍走统一 `call_tool`。这不新增 manifest permission 字段,规则见 [PLATFORM_ACTION_TOOLS.md](./PLATFORM_ACTION_TOOLS.md)
Runner 实际可用 LangBot 资源来自 Host 在 run 前冻结的授权快照:
@@ -151,7 +151,7 @@ effective_access = manifest.permissions ∩ binding.resource_policy ∩ current
1. `AgentResourceBuilder` 先用 manifest permissions 与 binding resource policy / runner config 求交,生成 `ctx.resources`
2. `AgentContextBuilder` 用 manifest permissions 与 binding state/storage policy 求交,生成 `ctx.context.available_apis`
3. `AgentRunSessionRegistry` 冻结 run-scoped resources 与 available APIs。
4. Runtime handler / `AgentRunAPIProxy` 按 active `run_id`、runner identity、caller plugin identity、resource id、scope、payload size、rate limit 和 deadline 校验每次调用。
4. Runtime handler / `RunnerAPIProxy` 按 active `run_id`、runner identity、caller plugin identity、resource id、scope、payload size、rate limit 和 deadline 校验每次调用。
反承诺:manifest permissions **只约束 LangBot 持有的资源访问**。它不承诺限制外部 harness 的 native shell、文件系统、CLI、MCP、网络或本机权限;这些能力由 operator/runtime/sandbox 另行约束,见 HOST_SDK §4.8 与 SECURITY_HARDENING。
@@ -167,7 +167,7 @@ context 边界的设计理由见 [AGENT_CONTEXT_PROTOCOL.md](./AGENT_CONTEXT_PRO
## 5. Run 协议
### 5.1 RUN_AGENT
### 5.1 RUN_RUNNER
Host 调用 Runtime
@@ -175,17 +175,17 @@ Host 调用 Runtime
class AgentRunRequest(BaseModel):
runner_id: str
runner_name: str
context: AgentRunContext
context: RunnerContext
```
Runtime 返回 `AgentRunResult` 异步流。底层 transport 可继续用 `plugin_author` / `plugin_name` / `runner_name` 定位组件,但协议语义以 `runner_id``context` 为准。
Runtime 返回 `RunnerResult` 异步流。底层 transport 可继续用 `plugin_author` / `plugin_name` / `runner_name` 定位组件,但协议语义以 `runner_id``context` 为准。
### 5.2 AgentRunContext
### 5.2 RunnerContext
这是 SDK 看到的**唯一权威 context 定义**。
```python
class AgentRunContext(BaseModel):
class RunnerContext(BaseModel):
run_id: str
trigger: AgentTrigger
event: AgentEventContext
@@ -361,7 +361,7 @@ class InteractionDeliveryCapabilities(BaseModel):
max_fields: int | None = None
```
Runner 使用 `AgentRunResult.interaction_requested()` 生成
Runner 使用 `RunnerResult.interaction_requested()` 生成
`action.requested(action="interaction.requested")`。Host 只能把请求投递到当前 run 冻结的
delivery targetRunner 不得通过 `target` 改写 bot、conversation 或用户。Host 为请求保存
`interaction_id -> processor/binding/conversation/expiry` 关联;平台 callback 必须先经过签名、
@@ -455,11 +455,11 @@ class AgentResources(BaseModel):
`skills` 是本次 run 中 pipeline-visible 的 skill facts`skill_name``display_name``description`)。**skill 通过统一 tool 形式消费,不是独立资源类别**:发现走 `list_skills` tool(或 `langbot_list_assets` 增加 skills 一类),激活走 `activate`,操作走 native exec/read/write。Host **不**把 skill 索引注入 system prompt,也不做 progressive-disclosure 注入;LLM 通过调用发现工具主动查询 skill 清单。Host **可选**在 ctx 提供预渲染的 `suggested_skill_prompt`(首轮延迟优化,runner 可忽略 / override),但它不是访问前提。`skills` 字段本身仅作为发现工具的数据来源与该可选预渲染的输入。
资源列表是本次 run 的授权结果。History / Event / State / Storage 访问通过 `ctx.context.available_apis` 和 Host 侧 run session 校验控制,不作为可枚举 resource list 暴露。Runner 只能通过 `AgentRunAPIProxy` 访问这些能力。当前事件的文件和工具大结果优先进入授权 sandbox/workspace,由 runner 通过 read/write/exec 类工具按需读取。
资源列表是本次 run 的授权结果。History / Event / State / Storage 访问通过 `ctx.context.available_apis` 和 Host 侧 run session 校验控制,不作为可枚举 resource list 暴露。Runner 只能通过 `RunnerAPIProxy` 访问这些能力。当前事件的文件和工具大结果优先进入授权 sandbox/workspace,由 runner 通过 read/write/exec 类工具按需读取。
## 7. Result Stream
### 7.1 AgentRunResult envelope
### 7.1 RunnerResult envelope
```python
JSONValue = str | int | float | bool | None | list["JSONValue"] | dict[str, "JSONValue"]
@@ -475,9 +475,9 @@ ResultType = Literal[
"run.failed",
]
class AgentRunResult(BaseModel):
class RunnerResult(BaseModel):
run_id: str
type: AgentRunResultType | str
type: RunnerResultType | str
data: dict[str, Any] = {}
usage: LLMTokenUsage | None = None
sequence: int | None = None
@@ -568,7 +568,7 @@ Host 必须校验 `state.updated` 的 scope、key、value 大小和 JSON 可序
{ "type": "action.requested", "data": { "action": "interaction.requested", "payload": { "interaction_id": "form_1", "kind": "choice", "title": "Approve?", "actions": [{"id": "approve", "label": "Approve", "style": "primary"}], "fallback_text": "Reply approve or reject." } } }
```
## 8. AgentRunAPIProxy
## 8. RunnerAPIProxy
所有 proxy action 必须携带 `run_id`。Host 必须校验:active run session 存在、caller plugin identity 匹配、resource 在本次 `ctx.resources` 中授权、scope 不越界、payload size / rate limit / deadline 合法。
@@ -777,7 +777,7 @@ Protocol v1 的安全边界在 Host
- 大 payload 不应塞进 result event;当前 run 的文件和工具大结果应进入授权 sandbox/workspace,由 read/write/exec 类工具按需访问。
- Host 必须记录 run_id、runner_id、action、resource、scope、result。
Host 不负责业务编排:不拼接全量历史、不替 runner 做 prompt assembly、不内置 agent memory / tool loop / 上下文压缩策略。这些由官方或第三方 AgentRunner 插件实现。
Host 不负责业务编排:不拼接全量历史、不替 runner 做 prompt assembly、不内置 agent memory / tool loop / 上下文压缩策略。这些由官方或第三方 Runner 插件实现。
外部 harness runner 的边界统一见 HOST_SDK §4.8。简言之:harness native permission mode、allowed/disallowed tools、shell/MCP 权限只是额外执行约束,不能替代 Host 对 LangBot 资源的授权。
@@ -786,7 +786,7 @@ Host 不负责业务编排:不拼接全量历史、不替 runner 做 prompt as
## 12. Pipeline AI Stage Adapter 边界
Pipeline 与 Agent 是 EBA 中平级的处理器:Pipeline 处理消息事件并执行完整
Stage 链,Agent 处理其声明支持的消息或非消息事件。本协议只约束 AgentRunner
Stage 链,Agent 处理其声明支持的消息或非消息事件。本协议只约束 Runner
调用,因此 Pipeline 仅在 AI Stage 调用 runner 时进入 Query entry adapter
该适配不会把 Pipeline 变成 Agent,也不会创建或更新持久 Agent。adapter 负责:
@@ -804,12 +804,12 @@ Stage 链,Agent 处理其声明支持的消息或非消息事件。本协议
## 13. 已确认约束
- EBA 路由层是 `one event -> one Processor target (Pipeline | Agent)`;同一 bot / channel 可以让不同事件绑定不同类型的处理器。
- 进入 AgentRunner Protocol 后,调用基数是 `one AgentBinding -> one run_id -> one runner`。这既适用于独立 Agent,也适用于 Pipeline AI Stage 的单次 runner 调用。
- 进入 Runner Protocol 后,调用基数是 `one AgentBinding -> one run_id -> one runner`。这既适用于独立 Agent,也适用于 Pipeline AI Stage 的单次 runner 调用。
- 一个 Agent 可以被多个 bot / channel 复用。如果 Agent 分支出现多个匹配 bindingBindingResolver 必须按明确规则选出一个或拒绝配置,不应默认 fan-out。
- observer agent、多 runner fan-out、并行裁决、result 合并等能力需要单独设计 delivery、state、platform action 和 audit 语义,不属于当前 v1 契约。
- `AgentRunnerDescriptor.source` 只允许 `plugin`Host 内置 adapter 不能作为 runner source 绕过插件/runtime/proxy 权限链。
- `RunnerDescriptor.source` 只允许 `plugin`Host 内置 adapter 不能作为 runner source 绕过插件/runtime/proxy 权限链。
- `ctx.resources` 与 proxy action 校验必须来自同一个 run authorization snapshotruntime handler 不应重新执行资源裁剪。
- v1 不要求 Agent、AgentRunner 插件实例或 runner id 全局串行。多个 bot / channel 可复用同一个 Agent;并发隔离依赖 `run_id`、binding、conversation / thread scope 和 Host authorization snapshot。
- v1 不要求 Agent、Runner 插件实例或 runner id 全局串行。多个 bot / channel 可复用同一个 Agent;并发隔离依赖 `run_id`、binding、conversation / thread scope 和 Host authorization snapshot。
- 外部 harness runner 当前是 MVP / dev path,证明协议可接入,不代表发布级安全边界或 Docker 生产可用性完成。
## 14. 开放问题
+31 -30
View File
@@ -1,23 +1,23 @@
# Agent Runner 插件化文档入口
# Runner 插件化文档入口
本文档是 agent-runner 插件化工作的路由页。具体设计拆到独立文档中维护,避免把 LangBot 宿主架构、SDK 协议、上下文管理、EBA 接入边界和官方 runner 迁移混在同一份 README 里。
## 背景与问题
旧 runner 路径主要围绕 Pipeline / Query 和 `pkg/provider/runners` 内置实现展开,扩展外部 agent runtime 时容易把 runner 选择、上下文裁剪、资源授权和消息投递绑在同一条聊天链路里。这个分支要把 LangBot 收敛成 Agent Host:Host 负责事件、绑定、授权、事实源和结果投递;AgentRunner 作为插件或外部 harness 消费统一协议并自主管理 prompt / history / memory。
旧 runner 路径主要围绕 Pipeline / Query 和 `pkg/provider/runners` 内置实现展开,扩展外部 agent runtime 时容易把 runner 选择、上下文裁剪、资源授权和消息投递绑在同一条聊天链路里。这个分支要把 LangBot 收敛成 Agent Host:Host 负责事件、绑定、授权、事实源和结果投递;Runner 作为插件或外部 harness 消费统一协议并自主管理 prompt / history / memory。
## 文档维护原则(单一事实源)
- **协议数据结构(schema)唯一定义在 [PROTOCOL_V1.md](./PROTOCOL_V1.md)。** 其他文档不得重抄 schema,只能引用,例如"见 PROTOCOL_V1 §4.2"。
- 当前实现状态、spec 差距与 runner 验收状态归 [STATUS.md](./STATUS.md);测试执行入口归 [AGENT_RUNNER_QA_GUIDE.md](./AGENT_RUNNER_QA_GUIDE.md),安全发布门槛归 [SECURITY_HARDENING.md](./SECURITY_HARDENING.md)。
- 当前实现状态、spec 差距与 runner 验收状态归 [STATUS.md](./STATUS.md);测试执行入口归 [RUNNER_QA_GUIDE.md](./RUNNER_QA_GUIDE.md),安全发布门槛归 [SECURITY_HARDENING.md](./SECURITY_HARDENING.md)。
- Host 内部模型(`AgentEventEnvelope``AgentBinding`、Descriptor、各 Store)定义在 [HOST_SDK_INFRASTRUCTURE.md](./HOST_SDK_INFRASTRUCTURE.md),不属于 SDK 协议。
- 其余专题文档只讲"为什么/边界/怎么用",避免重复叙述。
## 本分支目标
**本分支目标:AgentRunner 外化 / 插件化基础设施**
**本分支目标:Runner 外化 / 插件化基础设施**
本分支只做 LangBot 作为 Agent Host 的基础能力建设,让现有 Pipeline 与用户新建的独立 `Agent` 都能调用插件化 AgentRunner;不负责把两者做持久化迁移:
本分支只做 LangBot 作为 Agent Host 的基础能力建设,让现有 Pipeline 与用户新建的独立 `Agent` 都能调用插件化 Runner;不负责把两者做持久化迁移:
- LangBot 与 SDK 的稳定协议合同(Protocol v1
- Host-side `AgentEventEnvelope` / `AgentBinding` 模型
@@ -28,17 +28,17 @@
- Sandbox/workspace read/write/exec 文件能力,用于当前 run 的上传文件、工具大结果和临时产物
- SDK runtime forwarding pull APIs + `caller_plugin_identity` 验证路径
## 本分支不实现
## 当前已集成与后续扩展
以下能力由其他分支负责,本分支只保留 integration point。EBA 完整事件网关与事件路由当前由外部 EBA 分支联调
截至 2026-09-05`dev/4.11.x` 已合并 EBA 与 Runner 插件化。下面按当前代码划分实现边界
- **EventGateway / EventRouter**:完整事件网关实现、事件路由、事件持久化管理
- **Event subscription / Event notification**:事件订阅、推送通知
- **BindingResolver persistence UI**:绑定配置的持久化 UI 和 event router 集成(如由其他模块负责)
- **Scheduler / Background event source**定时任务、后台事件源
- **已实现的平台事件路由**`RuntimeBot` 负责事件转换后的 observer 广播、`event_bindings` 匹配和 Pipeline / Agent / discard 单目标分派;EventRouter 是逻辑职责,不是另一个独立服务。
- **已实现的持久化与 UI**:独立 Agent、Bot 事件绑定、处理器工作台、Runner 市场安装、事件范围与工具权限、路由诊断和调试入口。
- **后续的通用事件订阅与通知**:不把当前适配器回调和 Bot 路由理解成通用订阅产品。
- **后续的 Scheduler / Background event source**用户可配置的定时自动化和后台任务入口。
- **完整 Agent Platform / daemon control plane**Host-owned `AgentRun` / `AgentRunEvent`、run control primitives、最小 runtime heartbeat/claim lease 已作为 v2 foundation 落地;业务队列、Platform UI、daemon supervisor、runtime wakeup channel 和分布式 runtime 管控仍不属于 Protocol v1 主线。
EventGateway / EventRouter 在本文档中描述为 **external EBA branch integration point**,由外部 EBA 分支提供并联调。本分支只定义 host-side envelope/binding models 和 `run(event, binding)` orchestrator 入口
平台事件、Agent 配置及 Pipeline AI Stage 已共同使用 host-side envelope/binding models 和 `run(event, binding)`。后续入口应复用这条链路。当前实现与发布验收分别见 [STATUS.md](./STATUS.md),平台动作授权见 [PLATFORM_ACTION_TOOLS.md](./PLATFORM_ACTION_TOOLS.md)
本分支与外部 EBA / Agent Platform / Runtime Control Plane 的扩展边界见 [EXTENSION_SCOPE_MATRIX.md](./EXTENSION_SCOPE_MATRIX.md)。
@@ -48,26 +48,26 @@ EventGateway / EventRouter 在本文档中描述为 **external EBA branch integr
调度基数、Agent 复用、插件实例无状态、Pipeline adapter 和 fan-out 边界的规范来源是 [PROTOCOL_V1.md](./PROTOCOL_V1.md) §13;README 不复写这些约束。
## Pipeline 与 AgentRunner 的关系
## Pipeline 与 Runner 的关系
**Pipeline 与 Agent 是 EBA 中平级的处理器;`QueryEntryAdapter` 只适配 Pipeline 内部的 AgentRunner 调用。**
**Pipeline 与 Agent 是 EBA 中平级的处理器;`QueryEntryAdapter` 只适配 Pipeline 内部的 Runner 调用。**
EBA 先根据 `target_type` 选择 Pipeline 或 Agent。Pipeline 目标执行完整 Stage 链;当 Pipeline 的 AI Stage 调用 runner 时,`run_from_query()``QueryEntryAdapter``Query` 转换为 `AgentEventEnvelope` + `AgentBinding`,再委托到统一的 `run(event, binding, ...)`。Agent 目标则直接从自己的持久配置构造 binding。两条路径可以复用同一套 AgentRunner Host capabilities,但 Pipeline 本身不会被投影或持久化为 Agent。
EBA 先根据 `target_type` 选择 Pipeline 或 Agent。Pipeline 目标执行完整 Stage 链;当 Pipeline 的 AI Stage 调用 runner 时,`run_from_query()``QueryEntryAdapter``Query` 转换为 `AgentEventEnvelope` + `AgentBinding`,再委托到统一的 `run(event, binding, ...)`。Agent 目标则直接从自己的持久配置构造 binding。两条路径可以复用同一套 Runner Host capabilities,但 Pipeline 本身不会被投影或持久化为 Agent。
下一轮测试路径、状态定义和 smoke 记录见 [AGENT_RUNNER_QA_GUIDE.md](./AGENT_RUNNER_QA_GUIDE.md)。
下一轮测试路径、状态定义和 smoke 记录见 [RUNNER_QA_GUIDE.md](./RUNNER_QA_GUIDE.md)。
## 术语表
| 术语 | 含义 |
| --- | --- |
| Protocol v1 | Host 调用 AgentRunner 的 runner 可见合同:discovery、`AgentRunContext`、result stream、Host pull API 和错误模型。 |
| Protocol v1 | Host 调用 Runner 的 runner 可见合同:discovery、`RunnerContext`、result stream、Host pull API 和错误模型。 |
| Processor | EBA 的处理器上位概念;当前平级类型为 Pipeline 与 Agent。 |
| Agent | 目标产品层配置对象,保存 runner id、runner config 和资源/状态/投递策略;不等于插件实例。 |
| AgentConfig | Host 内部的单次 AgentRunner 调用配置投影,可由 Pipeline AI Stage 或持久 Agent 生成;投影本身不会创建 Agent。 |
| AgentConfig | Host 内部的单次 Runner 调用配置投影,可由 Pipeline AI Stage 或持久 Agent 生成;投影本身不会创建 Agent。 |
| AgentBinding / binding | Host 在一次事件运行前解析出的有效绑定,决定调用哪个 runner 以及带什么策略。 |
| envelope | Host 内部事件封装,即 `AgentEventEnvelope`runner 看到的是由它投影出的 `ctx.event`。 |
| descriptor / manifest | runner discovery 的能力和配置描述;manifest 来自插件,descriptor 是 Host 校验后的注册表视图。 |
| EBA | Event Based Agent,把消息、撤回、入群、定时任务等都统一成 host event 的接入方向;完整网关和路由在外部 EBA 分支联调。 |
| EBA | Event Based Agent;平台事件路由已集成,定时任务等通用事件源仍是后续扩展。 |
| harness runner | ACP、Claude Code、Codex 等已有自身 session / tool loop / MCP / 压缩机制的外部 runtime adapter。 |
| projection | Host 把内部事实源、授权资源或配置裁剪成 runner / harness 可消费视图的过程。 |
| Runtime Control Plane | v2 Host 能力层,当前已落地 Host-owned run/result ledger、run control primitives、最小 runtime heartbeat/claim lease;完整 daemon worker 管控、task wakeup 和 Agent Platform 产品形态不是 Protocol v1 主线。 |
@@ -76,17 +76,18 @@ EBA 先根据 `target_type` 选择 Pipeline 或 Agent。Pipeline 目标执行完
| 文档 | 关注点 |
| --- | --- |
| [PROTOCOL_V1.md](./PROTOCOL_V1.md) | **🔒 唯一 schema 事实源**。LangBot Host 与 SDK / Runtime / AgentRunner 的协议合同:版本协商、discovery、run context、result stream、proxy actions、错误和 adapter 边界。 |
| [PROTOCOL_V1.md](./PROTOCOL_V1.md) | **🔒 唯一 schema 事实源**。LangBot Host 与 SDK / Runtime / Runner 的协议合同:版本协商、discovery、run context、result stream、proxy actions、错误和 adapter 边界。 |
| [HOST_SDK_INFRASTRUCTURE.md](./HOST_SDK_INFRASTRUCTURE.md) | LangBot 宿主能力与分层架构、Host 内部模型(`AgentEventEnvelope` / `AgentBinding` / Descriptor / 各 Store)、runner 发现、绑定、资源授权、状态、存储、生命周期和调用链。 |
| [AGENT_CONTEXT_PROTOCOL.md](./AGENT_CONTEXT_PROTOCOL.md) | Agent-owned context 方向:事件到来时 LangBot 传什么,agent 如何按需拉取更多历史 / state、如何访问 sandbox/workspace 文件,以及如何支持 KV cache 友好的上下文管理。 |
| [EXTENSION_SCOPE_MATRIX.md](./EXTENSION_SCOPE_MATRIX.md) | AgentRunner 外化与外部 EBA / Agent Platform / Runtime Control Plane 的扩展边界矩阵,说明哪些是本分支底座、哪些由外部分支接入。 |
| [EVENT_BASED_AGENT.md](./EVENT_BASED_AGENT.md) | EBA 接入边界:事件模型、事件来源、触发绑定、非消息事件如何复用 AgentRunner 调度;完整 EventGateway / EventRouter 由外部 EBA 分支联调。 |
| [eba-productization-release.md](./eba-productization-release.md) | EBA 适配器与 AgentRunner 插件化合并后的产品化 / 发布计划,说明非技术用户快速上手差距、Bot 与处理器边界、未来 Solution 分发标的,以及多 namespace SaaS 支持要求。 |
| [EXTENSION_SCOPE_MATRIX.md](./EXTENSION_SCOPE_MATRIX.md) | Runner 外化与外部 EBA / Agent Platform / Runtime Control Plane 的扩展边界矩阵,说明哪些是本分支底座、哪些由外部分支接入。 |
| [EVENT_BASED_AGENT.md](./EVENT_BASED_AGENT.md) | 已集成的事件路由、处理器分派、平台工具和结构化交互边界。 |
| [eba-productization-release.md](./eba-productization-release.md) | EBA 适配器与 Runner 插件化合并后的产品化 / 发布计划,说明非技术用户快速上手差距、Bot 与处理器边界、未来 Solution 分发标的,以及多 namespace SaaS 支持要求。 |
| [RUNTIME_CONTROL_PLANE_V2.md](./RUNTIME_CONTROL_PLANE_V2.md) | Agent Platform v2 / runtime 管控面决策:`AgentRun` / `AgentRunEvent` / run control 已作为 Host 事实源落地,最小 runtime heartbeat/claim lease 已落地;完整 runtime registry / daemon 管控仍是后续可选阶段。 |
| [OFFICIAL_RUNNER_PLUGINS.md](./OFFICIAL_RUNNER_PLUGINS.md) | 官方 runner 插件迁移,包括 local-agent 和外部 runner。它是下游落地计划,不是 LangBot 基础能力设计的前置约束。 |
| [RUN_STEERING_AND_CHECKPOINT.md](./RUN_STEERING_AND_CHECKPOINT.md) | 运行中消息注入(steering / follow-up)与压缩摘要持久化(compaction checkpoint)的设计与落地状态记录;schema 仍以 PROTOCOL_V1 为准。 |
| [STATUS.md](./STATUS.md) | 当前实现状态、spec 与实现已知差距、runner 验收状态和历史高价值记录。 |
| [AGENT_RUNNER_QA_GUIDE.md](./AGENT_RUNNER_QA_GUIDE.md) | Agent Runner QA 指南:保留最高价值测试路径,指导 agent 开展下一轮 WebUI / runner smoke 验证。 |
| [PLATFORM_ACTION_TOOLS.md](./PLATFORM_ACTION_TOOLS.md) | 当前事件工具、平台动作和普通工具的配置、投射及执行授权。 |
| [RUNNER_QA_GUIDE.md](./RUNNER_QA_GUIDE.md) | Runner QA 指南:保留最高价值测试路径,指导 agent 开展下一轮 WebUI / runner smoke 验证。 |
| [SECURITY_HARDENING.md](./SECURITY_HARDENING.md) | 安全发布级 hardening 的后续发布门槛:路径隔离、权限边界、secret、资源配额、MCP / skill 投影和审计。 |
## 工作拆分
@@ -102,7 +103,7 @@ EBA 先根据 `target_type` 选择 Pipeline 或 Agent。Pipeline 目标执行完
- resource authorization 与 `run_id` 级权限校验
- host-owned state / storage / event log / transcript 能力
- sandbox/workspace 文件 staging 与 read/write/exec 能力
- SDK `AgentRunner``AgentRunContext``AgentRunResult``AgentRunAPIProxy`
- SDK `Runner``RunnerContext``RunnerResult``RunnerAPIProxy`
协议合同详见 [PROTOCOL_V1.md](./PROTOCOL_V1.md)。
@@ -116,13 +117,13 @@ Host 不定义通用历史窗口字段或策略;runner 通过 Host pull API
详见 [AGENT_CONTEXT_PROTOCOL.md](./AGENT_CONTEXT_PROTOCOL.md)。
### 3. Event Based AgentExternal Branch
### 3. Event Based Agent已集成
消息只是事件的一种。外部 EBA 分支中的 `message.received``message.recalled``group.member_joined``friend.request_received` 等事件都应能通过统一事件 envelope 触发 AgentRunner
消息只是事件的一种。当前平台适配器中的 `message.received``message.deleted``group.member_joined``friend.request_received` 等事件通过统一事件 envelope 进入处理器路由,具体支持范围以适配器能力声明为准
EBA dispatch 的基数和 fan-out 边界仍以 PROTOCOL_V1 §13 为准;本文档只列出本分支提供给外部 EBA 分支复用的入口点。
EBA dispatch 的基数和 fan-out 边界仍以 PROTOCOL_V1 §13 为准;新增事件源复用以下入口点。
**本分支不实现 EBA 完整能力,只提供**
**当前共同使用的执行底座**
- event-first envelope (`AgentEventEnvelope`)
- AgentBinding model
- `run(event, binding)` 入口
@@ -140,7 +141,7 @@ EBA dispatch 的基数和 fan-out 边界仍以 PROTOCOL_V1 §13 为准;本文
### 5. Runtime Control Plane v2Foundation Partial
当前 AgentRunner v1 主线仍以 `event -> binding -> runner.run(ctx) -> result stream` 为 runner 可见合同。Host 侧已经新增持久 `AgentRun` / `AgentRunEvent`、result persistence、cancel/finalize/query 等通用 run control primitives,并提供受权限保护的最小 runtime register/heartbeat/list、claim/renew/release 和 reconcile 原语。
当前 Runner v1 主线仍以 `event -> binding -> runner.run(ctx) -> result stream` 为 runner 可见合同。Host 侧已经新增持久 `AgentRun` / `AgentRunEvent`、result persistence、cancel/finalize/query 等通用 run control primitives,并提供受权限保护的最小 runtime register/heartbeat/list、claim/renew/release 和 reconcile 原语。
在这些 Host 能力之上,可以构建独立 agent 管控面插件;插件负责 UI、策略和编排体验,runtime/task 的事实源仍由 Host 持有。完整 daemon supervisor、任务唤醒/长轮询/WebSocket、跨 Host 分布式锁、provider 登录态诊断和产品化业务队列仍是后续工作。
@@ -1,10 +1,10 @@
# Agent Platform / Runtime Control Plane Decision Note
本文档记录 AgentRunner 插件化之后,LangBot 如何继续演进成 Agent Platform 基础设施层。这里讨论的是 Host capability layer,不是 `AgentRunner Protocol v2`,也不是把某个具体 Agent Platform 产品写进 LangBot core。
本文档记录 Runner 插件化之后,LangBot 如何继续演进成 Agent Platform 基础设施层。这里讨论的是 Host capability layer,不是 `Runner Protocol v2`,也不是把某个具体 Agent Platform 产品写进 LangBot core。
> 本文是当前决策版。协议数据结构仍以 [PROTOCOL_V1.md](./PROTOCOL_V1.md) 为准;测试执行入口见 [AGENT_RUNNER_QA_GUIDE.md](./AGENT_RUNNER_QA_GUIDE.md);扩展边界见 [EXTENSION_SCOPE_MATRIX.md](./EXTENSION_SCOPE_MATRIX.md)。
> 本文是当前决策版。协议数据结构仍以 [PROTOCOL_V1.md](./PROTOCOL_V1.md) 为准;测试执行入口见 [RUNNER_QA_GUIDE.md](./RUNNER_QA_GUIDE.md);扩展边界见 [EXTENSION_SCOPE_MATRIX.md](./EXTENSION_SCOPE_MATRIX.md)。
>
> 实现状态说明:本文描述的是 Runtime Control Plane v2 的目标能力和分阶段落地建议。当前 AgentRunner 插件化主线已经具备 event-first context、run-scoped authorization、EventLog / Transcript / State / sandbox 文件等 Host capability,并已落地持久 `AgentRun` / `AgentRunEvent` ledger、run control actions、最小 runtime heartbeat/claim lease 和 admin reconcile 原语。完整 Agent Platform 产品形态、daemon supervisor、runtime wakeup channel 和分布式 runtime 管控仍未完成。当前实现状态以 [STATUS.md](./STATUS.md) 为准。
> 实现状态说明:本文描述的是 Runtime Control Plane v2 的目标能力和分阶段落地建议。当前 Runner 插件化主线已经具备 event-first context、run-scoped authorization、EventLog / Transcript / State / sandbox 文件等 Host capability,并已落地持久 `AgentRun` / `AgentRunEvent` ledger、run control actions、最小 runtime heartbeat/claim lease 和 admin reconcile 原语。完整 Agent Platform 产品形态、daemon supervisor、runtime wakeup channel 和分布式 runtime 管控仍未完成。当前实现状态以 [STATUS.md](./STATUS.md) 为准。
## 1. 当前决策
@@ -14,7 +14,7 @@ LangBot 后续定位应更像 **Agent Host / infrastructure provider / transfer
- **Agent Platform 产品形态做成插件**。插件负责 agent 管理、策略、业务队列、UI、编排、多 agent 协作和产品体验。
- **Agent Platform 所需的基础事实源做进 Host**。当前 Host 已保存 event、state、transcript、sandbox 文件边界、active run 权限快照、持久 run/result ledger、审计关联和通用控制状态。
- **最小 runtime registry / heartbeat / claim lease 已作为 Host 原语落地,但不等于完整 daemon worker 管控**。远程 harness / daemon 的进程托管、wakeup channel、provider 登录态诊断和分布式调度仍可以先由 AgentRunner 插件和 SDK remote layer 自己维护。
- **最小 runtime registry / heartbeat / claim lease 已作为 Host 原语落地,但不等于完整 daemon worker 管控**。远程 harness / daemon 的进程托管、wakeup channel、provider 登录态诊断和分布式调度仍可以先由 Runner 插件和 SDK remote layer 自己维护。
- **不把业务调度写进 Host**。Host 提供通用 run/result/control primitivesPlatform 插件决定哪些事件触发哪些 agent、如何排队、如何分配、是否 fan-out。
推荐分层:
@@ -23,22 +23,23 @@ LangBot 后续定位应更像 **Agent Host / infrastructure provider / transfer
LangBot Host
Current base: EventLog / runtime AgentBinding / State / Transcript / sandbox files / active run authorization
Current v2 foundation: Run / RunEvent / audit / result persistence / control primitives / minimal runtime heartbeat and claim lease
Planned: Agent / Binding persistence / daemon supervisor / wakeup channel / distributed runtime operations
Current product: persisted Agent / Bot event_bindings / processor UI / event routing
Planned: external harness daemon supervisor / wakeup channel / distributed runtime operations
Agent Platform plugin
Agent management UI / project-task model / event routing policy
Business queue / multi-agent orchestration / runtime selection policy
AgentRunner plugin / external harness runtime
Runner plugin / external harness runtime
Connects ACP / remote daemon / local subprocess / HTTP API
Executes and converts provider-native events to AgentRunResult
Executes and converts provider-native events to RunnerResult
```
## 2. Platform 与非 Platform 的区别
当前 LangBot 已经具备 Agent Host 的核心特征:
- 抹平不同 AgentRunner。
- 抹平不同 Runner。
- 从 IM / Pipeline 入口触发 runner。
- 有 event-first context 方向。
- 有 Host-owned EventLog / Transcript / State 和 sandbox/workspace 文件边界。
@@ -73,7 +74,9 @@ Host 负责这些能力的通用事实源和安全边界;Platform 插件负责
- `AgentRunEvent` 保存 runner/result/admin event stream,按 `run_id + sequence` 做可回放分页。
- `AgentRuntime` 保存最小 runtime registry / heartbeat 事实,用于 runtime list、stale mark 和 claim lease reconcile。
因此本文后续提到的 `AgentRun` / `AgentRunEvent``run_append_result``run_finalize``run_cancel``runtime_register``runtime_heartbeat``run_claim` 等基础原语已经存在。仍未完成的是独立 platform `run_create` action、Host-owned Agent / Binding 持久模型、业务队列产品形态、daemon supervisor、runtime wakeup channel、跨 Host 分布式锁和 provider/runtime 诊断面。
因此本文后续提到的 `AgentRun` / `AgentRunEvent``run_append_result``run_finalize``run_cancel``runtime_register``runtime_heartbeat``run_claim` 等基础原语已经存在。2026-09-05 核对:独立 Agent 持久模型、Bot `event_bindings` 和处理器 UI 也已实现;`AgentBinding` 仍是单次运行投影,不是独立配置表。仍未完成的是独立 platform `run_create` action、业务队列产品形态、外部 harness daemon supervisor、runtime wakeup channel、跨 Host 分布式锁和 provider/runtime 诊断面。
SDK Plugin Runtime 已有 installation worker 的 supervisor、重启退避和 Runtime 重启协调器;这与这里规划的外部 Agent harness 进程托管不同。当前模型和状态以 [STATUS.md](./STATUS.md) 为准,本文后续阶段是能力拆分,不能作为尚未实现功能的清单。
## 3. 基础概念
@@ -113,7 +116,7 @@ metadata
### 3.3 RunEvent / RunResult
RunEvent 是一次 run 过程中产生的结果事件流,对应 runner 返回的 `AgentRunResult`。它不同于 EBA/EventLog 的输入事件:
RunEvent 是一次 run 过程中产生的结果事件流,对应 runner 返回的 `RunnerResult`。它不同于 EBA/EventLog 的输入事件:
```text
message.delta
@@ -148,8 +151,8 @@ Runtime / daemon 表示执行位置或执行能力,例如某台机器上的 Cl
当前决策:
- Host 不在第一阶段维护完整 runtime registry。
- AgentRunner 插件可以通过 SDK remote layer 与 daemon 保持连接、心跳和执行通道。
- 外部 harness / agent 不应直接访问 LangBot Host 或数据库。访问 LangBot 资源必须通过 daemon / AgentRunner plugin / SDK runtime / `AgentRunAPIProxy` / scoped MCP bridge,并接受 run-scoped authorization 校验。
- Runner 插件可以通过 SDK remote layer 与 daemon 保持连接、心跳和执行通道。
- 外部 harness / agent 不应直接访问 LangBot Host 或数据库。访问 LangBot 资源必须通过 daemon / Runner plugin / SDK runtime / `RunnerAPIProxy` / scoped MCP bridge,并接受 run-scoped authorization 校验。
- 如果后续多个插件都需要共享 runtime 状态,再把薄的 `RuntimeLease` / registry 下沉为 Host 通用能力。
## 4. Host 应新增的最小能力
@@ -219,7 +222,7 @@ metadata_json
- append 必须幂等,支持远程 daemon / plugin 重试。
- 未知 result type 可保存但 Host 只对已知类型执行副作用。
- 大 payload 仍应进入 sandbox/workspace,不直接塞入 result event。
- `usage_json` 保存 `AgentRunResult.usage` 原样结构;缺失表示 unknown,不等于 0。
- `usage_json` 保存 `RunnerResult.usage` 原样结构;缺失表示 unknown,不等于 0。
### 4.3 Run Control API
@@ -255,7 +258,7 @@ event -> binding -> context -> runner invocation -> result normalization
需要补齐:
- run 开始时创建 `AgentRun`
- 每个 `AgentRunResult` 进入 `AgentRunEvent`
- 每个 `RunnerResult` 进入 `AgentRunEvent`
- `run.completed` / 正常 generator 结束时标记 completed。
- `run.failed` / exception / timeout 标记 failed 或 timeout。
- terminal result 携带 usage 时,写入 `AgentRunEvent.usage_json` 并汇总到 `AgentRun.usage_json`
@@ -263,7 +266,7 @@ event -> binding -> context -> runner invocation -> result normalization
### 4.5 Usage / Cost Accounting
SDK 侧 `AgentRunResult` 已提供可选 `usage` 字段,用于把不同 runner / external harness / provider-native event 的 token usage 归一到同一个 run result envelope。
SDK 侧 `RunnerResult` 已提供可选 `usage` 字段,用于把不同 runner / external harness / provider-native event 的 token usage 归一到同一个 run result envelope。
语义:
@@ -306,7 +309,7 @@ RunCreateRequest / RunCreateResult
RunAppendResultRequest
```
这些是 Host control primitives,不替代 `AgentRunContext` / `AgentRunResult`
这些是 Host control primitives,不替代 `RunnerContext` / `RunnerResult`
### 5.2 Proxy Methods
@@ -324,17 +327,17 @@ finalize_run(run_id, status, error=None)
访问边界:
- 普通 AgentRunner 在同步 `run(ctx)` 内不一定需要直接调用这些 APIHost orchestrator 可自动记录。
- 普通 Runner 在同步 `run(ctx)` 内不一定需要直接调用这些 APIHost orchestrator 可自动记录。
- Platform 插件可以创建/查询/取消 run。
- AgentRunner 插件或 daemon bridge 可以 append/finalize 自己负责的 run。
- Runner 插件或 daemon bridge 可以 append/finalize 自己负责的 run。
- 外部 harness 仍不能直接调用 Host;必须经 SDK runtime / proxy / bridge。
### 5.3 Plugin-Daemon Heartbeat
远程 daemon 的初始心跳可以是 SDK / AgentRunner plugin 私有能力:
远程 daemon 的初始心跳可以是 SDK / Runner plugin 私有能力:
```text
daemon <-> AgentRunner plugin / SDK remote layer <-> LangBot plugin runtime <-> Host
daemon <-> Runner plugin / SDK remote layer <-> LangBot plugin runtime <-> Host
```
Host 第一阶段只需要知道:
@@ -362,7 +365,7 @@ Platform 插件不应负责:
- 在 Host Run Ledger 落地后,私有保存通用 run/result 事实源。
- 绕过 Host 直接写 transcript/state 或越权访问 sandbox/workspace 文件。
- 让外部 harness 直接访问 LangBot DB 或 Host 内部资源。
- 把某个业务队列语义强塞进 AgentRunner Protocol v1。
- 把某个业务队列语义强塞进 Runner Protocol v1。
## 7. 与 EBA 的关系
@@ -392,18 +395,18 @@ EventGateway
这两条路径最终应共享 Host run/result/state 事实源和 sandbox/workspace 文件边界。当前阶段可共享的是 event/transcript/state、sandbox 文件和同步执行链路;持久 run/result ledger 需要 Runtime Control Plane v2 Phase 1 补齐。区别在于是否有 Platform 插件参与产品化调度和业务队列。
## 8. 与 AgentRunner Protocol v1 的关系
## 8. 与 Runner Protocol v1 的关系
本设计不改变 v1 的 runner 可见合同:
```text
AgentRunContext -> AgentRunner.run(ctx) -> AgentRunResult stream
RunnerContext -> Runner.run(ctx) -> RunnerResult stream
```
必须保持:
- `AgentRunContext` 不塞入 daemon/worker/pod 细节。
- `AgentRunResult` 仍是 runner 输出的统一事件流。
- `RunnerContext` 不塞入 daemon/worker/pod 细节。
- `RunnerResult` 仍是 runner 输出的统一事件流。
- 普通 runner 不需要知道 task queue / runtime registry。
- 远程 harness 可以自管 session、tool loop、MCP、上下文压缩,但访问 LangBot 资源必须通过 SDK proxy / bridge。
- Runtime-managed execution 是 placement / transport 选择,不是普通 runner 协议的强制概念。
@@ -419,7 +422,7 @@ AgentRunContext -> AgentRunner.run(ctx) -> AgentRunResult stream
- `AgentRun` 表。
- `AgentRunEvent` 表。
- Orchestrator 自动创建/更新 run。
- Journal 持久化每个 `AgentRunResult`
- Journal 持久化每个 `RunnerResult`
- Run 查询和事件分页 API。
- SDK entities + proxy 方法。
@@ -514,9 +517,9 @@ Tests: 40+ 个文件
- Host 不写业务调度策略,但要保存通用状态、结果、权限和审计。
- EBA event 不是 queuequeue 是执行生命周期问题。
- 业务 queue 可以先在 Platform 插件里;执行 queue 只有在复用需求明确后再下沉 Host。
- Daemon registry 不应污染 AgentRunner Protocol v1。
- Daemon registry 不应污染 Runner Protocol v1。
- 外部 harness 不直接访问 LangBot Host 或 DB。
- 所有 LangBot 资源访问必须走 SDK runtime / `AgentRunAPIProxy` / scoped MCP bridge。
- 所有 LangBot 资源访问必须走 SDK runtime / `RunnerAPIProxy` / scoped MCP bridge。
- Docker / remote / local subprocess 只是 runtime placement,不是 runner 协议差异。
## 11. 非目标
@@ -525,8 +528,8 @@ Tests: 40+ 个文件
- 完整 Multica 式 runtime registry。
- Host 内置项目管理、任务板、agent team、workflow 产品逻辑。
- 把 daemon heartbeat / worker liveness 放进 `AgentRunContext`
- 把业务 queue 定义为 AgentRunner Protocol 字段。
- 把 daemon heartbeat / worker liveness 放进 `RunnerContext`
- 把业务 queue 定义为 Runner Protocol 字段。
- 让 Platform 插件私有保存 run/result 事实源。
- 让外部 agent/harness 直连 Host 内部资源。
@@ -16,7 +16,7 @@ local-agent 已移植 Pi 的事件生命周期、并行工具语义、hook 扩
### 1.1 问题
IM 场景下用户在 agent 运行中追加消息非常常见(补充信息、纠正方向、"算了别查了")。
EBA 先按事件选择一个 Pipeline 或 Agent 处理器;进入 AgentRunner 后,当前调用链是 `one AgentBinding -> one run_id -> one runner`
EBA 先按事件选择一个 Pipeline 或 Agent 处理器;进入 Runner 后,当前调用链是 `one AgentBinding -> one run_id -> one runner`
PROTOCOL_V1 §13):同会话的新消息要么等待当前 run 结束后触发新 run,
要么并发触发独立 run。两种行为都无法把新消息送进**正在执行的 tool loop**
用户体验是"agent 自顾自跑完过期任务,然后才看到新消息"。
@@ -59,7 +59,7 @@ pi-agent-core 区分两个队列,注入时机都在 turn 边界,不打断进
已落地的协议面(最终定义归 PROTOCOL_V1):
1. `ContextAccess.available_apis` 增加 steering pull 能力位。
2. `AgentRunAPIProxy` 增加 steering 拉取 action:默认 `mode=all`Host 保序返回全部
2. `RunnerAPIProxy` 增加 steering 拉取 action:默认 `mode=all`Host 保序返回全部
pending 输入;`one-at-a-time` 仅作为 runner 主动节流选项。
3. dispatch 层的"认领"规则:`message.received` 可被同 conversation 的 active run
吸收,原事件写 EventLog / Transcriptdispatch 行为写入 EventLog metadata。
@@ -1,4 +1,4 @@
# Agent Runner Security Boundary
# Runner Security Boundary
本文档记录 agent-runner 插件化后的安全边界和最小护栏。
@@ -6,7 +6,7 @@
**当前结论:不采用高强度监管模型。**
LangBot 的目标不是托管一个强隔离、不可信 code runner 平台。AgentRunner 插件,尤其是 ACP / Claude Code / Codex / OpenCode / Kimi Code 这类外部 harness,默认视为 **operator-owned execution**:用户或部署者显式配置并承担其文件系统、进程、网络、workspace、provider 登录态和 native tool 风险。
LangBot 的目标不是托管一个强隔离、不可信 code runner 平台。Runner 插件,尤其是 ACP / Claude Code / Codex / OpenCode / Kimi Code 这类外部 harness,默认视为 **operator-owned execution**:用户或部署者显式配置并承担其文件系统、进程、网络、workspace、provider 登录态和 native tool 风险。
LangBot 需要负责的是保护 **LangBot 自己持有的资源**,包括模型、知识库、LangBot tools、history、event、state、plugin/workspace storage、sandbox/workspace 文件访问等。只要这些资源访问是 run-scoped、permission-scoped、可校验、可诊断的,当前阶段即可接受。
@@ -171,7 +171,7 @@ LangBot 需要提供基本可控性:
截至 2026-06-15,已有实现覆盖:
- SDK typed AgentRunner manifest、capabilities、permissions。
- SDK typed Runner manifest、capabilities、permissions。
- Host resource builder 按 manifest permissions 和 binding policy 生成 `ctx.resources`
- Active run session snapshot 和 `caller_plugin_identity` 校验。
- History / event / state / tool / knowledge runtime action 的 run-scoped 校验。
@@ -204,7 +204,7 @@ LangBot 需要提供基本可控性:
可以对外说明:
> AgentRunner 插件通过 run-scoped authorization 和 scoped MCP gateway 保护 LangBot 持有资源。外部 code harness 的执行环境由用户或部署平台负责隔离;LangBot 当前不提供 managed sandbox。
> Runner 插件通过 run-scoped authorization 和 scoped MCP gateway 保护 LangBot 持有资源。外部 code harness 的执行环境由用户或部署平台负责隔离;LangBot 当前不提供 managed sandbox。
不能对外说明:
+35 -12
View File
@@ -1,14 +1,37 @@
# AgentRunner Pluginization Status
# Runner Pluginization Status
本文档是 `docs/agent-runner-pluginization/` 的状态事实源。协议 schema 仍以 [PROTOCOL_V1.md](./PROTOCOL_V1.md) 为准;测试步骤以 [AGENT_RUNNER_QA_GUIDE.md](./AGENT_RUNNER_QA_GUIDE.md) 为准;安全发布门槛以 [SECURITY_HARDENING.md](./SECURITY_HARDENING.md) 为准。
本文档是 `docs/agent-runner-pluginization/` 的状态事实源。协议 schema 仍以 [PROTOCOL_V1.md](./PROTOCOL_V1.md) 为准;测试步骤以 [RUNNER_QA_GUIDE.md](./RUNNER_QA_GUIDE.md) 为准;安全发布门槛以 [SECURITY_HARDENING.md](./SECURITY_HARDENING.md) 为准。
状态快照日期:2026-07-15
状态快照日期:2026-09-05。代码基线为 LangBot `dev/4.11.x` / `a4d36aa2d` 和 SDK `dev/4.11.x` / `f1da058`;本文是本地检视快照,不代表远端最新状态或正式发布批准
## 当前版本与验收边界
- Core 声明版本 `4.11.0``pyproject.toml` / `uv.lock` 仍依赖 `langbot-plugin==0.5.3`SDK 源码声明版本 `0.5.5`。本机 Core 实际导入旁边 SDK 的可编辑源码,不能用这个组合的测试通过证明 registry 安装可复现。
- 检视时 SDK 有四个未提交文件:`api/agent_tools/asset_gateway.py``api/agent_tools/external_tools.py``api/entities/builtin/runner/resources.py`(均位于 `src/langbot_plugin/`),以及 `tests/api/test_agent_tools_mcp_bridge.py`。这些改动增加 `platform_tools` 分类发现、可选 schema 和 gateway 引导。下面的 SDK 测试包含这些工作区改动;正式配套版本尚需冻结。
- 本轮仅重新执行下列定向测试与 TypeScript 检查,没有重跑全量 backend、真实平台、provider、浏览器 E2E 或 Cloud 部署门禁。后文旧日期的成功记录仍是历史证据。
| 2026-09-05 验证 | 结果 |
| --- | --- |
| Core`tests/unit_tests/agent``tests/unit_tests/api/service/test_agent_service.py``tests/unit_tests/platform/test_routing_rules.py``tests/unit_tests/api/service/test_maintenance_service.py` | 561 passed74 warnings |
| SDK`tests/api/entities/builtin/runner``tests/api/proxies``tests/api/test_agent_tools_mcp_bridge.py``tests/runtime/plugin/test_mgr_runner.py``tests/runtime/plugin/test_dependency_environment.py``tests/runtime/plugin/test_restart_coordinator.py` | 362 passed10 warnings |
| Web`pnpm exec tsc --noEmit` | pass |
| Web`pnpm test:unit` | 62 passed2 failed |
前端失败为源码形状断言:`oss-cloud-ui-privacy.test.mjs` 仍要求 fieldset 的精确旧 class`processor-detail-workbench.test.mjs` 仍要求 Agent 的旧四分区。当前表单已增加布局 class,并使用运行器、运行器配置、事件与工具分区。发布前需按当前设计更新断言或修正实现,不能记为全绿。
## 近期已集成内容
- **EBA 与独立 Agent**Bot 事件绑定、observer 广播、Pipeline / Agent / discard 单目标分派、Agent CRUD、处理器工作台与路由诊断已在同一分支。逻辑路由器实现在 `pkg/platform/botmgr.py::RuntimeBot`,不是外部独立 EventRouter 服务。
- **事件感知工具权限**Core 9 月 4 日提交已实现 `event_*` 自动事件工具、`allowed_platform_tools``allowed_tools`。Host 冻结事件目标,结合 Runner 权限、适配器能力和运行快照授权,执行时再次检查。详见 [PLATFORM_ACTION_TOOLS.md](./PLATFORM_ACTION_TOOLS.md)。SDK 分类发现的提交状态见上文。
- **产品流程**:Runner 市场内联安装、安装恢复和健康状态、事件范围选择、统一详情工作台、Agent 调试及 Bot 平台事件调试已实装。Agent 自身已无 enabled 开关;Bot 路由仍有 enabled。具体页面形态见 [处理器页面](../event-based-agents/08-agent-page-and-event-orchestration.md)。
- **Runtime 与存储**SDK 已加入 artifact 对应的独立依赖环境、Windows worker 连接路径、安装期间响应性测试;Core/Plugin Runtime/Box 分别报告拥有的存储目录。存储分析是观测,不提供硬配额。
- **工作空间**:OSS 单工作空间多人协作和资源作用域已合入。Cloud 隔离基础与生产激活分开验收,见 [Cloud 剩余事项](../multi-tenant/cloud-v2-pending-verification.md)。
## 实现状态
| 领域 | 状态 | 说明 |
| --- | --- | --- |
| SDK manifest schema | Done | `AgentRunnerManifest` 包含 typed `capabilities` / `permissions`;未知 capability / permission key 禁止进入 typed model。 |
| SDK manifest schema | Done | `RunnerManifest` 包含 typed `capabilities` / `permissions`;未知 capability / permission key 禁止进入 typed model。 |
| Runner discovery | Done | Runtime 返回 typed manifestHost registry 校验单个 runner,失败 warning + skip,不影响其它 runner。 |
| Host resource authorization | Done | `ctx.resources``ctx.context.available_apis` 由 manifest permissions 与 binding policy / run scope 求交后生成。 |
| Run authorization snapshot | Done | active run session 冻结 run-scoped resources 与 available APIsruntime handler 按 snapshot 校验 pull API。 |
@@ -25,32 +48,32 @@
## Spec 与实现已知差距
- `action.requested` 是严格白名单协议面:当前只执行 `interaction.requested`;其它 action 仍只记录 telemetry,不提供通用 platform action executor
- `action.requested` 是严格白名单协议面:当前只执行 `interaction.requested`;其它 action 仍只记录 telemetry。平台语义动作通过已授权的 `event_*` / `platform_*` 工具执行,不通过任意 result action 或原始 `call_platform_api`
- 结构化交互 SDK typed contract 与 DifyAgent continuation 已实现;SDK 正式发布、真实 Dify 凭据 E2E,以及需要长驻双向进程的 Claude Code 权限确认仍是后续验收项。Host 不持有 provider 私有 token。
- State 与 storage 的长期类型边界仍可继续收窄;当前合同只要求 JSON-safe state 与受控 storage API。
- `ToolResource.parameters` 已作为 best-effort full schema 由 Host 在构造 `ctx.resources` 时一次塞齐;无 schema 时 runner 仍需兼容 `parameters=None` 或按需调用 detail API。
- EventLog / Transcript 已提供显式 cleanup primitive;长期 retention 默认值、TTL 调度接入和 sandbox/workspace 文件清理仍是运维收尾项,应在 Runtime Control Plane 产品化前补齐。
- External harness 的 native shell / filesystem / CLI / MCP 权限不受 manifest permissions 约束;manifest permissions 只约束 LangBot 持有的资源访问。
- LangBot 当前不承诺 managed sandboxexternal harness 的 OS/process/network quota、workspace GC、provider-native tool 权限由用户或部署环境承担。
- Runtime Control Plane v2 当前只落地 Host 事实源和控制原语;还没有内置 Agent Platform UI、业务队列、daemon 进程托管、runtime wakeup channel、跨 Host 分布式锁 provider 登录态诊断。
- Runtime Control Plane v2 已有 Host 事实源和控制原语,独立 Agent 与处理器 UI 也已存在;仍缺业务任务队列、外部 harness daemon 托管、wakeup channel、跨 Host 分布式锁 provider 登录态诊断。SDK installation worker 的 supervisor/重启协调已实现,不能与外部 harness 管控混淆。
## Runner 验收状态
## Runner 历史验收记录(未在本轮重跑)
| Runner | 状态 | 最近证据 |
| --- | --- | --- |
| `plugin:langbot-team/LocalAgent/default` | Unit-pass; Marketplace UI pass; Debug Chat E2E pass | 2026-07-12 隔离 first-run 实例从真实 AgentRunner catalog 安装 `langbot-team/LocalAgent` 0.1.0Host 注册 `plugin:langbot-team/LocalAgent/default`,Wizard 自动选中并解锁后续操作。2026-07-15 `2026-07-15-08-44-10-770-08-00-sandbox-skill-authoring-edit-existing-e2e` 使用真实 `gpt-5.5` 完成 Skill 创建、注册、同 Query 激活、已激活包编辑与脚本执行;三阶段 UI、浏览器诊断和结构化文件系统检查全部通过,每阶段恰好新增一个 Bot 气泡,p95 14.6 秒、错误率 0。 |
| `plugin:langbot-team/ACPAgentRunner/default` | Unit-pass; Debug Chat E2E pass | 2026-07-15 从本地 0.1.4 发布包安装并注册 PascalCase runnerremote-ssh Claude ACP 通过反向隧道调用 run-scoped `langbot_get_current_event`,97.8 秒返回可见结果;Host 将增量 delta 和 `message.completed` 聚合为一个完整 Bot 气泡。 |
| `plugin:langbot-team/LocalAgent/default` | Unit-pass; Marketplace UI pass; Debug Chat E2E pass | 2026-07-12 隔离 first-run 实例从真实 Runner catalog 安装 `langbot-team/LocalAgent` 0.1.0Host 注册 `plugin:langbot-team/LocalAgent/default`,Wizard 自动选中并解锁后续操作。2026-07-15 `2026-07-15-08-44-10-770-08-00-sandbox-skill-authoring-edit-existing-e2e` 使用真实 `gpt-5.5` 完成 Skill 创建、注册、同 Query 激活、已激活包编辑与脚本执行;三阶段 UI、浏览器诊断和结构化文件系统检查全部通过,每阶段恰好新增一个 Bot 气泡,p95 14.6 秒、错误率 0。 |
| `plugin:langbot-team/ACPRunner/default` | Unit-pass; Debug Chat E2E pass | 2026-07-15 从本地 0.1.4 发布包安装并注册 PascalCase runnerremote-ssh Claude ACP 通过反向隧道调用 run-scoped `langbot_get_current_event`,97.8 秒返回可见结果;Host 将增量 delta 和 `message.completed` 聚合为一个完整 Bot 气泡。 |
| `plugin:langbot-team/ClaudeCodeAgent/default` / `plugin:langbot-team/CodexAgent/default` | Unit-pass; E2E pending | 通过 runner 仓库单测覆盖 session、run_id 注入和 LangBot MCP gateway;真实 harness E2E 取决于对应运行环境、CLI/daemon 可用性和 provider 登录态。 |
| Dify | Human-input unit-pass; credential E2E pending | `langbot-agent-runner/dify-agent` 已实现 `workflow_paused`、原子字段/确认交互、plugin-storage continuation、Dify submit/events 恢复与再次暂停;真实 Dify 凭据 E2E 待执行。 |
| n8n / Coze / DashScope / Langflow / Tbox / DeerFlow / WeKnora | Unit-pass; credential smoke optional | 2026-06-13 plugin layout / parser tests 通过;真实服务凭据 smoke 非每轮必跑。 |
## Host / SDK 验收状态
## Host / SDK 历史验收记录
| 范围 | 状态 | 最近证据 |
| --- | --- | --- |
| LangBot Runtime Control Plane v2 foundation | Unit-pass; EBA release gate 5/5 pass; AgentRunner preflight pass | 2026-07-12 `eba-functional-20260712-release-gate-rerun` 通过 Quick Start 场景筛选、隔离实例 Runner Marketplace 安装、Runner 健康状态、事件路由 dry-run / 合成派发,以及真实 OneBot `group.member_joined` → Agent → `send_group_msg` 链路。2026-07-15 AgentRunner release preflight 16 项通过、0 warningfixture contract、5 类 behavior matrix、ledger schema / async DB readiness / 100-run stress / 120-run 8-worker contention / claim-lease-auth concurrency、SDK runtime chaos 探针全部通过。 |
| LangBot Runtime Control Plane v2 foundation | Unit-pass; EBA release gate 5/5 pass; Runner preflight pass | 2026-07-12 `eba-functional-20260712-release-gate-rerun` 通过 Quick Start 场景筛选、隔离实例 Runner Marketplace 安装、Runner 健康状态、事件路由 dry-run / 合成派发,以及真实 OneBot `group.member_joined` → Agent → `send_group_msg` 链路。2026-07-15 Runner release preflight 16 项通过、0 warningfixture contract、5 类 behavior matrix、ledger schema / async DB readiness / 100-run stress / 120-run 8-worker contention / claim-lease-auth concurrency、SDK runtime chaos 探针全部通过。 |
| Host Skill / native tool integration | Unit-pass; WebUI E2E pass | 2026-07-15 provider / native / Skill / monitoring 定向测试 67 项通过,Pipeline / Chat / Wrapper 定向测试 61 项通过,Skills CLI 105 项通过;真实 Debug Chat 验证 `register_skill` 后同 Query `activate` 成功,监控工具调用不再把 SQL 行误取为字符串,结构化 JSON 文件检查不依赖格式空格,非流式多阶段 runner 结果只生成一个最终 Bot 气泡。 |
| SDK AgentRunner control entities / proxy | Unit-pass | 2026-06-23 SDK `tests/api/entities/builtin/agent_runner``tests/api/proxies``tests/api/test_agent_tools_mcp_bridge.py``tests/runtime/plugin/test_mgr_agent_runner.py``tests/runtime/test_pull_api_handlers.py``tests/runtime/io/handlers/test_plugin_handler.py`、EBA event entities 和 message tests 通过,覆盖 typed entities、AgentRunAPIProxy、MCP bridge、runtime manager 与 pull API handlers。 |
| SDK Runner control entities / proxy | Unit-pass | 2026-06-23 SDK `tests/api/entities/builtin/runner``tests/api/proxies``tests/api/test_agent_tools_mcp_bridge.py``tests/runtime/plugin/test_mgr_runner.py``tests/runtime/test_pull_api_handlers.py``tests/runtime/io/handlers/test_plugin_handler.py`、EBA event entities 和 message tests 通过,覆盖 typed entities、RunnerAPIProxy、MCP bridge、runtime manager 与 pull API handlers。 |
## 历史高价值记录
@@ -1,342 +1,66 @@
# EBA 产品化与发布计划
> 状态:规划草案,2026-07-01
>
> 范围:将已经合并的 AgentRunner 插件化和 Event Based Agent 适配器工作产品化,使非技术用户也能快速上手。本文聚焦产品缺口、发布门禁和 SaaS 多命名空间租户能力。本文不引入新的协议 schema;协议事实仍以 [PROTOCOL_V1.md](./PROTOCOL_V1.md) 为准,Host 模型事实仍以 [HOST_SDK_INFRASTRUCTURE.md](./HOST_SDK_INFRASTRUCTURE.md) 为准。
更新:2026-09-05,适用于合并后的 `dev/4.11.x`。本文维护产品边界和剩余交付顺序;具体实现、提交与验证结果统一记录在 [STATUS.md](./STATUS.md)。原 2026-07-01 草案中的功能缺口和 Phase 编号不再作为当前排期。
## 1. 产品方向
## 产品模型
当前技术方向是正确的:LangBot 应该把平台输入视为事件,为每个事件解析出一个有效路由,并通过 AgentRunner Host 边界调用一个处理资产。但这还不是一个非技术用户无需理解内部架构就能采用的产品
- **Bot**:平台连接、凭据和事件路由入口。用户在机器人上决定“发生什么时,使用哪个处理器”
- **Processor**:可复用处理逻辑的上位概念,当前类型为 Agent 与 Pipeline。
- **Pipeline**:保留完整 Stage 链的消息处理器,提供预处理、AI、后处理、扩展和输出控制。
- **Agent**:独立配置对象,选择 Runner 插件并配置事件范围、运行器与工具权限,可被多个 Bot 引用。
- **Workflow**:后续编排方向,当前尚无完整执行产品。
- **Solution**:后续分发单元,包含处理器、路由模板、依赖、变量和文档。
产品层模型应当是:
处理器入口聚合 Agent 与 Pipeline,不转换实体,不复制旧 Pipeline runner 配置生成 Agent。EBA 是内部术语,主要产品流程使用机器人、事件、处理器、流水线和工具等名称。
- **机器人(Bot**:平台连接与事件路由入口。机器人拥有适配器凭据、平台权限、入站事件可见性,以及这些事件的路由表。
- **处理器(Processor**:可复用的事件处理资产。当前处理器类型包括 **Agent****Pipeline**。未来可以增加 **Workflow**
- **Pipeline**:一等的无代码消息处理器,通过完整 Stage 链提供预处理、AI、后处理、扩展和输出控制。Pipeline 只处理消息,应当只能绑定到 `message.*` 事件。
- **Agent**:由 runner 驱动的事件优先处理器。Agent 可以根据自身声明的事件支持范围处理消息事件和非消息事件。
- **Solution(方案包)**:未来的分发/导出单元,包含处理器、路由模板、依赖清单、变量和文档。Solution 不应包含具体机器人凭据、租户密钥或已安装资产的 UUID 绑定。
## 已完成的产品化基础
`EBA` 是内部工程术语。它可以出现在内部设计文档中,但不应出现在主要产品流程里。面向用户的语言应优先使用“频道”“事件路由”“处理器”“消息流水线”“自动化”“路由模板”等表达。
## 2. 当前基础
已合并分支已经具备内部冒烟测试所需的技术基础:
- EBA 适配器可以把平台活动规范化为稳定的 Host 事件名。
- 机器人可以持久化 `event_bindings`,并将事件路由到 Agent、Pipeline 或丢弃目标。
- 旧消息输入可以投射到标准的 `message.received` 事件路径。
- Pipeline 仍可作为只处理消息的无代码处理器使用。
- AgentRunner 插件化提供了 Host 与 Runner 的契约、事件优先上下文、结果流和运行时集成边界。
- 官方 local runner 和 external runner 插件可以验证 runner 行为已经不再硬编码在 LangBot Core 中。
- WebUI 已具备 Agent/处理器管理页面,以及机器人侧事件路由页面。
- 机器人事件路由已具备 dry-run 诊断、运行时状态展示,以及安全的合成测试事件派发;测试事件会走已保存的 runtime 路由,但抑制真实平台出站动作。
- MCP 工具面已暴露机器人事件路由状态查询和合成测试事件派发,便于 QA agent 或外部调试工具复用。
这是一个技术收敛里程碑,还不是产品就绪版本。
## 3. 距离非技术产品的缺口
### 3.1 概念负担
当前用户仍需要理解过多内部概念:EBA、适配器事件名、runner 标识、插件运行时健康状态、事件模式、优先级和绑定目标。面向非技术用户的产品应当通过意图和结果来引导:
- “收到一条消息时,用这个处理器回复。”
- “有新群成员加入时,发送欢迎语。”
- “收到好友请求时,让 Agent 判断是否接受。”
`group.member_joined` 这类原始事件模式应继续保留在高级模式中,但默认 UI 应按友好名称和平台能力对事件进行分组。
### 3.2 上手路径
首次使用路径应当从用例开始,而不是从架构开始:
1. 选择一个频道。
2. 连接账号或 webhook。
3. 选择该频道支持的事件预设。
4. 选择或创建一个处理器。
5. 发送测试事件。
6. 如果失败,阅读简单的运行轨迹。
当前产品仍假设用户能诊断后端、插件运行时、Box 运行时、适配器和 runner 插件是否都已连接。对于开发者这是可接受的,但对非技术用户不可接受。
### 3.3 适配器就绪度
每个适配器都需要产品能力清单,而不仅是工程实现:
- 支持的事件列表及友好标签;
- 支持的出站动作;
- 所需凭据和配置步骤;
- 本地部署、自托管、SaaS 可用性;
- 测试信号可用性;
- 废弃/遗留状态;
- 已知限制。
废弃适配器应在产品中明确标记为“已废弃”或“遗留”。新的事件型适配器应按频道名称和能力描述,而不是使用 EBA 缩写。
### 3.4 处理器体验
处理器页面应管理可复用的 Agent 与 Pipeline,而不是让用户一次性理解所有事件路由决策。
- 创建 Agent 时应提供有倾向性的 runner 模板。
- 创建 Pipeline 时应继续保持无代码消息流水线路径。
- 未来 Workflow 的执行语义稳定后,可以作为另一种处理器类型引入。
- 支持的事件范围应作为能力信息和高级约束展示,而不是作为创建流程的主要概念。
- 依赖健康状态应可见:runner 插件是否安装、运行时是否连接、所需模型是否配置、所需资源是否可访问。
### 3.5 机器人事件路由体验
机器人页面应成为平台特定事件路由的主要配置位置,因为平台事件在已连接频道的上下文中最容易被用户理解。
最低产品要求:
- 基于适配器能力生成友好的事件选择器;
- 目标选择器按事件兼容性过滤;
- 对非消息事件隐藏 Pipeline;
- 对重叠路由给出冲突警告;
- 优先级先用视觉方式解释,而不是首先展示原始数字;
- 提供路由测试按钮,可以注入或重放样例事件;
- 每条路由展示状态,包括最近一次匹配的 run 和最近失败原因;
- 提供安全的兜底路由,包括显式丢弃。
### 3.6 可观测性
非技术用户需要的是简短运行轨迹,而不是原始日志:
```text
收到事件 -> 命中路由 -> 启动处理器 -> 动作已投递
```
当失败发生时,UI 应指出失败层级:
- 频道未连接;
- 适配器不支持该事件;
- 没有路由命中;
- 处理器已禁用;
- runner 插件不可用;
- 模型/资源缺失;
- 投递权限被拒绝。
### 3.7 文档和模板
发布需要面向产品场景的文档和模板:
- 客服机器人;
- 群欢迎和群管理;
- 好友请求审核;
- Dify 支持的外部 Agent
- 使用 LangBot 模型和知识库的本地 Agent;
- 用于多阶段消息处理的 Pipeline。
文档应先描述产品模型,只在高级架构章节中暴露内部术语。
## 4. 推荐 UX 边界
之前“把所有事件编排都放进 Agent”的方向应当收窄。更好的边界是:
- **机器人页面负责事件路由**,因为事件面是平台特定的,用户也自然会在机器人上配置频道行为。
- **处理器页面负责处理器资产**,因为 Agent 与 Pipeline 都应能跨机器人复用,并且未来可以被打包进 Solution。
- **Pipeline 保持为一种处理器类型**,而不是隐藏在 Agent 术语背后的历史对象。
这样可以降低心智负担:
- 用户在机器人页面问:“当这个机器人遇到某件事时,应该做什么?”
- 用户在处理器页面问:“我想复用什么处理逻辑?”
这也支持同一个机器人上的不同事件使用不同处理器类型:一个事件可以使用 Pipeline,另一个事件可以使用 Agent,未来另一个事件可以使用 Workflow。
## 5. 未来导出、分发和导入单元
导出/导入不在当前实现范围内,但产品边界不应阻塞它。
正确的未来分发单元是 **Solution**,不是机器人,也不是单独的 Agent。
Solution 应包含:
- 处理器:Agent、Pipeline、未来 Workflow 定义;
- 路由模板:事件模式、友好名称、目标逻辑引用、默认优先级和可选条件;
- 依赖清单:所需 runner 插件、适配器能力要求、模型、工具和资源;
- 变量:用户提供的值,例如 API key、频道选择、模型选择和 prompt 参数;
- 文档:配置意图和预期行为。
Solution 不应包含:
- 具体机器人凭据;
- 已安装运行时 token
- 租户或命名空间 UUID
- 密钥;
- 原始平台账号标识;
- 已解析的机器人事件绑定 UUID。
导入时,应在目标命名空间内解析路由模板。用户需要先选择机器人/频道,并授予所需权限。
## 6. SaaS 多命名空间架构
产品在公开 SaaS 发布前必须支持多命名空间 SaaS 架构。事件路由模型很敏感,因为适配器、凭据、处理器、运行时、状态和日志都会跨越信任边界。
### 6.1 命名空间模型
采用分层命名空间模型:
| 范围 | 用途 |
| 用户流程 | 当前实现 |
| --- | --- |
| 租户(Tenant) | 计费、法律归属、顶层隔离。 |
| 工作空间(Workspace) | 租户内的协作和产品工作区。 |
| 命名空间(Namespace) | 机器人、处理器、运行时 token、资源和日志的可部署隔离边界。自托管部署可以只有一个默认命名空间。 |
| 机器人范围 | 平台适配器实例和事件路由表。 |
| 处理器范围 | Agent、Pipeline、Workflow 及相关配置。 |
| 运行时范围 | 插件运行时、runner 注册、lease 和执行权限。 |
| 资源范围 | 知识库、模型凭据、文件、状态和密钥。 |
| 创建处理器 | Agent/Pipeline 类型选择、独立配置与统一详情工作台 |
| 选择运行器 | 已安装 Runner 动态 metadata、市场安装入口、安装进度/恢复和可用性反馈 |
| 配置 Agent | 运行器、运行器配置、事件与工具;基础信息在详情入口编辑 |
| 配置事件 | 分组事件选择、兼容目标过滤、路由排序、冲突提示和兜底匹配说明 |
| 验证行为 | 路由 dry-run、Agent 调试、Bot 配置中的平台事件调试;合成派发仍需遵循抑制真实出站的后端边界 |
| 理解故障 | 路由匹配/失败轨迹、Runner 状态、模型测试与调试错误反馈 |
| 首次使用 | 场景引导与插件化 Runner 安装流程 |
| 工具权限 | 自动事件工具、显式平台动作、普通工具白名单,运行时再次授权 |
| 存储诊断 | Core、Plugin Runtime、Box 分别报告拥有的目录与不可用状态 |
在 SaaS GA 前,核心持久化对象都应携带 `tenant_id``workspace_id``namespace_id`。自托管部署可以在迁移时种子化一个默认租户/工作空间/命名空间
这张表表示代码已实现,不代表所有平台/provider 和首次安装组合均完成当前版本真实验收。页面契约见 [处理器与事件编排](../event-based-agents/08-agent-page-and-event-orchestration.md),动作授权见 [平台工具](./PLATFORM_ACTION_TOOLS.md)
### 6.2 事件入口隔离
## 4.11 发布收尾
每个入站事件都必须先解析命名空间,再进行路由匹配:
| 顺序 | 工作 | 完成条件 |
| --- | --- | --- |
| 1 | 冻结跨仓库依赖 | 提交/确认 SDK 平台工具发现改动,固定配套 SDK 和 Runner 包,更新 Core 声明及 lock;验证 registry 或精确 commit 的干净安装 |
| 2 | 收敛自动化 | 按当前页面设计处理前端旧断言,执行 backend/SDK 定向测试、前端单测、类型检查及发布所需 lint/build/E2E |
| 3 | 验证用户路径 | 空白实例安装 Runner、创建 Agent/Pipeline、连接 Bot、保存路由、消息/非消息执行、工具允许与拒绝、错误诊断 |
| 4 | 补真实交互与平台证据 | 当前版本的选定平台媒体/回调、Dify continuation、外部 harness;记录支持、未支持、阻塞及未执行 |
| 5 | 完成生命周期 | EventLog/Transcript retention 调度、状态/文件清理、取消和重启后的行为有明确策略与测试 |
```text
adapter ingress -> tenant/workspace/namespace resolution -> event normalization -> event log append -> route match -> processor run
```
验收记录必须写明 Core/SDK/Runner 版本、操作系统、Runtime 连接方式、Box backend 和是否使用 editable 源码。真实平台测试、合成事件与 mock provider 测试分别记账。过去的成功报告不能自动作为当前 HEAD 的通过记录。
Webhook 和回调端点应编码或查找命名空间范围内的适配器安装。来自一个命名空间的平台事件,绝不能匹配另一个命名空间的路由,即使适配器名称、机器人名称或原始平台 ID 发生碰撞。
## Cloud 独立交付边界
### 6.3 路由目标规则
当前 OSS 支持单 Workspace 多成员及固定 RBACCloud 目录和计费归控制面,Core 负责资源作用域与执行边界。不能用 edition 字段或普通配置开启 OSS 多 Workspace。
运行时路由绑定可以使用已安装 UUID,但导出的路由模板必须使用逻辑引用。在 SaaS 中:
当前业务隔离单位是 Workspace 与 execution generation;不要求依据旧草案额外创建 Tenant/Workspace/Namespace 三层同构表。目录、安装绑定、存储与执行作用域以 [多租户架构](../multi-tenant/workspace-multi-user-architecture.md) 为准。
- 默认情况下,机器人路由只能指向同一命名空间内的处理器;
- 跨命名空间目标默认禁止,除非由策略明确共享;
- Pipeline 目标仍然只允许处理消息;
- 路由冲突评估应限制在命名空间内;
- 路由审计事件必须包含租户、工作空间、命名空间、机器人、路由、目标和 run 标识。
生产激活仍需关闭:
### 6.4 运行时和插件隔离
- 插件和可配置出站目标的网络/SSRF 策略。
- Plugin installation 与 Box 各存储面真正的 byte/inode 硬配额。
- 普通业务写入贯穿提交的 generation fence、业务 outbox 和持久对象引用切换保障。
- 最终 Linux/cgroup、持久卷、数据库权限、容量、故障恢复与 24 小时 soak 验收。
插件运行时和 runner 注册表需要命名空间范围的授权:
完整清单见 [Cloud 剩余验证](../multi-tenant/cloud-v2-pending-verification.md)。存储统计、局部压力测试或 OSS 功能验收不能替代这些门禁。
- 运行时注册 token 只作用于一个命名空间,或显式允许的一组命名空间;
- runner 发现结果按命名空间权限过滤;
- lease 和 heartbeat 按命名空间隔离;
- run-scoped API token 不能访问 run 所在命名空间之外的对象;
- 插件存储、状态和临时文件的 storage key 应包含命名空间;
- 早期 SaaS 可以接受共享运行时,但每个 API 调用都必须被 scoped 并审计;
- 企业或高风险租户应支持专用运行时。
## 后续产品方向
### 6.5 密钥、资源和状态
- **完整 Agent 管控面**:业务任务队列、唤醒、外部 harness daemon 管理、provider 登录态诊断和分布式执行。已有 run ledger/heartbeat/claim 是底座。
- **Workflow 与多 Agent**:先确定独立处理器或 Runner 扩展形式,再定义串并联、失败恢复、状态冲突和副作用幂等。
- **Solution 导入导出**:使用逻辑依赖和路由模板;导入时选择 Bot、模型和资源,不能保留源实例 UUID、凭据、token 或租户密钥。包格式与更新策略待定。
- **模型能力与评估**:国内 Provider reasoning 专项回归、reasoning token/生效策略记录、质量与延迟成本基线,见 [模型思考控制](../review/model-reasoning-control-design.md)。
密钥和资源不能全局寻址:
- 适配器凭据存放在命名空间范围的密钥存储中;
- 模型提供商凭据可以根据策略按租户/工作空间/命名空间设定范围;
- 知识资源声明允许哪些命名空间使用;
- Agent 持久状态按租户/工作空间/命名空间/处理器分区,除非共享策略另有规定;
- 事件日志和会话记录按命名空间隔离,并受保留策略约束。
### 6.6 Marketplace 和已安装资产
Marketplace 包可见性不等于已安装资产可见性:
- Marketplace 包可以是公开、租户私有或工作空间私有;
- 安装会创建命名空间本地资产,或命名空间本地引用;
- 已安装处理器和路由模板默认复制,避免意外跨租户变更;
- 更新必须显式且可审计。
### 6.7 SaaS 测试要求
SaaS beta 前必须验证:
- 租户 A 的事件不能匹配租户 B 的路由;
- 命名空间 A 的运行时不能 claim 命名空间 B 的 run
- 命名空间 A 的处理器不能读取命名空间 B 的资源;
- Solution 导入不能保留源租户 UUID 或密钥;
- 路由重放不能暴露另一个命名空间的原始事件 payload;
- 管理员可以查看审计轨迹,但不能访问密钥值。
## 7. 发布计划
### Phase 0:技术收敛
目标:证明合并分支可以基于事件绑定和外置 runner 运行。
必要门禁:
- 机器人事件路由以 `event_bindings` 作为唯一路由来源。
- 旧机器人 pipeline 路由字段已移除或迁移。
- Pipeline 可作为只处理消息的处理器运行。
- Agent runner 插件冒烟测试通过,覆盖 local runner 和 Dify runner。
- 迁移命名和 downgrade 路径有效。
- 主 UI 不再在面向用户的适配器名称中暴露 “EBA”。
### Phase 1:面向技术用户的私有 Beta
目标:让贡献者和早期自托管用户可用。
必要门禁:
- 文档和 UI 中存在适配器能力矩阵;
- 路由编辑器会过滤不兼容目标;
- runner/plugin 健康检查可见;
- local Agent 和 Dify Agent 有引导式配置路径;
- 每条路由有运行轨迹;
- 废弃适配器被一致标记;
- 失败信息能指出失败层级。
### Phase 2:面向非技术用户的产品 Beta
目标:让用户无需阅读架构文档也能完成常见场景。
必要门禁:
- 首次使用机器人向导从用例和频道开始;
- 事件预设默认隐藏原始事件模式;
- 存在路由模拟或测试事件能力;
- 存在常见处理器模板;
- 冲突警告和兜底行为清晰;
- 文档先使用产品语言,再介绍高级术语;
- SaaS 命名空间 schema 已实现,或已经具备迁移准备。
### Phase 3SaaS Beta
目标:安全地为多个租户运行产品。
必要门禁:
- 所有路由、运行时、状态、日志和资源事实都具备租户/工作空间/命名空间字段;
- 命名空间范围的运行时注册和 run claim 已强制执行;
- 命名空间范围的密钥和适配器安装已强制执行;
- 路由匹配和审计限制在命名空间内;
- 配额、保留策略和管理员审计界面存在;
- 自托管默认命名空间迁移已有文档。
### Phase 4GA
目标:让产品具备广泛采用所需的可靠性。
必要门禁:
- Solution 导出/导入已实现,并支持依赖和变量解析;
- Marketplace 分发支持命名空间本地安装;
- 跨命名空间共享策略是显式的;
- 安全评审覆盖适配器、运行时 token、runner API、密钥、日志和路由重放;
- 升级和回滚流程已有文档;
- 产品遥测可以衡量上手流失和路由失败类别。
## 8. 验收清单
只有满足以下条件,才可以认为发布版本达到产品就绪:
- 非技术用户可以连接一个受支持频道,选择场景,绑定处理器,测试它,并在不编辑原始 JSON 的情况下理解结果;
- 产品 UI 在主要流程中避免使用 EBA 这类内部术语;
- 机器人页面负责平台事件路由,处理器页面负责可复用的 Agent 与 Pipeline
- Pipeline 仍作为无代码消息处理器可见,并且不会出现在非消息事件目标中;
- 同一个机器人的不同事件可以路由到不同处理器类型;
- 路由失败能按层级解释;
- 命名空间隔离通过 schema、service check、运行时 token、storage key 和测试强制执行;
- 导出/导入实现后,使用带路由模板的 Solution 包,而不是具体机器人绑定。
## 9. 待决策问题
- 组合处理器入口统一使用 “Processor / 处理器”;Agent 与 Pipeline 是其中平级的类型。
- Workflow 应作为独立持久化处理器类型,还是作为 Agent runner 类别。
- SaaS 命名空间初期是否与工作空间一一映射,还是高级租户在首个 SaaS beta 就需要一个工作空间下多个命名空间。
- 哪些适配器允许在 SaaS 共享运行时中运行,哪些需要专用运行时隔离。
- 未来 Solution 导出/导入的确切包格式。
这些方向不自动构成 4.11 OSS 发布前置条件,也不承诺旧草案中的 5.0/GA 版本号或日期。
+11 -7
View File
@@ -1,5 +1,9 @@
# Event Based Agents 架构设计总览
> Product revision (2026-09-07): [Event processors and Pipeline plugin compatibility](./09-event-processors.md) defines an explicitly bound Runner alongside Pipeline and Agent. It supersedes the automatic EBA observer product model below; the new component and UI are planned, not yet implemented.
> 当前状态(2026-09-05):平台事件、Bot `event_bindings`、独立 Agent、Pipeline / Agent 平级路由及 WebUI 已集成到 `dev/4.11.x`。实现入口为 `pkg/platform/botmgr.py::RuntimeBot``pkg/agent/runner/`。下文“当前架构的局限性”“现有架构”描述改造前背景;EventBus / EventRouter 图表示职责划分,不表示存在同名独立服务。当前实现和验收以 [STATUS.md](../agent-runner-pluginization/STATUS.md) 为准,平台动作使用[授权工具](../agent-runner-pluginization/PLATFORM_ACTION_TOOLS.md)。
## 1. 背景与动机
### 当前架构的局限性
@@ -48,9 +52,9 @@ MessageAggregator (消息聚合)
QueryPool → Controller → Pipeline (固定阶段链)
│ │
│ ▼
AgentRunner Host orchestrator
│ Runner Host orchestrator
│ ▼
│ plugin AgentRunner
│ plugin Runner
adapter.reply_message() / adapter.send_message()
@@ -79,7 +83,7 @@ EventBus (统一事件总线)
EventRouter (读取 Bot 的 event_bindings)
├─→ Pipeline target — 完整 Stage 链,仅消息事件
├─→ Agent target — 独立 Agent,经插件 AgentRunner 执行
├─→ Agent target — 独立 Agent,经插件 Runner 执行
└─→ discard — 明确丢弃
@@ -146,9 +150,9 @@ pkg/platform/adapters/
### 3.4 事件响应目标与观察者
Pipeline 与 Agent 是长期并存、场景不同的同级处理器。Pipeline 保留完整 Stage 链,面向消息处理;Agent 是独立配置对象,选择一个已安装的插件 AgentRunner,并可声明消息或非消息事件能力。Bot 的 `event_bindings` 只负责把事件绑定到既有 Pipeline、独立 Agent 或 `discard`
Pipeline 与 Agent 是长期并存、场景不同的同级处理器。Pipeline 保留完整 Stage 链,面向消息处理;Agent 是独立配置对象,选择一个已安装的插件 Runner,并可声明消息或非消息事件能力。Bot 的 `event_bindings` 只负责把事件绑定到既有 Pipeline、独立 Agent 或 `discard`
插件 EventListener 是观察者:事件先广播给有权限的监听器,随后路由器再选择一个响应目标。Webhook、Dify、n8n 等外部执行方式若需要作为响应者,应由对应 AgentRunner 插件表达,而不是增加另一套 Host Handler 主链。
插件 EventListener 是观察者:事件先广播给有权限的监听器,随后路由器再选择一个响应目标。Webhook、Dify、n8n 等外部执行方式若需要作为响应者,应由对应 Runner 插件表达,而不是增加另一套 Host Handler 主链。
现有 Pipeline 不会被转换为 AgentPipeline 内的 runner 配置也不会复制到独立 Agent。用户需要 Agent 时自行创建并绑定。
@@ -170,7 +174,7 @@ Pipeline 与 Agent 是长期并存、场景不同的同级处理器。Pipeline
| 2 | 适配器特有 API | 统一抽象 + `call_platform_api` 透传 | 通用 API 覆盖大部分场景,透传机制保证灵活性,避免每个适配器导出独立的类型化 API 包 |
| 3 | 向后兼容策略 | 兼容层适配 | 保留旧事件类型和 API 作为新系统的 alias/wrapper,现有插件无需修改 |
| 4 | 处理器配置存储 | Bot 表使用 `event_bindings`,目标引用原始 Pipeline 或独立 Agent UUID | 路由关系不复制处理器配置,Pipeline/Agent 各自保持事实源 |
| 5 | Agent 处理器定位 | 独立 Agent + 插件 AgentRunner | Host 不再内置具体 runner;不同 AgentRunner 通过统一协议接入 |
| 5 | Agent 处理器定位 | 独立 Agent + 插件 Runner | Host 不再内置具体 runner;不同 Runner 通过统一协议接入 |
| 6 | 事件命名方式 | 命名空间式(`message.received` | 清晰的分类层级,便于通配匹配(`message.*`),与 WebUI 配置天然对应 |
## 5. 文档索引
@@ -190,7 +194,7 @@ Pipeline 与 Agent 是长期并存、场景不同的同级处理器。Pipeline
| 仓库 | 改动范围 |
|------|----------|
| **langbot-plugin-sdk** | 事件定义、实体模型、API 接口、适配器基类、通信协议扩展 |
| **LangBot**(后端) | 适配器实现、事件路由引擎、Bot/Agent 实体、AgentRunner Host 编排 |
| **LangBot**(后端) | 适配器实现、事件路由引擎、Bot/Agent 实体、Runner Host 编排 |
| **LangBot**(前端) | Bot 事件处理器编排面板 |
| **langbot-wiki** | 新架构文档、插件开发指南更新、适配器开发指南 |
| **langbot-plugin-demo** | 示例更新(使用新事件和 API) |
+1 -1
View File
@@ -499,7 +499,7 @@ class PlatformSpecificEvent(Event):
8. 目标处理事件
│ Pipeline → 进入完整 Pipeline 流水线(仅消息事件)
│ Agent → Host 编排已安装的插件 AgentRunner
│ Agent → Host 编排已安装的插件 Runner
│ discard → 不产生响应
9. 处理器执行完毕,可能通过 Host 授权 API 执行响应动作
+12 -11
View File
@@ -1,5 +1,7 @@
# 事件路由与编排
> Implementation update (2026-09-08): the EventListener observer-broadcast proposal below is superseded by [Event processors](09-event-processors.md). Legacy EventListener hooks run only inside Pipeline. New EBA handlers use explicitly created and bound Runner instances, a third peer processor type alongside Agent and Pipeline.
> 状态:当前实施模型(2026-07-12)。本文以 Pipeline / Agent 平级并存为准,不再保留早期 `pipeline / agent / webhook / plugin` 四种 Handler 草案。
## 1. 路由边界
@@ -14,10 +16,10 @@ Pipeline 与 Agent 是平级处理器:
| 处理器 | 配置事实源 | 执行路径 | 事件范围 |
| --- | --- | --- | --- |
| Pipeline | Pipeline 表与完整 Stage 配置 | MessageAggregator -> QueryPool -> RuntimePipeline | 消息事件,首版为 `message.received` |
| Agent | Agent 表中的 runner 与 runner config | AgentRunner Host orchestrator -> plugin AgentRunner | Agent/Runner 声明支持的消息或非消息事件 |
| Agent | Agent 表中的 runner 与 runner config | Runner Host orchestrator -> plugin Runner | Agent/Runner 声明支持的消息或非消息事件 |
| discard | 无处理器配置 | 明确结束路由 | 任意事件 |
插件 EventListener 不是第三种响应目标。Webhook、Dify、n8n、Coze 等外部系统需要响应事件时,由对应 AgentRunner 插件承接。
插件 EventListener 不是第三种响应目标。Webhook、Dify、n8n、Coze 等外部系统需要响应事件时,由对应 Runner 插件承接。
## 2. 数据模型
@@ -32,9 +34,8 @@ class Agent(Base):
description: str
emoji: str
kind: str # 固定为 "agent"
component_ref: str # AgentRunner id
component_ref: str # Runner id
config: dict # runner + runner_config
enabled: bool
supported_event_patterns: list[str]
```
@@ -56,7 +57,7 @@ class Agent(Base):
}
```
Runner id 来自已安装插件的 AgentRunner manifest。Host 不维护 LocalAgent、Dify 或其他具体实现的内置分支。
Runner id 来自已安装插件的 Runner manifest。Host 不维护 LocalAgent、Dify 或其他具体实现的内置分支。
### 2.2 EventBinding
@@ -113,7 +114,7 @@ Binding 只保存引用与路由条件。它不复制 Pipeline 或 Agent 配置
1. 忽略 `enabled = false` 的 binding。
2. 检查 `event_pattern` 与结构化 filters。
3. 校验目标存在、启用且声明支持该事件。
3. 校验目标存在且声明支持该事件。
4. 按 `priority` 从高到低选择;同优先级按稳定列表顺序。
5. 只执行一个响应目标。
@@ -128,20 +129,20 @@ Platform adapter
-> authorized Plugin EventListener observers
-> EventRouter
-> Pipeline target -> full Pipeline stage chain
-> Agent target -> AgentRunner Host orchestrator
-> Agent target -> Runner Host orchestrator
-> discard -> stop
-> Host delivery/platform API
```
### 4.1 Pipeline target
消息事件按原有方式构造 Query,经 MessageAggregator、QueryPool 和完整 Pipeline Stage 链执行。Pipeline 可以继续使用 AgentRunner 作为 AI stage 的实现,但 Pipeline 本身不会因此变成 Agent。
消息事件按原有方式构造 Query,经 MessageAggregator、QueryPool 和完整 Pipeline Stage 链执行。Pipeline 可以继续使用 Runner 作为 AI stage 的实现,但 Pipeline 本身不会因此变成 Agent。
### 4.2 Agent target
Host 读取独立 Agent 的 Runner id/config,构造 event-first context、run-scoped resources 与 delivery policy,再调用插件 AgentRunner。Runner 输出由 Host 统一归一化、记录和投递。
Host 读取独立 Agent 的 Runner id/config,构造 event-first context、run-scoped resources 与 delivery policy,再调用插件 Runner。Runner 输出由 Host 统一归一化、记录和投递。
AgentRunner 可通过 SDK/Python `AgentRunAPIProxy.call_tool` 或 SDK-owned scoped MCP bridge 回调 Host 能力。两条路径都映射到 `PluginToRuntimeAction.CALL_TOOL`,使用相同的 run authorization、Host execution Query、ToolManager 和 Box session 规则。Box session 是 Host canonical scope 的固定长度安全哈希;同一平台会话稳定、不同 scope 隔离、缺少 identity 时 fail closedRunner 不配置 sandbox scope。
Runner 可通过 SDK/Python `RunnerAPIProxy.call_tool` 或 SDK-owned scoped MCP bridge 回调 Host 能力。两条路径都映射到 `PluginToRuntimeAction.CALL_TOOL`,使用相同的 run authorization、Host execution Query、ToolManager 和 Box session 规则。Box session 是 Host canonical scope 的固定长度安全哈希;同一平台会话稳定、不同 scope 隔离、缺少 identity 时 fail closedRunner 不配置 sandbox scope。
### 4.3 Observer side effects
@@ -152,7 +153,7 @@ AgentRunner 可通过 SDK/Python `AgentRunAPIProxy.call_tool` 或 SDK-owned scop
1. Pipeline 与 Agent 保留各自的持久化、编辑和执行语义。
2. 处理器聚合页面可以统一展示二者,但不会创建第三份处理器记录。
3. 旧 Pipeline 仍是 Pipeline;其 runner config 不迁移、不复制为独立 Agent。
4. 需要 Agent 的用户新建 Agent、选择已安装 AgentRunner,再建立 event binding。
4. 需要 Agent 的用户新建 Agent、选择已安装 Runner,再建立 event binding。
5. 一个 Bot 可按不同事件同时绑定 Pipeline 与 Agent。
## 6. WebUI 约束
+2
View File
@@ -1,5 +1,7 @@
# 插件 SDK 改造
> Implementation update (2026-09-08): the EventListener observer-broadcast proposal below is superseded by [Event processors](09-event-processors.md). Legacy EventListener hooks run only inside Pipeline. New EBA handlers use explicitly created and bound Runner instances, a third peer processor type alongside Agent and Pipeline.
## 1. 概述
插件 SDK 需要配合 EBA 架构进行以下改造:
+13 -11
View File
@@ -1,6 +1,8 @@
# EBA 分阶段实施计划
> 更新:2026-07-12。文件名沿用早期设计,但这里的“迁移”仅指代码架构逐步接入 EBA,不代表 LangBot 3.x 数据库或配置升级。
> Implementation update (2026-09-08): the EventListener observer-broadcast proposal below is superseded by [Event processors](09-event-processors.md). Legacy EventListener hooks run only inside Pipeline. New EBA handlers use explicitly created and bound Runner instances, a third peer processor type alongside Agent and Pipeline.
> 更新:2026-09-05。P0P4 的主要实现已落入 `dev/4.11.x`,P5 仍需按当前版本验收;下文工作项用于维护实现边界,不表示全部待开发。文件名沿用早期设计,但这里的“迁移”仅指代码架构逐步接入 EBA,不代表 LangBot 3.x 数据库或配置升级。当前提交、定向测试及发布缺口见 [STATUS.md](../agent-runner-pluginization/STATUS.md)。
## 1. 发布边界
@@ -9,15 +11,15 @@ EBA 跨越 SDK、平台适配器、LangBot Host、WebUI 与插件生态,按可
- LangBot 4.x 不支持从 3.x 数据库或配置升级;不保留 legacy migration chain、旧 JSON 模板或旧 Runner 字段读取。
- Pipeline 与 Agent 平级且长期并存,分别保留持久化模型与执行链。
- 现有 Pipeline 不迁移为 AgentPipeline 内的 runner config 不复制到 Agent。
- 用户需要 Agent 时新建独立 Agent并选择已安装的 AgentRunner。
- 用户需要 Agent 时新建独立 Agent并选择已安装的 Runner。
- Host 不按 LocalAgent id 做运行时、Box 或 WebUI 特判。
- AgentRunner 的 SDK/Python 与 scoped MCP bridge 回调共享 Host 授权与事件 session 规则。
- Runner 的 SDK/Python 与 scoped MCP bridge 回调共享 Host 授权与事件 session 规则。
## 2. 阶段总览
| 阶段 | 目标 | 主要仓库 | 完成条件 |
| --- | --- | --- | --- |
| P0 | SDK 事件、能力与 AgentRunner 协议 | `langbot-plugin-sdk` | typed entities、manifest、proxy、runtime action 通过测试 |
| P0 | SDK 事件、能力与 Runner 协议 | `langbot-plugin-sdk` | typed entities、manifest、proxy、runtime action 通过测试 |
| P1 | 平台适配器 EBA 化 | LangBot + SDK | 事件转换、能力声明、通用/透传 API 通过 adapter checklist |
| P2 | Host 观察者与响应者路由 | LangBot backend | observer 广播 + Pipeline/Agent/discard 单目标仲裁可运行 |
| P3 | 独立 Agent 与 Runner 注册 | LangBot backend + plugins | Agent CRUD、registry、run authorization、delivery 可运行 |
@@ -30,8 +32,8 @@ EBA 跨越 SDK、平台适配器、LangBot Host、WebUI 与插件生态,按可
- 定义规范化平台事件、actor/subject/conversation/delivery context。
- 定义 adapter `supported_events``supported_apis` 与平台透传 API。
- 定义 AgentRunner manifest、run context/result、resource handles 和 pull/callback API。
- 提供 `AgentRunAPIProxy` 与 SDK-owned scoped MCP bridge。
- 定义 Runner manifest、run context/result、resource handles 和 pull/callback API。
- 提供 `RunnerAPIProxy` 与 SDK-owned scoped MCP bridge。
- 保持协议传输与权限校验可测试,不把 Host 私有 Query 对象暴露给插件。
### 验收
@@ -74,7 +76,7 @@ adapter event
- Plugin EventListener 是 observer,不作为 priority fallback。
- Pipeline 只处理消息事件并复用完整 Stage 链。
- Agent 使用独立 Agent 配置和 AgentRunner Host orchestrator。
- Agent 使用独立 Agent 配置和 Runner Host orchestrator。
- edit/reaction 等事件的 observer 副作用能力按事件和 adapter 能力过滤。
- dry-run 与合成派发必须使用同一匹配器,避免 UI 预览与真实路由漂移。
@@ -84,13 +86,13 @@ adapter event
- 同一事件最多一个响应目标,但 observer 仍能收到事件。
- Pipeline 与 Agent 可以在同一个 Bot 的不同 binding 中同时生效。
## 6. P3:独立 Agent 与 AgentRunner
## 6. P3:独立 Agent 与 Runner
### 工作项
- `agents` 只保存 Agent;Pipeline 继续使用自己的表和 API。
- Agent config 使用 `runner.id``runner_config[runner_id]`
- registry 只展示已安装、有效的插件 AgentRunner。
- registry 只展示已安装、有效的插件 Runner。
- Host 构造 run-scoped resources、state、delivery 与 event log/transcript。
- SDK/Python `call_tool` 和 scoped MCP bridge 都回到同一个 Host ToolManager。
- Box session 由 Host 将 instance/workspace/bot/adapter/target/thread scope 规范化并哈希为固定长度 `lb-box-<sha256>`;同 scope 稳定、不同 scope 隔离、缺少 identity 时 fail closed。
@@ -115,7 +117,7 @@ adapter event
### 验收
- 页面不出现 LocalAgent 专属 banner、变量隐藏或 Box/Pipeline 注入逻辑。
- 空 Runner 市场状态给出可安装 AgentRunner 的正常路径。
- 空 Runner 市场状态给出可安装 Runner 的正常路径。
- Pipeline Debug Chat/Monitoring 与 Agent 运行日志分别可用。
## 8. P5:发布门禁
@@ -123,7 +125,7 @@ adapter event
### 自动化
- LangBot backend unit/integration tests 与 Ruff。
- SDK AgentRunner/proxy/MCP bridge tests。
- SDK Runner/proxy/MCP bridge tests。
- Web lint/build 与关键 Playwright cases。
- `skills/bin/lbs validate``skills/bin/lbs index --check`
- LocalAgent 与其他官方 Runner plugin package/test gate。
@@ -1,6 +1,8 @@
# Agent 与 Pipeline 统一编排(产品最终形态)
> **状态**:方向修订稿(2026-06-12),供「适配器改造 / Agent 插件化 / 工作流引擎」三条工作线评审。
> Implementation update (2026-09-08): the EventListener observer-broadcast proposal below is superseded by [Event processors](09-event-processors.md). Legacy EventListener hooks run only inside Pipeline. New EBA handlers use explicitly created and bound Runner instances, a third peer processor type alongside Agent and Pipeline.
> **状态**:历史方向稿(2026-06-12);2026-09-05 标记归档用途。本文的示意 schema、5.0 发布火车、SDK 0.5.0aX 配套与多租户“预留”描述不再作为实施合同。当前 4.11 产品形态见 [08-agent-page-and-event-orchestration.md](./08-agent-page-and-event-orchestration.md),协议见 [PROTOCOL_V1.md](../agent-runner-pluginization/PROTOCOL_V1.md),已完成与剩余事项见 [STATUS.md](../agent-runner-pluginization/STATUS.md)。保留正文仅用于解释早期设计取舍。
>
> 本文档修订 [00-overview.md](./00-overview.md) §3.4 与 [04-event-routing.md](./04-event-routing.md) 中"四种 Handler"的编排模型:**所有编排目标统一进入处理器选择与事件绑定界面,但独立 Agent 与现有 Pipeline 保持不同类型**。事件路由的匹配机制、数据迁移策略、WebUI 交互骨架等内容仍以 04 为准,仅 handler 分类法被本文档取代。
@@ -18,7 +20,7 @@
EventRouter(事件 → 处理器绑定)
├─→ 选中的处理器(响应者,单一仲裁)
│ ├─ Pipeline:保留现有实体和执行链,仅处理消息事件
│ └─ Agent:用户新建并选择 AgentRunner 插件,可接本地、低代码或外部 runtime
│ └─ Agent:用户新建并选择 Runner 插件,可接本地、低代码或外部 runtime
└─→ 插件 EventListener(观察者,N 个广播,可 prevent_default
```
@@ -49,7 +51,7 @@ EventRouter(事件 → 处理器绑定)
04 文档中的 pipeline / agent / webhook / plugin 四种 handler_type,本质上都是"对事件作出响应的逻辑",差别只在编写和部署方式。产品层统一展示和绑定这些处理器,但不会把既有 Pipeline 持久化为 Agent
- **产品**:用户只需理解"给 Bot 的事件绑定处理器",处理器可以是 Pipeline 或 Agent
- **工程**:路由层按 `target_type` 分发到 Pipeline 或 AgentAgent 的扩展集中到 AgentRunner 抽象;
- **工程**:路由层按 `target_type` 分发到 Pipeline 或 AgentAgent 的扩展集中到 Runner 抽象;
- **生态**:Agent 成为市场上可分发、可复用的一等公民。
### 2.2 收编映射
@@ -57,7 +59,7 @@ EventRouter(事件 → 处理器绑定)
| 原 handler_type04 文档) | 收编后 |
|---------------------------|--------|
| `pipeline` | 保留 Pipeline 实体;binding 使用 `target_type=pipeline` 和原 `pipeline_uuid`,进程内直接复用 MessageAggregator → QueryPool → Pipeline 机制 |
| `agent`RequestRunner | 用户新建独立 Agent,并选择对应 AgentRunner 插件;不读取或复制旧 Pipeline 内嵌 runner 配置 |
| `agent`RequestRunner | 用户新建独立 Agent,并选择对应 Runner 插件;不读取或复制旧 Pipeline 内嵌 runner 配置 |
| `webhook` | 外部 Agent 的一种:事件 POST 出去、响应解析为动作(保留 04 §5.4 的请求/响应格式) |
| `plugin`EventListener 分发) | **不收编**——角色不同,见 §2.3 |
@@ -74,7 +76,7 @@ EventRouter(事件 → 处理器绑定)
### 3.1 独立 Agent 与现有 Pipeline
Agent 与 Pipeline 都是一等处理器。用户创建 Agent、选择已安装的 AgentRunner,再把适合的事件绑定到 AgentPipeline 继续保存在 Pipeline 表中,以完整 Stage 链处理消息事件。两者可在同一处理器列表中以不同 `kind` 展示和选择;这种聚合展示不会创建额外记录,也不会在两种模型之间复制配置。
Agent 与 Pipeline 都是一等处理器。用户创建 Agent、选择已安装的 Runner,再把适合的事件绑定到 AgentPipeline 继续保存在 Pipeline 表中,以完整 Stage 链处理消息事件。两者可在同一处理器列表中以不同 `kind` 展示和选择;这种聚合展示不会创建额外记录,也不会在两种模型之间复制配置。
```python
class Agent(Base):
@@ -82,7 +84,7 @@ class Agent(Base):
uuid: str # 主键
name: str
kind: str # 固定为 "agent"Pipeline 使用自己的持久模型
component_ref: str # AgentRunner id,例如 plugin:<author>/<plugin>/<runner>
component_ref: str # Runner id,例如 plugin:<author>/<plugin>/<runner>
config: dict # JSON — runner id、runner config 与资源/状态/投递策略
# 多租户预留:归属主体字段(tenant/workspace),首版可空
```
@@ -154,7 +156,7 @@ class AgentChunk:
```
**流式**:复用 SDK 通信协议既有的 `chunk_status: continue/end` 机制,`handle()` 的每次 yield 对应一个 chunk。
**Pipeline 与 Agent 分流**Pipeline target 继续走 LangBot 进程内的 Pipeline 执行链;独立 Agent 经 AgentRunner 插件 runtime 分发。路由层通过 binding 的 `target_type` 明确区分二者。
**Pipeline 与 Agent 分流**Pipeline target 继续走 LangBot 进程内的 Pipeline 执行链;独立 Agent 经 Runner 插件 runtime 分发。路由层通过 binding 的 `target_type` 明确区分二者。
### 4.3 执行语义与可靠性
@@ -171,7 +173,7 @@ class AgentChunk:
| 版本 | 内容 | 备注 |
|------|------|------|
| 4.11(可选) | 现状成果:12 个 EBA 适配器、插件全事件订阅、`call_platform_api` | 对用户不可见的管道工程 + 插件新能力,不动产品概念 |
| **5.0** | 产品形态首发:EventRouter + event→处理器绑定 + WebUI 编排 + 旧 Bot 路由迁移 + 独立 Agent / AgentRunner 插件 + SDK Agent 组件契约(可标 experimental | `use_pipeline_uuid` 仅改写为指向原 Pipeline 的 binding,不生成 Agent;配 SDK 0.5.0 正式版;走 beta 周期 |
| **5.0** | 产品形态首发:EventRouter + event→处理器绑定 + WebUI 编排 + 旧 Bot 路由迁移 + 独立 Agent / Runner 插件 + SDK Agent 组件契约(可标 experimental | `use_pipeline_uuid` 仅改写为指向原 Pipeline 的 binding,不生成 Agent;配 SDK 0.5.0 正式版;走 beta 周期 |
| 5.x | 工作流 Agent(工作流引擎线挂入)、Agent 市场生态、剩余适配器(satori 等)、Agent 插件化收尾 | 验证开放注册机制 |
| 多租户 | 独立评估:仅数据隔离 → 5.x 部署选项;伴随权限/计费/产品定位变化 → 6.0 | 前置条件是 §4.3 的归属主体预留已落实 |
@@ -1,6 +1,8 @@
# 处理器页面与事件编排产品设计
> 状态:实施稿(2026-06-23
> Implementation update (2026-09-08): the EventListener observer-broadcast proposal below is superseded by [Event processors](09-event-processors.md). Legacy EventListener hooks run only inside Pipeline. New EBA handlers use explicitly created and bound Runner instances, a third peer processor type alongside Agent and Pipeline.
> 状态:当前实现说明(2026-09-05),对应 `dev/4.11.x`。P0P3 已集成;发布验收见 [STATUS.md](../agent-runner-pluginization/STATUS.md)。
>
> 本文档修订 [07-agent-orchestration.md](./07-agent-orchestration.md) 中“Agent 替代 Pipeline”的表述。当前产品形态保留两种长期并存的同级处理器:**Agent** 与 **Pipeline**。处理器页面只是共享入口,不改变二者各自的持久化模型和执行语义。
@@ -10,7 +12,7 @@ LangBot 的处理逻辑分成两种同级形态:
| 形态 | 定位 | 可处理事件 | 典型用户 |
| --- | --- | --- | --- |
| Agent | runner 驱动的事件优先处理器,承载 AgentRunner / 外部 runner | `message.*``group.*``friend.*``bot.*``feedback.*``platform.*` 等声明范围 | 需要直接处理多类平台事件或接入外部 agent runtime 的用户 |
| Agent | runner 驱动的事件优先处理器,承载 Runner / 外部 runner | `message.*``group.*``friend.*``bot.*``feedback.*``platform.*` 等声明范围 | 需要直接处理多类平台事件或接入外部 agent runtime 的用户 |
| Pipeline | 可视化、可控、可组合的消息处理流水线,执行完整 Stage 链 | 仅 `message.*`,首版等价于 `message.received` | 需要预处理、AI、后处理、扩展和输出控制的消息场景 |
处理器页面负责统一管理这两种处理单元:
@@ -34,7 +36,9 @@ LangBot 的处理逻辑分成两种同级形态:
- Pipeline:创建一条独立 Pipeline,执行完整消息 Stage 链。
3. 编辑时按类型进入不同表单:
- Pipeline:沿用原 Pipeline 配置页,包括 AI、触发、安全、输出、扩展、Debug、Monitoring
- Agent配置基础信息、runner、runner config 和事件能力
- Agent:基础信息由详情入口编辑,主配置分为运行器、运行器配置、事件与工具;事件范围、自动事件工具、平台级动作和普通工具白名单在同一配置流程内维护
处理器详情复用 `ProcessorDetailWorkbench`Agent 与 Pipeline 保留各自的配置、调试和日志语义。`RunnerSelect` 提供已安装 Runner 和市场安装入口,安装状态可恢复;Runner 配置来自动态 metadata,不按 LocalAgent id 定制 Host 表单。调试事件选择位于输入区域;Bot 的平台事件调试位于机器人配置中,路由 dry-run 只解释匹配结果。
`/home/pipelines` 继续提供 Pipeline 直接编辑路径;共享处理器入口当前使用 `/home/agents`。URL 是实现路径,不代表 Agent 包含 Pipeline。
@@ -63,14 +67,14 @@ Pipeline 只能被绑定到 `message.*`。如果用户选择非消息事件,
```python
class Agent(Base):
workspace_uuid: str # workspace-scoped persistence
uuid: str
name: str
description: str
emoji: str
kind: str # 首版固定为 "agent"
component_ref: str # runner id / workflow id / future external ref
config: dict # runner runner_config
enabled: bool
component_ref: str # runner reference; execution config uses runner.id
config: dict # runner, runner_config, allowed_tools, allowed_platform_tools
supported_event_patterns: list[str]
created_at: datetime
updated_at: datetime
@@ -104,7 +108,7 @@ Pipeline 投影时固定:
### 3.2 Bot 事件绑定
Bot 新增 `event_bindings` JSON 字段,首版作为轻量配置面。后续当 EventRouter 查询、审计和多作用域规则稳定后,再拆成独立表
Bot 使用 `event_bindings` JSON 字段持久化路由。当前未引入独立路由表;是否拆表应由查询、审计和多作用域需求决定
```json
[
@@ -140,12 +144,12 @@ Bot 新增 `event_bindings` JSON 字段,首版作为轻量配置面。后续
## 5. 并存策略
1. Pipeline 与 Agent 长期并存,各自保存配置并执行自己的运行链路。
2. 现有 Bot 的 `use_pipeline_uuid` 转换为仍指向原 Pipeline 的消息事件绑定
3. 现有 `pipeline_routing_rules` 仍只作用于消息事件
2. 数据库升级中的旧路由转换由 Alembic 负责;当前运行时只读取 `event_bindings`,不再读取 `use_pipeline_uuid`
3. `pipeline_routing_rules` 不再作为第二个运行时路由来源;LangBot 4.x 不支持 3.x 数据库或配置升级
4. `event_bindings` 允许 `target_type=pipeline|agent|discard`Pipeline 目标只限 `message.*`
5. Pipeline 与 Agent 保留各自的持久化和编辑语义;处理器聚合入口只负责统一展示和选择。
## 6. 分阶段落地
## 6. 已集成阶段与维护范围
### P0:处理器入口统一
@@ -170,8 +174,8 @@ Bot 新增 `event_bindings` JSON 字段,首版作为轻量配置面。后续
- EBA 事件先广播插件 observer。
- 然后按 `event_bindings` 的事件模式、filters、priority 和顺序选择一个处理器。
- Pipeline 目标通过 MessageAggregator 进入完整 Pipeline Stage 链;Agent 目标直接进入 AgentRunner 链路。
- 非消息事件只选择声明支持该事件的 Agent,不调用 PipelineAgentRunner 输出有平台 reply target 时会投递回平台。
- Pipeline 目标通过 MessageAggregator 进入完整 Pipeline Stage 链;Agent 目标直接进入 Runner 链路。
- 非消息事件只选择声明支持该事件的 Agent,不调用 PipelineRunner 输出有平台 reply target 时会投递回平台。
## 7. 不做的事
@@ -0,0 +1,92 @@
# Runner components and Pipeline plugin compatibility
## Product boundary
| Product | Implementation | Event entry |
| --- | --- | --- |
| Pipeline | Pipeline stages and an agent-capable Runner | Received messages |
| Agent | A Runner with `spec.usages: [agent]` | Configured events |
| Plugin processor | A Runner with `spec.usages: [event]` | Events declared in `spec.events` |
Runner is the only component for these execution styles. `spec.usages` can contain
both `agent` and `event`; these are selection capabilities, not mutually exclusive
execution modes. An installed component is reusable code. Users create processor
instances, select a Runner and configure it, then bind Bot events to the instance.
Installation alone never subscribes a component to incoming events.
## Legacy EventListener contract
EventListener remains a Pipeline extension. Existing plugins retain their import
paths, handler registration syntax, event classes, Query-based APIs, and Pipeline
plugin selection behavior. No conversion of installed listeners into standalone
processor instances takes place.
Compatibility must cover execution behavior, not just successful deserialization:
| Hook | Required behavior |
| --- | --- |
| PersonMessageReceived / GroupMessageReceived | Read the returned EventContext before later stages; retain message edits and default prevention |
| PromptPreProcessing | Preserve timing and apply returned default_prompt and prompt |
| PersonNormalMessageReceived / GroupNormalMessageReceived | Preserve user_message_alter, default prevention, and replacement replies |
| PersonCommandSent / GroupCommandSent | Preserve command stage timing, default prevention, and replacement replies |
| NormalMessageResponded | Preserve response-stage timing, default prevention, and replacement message chains, including streaming behavior |
| All hooks | Preserve plugin ordering, prevent_postorder across installations, bound-plugin filtering, Query identity, and Workspace scope |
RPC responses are new Python objects. Host code must consume returned values
rather than assume mutations reached the original Query by object identity.
Host-only references, including the active Query and raw adapter message, must
remain available for legacy reply APIs without being exposed in serialized
plugin events.
Preserve source event fields across EBA-to-legacy conversion, including group
member permissions, bot group permissions, and member titles. Missing platform
information must be distinguished from fields that were dropped during conversion.
Direct Agent and Event processor execution must not synthesize Pipeline lifecycle
hooks. Those hooks describe actual Pipeline stages.
## SDK and runtime
`lbp comp Runner` generates `components/runner`. Every component uses
`plugin:author/plugin/name` as its identity. Names are unique within a plugin.
The component can override `async run(ctx)` and yield RunnerResult objects, or
register typed platform callbacks through `@self.handler(EventClass)` in
`initialize()`. Default run dispatches an exact handler, falling back to EBAEvent.
A custom run can delegate to this dispatch with `await super().run(ctx)`.
Both styles share RunnerContext, invocation-bound ctx.api, logs, replies, deadlines,
cancellation, worker isolation and the run ledger. ctx.event is the envelope;
ctx.platform_event is the typed platform payload. Each invocation owns its context;
never put the current context or run ID on a shared component or plugin instance.
The runtime emits completion on normal return unless the Runner already emitted a
terminal result. Exceptions fail the run and retain preceding results. Cancelling
the result stream cancels execution. There is no implicit retry or hidden model
loop. Returned text and logs do not send platform messages: replies are explicit
ctx.reply / ctx.reply_stream actions. Pipeline retains its configured output stage.
`self.plugin` continues to expose ordinary plugin APIs. ctx.api carries run-scoped
resource grants and records tool actions. Workspace and installation authorization
remain Host-enforced. Run identity and API operation scope are separate concepts.
## Selection and observability
Both product selectors discover the same Runner catalog and filter by usage.
Validate usage again before execution. Event-capable Runners must declare events;
users can route a subset, but cannot expand the manifest capability. Unconfigured
instances expose no event subscriptions. Workspace ownership, plugin scope and
instance identity are checked for routing, execution, cancellation and run reads.
Plugin processor details keep event debugging on the left and configuration/logs
on the right. Component settings use the existing schema form. Logs and action
results remain distinct from actual platform delivery; debug delivery is Mock.
Agent-native interactions stay on the Agent product path; typed handlers consume
platform events. Legacy Pipeline lifecycle hooks remain on the Pipeline path.
## Validation
SDK tests cover both execution styles, event matrices, concurrent contexts,
termination, cancellation and permissions. Packaged CLI tests generate, build and
execute the published component. Core tests cover usage-filtered discovery, event
routing, Workspace authorization and real plugin-runtime transport. RunnerDemo
provides multi-step actions, configuration isolation and controlled failures.
@@ -4,17 +4,17 @@ Date: May 10, 2026
Scope:
- `telegram-eba`
- `discord-eba`
- `aiocqhttp-eba`
- `dingtalk-eba`
- `lark-eba`
- `wecom-eba`
- `wecombot-eba`
- `wecomcs-eba`
- `officialaccount-eba`
- `qqofficial-eba`
- `slack-eba`
- `telegram-omni`
- `discord-omni`
- `aiocqhttp-omni`
- `dingtalk-omni`
- `lark-omni`
- `wecom-omni`
- `wecombot-omni`
- `wecomcs-omni`
- `officialaccount-omni`
- `qqofficial-omni`
- `slack-omni`
This report follows `acceptance-checklist.md`. Evidence levels are intentionally strict:
@@ -72,7 +72,7 @@ All four adapters deliver common SDK entities to plugins before LangBot core/plu
| Requirement | Telegram | Discord | aiocqhttp | DingTalk | Lark / Feishu |
|-------------|----------|---------|-----------|----------|---------------|
| `bot_uuid` filled | plugin-e2e | plugin-e2e | plugin-e2e | plugin-e2e | live plugin-e2e pending |
| `adapter_name` filled | `telegram` | `discord` | `aiocqhttp` | `dingtalk` | `lark-eba` in current unit/code; older live text evidence recorded `lark` before the naming fix |
| `adapter_name` filled | `telegram` | `discord` | `aiocqhttp` | `dingtalk` | `lark-omni` in current unit/code; older live text evidence recorded `lark` before the naming fix |
| common `MessageChain` delivered | `Plain`, group `At + Plain`, private `Image`, private `File` | `Source + Plain` | UI `Source + Plain`; protocol `Source + Plain + At + Face + Image + Voice + File + Quote + Plain` | `Source + Plain`, private `Source + Image`, private `Source + File` | live private `Source + Plain`; unit `Source + Plain + At/Image/File`; latest live image/file blocked |
| common user/group entities | plugin-e2e | plugin-e2e | plugin-e2e | plugin-e2e private user; group not completed | live private user; unit private/group |
| raw native object isolation | raw data stays in `source_platform_object` | raw data stays in `source_platform_object` | raw data stays in `source_platform_object` | raw data stays in `source_platform_object` | raw data stays in `source_platform_object` |
@@ -13,10 +13,10 @@ src/langbot/pkg/platform/adapters/aiocqhttp/
├── message_converter.py
├── platform_api.py
├── types.py
└── onebot.svg
└── onebot.png
```
The EBA adapter is registered as `aiocqhttp-eba`. The legacy adapter remains at `src/langbot/pkg/platform/sources/aiocqhttp.py`.
The EBA adapter is registered as `aiocqhttp-omni`. The legacy adapter remains at `src/langbot/pkg/platform/sources/aiocqhttp.py`.
## Configuration
+1 -1
View File
@@ -23,7 +23,7 @@ src/langbot/pkg/platform/adapters/discord/
└── voice.py
```
The adapter is registered as `discord-eba`.
The adapter is registered as `discord-omni`.
## Configuration
+1 -1
View File
@@ -15,7 +15,7 @@ src/langbot/pkg/platform/adapters/kook/
└── types.py
```
The adapter is registered as `kook-eba`.
The adapter is registered as `kook-omni`.
## Configuration
+1 -1
View File
@@ -15,7 +15,7 @@ The Lark/Feishu adapter now has an Event-Based Agents adapter package with:
- `api_impl.py` for common EBA API implementations.
- `platform_api.py` for Feishu-specific `call_platform_api` actions.
The legacy `lark` adapter remains available while the EBA adapter is registered separately as `lark-eba`.
The legacy `lark` adapter remains available while the EBA adapter is registered separately as `lark-omni`.
## Configuration
@@ -2,7 +2,7 @@
Adapter directory: `src/langbot/pkg/platform/adapters/officialaccount/`
Manifest name: `officialaccount-eba`
Manifest name: `officialaccount-omni`
Status: partial migration. Unit/API-shape coverage is present, and private text `plugin-e2e-ui` plus safe API evidence has been verified against the `dev.rockchin.top` Official Account fixture. Proactive outbound `send_message` remains not supported by this adapter because WeChat Official Account replies must be tied to inbound webhook windows.
@@ -70,7 +70,7 @@ Verified UI message: `EBA officialaccount single probe 2026-05-28 16:53`
Observed event/API evidence:
- `MessageReceived`: `bot_uuid=d7c46880-a9f8-431a-9172-5d3e0d663dbc`, `adapter_name=officialaccount-eba`, `chat_type=private`, `chat_id=ovH9L7OW6hNpWZWvp_NMmypVh26w`, `message_chain=[Source, Plain]`.
- `MessageReceived`: `bot_uuid=d7c46880-a9f8-431a-9172-5d3e0d663dbc`, `adapter_name=officialaccount-omni`, `chat_type=private`, `chat_id=ovH9L7OW6hNpWZWvp_NMmypVh26w`, `message_chain=[Source, Plain]`.
- Common safe APIs through probe platform sweep: `get_message`, `get_user_info`, `get_friend_list`.
- Platform APIs through `call_platform_api`: `get_mode`, `get_cached_response_status`.
- `send_message` and outbound component sweep returned explicit `NotSupportedError: send_message:official_account_requires_inbound_webhook_reply`, as expected for this adapter.
@@ -2,7 +2,7 @@
Adapter directory: `src/langbot/pkg/platform/adapters/qqofficial/`
Manifest name: `qqofficial-eba`
Manifest name: `qqofficial-omni`
Status: partial migration. The EBA adapter structure, manifest, converters, cache-backed safe APIs, platform API map, unit tests, and direct live probe scaffold are in place. A real QQ Official WebSocket bot on `dev.rockchin.top` received an inbound user message and drove LangBot into the normal pipeline path; the response path was blocked by the test environment model service returning `model_not_found` for `deepseek-v3`.
@@ -77,7 +77,7 @@ Status: partial migration. The EBA adapter structure, manifest, converters, cach
Test date: 2026-06-02
Endpoint/simulator: `dev.rockchin.top` with a real QQ Official WebSocket bot (`qqofficial-eba`, bot UUID `80a5560b-52b1-40e7-b7d6-4a2341eb4780`) and LangBot running from `/home/wgc/LangBotxg/LangBotEbaTest`.
Endpoint/simulator: `dev.rockchin.top` with a real QQ Official WebSocket bot (`qqofficial-omni`, bot UUID `80a5560b-52b1-40e7-b7d6-4a2341eb4780`) and LangBot running from `/home/wgc/LangBotxg/LangBotEbaTest`.
Observed evidence:
+2 -2
View File
@@ -9,7 +9,7 @@ Slack is migrated into `src/langbot/pkg/platform/adapters/slack/` with the stand
- `message_converter.py` maps common `MessageChain` components to Slack text fallback and maps inbound Slack text/image payloads back to EBA components.
- `api_impl.py` provides cache-backed common read APIs.
- `platform_api.py` declares safe Slack-specific API actions.
- `manifest.yaml` declares `slack-eba`.
- `manifest.yaml` declares `slack-omni`.
The legacy `src/langbot/pkg/platform/sources/slack.py` adapter is kept unchanged.
@@ -71,7 +71,7 @@ Evidence file: `/home/wgc/LangBotxg/LangBotEbaTest/data/temp/slack_eba_plugin_pr
Observed:
- Real Slack private text produced `MessageReceived` with `adapter_name=slack-eba`, `Source + Plain`, private chat type, and filled `bot_uuid`.
- Real Slack private text produced `MessageReceived` with `adapter_name=slack-omni`, `Source + Plain`, private chat type, and filled `bot_uuid`.
- Safe common APIs passed: `get_message`, `get_user_info`, `get_friend_list`.
- Outbound component fallback sweep passed through `send_message`: plain/at/face, image, quote, file, and forward.
- Declared Slack platform APIs passed: `get_mode`, `auth_test`.
+1 -1
View File
@@ -15,7 +15,7 @@ src/langbot/pkg/platform/adapters/telegram/
└── types.py
```
The adapter is registered as `telegram-eba`.
The adapter is registered as `telegram-omni`.
## Configuration
+4 -4
View File
@@ -15,9 +15,9 @@ src/langbot/pkg/platform/adapters/wecom/
└── types.py
```
The adapter is registered as `wecom-eba`.
The adapter is registered as `wecom-omni`.
This record covers the regular WeCom application-message adapter. WeCom AI Bot (`wecombot-eba`) uses a different protocol flow and is documented separately in `wecombot.md`. WeCom Customer Service (`wecomcs`) remains a separate follow-up migration.
This record covers the regular WeCom application-message adapter. WeCom AI Bot (`wecombot-omni`) uses a different protocol flow and is documented separately in `wecombot.md`. WeCom Customer Service (`wecomcs`) remains a separate follow-up migration.
## Configuration
@@ -102,13 +102,13 @@ uv --project /absolute/path/to/langbot-plugin-sdk run python -m langbot_plugin.c
Evidence:
- JSONL: `data/temp/wecom_eba_plugin_probe.jsonl`
- Bot: `wecom-eba`
- Bot: `wecom-omni`
- Client: real WeCom desktop client
- Environment: `dev.rockchin.top` test server
Observed and verified:
- A real private WeCom user message reached the plugin as `MessageReceived` with `adapter_name=wecom-eba`, common sender/chat fields, and `Source + Plain`.
- A real private WeCom user message reached the plugin as `MessageReceived` with `adapter_name=wecom-omni`, common sender/chat fields, and `Source + Plain`.
- SDK API calls succeeded through the standalone runtime, including `get_langbot_version`, `get_bots`, `get_bot_info`, `send_message`, plugin/workspace storage, and manifest/list APIs.
- Safe adapter API checks succeeded through the plugin path for cached message/user data and declared safe platform API actions.
+4 -4
View File
@@ -15,9 +15,9 @@ src/langbot/pkg/platform/adapters/wecombot/
└── types.py
```
The adapter is registered as `wecombot-eba`.
The adapter is registered as `wecombot-omni`.
This is separate from regular WeCom internal applications (`wecom-eba`). WeComBot supports WebSocket long connection mode, which does not require a webhook URL. Webhook mode remains available when `enable-webhook=true`.
This is separate from regular WeCom internal applications (`wecom-omni`). WeComBot supports WebSocket long connection mode, which does not require a webhook URL. Webhook mode remains available when `enable-webhook=true`.
## Configuration
@@ -118,13 +118,13 @@ Evidence:
- JSONL: `data/temp/wecombot_eba_plugin_probe.jsonl`
- Bot UUID: `9f5d4125-7b6d-4c98-8ca2-111111111111`
- Adapter: `wecombot-eba`
- Adapter: `wecombot-omni`
- Client: real WeCom desktop client, private `LangBot` BOT chat
- Mode: WebSocket long connection (`enable-webhook=false`)
Observed and verified:
- A real user-side message reached the plugin as `MessageReceived` with `adapter_name=wecombot-eba`, common sender/chat fields, and `Source + Plain`.
- A real user-side message reached the plugin as `MessageReceived` with `adapter_name=wecombot-omni`, common sender/chat fields, and `Source + Plain`.
- SDK API calls succeeded through the standalone runtime: `get_langbot_version`, `get_bots`, `get_bot_info`, `send_message`, plugin/workspace storage, manifest/list APIs, and safe cached common platform APIs.
- Outbound component sweep was visible in the WeCom client and returned `errcode=0`: plain/mention/face fallback, base64 image marker, quote fallback, file marker, and flattened forward fallback.
- Declared WeComBot platform APIs succeeded through `plugin.call_platform_api`: `is_websocket_mode`, `get_stream_session_status`, and `send_markdown`.
+2 -2
View File
@@ -15,7 +15,7 @@ src/langbot/pkg/platform/adapters/wecomcs/
└── types.py
```
The adapter is registered as `wecomcs-eba`. It is separate from regular WeCom application messages (`wecom-eba`) and WeCom AI Bot (`wecombot-eba`).
The adapter is registered as `wecomcs-omni`. It is separate from regular WeCom application messages (`wecom-omni`) and WeCom AI Bot (`wecombot-omni`).
## Configuration
@@ -120,7 +120,7 @@ Evidence:
- Server JSONL: `/home/wgc/LangBotxg/LangBotEbaTest/data/temp/wecomcs_eba_plugin_probe.jsonl`
- Trigger text: `EBA wecomcs dedupe probe 2026-05-27`
- `bot_uuid`: `cc810d2c-91f3-4f92-8f27-e1bf9f7b6cb4`
- `adapter_name`: `wecomcs-eba`
- `adapter_name`: `wecomcs-omni`
- Observed common event: `MessageReceived`, `event.type=message.received`
- Observed message chain: `Source + Plain`
- Observed chat: `chat_type=private`, `chat_id=external_userid|open_kfid`
@@ -4,6 +4,8 @@
更新日期:2026-07-29
> 2026-09-05 文档核对:下列版本、测试数量和镜像 pin 属于 7 月交付记录,不代表当前 `dev/4.11.x` 的依赖状态。当前 Core/SDK 配套与定向测试见 [STATUS.md](../agent-runner-pluginization/STATUS.md)。本次未重新验收闭源控制面、最终部署或 24 小时 soak,因此保留本页生产激活门禁;新增 Runtime 存储统计也不关闭 B-02 硬配额要求。
本文是 Cloud v2 首期上线前的剩余验证清单。它只记录尚不能由当前代码审查、
单元测试、集成测试、合成容量探针或短时 Linux 容器实验替代的证据。这里的项目
不属于 2026-07-29 代码与本地测试资源审查的完成条件,也不会让该审查持续保持未完成;
@@ -1,5 +1,7 @@
# Multi-tenant implementation checklist
> Status interpretation, 2026-09-05: this checklist retains the original multi-tenant implementation and activation evidence. The kernel is now also present on `dev/4.11.x`; the branch names below are historical workstream labels. SDK installation-worker restart coordination is implemented in `runtime/plugin/restart_coordinator.py`, with focused tests passing on the inspected workspace. An unchecked production gate does not necessarily mean its code is absent. Production fault injection, egress, hard storage quotas, and generation-aware business transaction/outbox gates remain separate. See [current 4.11 status](../agent-runner-pluginization/STATUS.md) and [Cloud activation gates](./cloud-v2-pending-verification.md).
This checklist turns the Workspace architecture into implementation and
verification gates. Exact commands and observed results are recorded in the
[verification report](./verification-report.md).
+7 -7
View File
@@ -13,7 +13,7 @@
┌──────────────────────────────────────────────────────────────────┐
│ LangBot 主进程 │
│ │
AgentRunner ──> SDK call_tool / scoped MCP bridge │
│ Runner ──> SDK call_tool / scoped MCP bridge │
│ │ │ │
│ └────────────────> ToolManager ──> NativeToolLoader │
│ │ │ │ │
@@ -87,8 +87,8 @@
**核心设计原则**:
- Box Runtime 作为独立进程运行,通过 Action RPC 与 LangBot 主进程通信,两者复用 SDK 的 IO 层(Handler → Connection → Controller
- 一个 session_id 对应一个容器/沙箱实例。同一 session 内可并存多条 mount 与多个 managed process
- AgentRunner 无权指定 session scope。SDK/Python `call_tool` 与 scoped MCP bridge 都发出同一个 `PluginToRuntimeAction.CALL_TOOL`,最终由 Host 的 ToolManager 执行,并使用当前 run 保存的同一个 execution Query
- Box 内托管的 stdio MCP server 使用独立的长期 `mcp-shared` session;它不是 AgentRunner 本次事件的 sandbox session(详见 [box-session-scope.md](./box-session-scope.md)
- Runner 无权指定 session scope。SDK/Python `call_tool` 与 scoped MCP bridge 都发出同一个 `PluginToRuntimeAction.CALL_TOOL`,最终由 Host 的 ToolManager 执行,并使用当前 run 保存的同一个 execution Query
- Box 内托管的 stdio MCP server 使用独立的长期 `mcp-shared` session;它不是 Runner 本次事件的 sandbox session(详见 [box-session-scope.md](./box-session-scope.md)
---
@@ -140,7 +140,7 @@ BoxService
**输出截断**: 默认 4000 字符上限,保留前 60% + 后 40%,中间插入 `[...truncated...]`
**Session 所有权**: `resolve_box_session_id(query)` 只接受 Host 已确定的私有 scope 或 Query launcher/session identity,并输出 `lb-box-` + 64 位小写 SHA-256 十六进制摘要(固定 71 个 ASCII 字符)。哈希输入是 canonical JSON,包含 instance、workspace、bot、platform adapter、target type/id 与 thread;原始用户、群组、conversation 或 event id 不会出现在 Box session id 中。相同 Host scope 稳定复用,不同 target/thread/workspace/bot/adapter/instance 相互隔离;缺少可用 identity 时 fail closed。Pipeline、Agent 或 AgentRunner 配置都不能覆盖该规则。
**Session 所有权**: `resolve_box_session_id(query)` 只接受 Host 已确定的私有 scope 或 Query launcher/session identity,并输出 `lb-box-` + 64 位小写 SHA-256 十六进制摘要(固定 71 个 ASCII 字符)。哈希输入是 canonical JSON,包含 instance、workspace、bot、platform adapter、target type/id 与 thread;原始用户、群组、conversation 或 event id 不会出现在 Box session id 中。相同 Host scope 稳定复用,不同 target/thread/workspace/bot/adapter/instance 相互隔离;缺少可用 identity 时 fail closed。Pipeline、Agent 或 Runner 配置都不能覆盖该规则。
**Skill 挂载合并**: `execute_tool()` 调用时,`build_skill_extra_mounts(query)` 会把当前 pipeline-bound 的所有 skill 的 `package_root` 作为 `extra_mounts` 加入 BoxSpec,挂在 `/workspace/.skills/<name>`。LLM 通过 `activate` 工具显式激活某个 skill 后,工具调用才允许引用这个 skill 的虚拟路径。
@@ -421,7 +421,7 @@ ToolManager.initialize()
3. 若 skill 是 Python 项目(有 `requirements.txt``pyproject.toml`),命令会被 venv bootstrap 包裹(在 skill 挂载点内创建 `.venv`
4. 调用 `box_service.execute_tool()` → 走 Host 从当前事件生成的 session_id 与已组装好的 `extra_mounts`**不再为每 skill 起独立 session**
AgentRunner 可以直接通过 SDK/Python `AgentRunAPIProxy.call_tool` 调用这些工具,也可以让外部 harness 通过 SDK-owned scoped MCP bridge 回调。两条入口都发送 `PluginToRuntimeAction.CALL_TOOL`,共享同一个 run authorization、Host session 中保存的 execution Query、ToolManager 与 `resolve_box_session_id(query)` 规则;Runner 不能提交自定义 Box session id。Pipeline run 保存原 Query;纯 EBA run 由 Host 构造 `pipeline_config=None``pipeline_uuid=None` 的最小 Query。
Runner 可以直接通过 SDK/Python `RunnerAPIProxy.call_tool` 调用这些工具,也可以让外部 harness 通过 SDK-owned scoped MCP bridge 回调。两条入口都发送 `PluginToRuntimeAction.CALL_TOOL`,共享同一个 run authorization、Host session 中保存的 execution Query、ToolManager 与 `resolve_box_session_id(query)` 规则;Runner 不能提交自定义 Box session id。Pipeline run 保存原 Query;纯 EBA run 由 Host 构造 `pipeline_config=None``pipeline_uuid=None` 的最小 Query。
### 4.3 MCP-in-Box (`mcp_stdio.py`, 354 行)
@@ -442,7 +442,7 @@ initialize()
每条 MCP server 是同一 session 中的一个 managed process,独立的 `process_id`、独立 attach URL,互不阻塞。
这里的 `mcp-shared` 只承载 LangBot 管理的 stdio MCP server 进程。AgentRunner 的 scoped MCP bridge 是回调 Host 工具的协议入口,不会把事件运行的 exec/read/write 改到 `mcp-shared`
这里的 `mcp-shared` 只承载 LangBot 管理的 stdio MCP server 进程。Runner 的 scoped MCP bridge 是回调 Host 工具的协议入口,不会把事件运行的 exec/read/write 改到 `mcp-shared`
---
@@ -582,7 +582,7 @@ volumes:
### Session scope
Pipeline 与 AgentRunner 配置不再暴露 sandbox session 模板。Host 将当前平台会话/事件 scope 规范化后哈希成固定长度的 `lb-box-<sha256>`;相同 scope 稳定复用,不同 scope 隔离,缺少 identity 时拒绝执行。SDK/Python 与 scoped MCP bridge 的工具调用遵守同一规则。详见 [box-session-scope.md](./box-session-scope.md)。
Pipeline 与 Runner 配置不再暴露 sandbox session 模板。Host 将当前平台会话/事件 scope 规范化后哈希成固定长度的 `lb-box-<sha256>`;相同 scope 稳定复用,不同 scope 隔离,缺少 identity 时拒绝执行。SDK/Python 与 scoped MCP bridge 的工具调用遵守同一规则。详见 [box-session-scope.md](./box-session-scope.md)。
### REST API
+11 -11
View File
@@ -7,7 +7,7 @@
## 1. Decision
The LangBot Host owns the Box session used by an event run. A Pipeline, Agent,
or AgentRunner cannot choose a global, per-user, per-conversation, or per-query
or Runner cannot choose a global, per-user, per-conversation, or per-query
sandbox mode.
`BoxService.resolve_box_session_id(query)` always returns this shape:
@@ -81,7 +81,7 @@ Host scope or launcher/session identity is also rejected. There is no
## 3. Host execution Query
AgentRunner callbacks need a Host-owned Query view because model/tool loaders
Runner callbacks need a Host-owned Query view because model/tool loaders
already consume that type. The Query is internal and is never exposed as a
Runner-controlled object.
@@ -96,11 +96,11 @@ Runner-controlled object.
This gives Pipeline and pure EBA execution the same Host tool path without
inventing a fake Pipeline for an independent Agent.
## 4. AgentRunner callback paths
## 4. Runner callback paths
AgentRunner implementations may use either callback transport:
Runner implementations may use either callback transport:
1. SDK/Python runners call `AgentRunAPIProxy.call_tool`.
1. SDK/Python runners call `RunnerAPIProxy.call_tool`.
2. External harnesses call the SDK-owned scoped MCP bridge.
Both transports emit the same `PluginToRuntimeAction.CALL_TOOL`. The Host then
@@ -108,8 +108,8 @@ validates the same run authorization, restores the same execution Query, and
dispatches to the same ToolManager and BoxService.
```text
AgentRunner
+-- AgentRunAPIProxy.call_tool --------+
Runner
+-- RunnerAPIProxy.call_tool --------+
| |
+-- SDK-owned scoped MCP bridge -------+--> PluginToRuntimeAction.CALL_TOOL
--> run authorization
@@ -119,7 +119,7 @@ AgentRunner
--> lb-box-<sha256>
```
An AgentRunner is not required to use MCP. Local Python runners can use the SDK
An Runner is not required to use MCP. Local Python runners can use the SDK
directly; code-agent harnesses can use the bridge. The transports do not define
different authorization or sandbox semantics.
@@ -143,16 +143,16 @@ This is separate from the scoped MCP bridge above:
| Path | Purpose | Session rule |
| --- | --- | --- |
| AgentRunner scoped MCP bridge | Call authorized Host tools for one active run | Host-owned `lb-box-<sha256>` from the run execution Query |
| Runner scoped MCP bridge | Call authorized Host tools for one active run | Host-owned `lb-box-<sha256>` from the run execution Query |
| MCP-in-Box stdio server | Keep configured MCP server processes running | Dedicated persistent `mcp-shared` session |
Calling a sandbox tool through the AgentRunner bridge never redirects the run
Calling a sandbox tool through the Runner bridge never redirects the run
workspace into `mcp-shared`. Conversely, an MCP server's managed-process
lifecycle does not inherit the current event scope.
## 7. Configuration and compatibility
There is no Box session scope field in Pipeline metadata, AgentRunner config,
There is no Box session scope field in Pipeline metadata, Runner config,
or the public Pipeline/Runner API. Operators configure the Box subsystem itself
(`box.enabled`, backend/runtime settings, profiles, mount allowlists, quotas,
and workspace roots), not per-Runner session templates.
+1 -1
View File
@@ -62,7 +62,7 @@
| Backend selection | 良好 | 显式 backend 优先级、local 探测顺序、配置变更触发 reselect |
| MCP Box 集成 | 良好 | config model、路径重写、payload、shared-session 多 process |
| Native tool loader | 良好 | 6 工具(exec/read/write/edit/glob/grep)、路径穿越拦截 |
| AgentRunner 工具入口 | 良好 | SDK proxy 与 MCP bridge 都映射到 `PluginToRuntimeAction.CALL_TOOL`Host action 测试覆盖 run-scoped execution Query 与纯 EBA native exec |
| Runner 工具入口 | 良好 | SDK proxy 与 MCP bridge 都映射到 `PluginToRuntimeAction.CALL_TOOL`Host action 测试覆盖 run-scoped execution Query 与纯 EBA native exec |
| Skill 系统 | 良好 | 加载、Tool Call 激活、marker、路径解析、authoring CRUD、HTTP service |
---
+6 -6
View File
@@ -3,24 +3,24 @@
> 更新日期: 2026-06-29
> 分支: `mcp_resources`
> PR: langbot-app/LangBot#2215
> 主题: MCP Resources 在 LangBot 中的产品价值、AgentRunner 集成方式与后续架构方向
> 主题: MCP Resources 在 LangBot 中的产品价值、Runner 集成方式与后续架构方向
## 结论
PR #2215 对 LangBot 有明确价值:它补齐了 MCP 协议中 Resources 这一重要能力,让 MCP server 不再只暴露 tools,也可以暴露文档、代码片段、配置、日志、图片等上下文资源。管理端可以发现和预览资源,Agent 也可以通过当前实现按需列出和读取资源。
但当前 AgentRunner 层的接入方式更接近一个可用的第一阶段方案,而不是最终架构。现在 MCP Resources 被包装成两个 synthetic tools
但当前 Runner 层的接入方式更接近一个可用的第一阶段方案,而不是最终架构。现在 MCP Resources 被包装成两个 synthetic tools
- `langbot_mcp_list_resources`
- `langbot_mcp_read_resource`
这让模型可以通过 function calling 主动探索资源,落地成本低,也复用了已有 `ToolManager` / `LocalAgentRunner` 的工具调用链路。不过从 MCP 规范和主流实现来看,Resources 更适合作为一种一等上下文来源,而不是长期隐藏在工具列表里。
这让模型可以通过 function calling 主动探索资源,落地成本低,也复用了已有 `ToolManager` / `LocalRunner` 的工具调用链路。不过从 MCP 规范和主流实现来看,Resources 更适合作为一种一等上下文来源,而不是长期隐藏在工具列表里。
建议保留当前 synthetic tools 作为探索能力,同时把后续主线设计调整为:MCP Resources 是 pipeline / conversation / message 级别可选择、可固定、可审计的上下文输入。
## 当前实现判断
当前 AgentRunner 集成路径如下:
当前 Runner 集成路径如下:
```text
Pipeline 绑定 MCP server
@@ -28,7 +28,7 @@ Pipeline 绑定 MCP server
-> Preproc 为 local-agent 加载工具
-> ToolManager.get_all_tools()
-> MCPLoader 注入 synthetic resource tools
-> LocalAgentRunner 将工具 schema 传给模型
-> LocalRunner 将工具 schema 传给模型
-> 模型发起 list/read tool call
-> ToolManager.execute_func_call()
-> MCPLoader 调 MCP session.list_resources/read_resource
@@ -178,7 +178,7 @@ LangBot 后续应支持模板发现、参数填写、实例化和绑定。否则
- 支持 resource templates。
- 支持资源订阅更新。
- 支持 chunk、summary、RAG 化接入。
- 为 DifyAgentRunner、LocalAgentRunner 等不同 runner 定义统一资源上下文接口。
- 为 DifyRunner、LocalRunner 等不同 runner 定义统一资源上下文接口。
## 最终建议
@@ -68,7 +68,7 @@ Anthropic、Google 和 LiteLLM 的官方文档域名在本次环境中被浏览
- `LLMModel.extra_args` 是 JSON 字段,Web 端已有通用高级参数编辑器。
- `LiteLLMRequester` 会按“模型级 `extra_args`,再调用级 `extra_args`”的顺序合并参数。
- LiteLLM 已统一处理多个 Provider 的 `reasoning_effort``thinking` 和返回的 `reasoning_content`
- `LocalAgentRunner` 的非流式、流式、工具调用和 fallback 路径都经过 `RuntimeProvider.invoke_llm*()`
- `LocalRunner` 的非流式、流式、工具调用和 fallback 路径都经过 `RuntimeProvider.invoke_llm*()`
- `remove-think` 已能控制 `<think>` 或独立 reasoning 内容是否进入展示文本。
- Gemini 工具调用所需的 `provider_specific_fields` / thought signature 已有保留逻辑和单元测试。
@@ -0,0 +1,63 @@
# Legacy and Omni adapter mainline audit
Audit date: 2026-09-10.
- Destination: `dev/4.11.x`, starting at `5ba25c39af85844283f7049beddc54ea71afea76`.
- Upstream: refreshed `origin/master`, `ce6b647fe7e620031c3dd9d16a8e8c08ba13b4b1`.
- Scope: adapter changes since the first Omni fork in March 2026, including shared platform clients, configuration manifests, and shutdown/resource handling.
- Existing local changes, including the Omni identifier rename, are preserved.
The two implementations do not inherit behavior automatically. A commit in `sources/` does not update `adapters/`. The audit compares the current legacy source with master, then checks its actual equivalent in the Omni converter, transport, client, configuration, and lifecycle code. Native interaction handling continues to use `interaction.request` and typed callbacks; old Dify runner-private state is not copied into the new event architecture.
## Platforms with an Omni adapter
| Platform | Relevant mainline work | Legacy result | Omni result |
| --- | --- | --- | --- |
| OneBot / aiocqhttp | JSON cards (`c75890874`, `0963fd544`), base64 (`ccc51522c`), group metadata (`cc7a13158`), listener lifecycle (`2c3e52c16`), bounded lookups (`e1ac5e0fc`), original image URLs (`463b12092`) | Already present | Added JSON-card parsing and normalization for image, voice, and file payloads. Reused the cached, timeout-bounded group/member lookup per adapter instance. Preserved native handler registration, typed events, and original image URLs. |
| Telegram | Streaming (`0755beebc`), bounded media/state (`e1ac5e0fc`), token-bearing image URLs (`9df021eb8`) | Already present | Shares the legacy message converter; ported persistent-message streaming, throttling/size fallback, bounded state, and shutdown. Token-bearing download URLs remain absent from the public Image component. |
| Discord | Streaming (`0755beebc`), media/resource limits (`e1ac5e0fc`), original image URLs (`463b12092`) | Already present | Added snapshot-based send/edit streaming with bounded state. Enforced the mainline media limit on Omni image, voice, and file loading. Incoming public attachment URLs are preserved. |
| Feishu / Lark | Files (`e06fac2bb`), proactive sends (`3680a8024`), service domain (`c7cb42bd7`), tables (`1d15798e5`), nonblocking connection discovery (`48952206d`), resource bounds (`e1ac5e0fc`), final duplicate text (`5ca30133a`), feedback association (`f8010a20e`) | Applied final-text fix; retained the newer SDK cache-task cleanup | Added identical domestic/international/custom domain fields and applied the selected domain to both HTTP and WebSocket clients. Reused nonblocking connection discovery and bounded upload helpers; bounded incoming downloads and callback tasks. Added table cards and feedback-to-monitoring association. The final card contains one text element and exits streaming mode; it does not use the legacy duplicate-placeholder layout. |
| DingTalk | Voice recognition (`de4d14fee`), files (`e06fac2bb`), interactive cards (`0755beebc`), task/card bounds (`e1ac5e0fc`), automatic layout (`8cf001550`) | Applied layout fix in the shared client | Shared client fix reaches both `create_and_card` and native interaction cards. Retained typed file/voice conversion and native callbacks, bounded normal stream cards, cleared callback/card state on shutdown, and exposed the template download. |
| QQ Official | Optional token (`cb45807b1`), Markdown (`c87548c0b`), complete stream snapshots (`79634772d`), resource bounds (`e1ac5e0fc`), original image URLs (`463b12092`) | Applied optional-token fix | Removed the token requirement in constructor and manifest. Added Markdown configuration and sending. Stream and non-stream fallback replace full snapshots instead of appending deltas. Restored original image URLs, bounded state, and closed the owned client on shutdown. |
| WeCom application | Media ID key (`d942bfe19`), original image URLs (`463b12092`), client lifetime (`e1ac5e0fc`) | Applied the `media_id` send fix | Media dispatch already used `media_id`; restored original image URLs and client shutdown. |
| WeCom AI Bot | WS/files/feedback (`d9378c3a8`, `14b1e0d33`, `c7efa4dd7`, `83ccb33fd`), sandbox media (`e934f08ad`), WS loading (`e69a80f5e`) | Already present | Outbound conversion now retains media items. Replies and final stream chunks use the shared upload/reply implementation instead of `[Image]` / `[File]` placeholders. Existing native inbound file/voice/quote handling, feedback, and shared WS fixes remain active. |
| WeCom customer service | Proactive text (`13dba887d`), `open_kfid` and unique outbound `msgid` (`cabde423a`), client limits (`e1ac5e0fc`) | Already present | Existing target parsing and generated outbound message IDs already match. Added owned-client shutdown and cache clearing; removed a shadowed obsolete image-send definition, retaining the bounded active implementation. |
| Slack | Original image URLs (`463b12092`), shared HTTP session (`e1ac5e0fc`) | Already present | Restored original image URLs alongside downloaded data. Existing shared-client behavior remains active; bounded retained typed-event caches. |
| KOOK | Bounded gateway decompression, HTTP responses, and shared sessions (`e1ac5e0fc`) | Already present | Reuses the bounded gateway decoder off the event loop and limited JSON response reader. Existing URL-based image conversion does not eagerly download media. |
| WeChat Official Account | Resource/state cleanup (`e1ac5e0fc`) and outbound IP display (`bca710dbd`) | Already present | Calls the shared client's cleanup on shutdown, bounds typed-event caches, and exposes the outbound IP configuration field. |
All 12 Omni manifests include the corresponding legacy configuration field names and current deployment help links. The Feishu/Lark `domain` and `custom_domain` fields are structurally identical, including labels, descriptions, options, defaults, and conditional display. Display names are unchanged.
## Platforms with no separate Omni implementation
| Platform | Result |
| --- | --- |
| Matrix | Applied `fc1c99843` to fix the unbound logout command during re-login. Existing connection/media/resource fixes already match master. |
| Mattermost | Added the complete mainline adapter, manifest, icon, and regression tests from `d6443b10b`. |
| LINE | Stable source-based sessions (`777fe1f20`), mention conversion (`855ae2bdb`), and resource limits already match master. |
| OpenClaw Weixin | Mainline adapter, outbound IP display, and resource/lifecycle changes already present. |
| WeChatPad | Mainline media/download and resource/lifecycle changes already present. |
| Satori | Mainline lifecycle/resource changes already present. |
| HTTP Bot | Mainline standalone HTTP integration and workspace/resource changes already present. |
| WebPage Bot | Delegated stream helpers and embedded-session behavior already match master. |
| WebSocket / browser debug | Mainline session isolation, scoped history, resource bounds, and routing work already present. Retained 4.11's image-reference retention and pipeline metadata additions. |
| Archived Gewechat, Nakuru, QQBotPy sources | Mainline source changes already present; no separate Omni counterpart exists. |
Documentation-link updates `1cfe87186` and `ec63978ec` were also applied to legacy manifests and matching Omni manifests.
## Intentional architectural differences
- New typed events and platform APIs remain in the Omni modules. Legacy Pipeline message/context conversion stays in the established compatibility path.
- Native interaction delivery uses the generic interaction request/callback protocol. The current 4.11 pipeline/provider code no longer emits `_form_data` or `_resume_from_form`; copying old Dify callback internals would reconnect the wrong execution model.
- This audit establishes parity for the mainline changes in scope, not a claim that every pre-fork legacy-only extension has a typed Omni API. For example, Discord voice-channel management still belongs to the legacy source; sending voice attachments and the post-fork streaming/resource fixes are covered here.
- Shared platform clients retain 4.11-specific return values and additional APIs instead of being overwritten with entire master files.
## Verification
- Full backend unit suite: **3751 passed, 1 skipped**. This includes platform transport tests, Pipeline/event routing, service validation, plugin actions, workspace isolation, and resource handling.
- Dedicated mainline-parity regression suite: **62 passed**. Coverage for configuration coverage, OneBot JSON/base64/metadata, QQ Markdown/snapshots, WeCom media replies, bounded media/gateway reads, lifecycle cleanup, streaming, and Feishu/Lark region selection.
- Ruff formatting and checks pass on the affected Python files.
- Restarted the local backend on port 5399; `/healthz` returns success and reports the plugin runtime connected. All 12 Omni icon endpoints and the new Mattermost icon return HTTP 200.
- Vendor requests in automated tests are mocked. Real delivery to every external platform has not been tested. The instance contains pre-existing incomplete Feishu and WeCom AI Bot configurations; the same missing-credential errors were present before this audit.
Tests: `tests/unit_tests/platform/test_omni_mainline_parity.py`, the platform-specific test files, and the imported Mattermost/WeCom/Lark/DingTalk regressions.
+5 -1
View File
@@ -69,7 +69,7 @@ dependencies = [
"langchain-text-splitters>=1.1.2",
"chromadb>=1.0.0,<2.0.0",
"qdrant-client (>=1.15.1,<2.0.0)",
"langbot-plugin==0.5.3",
"langbot-plugin==0.5.5",
"asyncpg>=0.30.0",
"line-bot-sdk>=3.19.0",
"matrix-nio>=0.25.2",
@@ -229,3 +229,7 @@ skip-magic-trailing-comma = false
# Like Black, automatically detect the appropriate line ending.
line-ending = "auto"
[tool.uv.sources]
# Development contract: update to the matching SDK release before publishing.
langbot-plugin = { git = "https://github.com/langbot-app/langbot-plugin-sdk", rev = "92a9e03fa9c791f4ed30cc3f5f0602c13b800d28" }
@@ -99,7 +99,7 @@ LangBot 是异步且集成度高的系统,有些问题不会直接表现为页
```text
Action list_plugins call timed out
Action list_agent_runners call timed out
Action list_runners call timed out
Action invoke_llm_stream call timed out
```
+3 -3
View File
@@ -61,7 +61,7 @@ bin/lbs fixture check
```
`env doctor` 会检查 URL、路径、代理变量等。代理变量是可选项;只有大小写代理变量互相冲突时才会报错。失败不一定代表仓库坏了,通常说明本地 LangBot 没启动、代理不一致或浏览器 profile 不存在。
`fixture check` 会检查仓库内测试 fixture 是否存在,例如 MCP stdio server、RAG 文档、多模态图片、qa-plugin-smoke 包和 QA AgentRunner 包。它也会校验 `.lbpkg` 是 zip 包,并检查 QA AgentRunner fixture 的入口文件未漂移。
`fixture check` 会检查仓库内测试 fixture 是否存在,例如 MCP stdio server、RAG 文档、多模态图片、qa-plugin-smoke 包和 QA Runner 包。它也会校验 `.lbpkg` 是 zip 包,并检查 QA Runner fixture 的入口文件未漂移。
4. 查看已有测试 case
@@ -344,8 +344,8 @@ npx playwright install chromium
脚本会尝试通过 `LANGBOT_PIPELINE_NAME` 从 Pipelines 页面进入目标 pipeline。两者都没有时,
该自动化会返回 `blocked`,不会伪造通过。
Runner 专用 case 不应复用通用 pipeline 变量。Local Agent、Codex AgentRunner 和
Claude Code AgentRunner 这类 case 会通过 `automation_pipeline_url_env` /
Runner 专用 case 不应复用通用 pipeline 变量。Local Agent、Codex Runner 和
Claude Code Runner 这类 case 会通过 `automation_pipeline_url_env` /
`automation_pipeline_name_env` 映射到 case-specific env,例如
`LANGBOT_LOCAL_AGENT_PIPELINE_URL`。这些 case 如果缺少专用变量,会返回 `blocked`
不会退回到 `LANGBOT_PIPELINE_URL`,避免跑错 pipeline 后产生假阳性。
+195 -164
View File
@@ -1,5 +1,5 @@
#!/usr/bin/env python3
"""Audit one persisted AgentRunner run without exposing authorization secrets."""
"""Audit one persisted Runner run without exposing authorization secrets."""
from __future__ import annotations
@@ -25,29 +25,29 @@ from agent_run_ledger_policy import (
def database_url(repo: pathlib.Path) -> str:
config = yaml.safe_load((repo / "data/config.yaml").read_text(encoding="utf-8")) or {}
database = config.get("database", {})
kind = database.get("use", "sqlite")
if kind == "sqlite":
path = pathlib.Path(database.get("sqlite", {}).get("path", "data/langbot.db"))
config = yaml.safe_load((repo / 'data/config.yaml').read_text(encoding='utf-8')) or {}
database = config.get('database', {})
kind = database.get('use', 'sqlite')
if kind == 'sqlite':
path = pathlib.Path(database.get('sqlite', {}).get('path', 'data/langbot.db'))
if not path.is_absolute():
path = repo / path
return f"sqlite+aiosqlite:///{path}"
if kind in {"postgres", "postgresql"}:
values = database.get("postgresql", {})
user = urllib.parse.quote_plus(str(values.get("user", "postgres")))
password = urllib.parse.quote_plus(str(values.get("password", "postgres")))
host = values.get("host", "127.0.0.1")
port = values.get("port", 5432)
name = values.get("database", "postgres")
return f"postgresql+asyncpg://{user}:{password}@{host}:{port}/{name}"
raise RuntimeError(f"Unsupported database backend: {kind}")
return f'sqlite+aiosqlite:///{path}'
if kind in {'postgres', 'postgresql'}:
values = database.get('postgresql', {})
user = urllib.parse.quote_plus(str(values.get('user', 'postgres')))
password = urllib.parse.quote_plus(str(values.get('password', 'postgres')))
host = values.get('host', '127.0.0.1')
port = values.get('port', 5432)
name = values.get('database', 'postgres')
return f'postgresql+asyncpg://{user}:{password}@{host}:{port}/{name}'
raise RuntimeError(f'Unsupported database backend: {kind}')
def parse_created_after(value: str | None) -> datetime.datetime | None:
if not value:
return None
parsed = datetime.datetime.fromisoformat(value.replace("Z", "+00:00"))
parsed = datetime.datetime.fromisoformat(value.replace('Z', '+00:00'))
if parsed.tzinfo is not None:
parsed = parsed.astimezone(datetime.timezone.utc).replace(tzinfo=None)
return parsed
@@ -55,19 +55,19 @@ def parse_created_after(value: str | None) -> datetime.datetime | None:
def event_matches_tool_call(data_json: str | None, tool_name: str, parameters: dict | None) -> bool:
try:
data = json.loads(data_json or "{}")
data = json.loads(data_json or '{}')
except (TypeError, ValueError):
return False
if not isinstance(data, dict) or data.get("tool_name") != tool_name:
if not isinstance(data, dict) or data.get('tool_name') != tool_name:
return False
return parameters is None or data.get("parameters") == parameters
return parameters is None or data.get('parameters') == parameters
def collect_result_texts(value: object) -> list[str]:
texts: list[str] = []
if isinstance(value, dict):
for key, item in value.items():
if key == "text" and isinstance(item, str):
if key == 'text' and isinstance(item, str):
texts.append(item)
else:
texts.extend(collect_result_texts(item))
@@ -85,7 +85,7 @@ async def audit(
expected_tool_name: str | None = None,
expected_parameters: dict | None = None,
expected_result_text: str | None = None,
tool_authorization_mode: str = "strict",
tool_authorization_mode: str = 'strict',
) -> dict:
engine = create_async_engine(database_url(repo))
failures: list[dict] = []
@@ -93,82 +93,106 @@ async def audit(
try:
async with engine.connect() as connection:
if run_id:
run_row = (await connection.execute(
sqlalchemy.text("SELECT * FROM agent_run WHERE run_id = :run_id"),
{"run_id": run_id},
)).mappings().first()
run_row = (
(
await connection.execute(
sqlalchemy.text('SELECT * FROM agent_run WHERE run_id = :run_id'),
{'run_id': run_id},
)
)
.mappings()
.first()
)
elif expected_tool_name:
query = "SELECT * FROM agent_run"
query = 'SELECT * FROM agent_run'
params = {}
if created_after is not None:
query += " WHERE created_at >= :created_after"
params["created_after"] = created_after
query += " ORDER BY id DESC LIMIT 100"
query += ' WHERE created_at >= :created_after'
params['created_after'] = created_after
query += ' ORDER BY id DESC LIMIT 100'
candidates = (await connection.execute(sqlalchemy.text(query), params)).mappings().all()
run_row = None
for candidate in candidates:
started_rows = (await connection.execute(
sqlalchemy.text(
"SELECT data_json FROM agent_run_event "
"WHERE run_id = :run_id AND type = 'tool.call.started' ORDER BY sequence"
),
{"run_id": str(candidate["run_id"])},
)).mappings().all()
started_rows = (
(
await connection.execute(
sqlalchemy.text(
'SELECT data_json FROM agent_run_event '
"WHERE run_id = :run_id AND type = 'tool.call.started' ORDER BY sequence"
),
{'run_id': str(candidate['run_id'])},
)
)
.mappings()
.all()
)
if any(
event_matches_tool_call(row.get("data_json"), expected_tool_name, expected_parameters)
event_matches_tool_call(row.get('data_json'), expected_tool_name, expected_parameters)
for row in started_rows
):
run_row = candidate
break
else:
run_row = (await connection.execute(
sqlalchemy.text("SELECT * FROM agent_run ORDER BY id DESC LIMIT 1")
)).mappings().first()
run_row = (
(await connection.execute(sqlalchemy.text('SELECT * FROM agent_run ORDER BY id DESC LIMIT 1')))
.mappings()
.first()
)
if run_row is None:
status = "fail" if expected_tool_name else "env_issue"
status = 'fail' if expected_tool_name else 'env_issue'
return {
"status": status,
"reason": "No AgentRunner run contains the expected tool call." if expected_tool_name else "No matching AgentRunner run exists.",
"failures": [{"kind": "expected_tool_call_missing"}] if expected_tool_name else [],
"warnings": [],
'status': status,
'reason': 'No Runner run contains the expected tool call.'
if expected_tool_name
else 'No matching Runner run exists.',
'failures': [{'kind': 'expected_tool_call_missing'}] if expected_tool_name else [],
'warnings': [],
}
selected_run_id = str(run_row["run_id"])
event_rows = (await connection.execute(
sqlalchemy.text("SELECT sequence, type, data_json, metadata_json FROM agent_run_event WHERE run_id = :run_id ORDER BY sequence"),
{"run_id": selected_run_id},
)).mappings().all()
selected_run_id = str(run_row['run_id'])
event_rows = (
(
await connection.execute(
sqlalchemy.text(
'SELECT sequence, type, data_json, metadata_json FROM agent_run_event WHERE run_id = :run_id ORDER BY sequence'
),
{'run_id': selected_run_id},
)
)
.mappings()
.all()
)
finally:
await engine.dispose()
authorization = load_ledger_json(
run_row.get("authorization_json"),
field="agent_run.authorization_json",
run_row.get('authorization_json'),
field='agent_run.authorization_json',
failures=failures,
)
tools = authorization.get("resources", {}).get("tools", []) if isinstance(authorization, dict) else []
tools = authorization.get('resources', {}).get('tools', []) if isinstance(authorization, dict) else []
allowed_tools: dict[str, dict] = {}
incomplete_tool_metadata: list[dict] = []
for tool in tools if isinstance(tools, list) else []:
if not isinstance(tool, dict):
incomplete_tool_metadata.append({"tool_name": "", "missing": ["tool object"]})
incomplete_tool_metadata.append({'tool_name': '', 'missing': ['tool object']})
continue
name = str(tool.get("tool_name", ""))
name = str(tool.get('tool_name', ''))
missing = []
if not name:
missing.append("tool_name")
if not str(tool.get("description", "")).strip():
missing.append("description")
if not isinstance(tool.get("parameters"), dict):
missing.append("parameters")
if not (tool.get("source") or tool.get("tool_type") or tool.get("source_id")):
missing.append("owner")
missing.append('tool_name')
if not str(tool.get('description', '')).strip():
missing.append('description')
if not isinstance(tool.get('parameters'), dict):
missing.append('parameters')
if not (tool.get('source') or tool.get('tool_type') or tool.get('source_id')):
missing.append('owner')
if missing:
incomplete_tool_metadata.append({"tool_name": name, "missing": missing})
incomplete_tool_metadata.append({'tool_name': name, 'missing': missing})
if name:
allowed_tools[name] = tool
if incomplete_tool_metadata:
failures.append({"kind": "incomplete_tool_metadata", "tools": incomplete_tool_metadata})
failures.append({'kind': 'incomplete_tool_metadata', 'tools': incomplete_tool_metadata})
starts: dict[str, list[dict]] = {}
completions: dict[str, list[dict]] = {}
@@ -178,7 +202,7 @@ async def audit(
invalid_tool_argument_errors: list[dict] = []
successful_tool_completion_sequences: list[int] = []
forbidden_pattern = re.compile(
r"invalid json(?! arguments)|unauthori[sz]ed|permission denied|forbidden|timed?\s*out|timeout",
r'invalid json(?! arguments)|unauthori[sz]ed|permission denied|forbidden|timed?\s*out|timeout',
re.I,
)
@@ -189,55 +213,53 @@ async def audit(
return collected
for key, item in value.items():
normalized = str(key).lower()
if normalized in {"error", "code", "status", "reason", "error_message"} and item is not None and item != "":
if normalized in {'error', 'code', 'status', 'reason', 'error_message'} and item is not None and item != '':
collected.append(str(item))
if isinstance(item, dict):
collected.extend(error_surface(item))
return collected
for row in event_rows:
event_type = str(row["type"])
event_type = str(row['type'])
event_types.append(event_type)
before = len(failures)
data = load_ledger_json(
row.get("data_json"),
field=f"agent_run_event[{row['sequence']}].data_json",
row.get('data_json'),
field=f'agent_run_event[{row["sequence"]}].data_json',
failures=failures,
)
invalid_event_json += int(len(failures) > before)
if not isinstance(data, dict):
failures.append({"kind": "invalid_event_payload", "sequence": row["sequence"], "type": event_type})
failures.append({'kind': 'invalid_event_payload', 'sequence': row['sequence'], 'type': event_type})
continue
if event_type in {"tool.call.started", "tool.call.completed"}:
call_id = str(data.get("tool_call_id", ""))
item = {"sequence": row["sequence"], "tool_name": str(data.get("tool_name", "")), "data": data}
if event_type in {'tool.call.started', 'tool.call.completed'}:
call_id = str(data.get('tool_call_id', ''))
item = {'sequence': row['sequence'], 'tool_name': str(data.get('tool_name', '')), 'data': data}
if not call_id:
failures.append({"kind": "missing_tool_call_id", "sequence": row["sequence"], "type": event_type})
elif event_type == "tool.call.started":
failures.append({'kind': 'missing_tool_call_id', 'sequence': row['sequence'], 'type': event_type})
elif event_type == 'tool.call.started':
starts.setdefault(call_id, []).append(item)
else:
completions.setdefault(call_id, []).append(item)
if not data.get("error") and data.get("result") is not None:
successful_tool_completion_sequences.append(row["sequence"])
diagnostic_text = "\n".join(error_surface(data))
if event_type == "run.failed":
diagnostic_text += "\n" + json.dumps(data, ensure_ascii=True)
if not data.get('error') and data.get('result') is not None:
successful_tool_completion_sequences.append(row['sequence'])
diagnostic_text = '\n'.join(error_surface(data))
if event_type == 'run.failed':
diagnostic_text += '\n' + json.dumps(data, ensure_ascii=True)
match = forbidden_pattern.search(diagnostic_text)
if match:
suspicious_errors.append({"sequence": row["sequence"], "type": event_type, "signal": match.group(0)})
elif event_type == "tool.call.completed":
suspicious_errors.append({'sequence': row['sequence'], 'type': event_type, 'signal': match.group(0)})
elif event_type == 'tool.call.completed':
signal = invalid_tool_argument_error_signal(diagnostic_text)
if signal:
invalid_tool_argument_errors.append(
{"sequence": row["sequence"], "type": event_type, "signal": signal}
)
invalid_tool_argument_errors.append({'sequence': row['sequence'], 'type': event_type, 'signal': signal})
if run_row["status"] != "completed":
failures.append({"kind": "run_status", "actual": run_row["status"], "expected": "completed"})
if "run.completed" not in event_types:
failures.append({"kind": "missing_run_completed_event"})
if "run.failed" in event_types:
failures.append({"kind": "run_failed_event"})
if run_row['status'] != 'completed':
failures.append({'kind': 'run_status', 'actual': run_row['status'], 'expected': 'completed'})
if 'run.completed' not in event_types:
failures.append({'kind': 'missing_run_completed_event'})
if 'run.failed' in event_types:
failures.append({'kind': 'run_failed_event'})
all_call_ids = sorted(set(starts) | set(completions))
unauthorized_calls = []
@@ -245,14 +267,21 @@ async def audit(
started = starts.get(call_id, [])
completed = completions.get(call_id, [])
if len(started) != 1 or len(completed) != 1:
failures.append({"kind": "tool_call_pairing", "tool_call_id": call_id, "started": len(started), "completed": len(completed)})
failures.append(
{
'kind': 'tool_call_pairing',
'tool_call_id': call_id,
'started': len(started),
'completed': len(completed),
}
)
continue
if started[0]["tool_name"] != completed[0]["tool_name"]:
failures.append({"kind": "tool_name_mismatch", "tool_call_id": call_id})
if started[0]["sequence"] >= completed[0]["sequence"]:
failures.append({"kind": "tool_call_order", "tool_call_id": call_id})
if started[0]["tool_name"] not in allowed_tools:
unauthorized_calls.append({"tool_call_id": call_id, "tool_name": started[0]["tool_name"]})
if started[0]['tool_name'] != completed[0]['tool_name']:
failures.append({'kind': 'tool_name_mismatch', 'tool_call_id': call_id})
if started[0]['sequence'] >= completed[0]['sequence']:
failures.append({'kind': 'tool_call_order', 'tool_call_id': call_id})
if started[0]['tool_name'] not in allowed_tools:
unauthorized_calls.append({'tool_call_id': call_id, 'tool_name': started[0]['tool_name']})
authorization_failures, authorization_warnings = classify_tool_authorization(
unauthorized_calls,
authorization_mode=tool_authorization_mode,
@@ -263,20 +292,18 @@ async def audit(
invalid_tool_argument_errors,
successful_tool_completion_sequences=successful_tool_completion_sequences,
run_completed=(
run_row["status"] == "completed"
and "run.completed" in event_types
and "run.failed" not in event_types
run_row['status'] == 'completed' and 'run.completed' in event_types and 'run.failed' not in event_types
),
)
if unrecovered_argument_errors:
suspicious_errors.extend(unrecovered_argument_errors)
warnings.extend(recovered_argument_warnings)
if suspicious_errors:
failures.append({"kind": "forbidden_error_signals", "events": suspicious_errors})
failures.append({'kind': 'forbidden_error_signals', 'events': suspicious_errors})
if not event_rows:
failures.append({"kind": "missing_run_events"})
failures.append({'kind': 'missing_run_events'})
if not tools:
warnings.append({"kind": "no_authorized_tools", "reason": "The run authorization snapshot exposes no tools."})
warnings.append({'kind': 'no_authorized_tools', 'reason': 'The run authorization snapshot exposes no tools.'})
expected_call_summary = None
if expected_tool_name:
@@ -284,97 +311,101 @@ async def audit(
item
for items in starts.values()
for item in items
if item["tool_name"] == expected_tool_name
and (expected_parameters is None or item["data"].get("parameters") == expected_parameters)
if item['tool_name'] == expected_tool_name
and (expected_parameters is None or item['data'].get('parameters') == expected_parameters)
]
if len(matching_starts) != 1:
failures.append({"kind": "expected_tool_call_count", "actual": len(matching_starts), "expected": 1})
failures.append({'kind': 'expected_tool_call_count', 'actual': len(matching_starts), 'expected': 1})
matching_completions = []
for started in matching_starts:
call_id = str(started["data"].get("tool_call_id", ""))
call_id = str(started['data'].get('tool_call_id', ''))
matching_completions.extend(completions.get(call_id, []))
result_text_match = expected_result_text is None or any(
expected_result_text in collect_result_texts(completed["data"].get("result"))
expected_result_text in collect_result_texts(completed['data'].get('result'))
for completed in matching_completions
)
if expected_result_text is not None and not result_text_match:
failures.append({"kind": "expected_tool_result_text_missing"})
failures.append({'kind': 'expected_tool_result_text_missing'})
expected_call_summary = {
"tool_name": expected_tool_name,
"parameters_match_required": expected_parameters is not None,
"matched_started_count": len(matching_starts),
"matched_completed_count": len(matching_completions),
"result_text_match_required": expected_result_text is not None,
"result_text_match": result_text_match,
'tool_name': expected_tool_name,
'parameters_match_required': expected_parameters is not None,
'matched_started_count': len(matching_starts),
'matched_completed_count': len(matching_completions),
'result_text_match_required': expected_result_text is not None,
'result_text_match': result_text_match,
}
metrics = {
"event_count": len(event_rows),
"tool_call_started": sum(len(items) for items in starts.values()),
"tool_call_completed": sum(len(items) for items in completions.values()),
"tool_call_ids": len(all_call_ids),
"authorized_tool_count": len(allowed_tools),
"tool_authorization_mode": tool_authorization_mode,
"runner_native_tool_call_count": len(unauthorized_calls) if tool_authorization_mode == "runner-native" else 0,
"invalid_event_json": invalid_event_json,
"suspicious_error_count": len(suspicious_errors),
"recovered_tool_argument_error_count": len(recovered_argument_warnings),
'event_count': len(event_rows),
'tool_call_started': sum(len(items) for items in starts.values()),
'tool_call_completed': sum(len(items) for items in completions.values()),
'tool_call_ids': len(all_call_ids),
'authorized_tool_count': len(allowed_tools),
'tool_authorization_mode': tool_authorization_mode,
'runner_native_tool_call_count': len(unauthorized_calls) if tool_authorization_mode == 'runner-native' else 0,
'invalid_event_json': invalid_event_json,
'suspicious_error_count': len(suspicious_errors),
'recovered_tool_argument_error_count': len(recovered_argument_warnings),
}
return {
"status": "pass" if not failures else "fail",
"reason": "Agent run ledger audit passed." if not failures else f"Agent run ledger audit found {len(failures)} invariant failure(s).",
"run": {
"run_id": selected_run_id,
"runner_id": run_row["runner_id"],
"status": run_row["status"],
"created_at": str(run_row["created_at"]),
"finished_at": str(run_row["finished_at"]),
'status': 'pass' if not failures else 'fail',
'reason': 'Agent run ledger audit passed.'
if not failures
else f'Agent run ledger audit found {len(failures)} invariant failure(s).',
'run': {
'run_id': selected_run_id,
'runner_id': run_row['runner_id'],
'status': run_row['status'],
'created_at': str(run_row['created_at']),
'finished_at': str(run_row['finished_at']),
},
"metrics": metrics,
"expected_tool_call": expected_call_summary,
"failures": failures,
"warnings": warnings,
'metrics': metrics,
'expected_tool_call': expected_call_summary,
'failures': failures,
'warnings': warnings,
}
def main() -> int:
parser = argparse.ArgumentParser()
parser.add_argument("--repo", required=True)
parser.add_argument("--run-id")
parser.add_argument("--created-after")
parser.add_argument("--expected-tool-name")
parser.add_argument("--expected-parameters-json")
parser.add_argument("--expected-result-text")
parser.add_argument('--repo', required=True)
parser.add_argument('--run-id')
parser.add_argument('--created-after')
parser.add_argument('--expected-tool-name')
parser.add_argument('--expected-parameters-json')
parser.add_argument('--expected-result-text')
parser.add_argument(
"--tool-authorization-mode",
choices=("strict", "runner-native"),
default="strict",
'--tool-authorization-mode',
choices=('strict', 'runner-native'),
default='strict',
)
parser.add_argument("--output", required=True)
parser.add_argument('--output', required=True)
args = parser.parse_args()
try:
expected_parameters = None
if args.expected_parameters_json:
expected_parameters = json.loads(args.expected_parameters_json)
if not isinstance(expected_parameters, dict):
raise ValueError("--expected-parameters-json must decode to an object")
raise ValueError('--expected-parameters-json must decode to an object')
if (expected_parameters is not None or args.expected_result_text) and not args.expected_tool_name:
raise ValueError("--expected-tool-name is required with expected parameters or result text")
report = asyncio.run(audit(
pathlib.Path(args.repo).resolve(),
args.run_id,
created_after=parse_created_after(args.created_after),
expected_tool_name=args.expected_tool_name,
expected_parameters=expected_parameters,
expected_result_text=args.expected_result_text,
tool_authorization_mode=args.tool_authorization_mode,
))
raise ValueError('--expected-tool-name is required with expected parameters or result text')
report = asyncio.run(
audit(
pathlib.Path(args.repo).resolve(),
args.run_id,
created_after=parse_created_after(args.created_after),
expected_tool_name=args.expected_tool_name,
expected_parameters=expected_parameters,
expected_result_text=args.expected_result_text,
tool_authorization_mode=args.tool_authorization_mode,
)
)
except Exception as exc: # noqa: BLE001 - probe must classify environment failures
report = {"status": "env_issue", "reason": str(exc), "failures": [], "warnings": []}
pathlib.Path(args.output).write_text(json.dumps(report, indent=2) + "\n", encoding="utf-8")
report = {'status': 'env_issue', 'reason': str(exc), 'failures': [], 'warnings': []}
pathlib.Path(args.output).write_text(json.dumps(report, indent=2) + '\n', encoding='utf-8')
print(json.dumps(report))
return 0 if report["status"] == "pass" else 2 if report["status"] == "env_issue" else 1
return 0 if report['status'] == 'pass' else 2 if report['status'] == 'env_issue' else 1
if __name__ == "__main__":
if __name__ == '__main__':
sys.exit(main())
@@ -143,7 +143,7 @@ try {
}
if (!runner?.name) {
result.status = "blocked";
throw new Error("No registered AgentRunner is available for the UI check.");
throw new Error("No registered Runner is available for the UI check.");
}
const runnerConfigStage = runnerTab.stages.find(
@@ -155,7 +155,7 @@ try {
body: {
kind: "agent",
name: `Runner Health ${paths.runId.slice(-40)}`,
description: "Temporary AgentRunner health visibility fixture",
description: "Temporary Runner health visibility fixture",
emoji: "H",
component_ref: runner.name,
config: {
@@ -229,7 +229,7 @@ try {
}
result.status = "pass";
result.reason =
"Agent Runner settings visibly distinguished a registered runner from a stale binding.";
"Runner settings visibly distinguished a registered runner from a stale binding.";
} catch (error) {
if (!["blocked", "env_issue"].includes(result.status)) result.status = "fail";
result.reason = result.reason || error.message;
@@ -24,7 +24,10 @@ function loadEnvDefaults(path) {
if (sep === -1) continue;
const key = line.slice(0, sep).trim();
if (env[key]) continue;
env[key] = line.slice(sep + 1).trim().replace(/^["']|["']$/g, "");
env[key] = line
.slice(sep + 1)
.trim()
.replace(/^["']|["']$/g, "");
}
}
@@ -46,12 +49,16 @@ function redactMessage(text) {
return String(text ?? "")
.replace(/\bbearer\s+[A-Za-z0-9._~+/=-]{8,}/gi, "Bearer [redacted]")
.replace(/\bsk-[A-Za-z0-9_-]{6,}\b/g, "[redacted]")
.replace(/(api[_-]?key|authorization|credential|jwt|oauth|password|secret|token)\s*[:=]\s*["']?[^"',\s]+/gi, "$1=[redacted]");
.replace(
/(api[_-]?key|authorization|credential|jwt|oauth|password|secret|token)\s*[:=]\s*["']?[^"',\s]+/gi,
"$1=[redacted]",
);
}
function isEnvironmentError(message) {
return /Playwright is not installed|LANGBOT_FRONTEND_URL|LANGBOT_BACKEND_URL|ERR_CONNECTION_REFUSED|ECONNREFUSED|net::ERR_|fetch failed|timed out/i
.test(message);
return /Playwright is not installed|LANGBOT_FRONTEND_URL|LANGBOT_BACKEND_URL|ERR_CONNECTION_REFUSED|ECONNREFUSED|net::ERR_|fetch failed|timed out/i.test(
message,
);
}
loadEnvDefaults("skills/.env");
@@ -80,9 +87,15 @@ const targets = [
},
{
id: "acp-agent-runner",
expected_runner_id: "plugin:langbot-team/ACPAgentRunner/default",
pipeline_url: firstEnv("LANGBOT_ACP_AGENT_RUNNER_PIPELINE_URL", "LANGBOT_AGENT_RUNNER_PIPELINE_URL"),
pipeline_name: firstEnv("LANGBOT_ACP_AGENT_RUNNER_PIPELINE_NAME", "LANGBOT_AGENT_RUNNER_PIPELINE_NAME"),
expected_runner_id: "plugin:langbot-team/ACPRunner/default",
pipeline_url: firstEnv(
"LANGBOT_ACP_RUNNER_PIPELINE_URL",
"LANGBOT_RUNNER_PIPELINE_URL",
),
pipeline_name: firstEnv(
"LANGBOT_ACP_RUNNER_PIPELINE_NAME",
"LANGBOT_RUNNER_PIPELINE_NAME",
),
require_func_call_model: false,
require_vision_model: false,
},
@@ -111,20 +124,29 @@ const result = {
automation_result_json: paths.automationResultJson,
result_json: paths.resultJson,
},
evidence_collected: ["ui", "screenshot", "console", "network", "api_diagnostic"],
evidence_collected: [
"ui",
"screenshot",
"console",
"network",
"api_diagnostic",
],
};
async function run() {
if (!backendUrl || !frontendUrl) {
result.status = "env_issue";
result.reason = "LANGBOT_FRONTEND_URL and LANGBOT_BACKEND_URL must be configured.";
result.reason =
"LANGBOT_FRONTEND_URL and LANGBOT_BACKEND_URL must be configured.";
return;
}
browser = await createBrowser(paths);
const { page } = browser;
await page.goto(frontendUrl, { waitUntil: "domcontentloaded" });
await page.waitForLoadState("networkidle", { timeout: 10_000 }).catch(() => {});
await page
.waitForLoadState("networkidle", { timeout: 10_000 })
.catch(() => {});
const workspace = await ensureBrowserWorkspace(page, backendUrl);
if (workspace.status !== "pass") {
result.status = workspace.status;
@@ -132,309 +154,428 @@ async function run() {
return;
}
const diagnostic = await page.evaluate(async ({ backendUrl, targets, testModels }) => {
const blockers = [];
const envIssues = [];
const warnings = [];
const checks = [];
const diagnostic = await page.evaluate(
async ({ backendUrl, targets, testModels }) => {
const blockers = [];
const envIssues = [];
const warnings = [];
const checks = [];
const addCheck = (name, status, detail = {}) => {
checks.push({ name, status, ...detail });
if (status === "blocked") blockers.push({ name, ...detail });
if (status === "env_issue") envIssues.push({ name, ...detail });
};
const safeMessage = (value) => String(value ?? "")
.replace(/\bbearer\s+[A-Za-z0-9._~+/=-]{8,}/gi, "Bearer [redacted]")
.replace(/\bsk-[A-Za-z0-9_-]{6,}\b/g, "[redacted]")
.replace(/(api[_-]?key|authorization|credential|jwt|oauth|password|secret|token)\s*[:=]\s*["']?[^"',\s]+/gi, "$1=[redacted]");
const token = localStorage.getItem("token");
if (!token) {
addCheck("browser-auth", "blocked", { reason: "Browser profile has no localStorage token." });
return { authenticated: false, blockers, env_issues: envIssues, warnings, checks };
}
const headers = {
Authorization: `Bearer ${token}`,
"Content-Type": "application/json",
"X-Workspace-Id": localStorage.getItem("langbot_active_workspace_uuid") || "",
};
const getJson = async (path) => {
const response = await fetch(`${backendUrl}${path}`, { headers });
return {
status: response.status,
json: await response.json().catch(() => ({})),
const addCheck = (name, status, detail = {}) => {
checks.push({ name, status, ...detail });
if (status === "blocked") blockers.push({ name, ...detail });
if (status === "env_issue") envIssues.push({ name, ...detail });
};
};
const postJson = async (path, body) => {
const response = await fetch(`${backendUrl}${path}`, {
method: "POST",
headers,
body: JSON.stringify(body),
});
return {
status: response.status,
json: await response.json().catch(() => ({})),
};
};
const safeMessage = (value) =>
String(value ?? "")
.replace(/\bbearer\s+[A-Za-z0-9._~+/=-]{8,}/gi, "Bearer [redacted]")
.replace(/\bsk-[A-Za-z0-9_-]{6,}\b/g, "[redacted]")
.replace(
/(api[_-]?key|authorization|credential|jwt|oauth|password|secret|token)\s*[:=]\s*["']?[^"',\s]+/gi,
"$1=[redacted]",
);
const tokenCheck = await getJson("/api/v1/user/check-token");
addCheck(
"browser-auth",
tokenCheck.status < 400 && (tokenCheck.json.code ?? 0) === 0 ? "pass" : "blocked",
{ http_status: tokenCheck.status, code: tokenCheck.json.code ?? null, reason: safeMessage(tokenCheck.json.msg || "") },
);
const systemInfo = await getJson("/api/v1/system/info");
addCheck(
"backend-system-info",
systemInfo.status < 400 ? "pass" : "env_issue",
{
http_status: systemInfo.status,
version: systemInfo.json.data?.version || systemInfo.json.data?.system?.version || "",
},
);
const pluginSystem = await getJson("/api/v1/system/status/plugin-system");
addCheck(
"plugin-system",
pluginSystem.status < 400 && (pluginSystem.json.code ?? 0) === 0 ? "pass" : "env_issue",
{
http_status: pluginSystem.status,
code: pluginSystem.json.code ?? null,
status: pluginSystem.json.data?.status || pluginSystem.json.data?.state || "",
reason: safeMessage(pluginSystem.json.msg || ""),
},
);
const boxStatus = await getJson("/api/v1/box/status");
addCheck(
"box-runtime",
boxStatus.status < 400 && (boxStatus.json.code ?? 0) === 0 ? "pass" : "env_issue",
{
http_status: boxStatus.status,
code: boxStatus.json.code ?? null,
status: boxStatus.json.data?.status || "",
backend: boxStatus.json.data?.backend || "",
reason: safeMessage(boxStatus.json.msg || ""),
},
);
const plugins = await getJson("/api/v1/plugins");
const installedPluginIds = (plugins.json.data?.plugins || [])
.map((plugin) => {
const metadata = plugin.manifest?.manifest?.metadata || plugin.manifest?.metadata || plugin.metadata || {};
return metadata.author && metadata.name ? `${metadata.author}/${metadata.name}` : "";
})
.filter(Boolean);
const requiredPlugins = ["langbot-team/LocalAgent", "langbot-team/ACPAgentRunner", "qa/plugin-smoke"];
const pluginPresence = Object.fromEntries(requiredPlugins.map((id) => [id, installedPluginIds.includes(id)]));
for (const [id, present] of Object.entries(pluginPresence)) {
addCheck(`plugin:${id}`, present ? "pass" : "blocked", { plugin_id: id, reason: present ? "" : "Required plugin is not listed by /api/v1/plugins." });
}
const tools = await getJson("/api/v1/tools");
const toolNames = (tools.json.data?.tools || [])
.map((tool) => tool.name || tool.tool_name || tool.function?.name || "")
.filter(Boolean)
.sort();
addCheck(
"tool:qa_plugin_echo",
toolNames.includes("qa_plugin_echo") ? "pass" : "blocked",
{ reason: toolNames.includes("qa_plugin_echo") ? "" : "qa-plugin-smoke tool qa_plugin_echo is not exposed through /api/v1/tools." },
);
if (!toolNames.includes("qa_mcp_echo")) {
warnings.push({
name: "tool:qa_mcp_echo",
reason: "qa_mcp_echo is not currently exposed. This is acceptable before mcp-stdio-register, but mcp-stdio-tool-call must run after registration.",
});
}
const modelResponse = await getJson("/api/v1/provider/models/llm");
const models = (modelResponse.json.data?.models || []).map((model) => ({
uuid: model.uuid,
name: model.name,
abilities: Array.isArray(model.abilities) ? model.abilities : [],
provider_uuid: model.provider_uuid || model.provider?.uuid || "",
provider_name: model.provider_name || model.provider?.name || "",
requester: model.requester || model.provider?.requester || "",
}));
addCheck(
"llm-model-list",
modelResponse.status < 400 && (modelResponse.json.code ?? 0) === 0 ? "pass" : "env_issue",
{ http_status: modelResponse.status, model_count: models.length, reason: safeMessage(modelResponse.json.msg || "") },
);
const modelById = new Map(models.map((model) => [model.uuid, model]));
const pipelineList = await getJson("/api/v1/pipelines");
const pipelines = pipelineList.json.data?.pipelines || [];
addCheck(
"pipeline-list",
pipelineList.status < 400 && (pipelineList.json.code ?? 0) === 0 ? "pass" : "blocked",
{ http_status: pipelineList.status, pipeline_count: pipelines.length, reason: safeMessage(pipelineList.json.msg || "") },
);
const resolvedPipelines = [];
const modelTested = new Set();
for (const target of targets) {
let pipelineId = "";
let matchedBy = "";
if (target.pipeline_url) {
try {
pipelineId = new URL(target.pipeline_url).searchParams.get("id") || "";
matchedBy = pipelineId ? "url" : "";
} catch {
pipelineId = "";
}
}
if (!pipelineId && target.pipeline_name) {
const match = pipelines.find((pipeline) => pipeline.name === target.pipeline_name);
if (match) {
pipelineId = match.uuid;
matchedBy = "name";
}
}
if (!pipelineId) {
addCheck(`pipeline:${target.id}`, "blocked", {
target: target.id,
reason: "Required pipeline env is missing or could not resolve to a pipeline id.",
const token = localStorage.getItem("token");
if (!token) {
addCheck("browser-auth", "blocked", {
reason: "Browser profile has no localStorage token.",
});
continue;
return {
authenticated: false,
blockers,
env_issues: envIssues,
warnings,
checks,
};
}
const response = await getJson(`/api/v1/pipelines/${encodeURIComponent(pipelineId)}`);
const pipeline = response.json.data?.pipeline;
if (response.status >= 400 || !pipeline) {
addCheck(`pipeline:${target.id}`, "blocked", {
target: target.id,
pipeline_id: pipelineId,
http_status: response.status,
reason: safeMessage(response.json.msg || "Could not load pipeline."),
});
continue;
}
const config = pipeline.config || {};
const aiConfig = config.ai && typeof config.ai === "object" ? config.ai : {};
const runner = aiConfig.runner && typeof aiConfig.runner === "object" ? aiConfig.runner : {};
const runnerId = runner.id || "";
const runnerConfigs = aiConfig.runner_config && typeof aiConfig.runner_config === "object" ? aiConfig.runner_config : {};
const runnerConfig = runnerConfigs[runnerId] && typeof runnerConfigs[runnerId] === "object" ? runnerConfigs[runnerId] : {};
const pipelineSummary = {
target: target.id,
pipeline_id: pipelineId,
pipeline_name: pipeline.name,
matched_by: matchedBy,
runner_id: runnerId,
expected_runner_id: target.expected_runner_id,
runner_config_keys: Object.keys(runnerConfig).sort(),
const headers = {
Authorization: `Bearer ${token}`,
"Content-Type": "application/json",
"X-Workspace-Id":
localStorage.getItem("langbot_active_workspace_uuid") || "",
};
const getJson = async (path) => {
const response = await fetch(`${backendUrl}${path}`, { headers });
return {
status: response.status,
json: await response.json().catch(() => ({})),
};
};
const postJson = async (path, body) => {
const response = await fetch(`${backendUrl}${path}`, {
method: "POST",
headers,
body: JSON.stringify(body),
});
return {
status: response.status,
json: await response.json().catch(() => ({})),
};
};
resolvedPipelines.push(pipelineSummary);
const tokenCheck = await getJson("/api/v1/user/check-token");
addCheck(
`pipeline:${target.id}:runner`,
runnerId === target.expected_runner_id ? "pass" : "blocked",
"browser-auth",
tokenCheck.status < 400 && (tokenCheck.json.code ?? 0) === 0
? "pass"
: "blocked",
{
...pipelineSummary,
reason: runnerId === target.expected_runner_id ? "" : `Expected ${target.expected_runner_id}, got ${runnerId || "<missing>"}.`,
http_status: tokenCheck.status,
code: tokenCheck.json.code ?? null,
reason: safeMessage(tokenCheck.json.msg || ""),
},
);
if (target.require_func_call_model || target.require_vision_model || (testModels && target.id === "local-agent")) {
const modelConfig = runnerConfig.model;
const primaryModelId = typeof modelConfig === "string"
? modelConfig
: modelConfig && typeof modelConfig === "object"
? modelConfig.primary || ""
const systemInfo = await getJson("/api/v1/system/info");
addCheck(
"backend-system-info",
systemInfo.status < 400 ? "pass" : "env_issue",
{
http_status: systemInfo.status,
version:
systemInfo.json.data?.version ||
systemInfo.json.data?.system?.version ||
"",
},
);
const pluginSystem = await getJson("/api/v1/system/status/plugin-system");
addCheck(
"plugin-system",
pluginSystem.status < 400 && (pluginSystem.json.code ?? 0) === 0
? "pass"
: "env_issue",
{
http_status: pluginSystem.status,
code: pluginSystem.json.code ?? null,
status:
pluginSystem.json.data?.status ||
pluginSystem.json.data?.state ||
"",
reason: safeMessage(pluginSystem.json.msg || ""),
},
);
const boxStatus = await getJson("/api/v1/box/status");
addCheck(
"box-runtime",
boxStatus.status < 400 && (boxStatus.json.code ?? 0) === 0
? "pass"
: "env_issue",
{
http_status: boxStatus.status,
code: boxStatus.json.code ?? null,
status: boxStatus.json.data?.status || "",
backend: boxStatus.json.data?.backend || "",
reason: safeMessage(boxStatus.json.msg || ""),
},
);
const plugins = await getJson("/api/v1/plugins");
const installedPluginIds = (plugins.json.data?.plugins || [])
.map((plugin) => {
const metadata =
plugin.manifest?.manifest?.metadata ||
plugin.manifest?.metadata ||
plugin.metadata ||
{};
return metadata.author && metadata.name
? `${metadata.author}/${metadata.name}`
: "";
if (!primaryModelId) {
addCheck(`pipeline:${target.id}:primary-model`, "blocked", {
...pipelineSummary,
reason: "Local-agent runner config has no primary model.",
});
continue;
}
const model = modelById.get(primaryModelId);
if (!model) {
addCheck(`pipeline:${target.id}:primary-model`, "blocked", {
...pipelineSummary,
model_uuid: primaryModelId,
reason: "Primary model is not listed by /api/v1/provider/models/llm.",
});
continue;
}
addCheck(`pipeline:${target.id}:primary-model`, "pass", {
...pipelineSummary,
model: {
uuid: model.uuid,
name: model.name,
abilities: model.abilities,
provider_name: model.provider_name,
requester: model.requester,
},
})
.filter(Boolean);
const requiredPlugins = [
"langbot-team/LocalAgent",
"langbot-team/ACPRunner",
"qa/plugin-smoke",
];
const pluginPresence = Object.fromEntries(
requiredPlugins.map((id) => [id, installedPluginIds.includes(id)]),
);
for (const [id, present] of Object.entries(pluginPresence)) {
addCheck(`plugin:${id}`, present ? "pass" : "blocked", {
plugin_id: id,
reason: present
? ""
: "Required plugin is not listed by /api/v1/plugins.",
});
if (target.require_func_call_model) {
addCheck(
`pipeline:${target.id}:func-call-model`,
model.abilities.includes("func_call") ? "pass" : "env_issue",
{
model_uuid: model.uuid,
model_name: model.name,
abilities: model.abilities,
reason: model.abilities.includes("func_call") ? "" : "Release gate includes tool-call cases; the local-agent primary model must advertise func_call.",
},
);
}
const tools = await getJson("/api/v1/tools");
const toolNames = (tools.json.data?.tools || [])
.map((tool) => tool.name || tool.tool_name || tool.function?.name || "")
.filter(Boolean)
.sort();
addCheck(
"tool:qa_plugin_echo",
toolNames.includes("qa_plugin_echo") ? "pass" : "blocked",
{
reason: toolNames.includes("qa_plugin_echo")
? ""
: "qa-plugin-smoke tool qa_plugin_echo is not exposed through /api/v1/tools.",
},
);
if (!toolNames.includes("qa_mcp_echo")) {
warnings.push({
name: "tool:qa_mcp_echo",
reason:
"qa_mcp_echo is not currently exposed. This is acceptable before mcp-stdio-register, but mcp-stdio-tool-call must run after registration.",
});
}
const modelResponse = await getJson("/api/v1/provider/models/llm");
const models = (modelResponse.json.data?.models || []).map((model) => ({
uuid: model.uuid,
name: model.name,
abilities: Array.isArray(model.abilities) ? model.abilities : [],
provider_uuid: model.provider_uuid || model.provider?.uuid || "",
provider_name: model.provider_name || model.provider?.name || "",
requester: model.requester || model.provider?.requester || "",
}));
addCheck(
"llm-model-list",
modelResponse.status < 400 && (modelResponse.json.code ?? 0) === 0
? "pass"
: "env_issue",
{
http_status: modelResponse.status,
model_count: models.length,
reason: safeMessage(modelResponse.json.msg || ""),
},
);
const modelById = new Map(models.map((model) => [model.uuid, model]));
const pipelineList = await getJson("/api/v1/pipelines");
const pipelines = pipelineList.json.data?.pipelines || [];
addCheck(
"pipeline-list",
pipelineList.status < 400 && (pipelineList.json.code ?? 0) === 0
? "pass"
: "blocked",
{
http_status: pipelineList.status,
pipeline_count: pipelines.length,
reason: safeMessage(pipelineList.json.msg || ""),
},
);
const resolvedPipelines = [];
const modelTested = new Set();
for (const target of targets) {
let pipelineId = "";
let matchedBy = "";
if (target.pipeline_url) {
try {
pipelineId =
new URL(target.pipeline_url).searchParams.get("id") || "";
matchedBy = pipelineId ? "url" : "";
} catch {
pipelineId = "";
}
}
if (target.require_vision_model) {
addCheck(
`pipeline:${target.id}:vision-model`,
model.abilities.includes("vision") ? "pass" : "env_issue",
{
model_uuid: model.uuid,
model_name: model.name,
abilities: model.abilities,
reason: model.abilities.includes("vision") ? "" : "Release gate includes multimodal cases; the local-agent primary model must advertise vision.",
},
if (!pipelineId && target.pipeline_name) {
const match = pipelines.find(
(pipeline) => pipeline.name === target.pipeline_name,
);
if (match) {
pipelineId = match.uuid;
matchedBy = "name";
}
}
if (testModels && !modelTested.has(model.uuid)) {
modelTested.add(model.uuid);
const modelTest = await postJson(`/api/v1/provider/models/llm/${encodeURIComponent(model.uuid)}/test`, { extra_args: {} });
const passed = modelTest.status < 400 && (modelTest.json.code ?? 0) === 0;
addCheck(
`model-test:${model.name}`,
passed ? "pass" : "env_issue",
{
model_uuid: model.uuid,
model_name: model.name,
http_status: modelTest.status,
code: modelTest.json.code ?? null,
reason: passed ? "" : safeMessage(modelTest.json.msg || modelTest.json.message || "Model test failed."),
if (!pipelineId) {
addCheck(`pipeline:${target.id}`, "blocked", {
target: target.id,
reason:
"Required pipeline env is missing or could not resolve to a pipeline id.",
});
continue;
}
const response = await getJson(
`/api/v1/pipelines/${encodeURIComponent(pipelineId)}`,
);
const pipeline = response.json.data?.pipeline;
if (response.status >= 400 || !pipeline) {
addCheck(`pipeline:${target.id}`, "blocked", {
target: target.id,
pipeline_id: pipelineId,
http_status: response.status,
reason: safeMessage(
response.json.msg || "Could not load pipeline.",
),
});
continue;
}
const config = pipeline.config || {};
const aiConfig =
config.ai && typeof config.ai === "object" ? config.ai : {};
const runner =
aiConfig.runner && typeof aiConfig.runner === "object"
? aiConfig.runner
: {};
const runnerId = runner.id || "";
const runnerConfigs =
aiConfig.runner_config && typeof aiConfig.runner_config === "object"
? aiConfig.runner_config
: {};
const runnerConfig =
runnerConfigs[runnerId] && typeof runnerConfigs[runnerId] === "object"
? runnerConfigs[runnerId]
: {};
const pipelineSummary = {
target: target.id,
pipeline_id: pipelineId,
pipeline_name: pipeline.name,
matched_by: matchedBy,
runner_id: runnerId,
expected_runner_id: target.expected_runner_id,
runner_config_keys: Object.keys(runnerConfig).sort(),
};
resolvedPipelines.push(pipelineSummary);
addCheck(
`pipeline:${target.id}:runner`,
runnerId === target.expected_runner_id ? "pass" : "blocked",
{
...pipelineSummary,
reason:
runnerId === target.expected_runner_id
? ""
: `Expected ${target.expected_runner_id}, got ${runnerId || "<missing>"}.`,
},
);
if (
target.require_func_call_model ||
target.require_vision_model ||
(testModels && target.id === "local-agent")
) {
const modelConfig = runnerConfig.model;
const primaryModelId =
typeof modelConfig === "string"
? modelConfig
: modelConfig && typeof modelConfig === "object"
? modelConfig.primary || ""
: "";
if (!primaryModelId) {
addCheck(`pipeline:${target.id}:primary-model`, "blocked", {
...pipelineSummary,
reason: "Local-agent runner config has no primary model.",
});
continue;
}
const model = modelById.get(primaryModelId);
if (!model) {
addCheck(`pipeline:${target.id}:primary-model`, "blocked", {
...pipelineSummary,
model_uuid: primaryModelId,
reason:
"Primary model is not listed by /api/v1/provider/models/llm.",
});
continue;
}
addCheck(`pipeline:${target.id}:primary-model`, "pass", {
...pipelineSummary,
model: {
uuid: model.uuid,
name: model.name,
abilities: model.abilities,
provider_name: model.provider_name,
requester: model.requester,
},
);
});
if (target.require_func_call_model) {
addCheck(
`pipeline:${target.id}:func-call-model`,
model.abilities.includes("func_call") ? "pass" : "env_issue",
{
model_uuid: model.uuid,
model_name: model.name,
abilities: model.abilities,
reason: model.abilities.includes("func_call")
? ""
: "Release gate includes tool-call cases; the local-agent primary model must advertise func_call.",
},
);
}
if (target.require_vision_model) {
addCheck(
`pipeline:${target.id}:vision-model`,
model.abilities.includes("vision") ? "pass" : "env_issue",
{
model_uuid: model.uuid,
model_name: model.name,
abilities: model.abilities,
reason: model.abilities.includes("vision")
? ""
: "Release gate includes multimodal cases; the local-agent primary model must advertise vision.",
},
);
}
if (testModels && !modelTested.has(model.uuid)) {
modelTested.add(model.uuid);
const modelTest = await postJson(
`/api/v1/provider/models/llm/${encodeURIComponent(model.uuid)}/test`,
{ extra_args: {} },
);
const passed =
modelTest.status < 400 && (modelTest.json.code ?? 0) === 0;
addCheck(
`model-test:${model.name}`,
passed ? "pass" : "env_issue",
{
model_uuid: model.uuid,
model_name: model.name,
http_status: modelTest.status,
code: modelTest.json.code ?? null,
reason: passed
? ""
: safeMessage(
modelTest.json.msg ||
modelTest.json.message ||
"Model test failed.",
),
},
);
}
}
}
}
return {
authenticated: true,
blockers,
env_issues: envIssues,
warnings,
checks,
resolved_pipelines: resolvedPipelines,
tools: {
required: ["qa_plugin_echo"],
optional_before_register: ["qa_mcp_echo"],
present: toolNames.filter((name) => ["qa_plugin_echo", "qa_mcp_echo"].includes(name)),
},
models,
};
}, { backendUrl, targets, testModels });
return {
authenticated: true,
blockers,
env_issues: envIssues,
warnings,
checks,
resolved_pipelines: resolvedPipelines,
tools: {
required: ["qa_plugin_echo"],
optional_before_register: ["qa_mcp_echo"],
present: toolNames.filter((name) =>
["qa_plugin_echo", "qa_mcp_echo"].includes(name),
),
},
models,
};
},
{ backendUrl, targets, testModels },
);
diagnostic.blockers = (diagnostic.blockers || []).map((item) => ({ ...item, reason: redactMessage(item.reason || "") }));
diagnostic.env_issues = (diagnostic.env_issues || []).map((item) => ({ ...item, reason: redactMessage(item.reason || "") }));
await writeFile(diagnosticPath, `${JSON.stringify(diagnostic, null, 2)}\n`, "utf8");
diagnostic.blockers = (diagnostic.blockers || []).map((item) => ({
...item,
reason: redactMessage(item.reason || ""),
}));
diagnostic.env_issues = (diagnostic.env_issues || []).map((item) => ({
...item,
reason: redactMessage(item.reason || ""),
}));
await writeFile(
diagnosticPath,
`${JSON.stringify(diagnostic, null, 2)}\n`,
"utf8",
);
await safeScreenshot(page, paths.screenshot);
const blockers = diagnostic.blockers || [];
@@ -447,31 +588,49 @@ async function run() {
result.reason = `Preflight environment issue: ${envIssues.map((item) => item.name).join(", ")}`;
} else {
result.status = "pass";
result.reason = "Release gate preflight passed: auth, plugin runtime, required pipelines, runner ids, tools, and local-agent model checks are ready.";
result.reason =
"Release gate preflight passed: auth, plugin runtime, required pipelines, runner ids, tools, and local-agent model checks are ready.";
}
result.check_count = Array.isArray(diagnostic.checks) ? diagnostic.checks.length : 0;
result.warning_count = Array.isArray(diagnostic.warnings) ? diagnostic.warnings.length : 0;
result.check_count = Array.isArray(diagnostic.checks)
? diagnostic.checks.length
: 0;
result.warning_count = Array.isArray(diagnostic.warnings)
? diagnostic.warnings.length
: 0;
}
try {
await run();
} catch (error) {
const message = redactMessage(error instanceof Error ? error.message : String(error));
const message = redactMessage(
error instanceof Error ? error.message : String(error),
);
result.status = isEnvironmentError(message) ? "env_issue" : "fail";
result.reason = message;
await writeFile(diagnosticPath, `${JSON.stringify({
authenticated: false,
blockers: [],
env_issues: result.status === "env_issue" ? [{ name: "preflight-runtime", reason: message }] : [],
warnings: [],
checks: [
await writeFile(
diagnosticPath,
`${JSON.stringify(
{
name: "preflight-runtime",
status: result.status,
reason: message,
authenticated: false,
blockers: [],
env_issues:
result.status === "env_issue"
? [{ name: "preflight-runtime", reason: message }]
: [],
warnings: [],
checks: [
{
name: "preflight-runtime",
status: result.status,
reason: message,
},
],
},
],
}, null, 2)}\n`, "utf8").catch(() => {});
null,
2,
)}\n`,
"utf8",
).catch(() => {});
} finally {
if (browser) await browser.close().catch(() => {});
const finishedAt = new Date();
+14 -15
View File
@@ -1,4 +1,4 @@
"""Policy helpers for classifying AgentRunner ledger error signals."""
"""Policy helpers for classifying Runner ledger error signals."""
from __future__ import annotations
@@ -7,7 +7,7 @@ import re
_INVALID_TOOL_ARGUMENT_PATTERN = re.compile(
r"invalid json arguments|\b\d+\s+validation errors?\s+for\s+[A-Za-z_][A-Za-z0-9_]*Args\b",
r'invalid json arguments|\b\d+\s+validation errors?\s+for\s+[A-Za-z_][A-Za-z0-9_]*Args\b',
re.IGNORECASE,
)
@@ -19,14 +19,14 @@ def load_ledger_json(value: str | None, *, field: str, failures: list[dict]) ->
try:
return json.loads(value)
except (TypeError, ValueError) as exc:
failures.append({"kind": "invalid_json", "field": field, "reason": str(exc)})
failures.append({'kind': 'invalid_json', 'field': field, 'reason': str(exc)})
return {}
def invalid_tool_argument_error_signal(value: str) -> str:
"""Return the persisted signal for malformed model-supplied tool arguments."""
match = _INVALID_TOOL_ARGUMENT_PATTERN.search(value)
return match.group(0) if match else ""
return match.group(0) if match else ''
def classify_invalid_tool_argument_errors(
@@ -40,15 +40,14 @@ def classify_invalid_tool_argument_errors(
warnings: list[dict] = []
for event in events:
recovered = run_completed and any(
sequence > event["sequence"]
for sequence in successful_tool_completion_sequences
sequence > event['sequence'] for sequence in successful_tool_completion_sequences
)
if recovered:
warnings.append(
{
"kind": "recovered_tool_argument_error",
"event": event,
"reason": "The model continued with a later successful tool call and the run completed.",
'kind': 'recovered_tool_argument_error',
'event': event,
'reason': 'The model continued with a later successful tool call and the run completed.',
}
)
else:
@@ -64,15 +63,15 @@ def classify_tool_authorization(
"""Classify tool names absent from the Host authorization snapshot."""
if not calls:
return [], []
if authorization_mode == "runner-native":
if authorization_mode == 'runner-native':
return [], [
{
"kind": "runner_native_tool_calls",
"calls": calls,
"reason": (
"External runner tool telemetry is not a LangBot Host tool call; "
'kind': 'runner_native_tool_calls',
'calls': calls,
'reason': (
'External runner tool telemetry is not a LangBot Host tool call; '
"the runner's own permission system governs it."
),
}
]
return [{"kind": "unauthorized_tool_calls", "calls": calls}], []
return [{'kind': 'unauthorized_tool_calls', 'calls': calls}], []
@@ -188,11 +188,7 @@ try {
.getByText(/Event Routing|事件路由|イベントルーティング/)
.first()
.waitFor({ timeout: 15_000 });
await page
.getByText(
/Events this adapter can receive|此适配器可接收的事件|このアダプターが受信できるイベント/,
)
.waitFor();
await page.getByText(/Supported events|支持的事件|対応イベント/).waitFor();
await page
.getByText(/Message received|收到消息|メッセージを受信/)
.first()
@@ -216,11 +212,11 @@ try {
);
await page
.getByRole("button", { name: /Test route|测试路由|ルートをテスト/ })
.getByRole("button", { name: /Check route|检查路由|ルートを確認/ })
.click();
await page.getByRole("dialog").waitFor();
await page
.getByRole("button", { name: /Preview route|预览路由|ルートをプレビュー/ })
.getByRole("button", { name: /View match|查看匹配结果|一致結果を確認/ })
.click();
await page
.getByText(/Route matched|已命中路由|ルートに一致しました/)
@@ -231,28 +227,67 @@ try {
.waitFor();
result.visible_signals.push("dry-run-matched", "discard-target");
await page
.getByRole("button", {
name: /Run saved route|运行已保存路由|保存済みルートを実行/,
})
.click();
await page
.getByText(
/saved route ran successfully|已保存路由运行成功|保存済みルートを実行しました/,
)
.waitFor({ timeout: 20_000 });
result.visible_signals.push("test-event-dispatched");
await page
.getByRole("button", { name: /Close|关闭|閉じる/ })
.first()
.click();
await page.getByRole("dialog").waitFor({ state: "hidden" });
await page
.getByText(/Discarded|已丢弃|破棄済み/)
.first()
.waitFor({ timeout: 10_000 });
result.visible_signals.push("route-status-discarded");
const adapterConfigCard = page.locator('[data-slot="card"]').filter({
has: page.getByText(/Adapter Configuration|适配器配置|アダプター設定/, {
exact: true,
}),
});
await adapterConfigCard
.getByRole("button", {
name: /Listen for platform events|监听平台事件|プラットフォームイベントを監視/,
})
.click();
const adapterDialog = page.getByRole("dialog");
await adapterDialog.waitFor();
await adapterDialog
.getByText(/Listening|正在监听|監視中/, { exact: true })
.waitFor({ timeout: 15_000 });
const inboundText = `adapter event ${paths.runId}`;
const inbound = await apiJson(
backendUrl,
`/bots/${encodeURIComponent(botId)}`,
{
method: "POST",
token,
body: {
session_id: `adapter-debug-${paths.runId}`,
session_type: "person",
sender: { id: "adapter-debug-user", name: "Adapter QA" },
message: [{ type: "Plain", text: inboundText }],
},
},
);
result.api.adapter_event_webhook = {
http_status: inbound.status,
code: inbound.json.code ?? null,
};
if (inbound.status >= 400 || inbound.json.code !== 0) {
throw new Error(
inbound.json.msg || "The HTTP Bot adapter rejected the inbound event.",
);
}
await adapterDialog
.getByText(/Message received|收到消息|メッセージ受信/, { exact: true })
.waitFor({ timeout: 15_000 });
await adapterDialog.getByText("message.received", { exact: true }).waitFor();
await adapterDialog.getByText(inboundText, { exact: true }).waitFor();
result.visible_signals.push(
"adapter-event-listening",
"adapter-event-received",
"adapter-event-raw-code",
);
await adapterDialog
.getByRole("button", { name: /Close|关闭|閉じる/ })
.click();
await adapterDialog.waitFor({ state: "hidden" });
const text = await bodyText(page);
if (/\bEBA event\b/.test(text)) {
@@ -308,7 +343,7 @@ try {
}
result.status = "pass";
result.reason =
"Bot event routing, dry-run, synthetic dispatch, and visible route status passed in the WebUI.";
"Bot event routing, dry-run, real adapter input, and visible route status passed in the WebUI.";
} catch (error) {
if (!["blocked", "env_issue"].includes(result.status)) result.status = "fail";
result.reason = result.reason || error.message;
@@ -12,7 +12,7 @@ import {
writeResult,
} from "./lib/langbot-e2e.mjs";
const RUNNER_ID = "plugin:langbot-team/ACPAgentRunner/default";
const RUNNER_ID = "plugin:langbot-team/ACPRunner/default";
const DEFAULT_PIPELINE_NAME = "Agent QA ACP Claude Debug Chat";
const DEFAULT_LOCAL_PASSWORD = "LangBotE2ELocalPass!2026";
const caseId = "ensure-acp-agent-runner-pipeline";
@@ -24,13 +24,18 @@ await ensureEvidence(paths);
const writeEnv = process.argv.includes("--write-env");
const frontendUrl = env.LANGBOT_FRONTEND_URL || "";
const backendUrl = env.LANGBOT_BACKEND_URL || "";
const pipelineName = env.LANGBOT_E2E_CREATE_PIPELINE_NAME || env.LANGBOT_ACP_AGENT_RUNNER_PIPELINE_NAME || DEFAULT_PIPELINE_NAME;
const sshTarget = env.LANGBOT_ACP_AGENT_RUNNER_SSH_TARGET || "yhh@101.34.71.12";
const sshConnectTimeout = env.LANGBOT_ACP_AGENT_RUNNER_SSH_CONNECT_TIMEOUT || "8";
const sshPort = env.LANGBOT_ACP_AGENT_RUNNER_SSH_PORT || "22";
const sshIdentityFile = env.LANGBOT_ACP_AGENT_RUNNER_SSH_IDENTITY_FILE || "";
const sshExtraOptions = env.LANGBOT_ACP_AGENT_RUNNER_SSH_EXTRA_OPTIONS || "";
const remoteWorkspace = env.LANGBOT_ACP_AGENT_RUNNER_REMOTE_WORKSPACE || "/home/yhh/langbot-e2e/acp-workspace";
const pipelineName =
env.LANGBOT_E2E_CREATE_PIPELINE_NAME ||
env.LANGBOT_ACP_RUNNER_PIPELINE_NAME ||
DEFAULT_PIPELINE_NAME;
const sshTarget = env.LANGBOT_ACP_RUNNER_SSH_TARGET || "yhh@101.34.71.12";
const sshConnectTimeout = env.LANGBOT_ACP_RUNNER_SSH_CONNECT_TIMEOUT || "8";
const sshPort = env.LANGBOT_ACP_RUNNER_SSH_PORT || "22";
const sshIdentityFile = env.LANGBOT_ACP_RUNNER_SSH_IDENTITY_FILE || "";
const sshExtraOptions = env.LANGBOT_ACP_RUNNER_SSH_EXTRA_OPTIONS || "";
const remoteWorkspace =
env.LANGBOT_ACP_RUNNER_REMOTE_WORKSPACE ||
"/home/yhh/langbot-e2e/acp-workspace";
const envLocalPath = resolve("skills/.env.local");
const result = {
@@ -64,7 +69,9 @@ try {
const user = env.LANGBOT_E2E_LOGIN_USER || "";
const password = env.LANGBOT_E2E_LOGIN_PASSWORD || DEFAULT_LOCAL_PASSWORD;
if (!user) {
throw new Error("LANGBOT_E2E_LOGIN_USER is required so this setup can create/update the pipeline via backend API.");
throw new Error(
"LANGBOT_E2E_LOGIN_USER is required so this setup can create/update the pipeline via backend API.",
);
}
const auth = await resetAndAuthLocalUser({ backendUrl, user, password });
@@ -116,13 +123,13 @@ try {
if (writeEnv && result.pipeline_id) {
await upsertEnvLocal(envLocalPath, {
LANGBOT_E2E_LOGIN_USER: user,
LANGBOT_ACP_AGENT_RUNNER_SSH_TARGET: sshTarget,
LANGBOT_ACP_AGENT_RUNNER_SSH_PORT: sshPort,
LANGBOT_ACP_AGENT_RUNNER_SSH_IDENTITY_FILE: sshIdentityFile,
LANGBOT_ACP_AGENT_RUNNER_SSH_EXTRA_OPTIONS: sshExtraOptions,
LANGBOT_ACP_AGENT_RUNNER_REMOTE_WORKSPACE: remoteWorkspace,
LANGBOT_ACP_AGENT_RUNNER_PIPELINE_URL: result.pipeline_url,
LANGBOT_ACP_AGENT_RUNNER_PIPELINE_NAME: result.pipeline_name || pipelineName,
LANGBOT_ACP_RUNNER_SSH_TARGET: sshTarget,
LANGBOT_ACP_RUNNER_SSH_PORT: sshPort,
LANGBOT_ACP_RUNNER_SSH_IDENTITY_FILE: sshIdentityFile,
LANGBOT_ACP_RUNNER_SSH_EXTRA_OPTIONS: sshExtraOptions,
LANGBOT_ACP_RUNNER_REMOTE_WORKSPACE: remoteWorkspace,
LANGBOT_ACP_RUNNER_PIPELINE_URL: result.pipeline_url,
LANGBOT_ACP_RUNNER_PIPELINE_NAME: result.pipeline_name || pipelineName,
});
result.wrote_env = true;
}
@@ -133,10 +140,20 @@ try {
console.log(JSON.stringify(result, null, 2));
}
process.exit(result.status === "pass" ? 0 : result.status === "env_issue" ? 2 : 1);
process.exit(
result.status === "pass" ? 0 : result.status === "env_issue" ? 2 : 1,
);
async function ensurePipeline({ backendUrl, token, pipelineName, runnerId, runnerConfig }) {
const pipelineList = await apiJson(backendUrl, "/api/v1/pipelines", { token });
async function ensurePipeline({
backendUrl,
token,
pipelineName,
runnerId,
runnerConfig,
}) {
const pipelineList = await apiJson(backendUrl, "/api/v1/pipelines", {
token,
});
if (isApiFailure(pipelineList)) {
return {
status: "fail",
@@ -155,7 +172,8 @@ async function ensurePipeline({ backendUrl, token, pipelineName, runnerId, runne
token,
body: {
name: pipelineName,
description: "Local QA pipeline for real ACP Claude AgentRunner Debug Chat smoke tests.",
description:
"Local QA pipeline for real ACP Claude Runner Debug Chat smoke tests.",
emoji: "QA",
},
});
@@ -167,7 +185,11 @@ async function ensurePipeline({ backendUrl, token, pipelineName, runnerId, runne
};
}
const pipelineId = createdResponse.json.data?.uuid || "";
const loaded = await apiJson(backendUrl, `/api/v1/pipelines/${encodeURIComponent(pipelineId)}`, { token });
const loaded = await apiJson(
backendUrl,
`/api/v1/pipelines/${encodeURIComponent(pipelineId)}`,
{ token },
);
pipeline = loaded.json.data?.pipeline || null;
created = true;
}
@@ -179,7 +201,11 @@ async function ensurePipeline({ backendUrl, token, pipelineName, runnerId, runne
};
}
const loaded = await apiJson(backendUrl, `/api/v1/pipelines/${encodeURIComponent(pipeline.uuid)}`, { token });
const loaded = await apiJson(
backendUrl,
`/api/v1/pipelines/${encodeURIComponent(pipeline.uuid)}`,
{ token },
);
if (isApiFailure(loaded) || !loaded.json.data?.pipeline) {
return {
status: "fail",
@@ -190,9 +216,15 @@ async function ensurePipeline({ backendUrl, token, pipelineName, runnerId, runne
}
pipeline = loaded.json.data.pipeline;
const config = pipeline.config && typeof pipeline.config === "object" ? pipeline.config : {};
const config =
pipeline.config && typeof pipeline.config === "object"
? pipeline.config
: {};
const ai = config.ai && typeof config.ai === "object" ? config.ai : {};
const runnerConfigs = ai.runner_config && typeof ai.runner_config === "object" ? ai.runner_config : {};
const runnerConfigs =
ai.runner_config && typeof ai.runner_config === "object"
? ai.runner_config
: {};
const updatedConfig = {
...config,
ai: {
@@ -209,16 +241,21 @@ async function ensurePipeline({ backendUrl, token, pipelineName, runnerId, runne
},
};
const updateResponse = await apiJson(backendUrl, `/api/v1/pipelines/${encodeURIComponent(pipeline.uuid)}`, {
method: "PUT",
token,
body: {
name: pipelineName,
description: "Local QA pipeline for real ACP Claude AgentRunner Debug Chat smoke tests.",
emoji: "QA",
config: updatedConfig,
const updateResponse = await apiJson(
backendUrl,
`/api/v1/pipelines/${encodeURIComponent(pipeline.uuid)}`,
{
method: "PUT",
token,
body: {
name: pipelineName,
description:
"Local QA pipeline for real ACP Claude Runner Debug Chat smoke tests.",
emoji: "QA",
config: updatedConfig,
},
},
});
);
if (isApiFailure(updateResponse)) {
return {
status: "fail",
@@ -230,7 +267,9 @@ async function ensurePipeline({ backendUrl, token, pipelineName, runnerId, runne
return {
status: "pass",
reason: created ? "ACP AgentRunner pipeline created and configured." : "ACP AgentRunner pipeline updated.",
reason: created
? "ACP Runner pipeline created and configured."
: "ACP Runner pipeline updated.",
pipeline_id: pipeline.uuid,
pipeline_name: pipelineName,
created,
@@ -239,7 +278,12 @@ async function ensurePipeline({ backendUrl, token, pipelineName, runnerId, runne
}
function isApiFailure(response) {
return response.status >= 400 || (response.json && response.json.code !== undefined && response.json.code !== 0);
return (
response.status >= 400 ||
(response.json &&
response.json.code !== undefined &&
response.json.code !== 0)
);
}
async function upsertEnvLocal(path, values) {
@@ -102,14 +102,16 @@ try {
backend_token_check: auth.check,
};
const pluginSetup = await ensureLocalAgentRunner({
const pluginSetup = await ensureLocalRunner({
backendUrl,
token: auth.token,
});
result.plugin_setup = pluginSetup;
if (pluginSetup.status !== "pass") {
result.status = pluginSetup.status === "env_issue" ? "env_issue" : "fail";
throw new Error(pluginSetup.reason || "Failed to prepare the LocalAgent runner plugin.");
throw new Error(
pluginSetup.reason || "Failed to prepare the LocalAgent runner plugin.",
);
}
const wizard = await skipWizard({ backendUrl, token: auth.token });
@@ -205,7 +207,7 @@ async function skipWizard({ backendUrl, token }) {
};
}
async function ensureLocalAgentRunner({ backendUrl, token }) {
async function ensureLocalRunner({ backendUrl, token }) {
const [author, name] = RUNNER_ID.replace(/^plugin:/, "").split("/");
const existingRunnerIds = await listRunnerIds(backendUrl, token);
if (existingRunnerIds.includes(RUNNER_ID)) {
@@ -264,10 +266,9 @@ async function ensureLocalAgentRunner({ backendUrl, token }) {
};
}
const spaceUrl = String(env.LANGBOT_SPACE_URL || "https://space.langbot.app").replace(
/\/$/,
"",
);
const spaceUrl = String(
env.LANGBOT_SPACE_URL || "https://space.langbot.app",
).replace(/\/$/, "");
let detailResponse;
try {
detailResponse = await fetch(
@@ -376,7 +377,8 @@ async function waitForRunnerRegistration({
}) {
const deadline = Date.now() + timeoutMs;
while (Date.now() < deadline) {
if ((await listRunnerIds(backendUrl, token)).includes(runnerId)) return true;
if ((await listRunnerIds(backendUrl, token)).includes(runnerId))
return true;
await sleep(1000);
}
return false;
@@ -501,8 +503,7 @@ async function ensureLocalAgentPipeline({
token,
body: {
name: pipelineName,
description:
"Local QA pipeline for AgentRunner Debug Chat smoke tests.",
description: "Local QA pipeline for Runner Debug Chat smoke tests.",
emoji: "QA",
},
});
@@ -640,8 +641,7 @@ async function ensureLocalAgentPipeline({
token,
body: {
name: pipelineName,
description:
"Local QA pipeline for AgentRunner Debug Chat smoke tests.",
description: "Local QA pipeline for Runner Debug Chat smoke tests.",
emoji: "QA",
config: updatedConfig,
},
@@ -24,7 +24,10 @@ await ensureEvidence(paths);
const writeEnv = process.argv.includes("--write-env");
const frontendUrl = env.LANGBOT_FRONTEND_URL || "";
const backendUrl = env.LANGBOT_BACKEND_URL || "";
const pipelineName = env.LANGBOT_E2E_CREATE_PIPELINE_NAME || env.LANGBOT_QA_AGENT_RUNNER_PIPELINE_NAME || DEFAULT_PIPELINE_NAME;
const pipelineName =
env.LANGBOT_E2E_CREATE_PIPELINE_NAME ||
env.LANGBOT_QA_RUNNER_PIPELINE_NAME ||
DEFAULT_PIPELINE_NAME;
const envLocalPath = resolve("skills/.env.local");
const result = {
@@ -55,7 +58,9 @@ try {
const user = env.LANGBOT_E2E_LOGIN_USER || "";
const password = env.LANGBOT_E2E_LOGIN_PASSWORD || DEFAULT_LOCAL_PASSWORD;
if (!user) {
throw new Error("LANGBOT_E2E_LOGIN_USER is required so this setup can create/update the pipeline via backend API.");
throw new Error(
"LANGBOT_E2E_LOGIN_USER is required so this setup can create/update the pipeline via backend API.",
);
}
const auth = await resetAndAuthLocalUser({ backendUrl, user, password });
@@ -80,8 +85,8 @@ try {
if (writeEnv && result.pipeline_id) {
await upsertEnvLocal(envLocalPath, {
LANGBOT_E2E_LOGIN_USER: user,
LANGBOT_QA_AGENT_RUNNER_PIPELINE_URL: result.pipeline_url,
LANGBOT_QA_AGENT_RUNNER_PIPELINE_NAME: result.pipeline_name || pipelineName,
LANGBOT_QA_RUNNER_PIPELINE_URL: result.pipeline_url,
LANGBOT_QA_RUNNER_PIPELINE_NAME: result.pipeline_name || pipelineName,
});
result.wrote_env = true;
}
@@ -92,10 +97,20 @@ try {
console.log(JSON.stringify(result, null, 2));
}
process.exit(result.status === "pass" ? 0 : result.status === "env_issue" ? 2 : 1);
process.exit(
result.status === "pass" ? 0 : result.status === "env_issue" ? 2 : 1,
);
async function ensurePipeline({ backendUrl, token, pipelineName, runnerId, runnerConfig }) {
const pipelineList = await apiJson(backendUrl, "/api/v1/pipelines", { token });
async function ensurePipeline({
backendUrl,
token,
pipelineName,
runnerId,
runnerConfig,
}) {
const pipelineList = await apiJson(backendUrl, "/api/v1/pipelines", {
token,
});
if (isApiFailure(pipelineList)) {
return {
status: "fail",
@@ -114,7 +129,8 @@ async function ensurePipeline({ backendUrl, token, pipelineName, runnerId, runne
token,
body: {
name: pipelineName,
description: "Local QA pipeline for deterministic QA AgentRunner Debug Chat smoke tests.",
description:
"Local QA pipeline for deterministic QA Runner Debug Chat smoke tests.",
emoji: "QA",
},
});
@@ -126,7 +142,11 @@ async function ensurePipeline({ backendUrl, token, pipelineName, runnerId, runne
};
}
const pipelineId = createdResponse.json.data?.uuid || "";
const loaded = await apiJson(backendUrl, `/api/v1/pipelines/${encodeURIComponent(pipelineId)}`, { token });
const loaded = await apiJson(
backendUrl,
`/api/v1/pipelines/${encodeURIComponent(pipelineId)}`,
{ token },
);
pipeline = loaded.json.data?.pipeline || null;
created = true;
}
@@ -138,7 +158,11 @@ async function ensurePipeline({ backendUrl, token, pipelineName, runnerId, runne
};
}
const loaded = await apiJson(backendUrl, `/api/v1/pipelines/${encodeURIComponent(pipeline.uuid)}`, { token });
const loaded = await apiJson(
backendUrl,
`/api/v1/pipelines/${encodeURIComponent(pipeline.uuid)}`,
{ token },
);
if (isApiFailure(loaded) || !loaded.json.data?.pipeline) {
return {
status: "fail",
@@ -149,9 +173,15 @@ async function ensurePipeline({ backendUrl, token, pipelineName, runnerId, runne
}
pipeline = loaded.json.data.pipeline;
const config = pipeline.config && typeof pipeline.config === "object" ? pipeline.config : {};
const config =
pipeline.config && typeof pipeline.config === "object"
? pipeline.config
: {};
const ai = config.ai && typeof config.ai === "object" ? config.ai : {};
const runnerConfigs = ai.runner_config && typeof ai.runner_config === "object" ? ai.runner_config : {};
const runnerConfigs =
ai.runner_config && typeof ai.runner_config === "object"
? ai.runner_config
: {};
const updatedConfig = {
...config,
ai: {
@@ -168,16 +198,21 @@ async function ensurePipeline({ backendUrl, token, pipelineName, runnerId, runne
},
};
const updateResponse = await apiJson(backendUrl, `/api/v1/pipelines/${encodeURIComponent(pipeline.uuid)}`, {
method: "PUT",
token,
body: {
name: pipelineName,
description: "Local QA pipeline for deterministic QA AgentRunner Debug Chat smoke tests.",
emoji: "QA",
config: updatedConfig,
const updateResponse = await apiJson(
backendUrl,
`/api/v1/pipelines/${encodeURIComponent(pipeline.uuid)}`,
{
method: "PUT",
token,
body: {
name: pipelineName,
description:
"Local QA pipeline for deterministic QA Runner Debug Chat smoke tests.",
emoji: "QA",
config: updatedConfig,
},
},
});
);
if (isApiFailure(updateResponse)) {
return {
status: "fail",
@@ -189,7 +224,9 @@ async function ensurePipeline({ backendUrl, token, pipelineName, runnerId, runne
return {
status: "pass",
reason: created ? "QA AgentRunner pipeline created and configured." : "QA AgentRunner pipeline updated.",
reason: created
? "QA Runner pipeline created and configured."
: "QA Runner pipeline updated.",
pipeline_id: pipeline.uuid,
pipeline_name: pipelineName,
created,
@@ -198,7 +235,12 @@ async function ensurePipeline({ backendUrl, token, pipelineName, runnerId, runne
}
function isApiFailure(response) {
return response.status >= 400 || (response.json && response.json.code !== undefined && response.json.code !== 0);
return (
response.status >= 400 ||
(response.json &&
response.json.code !== undefined &&
response.json.code !== 0)
);
}
async function upsertEnvLocal(path, values) {
@@ -20,7 +20,10 @@ await loadEnvFiles();
const paths = evidencePaths(caseId);
await ensureEvidence(paths);
const mobileScreenshot = paths.screenshot.replace(/\.png$/, "-mobile.png");
const installedScreenshot = paths.screenshot.replace(/\.png$/, "-installed.png");
const installedScreenshot = paths.screenshot.replace(
/\.png$/,
"-installed.png",
);
const startedAt = new Date();
let frontendUrl = "";
@@ -84,7 +87,7 @@ try {
}
try {
const payload = request.postDataJSON();
if (payload?.component_filter === "AgentRunner") {
if (payload?.component_filter === "Runner") {
result.marketplace_request = {
endpoint: new URL(request.url()).pathname,
component_filter: payload.component_filter,
@@ -98,14 +101,17 @@ try {
});
page.on("response", async (response) => {
const pathname = new URL(response.url()).pathname;
if (!/\/api\/v1\/marketplace\/(extensions|plugins)\/search$/.test(pathname)) {
if (
!/\/api\/v1\/marketplace\/(extensions|plugins)\/search$/.test(pathname)
) {
return;
}
try {
const payload = await response.json();
const entries = payload?.data?.extensions || payload?.data?.plugins || [];
const localAgent = entries.find(
(entry) => `${entry.author}/${entry.name}` === "langbot-team/LocalAgent",
(entry) =>
`${entry.author}/${entry.name}` === "langbot-team/LocalAgent",
);
result.marketplace_response = {
endpoint: pathname,
@@ -228,7 +234,7 @@ try {
});
await browseLink.waitFor();
const href = await browseLink.getAttribute("href");
if (href !== "/home/extensions?type=plugin&component=AgentRunner") {
if (href !== "/home/extensions?type=plugin&component=Runner") {
throw new Error(`Unexpected Runner marketplace URL: ${href}`);
}
const nextButton = page.getByRole("button", {
@@ -238,9 +244,7 @@ try {
throw new Error("Wizard allowed continuing without an installed Runner.");
}
if (!result.marketplace_request) {
throw new Error(
"Wizard did not request the AgentRunner Marketplace catalog.",
);
throw new Error("Wizard did not request the Runner Marketplace catalog.");
}
if (
!result.marketplace_response?.local_agent_present ||
@@ -287,10 +291,14 @@ try {
.then(() => "failed"),
]);
if (installOutcome === "failed") {
throw new Error("LocalAgent installation failed before Runner registration.");
throw new Error(
"LocalAgent installation failed before Runner registration.",
);
}
if (await nextButton.isDisabled()) {
throw new Error("Create & Deploy remained disabled after LocalAgent installation.");
throw new Error(
"Create & Deploy remained disabled after LocalAgent installation.",
);
}
const [installedPluginsResponse, installedMetadataResponse] =
@@ -298,8 +306,7 @@ try {
apiJson(backendUrl, "/api/v1/plugins", { token }),
apiJson(backendUrl, "/api/v1/pipelines/_/metadata", { token }),
]);
const postInstallPlugins =
installedPluginsResponse.json.data?.plugins || [];
const postInstallPlugins = installedPluginsResponse.json.data?.plugins || [];
const installedRunnerStage = installedMetadataResponse.json.data?.configs
?.find((config) => config.name === "ai")
?.stages?.find((stage) => stage.name === "runner");
@@ -338,7 +345,7 @@ try {
}
result.status = "pass";
result.reason =
"A clean first-run instance discovered LocalAgent in the AgentRunner catalog, installed and registered it, selected it, and enabled Create & Deploy.";
"A clean first-run instance discovered LocalAgent in the Runner catalog, installed and registered it, selected it, and enabled Create & Deploy.";
} catch (error) {
if (!["blocked", "env_issue"].includes(result.status)) result.status = "fail";
result.reason = result.reason || error.message;
@@ -42,26 +42,83 @@ const result = {
};
const repositories = [
{ id: "langbot", directory: "LangBot", envKey: "LANGBOT_REPO", manifest: false },
{ id: "plugin-sdk", directory: "langbot-plugin-sdk", envKey: "LANGBOT_PLUGIN_SDK_REPO", manifest: false },
{ id: "agent-runner", directory: "langbot-agent-runner", envKey: "LANGBOT_AGENT_RUNNER_REPO", manifest: false },
{ id: "local-agent", directory: "langbot-local-agent", envKey: "LANGBOT_LOCAL_AGENT_REPO", identity: "langbot-team/LocalAgent" },
{ id: "control-plane", directory: "langbot-agent-control-plane", envKey: "LANGBOT_AGENT_CONTROL_PLANE_REPO", identity: "langbot/agent-control-plane" },
{ id: "longterm-memory", directory: "langbot-longterm-memory", envKey: "LANGBOT_LONGTERM_MEMORY_REPO", identity: "langbot-team/LongTermMemory" },
{ id: "parser", directory: "langbot-parser", envKey: "LANGBOT_PARSER_PLUGIN_REPO", identity: "langbot-team/GeneralParsers" },
{ id: "rag", directory: "langbot-rag", envKey: "LANGBOT_RAG_PLUGIN_REPO", identity: "langbot-team/LangRAG" },
{ id: "skill-authoring", directory: "langbot-skill-authoring", envKey: "LANGBOT_SKILL_AUTHORING_REPO", identity: "huanghuoguoguo/skill-authoring" },
{
id: "langbot",
directory: "LangBot",
envKey: "LANGBOT_REPO",
manifest: false,
},
{
id: "plugin-sdk",
directory: "langbot-plugin-sdk",
envKey: "LANGBOT_PLUGIN_SDK_REPO",
manifest: false,
},
{
id: "agent-runner",
directory: "langbot-agent-runner",
envKey: "LANGBOT_RUNNER_REPO",
manifest: false,
},
{
id: "local-agent",
directory: "langbot-local-agent",
envKey: "LANGBOT_LOCAL_AGENT_REPO",
identity: "langbot-team/LocalAgent",
},
{
id: "control-plane",
directory: "langbot-agent-control-plane",
envKey: "LANGBOT_AGENT_CONTROL_PLANE_REPO",
identity: "langbot/agent-control-plane",
},
{
id: "longterm-memory",
directory: "langbot-longterm-memory",
envKey: "LANGBOT_LONGTERM_MEMORY_REPO",
identity: "langbot-team/LongTermMemory",
},
{
id: "parser",
directory: "langbot-parser",
envKey: "LANGBOT_PARSER_PLUGIN_REPO",
identity: "langbot-team/GeneralParsers",
},
{
id: "rag",
directory: "langbot-rag",
envKey: "LANGBOT_RAG_PLUGIN_REPO",
identity: "langbot-team/LangRAG",
},
{
id: "skill-authoring",
directory: "langbot-skill-authoring",
envKey: "LANGBOT_SKILL_AUTHORING_REPO",
identity: "huanghuoguoguo/skill-authoring",
},
];
function run(command, args, options = {}) {
return new Promise((resolvePromise) => {
const child = spawn(command, args, { ...options, encoding: "utf8", stdio: ["ignore", "pipe", "pipe"] });
const child = spawn(command, args, {
...options,
encoding: "utf8",
stdio: ["ignore", "pipe", "pipe"],
});
let stdout = "";
let stderr = "";
child.stdout.on("data", (chunk) => { stdout += chunk; });
child.stderr.on("data", (chunk) => { stderr += chunk; });
child.on("error", (error) => resolvePromise({ status: null, stdout, stderr, error }));
child.on("close", (status) => resolvePromise({ status, stdout, stderr, error: null }));
child.stdout.on("data", (chunk) => {
stdout += chunk;
});
child.stderr.on("data", (chunk) => {
stderr += chunk;
});
child.on("error", (error) =>
resolvePromise({ status: null, stdout, stderr, error }),
);
child.on("close", (status) =>
resolvePromise({ status, stdout, stderr, error: null }),
);
});
}
@@ -71,31 +128,50 @@ function addCheck(name, status, detail = {}) {
try {
const langbotRepo = await resolveLangBotRepo();
const workspaceRoot = resolve(env.LANGBOT_WORKSPACE_ROOT || dirname(langbotRepo));
const workspaceRoot = resolve(
env.LANGBOT_WORKSPACE_ROOT || dirname(langbotRepo),
);
const resolved = {};
for (const repository of repositories) {
const path = resolve(env[repository.envKey] || (repository.id === "langbot" ? langbotRepo : join(workspaceRoot, repository.directory)));
const path = resolve(
env[repository.envKey] ||
(repository.id === "langbot"
? langbotRepo
: join(workspaceRoot, repository.directory)),
);
resolved[repository.id] = path;
try {
await access(join(path, ".git"));
} catch {
addCheck(`repo:${repository.id}`, "fail", { path, reason: "Git checkout is missing." });
addCheck(`repo:${repository.id}`, "fail", {
path,
reason: "Git checkout is missing.",
});
continue;
}
const branch = await run("git", ["branch", "--show-current"], { cwd: path });
const branch = await run("git", ["branch", "--show-current"], {
cwd: path,
});
const branchName = branch.stdout.trim();
const compatible = branch.status === 0 && /^(?:main|dev\/4\.11\.x)$/.test(branchName);
const compatible =
branch.status === 0 && /^(?:main|dev\/4\.11\.x)$/.test(branchName);
addCheck(`repo:${repository.id}`, compatible ? "pass" : "fail", {
path,
branch: branchName,
reason: compatible ? "" : "Expected main or dev/4.11.x compatibility branch.",
reason: compatible
? ""
: "Expected main or dev/4.11.x compatibility branch.",
});
const dirty = await run("git", ["status", "--short"], { cwd: path });
if (dirty.stdout.trim()) {
result.warnings.push({ name: `dirty:${repository.id}`, path, entries: dirty.stdout.trim().split(/\r?\n/).length });
result.warnings.push({
name: `dirty:${repository.id}`,
path,
entries: dirty.stdout.trim().split(/\r?\n/).length,
});
}
if (repository.identity) {
@@ -105,13 +181,20 @@ try {
const author = manifest.match(/^\s{2}author:\s*([^\s#]+)/m)?.[1] || "";
const name = manifest.match(/^\s{2}name:\s*([^\s#]+)/m)?.[1] || "";
const identity = `${author}/${name}`;
addCheck(`manifest:${repository.id}`, identity === repository.identity ? "pass" : "fail", {
path: manifestPath,
identity,
expected_identity: repository.identity,
});
addCheck(
`manifest:${repository.id}`,
identity === repository.identity ? "pass" : "fail",
{
path: manifestPath,
identity,
expected_identity: repository.identity,
},
);
} catch (error) {
addCheck(`manifest:${repository.id}`, "fail", { path: manifestPath, reason: error.message });
addCheck(`manifest:${repository.id}`, "fail", {
path: manifestPath,
reason: error.message,
});
}
}
}
@@ -121,41 +204,69 @@ try {
await access(python);
addCheck("langbot-venv", "pass", { python });
} catch {
addCheck("langbot-venv", "fail", { python, reason: "LangBot virtualenv Python is missing." });
addCheck("langbot-venv", "fail", {
python,
reason: "LangBot virtualenv Python is missing.",
});
}
const sdkSrc = join(resolved["plugin-sdk"], "src");
const importProbe = await run(python, ["-c", [
"import json, pathlib, langbot_plugin",
"from langbot_plugin.api.entities.builtin.agent_runner.input import AgentInput",
"from langbot_plugin.api.entities.builtin.agent_runner.result import AgentRunResult",
"print(json.dumps({'path': str(pathlib.Path(langbot_plugin.__file__).resolve()), 'entities': [AgentInput.__name__, AgentRunResult.__name__]}))",
].join("; ")], {
cwd: resolved.langbot,
env: { ...env, PYTHONPATH: [sdkSrc, env.PYTHONPATH].filter(Boolean).join(delimiter) },
});
const importProbe = await run(
python,
[
"-c",
[
"import json, pathlib, langbot_plugin",
"from langbot_plugin.api.entities.builtin.runner.input import AgentInput",
"from langbot_plugin.api.entities.builtin.runner.result import RunnerResult",
"print(json.dumps({'path': str(pathlib.Path(langbot_plugin.__file__).resolve()), 'entities': [AgentInput.__name__, RunnerResult.__name__]}))",
].join("; "),
],
{
cwd: resolved.langbot,
env: {
...env,
PYTHONPATH: [sdkSrc, env.PYTHONPATH].filter(Boolean).join(delimiter),
},
},
);
let importDetail = {};
try { importDetail = JSON.parse(importProbe.stdout.trim()); } catch { importDetail = { stderr: importProbe.stderr.trim() }; }
const localSdkLoaded = importProbe.status === 0 && resolve(importDetail.path || "").startsWith(resolve(sdkSrc));
try {
importDetail = JSON.parse(importProbe.stdout.trim());
} catch {
importDetail = { stderr: importProbe.stderr.trim() };
}
const localSdkLoaded =
importProbe.status === 0 &&
resolve(importDetail.path || "").startsWith(resolve(sdkSrc));
addCheck("local-sdk-import", localSdkLoaded ? "pass" : "fail", {
expected_root: resolve(sdkSrc),
...importDetail,
reason: localSdkLoaded ? "" : "langbot_plugin did not load from the workspace SDK source tree.",
reason: localSdkLoaded
? ""
: "langbot_plugin did not load from the workspace SDK source tree.",
});
const failures = result.checks.filter((check) => check.status === "fail");
result.status = failures.length === 0 ? "pass" : "fail";
result.reason = failures.length === 0
? `Workspace compatibility preflight passed with ${result.warnings.length} non-blocking dirty-worktree warning(s).`
: `Workspace compatibility preflight found ${failures.length} blocking check(s).`;
result.reason =
failures.length === 0
? `Workspace compatibility preflight passed with ${result.warnings.length} non-blocking dirty-worktree warning(s).`
: `Workspace compatibility preflight found ${failures.length} blocking check(s).`;
} catch (error) {
result.status = /missing|ENOENT|not found/i.test(error.message) ? "env_issue" : "fail";
result.status = /missing|ENOENT|not found/i.test(error.message)
? "env_issue"
: "fail";
result.reason = error.message;
} finally {
const finishedAt = new Date();
result.finished_at = finishedAt.toISOString();
result.finished_at_local = localIsoWithOffset(finishedAt);
await writeFile(detailsPath, `${JSON.stringify({ checks: result.checks, warnings: result.warnings }, null, 2)}\n`, "utf8");
await writeFile(
detailsPath,
`${JSON.stringify({ checks: result.checks, warnings: result.warnings }, null, 2)}\n`,
"utf8",
);
await writeResult(paths, result);
console.log(JSON.stringify(result, null, 2));
}
+25 -25
View File
@@ -210,7 +210,7 @@
"case_summaries": [
{
"id": "acp-agent-runner-debug-chat",
"title": "ACP AgentRunner can answer through Debug Chat using real remote Claude",
"title": "ACP Runner can answer through Debug Chat using real remote Claude",
"mode": "agent-browser",
"area": "pipeline",
"type": "regression",
@@ -229,8 +229,8 @@
"node:scripts/e2e/ensure-acp-agent-runner-pipeline.mjs --write-env"
],
"setup_provides_env": [
"LANGBOT_ACP_AGENT_RUNNER_PIPELINE_URL",
"LANGBOT_ACP_AGENT_RUNNER_PIPELINE_NAME"
"LANGBOT_ACP_RUNNER_PIPELINE_URL",
"LANGBOT_ACP_RUNNER_PIPELINE_NAME"
],
"evidence_required": [
"ui",
@@ -240,7 +240,7 @@
},
{
"id": "agent-run-ledger-audit",
"title": "Persisted AgentRunner run ledger passes end-to-end invariants",
"title": "Persisted Runner run ledger passes end-to-end invariants",
"mode": "probe",
"area": "agent",
"type": "regression",
@@ -264,7 +264,7 @@
},
{
"id": "agent-runner-async-db-readiness",
"title": "AgentRunner async DB readiness probe",
"title": "Runner async DB readiness probe",
"mode": "probe",
"area": "release",
"type": "smoke",
@@ -286,7 +286,7 @@
},
{
"id": "agent-runner-behavior-matrix",
"title": "AgentRunner deterministic behavior matrix probe",
"title": "Runner deterministic behavior matrix probe",
"mode": "probe",
"area": "release",
"type": "regression",
@@ -308,7 +308,7 @@
},
{
"id": "agent-runner-fixture-contract",
"title": "QA AgentRunner fixture contract probe",
"title": "QA Runner fixture contract probe",
"mode": "probe",
"area": "release",
"type": "regression",
@@ -355,7 +355,7 @@
},
{
"id": "agent-runner-ledger-concurrency",
"title": "AgentRunner run ledger concurrency and auth pytest probe",
"title": "Runner run ledger concurrency and auth pytest probe",
"mode": "probe",
"area": "release",
"type": "regression",
@@ -378,7 +378,7 @@
},
{
"id": "agent-runner-ledger-contention",
"title": "AgentRunner ledger SQLite contention probe",
"title": "Runner ledger SQLite contention probe",
"mode": "probe",
"area": "release",
"type": "regression",
@@ -401,7 +401,7 @@
},
{
"id": "agent-runner-ledger-invariants",
"title": "AgentRunner ledger schema and status invariants probe",
"title": "Runner ledger schema and status invariants probe",
"mode": "probe",
"area": "release",
"type": "regression",
@@ -423,7 +423,7 @@
},
{
"id": "agent-runner-ledger-stress",
"title": "AgentRunner ledger lightweight stress probe",
"title": "Runner ledger lightweight stress probe",
"mode": "probe",
"area": "release",
"type": "regression",
@@ -445,7 +445,7 @@
},
{
"id": "agent-runner-live-install",
"title": "QA AgentRunner package installs and registers in LangBot",
"title": "QA Runner package installs and registers in LangBot",
"mode": "probe",
"area": "release",
"type": "regression",
@@ -468,7 +468,7 @@
},
{
"id": "agent-runner-qa-debug-chat",
"title": "QA AgentRunner returns deterministic output through Debug Chat",
"title": "QA Runner returns deterministic output through Debug Chat",
"mode": "agent-browser",
"area": "pipeline",
"type": "regression",
@@ -487,8 +487,8 @@
"node:scripts/e2e/ensure-qa-agent-runner-pipeline.mjs --write-env"
],
"setup_provides_env": [
"LANGBOT_QA_AGENT_RUNNER_PIPELINE_URL",
"LANGBOT_QA_AGENT_RUNNER_PIPELINE_NAME"
"LANGBOT_QA_RUNNER_PIPELINE_URL",
"LANGBOT_QA_RUNNER_PIPELINE_NAME"
],
"evidence_required": [
"ui",
@@ -526,7 +526,7 @@
},
{
"id": "agent-runner-runtime-chaos",
"title": "AgentRunner SDK runtime chaos pytest probe",
"title": "Runner SDK runtime chaos pytest probe",
"mode": "probe",
"area": "release",
"type": "regression",
@@ -549,7 +549,7 @@
},
{
"id": "bot-event-routing-product-flow",
"title": "Bot event routing can be configured and tested from the WebUI",
"title": "Bot event routing and adapter input can be inspected from the WebUI",
"mode": "agent-browser",
"area": "bot",
"type": "feature",
@@ -657,7 +657,7 @@
},
{
"id": "dify-agent-debug-chat",
"title": "Dify AgentRunner returns a response through Pipeline Debug Chat",
"title": "Dify Runner returns a response through Pipeline Debug Chat",
"mode": "agent-browser",
"area": "pipeline",
"type": "provider",
@@ -1937,7 +1937,7 @@
},
{
"id": "wizard-runner-marketplace-catalog",
"title": "Quick Start installs a published AgentRunner on a clean instance",
"title": "Quick Start installs a published Runner on a clean instance",
"mode": "agent-browser",
"area": "wizard",
"type": "feature",
@@ -2289,7 +2289,7 @@
{
"id": "langbot-workspace-release-gate",
"title": "LangBot workspace top-down release gate",
"description": "Broad release gate combining deterministic repository contracts with representative browser workflows, plugin pages, RAG/parser, EBA, external AgentRunner, and one complex LocalAgent task.",
"description": "Broad release gate combining deterministic repository contracts with representative browser workflows, plugin pages, RAG/parser, EBA, external Runner, and one complex LocalAgent task.",
"type": "release_gate",
"priority": "p0",
"tags": [
@@ -2352,7 +2352,7 @@
"fixtures": [
{
"id": "qa-agent-runner-behaviors",
"title": "Deterministic AgentRunner behavior matrix",
"title": "Deterministic Runner behavior matrix",
"kind": "json",
"path": "fixtures/agent-runner/qa-runner-behaviors.json",
"related_cases": [
@@ -2363,7 +2363,7 @@
},
{
"id": "qa-agent-runner-source",
"title": "QA deterministic AgentRunner fixture source",
"title": "QA deterministic Runner fixture source",
"kind": "plugin_source",
"path": "fixtures/plugins/qa-agent-runner/manifest.yaml",
"related_cases": [
@@ -2375,7 +2375,7 @@
},
{
"id": "qa-agent-runner-package",
"title": "QA deterministic AgentRunner prebuilt package",
"title": "QA deterministic Runner prebuilt package",
"kind": "plugin_package",
"path": "fixtures/plugins/qa-agent-runner/dist/qa-agent-runner-0.1.0.lbpkg",
"related_cases": [
@@ -2486,7 +2486,7 @@
"troubleshooting_summaries": [
{
"id": "agent-runner-actor-context-fields",
"title": "AgentRunner reads old actor.type and actor.id fields",
"title": "Runner reads old actor.type and actor.id fields",
"category": "product",
"related_cases": [
"dify-agent-debug-chat",
@@ -2504,7 +2504,7 @@
},
{
"id": "ambiguous-runner-default-label",
"title": "AgentRunner selector shows multiple Default or 默认 options",
"title": "Runner selector shows multiple Default or 默认 options",
"category": "product",
"related_cases": [
"dify-agent-debug-chat",
+7 -7
View File
@@ -37,10 +37,10 @@ LANGBOT_NO_PROXY=localhost,127.0.0.1,::1
# LANGBOT_PIPELINE_NAME=Generic QA Pipeline
# LANGBOT_LOCAL_AGENT_PIPELINE_URL=http://127.0.0.1:3000/home/pipelines?id=<local-agent-pipeline-uuid>
# LANGBOT_LOCAL_AGENT_PIPELINE_NAME=Local Agent QA Pipeline
# LANGBOT_ACP_AGENT_RUNNER_PIPELINE_URL=http://127.0.0.1:3000/home/pipelines?id=<acp-agent-runner-pipeline-uuid>
# LANGBOT_ACP_AGENT_RUNNER_PIPELINE_NAME=ACP AgentRunner QA Pipeline
# LANGBOT_ACP_AGENT_RUNNER_SSH_TARGET=yhh@101.34.71.12
# LANGBOT_ACP_AGENT_RUNNER_SSH_PORT=22
# LANGBOT_ACP_AGENT_RUNNER_SSH_IDENTITY_FILE=
# LANGBOT_ACP_AGENT_RUNNER_SSH_EXTRA_OPTIONS=
# LANGBOT_ACP_AGENT_RUNNER_REMOTE_WORKSPACE=/home/yhh/langbot-e2e/acp-workspace
# LANGBOT_ACP_RUNNER_PIPELINE_URL=http://127.0.0.1:3000/home/pipelines?id=<acp-agent-runner-pipeline-uuid>
# LANGBOT_ACP_RUNNER_PIPELINE_NAME=ACP Runner QA Pipeline
# LANGBOT_ACP_RUNNER_SSH_TARGET=yhh@101.34.71.12
# LANGBOT_ACP_RUNNER_SSH_PORT=22
# LANGBOT_ACP_RUNNER_SSH_IDENTITY_FILE=
# LANGBOT_ACP_RUNNER_SSH_EXTRA_OPTIONS=
# LANGBOT_ACP_RUNNER_REMOTE_WORKSPACE=/home/yhh/langbot-e2e/acp-workspace
+21 -8
View File
@@ -64,8 +64,11 @@ The tools wrap the LangBot service layer. Current tools (v1):
| --- | --- |
| `get_system_info` | Version, edition, instance id |
| `list_bots` / `get_bot` / `create_bot` / `update_bot` / `delete_bot` | Manage messaging-platform bots (secrets redacted on read) |
| `list_bot_event_route_statuses` / `test_bot_event_route` | Inspect bot event-route runtime status and dispatch a synthetic test event through saved routes without sending real outbound platform messages |
| `list_processors` / `get_processor` / `create_processor` / `update_processor` / `delete_processor` | Manage the peer Agent and Pipeline processor types |
| `list_bot_event_route_statuses` | Inspect bot event-route runtime status |
| `list_processors` / `get_processor` / `create_processor` / `update_processor` / `delete_processor` | Manage the peer Agent, Pipeline and Event processor types |
| `get_processor_metadata` | Discover installed event-capable Runner components, schemas and supported event patterns. |
| `list_processor_runs` / `get_processor_run_events` | Read one Event processor instance run history and logs; paginate with `before_id` / `after_sequence`. |
| `debug_agent` | Execute a synthetic Agent event (`processor_uuid`, `payload`); requires `runtime.operate`. Returns final text and up to 1000 execution events (thinking, text, tool arguments/results). Platform tools use Mock; other configured tools execute normally. Optional `payload.mock`: `errors`/`results` keyed by platform tool name, `unsupported_apis` lists unavailable platform APIs. |
| `list_pipelines` / `get_pipeline` / `create_pipeline` / `update_pipeline` / `delete_pipeline` | Manage pipelines |
| `list_llm_models` / `get_llm_model` / `list_embedding_models` / `list_model_providers` | Inspect models & providers |
| `list_knowledge_bases` / `get_knowledge_base` / `retrieve_knowledge_base` | RAG knowledge bases (incl. semantic search) |
@@ -77,12 +80,8 @@ shape as the corresponding HTTP API request body. Discover resources with the
`list_*` / `get_*` tools before mutating; identifiers are UUIDs. Reads require
`resource.view`; mutations require `resource.manage`. All service calls inherit
the immutable Workspace context authenticated at the MCP transport boundary.
`test_bot_event_route` uses the bot's saved runtime route table, injects a
synthetic event such as `message.received`, and suppresses platform delivery.
It still executes the selected processor, so tools and external services may
have side effects. Use `payload` for sample event fields, for example
`{"message_text": "hello", "chat_type": "private", "chat_id": "u1"}`.
Pass `is_default: true` to `create_pipeline` only when the Workspace does not
already have a default pipeline.
## How to use
@@ -114,3 +113,17 @@ have side effects. Use `payload` for sample event fields, for example
- A `403` means the key is valid but lacks the permission required by the tool.
- The global key is plaintext in config.yaml — only enable it on trusted/internal
deployments and serve over HTTPS.
## Event processors
Create a processor with `kind: "event_processor"` and basic information. Without
a component it supports no events. Discover installed components with
`get_processor_metadata`, then use `update_processor` with `component_ref` and
optional `parameters`. API callers may also supply these when creating an instance. Bind bot events to this instance with `target_type: "event_processor"`
and `target_id` equal to its UUID. Installation alone never activates a handler.
`debug_agent` accepts the complete typed event in `payload.data` for this kind.
Legacy EventListener plugins remain in the Pipeline lifecycle.
`list_processor_runs` includes `created_at_ms`, `started_at_ms`, and
`finished_at_ms`: Host lifecycle times in epoch milliseconds. Use the start and finish times for elapsed processing time; select a run and call `get_processor_run_events` for its
logs and action results. These times are not internal plugin profiling data.
+5 -5
View File
@@ -11,13 +11,13 @@ Use this skill when an agent needs to verify LangBot behavior through the WebUI
- **General WebUI testing**: read `references/web-ui-testing.md`.
- **Pipeline Debug Chat**: read `references/pipeline-debug-chat.md`.
- **Dify AgentRunner**: read `references/dify-agent-runner.md`.
- **Dify Runner**: read `references/dify-agent-runner.md`.
- **Model provider setup or test button**: read `references/model-provider-testing.md`.
- **Plugin install/runtime/tool/page smoke**: read `references/plugin-e2e-smoke.md`.
- **Local Agent Runner**: read `references/local-agent-runner.md`.
- **Local Agent Runner path coverage**: read `references/local-agent-runner-coverage.md`.
- **Diff-aware AgentRunner QA after code changes**: read `references/agent-runner-qa-workflow.md`.
- **Agent Runner release gate**: read `references/agent-runner-release-gate.md`.
- **Local Runner**: read `references/local-agent-runner.md`.
- **Local Runner path coverage**: read `references/local-agent-runner-coverage.md`.
- **Diff-aware Runner QA after code changes**: read `references/agent-runner-qa-workflow.md`.
- **Runner release gate**: read `references/agent-runner-release-gate.md`.
- **Sandbox-backed skill authoring**: read `references/sandbox-skill-authoring.md`.
- **LangRAG knowledge bases**: read `references/langrag-knowledge-base.md`.
- **MCP stdio tool testing**: read `references/mcp-stdio-testing.md`.
@@ -1,5 +1,5 @@
id: acp-agent-runner-debug-chat
title: "ACP AgentRunner can answer through Debug Chat using real remote Claude"
title: "ACP Runner can answer through Debug Chat using real remote Claude"
mode: agent-browser
area: pipeline
type: regression
@@ -19,42 +19,42 @@ env:
- LANGBOT_FRONTEND_URL
- LANGBOT_BACKEND_URL
env_any:
- LANGBOT_ACP_AGENT_RUNNER_PIPELINE_URL|LANGBOT_ACP_AGENT_RUNNER_PIPELINE_NAME
- LANGBOT_ACP_RUNNER_PIPELINE_URL|LANGBOT_ACP_RUNNER_PIPELINE_NAME
automation: scripts/e2e/pipeline-debug-chat.mjs
automation_env:
- LANGBOT_FRONTEND_URL
- LANGBOT_BACKEND_URL
- LANGBOT_BROWSER_PROFILE
- LANGBOT_CHROMIUM_EXECUTABLE
- LANGBOT_ACP_AGENT_RUNNER_PIPELINE_URL
- LANGBOT_ACP_AGENT_RUNNER_PIPELINE_NAME
- LANGBOT_ACP_RUNNER_PIPELINE_URL
- LANGBOT_ACP_RUNNER_PIPELINE_NAME
- LANGBOT_E2E_PROMPT
- LANGBOT_E2E_EXPECTED_TEXT
- LANGBOT_E2E_EXPECTED_RUNNER_ID
- LANGBOT_E2E_RESPONSE_TIMEOUT_MS
automation_pipeline_url_env: LANGBOT_ACP_AGENT_RUNNER_PIPELINE_URL
automation_pipeline_name_env: LANGBOT_ACP_AGENT_RUNNER_PIPELINE_NAME
automation_expected_runner_id: "plugin:langbot-team/ACPAgentRunner/default"
automation_prompt: "Do not launch any background agent, subagent, task, or worker. In this current ACP session, directly call the MCP tool named langbot_get_current_event exactly once and wait for its result. After it returns, reply exactly ACP_AGENT_RUNNER_E2E_OK with no other text."
automation_expected_text: "ACP_AGENT_RUNNER_E2E_OK"
automation_pipeline_url_env: LANGBOT_ACP_RUNNER_PIPELINE_URL
automation_pipeline_name_env: LANGBOT_ACP_RUNNER_PIPELINE_NAME
automation_expected_runner_id: "plugin:langbot-team/ACPRunner/default"
automation_prompt: "Do not launch any background agent, subagent, task, or worker. In this current ACP session, directly call the MCP tool named langbot_get_current_event exactly once and wait for its result. After it returns, reply exactly ACP_RUNNER_E2E_OK with no other text."
automation_expected_text: "ACP_RUNNER_E2E_OK"
automation_response_timeout_ms: "300000"
setup_automation:
- "node:scripts/e2e/ensure-acp-agent-runner-pipeline.mjs --write-env"
setup_provides_env:
- LANGBOT_ACP_AGENT_RUNNER_PIPELINE_URL
- LANGBOT_ACP_AGENT_RUNNER_PIPELINE_NAME
- LANGBOT_ACP_RUNNER_PIPELINE_URL
- LANGBOT_ACP_RUNNER_PIPELINE_NAME
preconditions:
- "The remote machine has a working Claude Code login and can run npx -y @agentclientprotocol/claude-agent-acp."
- "LangBot can non-interactively SSH to the remote machine; the runner opens the MCP reverse tunnel automatically."
steps:
- "Open LANGBOT_FRONTEND_URL."
- "Open the ACP AgentRunner QA pipeline."
- "Confirm the pipeline AI runner is plugin:langbot-team/ACPAgentRunner/default."
- "Open the ACP Runner QA pipeline."
- "Confirm the pipeline AI runner is plugin:langbot-team/ACPRunner/default."
- "Open Debug Chat."
- "Ask the real remote Claude ACP agent to call langbot_get_current_event and return ACP_AGENT_RUNNER_E2E_OK exactly."
- "Ask the real remote Claude ACP agent to call langbot_get_current_event and return ACP_RUNNER_E2E_OK exactly."
checks:
- "UI: Debug Chat shows the user prompt."
- "UI: Debug Chat shows a Bot response containing ACP_AGENT_RUNNER_E2E_OK."
- "UI: Debug Chat shows a Bot response containing ACP_RUNNER_E2E_OK."
- "Logs: Backend logs include Processing request from person_websocket and Streaming completed for this run."
- "Logs: No acp runner request error appears for this run."
- "Console: No unexpected frontend errors appear during Debug Chat."
@@ -66,13 +66,13 @@ diagnostics:
- "Use scripts/e2e/ensure-acp-agent-runner-pipeline.mjs --write-env to create/update the pipeline."
- "For remote Claude on 101, verify ssh yhh@101.34.71.12 can run without password prompts; no separate ssh -R process is required."
success_patterns:
- "ACP_AGENT_RUNNER_E2E_OK"
- "ACP_RUNNER_E2E_OK"
- "Processing request from person_websocket"
- "Streaming completed"
failure_patterns:
- "acp.command_not_found"
- "acp.process_exited"
- "Agent runner plugin:langbot-team/ACPAgentRunner/default execution failed"
- "Agent runner plugin:langbot-team/ACPRunner/default execution failed"
troubleshooting:
- backend-not-listening
- plugin-runtime-timeout
@@ -1,5 +1,5 @@
id: agent-run-ledger-audit
title: "Persisted AgentRunner run ledger passes end-to-end invariants"
title: "Persisted Runner run ledger passes end-to-end invariants"
mode: probe
area: agent
type: regression
@@ -15,7 +15,7 @@ skills:
- langbot-testing
automation: scripts/e2e/agent-run-ledger-audit.mjs
steps:
- "Set LANGBOT_AGENT_RUN_ID to audit a specific run, or leave it unset to audit the latest persisted AgentRunner run."
- "Set LANGBOT_AGENT_RUN_ID to audit a specific run, or leave it unset to audit the latest persisted Runner run."
- "For an external runner's own CLI tools, set LANGBOT_AGENT_TOOL_AUTHORIZATION_MODE=runner-native; keep the default strict mode for Host tool calls."
- "Read the active LangBot database configuration and inspect the selected run and its ordered events."
- "Verify completed terminal state, run.completed, paired tool.call.started/completed events, stable tool names, and monotonic ordering."
@@ -1,5 +1,5 @@
id: agent-runner-async-db-readiness
title: "AgentRunner async DB readiness probe"
title: "Runner async DB readiness probe"
mode: probe
area: release
type: smoke
@@ -1,5 +1,5 @@
id: agent-runner-behavior-matrix
title: "AgentRunner deterministic behavior matrix probe"
title: "Runner deterministic behavior matrix probe"
mode: probe
area: release
type: regression
@@ -1,5 +1,5 @@
id: agent-runner-fixture-contract
title: "QA AgentRunner fixture contract probe"
title: "QA Runner fixture contract probe"
mode: probe
area: release
type: regression
@@ -17,19 +17,19 @@ env:
automation: skills/langbot-testing/probes/agent-runner-fixture-contract.mjs
steps:
- "Run `rtk bin/lbs test run agent-runner-fixture-contract --dry-run` first; remove `--dry-run` after checking the planned evidence directory."
- "Automation imports the QA AgentRunner fixture source and executes normal, streaming, and controlled-failure paths with SDK entities."
- "Automation imports the QA Runner fixture source and executes normal, streaming, and controlled-failure paths with SDK entities."
checks:
- "automation-result.json status is pass."
- "probe-stdout.log contains QA_AGENT_RUNNER_FIXTURE_CONTRACT_OK."
- "Normal input returns QA_AGENT_RUNNER_OK:<input>."
- "probe-stdout.log contains QA_RUNNER_FIXTURE_CONTRACT_OK."
- "Normal input returns QA_RUNNER_OK:<input>."
- "Streaming input emits message.delta chunks and completes."
- "Failure input returns QA_AGENT_RUNNER_CONTROLLED_FAILURE."
- "Failure input returns QA_RUNNER_CONTROLLED_FAILURE."
evidence_required:
- filesystem
diagnostics:
- "This validates the deterministic fixture source contract. It does not prove the plugin package is installed in a live LangBot instance."
success_patterns:
- "QA_AGENT_RUNNER_FIXTURE_CONTRACT_OK"
- "QA_RUNNER_FIXTURE_CONTRACT_OK"
failure_patterns:
- "AssertionError"
- "fixture contract exited"
@@ -25,7 +25,7 @@ automation_env:
- LANGBOT_BROWSER_PROFILE
- LANGBOT_CHROMIUM_EXECUTABLE
preconditions:
- "The plugin runtime is enabled and connected, with at least one AgentRunner registered."
- "The plugin runtime is enabled and connected, with at least one Runner registered."
- "The target is a local test instance where a temporary Agent may be created and deleted."
steps:
- "Read the live plugin runtime status and select a registered runner from Agent metadata."
@@ -1,5 +1,5 @@
id: agent-runner-ledger-concurrency
title: "AgentRunner run ledger concurrency and auth pytest probe"
title: "Runner run ledger concurrency and auth pytest probe"
mode: probe
area: release
type: regression
@@ -1,5 +1,5 @@
id: agent-runner-ledger-contention
title: "AgentRunner ledger SQLite contention probe"
title: "Runner ledger SQLite contention probe"
mode: probe
area: release
type: regression
@@ -1,5 +1,5 @@
id: agent-runner-ledger-invariants
title: "AgentRunner ledger schema and status invariants probe"
title: "Runner ledger schema and status invariants probe"
mode: probe
area: release
type: regression
@@ -1,5 +1,5 @@
id: agent-runner-ledger-stress
title: "AgentRunner ledger lightweight stress probe"
title: "Runner ledger lightweight stress probe"
mode: probe
area: release
type: regression
@@ -1,5 +1,5 @@
id: agent-runner-live-install
title: "QA AgentRunner package installs and registers in LangBot"
title: "QA Runner package installs and registers in LangBot"
mode: probe
area: release
type: regression
@@ -25,7 +25,7 @@ automation_expected_tool: ""
automation_expected_runner_id: "plugin:qa/agent-runner/default"
steps:
- "Run `rtk bin/lbs test run agent-runner-live-install --dry-run` first; remove `--dry-run` only after readiness points at a local test LangBot instance."
- "Automation authenticates the local test user, uploads the QA AgentRunner .lbpkg package, waits for the install task, and reads pipeline metadata."
- "Automation authenticates the local test user, uploads the QA Runner .lbpkg package, waits for the install task, and reads pipeline metadata."
checks:
- "automation-result.json status is pass."
- "/api/v1/plugins lists qa/agent-runner after install."
@@ -1,5 +1,5 @@
id: agent-runner-qa-debug-chat
title: "QA AgentRunner returns deterministic output through Debug Chat"
title: "QA Runner returns deterministic output through Debug Chat"
mode: agent-browser
area: pipeline
type: regression
@@ -17,21 +17,21 @@ skills:
env:
- LANGBOT_FRONTEND_URL
- LANGBOT_BACKEND_URL
- LANGBOT_QA_AGENT_RUNNER_PIPELINE_URL
- LANGBOT_QA_AGENT_RUNNER_PIPELINE_NAME
- LANGBOT_QA_RUNNER_PIPELINE_URL
- LANGBOT_QA_RUNNER_PIPELINE_NAME
automation: scripts/e2e/pipeline-debug-chat.mjs
automation_env:
- LANGBOT_FRONTEND_URL
- LANGBOT_BACKEND_URL
- LANGBOT_BROWSER_PROFILE
- LANGBOT_CHROMIUM_EXECUTABLE
- LANGBOT_QA_AGENT_RUNNER_PIPELINE_URL
- LANGBOT_QA_AGENT_RUNNER_PIPELINE_NAME
automation_pipeline_url_env: LANGBOT_QA_AGENT_RUNNER_PIPELINE_URL
automation_pipeline_name_env: LANGBOT_QA_AGENT_RUNNER_PIPELINE_NAME
- LANGBOT_QA_RUNNER_PIPELINE_URL
- LANGBOT_QA_RUNNER_PIPELINE_NAME
automation_pipeline_url_env: LANGBOT_QA_RUNNER_PIPELINE_URL
automation_pipeline_name_env: LANGBOT_QA_RUNNER_PIPELINE_NAME
automation_expected_runner_id: "plugin:qa/agent-runner/default"
automation_prompt: "hello-live"
automation_expected_text: "QA_AGENT_RUNNER_OK:hello-live"
automation_expected_text: "QA_RUNNER_OK:hello-live"
automation_response_timeout_ms: "120000"
automation_debug_chat_response_p95_ms: "120000"
automation_reset_debug_chat: "1"
@@ -39,17 +39,17 @@ setup_automation:
- "case:agent-runner-live-install"
- "node:scripts/e2e/ensure-qa-agent-runner-pipeline.mjs --write-env"
setup_provides_env:
- LANGBOT_QA_AGENT_RUNNER_PIPELINE_URL
- LANGBOT_QA_AGENT_RUNNER_PIPELINE_NAME
- LANGBOT_QA_RUNNER_PIPELINE_URL
- LANGBOT_QA_RUNNER_PIPELINE_NAME
steps:
- "Open LANGBOT_FRONTEND_URL."
- "Open the pipeline from LANGBOT_QA_AGENT_RUNNER_PIPELINE_URL or LANGBOT_QA_AGENT_RUNNER_PIPELINE_NAME."
- "Open the pipeline from LANGBOT_QA_RUNNER_PIPELINE_URL or LANGBOT_QA_RUNNER_PIPELINE_NAME."
- "Confirm the pipeline AI runner is plugin:qa/agent-runner/default."
- "Open Debug Chat."
- "Send: hello-live."
checks:
- "UI: The user message appears in Debug Chat."
- "UI: A Bot message appears and contains QA_AGENT_RUNNER_OK:hello-live."
- "UI: A Bot message appears and contains QA_RUNNER_OK:hello-live."
- "API diagnostic: pipeline config uses plugin:qa/agent-runner/default."
- "Console: No unexpected frontend runtime errors appear during the send/receive path."
evidence_required:
@@ -62,7 +62,7 @@ diagnostics:
- "This is the deterministic live execution proof that sits after fixture contract and live install."
- "If the runner id mismatch is reported, rerun ensure-qa-agent-runner-pipeline.mjs --write-env."
success_patterns:
- "QA_AGENT_RUNNER_OK:hello-live"
- "QA_RUNNER_OK:hello-live"
failure_patterns:
- "plugin:qa/agent-runner/default execution failed"
- "Action invoke_llm_stream call timed out"
@@ -19,7 +19,7 @@ env:
- LANGBOT_BACKEND_URL
env_any:
- LANGBOT_LOCAL_AGENT_PIPELINE_URL|LANGBOT_LOCAL_AGENT_PIPELINE_NAME
- LANGBOT_ACP_AGENT_RUNNER_PIPELINE_URL|LANGBOT_ACP_AGENT_RUNNER_PIPELINE_NAME
- LANGBOT_ACP_RUNNER_PIPELINE_URL|LANGBOT_ACP_RUNNER_PIPELINE_NAME
automation: scripts/e2e/agent-runner-release-preflight.mjs
automation_env:
- LANGBOT_FRONTEND_URL
@@ -28,24 +28,24 @@ automation_env:
- LANGBOT_CHROMIUM_EXECUTABLE
automation_env_any:
- LANGBOT_LOCAL_AGENT_PIPELINE_URL|LANGBOT_LOCAL_AGENT_PIPELINE_NAME
- LANGBOT_ACP_AGENT_RUNNER_PIPELINE_URL|LANGBOT_ACP_AGENT_RUNNER_PIPELINE_NAME
- LANGBOT_ACP_RUNNER_PIPELINE_URL|LANGBOT_ACP_RUNNER_PIPELINE_NAME
preconditions:
- "LANGBOT_LOCAL_AGENT_PIPELINE_URL or LANGBOT_LOCAL_AGENT_PIPELINE_NAME points to the local-agent release pipeline."
- "LANGBOT_ACP_AGENT_RUNNER_PIPELINE_URL or LANGBOT_ACP_AGENT_RUNNER_PIPELINE_NAME points to the ACP AgentRunner release pipeline."
- "LANGBOT_ACP_RUNNER_PIPELINE_URL or LANGBOT_ACP_RUNNER_PIPELINE_NAME points to the ACP Runner release pipeline."
- "The active browser profile is authenticated for the same LangBot backend."
- "By default the preflight performs a cheap model test for the local-agent primary model; set LANGBOT_PREFLIGHT_TEST_MODELS=0 only when deliberately classifying model credentials outside this run."
steps:
- "Open LANGBOT_FRONTEND_URL with the configured browser profile."
- "Use the browser token to call LangBot backend readiness APIs without printing token values."
- "Check plugin runtime status, Box status, required runner plugins, qa-plugin-smoke, and qa_plugin_echo."
- "Resolve the local-agent and ACP AgentRunner QA pipelines from their case-specific env vars."
- "Resolve the local-agent and ACP Runner QA pipelines from their case-specific env vars."
- "Assert each pipeline uses the expected runner id."
- "Assert the external runner pipeline uses the expected runner id."
- "Assert the local-agent primary model advertises func_call and vision for the full release gate."
- "Run the local-agent primary model test endpoint unless LANGBOT_PREFLIGHT_TEST_MODELS=0."
checks:
- "API diagnostic: api-diagnostic.json has no blockers and no env_issues."
- "API diagnostic: required pipelines resolve to plugin:langbot-team/LocalAgent/default and plugin:langbot-team/ACPAgentRunner/default."
- "API diagnostic: required pipelines resolve to plugin:langbot-team/LocalAgent/default and plugin:langbot-team/ACPRunner/default."
- "API diagnostic: qa_plugin_echo is exposed by /api/v1/tools."
- "API diagnostic: local-agent model check catches invalid credentials or missing func_call/vision before release E2E starts."
- "Secret safety: token values, api keys, and provider secrets are not printed."
@@ -1,5 +1,5 @@
id: agent-runner-runtime-chaos
title: "AgentRunner SDK runtime chaos pytest probe"
title: "Runner SDK runtime chaos pytest probe"
mode: probe
area: release
type: regression
@@ -19,10 +19,10 @@ automation: skills/langbot-testing/probes/agent-runner-runtime-chaos.mjs
steps:
- "Run `rtk bin/lbs test run agent-runner-runtime-chaos --dry-run` first; remove `--dry-run` after checking the SDK repo target and evidence directory."
- "Automation resolves LANGBOT_PLUGIN_SDK_REPO, defaulting to ../../langbot-plugin-sdk when the env var is unset."
- "Automation runs the existing SDK pytest files tests/runtime/plugin/test_mgr_agent_runner.py and tests/runtime/test_pull_api_handlers.py."
- "Automation runs the existing SDK pytest files tests/runtime/plugin/test_mgr_runner.py and tests/runtime/test_pull_api_handlers.py."
checks:
- "automation-result.json status is pass."
- "pytest exit status is 0 for the existing AgentRunner runtime and pull API handler tests."
- "pytest exit status is 0 for the existing Runner runtime and pull API handler tests."
- "pytest-stdout.log and pytest-stderr.log are written under LBS_EVIDENCE_DIR."
evidence_required:
- filesystem
@@ -1,5 +1,5 @@
id: bot-event-routing-product-flow
title: "Bot event routing can be configured and tested from the WebUI"
title: "Bot event routing and adapter input can be inspected from the WebUI"
mode: agent-browser
area: bot
type: feature
@@ -33,16 +33,18 @@ steps:
- "Confirm the adapter capability summary, friendly event name, target, and route status are visible."
- "Confirm overlapping routes and unmatched-event fallback behavior are explained before save."
- "Open Test event route and run a dry-run against the current form."
- "Run the saved runtime route with a synthetic event."
- "Close the dialog and confirm the route card shows the latest discarded status."
- "Open Platform event debugging from Adapter Configuration and send a real inbound event through the HTTP Bot adapter."
- "Confirm the normalized event, raw event code, payload, and latest discarded route status are visible."
checks:
- "UI: A user can choose a channel and add a scenario-labeled behavior during initial Bot creation."
- "UI: Event routing uses user-facing labels and does not require the raw event name in the primary route card."
- "UI: Definite route shadowing and unmatched-event fallback behavior are visible without opening raw logs."
- "UI: Dry-run visibly reports that the route matched the discard processor."
- "UI: Saved-route execution visibly succeeds, explains its side-effect boundary, and updates route status to discarded."
- "UI: Adapter event debugging lives under Adapter Configuration, remains separate from route preview, and starts listening only after the dialog opens."
- "UI: A real adapter event shows its friendly name, raw code, and normalized event data."
- "UI: The route card updates to discarded after the real inbound event is handled."
- "Console: No unexpected frontend errors appear during the flow."
- "Network: Bot, dry-run, route-status, and test-event requests return without 5xx responses."
- "Network: Bot, dry-run, route-status, log, and HTTP Bot webhook requests return without 5xx responses."
- "Cleanup: The temporary Bot is deleted after evidence is collected."
evidence_required:
- ui
@@ -51,7 +53,8 @@ evidence_required:
- api_diagnostic
diagnostics:
- "The fixture deliberately uses the discard processor so the product-flow test cannot invoke a model, tool, or external callback."
- "A passing API call without the visible matched and discarded UI states is not a pass."
- "The adapter dialog observes normalized platform events; it does not simulate route matching."
- "A passing webhook call without the visible adapter event and discarded UI states is not a pass."
troubleshooting:
- backend-not-listening
- proxy-env-mismatch
@@ -45,7 +45,7 @@ steps:
- "Select exactly one Box child whose parent is main.py running from LANGBOT_REPO; abort on zero or multiple matches."
- "Send SIGTERM to that Box child and poll process, Box status, MCP runtime info, and global tools for up to 30 seconds."
- "Without running MCP setup or registration, reset Debug Chat and call qa_mcp_echo with a unique per-run value through the browser."
- "Audit the matching AgentRunner ledger run and require the exact qa_mcp_echo arguments plus the complete tool result text."
- "Audit the matching Runner ledger run and require the exact qa_mcp_echo arguments plus the complete tool result text."
checks:
- "The old Box PID exits and a new Box PID appears under the same LangBot parent."
- "Box returns available=true with at least one active session and managed process."
@@ -1,5 +1,5 @@
id: dify-agent-debug-chat
title: "Dify AgentRunner returns a response through Pipeline Debug Chat"
title: "Dify Runner returns a response through Pipeline Debug Chat"
mode: agent-browser
area: pipeline
type: provider
@@ -18,8 +18,8 @@ skills:
env:
- LANGBOT_FRONTEND_URL
- LANGBOT_BACKEND_URL
- LANGBOT_ACP_AGENT_RUNNER_PIPELINE_URL
- LANGBOT_ACP_AGENT_RUNNER_PIPELINE_NAME
- LANGBOT_ACP_RUNNER_PIPELINE_URL
- LANGBOT_ACP_RUNNER_PIPELINE_NAME
preconditions:
- "An external-harness runner pipeline (e.g. ACP remote claude-code) is configured with langbot-assets-enabled=true so the LangBot MCP gateway is exposed to the harness."
- "The remote harness (claude-code) is reachable and responsive (claude -p returns within the runner timeout)."
@@ -29,10 +29,10 @@ automation_env:
- LANGBOT_FRONTEND_URL
- LANGBOT_BROWSER_PROFILE
- LANGBOT_CHROMIUM_EXECUTABLE
- LANGBOT_ACP_AGENT_RUNNER_PIPELINE_URL
- LANGBOT_ACP_AGENT_RUNNER_PIPELINE_NAME
automation_pipeline_url_env: LANGBOT_ACP_AGENT_RUNNER_PIPELINE_URL
automation_pipeline_name_env: LANGBOT_ACP_AGENT_RUNNER_PIPELINE_NAME
- LANGBOT_ACP_RUNNER_PIPELINE_URL
- LANGBOT_ACP_RUNNER_PIPELINE_NAME
automation_pipeline_url_env: LANGBOT_ACP_RUNNER_PIPELINE_URL
automation_pipeline_name_env: LANGBOT_ACP_RUNNER_PIPELINE_NAME
automation_prompt: "You have LangBot tools available via an MCP server (tools prefixed langbot_). Call langbot_list_assets with asset_types = [\"skills\",\"tools\"]. Then reply with one single line: the literal token PROBEDONE, a space, the number of skills you found, a space, and the number of tools you found."
automation_expected_text: "PROBEDONE"
automation_response_timeout_ms: "540000"
@@ -1,5 +1,5 @@
id: wizard-runner-marketplace-catalog
title: "Quick Start installs a published AgentRunner on a clean instance"
title: "Quick Start installs a published Runner on a clean instance"
mode: agent-browser
area: wizard
type: feature
@@ -27,19 +27,19 @@ preconditions:
steps:
- "Start an isolated first-run instance and confirm zero installed plugins and zero registered runners."
- "Resume Quick Start at the AI Engine step with a temporary disabled Bot."
- "Confirm the browser requests Marketplace plugins with component_filter=AgentRunner."
- "Confirm the browser requests Marketplace plugins with component_filter=Runner."
- "Confirm langbot-team/LocalAgent is published with an installable version and the Runner Extensions link is correct."
- "Install LocalAgent and wait for plugin initialization and AgentRunner registration."
- "Install LocalAgent and wait for plugin initialization and Runner registration."
- "Confirm Create & Deploy is disabled before installation and enabled after LocalAgent is selected."
- "Verify the layout at desktop and mobile widths."
checks:
- "API: The instance has zero installed plugins and zero registered runners."
- "API: The instance wizard status is none."
- "Network: Marketplace search uses component_filter=AgentRunner and type_filter=plugin."
- "Network: Marketplace search uses component_filter=Runner and type_filter=plugin."
- "UI: The AI Engine step displays the published langbot-team/LocalAgent card."
- "Marketplace: LocalAgent includes latest_version so installation can proceed."
- "Runtime: LocalAgent installs and registers plugin:langbot-team/LocalAgent/default."
- "UI: Browse Runner Extensions links to the AgentRunner-filtered market."
- "UI: Browse Runner Extensions links to the Runner-filtered market."
- "UI: Create & Deploy transitions from disabled to enabled only after Runner selection."
- "Console: No unexpected frontend errors appear during the flow."
- "Cleanup: Wizard progress and the temporary Bot are removed."
@@ -18,7 +18,7 @@ steps:
- "Discover the active LangBot checkout and sibling workspace repositories, with LANGBOT_WORKSPACE_ROOT and repository-specific env overrides available for non-default layouts."
- "Verify every checkout is on main or dev/4.11.x and record dirty worktrees as warnings only."
- "Validate plugin manifest identities for LocalAgent, Control Plane, LongTermMemory, GeneralParsers, LangRAG, and Skill Authoring."
- "Use the LangBot virtualenv to import langbot_plugin and AgentRunner Protocol entities from the workspace SDK source tree."
- "Use the LangBot virtualenv to import langbot_plugin and Runner Protocol entities from the workspace SDK source tree."
checks:
- "workspace-preflight.json contains no failed checks."
- "The langbot_plugin import path is inside the discovered langbot-plugin-sdk/src directory."
@@ -16,7 +16,7 @@ skills:
automation: scripts/e2e/workspace-repository-contracts.mjs
steps:
- "Resolve the active LangBot virtualenv and workspace SDK source path."
- "Run tests independently for AgentRunner, Control Plane, LongTermMemory, Parser, RAG, Skill Authoring, LocalAgent, LangBot Agent/Provider, the skills CLI, and SDK runtime contracts."
- "Run tests independently for Runner, Control Plane, LongTermMemory, Parser, RAG, Skill Authoring, LocalAgent, LangBot Agent/Provider, the skills CLI, and SDK runtime contracts."
- "Run SDK packaging blackbox separately so an isolated build dependency network failure is classified as env_issue without masking product test failures."
- "Write per-repository stdout, stderr, status, and duration under repository-contracts/."
checks:
@@ -1,7 +1,7 @@
[
{
"id": "qa-agent-runner-behaviors",
"title": "Deterministic AgentRunner behavior matrix",
"title": "Deterministic Runner behavior matrix",
"kind": "json",
"path": "fixtures/agent-runner/qa-runner-behaviors.json",
"related_cases": [
@@ -13,7 +13,7 @@
},
{
"id": "qa-agent-runner-source",
"title": "QA deterministic AgentRunner fixture source",
"title": "QA deterministic Runner fixture source",
"kind": "plugin_source",
"path": "fixtures/plugins/qa-agent-runner/manifest.yaml",
"related_cases": [
@@ -22,11 +22,11 @@
"agent-runner-live-install",
"agent-runner-qa-debug-chat"
],
"checks": ["exists", "qa_agent_runner_source"]
"checks": ["exists", "qa_runner_source"]
},
{
"id": "qa-agent-runner-package",
"title": "QA deterministic AgentRunner prebuilt package",
"title": "QA deterministic Runner prebuilt package",
"kind": "plugin_package",
"path": "fixtures/plugins/qa-agent-runner/dist/qa-agent-runner-0.1.0.lbpkg",
"related_cases": [
@@ -1,6 +1,6 @@
# QA AgentRunner Fixture
# QA Runner Fixture
Deterministic AgentRunner plugin source used by `langbot-skills` probes and future browser release-gate cases.
Deterministic Runner plugin source used by `langbot-skills` probes and future browser release-gate cases.
Runner id after installation should be:
@@ -10,6 +10,6 @@ plugin:qa/agent-runner/default
Expected behavior:
- normal input returns `QA_AGENT_RUNNER_OK:<input>`
- normal input returns `QA_RUNNER_OK:<input>`
- input containing `stream` emits streaming chunks then completes
- input containing `fail` returns `QA_AGENT_RUNNER_CONTROLLED_FAILURE`
- input containing `fail` returns `QA_RUNNER_CONTROLLED_FAILURE`
@@ -1,4 +1,4 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 64 64" role="img" aria-label="QA AgentRunner icon">
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 64 64" role="img" aria-label="QA Runner icon">
<rect width="64" height="64" rx="12" fill="#111827"/>
<path d="M16 20h32v22H35l-7 8v-8H16z" fill="#22c55e"/>
<path d="M24 30h16" stroke="#111827" stroke-width="4" stroke-linecap="round"/>

Before

Width:  |  Height:  |  Size: 306 B

After

Width:  |  Height:  |  Size: 301 B

@@ -1,39 +0,0 @@
from __future__ import annotations
import typing
from langbot_plugin.api.definition.components.agent_runner.runner import AgentRunner
from langbot_plugin.api.entities.builtin.agent_runner import AgentRunContext, AgentRunResult
from langbot_plugin.api.entities.builtin.provider.message import Message, MessageChunk
class DefaultAgentRunner(AgentRunner):
async def run(
self,
ctx: AgentRunContext,
) -> typing.AsyncGenerator[AgentRunResult, None]:
text = (ctx.input.to_text() or "").strip()
if "fail" in text.lower():
yield AgentRunResult.run_failed(
ctx.run_id,
error="QA_AGENT_RUNNER_CONTROLLED_FAILURE",
code="qa.controlled_failure",
retryable=False,
)
return
content = f"QA_AGENT_RUNNER_OK:{text or 'empty'}"
if "stream" in text.lower():
for chunk in ("QA_", "AGENT_", f"RUNNER_OK:{text}"):
yield AgentRunResult.message_delta(
ctx.run_id,
MessageChunk(role="assistant", content=chunk),
)
yield AgentRunResult.run_completed(ctx.run_id, finish_reason="stop")
return
yield AgentRunResult.run_completed(
ctx.run_id,
Message(role="assistant", content=content),
finish_reason="stop",
)
@@ -0,0 +1,39 @@
from __future__ import annotations
import typing
from langbot_plugin.api.definition.components.runner.runner import Runner
from langbot_plugin.api.entities.builtin.runner import RunnerContext, RunnerResult
from langbot_plugin.api.entities.builtin.provider.message import Message, MessageChunk
class DefaultRunner(Runner):
async def run(
self,
ctx: RunnerContext,
) -> typing.AsyncGenerator[RunnerResult, None]:
text = (ctx.input.to_text() or '').strip()
if 'fail' in text.lower():
yield RunnerResult.run_failed(
ctx.run_id,
error='QA_RUNNER_CONTROLLED_FAILURE',
code='qa.controlled_failure',
retryable=False,
)
return
content = f'QA_RUNNER_OK:{text or "empty"}'
if 'stream' in text.lower():
for chunk in ('QA_', 'AGENT_', f'RUNNER_OK:{text}'):
yield RunnerResult.message_delta(
ctx.run_id,
MessageChunk(role='assistant', content=chunk),
)
yield RunnerResult.run_completed(ctx.run_id, finish_reason='stop')
return
yield RunnerResult.run_completed(
ctx.run_id,
Message(role='assistant', content=content),
finish_reason='stop',
)
@@ -1,5 +1,5 @@
apiVersion: langbot/v1
kind: AgentRunner
kind: Runner
metadata:
name: default
label:
@@ -27,4 +27,4 @@ spec:
execution:
python:
path: default.py
attr: DefaultAgentRunner
attr: DefaultRunner
@@ -3,6 +3,6 @@ from __future__ import annotations
from langbot_plugin.api.definition.plugin import BasePlugin
class QAAgentRunnerPlugin(BasePlugin):
class QARunnerPlugin(BasePlugin):
async def initialize(self) -> None:
self.ready_marker = "qa-agent-runner-ready"
self.ready_marker = 'qa-agent-runner-ready'
@@ -6,20 +6,20 @@ metadata:
repository: https://example.invalid/langbot/qa-agent-runner
version: 0.1.0
description:
en_US: Deterministic AgentRunner fixture for LangBot QA.
zh_Hans: LangBot QA 使用的确定性 AgentRunner 夹具。
en_US: Deterministic Runner fixture for LangBot QA.
zh_Hans: LangBot QA 使用的确定性 Runner 夹具。
label:
en_US: QA AgentRunner
zh_Hans: QA AgentRunner
en_US: QA Runner
zh_Hans: QA Runner
icon: assets/icon.svg
spec:
config: []
components:
AgentRunner:
Runner:
fromDirs:
- path: components/agent_runner/
- path: components/runner/
maxDepth: 1
execution:
python:
path: main.py
attr: QAAgentRunnerPlugin
attr: QARunnerPlugin
@@ -7,7 +7,11 @@ import { delimiter, join, resolve } from "node:path";
import { env } from "node:process";
function timestampSlug(date = new Date()) {
return date.toISOString().replace(/\.\d{3}Z$/, "Z").replace(/[^0-9A-Za-z]+/g, "-").replace(/^-|-$/g, "");
return date
.toISOString()
.replace(/\.\d{3}Z$/, "Z")
.replace(/[^0-9A-Za-z]+/g, "-")
.replace(/^-|-$/g, "");
}
function localIsoWithOffset(date = new Date()) {
@@ -51,7 +55,14 @@ function run(command, timeoutMs, childEnv) {
});
child.on("error", (error) => {
clearTimeout(timeout);
resolveDone({ stdout, stderr, error, timedOut, status: null, signal: null });
resolveDone({
stdout,
stderr,
error,
timedOut,
status: null,
signal: null,
});
});
child.on("close", (status, signal) => {
clearTimeout(timeout);
@@ -66,7 +77,7 @@ import json
import sys
from pathlib import Path
from langbot.pkg.agent.runner.descriptor import AgentRunnerDescriptor
from langbot.pkg.agent.runner.descriptor import RunnerDescriptor
from langbot.pkg.agent.runner.errors import RunnerExecutionError, RunnerProtocolError
from langbot.pkg.agent.runner.result_normalizer import AgentResultNormalizer
@@ -80,10 +91,10 @@ class App:
logger = Logger()
def descriptor():
return AgentRunnerDescriptor(
return RunnerDescriptor(
id='plugin:qa/agent-runner/default',
source='plugin',
label={'en_US': 'QA AgentRunner'},
label={'en_US': 'QA Runner'},
plugin_author='qa',
plugin_name='agent-runner',
runner_name='default',
@@ -139,18 +150,26 @@ async function main() {
const root = resolve(env.LBS_ROOT || process.cwd());
const caseId = "agent-runner-behavior-matrix";
const runId = env.LBS_RUN_ID || `${timestampSlug()}-${caseId}`;
const evidenceDir = resolve(env.LBS_EVIDENCE_DIR || join(root, "reports", "evidence", runId));
const evidenceDir = resolve(
env.LBS_EVIDENCE_DIR || join(root, "reports", "evidence", runId),
);
await mkdir(evidenceDir, { recursive: true });
const startedAt = new Date();
const langbotRepo = resolve(root, env.LANGBOT_REPO || "..");
const sdkRepo = resolve(root, env.LANGBOT_PLUGIN_SDK_REPO || "../../langbot-plugin-sdk");
const sdkRepo = resolve(
root,
env.LANGBOT_PLUGIN_SDK_REPO || "../../langbot-plugin-sdk",
);
const sdkSrc = resolve(sdkRepo, "src");
const fixturePath = resolve(root, "skills/langbot-testing/fixtures/agent-runner/qa-runner-behaviors.json");
const fixturePath = resolve(
root,
"skills/langbot-testing/fixtures/agent-runner/qa-runner-behaviors.json",
);
const stdoutLog = join(evidenceDir, "probe-stdout.log");
const stderrLog = join(evidenceDir, "probe-stderr.log");
const automationResultJson = join(evidenceDir, "automation-result.json");
const resultJson = join(evidenceDir, "result.json");
const timeoutMs = Number(env.LANGBOT_AGENT_RUNNER_PROBE_TIMEOUT_MS || "30000");
const timeoutMs = Number(env.LANGBOT_RUNNER_PROBE_TIMEOUT_MS || "30000");
const command = {
executable: "rtk",
args: [resolve(langbotRepo, ".venv/bin/python"), "-c", script, fixturePath],
@@ -175,7 +194,12 @@ async function main() {
timeout_ms: timeoutMs,
exit_status: null,
signal: null,
evidence: { stdout_log: stdoutLog, stderr_log: stderrLog, automation_result_json: automationResultJson, result_json: resultJson },
evidence: {
stdout_log: stdoutLog,
stderr_log: stderrLog,
automation_result_json: automationResultJson,
result_json: resultJson,
},
evidence_collected: ["filesystem"],
};
try {
@@ -185,7 +209,9 @@ async function main() {
} else {
const proc = await run(command, timeoutMs, {
...process.env,
PYTHONPATH: [sdkSrc, process.env.PYTHONPATH].filter(Boolean).join(delimiter),
PYTHONPATH: [sdkSrc, process.env.PYTHONPATH]
.filter(Boolean)
.join(delimiter),
UV_CACHE_DIR: env.UV_CACHE_DIR || join(evidenceDir, ".uv-cache"),
});
await writeFile(stdoutLog, proc.stdout, "utf8");
@@ -198,7 +224,10 @@ async function main() {
} else if (proc.timedOut) {
result.status = "fail";
result.reason = `behavior matrix timed out after ${timeoutMs}ms`;
} else if (proc.status === 0 && proc.stdout.includes("QA_RUNNER_BEHAVIOR_MATRIX_OK")) {
} else if (
proc.status === 0 &&
proc.stdout.includes("QA_RUNNER_BEHAVIOR_MATRIX_OK")
) {
result.status = "pass";
result.reason = "behavior matrix passed";
} else {
@@ -219,7 +248,9 @@ async function main() {
await writeFile(resultJson, resultText, "utf8");
console.log(JSON.stringify(result, null, 2));
}
process.exit(result.status === "pass" ? 0 : result.status === "env_issue" ? 2 : 1);
process.exit(
result.status === "pass" ? 0 : result.status === "env_issue" ? 2 : 1,
);
}
await main();
@@ -7,7 +7,11 @@ import { delimiter, join, resolve } from "node:path";
import { env } from "node:process";
function timestampSlug(date = new Date()) {
return date.toISOString().replace(/\.\d{3}Z$/, "Z").replace(/[^0-9A-Za-z]+/g, "-").replace(/^-|-$/g, "");
return date
.toISOString()
.replace(/\.\d{3}Z$/, "Z")
.replace(/[^0-9A-Za-z]+/g, "-")
.replace(/^-|-$/g, "");
}
function localIsoWithOffset(date = new Date()) {
@@ -51,7 +55,14 @@ function run(command, timeoutMs, childEnv) {
});
child.on("error", (error) => {
clearTimeout(timeout);
resolveDone({ stdout, stderr, error, timedOut, status: null, signal: null });
resolveDone({
stdout,
stderr,
error,
timedOut,
status: null,
signal: null,
});
});
child.on("close", (status, signal) => {
clearTimeout(timeout);
@@ -66,27 +77,27 @@ import importlib.util
import sys
from pathlib import Path
from langbot_plugin.api.entities.builtin.agent_runner.context import AgentRunContext
from langbot_plugin.api.entities.builtin.agent_runner.delivery import DeliveryContext
from langbot_plugin.api.entities.builtin.agent_runner.event import AgentEventContext
from langbot_plugin.api.entities.builtin.agent_runner.input import AgentInput
from langbot_plugin.api.entities.builtin.agent_runner.resources import AgentResources
from langbot_plugin.api.entities.builtin.agent_runner.runtime import AgentRuntimeContext
from langbot_plugin.api.entities.builtin.agent_runner.trigger import AgentTrigger
from langbot_plugin.api.entities.builtin.runner.context import RunnerContext
from langbot_plugin.api.entities.builtin.runner.delivery import DeliveryContext
from langbot_plugin.api.entities.builtin.runner.event import AgentEventContext
from langbot_plugin.api.entities.builtin.runner.input import AgentInput
from langbot_plugin.api.entities.builtin.runner.resources import AgentResources
from langbot_plugin.api.entities.builtin.runner.runtime import AgentRuntimeContext
from langbot_plugin.api.entities.builtin.runner.trigger import AgentTrigger
fixture = Path(sys.argv[1])
runner_py = fixture / "components" / "agent_runner" / "default.py"
runner_py = fixture / "components" / "runner" / "default.py"
manifest = fixture / "manifest.yaml"
runner_yaml = fixture / "components" / "agent_runner" / "default.yaml"
runner_yaml = fixture / "components" / "runner" / "default.yaml"
assert manifest.exists(), manifest
assert runner_yaml.exists(), runner_yaml
spec = importlib.util.spec_from_file_location("qa_agent_runner_fixture", runner_py)
spec = importlib.util.spec_from_file_location("qa_runner_fixture", runner_py)
module = importlib.util.module_from_spec(spec)
assert spec and spec.loader
spec.loader.exec_module(module)
def context(run_id, text):
return AgentRunContext(
return RunnerContext(
run_id=run_id,
trigger=AgentTrigger(type="message.received", source="webui"),
event=AgentEventContext(event_id=f"evt-{run_id}", event_type="message.received", source="webui"),
@@ -97,7 +108,7 @@ def context(run_id, text):
)
async def collect(text):
runner = module.DefaultAgentRunner()
runner = module.DefaultRunner()
results = []
async for result in runner.run(context(f"run-{len(text)}", text)):
results.append(result)
@@ -107,17 +118,17 @@ async def main():
normal = await collect("hello")
assert len(normal) == 1, normal
assert normal[0].type.value == "run.completed"
assert normal[0].data["message"]["content"] == "QA_AGENT_RUNNER_OK:hello"
assert normal[0].data["message"]["content"] == "QA_RUNNER_OK:hello"
stream = await collect("stream hello")
assert [item.type.value for item in stream] == ["message.delta", "message.delta", "message.delta", "run.completed"]
assert "".join(item.data["chunk"]["content"] for item in stream[:3]) == "QA_AGENT_RUNNER_OK:stream hello"
assert "".join(item.data["chunk"]["content"] for item in stream[:3]) == "QA_RUNNER_OK:stream hello"
failed = await collect("please fail")
assert len(failed) == 1
assert failed[0].type.value == "run.failed"
assert failed[0].data["error"] == "QA_AGENT_RUNNER_CONTROLLED_FAILURE"
print("QA_AGENT_RUNNER_FIXTURE_CONTRACT_OK")
assert failed[0].data["error"] == "QA_RUNNER_CONTROLLED_FAILURE"
print("QA_RUNNER_FIXTURE_CONTRACT_OK")
asyncio.run(main())
`;
@@ -126,18 +137,30 @@ async function main() {
const root = resolve(env.LBS_ROOT || process.cwd());
const caseId = "agent-runner-fixture-contract";
const runId = env.LBS_RUN_ID || `${timestampSlug()}-${caseId}`;
const evidenceDir = resolve(env.LBS_EVIDENCE_DIR || join(root, "reports", "evidence", runId));
const evidenceDir = resolve(
env.LBS_EVIDENCE_DIR || join(root, "reports", "evidence", runId),
);
await mkdir(evidenceDir, { recursive: true });
const startedAt = new Date();
const sdkRepo = resolve(root, env.LANGBOT_PLUGIN_SDK_REPO || "../../langbot-plugin-sdk");
const sdkRepo = resolve(
root,
env.LANGBOT_PLUGIN_SDK_REPO || "../../langbot-plugin-sdk",
);
const sdkSrc = resolve(sdkRepo, "src");
const fixturePath = resolve(root, "skills/langbot-testing/fixtures/plugins/qa-agent-runner");
const fixturePath = resolve(
root,
"skills/langbot-testing/fixtures/plugins/qa-agent-runner",
);
const stdoutLog = join(evidenceDir, "probe-stdout.log");
const stderrLog = join(evidenceDir, "probe-stderr.log");
const automationResultJson = join(evidenceDir, "automation-result.json");
const resultJson = join(evidenceDir, "result.json");
const timeoutMs = Number(env.LANGBOT_AGENT_RUNNER_PROBE_TIMEOUT_MS || "30000");
const command = { executable: "rtk", args: ["uv", "run", "--no-sync", "python", "-c", script, fixturePath], cwd: sdkRepo };
const timeoutMs = Number(env.LANGBOT_RUNNER_PROBE_TIMEOUT_MS || "30000");
const command = {
executable: "rtk",
args: ["uv", "run", "--no-sync", "python", "-c", script, fixturePath],
cwd: sdkRepo,
};
const result = {
source: "automation",
probe: "agent-runner-fixture-contract",
@@ -156,7 +179,12 @@ async function main() {
timeout_ms: timeoutMs,
exit_status: null,
signal: null,
evidence: { stdout_log: stdoutLog, stderr_log: stderrLog, automation_result_json: automationResultJson, result_json: resultJson },
evidence: {
stdout_log: stdoutLog,
stderr_log: stderrLog,
automation_result_json: automationResultJson,
result_json: resultJson,
},
evidence_collected: ["filesystem"],
};
try {
@@ -166,7 +194,9 @@ async function main() {
} else {
const proc = await run(command, timeoutMs, {
...process.env,
PYTHONPATH: [sdkSrc, process.env.PYTHONPATH].filter(Boolean).join(delimiter),
PYTHONPATH: [sdkSrc, process.env.PYTHONPATH]
.filter(Boolean)
.join(delimiter),
UV_CACHE_DIR: env.UV_CACHE_DIR || join(evidenceDir, ".uv-cache"),
});
await writeFile(stdoutLog, proc.stdout, "utf8");
@@ -179,9 +209,12 @@ async function main() {
} else if (proc.timedOut) {
result.status = "fail";
result.reason = `fixture contract probe timed out after ${timeoutMs}ms`;
} else if (proc.status === 0 && proc.stdout.includes("QA_AGENT_RUNNER_FIXTURE_CONTRACT_OK")) {
} else if (
proc.status === 0 &&
proc.stdout.includes("QA_RUNNER_FIXTURE_CONTRACT_OK")
) {
result.status = "pass";
result.reason = "QA AgentRunner fixture contract passed";
result.reason = "QA Runner fixture contract passed";
} else {
result.status = "fail";
result.reason = `fixture contract exited with status ${proc.status}`;
@@ -200,7 +233,9 @@ async function main() {
await writeFile(resultJson, resultText, "utf8");
console.log(JSON.stringify(result, null, 2));
}
process.exit(result.status === "pass" ? 0 : result.status === "env_issue" ? 2 : 1);
process.exit(
result.status === "pass" ? 0 : result.status === "env_issue" ? 2 : 1,
);
}
await main();
@@ -8,7 +8,8 @@ await runPytestProbe({
defaultRepo: "..",
pythonPathEnvKeys: ["LANGBOT_PLUGIN_SDK_REPO"],
defaultPythonPaths: ["../../langbot-plugin-sdk/src"],
description: "LangBot AgentRunner run ledger claim, lease, authorization, and runtime-admin pytest probe.",
description:
"LangBot Runner run ledger claim, lease, authorization, and runtime-admin pytest probe.",
testTargets: [
"tests/unit_tests/agent/test_run_ledger_store.py::test_create_queued_run_claim_renew_release",
"tests/unit_tests/agent/test_run_ledger_store.py::test_expired_claim_can_be_reclaimed",
@@ -7,7 +7,11 @@ import { delimiter, join, resolve } from "node:path";
import { env } from "node:process";
function timestampSlug(date = new Date()) {
return date.toISOString().replace(/\.\d{3}Z$/, "Z").replace(/[^0-9A-Za-z]+/g, "-").replace(/^-|-$/g, "");
return date
.toISOString()
.replace(/\.\d{3}Z$/, "Z")
.replace(/[^0-9A-Za-z]+/g, "-")
.replace(/^-|-$/g, "");
}
function localIsoWithOffset(date = new Date()) {
@@ -51,7 +55,14 @@ function run(command, timeoutMs, childEnv) {
});
child.on("error", (error) => {
clearTimeout(timeout);
resolveDone({ stdout, stderr, error, timedOut, status: null, signal: null });
resolveDone({
stdout,
stderr,
error,
timedOut,
status: null,
signal: null,
});
});
child.on("close", (status, signal) => {
clearTimeout(timeout);
@@ -149,18 +160,23 @@ async function main() {
const root = resolve(env.LBS_ROOT || process.cwd());
const caseId = "agent-runner-ledger-contention";
const runId = env.LBS_RUN_ID || `${timestampSlug()}-${caseId}`;
const evidenceDir = resolve(env.LBS_EVIDENCE_DIR || join(root, "reports", "evidence", runId));
const evidenceDir = resolve(
env.LBS_EVIDENCE_DIR || join(root, "reports", "evidence", runId),
);
await mkdir(evidenceDir, { recursive: true });
const startedAt = new Date();
const langbotRepo = resolve(root, env.LANGBOT_REPO || "..");
const sdkRepo = resolve(root, env.LANGBOT_PLUGIN_SDK_REPO || "../../langbot-plugin-sdk");
const sdkRepo = resolve(
root,
env.LANGBOT_PLUGIN_SDK_REPO || "../../langbot-plugin-sdk",
);
const sdkSrc = resolve(sdkRepo, "src");
const dbPath = join(evidenceDir, "ledger-contention.sqlite3");
const stdoutLog = join(evidenceDir, "probe-stdout.log");
const stderrLog = join(evidenceDir, "probe-stderr.log");
const automationResultJson = join(evidenceDir, "automation-result.json");
const resultJson = join(evidenceDir, "result.json");
const timeoutMs = Number(env.LANGBOT_AGENT_RUNNER_PROBE_TIMEOUT_MS || "30000");
const timeoutMs = Number(env.LANGBOT_RUNNER_PROBE_TIMEOUT_MS || "30000");
const command = {
executable: "rtk",
args: [resolve(langbotRepo, ".venv/bin/python"), "-c", script, dbPath],
@@ -185,7 +201,13 @@ async function main() {
timeout_ms: timeoutMs,
exit_status: null,
signal: null,
evidence: { stdout_log: stdoutLog, stderr_log: stderrLog, database: dbPath, automation_result_json: automationResultJson, result_json: resultJson },
evidence: {
stdout_log: stdoutLog,
stderr_log: stderrLog,
database: dbPath,
automation_result_json: automationResultJson,
result_json: resultJson,
},
evidence_collected: ["filesystem"],
};
try {
@@ -195,7 +217,9 @@ async function main() {
} else {
const proc = await run(command, timeoutMs, {
...process.env,
PYTHONPATH: [sdkSrc, process.env.PYTHONPATH].filter(Boolean).join(delimiter),
PYTHONPATH: [sdkSrc, process.env.PYTHONPATH]
.filter(Boolean)
.join(delimiter),
UV_CACHE_DIR: env.UV_CACHE_DIR || join(evidenceDir, ".uv-cache"),
});
await writeFile(stdoutLog, proc.stdout, "utf8");
@@ -208,7 +232,10 @@ async function main() {
} else if (proc.timedOut) {
result.status = "fail";
result.reason = `ledger contention timed out after ${timeoutMs}ms`;
} else if (proc.status === 0 && proc.stdout.includes("LEDGER_CONTENTION_OK")) {
} else if (
proc.status === 0 &&
proc.stdout.includes("LEDGER_CONTENTION_OK")
) {
result.status = "pass";
result.reason = "ledger contention probe passed";
} else {
@@ -229,7 +256,9 @@ async function main() {
await writeFile(resultJson, resultText, "utf8");
console.log(JSON.stringify(result, null, 2));
}
process.exit(result.status === "pass" ? 0 : result.status === "env_issue" ? 2 : 1);
process.exit(
result.status === "pass" ? 0 : result.status === "env_issue" ? 2 : 1,
);
}
await main();
@@ -7,7 +7,11 @@ import { delimiter, join, resolve } from "node:path";
import { env } from "node:process";
function timestampSlug(date = new Date()) {
return date.toISOString().replace(/\.\d{3}Z$/, "Z").replace(/[^0-9A-Za-z]+/g, "-").replace(/^-|-$/g, "");
return date
.toISOString()
.replace(/\.\d{3}Z$/, "Z")
.replace(/[^0-9A-Za-z]+/g, "-")
.replace(/^-|-$/g, "");
}
function localIsoWithOffset(date = new Date()) {
@@ -55,7 +59,14 @@ function runProcess(command, timeoutMs, childEnv) {
});
child.on("error", (error) => {
clearTimeout(timeout);
resolveDone({ stdout, stderr, error, timedOut, status: null, signal: null });
resolveDone({
stdout,
stderr,
error,
timedOut,
status: null,
signal: null,
});
});
child.on("close", (status, signal) => {
clearTimeout(timeout);
@@ -124,11 +135,16 @@ async function main() {
const root = resolve(env.LBS_ROOT || process.cwd());
const caseId = "agent-runner-ledger-invariants";
const runId = env.LBS_RUN_ID || `${timestampSlug()}-${caseId}`;
const evidenceDir = resolve(env.LBS_EVIDENCE_DIR || join(root, "reports", "evidence", runId));
const evidenceDir = resolve(
env.LBS_EVIDENCE_DIR || join(root, "reports", "evidence", runId),
);
await mkdir(evidenceDir, { recursive: true });
const startedAt = new Date();
const langbotRepo = resolveFromRoot(root, env.LANGBOT_REPO || "..");
const sdkRepo = resolveFromRoot(root, env.LANGBOT_PLUGIN_SDK_REPO || "../../langbot-plugin-sdk");
const sdkRepo = resolveFromRoot(
root,
env.LANGBOT_PLUGIN_SDK_REPO || "../../langbot-plugin-sdk",
);
const sdkSrc = resolve(sdkRepo, "src");
const stdoutLog = join(evidenceDir, "probe-stdout.log");
const stderrLog = join(evidenceDir, "probe-stderr.log");
@@ -139,7 +155,7 @@ async function main() {
args: [resolve(langbotRepo, ".venv/bin/python"), "-c", probeScript],
cwd: langbotRepo,
};
const timeoutMs = Number(env.LANGBOT_AGENT_RUNNER_PROBE_TIMEOUT_MS || "30000");
const timeoutMs = Number(env.LANGBOT_RUNNER_PROBE_TIMEOUT_MS || "30000");
const result = {
source: "automation",
probe: "python-sync",
@@ -174,7 +190,9 @@ async function main() {
} else {
const childEnv = {
...process.env,
PYTHONPATH: [sdkSrc, process.env.PYTHONPATH].filter(Boolean).join(delimiter),
PYTHONPATH: [sdkSrc, process.env.PYTHONPATH]
.filter(Boolean)
.join(delimiter),
UV_CACHE_DIR: env.UV_CACHE_DIR || join(evidenceDir, ".uv-cache"),
};
await mkdir(childEnv.UV_CACHE_DIR, { recursive: true });
@@ -210,7 +228,9 @@ async function main() {
await writeFile(resultJson, resultText, "utf8");
console.log(JSON.stringify(result, null, 2));
}
process.exit(result.status === "pass" ? 0 : result.status === "env_issue" ? 2 : 1);
process.exit(
result.status === "pass" ? 0 : result.status === "env_issue" ? 2 : 1,
);
}
await main();
@@ -7,7 +7,11 @@ import { delimiter, join, resolve } from "node:path";
import { env } from "node:process";
function timestampSlug(date = new Date()) {
return date.toISOString().replace(/\.\d{3}Z$/, "Z").replace(/[^0-9A-Za-z]+/g, "-").replace(/^-|-$/g, "");
return date
.toISOString()
.replace(/\.\d{3}Z$/, "Z")
.replace(/[^0-9A-Za-z]+/g, "-")
.replace(/^-|-$/g, "");
}
function localIsoWithOffset(date = new Date()) {
@@ -51,7 +55,14 @@ function run(command, timeoutMs, childEnv) {
});
child.on("error", (error) => {
clearTimeout(timeout);
resolveDone({ stdout, stderr, error, timedOut, status: null, signal: null });
resolveDone({
stdout,
stderr,
error,
timedOut,
status: null,
signal: null,
});
});
child.on("close", (status, signal) => {
clearTimeout(timeout);
@@ -116,17 +127,22 @@ async function main() {
const root = resolve(env.LBS_ROOT || process.cwd());
const caseId = "agent-runner-ledger-stress";
const runId = env.LBS_RUN_ID || `${timestampSlug()}-${caseId}`;
const evidenceDir = resolve(env.LBS_EVIDENCE_DIR || join(root, "reports", "evidence", runId));
const evidenceDir = resolve(
env.LBS_EVIDENCE_DIR || join(root, "reports", "evidence", runId),
);
await mkdir(evidenceDir, { recursive: true });
const startedAt = new Date();
const langbotRepo = resolve(root, env.LANGBOT_REPO || "..");
const sdkRepo = resolve(root, env.LANGBOT_PLUGIN_SDK_REPO || "../../langbot-plugin-sdk");
const sdkRepo = resolve(
root,
env.LANGBOT_PLUGIN_SDK_REPO || "../../langbot-plugin-sdk",
);
const sdkSrc = resolve(sdkRepo, "src");
const stdoutLog = join(evidenceDir, "probe-stdout.log");
const stderrLog = join(evidenceDir, "probe-stderr.log");
const automationResultJson = join(evidenceDir, "automation-result.json");
const resultJson = join(evidenceDir, "result.json");
const timeoutMs = Number(env.LANGBOT_AGENT_RUNNER_PROBE_TIMEOUT_MS || "30000");
const timeoutMs = Number(env.LANGBOT_RUNNER_PROBE_TIMEOUT_MS || "30000");
const command = {
executable: "rtk",
args: [resolve(langbotRepo, ".venv/bin/python"), "-c", script],
@@ -150,7 +166,12 @@ async function main() {
timeout_ms: timeoutMs,
exit_status: null,
signal: null,
evidence: { stdout_log: stdoutLog, stderr_log: stderrLog, automation_result_json: automationResultJson, result_json: resultJson },
evidence: {
stdout_log: stdoutLog,
stderr_log: stderrLog,
automation_result_json: automationResultJson,
result_json: resultJson,
},
evidence_collected: ["filesystem"],
};
try {
@@ -160,7 +181,9 @@ async function main() {
} else {
const proc = await run(command, timeoutMs, {
...process.env,
PYTHONPATH: [sdkSrc, process.env.PYTHONPATH].filter(Boolean).join(delimiter),
PYTHONPATH: [sdkSrc, process.env.PYTHONPATH]
.filter(Boolean)
.join(delimiter),
UV_CACHE_DIR: env.UV_CACHE_DIR || join(evidenceDir, ".uv-cache"),
});
await writeFile(stdoutLog, proc.stdout, "utf8");
@@ -173,7 +196,10 @@ async function main() {
} else if (proc.timedOut) {
result.status = "fail";
result.reason = `ledger stress timed out after ${timeoutMs}ms`;
} else if (proc.status === 0 && proc.stdout.includes("LEDGER_STRESS_OK")) {
} else if (
proc.status === 0 &&
proc.stdout.includes("LEDGER_STRESS_OK")
) {
result.status = "pass";
result.reason = "ledger stress probe passed";
} else {
@@ -194,7 +220,9 @@ async function main() {
await writeFile(resultJson, resultText, "utf8");
console.log(JSON.stringify(result, null, 2));
}
process.exit(result.status === "pass" ? 0 : result.status === "env_issue" ? 2 : 1);
process.exit(
result.status === "pass" ? 0 : result.status === "env_issue" ? 2 : 1,
);
}
await main();
@@ -6,9 +6,10 @@ await runPytestProbe({
caseId: "agent-runner-runtime-chaos",
repoEnvKey: "LANGBOT_PLUGIN_SDK_REPO",
defaultRepo: "../../langbot-plugin-sdk",
description: "LangBot plugin SDK AgentRunner runtime failure, timeout, forwarding, and pull API pytest probe.",
description:
"LangBot plugin SDK Runner runtime failure, timeout, forwarding, and pull API pytest probe.",
testTargets: [
"tests/runtime/plugin/test_mgr_agent_runner.py",
"tests/runtime/plugin/test_mgr_runner.py",
"tests/runtime/test_pull_api_handlers.py",
],
});
@@ -5,7 +5,10 @@ import { basename, delimiter, join, resolve } from "node:path";
import { env } from "node:process";
function loadEnvDefaults(root) {
for (const path of [join(root, "skills/.env"), join(root, "skills/.env.local")]) {
for (const path of [
join(root, "skills/.env"),
join(root, "skills/.env.local"),
]) {
if (!existsSync(path)) continue;
for (const rawLine of readFileSync(path, "utf8").split(/\r?\n/)) {
const line = rawLine.trim();
@@ -14,13 +17,20 @@ function loadEnvDefaults(root) {
if (sep === -1) continue;
const key = line.slice(0, sep).trim();
if (env[key]) continue;
env[key] = line.slice(sep + 1).trim().replace(/^["']|["']$/g, "");
env[key] = line
.slice(sep + 1)
.trim()
.replace(/^["']|["']$/g, "");
}
}
}
function timestampSlug(date = new Date()) {
return date.toISOString().replace(/\.\d{3}Z$/, "Z").replace(/[^0-9A-Za-z]+/g, "-").replace(/^-|-$/g, "");
return date
.toISOString()
.replace(/\.\d{3}Z$/, "Z")
.replace(/[^0-9A-Za-z]+/g, "-")
.replace(/^-|-$/g, "");
}
function localIsoWithOffset(date = new Date()) {
@@ -88,7 +98,14 @@ async function runProcess(command, timeoutMs, childEnv) {
});
child.on("error", (error) => {
clearTimeout(timeout);
resolveDone({ stdout, stderr, error, timedOut, status: null, signal: null });
resolveDone({
stdout,
stderr,
error,
timedOut,
status: null,
signal: null,
});
});
child.on("close", (status, signal) => {
clearTimeout(timeout);
@@ -109,10 +126,14 @@ export async function runPytestProbe({
}) {
const root = resolve(env.LBS_ROOT || process.cwd());
loadEnvDefaults(root);
const resolvedTimeoutMs = Number(timeoutMs || env.LANGBOT_AGENT_RUNNER_PROBE_TIMEOUT_MS || "180000");
const resolvedTimeoutMs = Number(
timeoutMs || env.LANGBOT_RUNNER_PROBE_TIMEOUT_MS || "180000",
);
const runId = env.LBS_RUN_ID || `${timestampSlug()}-${caseId}`;
const evidenceDir = resolve(env.LBS_EVIDENCE_DIR || join(root, "reports", "evidence", runId));
const evidenceDir = resolve(
env.LBS_EVIDENCE_DIR || join(root, "reports", "evidence", runId),
);
await mkdir(evidenceDir, { recursive: true });
const uvCacheDir = env.UV_CACHE_DIR || join(evidenceDir, ".uv-cache");
await mkdir(uvCacheDir, { recursive: true });
@@ -171,14 +192,21 @@ export async function runPytestProbe({
result.status = "env_issue";
result.reason = `${repoEnvKey || "repo"} did not resolve to an existing directory: ${repoPath}`;
} else {
const missingTargets = testTargets.filter((target) => !existsSync(join(repoPath, target.split("::")[0])));
const missingTargets = testTargets.filter(
(target) => !existsSync(join(repoPath, target.split("::")[0])),
);
if (missingTargets.length > 0) {
result.status = "env_issue";
result.reason = `pytest target file(s) not found in ${basename(repoPath)}: ${missingTargets.join(", ")}`;
} else {
const childEnv = { ...process.env, UV_CACHE_DIR: uvCacheDir };
if (pythonPaths.length > 0) {
childEnv.PYTHONPATH = [pythonPaths.join(delimiter), childEnv.PYTHONPATH].filter(Boolean).join(delimiter);
childEnv.PYTHONPATH = [
pythonPaths.join(delimiter),
childEnv.PYTHONPATH,
]
.filter(Boolean)
.join(delimiter);
}
const proc = await runProcess(command, resolvedTimeoutMs, childEnv);
result.exit_status = proc.status;
@@ -195,7 +223,11 @@ export async function runPytestProbe({
} else if (proc.status === 0) {
result.status = "pass";
result.reason = `pytest passed for ${testTargets.join(", ")}.`;
} else if (/command not found|no such file or directory|executable file not found/i.test(`${proc.stdout}\n${proc.stderr}`)) {
} else if (
/command not found|no such file or directory|executable file not found/i.test(
`${proc.stdout}\n${proc.stderr}`,
)
) {
result.status = "env_issue";
result.reason = `pytest command could not run in ${repoPath}. See ${stdoutLog} and ${stderrLog}.`;
} else {
@@ -1,6 +1,6 @@
# AgentRunner QA Workflow
# Runner QA Workflow
Use this workflow when an agent finishes AgentRunner-related code and enters a
Use this workflow when an agent finishes Runner-related code and enters a
test phase.
## Order
@@ -25,7 +25,7 @@ test phase.
backend is available and installing the QA fixture is acceptable.
- `rtk bin/lbs test run agent-runner-qa-debug-chat --dry-run` when WebUI live
execution needs deterministic coverage without a model provider. This
case runs its setup automation first: install the QA AgentRunner fixture,
case runs its setup automation first: install the QA Runner fixture,
create/update the QA pipeline, write the case-specific pipeline env, then
execute Debug Chat.
- `rtk bin/lbs test run agent-runner-ledger-invariants --dry-run`
@@ -51,8 +51,8 @@ only to review or adjust the generated list.
| --- | --- | --- |
| `LangBot/src/langbot/pkg/agent/runner/*`, `tests/unit_tests/agent/test_result_normalizer.py`, protocol/result/context/resource builders | `rtk bin/lbs test run agent-runner-fixture-contract --dry-run`; `rtk bin/lbs test run agent-runner-behavior-matrix --dry-run`; targeted LangBot unit tests for touched files | Result shape, user-visible runner output, or Debug Chat delivery changed: add `pipeline-debug-chat` or `local-agent-basic-debug-chat`. |
| `LangBot/src/langbot/pkg/entity/persistence/agent_run.py`, `run_journal.py`, run ledger store/API/auth tests, claim/lease/status code | `rtk bin/lbs test run agent-runner-ledger-invariants --dry-run`; `rtk bin/lbs test run agent-runner-ledger-stress --dry-run`; `rtk bin/lbs test run agent-runner-ledger-contention --dry-run`; `rtk bin/lbs test run agent-runner-async-db-readiness --dry-run` before `rtk bin/lbs test run agent-runner-ledger-concurrency --dry-run` | Debug Chat run lifecycle, resume, or visible completion changed: add `local-agent-basic-debug-chat`. |
| `langbot-plugin-sdk/src/langbot_plugin/api/entities/builtin/agent_runner/*`, `api/proxies/agent_run_api.py`, runtime pull handlers, plugin manager/runtime IO | `rtk bin/lbs test run agent-runner-runtime-chaos --dry-run`; `rtk bin/lbs test run agent-runner-behavior-matrix --dry-run`; targeted SDK pytest | Runtime delivery or tool-call surface changed: add `agent-runner-release-preflight`, then `local-agent-basic-debug-chat`. |
| `langbot-agent-runner/*/components/agent_runner/*`, external runner daemon/client code, ACP/Codex/Claude runner command wrappers | Repo-local targeted tests; `rtk bin/lbs test run agent-runner-runtime-chaos --dry-run`; `rtk bin/lbs test run agent-runner-release-preflight --dry-run` | ACP or external coding runner behavior changed: add `acp-agent-runner-debug-chat`. |
| `langbot-plugin-sdk/src/langbot_plugin/api/entities/builtin/runner/*`, `api/proxies/agent_run_api.py`, runtime pull handlers, plugin manager/runtime IO | `rtk bin/lbs test run agent-runner-runtime-chaos --dry-run`; `rtk bin/lbs test run agent-runner-behavior-matrix --dry-run`; targeted SDK pytest | Runtime delivery or tool-call surface changed: add `agent-runner-release-preflight`, then `local-agent-basic-debug-chat`. |
| `langbot-agent-runner/*/components/runner/*`, external runner daemon/client code, ACP/Codex/Claude runner command wrappers | Repo-local targeted tests; `rtk bin/lbs test run agent-runner-runtime-chaos --dry-run`; `rtk bin/lbs test run agent-runner-release-preflight --dry-run` | ACP or external coding runner behavior changed: add `acp-agent-runner-debug-chat`. |
| Prompt preprocessing, effective prompt, pipeline AI config, runner binding/default runner migration | `rtk bin/lbs test run agent-runner-behavior-matrix --dry-run`; targeted LangBot pipeline/agent tests | The runner reads host-provided prompt or saved runner config: add `local-agent-effective-prompt-debug-chat`. |
| Context window, transcript, history/event state, compaction, checkpoint/steering | `rtk bin/lbs test run agent-runner-behavior-matrix --dry-run`; targeted LangBot agent state/context tests | Multi-turn memory, compaction, or steering behavior changed: add `local-agent-context-compaction-debug-chat` and, for steering-specific changes, `local-agent-steering-debug-chat`. |
| Plugin tool authorization, host tool listing, MCP tool bridge, function-call conversion | `rtk bin/lbs test run agent-runner-behavior-matrix --dry-run`; targeted plugin/MCP/tool tests | Tool execution is user-visible: add `local-agent-plugin-tool-call-debug-chat`; for MCP-specific changes add `mcp-stdio-register` then `mcp-stdio-tool-call`. |

Some files were not shown because too many files have changed in this diff Show More