xinyin025/LangBot
1
0
Fork 0
mirror of https://github.com/langbot-app/LangBot.git synced 2026-07-17 18:06:06 +00:00
Code Issues Packages Projects Releases Wiki Activity

feat(agent-runner): enforce 4.x host-owned execution

Browse Source
This commit is contained in:
huanghuoguoguo
2026-07-12 20:36:32 +08:00
parent e6384aae5d
commit 99d9c227f9
171 changed files with 6958 additions and 5385 deletions
Download Patch File Download Diff File Expand all files Collapse all files
AGENTS.md
+1 -1
View File
@@ -83,7 +83,7 @@ Config keys to verify in `data/config.yaml` / `src/langbot/templates/config.yaml
## Change Rules
- HTTP API changes that should be agent-accessible must update the matching MCP tool in `src/langbot/pkg/api/mcp/server.py` and the relevant skill under `skills/` in the same pass.
- New schema changes use Alembic under `src/langbot/pkg/persistence/alembic/versions/`; do not add legacy `dbmXXX` migrations.
- New schema changes use Alembic under `src/langbot/pkg/persistence/alembic/versions/`. LangBot 4.x does not support upgrading 3.x databases.
- New platform behavior belongs in platform adapters only for platform translation; pipeline/business logic belongs in `pkg/pipeline/` or services.
- User-facing strings must support i18n (`en_US`, `zh_Hans`; include `ja_JP` where the repo already does).
- Code comments and docstrings must be English.
ARCHITECTURE.md
+11 -10
View File
@@ -52,12 +52,13 @@ LangBot/
│ │ ├── api/ # HTTP API + MCP server mount
│ │ ├── platform/ # IM adapters and runtime bot manager
│ │ ├── pipeline/ # Message routing and pipeline stages
│ │ ├── provider/ # LLM runners, model manager, tools
│ │ ├── provider/ # Model providers and Host-owned tools
│ │ ├── agent/ # Agent/AgentRunner orchestration and run state
│ │ ├── plugin/ # LangBot-side Plugin Runtime connector/handler
│ │ ├── box/ # LangBot-side Box service/connector
│ │ ├── skill/ # Skill metadata/activation integration
│ │ ├── rag/ , vector/ # Knowledge-base and vector DB integration
│ │ ├── persistence/ # SQLAlchemy/SQLModel, Alembic, legacy migrations
│ │ ├── persistence/ # SQLAlchemy/SQLModel and Alembic migrations
│ │ ├── storage/ # Local/S3 file storage abstraction
│ │ └── config/, entity/, utils/, telemetry/, survey/
│ ├── libs/ # Vendored third-party platform SDKs
@@ -80,7 +81,7 @@ Platform adapter
→ Controller
→ RuntimePipeline
→ PipelineStage chain
→ RequestRunner / ToolManager / PluginRuntimeConnector / BoxService
→ AgentRunner orchestrator / ToolManager / PluginRuntimeConnector / BoxService
→ response via adapter
```
@@ -107,7 +108,7 @@ Inbound platform messages enter through adapter-specific SDK callbacks. The comm
3. `MessageAggregator` batches/normalizes messages before adding a `Query` to `QueryPool`.
4. `Controller` in `pkg/pipeline/controller.py` selects queries subject to global pipeline concurrency and per-session concurrency.
5. `RuntimePipeline` in `pkg/pipeline/pipelinemgr.py` runs configured pipeline stages using a responsibility-chain style executor that supports generator stages.
6. The chat stage emits plugin events, calls a configured `RequestRunner`, handles streaming/non-streaming responses, records telemetry, and appends conversation history.
6. The chat stage emits plugin events and projects the current query into the AgentRunner Host orchestrator. The selected plugin AgentRunner returns streaming or final results while the Host owns authorization, tools, telemetry, and conversation history.
7. Output stages send text, cards, chunks, files, or error notices back through the original platform adapter.
Pipeline components are registered by decorators and package import side effects. When adding a new stage, loader, runner, or adapter, check the corresponding preregistration mechanism instead of inventing a second registry.
@@ -136,12 +137,12 @@ Important pieces:
Pipelines are configuration-driven. Prefer adding a stage or extending an existing stage family over hard-coding behavior in platform adapters.
## Provider, RAG, and Tools
## Agents, Providers, RAG, and Tools
Provider code lives under `pkg/provider/`.
Agent orchestration lives under `pkg/agent/`; model providers and tools live under `pkg/provider/`.
- `modelmgr/` manages configured model providers and requesters.
- `runners/` implements request runners such as the local agent runner and external workflow integrations.
- `pkg/agent/runner/` discovers plugin AgentRunner components, resolves bindings, constructs run-scoped context/resources, and records execution state.
- `tools/toolmgr.py` aggregates tools from native tools, plugin tools, external MCP servers, and skill-authoring tools.
- `tools/loaders/mcp.py` is the MCP client side: external MCP servers that LangBot connects to for agent tools.
- RAG lives across `pkg/rag/`, `pkg/vector/`, model services, and plugin KnowledgeEngine actions.
@@ -206,8 +207,8 @@ Persistence is centered on `pkg/persistence/mgr.py`.
- SQLite is the default database; PostgreSQL is supported.
- Models live under `pkg/entity/persistence/`.
- Fresh schemas are created from metadata, then legacy migrations run up to the frozen 3.x baseline, then Alembic migrations run to head.
- New schema changes should use Alembic under `pkg/persistence/alembic/versions/`; do not extend the frozen legacy migration chain.
- Fresh schemas are created from current metadata, then Alembic migrations run to head. LangBot 4.x does not upgrade 3.x databases.
- New schema changes use Alembic under `pkg/persistence/alembic/versions/`; there is no legacy migration chain in 4.x.
Configuration starts from `src/langbot/templates/config.yaml` and is generated into `data/config.yaml` on first run. Most long-lived managers read from `ap.instance_config.data`.
@@ -239,7 +240,7 @@ When one of these changes, update the others if the behavior or contract changed
- New LLM tool source: extend `pkg/provider/tools/loaders/` and `ToolManager` intentionally.
- New plugin component/API/protocol: change `langbot-plugin-sdk` first or in lockstep, then update LangBot bridge code.
- New Box capability: change both `pkg/box/` and `langbot-plugin-sdk/src/langbot_plugin/box/`, plus config and tests.
- New database schema: add an Alembic migration, not a legacy `dbmXXX` migration.
- New database schema: add an Alembic migration.
## Design Biases
docs/agent-runner-pluginization/EVENT_BASED_AGENT.md
+12 -3
View File
@@ -80,13 +80,22 @@ EBA 后 `action.requested`（PROTOCOL_V1 §7.3，当前仅 telemetry 不执行
Host 必须校验：binding / platform action policy 是否授权该 action、actor / bot / workspace 是否允许、是否需要人工审批，以及当前 run session / caller identity 是否匹配。EBA 还可能预留 `delivery.requested`（请求投递到某 surface）。
Delivery 方面，event 不一定回复到当前聊天窗口：消息事件通常带 reply target；系统事件可能没有默认 reply target，需要 runner 返回 `action.requested` 或由 binding 的 delivery policy 决定投递位置（`DeliveryContext` 见 PROTOCOL_V1 §5.7）。
当前 Host 会把 adapter 声明的通用 API 投影到
`DeliveryContext.platform_capabilities.supported_apis`，并据此设置
`supports_edit` / `supports_reaction`。该投影只供 runner 选择输出形态，不构成
平台动作授权；合成测试 adapter 会移除副作用能力并抑制实际出站调用。
## 7. 与 Context 协议的关系
EBA 事件进入 AgentRunner 时仍遵循 [AGENT_CONTEXT_PROTOCOL.md](./AGENT_CONTEXT_PROTOCOL.md)：inline 当前事件、大 payload 用 raw/staged file ref、不默认 inline 完整 history、agent 按需通过 API 拉取、Host 保留 EventLog 和权限 guardrail。非消息事件可以被投影进 Transcript，但不能强制伪装为 user message；AgentRunner 根据 event type 自己决定是否纳入模型上下文。
## 8. EBA 分支联调内容
## 8. 当前集成状态
外部 EBA 分支负责联调 EventGateway 完整实现、Pipeline / Agent 处理器路由、AgentBindingResolver 集成、事件绑定持久模型和 UI、`DeliveryContext` 完整实现、platform action permission model 和执行器、真实平台事件接入。
当前分支已完成 EventRouter、Pipeline / Agent 平级处理器路由、Bot
`event_bindings` 持久化与 WebUI、AgentBinding 投影、路由 dry-run、合成测试事件、
运行状态和真实 OneBot 非消息事件到 Agent 的闭环。Pipeline 消息链和独立 Agent
均复用同一个 AgentRunner orchestrator / context / result 协议。
当前底座已完成：① Pipeline 消息链可投影 `message.received` 并在 AI Stage 复用 AgentRunner → ② 独立 Agent 可从 EBA 路由直接生成 `AgentBinding` → ③ context builder 从 event + binding 构造 runner context → ④ 引入 EventLog / Transcript。外部 EBA 分支在此基础上联调真实事件来源、处理器路由、binding persistence / UI 和 platform action。
尚未落地的是 platform action permission model 和 `action.requested` 执行器；在显式
action allowlist、binding policy、adapter capability 和审批模型完成前，该 result 仍只
记录 telemetry，不执行平台副作用。
docs/agent-runner-pluginization/PROTOCOL_V1.md
+5
View File
@@ -295,6 +295,11 @@ class DeliveryContext(BaseModel):
```
Runner 可参考 delivery 能力决定返回 `message.delta`、`message.completed` 或 `action.requested`。
平台事件进入独立 Agent 时，Host 会从当前 adapter 的 `get_supported_apis()` 投影
`supports_edit`、`supports_reaction`，并把去重后的 API 名称写入
`platform_capabilities.supported_apis`。这些字段只描述当前投递表面的能力，不授予
平台动作权限；`action.requested` 仍受 §7.3 的 reserved 约束。合成路由测试使用的
adapter 会过滤所有已知副作用 API，因此测试事件不会向 runner 宣告真实出站能力。
### 5.8 ContextAccess
docs/agent-runner-pluginization/STATUS.md
+4 -3
View File
@@ -2,7 +2,7 @@
本文档是 `docs/agent-runner-pluginization/` 的状态事实源。协议 schema 仍以 [PROTOCOL_V1.md](./PROTOCOL_V1.md) 为准；测试步骤以 [AGENT_RUNNER_QA_GUIDE.md](./AGENT_RUNNER_QA_GUIDE.md) 为准；安全发布门槛以 [SECURITY_HARDENING.md](./SECURITY_HARDENING.md) 为准。
状态快照日期：2026-06-23。
状态快照日期：2026-07-12。
## 实现状态
@@ -20,11 +20,12 @@
| Security boundary | Done | 当前口径降级为轻量边界：LangBot 保护自身持有资源；external harness 的 OS / process / network / workspace 风险由用户或部署环境承担；managed sandbox 不是当前承诺。 |
| Steering control path | Done | claim 异常不再逃逸 consumer loop；queue 有上限；未 pull 的 claimed 输入在 run 结束时写 `steering.dropped` 审计终态。 |
| SDK v1 contract closure | Done | SDK 提供 `AgentAPIError` / `AgentAPIException`、typed `SteeringPullResult`、未知 result type 宽容解析、result `sequence` 注入与取消传播。 |
| EBA processor routing | Done; clean-instance catalog gate pending | Bot `event_bindings`、Pipeline / Agent 平级路由、WebUI dry-run / 合成测试 / 状态、OneBot 非消息事件到 Agent 及平台回复已闭环；全新实例 Runner Marketplace 用例仍需独立空白环境。 |
## Spec 与实现已知差距
- `action.requested` 仍只作为 telemetry / reserved surface；platform action executor 不在本分支执行。
- EventGateway / EventRouter 完整实现由外部 EBA 分支联调；本分支只提供 event-first host envelope / binding / run 入口。
- `action.requested` 权限模型完成前，DeliveryContext 的 adapter capability 投影只用于输出决策，不提供平台动作执行权限。
- State 与 storage 的长期类型边界仍可继续收窄；当前合同只要求 JSON-safe state 与受控 storage API。
- `ToolResource.parameters` 已作为 best-effort full schema 由 Host 在构造 `ctx.resources` 时一次塞齐；无 schema 时 runner 仍需兼容 `parameters=None` 或按需调用 detail API。
- EventLog / Transcript 已提供显式 cleanup primitive；长期 retention 默认值、TTL 调度接入和 sandbox/workspace 文件清理仍是运维收尾项，应在 Runtime Control Plane 产品化前补齐。
@@ -44,7 +45,7 @@
| 范围 | 状态 | 最近证据 |
| --- | --- | --- |
| LangBot Runtime Control Plane v2 foundation | Unit-pass; product E2E pending | 2026-06-23 `tests/unit_tests/agent`、`tests/unit_tests/plugin/test_handler_actions.py`、`tests/unit_tests/provider/test_skill_tools.py`、pipeline preproc/chat handler tests 和 Telegram EBA adapter tests 通过，覆盖 ledger、admin permissions、runtime heartbeat、claim/reconcile、orchestrator 持久化、取消传播、skill activation persistence 和插件化 runner pipeline path。 |
| LangBot Runtime Control Plane v2 foundation | Unit-pass; EBA product flow pass | 2026-07-12 事件路由与 Agent 协议针对性测试通过；WebUI 已验证 Quick Start 场景筛选、事件路由 dry-run / 合成派发、Runner 健康状态，以及真实 OneBot `group.member_joined` → Agent → `send_group_msg` 链路。clean-instance Runner Marketplace 用例因当前实例已有插件与 runner 未执行。 |
| SDK AgentRunner control entities / proxy | Unit-pass | 2026-06-23 SDK `tests/api/entities/builtin/agent_runner`、`tests/api/proxies`、`tests/api/test_agent_tools_mcp_bridge.py`、`tests/runtime/plugin/test_mgr_agent_runner.py`、`tests/runtime/test_pull_api_handlers.py`、`tests/runtime/io/handlers/test_plugin_handler.py`、EBA event entities 和 message tests 通过，覆盖 typed entities、AgentRunAPIProxy、MCP bridge、runtime manager 与 pull API handlers。 |
## 历史高价值记录
docs/event-based-agents/00-overview.md
+17 -21
View File
@@ -48,7 +48,9 @@ MessageAggregator (消息聚合)
QueryPool → Controller → Pipeline (固定阶段链)
│ │
│ ▼
│ RequestRunner (local-agent / dify / n8n / ...)
│ AgentRunner Host orchestrator
│ ▼
│ plugin AgentRunner
│
▼
adapter.reply_message() / adapter.send_message()
@@ -71,13 +73,14 @@ adapter.reply_message() / adapter.send_message()
▼
EventBus (统一事件总线)
│
▼
EventRouter (事件路由引擎, 读取 Bot 的 event_handlers 配置)
├─→ Plugin EventListener observers（始终广播，不参与响应仲裁）
│
├─→ PipelineHandler — 现有流水线（完整 Stage 链）
├─→ AgentHandler — 直接调用 RequestRunner（轻量 AI 处理）
├─→ WebhookHandler — POST 到外部服务（Dify/n8n webhook 等）
└─→ PluginHandler — 分发给插件 EventListener
▼
EventRouter (读取 Bot 的 event_bindings)
│
├─→ Pipeline target — 完整 Stage 链，仅消息事件
├─→ Agent target — 独立 Agent，经插件 AgentRunner 执行
└─→ discard — 明确丢弃
│
▼
统一平台 API
@@ -141,20 +144,13 @@ pkg/platform/adapters/
详见 [03-adapter-structure.md](./03-adapter-structure.md)。
### 3.4 事件处理器（Event Handler）
### 3.4 事件响应目标与观察者
> **2026-06 方向修订**：四种 Handler 分类法已演进为「事件 → 处理器」统一路由。Pipeline 与 Agent 是长期并存、场景不同的同级处理器：Pipeline 面向消息事件的完整 Stage 链，Agent 面向其声明支持的消息或非消息事件；插件 EventListener 保留为观察者角色。详见 [07-agent-orchestration.md](./07-agent-orchestration.md)。本节保留原设计供对照。
Pipeline 与 Agent 是长期并存、场景不同的同级处理器。Pipeline 保留完整 Stage 链，面向消息处理；Agent 是独立配置对象，选择一个已安装的插件 AgentRunner，并可声明消息或非消息事件能力。Bot 的 `event_bindings` 只负责把事件绑定到既有 Pipeline、独立 Agent 或 `discard`。
四种处理器类型，用户在 WebUI 的 Bot 管理页面配置：
插件 EventListener 是观察者：事件先广播给有权限的监听器，随后路由器再选择一个响应目标。Webhook、Dify、n8n 等外部执行方式若需要作为响应者，应由对应 AgentRunner 插件表达，而不是增加另一套 Host Handler 主链。
| 类型 | 说明 | 适用场景 |
|------|------|----------|
| **pipeline** | 现有流水线机制，完整的多 Stage 处理链（PreProcessor → MessageProcessor → PostProcessor 等） | 复杂消息处理，需要完整的预处理/后处理流程 |
| **agent** | 直接调用 RequestRunner（local-agent / dify / n8n / coze / dashscope / langflow / tbox），从 Pipeline 中解耦 | 轻量级 AI 处理、直接对接外部 LLMOps 平台处理各类事件 |
| **webhook** | 将事件 POST 到外部 URL，根据响应执行动作 | 对接自建服务、Dify/n8n 的 Webhook 触发器、自定义后端 |
| **plugin** | 分发给插件 EventListener 处理 | 插件自定义逻辑 |
配置存储在 Bot 表的 `event_handlers` JSON 字段中，通过 WebUI 编排面板管理。
现有 Pipeline 不会被转换为 Agent，Pipeline 内的 runner 配置也不会复制到独立 Agent。用户需要 Agent 时自行创建并绑定。
详见 [04-event-routing.md](./04-event-routing.md)。
@@ -173,8 +169,8 @@ pkg/platform/adapters/
| 1 | 事件处理器配置粒度 | 每个 Bot 独立配置 | Bot 是用户操作的核心单元，不同 Bot 可能对接不同业务场景 |
| 2 | 适配器特有 API | 统一抽象 + `call_platform_api` 透传 | 通用 API 覆盖大部分场景，透传机制保证灵活性，避免每个适配器导出独立的类型化 API 包 |
| 3 | 向后兼容策略 | 兼容层适配 | 保留旧事件类型和 API 作为新系统的 alias/wrapper，现有插件无需修改 |
| 4 | 处理器配置存储 | Bot 表新增 `event_handlers` JSON 字段 | 简单直接，避免新增关联表；替代现有 `use_pipeline_uuid` |
| 5 | Agent 处理器定位 | 从 Pipeline 中解耦 RequestRunner | 不是所有事件都需要完整 Pipeline Stage 链；Agent 处理器提供轻量级 AI 处理路径，支持所有现有 Runner |
| 4 | 处理器配置存储 | Bot 表使用 `event_bindings`，目标引用原始 Pipeline 或独立 Agent UUID | 路由关系不复制处理器配置，Pipeline/Agent 各自保持事实源 |
| 5 | Agent 处理器定位 | 独立 Agent + 插件 AgentRunner | Host 不再内置具体 runner；不同 AgentRunner 通过统一协议接入 |
| 6 | 事件命名方式 | 命名空间式（`message.received`） | 清晰的分类层级，便于通配匹配（`message.*`），与 WebUI 配置天然对应 |
## 5. 文档索引
@@ -194,7 +190,7 @@ pkg/platform/adapters/
| 仓库 | 改动范围 |
|------|----------|
| **langbot-plugin-sdk** | 事件定义、实体模型、API 接口、适配器基类、通信协议扩展 |
| **LangBot**（后端） | 适配器实现、事件路由引擎、Bot 实体扩展、数据库迁移、RequestRunner 解耦 |
| **LangBot**（后端） | 适配器实现、事件路由引擎、Bot/Agent 实体、AgentRunner Host 编排 |
| **LangBot**（前端） | Bot 事件处理器编排面板 |
| **langbot-wiki** | 新架构文档、插件开发指南更新、适配器开发指南 |
| **langbot-plugin-demo** | 示例更新（使用新事件和 API） |
docs/event-based-agents/01-event-system.md
+11 -10
View File
@@ -490,18 +490,19 @@ class PlatformSpecificEvent(Event):
│
5. EventBus 分发
│
6. EventRouter 查询 Bot 的 event_handlers 配置
│ 匹配事件类型 → 找到对应的 Handler
│ 支持通配符：'message.*' 匹配所有消息事件
│ 未匹配到 → 走默认 Handler（plugin，保持向后兼容）
6. EventBus 向有权限的 Plugin EventListener 广播观察者事件
│ observer 不占用响应目标，也不参与 priority 仲裁
│
7. Handler 处理事件
│ PipelineHandler → 进入 Pipeline 流水线
│ AgentHandler → 调用 RequestRunner
│ WebhookHandler → POST 到外部 URL
│ PluginHandler → 分发给插件 EventListener
7. EventRouter 查询 Bot 的 event_bindings
│ 精确/通配匹配 event_pattern，应用 filters 与 priority
│ 选择一个 Pipeline、Agent 或 discard 目标
│
8. Handler 执行完毕，可能通过 API 执行响应动作
8. 目标处理事件
│ Pipeline → 进入完整 Pipeline 流水线（仅消息事件）
│ Agent → Host 编排已安装的插件 AgentRunner
│ discard → 不产生响应
│
9. 处理器执行完毕，可能通过 Host 授权 API 执行响应动作
（发消息、编辑消息、踢人、同意好友请求等）
```
docs/event-based-agents/04-event-routing.md
+125 -698
View File
@@ -1,747 +1,174 @@
# 事件路由与编排
> **2026-06 方向修订**：本文档的四种 handler_type（pipeline / agent / webhook / plugin）分类法已被「事件 → 处理器（Agent / Pipeline）」统一编排取代，收编映射与新数据模型见 [07-agent-orchestration.md](./07-agent-orchestration.md)。本文档中的事件匹配规则（§4）、`use_pipeline_uuid` 路由迁移策略（§6）、WebUI 交互骨架（§7）与 webhook 请求/响应格式（§5.4）仍然有效。
> 状态：当前实施模型（2026-07-12）。本文以 Pipeline / Agent 平级并存为准，不再保留早期 `pipeline / agent / webhook / plugin` 四种 Handler 草案。
## 1. 概述
## 1. 路由边界
事件路由是 EBA 架构的核心机制：事件从适配器产生后，经由 EventBus 进入 EventRouter，由 EventRouter 根据 Bot 的配置将事件分发到对应的处理器（Handler）。
EBA 将事件处理拆成两个互不替代的阶段：
**配置方式**：用户在 WebUI 的 Bot 管理页面通过可视化编排面板管理事件处理器配置，配置数据存储在数据库的 Bot 表 `event_handlers` JSON 字段中。
1. **观察者广播**：EventBus 把事件广播给有权限的插件 EventListener。观察者可记录、同步或执行受控副作用，但不占用响应目标。
2. **响应者仲裁**：EventRouter 按 Bot 的 `event_bindings` 选择一个 Pipeline、独立 Agent 或 `discard`。
Pipeline 与 Agent 是平级处理器：
| 处理器 | 配置事实源 | 执行路径 | 事件范围 |
| --- | --- | --- | --- |
| Pipeline | Pipeline 表与完整 Stage 配置 | MessageAggregator -> QueryPool -> RuntimePipeline | 消息事件，首版为 `message.received` |
| Agent | Agent 表中的 runner 与 runner config | AgentRunner Host orchestrator -> plugin AgentRunner | Agent/Runner 声明支持的消息或非消息事件 |
| discard | 无处理器配置 | 明确结束路由 | 任意事件 |
插件 EventListener 不是第三种响应目标。Webhook、Dify、n8n、Coze 等外部系统需要响应事件时，由对应 AgentRunner 插件承接。
## 2. 数据模型
### 2.1 Bot 实体扩展
### 2.1 独立 Agent
在 `bots` 表新增 `event_handlers` 字段：
Agent 保存自己的 Runner 选择与配置，不嵌入 Pipeline，也不复制 Pipeline 的 AI stage：
```python
class Bot(Base):
__tablename__ = "bots"
uuid: str # 主键
class Agent(Base):
uuid: str
name: str
description: str
adapter: str
adapter_config: dict # JSON
enable: bool
# 新增
event_handlers: list # JSON — 事件处理器配置列表
# 保留（过渡期后弃用）
use_pipeline_name: str # deprecated
use_pipeline_uuid: str # deprecated
created_at: datetime
updated_at: datetime
emoji: str
kind: str # 固定为 "agent"
component_ref: str # AgentRunner id
config: dict # runner + runner_config
enabled: bool
supported_event_patterns: list[str]
```
### 2.2 EventHandler 配置结构
当前配置形状：
`event_handlers` 字段存储一个 JSON 数组，每个元素定义一条事件路由规则：
```json
{
"runner": {
"id": "plugin:langbot-team/LocalAgent/default"
},
"runner_config": {
"plugin:langbot-team/LocalAgent/default": {
"model": {
"primary": "model-uuid",
"fallbacks": []
}
}
}
}
```
Runner id 来自已安装插件的 AgentRunner manifest。Host 不维护 LocalAgent、Dify 或其他具体实现的内置分支。
### 2.2 EventBinding
Bot 维护事件到处理器的引用：
```python
class EventHandlerConfig(pydantic.BaseModel):
"""单条事件处理器配置"""
event_type: str
"""匹配的事件类型
支持精确匹配和通配符：
- "message.received" — 精确匹配
- "message.*" — 匹配 message 命名空间下所有事件
- "group.*" — 匹配 group 命名空间下所有事件
- "*" — 匹配所有事件（兜底）
"""
handler_type: str
"""处理器类型: "pipeline" | "agent" | "webhook" | "plugin" """
handler_config: dict = {}
"""处理器的具体配置，结构取决于 handler_type"""
enabled: bool = True
"""是否启用此规则"""
priority: int = 0
"""优先级，数字越大越先匹配（同一事件类型有多条规则时）"""
description: str = ""
"""规则描述（供 WebUI 显示）"""
class EventBinding(BaseModel):
id: str
event_pattern: str # 精确、namespace.* 或 *
target_type: str # agent | pipeline | discard
target_uuid: str | None # Agent/Pipeline 原始 UUID
filters: list[dict]
priority: int
enabled: bool
description: str
```
### 2.3 各 Handler 类型的 handler_config 结构
#### pipeline
```json
{
"handler_type": "pipeline",
"handler_config": {
"pipeline_uuid": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"
}
}
```
将事件作为消息事件传入现有 Pipeline 流水线。仅适用于 `message.received` 事件。
#### agent
```json
{
"handler_type": "agent",
"handler_config": {
"runner": "local-agent",
"runner_config": {
"model_uuid": "...",
"prompt": "你是一个群组助理，请处理以下事件：{event_summary}",
"tools_enabled": true
}
}
}
```
```json
{
"handler_type": "agent",
"handler_config": {
"runner": "dify-service-api",
"runner_config": {
"base_url": "https://api.dify.ai/v1",
"api_key": "...",
"app_type": "agent"
}
}
}
```
直接调用 RequestRunner 处理事件。可用的 runner 包括：
- `local-agent` — 内置 LLM Agent
- `dify-service-api` — Dify 平台
- `n8n-service-api` — n8n 工作流
- `coze-api` — Coze (扣子)
- `dashscope-app-api` — 阿里百炼
- `langflow-api` — Langflow
- `tbox-app-api` — 蚂蚁 Tbox
Agent 处理器不经过 Pipeline 的多 Stage 流程，而是直接构建上下文并调用 Runner。适用于所有事件类型。
**Agent Handler 与 Pipeline 的关系**：
- Pipeline 是完整的多 Stage 处理链（PreProcessor → MessageProcessor(内含Runner) → PostProcessor → ...），适合复杂消息处理
- Agent Handler 是轻量级的，直接调用 Runner，跳过 PreProcessor/PostProcessor 等阶段
- Pipeline 内部的 AI Stage 仍然使用 Runner，所以 Runner 本身被两种 Handler 共享
- 用户可以根据场景选择：消息处理用 Pipeline（更多控制），其他事件用 Agent（更直接）
#### webhook
```json
{
"handler_type": "webhook",
"handler_config": {
"url": "https://example.com/webhook/langbot-events",
"method": "POST",
"headers": {
"Authorization": "Bearer xxx"
},
"timeout": 30,
"retry_count": 3,
"retry_interval": 5,
"response_actions": true
}
}
```
将事件序列化为 JSON POST 到外部 URL。支持的特性：
- **认证**：通过 headers 配置（Bearer Token、API Key 等）
- **重试**：配置重试次数和间隔
- **响应动作**：如果 `response_actions` 为 true，解析响应 JSON 中的 `actions` 字段并执行（如发送消息、同意好友请求等）
Webhook 请求体格式：
```json
{
"event": {
"type": "group.member_joined",
"timestamp": 1700000000.0,
"bot_uuid": "...",
"adapter_name": "telegram",
"group": { "id": "...", "name": "..." },
"member": { "id": "...", "nickname": "..." }
},
"bot": {
"uuid": "...",
"name": "...",
"adapter": "telegram"
}
}
```
响应体格式（当 `response_actions` 为 true 时）：
```json
{
"actions": [
{
"type": "send_message",
"params": {
"target_type": "group",
"target_id": "123456",
"message": [{ "type": "Plain", "text": "欢迎新成员！" }]
}
},
{
"type": "call_platform_api",
"params": {
"action": "pin_message",
"params": { "chat_id": "123456", "message_id": "789" }
}
}
]
}
```
#### plugin
```json
{
"handler_type": "plugin",
"handler_config": {
"plugin_filter": []
}
}
```
将事件分发给插件的 EventListener 处理。
- `plugin_filter`：可选的插件名过滤列表，为空表示分发给所有插件
- 沿用现有的插件事件分发机制（按优先级遍历插件，支持 `prevent_postorder`）
### 2.4 完整配置示例
一个 Bot 的 `event_handlers` 配置示例：
示例：
```json
[
{
"event_type": "message.received",
"handler_type": "pipeline",
"handler_config": {
"pipeline_uuid": "default-pipeline-uuid"
},
"enabled": true,
"priority": 10,
"description": "消息事件使用默认流水线处理"
},
{
"event_type": "group.member_joined",
"handler_type": "agent",
"handler_config": {
"runner": "local-agent",
"runner_config": {
"model_uuid": "gpt-4o-mini",
"prompt": "有新成员 {member_name} 加入了群组 {group_name}，请生成一条欢迎消息。"
}
},
"enabled": true,
"priority": 0,
"description": "新成员入群时用 AI 生成欢迎消息"
},
{
"event_type": "friend.request_received",
"handler_type": "webhook",
"handler_config": {
"url": "https://my-server.com/api/friend-request",
"response_actions": true
},
"enabled": true,
"priority": 0,
"description": "好友请求转发到自建服务处理"
},
{
"event_type": "*",
"handler_type": "plugin",
"handler_config": {},
"enabled": true,
"priority": -100,
"description": "所有事件兜底发给插件处理"
}
{
"id": "binding-message",
"event_pattern": "message.received",
"target_type": "pipeline",
"target_uuid": "pipeline-uuid",
"filters": [],
"priority": 100,
"enabled": true
},
{
"id": "binding-member-joined",
"event_pattern": "group.member_joined",
"target_type": "agent",
"target_uuid": "agent-uuid",
"filters": [],
"priority": 50,
"enabled": true
}
]
```
## 3. EventBus 设计
Binding 只保存引用与路由条件。它不复制 Pipeline 或 Agent 配置。
EventBus 是事件的中转站，接收来自各个 RuntimeBot 的事件，交由 EventRouter 处理。
## 3. 匹配与仲裁
```python
class EventBus:
"""事件总线"""
事件模式支持：
def __init__(self, ap: Application):
self.ap = ap
self.event_router = EventRouter(ap)
- 精确匹配：`group.member_joined`
- 命名空间通配：`group.*`
- 全局通配：`*`
async def emit(
self,
event: Event,
adapter: AbstractPlatformAdapter,
):
"""接收并分发事件
路由按以下顺序处理：
Args:
event: 统一事件对象
adapter: 产生此事件的适配器实例
"""
# 1. 全局事件日志
self.ap.logger.debug(
f"EventBus: {event.type} from bot {event.bot_uuid}"
)
1. 忽略 `enabled = false` 的 binding。
2. 检查 `event_pattern` 与结构化 filters。
3. 校验目标存在、启用且声明支持该事件。
4. 按 `priority` 从高到低选择；同优先级按稳定列表顺序。
5. 只执行一个响应目标。
# 2. 交由 EventRouter 路由处理
await self.event_router.route(event, adapter)
Pipeline 目标只能匹配消息事件。非消息事件不得伪装成用户文本塞进 Pipeline。
## 4. 执行流程
```text
Platform adapter
-> normalized event
-> EventBus
-> authorized Plugin EventListener observers
-> EventRouter
-> Pipeline target -> full Pipeline stage chain
-> Agent target -> AgentRunner Host orchestrator
-> discard -> stop
-> Host delivery/platform API
```
## 4. EventRouter 设计
### 4.1 Pipeline target
EventRouter 是事件路由引擎，根据 Bot 的 `event_handlers` 配置决定事件的处理方式。
消息事件按原有方式构造 Query，经 MessageAggregator、QueryPool 和完整 Pipeline Stage 链执行。Pipeline 可以继续使用 AgentRunner 作为 AI stage 的实现，但 Pipeline 本身不会因此变成 Agent。
```python
class EventRouter:
"""事件路由引擎"""
### 4.2 Agent target
def __init__(self, ap: Application):
self.ap = ap
self.handlers: dict[str, AbstractEventHandler] = {
"pipeline": PipelineHandler(ap),
"agent": AgentHandler(ap),
"webhook": WebhookHandler(ap),
"plugin": PluginHandler(ap),
}
Host 读取独立 Agent 的 Runner id/config，构造 event-first context、run-scoped resources 与 delivery policy，再调用插件 AgentRunner。Runner 输出由 Host 统一归一化、记录和投递。
async def route(
self,
event: Event,
adapter: AbstractPlatformAdapter,
):
"""路由事件到对应处理器"""
AgentRunner 可通过 SDK/Python `AgentRunAPIProxy.call_tool` 或 SDK-owned scoped MCP bridge 回调 Host 能力。两条路径都映射到 `PluginToRuntimeAction.CALL_TOOL`，使用相同的 run authorization、Host execution Query、ToolManager 和 Box session 规则。Box session 是 Host canonical scope 的固定长度安全哈希；同一平台会话稳定、不同 scope 隔离、缺少 identity 时 fail closed，Runner 不配置 sandbox scope。
# 1. 获取 Bot 配置
bot = await self.ap.platform_mgr.get_bot_by_uuid(event.bot_uuid)
if not bot:
return
### 4.3 Observer side effects
# 2. 获取事件处理器配置
event_handlers = bot.bot_entity.event_handlers or []
观察者与响应者可以同时工作。为避免编辑、reaction 等合成事件重复触发不可逆操作，Host 应按事件能力和授权过滤观察者可用 API，并记录副作用结果。Observer 广播不作为 fallback 响应。
# 3. 匹配规则（按 priority 降序排列）
matched_handlers = self._match_handlers(event.type, event_handlers)
## 5. Pipeline 与 Agent 的并存规则
if not matched_handlers:
# 未匹配到任何规则 → 默认交给插件处理（向后兼容）
await self.handlers["plugin"].handle(event, adapter, {})
return
1. Pipeline 与 Agent 保留各自的持久化、编辑和执行语义。
2. 处理器聚合页面可以统一展示二者，但不会创建第三份处理器记录。
3. 旧 Pipeline 仍是 Pipeline；其 runner config 不迁移、不复制为独立 Agent。
4. 需要 Agent 的用户新建 Agent、选择已安装 AgentRunner，再建立 event binding。
5. 一个 Bot 可按不同事件同时绑定 Pipeline 与 Agent。
# 4. 执行第一个匹配的 Handler
# （未来可扩展为多个 Handler 串行/并行执行）
handler_config = matched_handlers[0]
handler = self.handlers.get(handler_config.handler_type)
## 6. WebUI 约束
if handler:
await handler.handle(event, adapter, handler_config.handler_config)
else:
self.ap.logger.warning(
f"Unknown handler type: {handler_config.handler_type}"
)
处理器入口展示带类型标识的 Agent 与 Pipeline。Bot 事件编排器应：
def _match_handlers(
self,
event_type: str,
handlers: list[EventHandlerConfig],
) -> list[EventHandlerConfig]:
"""匹配事件类型到处理器配置
- 按 adapter manifest 展示可用事件；
- 非消息事件不提供 Pipeline 目标；
- Agent 目标按 `supported_event_patterns` 过滤；
- 展示 priority、filters、enabled 与 discard；
- 保存前校验目标 UUID 与 `target_type` 一致。
匹配规则：
1. 精确匹配：event_type == handler.event_type
2. 命名空间通配：handler.event_type 为 "message.*" 时匹配所有 "message.xxx"
3. 全局通配：handler.event_type 为 "*" 时匹配所有事件
4. 按 priority 降序排列
5. 只返回 enabled=True 的规则
"""
matched = []
for handler in handlers:
if not handler.enabled:
continue
if self._event_type_matches(event_type, handler.event_type):
matched.append(handler)
Pipeline 的配置、Debug Chat 和 Monitoring 继续使用 Pipeline 页面；Agent 使用独立表单配置 Runner 与事件能力。
matched.sort(key=lambda h: h.priority, reverse=True)
return matched
## 7. 版本与迁移边界
@staticmethod
def _event_type_matches(event_type: str, pattern: str) -> bool:
"""判断事件类型是否匹配模式"""
if pattern == "*":
return True
if pattern == event_type:
return True
if pattern.endswith(".*"):
namespace = pattern[:-2]
return event_type.startswith(namespace + ".")
return False
```
此功能按当前 4.x schema 直接实现，不提供 LangBot 3.x 数据库或配置升级路径，也不读取旧 Runner 字段作为 fallback。旧 Pipeline 中的 runner 配置不会生成独立 Agent；用户按新产品模型添加 Agent 即可。
## 5. 事件处理器（Handler）实现
### 5.1 Handler 基类
```python
class AbstractEventHandler(abc.ABC):
"""事件处理器基类"""
def __init__(self, ap: Application):
self.ap = ap
@abc.abstractmethod
async def handle(
self,
event: Event,
adapter: AbstractPlatformAdapter,
config: dict,
) -> None:
"""处理事件
Args:
event: 统一事件对象
adapter: 适配器实例（用于调用平台 API 发送响应）
config: handler_config 配置
"""
...
```
### 5.2 PipelineHandler
将消息事件注入现有 Pipeline 流水线处理。
```python
class PipelineHandler(AbstractEventHandler):
"""Pipeline 处理器 — 将事件送入现有 Pipeline 流水线"""
async def handle(self, event, adapter, config):
pipeline_uuid = config.get("pipeline_uuid")
if not isinstance(event, MessageReceivedEvent):
self.ap.logger.warning(
f"PipelineHandler only supports MessageReceivedEvent, "
f"got {event.type}"
)
return
# 将 MessageReceivedEvent 转换为现有的 Query 并投入 QueryPool
# 复用现有的 MessageAggregator + QueryPool + Pipeline 机制
launcher_type = (
LauncherTypes.PERSON
if event.chat_type == ChatType.PRIVATE
else LauncherTypes.GROUP
)
await self.ap.msg_aggregator.add_message(
bot_uuid=event.bot_uuid,
launcher_type=launcher_type,
launcher_id=event.chat_id,
sender_id=event.sender.id,
message_event=event.to_legacy_event(), # 转换为 FriendMessage/GroupMessage
message_chain=event.message_chain,
adapter=adapter,
pipeline_uuid=pipeline_uuid,
)
```
### 5.3 AgentHandler
直接调用 RequestRunner 处理事件，不经过 Pipeline Stage 链。
```python
class AgentHandler(AbstractEventHandler):
"""Agent 处理器 — 直接调用 RequestRunner 处理事件"""
async def handle(self, event, adapter, config):
runner_name = config.get("runner", "local-agent")
runner_config = config.get("runner_config", {})
# 1. 查找 Runner 类
runner_cls = None
for r in preregistered_runners:
if r.name == runner_name:
runner_cls = r
break
if not runner_cls:
self.ap.logger.error(f"Runner not found: {runner_name}")
return
# 2. 构建事件上下文（将事件信息整理为 Runner 可处理的格式）
event_context = self._build_event_context(event, runner_config)
# 3. 实例化并调用 Runner
runner = runner_cls(self.ap, self._build_runner_pipeline_config(config))
response_messages = []
async for result in runner.run(event_context):
response_messages.append(result)
# 4. 发送响应（如果 Runner 产生了回复）
if response_messages and isinstance(event, MessageReceivedEvent):
# 将 Runner 输出转换为 MessageChain 并回复
reply_chain = self._build_reply_chain(response_messages)
await adapter.reply_message(event, reply_chain)
def _build_event_context(self, event, runner_config):
"""将事件构建为 Runner 可处理的上下文
对于消息事件，直接使用消息内容。
对于其他事件，根据 runner_config 中的 prompt 模板生成描述文本。
"""
...
def _build_runner_pipeline_config(self, config):
"""将 handler_config 转换为 Runner 需要的 pipeline_config 格式"""
...
```
### 5.4 WebhookHandler
将事件 POST 到外部 URL。
```python
class WebhookHandler(AbstractEventHandler):
"""Webhook 处理器 — 将事件 POST 到外部 URL"""
async def handle(self, event, adapter, config):
url = config.get("url")
method = config.get("method", "POST")
headers = config.get("headers", {})
timeout = config.get("timeout", 30)
retry_count = config.get("retry_count", 3)
response_actions = config.get("response_actions", False)
# 1. 构建请求体
bot = await self.ap.platform_mgr.get_bot_by_uuid(event.bot_uuid)
payload = {
"event": event.model_dump(),
"bot": {
"uuid": bot.bot_entity.uuid,
"name": bot.bot_entity.name,
"adapter": bot.bot_entity.adapter,
}
}
# 2. 发送请求（带重试）
response = await self._send_with_retry(
url, method, headers, payload, timeout, retry_count
)
# 3. 处理响应动作
if response_actions and response:
await self._execute_response_actions(
response, adapter, event
)
async def _execute_response_actions(self, response, adapter, event):
"""执行响应中的动作列表"""
actions = response.get("actions", [])
for action in actions:
action_type = action.get("type")
params = action.get("params", {})
if action_type == "send_message":
chain = MessageChain.model_validate(params.get("message", []))
await adapter.send_message(
params["target_type"],
params["target_id"],
chain,
)
elif action_type == "reply":
chain = MessageChain.model_validate(params.get("message", []))
await adapter.reply_message(event, chain)
elif action_type == "call_platform_api":
await adapter.call_platform_api(
params["action"],
params.get("params", {}),
)
elif action_type == "approve_friend_request":
await adapter.approve_friend_request(
params["request_id"],
params.get("approve", True),
)
# ... 更多动作类型
```
### 5.5 PluginHandler
将事件分发给插件的 EventListener。
```python
class PluginHandler(AbstractEventHandler):
"""Plugin 处理器 — 分发给插件 EventListener"""
async def handle(self, event, adapter, config):
plugin_filter = config.get("plugin_filter", [])
# 复用现有的插件事件分发机制
# 通过 plugin_connector 将事件发送给 Plugin Runtime
await self.ap.plugin_connector.emit_event(
event=event,
adapter=adapter,
plugin_filter=plugin_filter,
)
```
## 6. use_pipeline_uuid 迁移
### 6.1 自动迁移
数据库迁移脚本将现有的 `use_pipeline_uuid` 自动转换为 `event_handlers`：
这里迁移的只有 Bot 路由字段：binding 仍指向原 Pipeline，不会新建 Agent，也不会复制 Pipeline 内嵌的 runner 配置。
```python
# 迁移逻辑
for bot in all_bots:
if bot.use_pipeline_uuid and not bot.event_handlers:
bot.event_handlers = [
{
"event_type": "message.received",
"handler_type": "pipeline",
"handler_config": {
"pipeline_uuid": bot.use_pipeline_uuid
},
"enabled": True,
"priority": 10,
"description": "Auto-migrated from use_pipeline_uuid"
},
{
"event_type": "*",
"handler_type": "plugin",
"handler_config": {},
"enabled": True,
"priority": -100,
"description": "Default plugin handler"
}
]
```
### 6.2 过渡期兼容
在过渡期内，如果 `event_handlers` 为空且 `use_pipeline_uuid` 非空，EventRouter 自动回退到旧行为：
```python
# EventRouter.route() 中的兼容逻辑
if not event_handlers and bot.bot_entity.use_pipeline_uuid:
# 回退：消息事件走 Pipeline，其他事件走 Plugin
if isinstance(event, MessageReceivedEvent):
await self.handlers["pipeline"].handle(
event, adapter,
{"pipeline_uuid": bot.bot_entity.use_pipeline_uuid}
)
else:
await self.handlers["plugin"].handle(event, adapter, {})
return
```
## 7. WebUI 编排面板数据模型
### 7.1 交互设计概要
在 WebUI 的 Bot 管理页面，新增"事件处理器"标签页（或区域），呈现为一个**规则列表**：
```
┌─────────────────────────────────────────────────────────────┐
│ 事件处理器 [+ 添加规则] │
├─────────────────────────────────────────────────────────────┤
│ │
│ ┌─ 规则 1 ─────────────────────────────────── [启用] [删除] ─┐ │
│ │ 事件类型: [message.received ▾] │ │
│ │ 处理器: [Pipeline ▾] │ │
│ │ Pipeline: [默认流水线 ▾] │ │
│ │ 优先级: [10] │ │
│ │ 描述: 消息事件使用默认流水线处理 │ │
│ └──────────────────────────────────────────────────────────┘ │
│ │
│ ┌─ 规则 2 ─────────────────────────────────── [启用] [删除] ─┐ │
│ │ 事件类型: [group.member_joined ▾] │ │
│ │ 处理器: [Agent ▾] │ │
│ │ Runner: [local-agent ▾] │ │
│ │ 模型: [gpt-4o-mini ▾] │ │
│ │ Prompt: [有新成员加入...] │ │
│ │ 优先级: [0] │ │
│ └──────────────────────────────────────────────────────────┘ │
│ │
│ ┌─ 规则 3 (兜底) ──────────────────────────── [启用] [删除] ─┐ │
│ │ 事件类型: [* ▾] │ │
│ │ 处理器: [Plugin ▾] │ │
│ │ 优先级: [-100] │ │
│ └──────────────────────────────────────────────────────────┘ │
│ │
└─────────────────────────────────────────────────────────────┘
```
### 7.2 前端数据结构
```typescript
interface EventHandlerRule {
event_type: string; // 下拉选择，选项从适配器 manifest 的 supported_events 获取
handler_type: string; // "pipeline" | "agent" | "webhook" | "plugin"
handler_config: Record<string, any>; // 根据 handler_type 动态渲染不同的配置表单
enabled: boolean;
priority: number;
description: string;
}
// Bot 编辑接口扩展
interface BotConfig {
uuid: string;
name: string;
adapter: string;
adapter_config: Record<string, any>;
enable: boolean;
event_handlers: EventHandlerRule[]; // 新增
}
```
### 7.3 事件类型下拉选项
从 Bot 关联的适配器 manifest 中获取 `supported_events`，加上通配符选项：
```
- message.received
- message.edited
- message.deleted
- message.reaction
- feedback.received
- group.member_joined
- group.member_left
- group.member_banned
- group.info_updated
- friend.request_received
- friend.added
- bot.invited_to_group
- bot.removed_from_group
- bot.muted
- bot.unmuted
- platform.specific
─────────────────
- message.* (所有消息事件)
- feedback.* (所有反馈事件)
- group.* (所有群组事件)
- friend.* (所有好友事件)
- bot.* (所有 Bot 事件)
- * (所有事件)
```
### 7.4 HTTP API
```
GET /api/v1/bots/{uuid}/event-handlers 获取 Bot 的事件处理器配置
PUT /api/v1/bots/{uuid}/event-handlers 更新 Bot 的事件处理器配置
GET /api/v1/adapters/{name}/supported-events 获取适配器支持的事件类型
GET /api/v1/adapters/{name}/supported-apis 获取适配器支持的 API
```
详见 [07-agent-orchestration.md](./07-agent-orchestration.md) 与 [08-agent-page-and-event-orchestration.md](./08-agent-page-and-event-orchestration.md)。
docs/event-based-agents/06-migration-plan.md
+103 -391
View File
@@ -1,433 +1,145 @@
# 分阶段迁移计划
# EBA 分阶段实施计划
> **2026-06 方向修订**：Phase 3 的「四种 Handler 框架」与 Phase 5 的编排面板形态，按 [07-agent-orchestration.md](./07-agent-orchestration.md) 调整为「事件 → 处理器」统一路由。EventRouter 通过 `target_type` 在同级 Pipeline / Agent 间选择；二者分别保留自己的实体、配置和执行链。阶段划分、依赖关系与验收标准按 Processor 模型解读；发布节奏见 07 §5「发布火车」。
> 更新：2026-07-12。文件名沿用早期设计，但这里的“迁移”仅指代码架构逐步接入 EBA，不代表 LangBot 3.x 数据库或配置升级。
## 1. 概述
## 1. 发布边界
EBA 架构涉及 langbot-plugin-sdk、LangBot 后端、LangBot 前端、文档和示例插件等多个仓库的改动。为降低风险、保证系统稳定性，采用分阶段渐进式迁移策略。
EBA 跨越 SDK、平台适配器、LangBot Host、WebUI 与插件生态，按可验证阶段落地。当前发布遵守以下硬边界：
### 1.1 阶段总览
- LangBot 4.x 不支持从 3.x 数据库或配置升级；不保留 legacy migration chain、旧 JSON 模板或旧 Runner 字段读取。
- Pipeline 与 Agent 平级且长期并存，分别保留持久化模型与执行链。
- 现有 Pipeline 不迁移为 Agent，Pipeline 内的 runner config 不复制到 Agent。
- 用户需要 Agent 时新建独立 Agent并选择已安装的 AgentRunner。
- Host 不按 LocalAgent id 做运行时、Box 或 WebUI 特判。
- AgentRunner 的 SDK/Python 与 scoped MCP bridge 回调共享 Host 授权与事件 session 规则。
| 阶段 | 名称 | 范围 | 依赖 |
|------|------|------|------|
| Phase 1 | SDK 实体层 | langbot-plugin-sdk | 无 |
| Phase 2 | 适配器重构 | LangBot 后端 | Phase 1 |
| Phase 3 | 核心系统 | LangBot 后端 | Phase 2 |
| Phase 4 | 插件 SDK 集成 | langbot-plugin-sdk + LangBot | Phase 3 |
| Phase 5 | WebUI 编排面板 | LangBot 前端 | Phase 3 |
| Phase 6 | 文档与示例 | langbot-wiki + langbot-plugin-demo | Phase 4, 5 |
## 2. 阶段总览
### 1.2 核心原则
| 阶段 | 目标 | 主要仓库 | 完成条件 |
| --- | --- | --- | --- |
| P0 | SDK 事件、能力与 AgentRunner 协议 | `langbot-plugin-sdk` | typed entities、manifest、proxy、runtime action 通过测试 |
| P1 | 平台适配器 EBA 化 | LangBot + SDK | 事件转换、能力声明、通用/透传 API 通过 adapter checklist |
| P2 | Host 观察者与响应者路由 | LangBot backend | observer 广播 + Pipeline/Agent/discard 单目标仲裁可运行 |
| P3 | 独立 Agent 与 Runner 注册 | LangBot backend + plugins | Agent CRUD、registry、run authorization、delivery 可运行 |
| P4 | WebUI 处理器与事件编排 | LangBot web | Agent/Pipeline 聚合入口和 Bot binding 编辑器可用 |
| P5 | 发布门禁与文档 | LangBot skills/docs + runner plugins | 单测、UI E2E、真实 adapter smoke 和插件预检通过 |
- **每个阶段结束后系统可运行**：任何阶段完成后，现有功能不受影响
- **向后兼容贯穿全程**：旧接口在整个迁移期间保持可用
- **先 SDK 后实现**：先定义好接口和模型，再做具体实现
- **先核心适配器后边缘**：优先迁移用户量大的适配器
## 3. P0：SDK 契约
---
### 工作项
## 2. Phase 1：SDK 实体层
- 定义规范化平台事件、actor/subject/conversation/delivery context。
- 定义 adapter `supported_events`、`supported_apis` 与平台透传 API。
- 定义 AgentRunner manifest、run context/result、resource handles 和 pull/callback API。
- 提供 `AgentRunAPIProxy` 与 SDK-owned scoped MCP bridge。
- 保持协议传输与权限校验可测试，不把 Host 私有 Query 对象暴露给插件。
**目标**：在 langbot-plugin-sdk 中定义新的事件体系、通用实体、API 接口和适配器基类。
### 验收
**仓库**：`langbot-plugin-sdk`
- stdio/WebSocket runtime 对 typed action 的序列化一致。
- 无 `run_id` 或越权资源调用被拒绝。
- Python proxy 与 MCP bridge 对同一 Host tool 呈现一致结果/错误形状。
### 2.1 任务清单
## 4. P1：平台适配器
| # | 任务 | 文件/模块 | 说明 |
|---|------|----------|------|
| 1.1 | 定义通用事件基类层次 | `api/entities/builtin/platform/events.py` | 新增 `MessageReceivedEvent`, `MessageEditedEvent`, `GroupMemberJoinedEvent` 等，保留现有 `FriendMessage`/`GroupMessage` |
| 1.2 | 定义平台特有事件基类 | `api/entities/builtin/platform/events.py` | 新增 `PlatformSpecificEvent` |
| 1.3 | 扩展通用实体 | `api/entities/builtin/platform/entities.py` | 新增 `User`（统一 Friend/GroupMember 的基础）、`Channel` 等，保留现有实体 |
| 1.4 | 清理消息组件 | `api/entities/builtin/platform/message.py` | 将 `WeChatMiniPrograms` 等 WeChat 特有组件标记为 platform-specific，不再作为通用组件 |
| 1.5 | 定义新适配器基类 | `api/definition/abstract/platform/adapter.py` | 新增 `AbstractPlatformAdapter`（继承现有 `AbstractMessagePlatformAdapter` 并扩展通用 API 方法），保留旧基类 |
| 1.6 | 定义 API 能力声明 | `api/definition/abstract/platform/capabilities.py`（新文件） | `AdapterCapabilities` 数据类，声明适配器支持的事件和 API |
| 1.7 | 定义 `NotSupportedError` | `api/entities/builtin/platform/errors.py`（新文件） | 可选 API 未实现时抛出的异常 |
每个适配器按自己的能力增量接入，而不是要求所有平台一次性支持全部事件。
### 2.2 关键设计约束
### 单适配器步骤
- 所有新增定义以**新增文件或新增类**的方式引入，**不修改**现有类的字段和方法签名
- 现有 `AbstractMessagePlatformAdapter` 保留不动，新基类 `AbstractPlatformAdapter` 继承它
- 新事件类与旧事件类并存，通过 `event_type` 字段（命名空间字符串）区分
1. 将原生 SDK callback 转换为规范化事件。
2. 声明实际支持的事件与 API，不把缺失能力伪装为成功。
3. 实现 send/reply/edit/delete/group/user 等适用 API 与平台透传。
4. 对消息链、媒体、reply target 和错误语义写单元测试。
5. 按 `adapters/acceptance-checklist.md` 记录真实平台 probe。
### 2.3 验收标准
### 验收
- [ ] 所有新增类可正常 import 且通过类型检查
- [ ] 现有 `FriendMessage`, `GroupMessage`, `AbstractMessagePlatformAdapter` 等类行为不变
- [ ] 新增单元测试覆盖事件序列化/反序列化、实体构造
- [ ] SDK 版本号 minor bump（如 `0.x.0` → `0.x+1.0`）
- `message.received` 保持正常收发。
- 新事件不会重复转换或产生循环合成事件。
- `supported_apis` 与实际调用能力一致。
---
## 5. P2：Host 路由
## 3. Phase 2：适配器重构
### 执行顺序
**目标**：将现有单文件适配器迁移到独立目录结构，实现新事件监听和通用 API。
**仓库**：`LangBot`（后端）
### 3.1 适配器迁移优先级
根据用户量和代表性，建议按以下顺序迁移：
| 优先级 | 适配器 | 理由 |
|--------|--------|------|
| P0 | **Telegram** | 用户量大，API 最完善，适合作为参考实现 |
| P0 | **Discord** | 国际用户主要平台，事件类型丰富 |
| P1 | **aiocqhttp**（OneBot v11） | 国内 QQ 用户主要适配器 |
| P1 | **Satori** | 通用协议适配器，覆盖多个平台 |
| P2 | **Lark** / **DingTalk** / **Slack** | 企业平台，用户量中等 |
| P2 | **qqofficial** / **WeChat 系列** | 国内用户 |
| P3 | **Kook** / **LINE** / **WeCom 系列** | 用户量较小 |
| P3 | **WebSocket** | 内置适配器，相对简单 |
| P4 | **legacy/*** | 遗留适配器，按需决定是否迁移或废弃 |
### 3.2 单个适配器迁移步骤（以 Telegram 为例）
| # | 任务 | 说明 |
|---|------|------|
| 2.1 | 创建目录结构 | `pkg/platform/adapters/telegram/` 下创建 `__init__.py`, `adapter.py`, `event_converter.py`, `message_converter.py`, `api_impl.py`, `types.py`, `manifest.yaml` |
| 2.2 | 迁移消息转换器 | 将 `TelegramMessageConverter` 从 `sources/telegram.py` 搬到 `adapters/telegram/message_converter.py`，逻辑不变 |
| 2.3 | 重写事件转换器 | 新的 `TelegramEventConverter` 支持将 Telegram Update 转换为所有通用事件类型（不只是消息），不支持的事件转为 `PlatformSpecificEvent` |
| 2.4 | 实现通用 API | 在 `api_impl.py` 中实现 `edit_message`, `delete_message`, `get_group_info` 等 Telegram 支持的通用 API |
| 2.5 | 实现透传 API | 在 `adapter.py` 中实现 `call_platform_api`，将 action 映射到 Telegram Bot API 调用 |
| 2.6 | 声明能力 | 在 `manifest.yaml` 或适配器类中声明支持的事件和 API 列表 |
| 2.7 | 新建 Adapter 主类 | `TelegramAdapter` 继承 `AbstractPlatformAdapter`（新基类），委托各模块实现 |
| 2.8 | 更新 manifest.yaml | 更新 `execution.python.path` 指向新位置 |
| 2.9 | 验证 | 确保新适配器通过现有消息收发流程的测试 |
### 3.3 基础设施任务
| # | 任务 | 说明 |
|---|------|------|
| 2.A | 创建 `adapters/_base/` | 将 SDK 中新基类的运行时辅助代码放在此处（如事件分发辅助函数） |
| 2.B | 更新 ComponentDiscovery | 使 `discover_blueprint` 支持扫描 `adapters/` 子目录中的 YAML |
| 2.C | 更新 `templates/components.yaml` | 将 `fromDirs` 从 `pkg/platform/sources/` 改为 `pkg/platform/adapters/`（过渡期两个都扫描） |
| 2.D | 保留旧 sources/ | 过渡期不删除旧文件，通过 manifest 的 `deprecated: true` 标记 |
### 3.4 验收标准
- [ ] 已迁移的适配器在新目录结构下正常启动和收发消息
- [ ] 新事件（如 `message.edited`）在支持的平台上正确触发
- [ ] 通用 API（如 `edit_message`）在支持的平台上正确执行
- [ ] 未迁移的适配器（仍在 `sources/`）继续正常工作
- [ ] ComponentDiscovery 同时扫描新旧目录
---
## 4. Phase 3：核心系统
**目标**：实现 EventBus、EventRouter 和事件处理器框架，将事件从适配器分发到不同的处理器。
**仓库**：`LangBot`（后端）
### 4.1 任务清单
| # | 任务 | 文件/模块 | 说明 |
|---|------|----------|------|
| 3.1 | 实现 EventBus | `pkg/platform/event_bus.py`（新文件） | 事件总线：接收适配器事件，进行日志记录，分发给 EventRouter |
| 3.2 | 实现 EventRouter | `pkg/platform/event_router.py`（新文件） | 事件路由引擎：读取 Bot 的 `event_handlers` 配置，匹配事件类型，分发到对应 Handler |
| 3.3 | 实现 PipelineHandler | `pkg/platform/handlers/pipeline_handler.py` | 将 `message.received` 事件转为现有 Query，进入 Pipeline 流水线 |
| 3.4 | 实现 AgentHandler | `pkg/platform/handlers/agent_handler.py` | 直接调用 RequestRunner 处理事件，不经过 Pipeline 多 Stage 流程 |
| 3.5 | 实现 WebhookHandler | `pkg/platform/handlers/webhook_handler.py` | 将事件 POST 到外部 URL，解析响应执行动作（重构现有 WebhookPusher） |
| 3.6 | 实现 PluginHandler | `pkg/platform/handlers/plugin_handler.py` | 将事件分发给插件 EventListener（复用现有 plugin_connector 机制） |
| 3.7 | Bot 实体扩展 | `pkg/entity/persistence/bot.py` | 新增 `event_handlers` JSON 字段 |
| 3.8 | 数据库迁移 | `pkg/persistence/migrations/` | 新增迁移脚本：添加 `event_handlers` 列，将现有 `use_pipeline_uuid` 数据迁移为 `event_handlers` 格式 |
| 3.9 | 重构 RuntimeBot | `pkg/platform/botmgr.py` | 将 `initialize()` 中硬编码的 `on_friend_message`/`on_group_message` 回调替换为通过 EventBus 分发所有事件 |
| 3.10 | 重构 MessageAggregator | `pkg/pipeline/aggregator.py` | 从 RuntimeBot 解耦，作为 PipelineHandler 的内部机制（只对 `message.received` 事件生效） |
| 3.11 | Agent Handler 中 RequestRunner 解耦 | `pkg/provider/runner.py` + handlers | RequestRunner 需要能独立于 Pipeline Stage 运行，为 Agent Handler 提供轻量调用路径 |
| 3.12 | HTTP API 扩展 | `pkg/api/http/controller/` | 新增/更新 Bot API 端点以支持 `event_handlers` 的 CRUD |
### 4.2 数据迁移策略
现有 Bot 表有 `use_pipeline_uuid` 字段，需要自动迁移为 `event_handlers`：
该步骤只改写 Bot 的路由表示，仍通过原 UUID 指向原 Pipeline；不得创建独立 Agent，也不得复制 Pipeline 内嵌的 runner 配置。用户需要 Agent 时自行新增并绑定。
```python
# 迁移逻辑伪代码
for bot in all_bots:
if bot.use_pipeline_uuid:
bot.event_handlers = [
{
"event_type": "message.received",
"handler_type": "pipeline",
"handler_config": {
"pipeline_uuid": bot.use_pipeline_uuid
}
}
]
else:
bot.event_handlers = []
```text
adapter event
-> EventBus observer broadcast
-> EventRouter match event_bindings
-> one target: Pipeline | Agent | discard
-> Host delivery
```
### 4.3 RuntimeBot 重构要点
### 约束
当前 `RuntimeBot.initialize()` 硬编码注册两个回调：
- Plugin EventListener 是 observer，不作为 priority fallback。
- Pipeline 只处理消息事件并复用完整 Stage 链。
- Agent 使用独立 Agent 配置和 AgentRunner Host orchestrator。
- edit/reaction 等事件的 observer 副作用能力按事件和 adapter 能力过滤。
- dry-run 与合成派发必须使用同一匹配器，避免 UI 预览与真实路由漂移。
```python
# 现有代码 (botmgr.py)
self.adapter.register_listener(FriendMessage, on_friend_message)
self.adapter.register_listener(GroupMessage, on_group_message)
```
### 验收
重构后改为注册通用事件回调：
- 精确、namespace wildcard、全局 wildcard、filters 与 priority 有单元覆盖。
- 同一事件最多一个响应目标，但 observer 仍能收到事件。
- Pipeline 与 Agent 可以在同一个 Bot 的不同 binding 中同时生效。
```python
# 新代码
async def on_event(event: Event, adapter: AbstractPlatformAdapter):
await self.event_bus.emit(
bot_uuid=self.bot_entity.uuid,
event=event,
adapter=adapter,
)
## 6. P3：独立 Agent 与 AgentRunner
# 注册所有事件类型的统一回调
self.adapter.register_listener(Event, on_event)
```
### 工作项
EventBus 接收事件后，调用 EventRouter 按配置分发。
- `agents` 只保存 Agent；Pipeline 继续使用自己的表和 API。
- Agent config 使用 `runner.id` 与 `runner_config[runner_id]`。
- registry 只展示已安装、有效的插件 AgentRunner。
- Host 构造 run-scoped resources、state、delivery 与 event log/transcript。
- SDK/Python `call_tool` 和 scoped MCP bridge 都回到同一个 Host ToolManager。
- Box session 由 Host 将 instance/workspace/bot/adapter/target/thread scope 规范化并哈希为固定长度 `lb-box-<sha256>`；同 scope 稳定、不同 scope 隔离、缺少 identity 时 fail closed。
### 4.4 事件处理器执行流程
### 验收
```
EventBus.emit(bot_uuid, event, adapter)
│
▼
EventRouter.route(bot_uuid, event)
│ 查询 bot.event_handlers 配置
│ 匹配 event_type（精确匹配 > 通配符 *）
▼
匹配到的 Handler(s)
│
├── PipelineHandler.handle(event, adapter)
│ │ 仅支持 message.received
│ │ 构造 Query → MessageAggregator → QueryPool → Pipeline
│ └── 沿用现有完整流水线机制
│
├── AgentHandler.handle(event, adapter)
│ │ 根据 handler_config 选择 RequestRunner
│ │ 直接调用 runner.run() 处理事件
│ └── 将结果通过 adapter API 回复
│
├── WebhookHandler.handle(event, adapter)
│ │ 序列化事件为 JSON
│ │ POST 到 handler_config.url
│ └── 解析响应，执行动作（回复消息、调用 API 等）
│
└── PluginHandler.handle(event, adapter)
│ 通过 plugin_connector 分发给插件
└── 插件 EventListener 处理
```
- 安装/卸载 Runner 后 metadata 与表单选项同步。
- Runner 无法调用未授权 model/tool/knowledge/state。
- 两种 Host callback transport 不能覆盖 sandbox session id。
- LocalAgent、ACP/ClaudeCode/Codex 与外部服务 Runner 不需要 Host id 特判。
### 4.5 验收标准
## 7. P4：WebUI
- [ ] `message.received` 事件通过 PipelineHandler 正确进入现有 Pipeline（与旧行为一致）
- [ ] 新增事件（如 `group.member_joined`）能通过 PluginHandler 分发给插件
- [ ] AgentHandler 能直接调用 RequestRunner（至少 `local-agent`）处理事件并回复
- [ ] WebhookHandler 能将事件 POST 到外部 URL
- [ ] 数据库迁移正确执行，`use_pipeline_uuid` 数据迁移到 `event_handlers`
- [ ] 现有 Bot 在不修改配置的情况下行为不变（自动迁移保证）
### 工作项
---
- `/home/agents` 聚合显示 Agent 与 Pipeline，并明确类型。
- 创建时选择 Agent 或 Pipeline，编辑时进入各自表单。
- Agent 表单读取动态 Runner metadata，保存当前 `runner` / `runner_config` 形状。
- Bot 事件编排器编辑 `event_pattern`、target、filters、priority 与 enabled。
- 非消息事件过滤 Pipeline；Agent 按声明事件能力过滤。
## 5. Phase 4：插件 SDK 集成
### 验收
**目标**：将新事件和 API 通过插件 SDK 暴露给插件开发者，同时实现兼容层。
- 页面不出现 LocalAgent 专属 banner、变量隐藏或 Box/Pipeline 注入逻辑。
- 空 Runner 市场状态给出可安装 AgentRunner 的正常路径。
- Pipeline Debug Chat/Monitoring 与 Agent 运行日志分别可用。
**仓库**：`langbot-plugin-sdk` + `LangBot`
## 8. P5：发布门禁
### 5.1 任务清单
### 自动化
| # | 任务 | 说明 |
|---|------|------|
| 4.1 | 新增插件事件包装 | 在 `api/entities/events.py` 中为每个通用事件新增插件级事件类（如 `MessageEditedReceived`, `MemberJoinedReceived`） |
| 4.2 | 兼容层实现 | `PersonMessageReceived` / `GroupMessageReceived` 由新的 `MessageReceivedEvent` 自动生成，旧事件作为新事件的 alias |
| 4.3 | 新 API 暴露 | 在 `LangBotAPIProxy` 中新增方法：`edit_message`, `delete_message`, `get_group_info`, `get_user_info`, `call_platform_api` 等 |
| 4.4 | 通信协议扩展 | 在 `entities/io/actions/enums.py` 中新增 action 枚举（如 `EDIT_MESSAGE`, `DELETE_MESSAGE`, `GET_GROUP_INFO`, `CALL_PLATFORM_API`） |
| 4.5 | Runtime Handler 扩展 | 在 PluginConnectionHandler / ControlConnectionHandler 中添加新 action 的处理逻辑 |
| 4.6 | EventListener 扩展 | 确保 `@handler()` 装饰器支持注册新事件类型 |
| 4.7 | QueryBasedAPI 扩展 | 在 `QueryBasedAPIProxy` 中新增事件上下文相关的 API（如 `get_event_source_adapter`） |
- LangBot backend unit/integration tests 与 Ruff。
- SDK AgentRunner/proxy/MCP bridge tests。
- Web lint/build 与关键 Playwright cases。
- `skills/bin/lbs validate`、`skills/bin/lbs index --check`。
- LocalAgent 与其他官方 Runner plugin package/test gate。
### 5.2 兼容层详细设计
### 真实环境
```
新事件系统 旧事件系统（兼容层）
───────────── ─────────────────
MessageReceivedEvent ┌→ PersonMessageReceived (chat_type == "private")
(chat_type: "private"|"group") ┤
└→ GroupMessageReceived (chat_type == "group")
```
- 至少一个消息事件走 Pipeline。
- 至少一个非消息事件走独立 Agent 并执行平台动作。
- SDK/Python 和 MCP bridge 各完成一次受权工具调用，并证明二者都进入同一个 `PluginToRuntimeAction.CALL_TOOL` Host handler。
- Box 可用/不可用的降级路径可观察且无 Runner 特判。
**实现方式**：在 RuntimeEventDispatcher 中，当分发 `MessageReceivedEvent` 给插件时，同时生成对应的旧事件类实例。插件可以用新事件类或旧事件类注册 handler，都能收到。
## 9. 非目标
### 5.3 验收标准
- [ ] 现有插件（使用旧事件和 API）无需修改即可运行
- [ ] 新插件可以使用新事件类型（如 `MemberJoinedReceived`）注册 handler
- [ ] 新 API（如 `edit_message`）可通过 `self.edit_message()` 或 `event_context.edit_message()` 调用
- [ ] 透传 API `call_platform_api` 可正常调用适配器特有功能
- [ ] 所有新 action 的通信协议正确工作（stdio / WebSocket）
---
## 6. Phase 5：WebUI 编排面板
**目标**：在 WebUI 的 Bot 管理页面实现事件处理器的可视化编排。
**仓库**：`LangBot`（前端 `web/`）
### 6.1 任务清单
| # | 任务 | 说明 |
|---|------|------|
| 5.1 | Bot 编辑页面扩展 | 在 Bot 编辑页面新增「事件处理」面板 |
| 5.2 | 事件处理器列表组件 | 可视化展示当前 Bot 的 `event_handlers` 列表，支持增删改排序 |
| 5.3 | 事件类型选择器 | 下拉选择事件类型（命名空间分组展示），支持通配符 `*` |
| 5.4 | Handler 类型选择与配置 | 选择 handler 类型后展示对应的配置表单（Pipeline 选择器、Runner 选择器、Webhook URL 等） |
| 5.5 | Pipeline Handler 配置 | 复用现有的 Pipeline 选择 UI（从现有 `use_pipeline_uuid` 选择器迁移） |
| 5.6 | Agent Handler 配置 | Runner 选择器（local-agent / dify / n8n / coze 等）+ Runner 参数配置表单 |
| 5.7 | Webhook Handler 配置 | URL 输入、认证方式选择、Header 配置 |
| 5.8 | Plugin Handler 配置 | 通常无需额外配置，分发给所有匹配的插件 EventListener |
| 5.9 | HTTP API 对接 | 前端调用后端 API 保存/读取 `event_handlers` 配置 |
| 5.10 | 迁移提示 | 对于从旧版本升级的用户，如果检测到 `use_pipeline_uuid` 已自动迁移，展示提示说明 |
### 6.2 UI 交互设计概要
```
┌─ Bot 编辑页面 ─────────────────────────────────────┐
│ │
│ 基本信息 │ 适配器配置 │ ★ 事件处理 │ │
│ │
│ ┌─ 事件处理器列表 ────────────────────────────┐ │
│ │ │ │
│ │ ① message.received → Pipeline: "主流水线" │ │
│ │ [编辑] [删除] │ │
│ │ │ │
│ │ ② group.member_joined → Agent: local-agent │ │
│ │ [编辑] [删除] │ │
│ │ │ │
│ │ ③ * (默认) → Plugin │ │
│ │ [编辑] [删除] │ │
│ │ │ │
│ │ [+ 添加事件处理器] │ │
│ │ │ │
│ └──────────────────────────────────────────────┘ │
│ │
│ [保存] [取消] │
└─────────────────────────────────────────────────────┘
```
### 6.3 验收标准
- [ ] 用户可以在 WebUI 上为 Bot 添加/编辑/删除事件处理器
- [ ] 四种 Handler 类型均有对应的配置表单
- [ ] 配置保存后正确写入数据库 `event_handlers` 字段
- [ ] 旧版本升级后，自动迁移的配置在 UI 上正确展示
- [ ] Pipeline Handler 的行为与旧的 `use_pipeline_uuid` 完全一致
---
## 7. Phase 6：文档与示例
**目标**：更新所有面向开发者的文档和示例。
**仓库**：`langbot-wiki`, `langbot-plugin-demo`
### 7.1 任务清单
| # | 任务 | 仓库 | 说明 |
|---|------|------|------|
| 6.1 | EBA 架构概览文档 | langbot-wiki | 面向用户的新架构说明 |
| 6.2 | 适配器开发指南更新 | langbot-wiki | 如何开发一个新的适配器（新目录结构、新基类、事件转换等） |
| 6.3 | 插件开发指南更新 | langbot-wiki | 新事件类型、新 API 的使用说明 |
| 6.4 | 插件迁移指南 | langbot-wiki | 现有插件如何迁移到新事件/API（如果需要使用新能力） |
| 6.5 | 事件处理器配置指南 | langbot-wiki | WebUI 上如何配置事件处理器 |
| 6.6 | 示例插件更新 | langbot-plugin-demo | HelloPlugin 增加新事件监听示例、新 API 调用示例 |
| 6.7 | 新示例插件 | langbot-plugin-demo | 新建一个示例展示非消息事件处理（如入群欢迎） |
---
## 8. 风险评估与缓解
### 8.1 技术风险
| 风险 | 影响 | 概率 | 缓解措施 |
|------|------|------|----------|
| 适配器迁移中断现有功能 | 高 | 中 | 新旧目录并存，ComponentDiscovery 同时扫描两个目录，逐个适配器迁移验证 |
| 事件模型不兼容导致插件崩溃 | 高 | 低 | 兼容层保证旧事件类型继续工作，新增类不修改旧类 |
| 数据库迁移失败 | 高 | 低 | 迁移脚本做前置校验，`use_pipeline_uuid` 在过渡期保留不删除 |
| RequestRunner 解耦破坏 Pipeline | 高 | 中 | Agent Handler 调用 Runner 的路径独立于 Pipeline，不修改现有 Pipeline Stage 中的 Runner 调用逻辑 |
| 性能回退（EventBus 额外开销） | 中 | 低 | EventBus 在进程内同步分发，无额外序列化/网络开销 |
| 各平台事件差异大难以统一 | 中 | 中 | 通用事件只抽象最大公约数字段，差异部分保留在 `source_platform_object`；不支持的事件走 `PlatformSpecificEvent` |
### 8.2 兼容性风险
| 风险 | 缓解措施 |
|------|----------|
| 现有插件使用旧事件类 | 兼容层自动将新事件转为旧事件分发，两种事件类都能注册 handler |
| 现有插件调用 `reply()` / `send_message()` | 这两个 API 保持不变，只是底层实现可能微调 |
| 第三方基于 `AbstractMessagePlatformAdapter` 开发的适配器 | 旧基类保留，新基类继承旧基类，第三方适配器无需立即迁移 |
| 用户自定义 Pipeline 配置 | Pipeline 机制完整保留，PipelineHandler 只是入口变了（从 RuntimeBot 硬编码变为 EventRouter 配置） |
### 8.3 回滚策略
每个 Phase 独立可回滚：
- **Phase 1**（SDK 新增类）：删除新增文件，回退 SDK 版本号
- **Phase 2**（适配器目录）：恢复 `components.yaml` 的 `fromDirs` 指向旧目录，旧 sources/ 未删除
- **Phase 3**（核心系统）：回退数据库迁移，恢复 RuntimeBot 旧的硬编码回调
- **Phase 4**（插件集成）：回退 SDK 版本，插件使用旧版 SDK
- **Phase 5**（WebUI）：前端回退，Bot 编辑页面隐藏事件处理面板
---
## 9. 里程碑与时间线建议
| 里程碑 | 阶段 | 预期产出 |
|--------|------|----------|
| M1 | Phase 1 完成 | SDK 新版本发布，包含新事件/实体/基类定义 |
| M2 | Phase 2 首批适配器（Telegram + Discord） | 两个参考实现，验证目录结构和事件/API 体系 |
| M3 | Phase 3 核心系统 | EventBus + EventRouter + 四种 Handler 可用 |
| M4 | Phase 2 剩余适配器 | 所有活跃适配器迁移完成 |
| M5 | Phase 4 插件集成 | 新 SDK 发布，插件可使用新事件和 API |
| M6 | Phase 5 WebUI | 事件处理器编排面板上线 |
| M7 | Phase 6 文档 | 开发者文档和示例更新完毕 |
建议 M1-M3 作为第一个大版本发布（如 v5.0），M4-M7 在后续小版本迭代中完成。
---
## 10. 开发指引
### 10.1 分支策略
建议在主仓库创建 `feature/eba` 长期特性分支，各 Phase 在子分支上开发后合入特性分支：
```
main
└── feature/eba
├── feature/eba-sdk-entities (Phase 1)
├── feature/eba-adapter-telegram (Phase 2)
├── feature/eba-adapter-discord (Phase 2)
├── feature/eba-core-system (Phase 3)
├── feature/eba-plugin-sdk (Phase 4)
└── feature/eba-webui (Phase 5)
```
### 10.2 测试策略
| 层次 | 测试内容 | 工具 |
|------|----------|------|
| 单元测试 | 事件序列化/反序列化、实体构造、API 调用 mock | pytest |
| 集成测试 | EventBus → EventRouter → Handler 全链路 | pytest + asyncio |
| 适配器测试 | 各适配器的事件转换、消息转换、API 调用 | pytest + mock SDK |
| 端到端测试 | 从模拟平台事件到完整处理流程 | staging 环境 |
| 插件兼容性测试 | 旧插件在新系统下的行为 | langbot-plugin-demo |
### 10.3 代码审查关注点
- 新增代码是否影响现有行为
- 兼容层是否正确映射所有旧事件/API 场景
- 数据库迁移是否可逆
- 新 API 的错误处理（`NotSupportedError`）是否一致
- 事件模型的序列化在 stdio/WebSocket 通信中是否正确
- 不把 Pipeline 改名或包装成 Agent。
- 不自动把 Pipeline 内 runner 配置迁移成 Agent。
- 不为 3.x 保留数据库迁移、旧模板或配置 fallback。
- 不要求所有 Runner 经 MCP；本地 Python Runner 可以直接使用 SDK。
- 不允许 Runner 自定义 Box session scope。
- 不在首版实现多 Agent 串并联；多步骤编排留给后续 workflow。
docs/review/box-architecture.md
+19 -18
View File
@@ -1,6 +1,6 @@
# Box 系统架构深度分析
> 更新日期: 2026-06-02
> 更新日期: 2026-07-12
> 状态更新: 自部署社区版已具备发布条件（box 可选、降级完善、无迁移欠债）；工具调用循环上限、配额遍历异步化、`host_path` 挂载白名单等已落地。剩余多租户 / 安全硬化项见 [SaaS 阻塞项清单](./box-issues.md)。
> 分支: `feat/sandbox` (LangBot + langbot-plugin-sdk)
> 相关文档: [SaaS 阻塞项](./box-issues.md) | [Session 作用域](./box-session-scope.md) | [Runtime 对比](./box-vs-plugin-runtime.md) | [测试覆盖](./box-test-coverage.md) | [toB 分析](./box-tob-analysis.md)
@@ -13,7 +13,9 @@
┌──────────────────────────────────────────────────────────────────┐
│ LangBot 主进程 │
│ │
│ LocalAgentRunner ──> ToolManager ──> NativeToolLoader │
│ AgentRunner ──> SDK call_tool / scoped MCP bridge │
│ │ │ │
│ └────────────────> ToolManager ──> NativeToolLoader │
│ │ │ │ │
│ │ │ exec / read / write / edit │
│ │ │ glob / grep │
@@ -32,7 +34,7 @@
│ ├─ Host mount 校验 (allowed_mount_roots) │
│ ├─ Workspace quota 检查 │
│ ├─ 输出截断 (head+tail) │
│ ├─ Session ID 模板解析 (resolve_box_session_id) │
│ ├─ Host scope 哈希 (resolve_box_session_id) │
│ ├─ 技能挂载组装 (build_skill_extra_mounts) │
│ ├─ 重连循环 (_reconnect_loop, 指数退避) │
│ └─ BoxRuntimeConnector │
@@ -85,7 +87,8 @@
**核心设计原则**:
- Box Runtime 作为独立进程运行，通过 Action RPC 与 LangBot 主进程通信，两者复用 SDK 的 IO 层（Handler → Connection → Controller）
- 一个 session_id 对应一个容器/沙箱实例。同一 session 内可并存多条 mount 与多个 managed process
- Skill / 默认 exec / MCP Server 共享同一个 session 容器（详见 [box-session-scope.md](./box-session-scope.md)）
- AgentRunner 无权指定 session scope。SDK/Python `call_tool` 与 scoped MCP bridge 都发出同一个 `PluginToRuntimeAction.CALL_TOOL`，最终由 Host 的 ToolManager 执行，并使用当前 run 保存的同一个 execution Query
- Box 内托管的 stdio MCP server 使用独立的长期 `mcp-shared` session；它不是 AgentRunner 本次事件的 sandbox session（详见 [box-session-scope.md](./box-session-scope.md)）
---
@@ -93,7 +96,7 @@
### 2.1 BoxService (`pkg/box/service.py`, 722 行)
应用层门面，协调 Profile、安全校验、配额、连接、Skill 挂载与 Session 模板：
应用层门面，协调 Profile、安全校验、配额、连接、Skill 挂载与 Host scope 哈希：
主要公开方法（按定义顺序）：
@@ -104,7 +107,7 @@ BoxService
├─ _reconnect_loop(connector) 指数退避重连
├─ available (property) 连接状态
│
├─ resolve_box_session_id(query) 从 pipeline 模板解析 session_id
├─ resolve_box_session_id(query) 哈希 Host 私有 scope，生成固定长度 session_id
├─ build_skill_extra_mounts(query) 组装 pipeline-bound skill 的挂载列表
│
├─ execute_tool(parameters, query) Agent 调用 exec 时的入口
@@ -137,6 +140,8 @@ BoxService
**输出截断**: 默认 4000 字符上限，保留前 60% + 后 40%，中间插入 `[...truncated...]`。
**Session 所有权**: `resolve_box_session_id(query)` 只接受 Host 已确定的私有 scope 或 Query launcher/session identity，并输出 `lb-box-` + 64 位小写 SHA-256 十六进制摘要（固定 71 个 ASCII 字符）。哈希输入是 canonical JSON，包含 instance、workspace、bot、platform adapter、target type/id 与 thread；原始用户、群组、conversation 或 event id 不会出现在 Box session id 中。相同 Host scope 稳定复用，不同 target/thread/workspace/bot/adapter/instance 相互隔离；缺少可用 identity 时 fail closed。Pipeline、Agent 或 AgentRunner 配置都不能覆盖该规则。
**Skill 挂载合并**: `execute_tool()` 调用时，`build_skill_extra_mounts(query)` 会把当前 pipeline-bound 的所有 skill 的 `package_root` 作为 `extra_mounts` 加入 BoxSpec，挂在 `/workspace/.skills/<name>`。LLM 通过 `activate` 工具显式激活某个 skill 后，工具调用才允许引用这个 skill 的虚拟路径。
### 2.2 BoxRuntimeConnector (`pkg/box/connector.py`, 357 行)
@@ -414,7 +419,9 @@ ToolManager.initialize()
1. 验证 skill 已激活
2. 单次 exec 只能引用一个 skill 包
3. 若 skill 是 Python 项目（有 `requirements.txt` 或 `pyproject.toml`），命令会被 venv bootstrap 包裹（在 skill 挂载点内创建 `.venv`）
4. 调用 `box_service.execute_tool()` → 走默认 session_id 与已组装好的 `extra_mounts`，**不再为每 skill 起独立 session**
4. 调用 `box_service.execute_tool()` → 走 Host 从当前事件生成的 session_id 与已组装好的 `extra_mounts`，**不再为每 skill 起独立 session**
AgentRunner 可以直接通过 SDK/Python `AgentRunAPIProxy.call_tool` 调用这些工具，也可以让外部 harness 通过 SDK-owned scoped MCP bridge 回调。两条入口都发送 `PluginToRuntimeAction.CALL_TOOL`，共享同一个 run authorization、Host session 中保存的 execution Query、ToolManager 与 `resolve_box_session_id(query)` 规则；Runner 不能提交自定义 Box session id。Pipeline run 保存原 Query；纯 EBA run 由 Host 构造 `pipeline_config=None`、`pipeline_uuid=None` 的最小 Query。
### 4.3 MCP-in-Box (`mcp_stdio.py`, 354 行)
@@ -422,7 +429,7 @@ ToolManager.initialize()
```
initialize()
1. 复用/创建共享 session (session_id = _build_box_session_id())
1. 复用/创建共享 session (`session_id = mcp-shared`)
- persistent=True，长期保持
2. workspace.execute_raw(install_cmd) 安装依赖 (可选)
3. 将每个 MCP server 文件 stage 到 /workspace/.mcp/<process_id>/
@@ -435,6 +442,8 @@ initialize()
每条 MCP server 是同一 session 中的一个 managed process，独立的 `process_id`、独立 attach URL，互不阻塞。
这里的 `mcp-shared` 只承载 LangBot 管理的 stdio MCP server 进程。AgentRunner 的 scoped MCP bridge 是回调 Host 工具的协议入口，不会把事件运行的 exec/read/write 改到 `mcp-shared`。
---
## 5. 启动与生命周期
@@ -566,22 +575,14 @@ volumes:
| http/sse MCP server | 正常 | 正常(不依赖 Box) |
| Skill 列表/读取 (`list_skills`/`get_skill`/`read_skill_file`) | 走 Box runtime | 走 LangBot 本地 `data/skills/` 只读 fallback |
| Skill 创建/编辑/安装/写文件 | 走 Box runtime | **HTTP 400** + 明确错误信息(`_require_box_for_write`) |
| Pipeline AI 配置中 `box-session-id-template` | 正常生效 | **前端 banner** 提示字段无效 |
| Pipeline 扩展页 `enable_all_skills` / 绑定 skill | 可编辑 | **前端禁用** + banner |
| 仪表盘 Box 状态卡片 | 绿点 / "已连接" | 灰点 / "已禁用"(disabled) 或 红点 / "已断开"(failed) |
> 后端拒写的边界条件:如果 `ap.box_service` **完全没装**(老式 dev mode,没经过 BuildAppStage),`_require_box_for_write` 视作 no-op,保留 `data/skills/` 本地路径——以兼容历史测试与最小化设置。生产环境总会装 `ap.box_service`,因此该 fallback 不会被触发。
### Pipeline 配置 (templates/metadata/pipeline/ai.yaml)
### Session scope
`local-agent.config.box-session-id-template` 控制 session 作用域，预设：
- `{launcher_type}_{launcher_id}` — 每个会话 (推荐，默认)
- `{launcher_type}_{launcher_id}_{sender_id}` — 群聊每个用户
- `{launcher_type}_{launcher_id}_{conversation_id}` — 每个对话上下文
- `{query_id}` — 每条消息（完全隔离）
详见 [box-session-scope.md](./box-session-scope.md)。
Pipeline 与 AgentRunner 配置不再暴露 sandbox session 模板。Host 将当前平台会话/事件 scope 规范化后哈希成固定长度的 `lb-box-<sha256>`；相同 scope 稳定复用，不同 scope 隔离，缺少 identity 时拒绝执行。SDK/Python 与 scoped MCP bridge 的工具调用遵守同一规则。详见 [box-session-scope.md](./box-session-scope.md)。
### REST API
docs/review/box-session-scope.md
+144 -365
View File
@@ -1,402 +1,181 @@
# Box Session Scope Design
# Box Session Scope
> Date: 2026-04-18 (last reviewed 2026-06-02)
> Status (2026-06-02): the self-hosted community edition is release-ready (box optional, clean degradation, no migration debt). Tool-call loop cap, async quota scan, and the host_path mount allowlist have landed. Remaining multi-tenant / security hardening is tracked in [box-issues.md](./box-issues.md).
> Branch: `feat/sandbox` (LangBot + langbot-plugin-sdk)
> Last reviewed: 2026-07-12
> Status: implemented Host-owned, hashed execution scope; Runner/Pipeline session templates are removed.
> Related: [Box Architecture](./box-architecture.md) | [Box vs Plugin Runtime](./box-vs-plugin-runtime.md)
---
## 1. Decision
## 0. Implementation Status (2026-05-19)
The LangBot Host owns the Box session used by an event run. A Pipeline, Agent,
or AgentRunner cannot choose a global, per-user, per-conversation, or per-query
sandbox mode.
This document was authored as a design proposal. The current `feat/sandbox` branch
has shipped the design largely as written:
`BoxService.resolve_box_session_id(query)` always returns this shape:
| Item | Status | Notes |
|------|--------|-------|
| `BoxMountSpec` + `BoxSpec.extra_mounts` | ✅ Shipped | SDK `box/models.py` |
| Docker / nsjail / E2B backends apply extra mounts | ✅ Shipped | Last gap closed by SDK commit `0fea9b1` (E2B) |
| `box-session-id-template` in `local-agent` pipeline config | ✅ Shipped | `templates/metadata/pipeline/ai.yaml`, default `{launcher_type}_{launcher_id}` |
| `BoxService.resolve_box_session_id(query)` | ✅ Shipped | `pkg/box/service.py:166` |
| `BoxService.build_skill_extra_mounts(query)` | ✅ Shipped | `pkg/box/service.py:189` |
| Skill exec uses unified container + extra mounts | ✅ Shipped | `pkg/provider/tools/loaders/native.py` skill branch |
| MCP-in-Box uses shared persistent session, multi-process | ✅ Shipped (earlier than originally scoped) | SDK commit `529088e`, LangBot `mcp_stdio.py:_build_box_session_id` |
| `BoxManagedProcessSpec.process_id` + multi-process per session | ✅ Shipped | `BoxRuntime` keeps `managed_processes: dict[pid, _ManagedProcess]` |
| Per-tenant / quota integration with templates | ❌ Not started | See [box-tob-analysis.md](./box-tob-analysis.md) |
The "Phase 2 deferred" note in §10 is **out of date** — MCP unification went in on
the same line. Pipeline-scoped (not user-scoped) MCP container is the realized
behavior: each pipeline's MCP servers share one `mcp-<pipeline>` session, and
user exec sessions use the template-derived id.
The remaining open work is multi-tenant overlays (tenant_id in session_id,
quota counters keyed by tenant), tracked in the toB analysis doc rather than here.
---
## 1. Problems
### 1.1 Default exec: per-message containers
Currently, `BoxService.execute_tool()` sets `session_id = str(query.query_id)` — an
auto-incrementing integer per incoming message. Every user message creates a new sandbox
container. Dependencies installed and in-container state are lost between messages.
### 1.2 Three isolated container pools
Default exec, skills, and MCP servers each manage their own containers with
independent session IDs:
| Path | Session ID | Container |
|--------------|-----------------------------------------------|-------------|
| Default exec | `str(query_id)` (per message) | Ephemeral |
| Skill exec | `skill-{launcher}_{id}-{skill_name}` | Per skill |
| MCP stdio | `mcp-{server_uuid}` | Per server |
This means a single logical user interaction can spawn 3+ containers that cannot
share state, see each other's files, or reuse installed dependencies.
### 1.3 Single bind mount limitation
`BoxSpec` currently supports only **one** `host_path` → `mount_path` bind mount.
This prevents mounting both a default workspace and skill directories into the
same container.
---
## 2. Concept Model
```
Platform Message
→ Query (query_id: int, auto-increment, per message)
→ Session (launcher_type + launcher_id, per chat window)
→ Conversation (uuid, per dialogue context within a Session)
```text
lb-box-<64 lowercase SHA-256 hex characters>
```
| Concept | Key | Example | Scope |
|---------------|-------------------------------------|----------------------------|------------------------------|
| Query | `query_id` | `42` | Single message |
| Session | `launcher_type` + `launcher_id` | `group_123456` | Chat window (group or PM) |
| Conversation | `conversation_id` (UUID) | `a1b2c3d4-...` | Dialogue context within a Session |
| Sender | `sender_id` | `789` | Individual user |
The result is exactly 71 ASCII characters. Raw platform, user, group,
conversation, thread, and event identifiers never appear in the Box session
id. This avoids unsafe path characters, unbounded identifier length, and
identity leakage through runtime/container metadata.
Note: in a **group chat**, all users share the same Session (keyed by `group_id`). The
individual sender is tracked as `sender_id` but does not affect Session/Conversation routing.
This rule replaces all former concepts of:
---
- Pipeline or Runner `box-session-id-template` fields;
- a global forced session template;
- API fields that let a caller supply sandbox scope;
- LocalAgent-specific Host injection of Box availability, scope, or Pipeline id.
## 3. Target Scenarios
## 2. Canonical Host scope
| # | Scenario | Box Granularity | Desired `session_id` |
|----|--------------------------------|------------------------------------------|---------------------------------------------------------|
| 1 | Personal assistant | 1 Box per user, long-lived | `{launcher_type}_{launcher_id}` |
| 2 | Customer service | 1 Box per customer, cross-pipeline | `{launcher_type}_{launcher_id}` |
| 3 | Internal employee tool | 1 Box per employee | `{launcher_type}_{launcher_id}` |
| 4 | Group chat shared assistant | 1 Box per group | `{launcher_type}_{launcher_id}` |
| 5 | Group chat isolated per user | 1 Box per user within a group | `{launcher_type}_{launcher_id}_{sender_id}` |
| 6 | Teaching (cross-channel) | 1 Box per student across groups/PMs | `{sender_id}` |
| 7 | One-off execution | 1 Box per message (current behavior) | `{query_id}` |
| 8 | Multi-project development | 1 Box per conversation context | `{launcher_type}_{launcher_id}_{conversation_id}` |
Before hashing, the Host creates a canonical, sorted JSON scope with these
dimensions:
No single fixed granularity covers all scenarios. A template-based approach is needed.
| Dimension | Purpose |
| --- | --- |
| `instance_id` | Isolate separate LangBot installations |
| `workspace_id` | Preserve workspace/tenant boundary when available |
| `bot_id` | Prevent two bots from sharing a sandbox accidentally |
| `platform_adapter` | Separate identical target ids from different adapters |
| `target_type` / `target_id` | Identify the platform session or event target |
| `thread_id` | Isolate threads within a target when available |
---
The canonical JSON is domain-separated and hashed by the Host. Runner input,
runner config, and tool parameters are not trusted sources for this scope.
## 4. Design Overview
### 2.1 Target identity priority
Two key changes:
The Host resolves `target_type` / `target_id` in this order:
1. **Unified container**: exec, skills, and MCP all share the same container per
session scope. No more separate container pools.
2. **Configurable session scope**: `session_id` is generated from a template with
pipeline variables, configurable per pipeline.
1. For a Pipeline-backed run, use the exact Query launcher tuple.
2. For a pure EBA run, use `delivery.reply_target.target_type/target_id`
(`launcher_type/launcher_id` aliases are accepted).
3. If there is no delivery target, use `conversation_id`.
4. For a non-message event without a conversation, use `event_id`, producing
an event-scoped sandbox.
### 4.1 Unified Container with Multiple Mounts
The adapter class or declared adapter capability supplies platform adapter
identity. The Host includes the active LangBot instance, workspace, bot, and
thread dimensions when they exist.
A single container per session scope is created on first use. It has:
### 2.2 Stability and isolation
- **Primary mount**: default workspace at `/workspace` (from `default_host_workspace`)
- **Skill mounts**: each pipeline-bound skill's `package_root` mounted at
`/workspace/.skills/{skill_name}/`
- **MCP servers**: run as managed processes inside the same container
The same normalized scope always produces the same hash, so repeated runs in
the same platform conversation reuse the same Box workspace. A rotating
transcript/conversation id does not change the scope when an explicit platform
reply target remains the same.
```
Container (session_id = "group_123456")
/workspace/ ← default workspace (bind mount, rw)
/workspace/.skills/web-search/ ← skill package (bind mount, rw)
/workspace/.skills/data-analysis/ ← skill package (bind mount, rw)
[managed process: mcp-server-a] ← MCP server running inside
[managed process: mcp-server-b] ← MCP server running inside
A different target, thread, workspace, bot, platform adapter, or LangBot
instance changes the hash. If delivery target is unavailable and
`conversation_id` is the fallback, different conversations also produce
different hashes. Event-scoped fallback isolates unrelated non-message events.
### 2.3 Fail closed
If the private Host scope marker is present but empty or malformed, Box rejects
execution with `BoxValidationError`. A direct Query without either a valid
Host scope or launcher/session identity is also rejected. There is no
`unknown`, raw query id, global, or caller-selected fallback.
## 3. Host execution Query
AgentRunner callbacks need a Host-owned Query view because model/tool loaders
already consume that type. The Query is internal and is never exposed as a
Runner-controlled object.
- A Pipeline run stores the exact current Query in `AgentRunSession`.
- A pure EBA run builds a minimal Query with a valid Session and
`pipeline_config=None`, `pipeline_uuid=None`.
- The Host attaches canonical `_host_box_scope` and the authorized skill names
in `_pipeline_bound_skills`.
- `PluginToRuntimeAction.CALL_TOOL` restores this Query from the active
`run_id` before dispatching to `ToolManager`.
This gives Pipeline and pure EBA execution the same Host tool path without
inventing a fake Pipeline for an independent Agent.
## 4. AgentRunner callback paths
AgentRunner implementations may use either callback transport:
1. SDK/Python runners call `AgentRunAPIProxy.call_tool`.
2. External harnesses call the SDK-owned scoped MCP bridge.
Both transports emit the same `PluginToRuntimeAction.CALL_TOOL`. The Host then
validates the same run authorization, restores the same execution Query, and
dispatches to the same ToolManager and BoxService.
```text
AgentRunner
+-- AgentRunAPIProxy.call_tool --------+
| |
+-- SDK-owned scoped MCP bridge -------+--> PluginToRuntimeAction.CALL_TOOL
--> run authorization
--> execution Query
--> ToolManager
--> BoxService
--> lb-box-<sha256>
```
This requires extending `BoxSpec` to support multiple mounts (see §5).
An AgentRunner is not required to use MCP. Local Python runners can use the SDK
directly; code-agent harnesses can use the bridge. The transports do not define
different authorization or sandbox semantics.
### 4.2 Session ID Template
## 5. Skills and mounts
A new field `box-session-id-template` in the `local-agent` pipeline runner config
controls the session scope:
Native exec and skill-backed exec for one Host scope use the same hashed
session. `BoxService.build_skill_extra_mounts(query)` adds visible, authorized
skill packages under `/workspace/.skills/<name>` when the session is created.
```yaml
# templates/metadata/pipeline/ai.yaml (under local-agent.config)
- name: box-session-id-template
label:
en_US: Sandbox Scope
zh_Hans: 沙箱作用域
description:
en_US: >-
Determines how sandbox environments are shared. Use variables to
control isolation granularity.
zh_Hans: >-
决定沙箱环境的共享方式。使用变量控制隔离粒度。
type: select
required: false
default: "{launcher_type}_{launcher_id}"
options:
- value: "{launcher_type}_{launcher_id}"
label:
en_US: Per chat (Recommended)
zh_Hans: 每个会话（推荐）
- value: "{launcher_type}_{launcher_id}_{sender_id}"
label:
en_US: Per user in chat
zh_Hans: 会话中每个用户
- value: "{launcher_type}_{launcher_id}_{conversation_id}"
label:
en_US: Per conversation context
zh_Hans: 每个对话上下文
- value: "{query_id}"
label:
en_US: Per message (isolated)
zh_Hans: 每条消息（完全隔离）
```
Skill activation controls which skill-backed tools and paths are available. It
does not create a different session and does not grant the Runner authority to
change the session id.
Available template variables (populated by PreProcessor in `query.variables`):
## 6. `mcp-shared` is a different session
| Variable | Source | Example |
|---------------------|---------------------------------|----------------------|
| `{launcher_type}` | `query.session.launcher_type` | `person` / `group` |
| `{launcher_id}` | `query.session.launcher_id` | `123456` |
| `{sender_id}` | `query.sender_id` | `789` |
| `{conversation_id}` | `conversation.uuid` | `a1b2c3d4-...` |
| `{query_id}` | `query.query_id` | `42` |
LangBot can host configured stdio MCP servers as managed processes inside Box.
Those long-lived infrastructure processes share the dedicated `mcp-shared`
session and are isolated from one another by `process_id`.
Default `{launcher_type}_{launcher_id}` covers scenarios 1–4 out of the box.
This is separate from the scoped MCP bridge above:
---
| Path | Purpose | Session rule |
| --- | --- | --- |
| AgentRunner scoped MCP bridge | Call authorized Host tools for one active run | Host-owned `lb-box-<sha256>` from the run execution Query |
| MCP-in-Box stdio server | Keep configured MCP server processes running | Dedicated persistent `mcp-shared` session |
## 5. SDK Changes: Multi-Mount BoxSpec
Calling a sandbox tool through the AgentRunner bridge never redirects the run
workspace into `mcp-shared`. Conversely, an MCP server's managed-process
lifecycle does not inherit the current event scope.
### 5.1 Model Extension
## 7. Configuration and compatibility
```python
# box/models.py
There is no Box session scope field in Pipeline metadata, AgentRunner config,
or the public Pipeline/Runner API. Operators configure the Box subsystem itself
(`box.enabled`, backend/runtime settings, profiles, mount allowlists, quotas,
and workspace roots), not per-Runner session templates.
class BoxMountSpec(pydantic.BaseModel):
"""A single bind mount specification."""
host_path: str
mount_path: str
mode: BoxHostMountMode = BoxHostMountMode.READ_WRITE
Old configuration containing `box-session-id-template` is unsupported in the
4.x contract. LangBot 4.x does not migrate LangBot 3.x configuration or
databases, so the removed field is not read as a compatibility fallback.
class BoxSpec(pydantic.BaseModel):
# ... existing fields ...
host_path: str | None = None # Primary mount (backward compat)
host_path_mode: BoxHostMountMode = BoxHostMountMode.READ_WRITE
mount_path: str = DEFAULT_BOX_MOUNT_PATH
extra_mounts: list[BoxMountSpec] = [] # NEW: additional mounts
```
## 8. Regression coverage
`extra_mounts` is additive — the existing `host_path` / `mount_path` pair remains
the primary mount for backward compatibility.
Release tests should prove:
### 5.2 Backend: Apply Extra Mounts
```python
# box/backend.py — CLISandboxBackend.start_session()
# Primary mount (unchanged)
if spec.host_path is not None and spec.host_path_mode != BoxHostMountMode.NONE:
args.extend(['-v', f'{spec.host_path}:{spec.mount_path}:{spec.host_path_mode.value}'])
# Extra mounts (NEW)
for mount in spec.extra_mounts:
if mount.mode != BoxHostMountMode.NONE:
args.extend(['-v', f'{mount.host_path}:{mount.mount_path}:{mount.mode.value}'])
```
Same pattern for nsjail backend.
---
## 6. LangBot Changes
### 6.1 Session ID Resolution
In `BoxService.execute_tool()`:
```python
# Before:
spec_payload.setdefault('session_id', str(query.query_id))
# After:
template = (query.pipeline_config or {}).get('ai', {}) \
.get('local-agent', {}).get('box-session-id-template',
'{launcher_type}_{launcher_id}')
variables = query.variables or {}
session_id = template.format_map(collections.defaultdict(
lambda: 'unknown', variables
))
spec_payload.setdefault('session_id', session_id)
```
### 6.2 Skill Exec: Use Same Container
Currently `native.py:_invoke_exec` creates a separate `BoxWorkspaceSession` per
skill with `host_path=package_root`. Instead:
1. Use the **same session_id** as default exec (from the template).
2. Pass the skill's `package_root` as an **extra mount** at
`/workspace/.skills/{skill_name}/` instead of replacing `/workspace`.
3. The container already has the default workspace at `/workspace`.
```python
# native.py — _invoke_exec, skill branch (REVISED)
# Same session_id as default exec
session_id = resolve_box_session_id(query)
spec_payload = {
'cmd': rewritten_command,
'workdir': rewritten_workdir,
'session_id': session_id,
'extra_mounts': [{
'host_path': package_root,
'mount_path': f'/workspace/.skills/{selected_skill_name}',
'mode': 'rw',
}],
}
result = await self.ap.box_service.execute_spec_payload(spec_payload, query)
```
The virtual path `/workspace/.skills/{name}` no longer needs rewriting at the
command level — it maps directly to the bind mount path inside the container.
### 6.3 MCP: Use Same Container
MCP servers should run inside the same container as exec and skills. Changes:
1. `BoxStdioSessionRuntime` uses the pipeline's session_id template instead of
`mcp-{server_uuid}`.
2. MCP server's working directory is a subdirectory (e.g. `/workspace/.mcp/{name}/`).
3. MCP server's dependencies are mounted or installed into that subdirectory.
4. The MCP server runs as a managed process inside the shared container.
Since MCP servers start at LangBot boot (not per-query), the session must be
created eagerly. The container will be kept alive by the managed process
exemption in TTL reaping (`runtime.py:259`).
**Note**: MCP sessions are pipeline-scoped (not per-launcher), so their session_id
should be a **fixed identifier per pipeline** rather than the user-facing template.
This means one shared MCP container per pipeline, with user exec sessions separate.
Alternatively, in a future iteration, MCP managed processes could be launched
lazily into the user's container on first MCP tool call. This is more complex
but maximizes sharing. For V1, keeping MCP containers at pipeline scope is
simpler and more predictable.
---
## 7. Mount Layout Summary
### Default exec (no skills activated)
```
Container (session_id from template)
/workspace/ ← default_host_workspace (rw)
```
### Exec with activated skills
```
Container (same session_id)
/workspace/ ← default_host_workspace (rw)
/workspace/.skills/web-search/ ← skill package_root (rw)
/workspace/.skills/data-analysis/ ← skill package_root (rw)
```
Extra mounts are **additive** — they are added when the container is first
created (or on the first exec that references a skill). Since Docker bind
mounts are specified at container creation time, skills must be known at
creation time.
**Resolution**: When creating a container, inject `extra_mounts` for **all
pipeline-bound skills** (from `extensions_preferences`), not just the
currently activated one. This way any skill can be activated later without
recreating the container.
### MCP servers (V1: pipeline-scoped)
```
Container (session_id = "mcp-pipeline-{pipeline_uuid}")
/workspace/ ← MCP shared workspace
/workspace/.mcp/server-a/ ← MCP server A files
/workspace/.mcp/server-b/ ← MCP server B files
[managed process: server-a]
[managed process: server-b]
```
---
## 8. Data Migration
Existing pipelines do not have `box-session-id-template`. The backend uses
`.get(..., default)` so missing keys fall back to `{launcher_type}_{launcher_id}`.
This changes behavior from per-message to per-launcher for existing pipelines.
Recommendation: **accept the behavior change** — per-launcher is the more
intuitive default, and the old per-message behavior was rarely desired.
---
## 9. Cloud Quota Implications
| Scope | Typical concurrent containers |
|-----------------------------------------------|-------------------------------|
| `{query_id}` (per message) | Many, short-lived |
| `{launcher_type}_{launcher_id}` (per chat) | = active chat count |
| `{sender_id}` (per user) | = active user count |
| `{conversation_id}` (per conversation) | Between per-chat and per-msg |
With the unified container model, each scope value maps to exactly **one**
container (instead of potentially 3+ per-message). This significantly reduces
resource usage.
Quota enforcement point: `BoxRuntime._get_or_create_session()` in the SDK.
---
## 10. Implementation Phases
### Phase 1: Session scope + skill unification (this PR)
1. **SDK**: Extend `BoxSpec` with `extra_mounts: list[BoxMountSpec]`.
2. **SDK**: Update Docker/nsjail backends to apply extra mounts.
3. **LangBot**: Add `box-session-id-template` to `local-agent` YAML metadata
and default pipeline config JSON.
4. **LangBot**: Update `BoxService.execute_tool()` to use template interpolation.
5. **LangBot**: Update `native.py:_invoke_exec` skill branch to use same
session_id + extra mounts instead of separate `BoxWorkspaceSession`.
6. **LangBot**: On container creation, inject extra mounts for all
pipeline-bound skills.
7. **Frontend**: No code change — `DynamicFormComponent` renders `select` fields.
8. **Tests**: Unit tests for template interpolation and multi-mount specs.
### Phase 2: MCP unification (future)
1. Refactor `BoxStdioSessionRuntime` to use pipeline-scoped shared container.
2. MCP servers become managed processes in the shared container.
3. Support multiple concurrent managed processes per container.
MCP unification is deferred because it requires changes to the managed process
model (currently 1 managed process per session) and has startup ordering
concerns (MCP servers start at boot, before any user query determines
a session_id).
- every event-run session id matches `lb-box-[0-9a-f]{64}` and contains no raw
identity;
- the same canonical Host scope is stable while different targets,
conversations, threads, bots, adapters, workspaces, or instances are
isolated;
- Pipeline and pure EBA runs representing the same platform session produce
the same canonical scope;
- missing Host/Query identity fails closed;
- SDK/Python `call_tool` and the scoped MCP bridge both enter
`PluginToRuntimeAction.CALL_TOOL` and restore the run execution Query;
- Runner payload/config cannot override the session id;
- stdio MCP processes remain in `mcp-shared` and are isolated by process id;
- authorized skills are mounted into the hashed run session without creating
per-skill sessions.
docs/review/box-test-coverage.md
+7 -6
View File
@@ -1,6 +1,6 @@
# Box 系统测试覆盖分析
> 更新日期: 2026-06-02
> 更新日期: 2026-07-12
> 状态更新: 自部署社区版已具备发布条件（box 可选、降级完善、无迁移欠债）；工具调用循环上限、配额遍历异步化、`host_path` 挂载白名单等已落地。剩余多租户 / 安全硬化项见 [SaaS 阻塞项清单](./box-issues.md)。
> 分支: `feat/sandbox` (LangBot + langbot-plugin-sdk)
@@ -15,13 +15,14 @@
| `tests/unit_tests/box/test_box_connector.py` | 106 | 是 | Connector 传输决策、WS relay URL、dispose、心跳/重连 |
| `tests/unit_tests/box/test_box_service.py` | 1224 | 是 | Service 核心逻辑（最全面） |
| `tests/unit_tests/box/test_workspace.py` | 147 | 是 | WorkspaceSession 路径重写、payload 构建 |
| `tests/unit_tests/agent/test_execution_context.py` | 144 | 是 | Pipeline/纯 EBA execution Query、canonical Host scope、adapter/instance 隔离 |
| `tests/unit_tests/plugin/test_handler_actions.py` | 1071 | 是 | AgentRun CALL_TOOL 恢复 execution Query、纯 EBA native exec |
| `tests/unit_tests/provider/test_mcp_box_integration.py` | 707 | 是 | MCP Box 配置、路径重写、payload、shared-session/multi-process、runtime info |
| `tests/unit_tests/provider/test_localagent_sandbox_exec.py` | 444 | 是 | LocalAgent exec 流程、流式、Skill 激活 (Tool Call) |
| `tests/unit_tests/provider/test_tool_manager_native.py` | 249 | 是 | ToolManager 路由、native tool CRUD、路径穿越、6 工具暴露 |
| `tests/unit_tests/provider/test_skill_tools.py` | 582 | 是 | Skill 管理、Tool Call 激活、路径、authoring CRUD |
| `tests/unit_tests/test_skill_service.py` | 396 | 是 | HTTP service：skill CRUD、zip/GitHub install、文件浏览 |
| `tests/unit_tests/test_paths.py` | 23 | 是 | paths 工具 |
| `tests/unit_tests/test_preproc.py` | 134 | 是 | PreProcessor 注入 session 变量、bound skill 解析 |
| `tests/unit_tests/test_preproc.py` | 134 | 是 | PreProcessor 的模型、历史与 bound skill 解析 |
| `tests/unit_tests/pipeline/test_chat_handler_logging.py` | 78 | 是 | Chat handler 日志相关回归 |
| `tests/integration_tests/box/test_box_integration.py` | 329 | **否** | 真实容器执行、超时、网络隔离 |
| `tests/integration_tests/box/test_box_mcp_integration.py` | 368 | **否** | Managed process、WS attach、shared-session 清理 |
@@ -35,7 +36,7 @@
| `tests/box/test_e2b_backend.py` | 482 | 是 | E2B SDK mock、session 生命周期、extra_mounts 同步 |
| `tests/box/test_skill_store.py` | 88 | 是 | zip preview/install、基础 file CRUD |
**总计**: 17 个测试文件, ~6,500 行测试代码; 其中 2 个集成测试（约 700 行）在 CI 中不运行。
**说明**: 本表按当前主链列出 Box 相关测试；其中 2 个真实容器集成测试默认不在 CI 中运行。
> 较 2026-04-16 版增加：`test_skill_service.py`、`test_paths.py`、`test_preproc.py`、`test_chat_handler_logging.py` (LangBot)，`test_backend_selection.py`、`test_e2b_backend.py`、`test_skill_store.py` (SDK)。`test_nsjail_backend.py` 增加 CLI 兼容性 case (commit `feed530`)。
@@ -51,7 +52,7 @@
| BoxService workspace quota | 优秀 | 前置/后置配额检查、超额清理 |
| BoxService 输出截断 | 优秀 | 短/精确边界/长输出、独立 stderr |
| BoxService 可观测性 | 优秀 | 状态报告、error ring buffer、buffer 上限 |
| BoxService session 模板 | 良好 | `resolve_box_session_id` + `build_skill_extra_mounts` 在 service / native / mcp 三处都有覆盖 |
| BoxService Host-owned session | 优秀 | 覆盖 `lb-box-<sha256>` 固定格式、原始 identity 不泄露、同 scope 稳定、不同 conversation/scope 隔离、缺 identity fail closed |
| RPC client/server 协议 | 优秀 | execute/get_sessions/delete/create/conflict error |
| BoxRuntimeConnector | 良好 | local/remote 模式、Docker 平台、relay URL、心跳与重连回调 |
| BoxWorkspaceSession | 良好 | payload 构建、managed process 路径重写、stage host file |
@@ -61,7 +62,7 @@
| Backend selection | 良好 | 显式 backend 优先级、local 探测顺序、配置变更触发 reselect |
| MCP Box 集成 | 良好 | config model、路径重写、payload、shared-session 多 process |
| Native tool loader | 良好 | 6 工具（exec/read/write/edit/glob/grep）、路径穿越拦截 |
| LocalAgent exec 流程 | 良好 | 完整 tool call 循环、流式、system prompt 注入、Tool Call 激活 |
| AgentRunner 工具入口 | 良好 | SDK proxy 与 MCP bridge 都映射到 `PluginToRuntimeAction.CALL_TOOL`；Host action 测试覆盖 run-scoped execution Query 与纯 EBA native exec |
| Skill 系统 | 良好 | 加载、Tool Call 激活、marker、路径解析、authoring CRUD、HTTP service |
---
skills/scripts/e2e/ensure-fake-provider-pipeline.mjs
-1
View File
@@ -522,7 +522,6 @@ async function ensurePipeline({ backendUrl, token, name, modelUuid }) {
prompt: [{ role: "system", content: "You are a deterministic QA assistant. Reply exactly as instructed." }],
"remove-think": false,
"knowledge-bases": [],
"box-session-id-template": "{launcher_type}_{launcher_id}",
"retrieval-top-k": 5,
"rerank-model": "",
"rerank-top-k": 5,
skills/scripts/e2e/pipeline-debug-chat.mjs
+3 -1
View File
@@ -112,7 +112,9 @@ function parseJsonEnv(key, fallback) {
}
function positiveNumberEnv(key, fallback) {
const value = Number(env[key] || "");
const raw = env[key];
if (raw === undefined || raw === "") return fallback;
const value = Number(raw);
return Number.isFinite(value) && value >= 0 ? value : fallback;
}
skills/skills/langbot-testing/cases/agent-runner-ledger-invariants.yaml
+1 -1
View File
@@ -17,7 +17,7 @@ env:
automation: skills/langbot-testing/probes/agent-runner-ledger-invariants.mjs
steps:
- "Run `rtk bin/lbs test run agent-runner-ledger-invariants --dry-run` first; remove `--dry-run` after checking the planned evidence directory."
- "Automation resolves LANGBOT_REPO, defaulting to ../LangBot, and imports the sibling SDK from LANGBOT_PLUGIN_SDK_REPO or ../langbot-plugin-sdk/src."
- "Automation resolves LANGBOT_REPO, defaulting to the parent LangBot checkout, and imports the sibling SDK from LANGBOT_PLUGIN_SDK_REPO or ../../langbot-plugin-sdk/src."
- "Automation checks run status sets, terminal status validation, ledger table/index DDL, and a minimal synchronous insert/read path."
checks:
- "automation-result.json status is pass."
skills/skills/langbot-testing/cases/agent-runner-qa-debug-chat.yaml
+1
View File
@@ -33,6 +33,7 @@ automation_expected_runner_id: "plugin:qa/agent-runner/default"
automation_prompt: "hello-live"
automation_expected_text: "QA_AGENT_RUNNER_OK:hello-live"
automation_response_timeout_ms: "120000"
automation_debug_chat_response_p95_ms: "120000"
automation_reset_debug_chat: "1"
setup_automation:
- "case:agent-runner-live-install"
skills/skills/langbot-testing/cases/agent-runner-runtime-chaos.yaml
+2 -2
View File
@@ -18,7 +18,7 @@ env:
automation: skills/langbot-testing/probes/agent-runner-runtime-chaos.mjs
steps:
- "Run `rtk bin/lbs test run agent-runner-runtime-chaos --dry-run` first; remove `--dry-run` after checking the SDK repo target and evidence directory."
- "Automation resolves LANGBOT_PLUGIN_SDK_REPO, defaulting to ../langbot-plugin-sdk when the env var is unset."
- "Automation resolves LANGBOT_PLUGIN_SDK_REPO, defaulting to ../../langbot-plugin-sdk when the env var is unset."
- "Automation runs the existing SDK pytest files tests/runtime/plugin/test_mgr_agent_runner.py and tests/runtime/test_pull_api_handlers.py."
checks:
- "automation-result.json status is pass."
@@ -28,7 +28,7 @@ evidence_required:
- filesystem
diagnostics:
- "This probe does not open the WebUI; it runs SDK pytest targets directly."
- "env_issue means LANGBOT_PLUGIN_SDK_REPO/default ../langbot-plugin-sdk did not resolve, rtk/uv was unavailable, or the expected test files are missing."
- "env_issue means LANGBOT_PLUGIN_SDK_REPO/default ../../langbot-plugin-sdk did not resolve, rtk/uv was unavailable, or the expected test files are missing."
- "fail means the existing SDK runtime pytest target failed or timed out."
success_patterns:
- "pytest passed"
skills/skills/langbot-testing/cases/local-agent-combo-rag-compaction-tool-debug-chat.yaml
+1 -1
View File
@@ -37,7 +37,7 @@ automation_pipeline_url_env: LANGBOT_LOCAL_AGENT_PIPELINE_URL
automation_pipeline_name_env: LANGBOT_LOCAL_AGENT_PIPELINE_NAME
automation_expected_runner_id: "plugin:langbot-team/LocalAgent/default"
automation_runner_config_patch_json: '{"knowledge-bases":["${LANGBOT_LOCAL_AGENT_RAG_KB_UUID}"],"retrieval-top-k":1,"context-window-tokens":650,"context-reserve-tokens":180,"context-keep-recent-tokens":120,"context-summary-tokens":220,"max-tool-iterations":4,"tool-execution-mode":"serial"}'
automation_extensions_patch_json: '{"enable_all_plugins":false,"bound_plugins":[{"author":"langbot","name":"local-agent"},{"author":"qa","name":"plugin-smoke"}],"enable_all_mcp_servers":false,"bound_mcp_servers":[],"enable_all_skills":false,"bound_skills":[]}'
automation_extensions_patch_json: '{"enable_all_plugins":false,"bound_plugins":[{"author":"langbot-team","name":"LocalAgent"},{"author":"qa","name":"plugin-smoke"}],"enable_all_mcp_servers":false,"bound_mcp_servers":[],"enable_all_skills":false,"bound_skills":[]}'
automation_restore_runner_config: "1"
automation_restore_extensions: "1"
automation_reset_debug_chat: "1"
skills/skills/langbot-testing/cases/local-agent-context-compaction-debug-chat.yaml
+1 -1
View File
@@ -33,7 +33,7 @@ automation_pipeline_url_env: LANGBOT_LOCAL_AGENT_PIPELINE_URL
automation_pipeline_name_env: LANGBOT_LOCAL_AGENT_PIPELINE_NAME
automation_expected_runner_id: "plugin:langbot-team/LocalAgent/default"
automation_runner_config_patch_json: '{"context-window-tokens":225,"context-reserve-tokens":50,"context-keep-recent-tokens":30,"context-summary-tokens":105,"knowledge-bases":[]}'
automation_extensions_patch_json: '{"enable_all_plugins":false,"bound_plugins":[{"author":"langbot","name":"local-agent"}],"enable_all_mcp_servers":false,"bound_mcp_servers":[],"enable_all_skills":false,"bound_skills":[]}'
automation_extensions_patch_json: '{"enable_all_plugins":false,"bound_plugins":[{"author":"langbot-team","name":"LocalAgent"}],"enable_all_mcp_servers":false,"bound_mcp_servers":[],"enable_all_skills":false,"bound_skills":[]}'
automation_restore_runner_config: "1"
automation_restore_extensions: "1"
automation_reset_debug_chat: "1"
skills/skills/langbot-testing/cases/local-agent-multitool-rag-compaction-debug-chat.yaml
+1 -1
View File
@@ -38,7 +38,7 @@ automation_pipeline_url_env: LANGBOT_LOCAL_AGENT_PIPELINE_URL
automation_pipeline_name_env: LANGBOT_LOCAL_AGENT_PIPELINE_NAME
automation_expected_runner_id: "plugin:langbot-team/LocalAgent/default"
automation_runner_config_patch_json: '{"knowledge-bases":["${LANGBOT_LOCAL_AGENT_RAG_KB_UUID}"],"retrieval-top-k":1,"context-window-tokens":900,"context-reserve-tokens":220,"context-keep-recent-tokens":160,"context-summary-tokens":260,"max-tool-iterations":5,"tool-execution-mode":"serial"}'
automation_extensions_patch_json: '{"enable_all_plugins":false,"bound_plugins":[{"author":"langbot","name":"local-agent"},{"author":"qa","name":"plugin-smoke"}],"enable_all_mcp_servers":false,"bound_mcp_servers":[],"enable_all_skills":false,"bound_skills":[]}'
automation_extensions_patch_json: '{"enable_all_plugins":false,"bound_plugins":[{"author":"langbot-team","name":"LocalAgent"},{"author":"qa","name":"plugin-smoke"}],"enable_all_mcp_servers":false,"bound_mcp_servers":[],"enable_all_skills":false,"bound_skills":[]}'
automation_restore_runner_config: "1"
automation_restore_extensions: "1"
automation_reset_debug_chat: "1"
skills/skills/langbot-testing/cases/local-agent-parallel-tools-rag-compaction-debug-chat.yaml
+1 -1
View File
@@ -38,7 +38,7 @@ automation_pipeline_url_env: LANGBOT_LOCAL_AGENT_PIPELINE_URL
automation_pipeline_name_env: LANGBOT_LOCAL_AGENT_PIPELINE_NAME
automation_expected_runner_id: "plugin:langbot-team/LocalAgent/default"
automation_runner_config_patch_json: '{"knowledge-bases":["${LANGBOT_LOCAL_AGENT_RAG_KB_UUID}"],"retrieval-top-k":1,"context-window-tokens":900,"context-reserve-tokens":220,"context-keep-recent-tokens":160,"context-summary-tokens":260,"max-tool-iterations":3,"tool-execution-mode":"parallel"}'
automation_extensions_patch_json: '{"enable_all_plugins":false,"bound_plugins":[{"author":"langbot","name":"local-agent"},{"author":"qa","name":"plugin-smoke"}],"enable_all_mcp_servers":false,"bound_mcp_servers":[],"enable_all_skills":false,"bound_skills":[]}'
automation_extensions_patch_json: '{"enable_all_plugins":false,"bound_plugins":[{"author":"langbot-team","name":"LocalAgent"},{"author":"qa","name":"plugin-smoke"}],"enable_all_mcp_servers":false,"bound_mcp_servers":[],"enable_all_skills":false,"bound_skills":[]}'
automation_restore_runner_config: "1"
automation_restore_extensions: "1"
automation_reset_debug_chat: "1"
skills/skills/langbot-testing/cases/local-agent-tool-error-recovery-debug-chat.yaml
+1 -1
View File
@@ -34,7 +34,7 @@ automation_pipeline_url_env: LANGBOT_LOCAL_AGENT_PIPELINE_URL
automation_pipeline_name_env: LANGBOT_LOCAL_AGENT_PIPELINE_NAME
automation_expected_runner_id: "plugin:langbot-team/LocalAgent/default"
automation_runner_config_patch_json: '{"knowledge-bases":[],"max-tool-iterations":3,"tool-execution-mode":"serial"}'
automation_extensions_patch_json: '{"enable_all_plugins":false,"bound_plugins":[{"author":"langbot","name":"local-agent"},{"author":"qa","name":"plugin-smoke"}],"enable_all_mcp_servers":false,"bound_mcp_servers":[],"enable_all_skills":false,"bound_skills":[]}'
automation_extensions_patch_json: '{"enable_all_plugins":false,"bound_plugins":[{"author":"langbot-team","name":"LocalAgent"},{"author":"qa","name":"plugin-smoke"}],"enable_all_mcp_servers":false,"bound_mcp_servers":[],"enable_all_skills":false,"bound_skills":[]}'
automation_restore_runner_config: "1"
automation_restore_extensions: "1"
automation_reset_debug_chat: "1"
skills/skills/langbot-testing/cases/local-agent-tool-loop-limit-debug-chat.yaml
+1 -1
View File
@@ -35,7 +35,7 @@ automation_pipeline_url_env: LANGBOT_LOCAL_AGENT_PIPELINE_URL
automation_pipeline_name_env: LANGBOT_LOCAL_AGENT_PIPELINE_NAME
automation_expected_runner_id: "plugin:langbot-team/LocalAgent/default"
automation_runner_config_patch_json: '{"knowledge-bases":[],"max-tool-iterations":2,"tool-execution-mode":"serial"}'
automation_extensions_patch_json: '{"enable_all_plugins":false,"bound_plugins":[{"author":"langbot","name":"local-agent"},{"author":"qa","name":"plugin-smoke"}],"enable_all_mcp_servers":false,"bound_mcp_servers":[],"enable_all_skills":false,"bound_skills":[]}'
automation_extensions_patch_json: '{"enable_all_plugins":false,"bound_plugins":[{"author":"langbot-team","name":"LocalAgent"},{"author":"qa","name":"plugin-smoke"}],"enable_all_mcp_servers":false,"bound_mcp_servers":[],"enable_all_skills":false,"bound_skills":[]}'
automation_restore_runner_config: "1"
automation_restore_extensions: "1"
automation_reset_debug_chat: "1"
skills/skills/langbot-testing/cases/mcp-stdio-tool-call.yaml
+1 -1
View File
@@ -32,7 +32,7 @@ automation_env:
automation_pipeline_url_env: LANGBOT_LOCAL_AGENT_PIPELINE_URL
automation_pipeline_name_env: LANGBOT_LOCAL_AGENT_PIPELINE_NAME
automation_expected_runner_id: "plugin:langbot-team/LocalAgent/default"
automation_extensions_patch_json: '{"enable_all_plugins":false,"bound_plugins":[{"author":"langbot","name":"local-agent"}],"enable_all_mcp_servers":false,"bound_mcp_servers":["${LANGBOT_MCP_QA_STDIO_SERVER_UUID}"],"enable_all_skills":false,"bound_skills":[]}'
automation_extensions_patch_json: '{"enable_all_plugins":false,"bound_plugins":[{"author":"langbot-team","name":"LocalAgent"}],"enable_all_mcp_servers":false,"bound_mcp_servers":["${LANGBOT_MCP_QA_STDIO_SERVER_UUID}"],"enable_all_skills":false,"bound_skills":[]}'
automation_restore_extensions: "1"
automation_reset_debug_chat: "1"
automation_prompt: "Call the qa_mcp_echo MCP tool with exactly this text: mcp-ok-local-agent. Return only the tool result."
skills/skills/langbot-testing/probes/agent-runner-async-db-readiness.mjs
+6 -2
View File
@@ -80,13 +80,17 @@ async function main() {
const evidenceDir = resolve(env.LBS_EVIDENCE_DIR || join(root, "reports", "evidence", runId));
await mkdir(evidenceDir, { recursive: true });
const startedAt = new Date();
const langbotRepo = resolve(root, env.LANGBOT_REPO || "../LangBot");
const langbotRepo = resolve(root, env.LANGBOT_REPO || "..");
const stdoutLog = join(evidenceDir, "probe-stdout.log");
const stderrLog = join(evidenceDir, "probe-stderr.log");
const automationResultJson = join(evidenceDir, "automation-result.json");
const resultJson = join(evidenceDir, "result.json");
const timeoutMs = Number(env.LANGBOT_ASYNC_DB_READINESS_TIMEOUT_MS || "5000");
const command = { executable: "rtk", args: ["uv", "run", "python", "-c", script], cwd: langbotRepo };
const command = {
executable: "rtk",
args: [resolve(langbotRepo, ".venv/bin/python"), "-c", script],
cwd: langbotRepo,
};
const result = {
source: "automation",
probe: "aiosqlite-readiness",
skills/skills/langbot-testing/probes/agent-runner-behavior-matrix.mjs
+8 -3
View File
@@ -142,15 +142,20 @@ async function main() {
const evidenceDir = resolve(env.LBS_EVIDENCE_DIR || join(root, "reports", "evidence", runId));
await mkdir(evidenceDir, { recursive: true });
const startedAt = new Date();
const langbotRepo = resolve(root, env.LANGBOT_REPO || "../LangBot");
const sdkSrc = resolve(root, env.LANGBOT_PLUGIN_SDK_REPO || "../langbot-plugin-sdk/src");
const langbotRepo = resolve(root, env.LANGBOT_REPO || "..");
const sdkRepo = resolve(root, env.LANGBOT_PLUGIN_SDK_REPO || "../../langbot-plugin-sdk");
const sdkSrc = resolve(sdkRepo, "src");
const fixturePath = resolve(root, "skills/langbot-testing/fixtures/agent-runner/qa-runner-behaviors.json");
const stdoutLog = join(evidenceDir, "probe-stdout.log");
const stderrLog = join(evidenceDir, "probe-stderr.log");
const automationResultJson = join(evidenceDir, "automation-result.json");
const resultJson = join(evidenceDir, "result.json");
const timeoutMs = Number(env.LANGBOT_AGENT_RUNNER_PROBE_TIMEOUT_MS || "30000");
const command = { executable: "rtk", args: ["uv", "run", "python", "-c", script, fixturePath], cwd: langbotRepo };
const command = {
executable: "rtk",
args: [resolve(langbotRepo, ".venv/bin/python"), "-c", script, fixturePath],
cwd: langbotRepo,
};
const result = {
source: "automation",
probe: "agent-runner-behavior-matrix",
skills/skills/langbot-testing/probes/agent-runner-fixture-contract.mjs
+2 -2
View File
@@ -129,7 +129,7 @@ async function main() {
const evidenceDir = resolve(env.LBS_EVIDENCE_DIR || join(root, "reports", "evidence", runId));
await mkdir(evidenceDir, { recursive: true });
const startedAt = new Date();
const sdkRepo = resolve(root, env.LANGBOT_PLUGIN_SDK_REPO || "../langbot-plugin-sdk");
const sdkRepo = resolve(root, env.LANGBOT_PLUGIN_SDK_REPO || "../../langbot-plugin-sdk");
const sdkSrc = resolve(sdkRepo, "src");
const fixturePath = resolve(root, "skills/langbot-testing/fixtures/plugins/qa-agent-runner");
const stdoutLog = join(evidenceDir, "probe-stdout.log");
@@ -137,7 +137,7 @@ async function main() {
const automationResultJson = join(evidenceDir, "automation-result.json");
const resultJson = join(evidenceDir, "result.json");
const timeoutMs = Number(env.LANGBOT_AGENT_RUNNER_PROBE_TIMEOUT_MS || "30000");
const command = { executable: "rtk", args: ["uv", "run", "python", "-c", script, fixturePath], cwd: sdkRepo };
const command = { executable: "rtk", args: ["uv", "run", "--no-sync", "python", "-c", script, fixturePath], cwd: sdkRepo };
const result = {
source: "automation",
probe: "agent-runner-fixture-contract",
skills/skills/langbot-testing/probes/agent-runner-ledger-concurrency.mjs
+2 -2
View File
@@ -5,9 +5,9 @@ import { runPytestProbe } from "./pytest-probe.mjs";
await runPytestProbe({
caseId: "agent-runner-ledger-concurrency",
repoEnvKey: "LANGBOT_REPO",
defaultRepo: "../LangBot",
defaultRepo: "..",
pythonPathEnvKeys: ["LANGBOT_PLUGIN_SDK_REPO"],
defaultPythonPaths: ["../langbot-plugin-sdk/src"],
defaultPythonPaths: ["../../langbot-plugin-sdk/src"],
description: "LangBot AgentRunner run ledger claim, lease, authorization, and runtime-admin pytest probe.",
testTargets: [
"tests/unit_tests/agent/test_run_ledger_store.py::test_create_queued_run_claim_renew_release",
skills/skills/langbot-testing/probes/agent-runner-ledger-contention.mjs
+8 -3
View File
@@ -152,15 +152,20 @@ async function main() {
const evidenceDir = resolve(env.LBS_EVIDENCE_DIR || join(root, "reports", "evidence", runId));
await mkdir(evidenceDir, { recursive: true });
const startedAt = new Date();
const langbotRepo = resolve(root, env.LANGBOT_REPO || "../LangBot");
const sdkSrc = resolve(root, env.LANGBOT_PLUGIN_SDK_REPO || "../langbot-plugin-sdk/src");
const langbotRepo = resolve(root, env.LANGBOT_REPO || "..");
const sdkRepo = resolve(root, env.LANGBOT_PLUGIN_SDK_REPO || "../../langbot-plugin-sdk");
const sdkSrc = resolve(sdkRepo, "src");
const dbPath = join(evidenceDir, "ledger-contention.sqlite3");
const stdoutLog = join(evidenceDir, "probe-stdout.log");
const stderrLog = join(evidenceDir, "probe-stderr.log");
const automationResultJson = join(evidenceDir, "automation-result.json");
const resultJson = join(evidenceDir, "result.json");
const timeoutMs = Number(env.LANGBOT_AGENT_RUNNER_PROBE_TIMEOUT_MS || "30000");
const command = { executable: "rtk", args: ["uv", "run", "python", "-c", script, dbPath], cwd: langbotRepo };
const command = {
executable: "rtk",
args: [resolve(langbotRepo, ".venv/bin/python"), "-c", script, dbPath],
cwd: langbotRepo,
};
const result = {
source: "automation",
probe: "agent-runner-ledger-contention",
skills/skills/langbot-testing/probes/agent-runner-ledger-invariants.mjs
+8 -3
View File
@@ -127,13 +127,18 @@ async function main() {
const evidenceDir = resolve(env.LBS_EVIDENCE_DIR || join(root, "reports", "evidence", runId));
await mkdir(evidenceDir, { recursive: true });
const startedAt = new Date();
const langbotRepo = resolveFromRoot(root, env.LANGBOT_REPO || "../LangBot");
const sdkSrc = resolveFromRoot(root, env.LANGBOT_PLUGIN_SDK_REPO || "../langbot-plugin-sdk/src");
const langbotRepo = resolveFromRoot(root, env.LANGBOT_REPO || "..");
const sdkRepo = resolveFromRoot(root, env.LANGBOT_PLUGIN_SDK_REPO || "../../langbot-plugin-sdk");
const sdkSrc = resolve(sdkRepo, "src");
const stdoutLog = join(evidenceDir, "probe-stdout.log");
const stderrLog = join(evidenceDir, "probe-stderr.log");
const automationResultJson = join(evidenceDir, "automation-result.json");
const resultJson = join(evidenceDir, "result.json");
const command = { executable: "rtk", args: ["uv", "run", "python", "-c", probeScript], cwd: langbotRepo };
const command = {
executable: "rtk",
args: [resolve(langbotRepo, ".venv/bin/python"), "-c", probeScript],
cwd: langbotRepo,
};
const timeoutMs = Number(env.LANGBOT_AGENT_RUNNER_PROBE_TIMEOUT_MS || "30000");
const result = {
source: "automation",
skills/skills/langbot-testing/probes/agent-runner-ledger-stress.mjs
+8 -3
View File
@@ -119,14 +119,19 @@ async function main() {
const evidenceDir = resolve(env.LBS_EVIDENCE_DIR || join(root, "reports", "evidence", runId));
await mkdir(evidenceDir, { recursive: true });
const startedAt = new Date();
const langbotRepo = resolve(root, env.LANGBOT_REPO || "../LangBot");
const sdkSrc = resolve(root, env.LANGBOT_PLUGIN_SDK_REPO || "../langbot-plugin-sdk/src");
const langbotRepo = resolve(root, env.LANGBOT_REPO || "..");
const sdkRepo = resolve(root, env.LANGBOT_PLUGIN_SDK_REPO || "../../langbot-plugin-sdk");
const sdkSrc = resolve(sdkRepo, "src");
const stdoutLog = join(evidenceDir, "probe-stdout.log");
const stderrLog = join(evidenceDir, "probe-stderr.log");
const automationResultJson = join(evidenceDir, "automation-result.json");
const resultJson = join(evidenceDir, "result.json");
const timeoutMs = Number(env.LANGBOT_AGENT_RUNNER_PROBE_TIMEOUT_MS || "30000");
const command = { executable: "rtk", args: ["uv", "run", "python", "-c", script], cwd: langbotRepo };
const command = {
executable: "rtk",
args: [resolve(langbotRepo, ".venv/bin/python"), "-c", script],
cwd: langbotRepo,
};
const result = {
source: "automation",
probe: "agent-runner-ledger-stress",
skills/skills/langbot-testing/probes/agent-runner-runtime-chaos.mjs
+1 -1
View File
@@ -5,7 +5,7 @@ import { runPytestProbe } from "./pytest-probe.mjs";
await runPytestProbe({
caseId: "agent-runner-runtime-chaos",
repoEnvKey: "LANGBOT_PLUGIN_SDK_REPO",
defaultRepo: "../langbot-plugin-sdk",
defaultRepo: "../../langbot-plugin-sdk",
description: "LangBot plugin SDK AgentRunner runtime failure, timeout, forwarding, and pull API pytest probe.",
testTargets: [
"tests/runtime/plugin/test_mgr_agent_runner.py",
skills/skills/langbot-testing/probes/pytest-probe.mjs
+1 -1
View File
@@ -129,7 +129,7 @@ export async function runPytestProbe({
const resultJson = join(evidenceDir, "result.json");
const command = {
executable: "rtk",
args: ["uv", "run", "pytest", "-q", ...testTargets],
args: ["uv", "run", "--no-sync", "pytest", "-q", ...testTargets],
cwd: repoPath,
};
const result = {
skills/test/lbs-cli.test.ts
+5 -1
View File
@@ -3589,7 +3589,7 @@ test("MCP stdio tool-call case setups pipeline and registered MCP server", () =>
JSON.parse(run.automation.env_defaults.LANGBOT_E2E_EXTENSIONS_PATCH_JSON),
{
enable_all_plugins: false,
bound_plugins: [{ author: "langbot", name: "local-agent" }],
bound_plugins: [{ author: "langbot-team", name: "LocalAgent" }],
enable_all_mcp_servers: false,
bound_mcp_servers: ["mcp-server-uuid"],
enable_all_skills: false,
@@ -3695,6 +3695,10 @@ test("AgentRunner QA Debug Chat case uses dedicated pipeline env", () => {
run.automation.env_defaults.LANGBOT_E2E_EXPECTED_RUNNER_ID,
"plugin:qa/agent-runner/default",
);
assert.equal(
run.automation.env_defaults.LANGBOT_E2E_DEBUG_CHAT_RESPONSE_P95_MS,
"120000",
);
assert.deepEqual(
run.setup_automation.map((item: { entry: string }) => item.entry),
[
src/langbot/pkg/agent/__init__.py
+4 -3
View File
@@ -1,4 +1,5 @@
"""Agent runner subsystem for LangBot."""
from __future__ import annotations
from .runner.descriptor import AgentRunnerDescriptor
@@ -15,7 +16,7 @@ from .runner.context_builder import AgentRunContextBuilder
from .runner.resource_builder import AgentResourceBuilder
from .runner.result_normalizer import AgentResultNormalizer
from .runner.orchestrator import AgentRunOrchestrator
from .runner.config_migration import ConfigMigration
from .runner.config_resolver import RunnerConfigResolver
__all__ = [
'AgentRunnerDescriptor',
@@ -33,5 +34,5 @@ __all__ = [
'AgentResourceBuilder',
'AgentResultNormalizer',
'AgentRunOrchestrator',
'ConfigMigration',
]
'RunnerConfigResolver',
]
src/langbot/pkg/agent/runner/__init__.py
+2 -2
View File
@@ -16,7 +16,7 @@ from .context_builder import AgentRunContextBuilder
from .resource_builder import AgentResourceBuilder
from .result_normalizer import AgentResultNormalizer
from .orchestrator import AgentRunOrchestrator
from .config_migration import ConfigMigration
from .config_resolver import RunnerConfigResolver
from .default_config import AgentRunnerDefaultConfigService
from .binding_resolver import AgentBindingResolver, AgentBindingResolutionError
from .session_registry import (
@@ -49,7 +49,7 @@ __all__ = [
'AgentResourceBuilder',
'AgentResultNormalizer',
'AgentRunOrchestrator',
'ConfigMigration',
'RunnerConfigResolver',
'AgentRunnerDefaultConfigService',
'AgentBindingResolver',
'AgentBindingResolutionError',
src/langbot/pkg/agent/runner/config_migration.py
-64
View File
@@ -1,64 +0,0 @@
"""Helpers for the current AgentRunner config shape."""
from __future__ import annotations
import typing
class ConfigMigration:
"""Configuration helpers for the current AgentRunner shape.
Responsibilities:
- Resolve runner ID from ai.runner.id
- Extract Agent/runner config from ai.runner_config
- Read current conversation expiry settings
"""
@staticmethod
def resolve_runner_id(pipeline_config: dict[str, typing.Any]) -> str | None:
"""Resolve runner ID from current configuration.
Args:
pipeline_config: Current configuration container
Returns:
Runner ID string, or None if not configured
"""
ai_config = pipeline_config.get('ai', {}) if isinstance(pipeline_config, dict) else {}
runner = ai_config.get('runner', {}) if isinstance(ai_config, dict) else {}
runner_id = runner.get('id') if isinstance(runner, dict) else None
return runner_id if isinstance(runner_id, str) and runner_id else None
@staticmethod
def resolve_runner_config(
pipeline_config: dict[str, typing.Any],
runner_id: str,
) -> dict[str, typing.Any]:
"""Resolve Agent/runner configuration from the current container.
Args:
pipeline_config: Current configuration container
runner_id: Resolved runner ID
Returns:
Runner configuration dict (empty if not found)
"""
ai_config = pipeline_config.get('ai', {}) if isinstance(pipeline_config, dict) else {}
runner_configs = ai_config.get('runner_config', {}) if isinstance(ai_config, dict) else {}
runner_config = runner_configs.get(runner_id, {}) if isinstance(runner_configs, dict) else {}
return runner_config if isinstance(runner_config, dict) else {}
@staticmethod
def get_expire_time(pipeline_config: dict[str, typing.Any]) -> int:
"""Get conversation expire time from configuration.
Args:
pipeline_config: Current configuration container
Returns:
Expire time in seconds (0 means no expiry)
"""
ai_config = pipeline_config.get('ai', {}) if isinstance(pipeline_config, dict) else {}
runner = ai_config.get('runner', {}) if isinstance(ai_config, dict) else {}
expire_time = runner.get('expire-time', 0) if isinstance(runner, dict) else 0
return expire_time if isinstance(expire_time, int) else 0
src/langbot/pkg/agent/runner/config_resolver.py
+199
View File
@@ -0,0 +1,199 @@
"""Resolve the current AgentRunner configuration shape."""
from __future__ import annotations
import typing
HOST_SECURITY_BOOLEAN_FIELDS = (
'enable-all-tools',
'mcp-resource-agent-read-enabled',
)
class RunnerConfigResolver:
"""Configuration helpers for the current AgentRunner shape.
Responsibilities:
- Resolve runner ID from ai.runner.id
- Extract Agent/runner config from ai.runner_config
- Read current conversation expiry settings
- Validate the persisted 4.x Agent binding container
"""
@staticmethod
def validate_agent_config(config: typing.Any) -> dict[str, typing.Any]:
"""Validate and return the persisted 4.x Agent config container."""
if not isinstance(config, dict):
raise ValueError('Agent config must be an object')
runner = config.get('runner')
if not isinstance(runner, dict):
raise ValueError("Agent config field 'runner' must be an object")
runner_id = runner.get('id')
if not isinstance(runner_id, str):
raise ValueError("Agent config field 'runner.id' must be a string")
runner_configs = config.get('runner_config')
if not isinstance(runner_configs, dict):
raise ValueError("Agent config field 'runner_config' must be an object")
for configured_runner_id, runner_config in runner_configs.items():
if not isinstance(configured_runner_id, str) or not configured_runner_id:
raise ValueError("Agent config field 'runner_config' must use non-empty string runner IDs")
if not isinstance(runner_config, dict):
raise ValueError(f'Agent runner_config[{configured_runner_id!r}] must be an object')
if runner_id and runner_id not in runner_configs:
raise ValueError(f'Agent runner_config is missing selected runner {runner_id!r}')
if runner_id:
RunnerConfigResolver.validate_runner_security_fields(
runner_configs[runner_id],
context=f'Agent runner_config[{runner_id!r}]',
)
return config
@staticmethod
def validate_runner_security_fields(
runner_config: dict[str, typing.Any],
*,
context: str = 'Runner config',
) -> dict[str, typing.Any]:
"""Reject malformed Host-owned security toggles instead of enabling them."""
for field_name in HOST_SECURITY_BOOLEAN_FIELDS:
if field_name in runner_config and not isinstance(runner_config[field_name], bool):
raise ValueError(f'{context} field {field_name!r} must be a boolean')
RunnerConfigResolver.validate_mcp_resource_attachments(
runner_config.get('mcp-resources'),
context=context,
field_name='mcp-resources',
)
return runner_config
@staticmethod
def validate_mcp_resource_attachments(
resources: typing.Any,
*,
context: str,
field_name: str,
) -> typing.Any:
"""Validate explicit attachment enable flags shared by Agent and Pipeline inputs."""
if not isinstance(resources, list):
return resources
for index, resource in enumerate(resources):
if not isinstance(resource, dict) or 'enabled' not in resource:
continue
if not isinstance(resource['enabled'], bool):
raise ValueError(f"{context} field '{field_name}[{index}].enabled' must be a boolean")
return resources
@classmethod
def validate_pipeline_config(cls, config: typing.Any) -> dict[str, typing.Any]:
"""Validate the selected Runner container in a current 4.x Pipeline config."""
if not isinstance(config, dict):
raise ValueError('Pipeline config must be an object')
ai_config = config.get('ai')
if ai_config is None:
return config
if not isinstance(ai_config, dict):
raise ValueError("Pipeline config field 'ai' must be an object")
runner = ai_config.get('runner')
if runner is None:
return config
if not isinstance(runner, dict):
raise ValueError("Pipeline config field 'ai.runner' must be an object")
runner_id = runner.get('id')
if not isinstance(runner_id, str):
raise ValueError("Pipeline config field 'ai.runner.id' must be a string")
if not runner_id:
return config
runner_configs = ai_config.get('runner_config')
if not isinstance(runner_configs, dict):
raise ValueError("Pipeline config field 'ai.runner_config' must be an object")
if runner_id not in runner_configs:
raise ValueError(f'Pipeline runner_config is missing selected runner {runner_id!r}')
selected_config = runner_configs[runner_id]
if not isinstance(selected_config, dict):
raise ValueError(f'Pipeline runner_config[{runner_id!r}] must be an object')
cls.validate_runner_security_fields(
selected_config,
context=f'Pipeline runner_config[{runner_id!r}]',
)
return config
@staticmethod
def resolve_agent_runner_id(config: dict[str, typing.Any]) -> str | None:
"""Resolve a runner ID from a validated persisted Agent config."""
runner = config.get('runner', {})
runner_id = runner.get('id') if isinstance(runner, dict) else None
return runner_id if isinstance(runner_id, str) and runner_id else None
@classmethod
def resolve_agent_runner_config(
cls,
config: typing.Any,
) -> tuple[dict[str, typing.Any], str | None, dict[str, typing.Any]]:
"""Validate an Agent config and return its selected runner configuration."""
validated = cls.validate_agent_config(config)
runner_id = cls.resolve_agent_runner_id(validated)
runner_configs = typing.cast(dict[str, typing.Any], validated['runner_config'])
runner_config = runner_configs.get(runner_id, {}) if runner_id else {}
return validated, runner_id, typing.cast(dict[str, typing.Any], runner_config)
@staticmethod
def resolve_runner_id(pipeline_config: dict[str, typing.Any]) -> str | None:
"""Resolve runner ID from current configuration.
Args:
pipeline_config: Current configuration container
Returns:
Runner ID string, or None if not configured
"""
ai_config = pipeline_config.get('ai', {}) if isinstance(pipeline_config, dict) else {}
runner = ai_config.get('runner', {}) if isinstance(ai_config, dict) else {}
runner_id = runner.get('id') if isinstance(runner, dict) else None
return runner_id if isinstance(runner_id, str) and runner_id else None
@staticmethod
def resolve_runner_config(
pipeline_config: dict[str, typing.Any],
runner_id: str,
) -> dict[str, typing.Any]:
"""Resolve Agent/runner configuration from the current container.
Args:
pipeline_config: Current configuration container
runner_id: Resolved runner ID
Returns:
Runner configuration dict (empty if not found)
"""
ai_config = pipeline_config.get('ai', {}) if isinstance(pipeline_config, dict) else {}
runner_configs = ai_config.get('runner_config', {}) if isinstance(ai_config, dict) else {}
runner_config = runner_configs.get(runner_id, {}) if isinstance(runner_configs, dict) else {}
return runner_config if isinstance(runner_config, dict) else {}
@staticmethod
def get_expire_time(pipeline_config: dict[str, typing.Any]) -> int:
"""Get conversation expire time from configuration.
Args:
pipeline_config: Current configuration container
Returns:
Expire time in seconds (0 means no expiry)
"""
ai_config = pipeline_config.get('ai', {}) if isinstance(pipeline_config, dict) else {}
runner = ai_config.get('runner', {}) if isinstance(ai_config, dict) else {}
expire_time = runner.get('expire-time', 0) if isinstance(runner, dict) else 0
return expire_time if isinstance(expire_time, int) else 0
src/langbot/pkg/agent/runner/context_builder.py
+3
View File
@@ -75,6 +75,9 @@ class ToolResource(typing.TypedDict):
tool_type: str | None
description: str | None
operations: list[str]
parameters: dict[str, typing.Any] | None
source: typing.NotRequired[str]
source_id: typing.NotRequired[str | None]
class KnowledgeBaseResource(typing.TypedDict):
src/langbot/pkg/agent/runner/default_config.py
+2 -2
View File
@@ -7,7 +7,7 @@ import sqlalchemy
from ...core import app
from ...entity.persistence import pipeline as persistence_pipeline
from . import config_schema
from .config_migration import ConfigMigration
from .config_resolver import RunnerConfigResolver
class AgentRunnerDefaultConfigService:
@@ -53,7 +53,7 @@ class AgentRunnerDefaultConfigService:
if not isinstance(pipeline_config, dict):
return False
runner_id = ConfigMigration.resolve_runner_id(pipeline_config)
runner_id = RunnerConfigResolver.resolve_runner_id(pipeline_config)
if not runner_id:
return False
src/langbot/pkg/agent/runner/execution_context.py
+319
View File
@@ -0,0 +1,319 @@
"""Host-only Query compatibility views for AgentRunner tool execution."""
from __future__ import annotations
import copy
import hashlib
import json
import typing
from langbot_plugin.api.entities.builtin.pipeline import query as pipeline_query
from langbot_plugin.api.entities.builtin.platform import events as platform_events
from langbot_plugin.api.entities.builtin.platform import message as platform_message
from langbot_plugin.api.entities.builtin.provider import message as provider_message
from langbot_plugin.api.entities.builtin.provider import session as provider_session
from ...utils import constants
from .host_models import AgentEventEnvelope
HOST_BOX_SCOPE_VARIABLE = '_host_box_scope'
AUTHORIZED_SKILLS_VARIABLE = '_pipeline_bound_skills'
MCP_RESOURCE_ATTACHMENTS_VARIABLE = '_pipeline_mcp_resource_attachments'
MCP_RESOURCE_AGENT_READ_ENABLED_VARIABLE = '_pipeline_mcp_resource_agent_read_enabled'
def project_mcp_resource_config(
query: pipeline_query.Query,
runner_config: dict[str, typing.Any],
) -> dict[str, typing.Any]:
"""Attach standard MCP resource settings to a Host execution Query."""
variables = getattr(query, 'variables', None)
if not isinstance(variables, dict):
variables = {}
query.variables = variables
attachments = runner_config.get('mcp-resources', [])
variables.setdefault(
MCP_RESOURCE_ATTACHMENTS_VARIABLE,
list(attachments) if isinstance(attachments, list) else [],
)
variables.setdefault(
MCP_RESOURCE_AGENT_READ_ENABLED_VARIABLE,
runner_config.get('mcp-resource-agent-read-enabled', True) is True,
)
return variables
async def build_mcp_resource_context_addition(
ap: typing.Any,
query: pipeline_query.Query,
) -> str:
"""Build model-facing MCP context without mutating the canonical input."""
tool_mgr = getattr(ap, 'tool_mgr', None)
if tool_mgr is None:
return ''
mcp_loader = getattr(tool_mgr, '__dict__', {}).get('mcp_tool_loader')
if mcp_loader is None:
return ''
resource_context = await mcp_loader.build_resource_context_for_query(query)
if not resource_context:
return ''
addition = (
'\n\nMCP resource context selected by LangBot host:\n'
f'{resource_context}\n\n'
'Use this context as read-only reference material. If it conflicts with the user message, '
'ask for clarification before taking external actions.'
)
return addition
def append_mcp_resource_context_to_event(event: AgentEventEnvelope, addition: str) -> None:
"""Append pinned context only to the run-scoped execution input."""
if not addition:
return
has_text = event.input.text is not None
has_structured_content = bool(event.input.contents)
if has_text:
event.input.text += addition
if has_structured_content or not has_text:
if not _append_text_to_content(event.input.contents, addition):
event.input.contents.append(provider_message.ContentElement.from_text(addition.strip()))
def _append_text_to_content(content: typing.Any, addition: str) -> bool:
if isinstance(content, str):
return False
if not isinstance(content, list):
return False
for content_element in content:
if getattr(content_element, 'type', None) == 'text':
content_element.text = (content_element.text or '') + addition
return True
content.append(provider_message.ContentElement.from_text(addition.strip()))
return True
def prepare_execution_query(
query: pipeline_query.Query,
event: AgentEventEnvelope,
authorized_skill_names: list[str],
) -> pipeline_query.Query:
"""Attach Host-owned execution metadata without changing Query identity."""
variables = prepare_box_scope(query, event, preserve_existing=True)
variables[AUTHORIZED_SKILLS_VARIABLE] = list(dict.fromkeys(authorized_skill_names))
return query
def prepare_box_scope(
query: pipeline_query.Query,
event: AgentEventEnvelope,
*,
preserve_existing: bool = False,
) -> dict[str, typing.Any]:
"""Attach the Host Box scope before any Query-side file staging."""
variables = getattr(query, 'variables', None)
if not isinstance(variables, dict):
variables = {}
query.variables = variables
existing_scope = variables.get(HOST_BOX_SCOPE_VARIABLE)
if preserve_existing and isinstance(existing_scope, str) and existing_scope.strip():
return variables
variables[HOST_BOX_SCOPE_VARIABLE] = build_host_box_scope(event, query=query)
return variables
def build_execution_query(
event: AgentEventEnvelope,
authorized_skill_names: list[str],
) -> pipeline_query.Query:
"""Build the minimum Query view required by Host-owned tool loaders."""
launcher_type, launcher_id, sender_id = _resolve_session_identity(event)
message_chain = _build_message_chain(event)
message_event = platform_events.MessageEvent(
type=event.source_event_type or event.event_type,
message_chain=message_chain,
time=float(event.event_time) if event.event_time is not None else None,
)
session = provider_session.Session(
launcher_type=launcher_type,
launcher_id=launcher_id,
sender_id=sender_id,
)
contents = copy.deepcopy(event.input.contents)
user_content: str | list[provider_message.ContentElement]
if contents:
user_content = contents
else:
user_content = event.input.text or ''
query = pipeline_query.Query(
query_id=_synthetic_query_id(event.event_id),
launcher_type=launcher_type,
launcher_id=launcher_id,
sender_id=sender_id,
message_event=message_event,
message_chain=message_chain,
bot_uuid=event.bot_id,
pipeline_uuid=None,
pipeline_config=None,
session=session,
messages=[],
user_message=provider_message.Message(role='user', content=user_content),
variables={},
resp_messages=[],
)
query.variables[HOST_BOX_SCOPE_VARIABLE] = build_host_box_scope(event)
query.variables[AUTHORIZED_SKILLS_VARIABLE] = list(dict.fromkeys(authorized_skill_names))
return query
def build_host_box_scope(
event: AgentEventEnvelope,
*,
query: pipeline_query.Query | None = None,
) -> str | None:
"""Return a stable Host scope for a Query session or event."""
target_type, target_id = _resolve_box_target(event, query)
if target_type is None or target_id is None:
return None
capabilities = event.delivery.platform_capabilities or {}
adapter_identity = None
if query is not None:
adapter = getattr(query, 'adapter', None)
if adapter is not None:
adapter_identity = adapter.__class__.__name__
adapter_identity = _first_present(
adapter_identity,
capabilities.get('adapter'),
capabilities.get('source'),
event.source if query is None else None,
)
return json.dumps(
{
'instance_id': _nonempty(constants.instance_id),
'workspace_id': _nonempty(event.workspace_id),
'bot_id': _nonempty(event.bot_id),
'platform_adapter': adapter_identity,
'target_type': target_type,
'target_id': target_id,
'thread_id': _nonempty(event.thread_id),
},
ensure_ascii=False,
sort_keys=True,
separators=(',', ':'),
)
def _resolve_session_identity(
event: AgentEventEnvelope,
) -> tuple[provider_session.LauncherTypes, str, str]:
subject_data = event.subject.data if event.subject is not None else {}
reply_target = event.delivery.reply_target or {}
launcher_type_value = _first_present(
reply_target.get('target_type'),
subject_data.get('launcher_type'),
event.data.get('launcher_type'),
reply_target.get('launcher_type'),
)
if launcher_type_value == provider_session.LauncherTypes.GROUP.value:
launcher_type_value = provider_session.LauncherTypes.GROUP.value
else:
launcher_type_value = provider_session.LauncherTypes.PERSON.value
launcher_id = _first_nonempty(
reply_target.get('target_id'),
subject_data.get('launcher_id'),
event.data.get('launcher_id'),
reply_target.get('launcher_id'),
event.conversation_id,
event.subject.subject_id if event.subject is not None else None,
event.actor.actor_id if event.actor is not None else None,
event.event_id,
)
sender_id = _first_nonempty(
subject_data.get('sender_id'),
event.data.get('sender_id'),
event.actor.actor_id if event.actor is not None else None,
launcher_id,
)
return provider_session.LauncherTypes(launcher_type_value), launcher_id, sender_id
def _resolve_box_target(
event: AgentEventEnvelope,
query: pipeline_query.Query | None,
) -> tuple[str | None, str | None]:
if query is not None:
launcher_type = getattr(query, 'launcher_type', None)
if hasattr(launcher_type, 'value'):
launcher_type = launcher_type.value
launcher_id = getattr(query, 'launcher_id', None)
normalized_type = _nonempty(launcher_type)
normalized_id = _nonempty(launcher_id)
if normalized_type is not None and normalized_id is not None:
return normalized_type, normalized_id
reply_target = event.delivery.reply_target or {}
target_type = _first_present(
reply_target.get('target_type'),
reply_target.get('launcher_type'),
)
target_id = _first_present(
reply_target.get('target_id'),
reply_target.get('launcher_id'),
)
if target_type is not None and target_id is not None:
return target_type, target_id
conversation_id = _nonempty(event.conversation_id)
if conversation_id is not None:
return 'conversation', conversation_id
event_id = _nonempty(event.event_id)
if event_id is not None:
return 'event', event_id
return None, None
def _build_message_chain(event: AgentEventEnvelope) -> platform_message.MessageChain:
text = event.input.to_text()
if not text:
return platform_message.MessageChain([])
return platform_message.MessageChain([platform_message.Plain(text=text)])
def _synthetic_query_id(event_id: str) -> int:
digest = hashlib.sha256(event_id.encode('utf-8')).hexdigest()
return int(digest[:15], 16) or 1
def _first_nonempty(*values: typing.Any) -> str:
normalized = _first_present(*values)
if normalized is not None:
return normalized
raise ValueError('Agent event does not contain a usable execution identity')
def _first_present(*values: typing.Any) -> str | None:
for value in values:
normalized = _nonempty(value)
if normalized is not None:
return normalized
return None
def _nonempty(value: typing.Any) -> str | None:
if value is None:
return None
normalized = str(value).strip()
return normalized or None
src/langbot/pkg/agent/runner/host_models.py
+12 -5
View File
@@ -2,6 +2,7 @@
These are Host-internal models, not exposed to SDK.
"""
from __future__ import annotations
import typing
@@ -73,7 +74,7 @@ class AgentEventEnvelope(pydantic.BaseModel):
class BindingScope(pydantic.BaseModel):
"""Scope for agent binding."""
scope_type: typing.Literal["agent", "bot", "workspace", "global"] = "agent"
scope_type: typing.Literal['agent', 'bot', 'workspace', 'global'] = 'agent'
"""Scope type."""
scope_id: str | None = None
@@ -92,6 +93,12 @@ class ResourcePolicy(pydantic.BaseModel):
allowed_tool_names: list[str] | None = None
"""Additional tool name grants. None means no additional tool grants."""
allowed_tool_sources: dict[str, dict[str, str | None]] | None = None
"""Host-resolved implementation identity for each allowed tool name."""
allow_all_tools: bool = False
"""Whether all tools visible to the current Host scope are granted."""
allowed_kb_uuids: list[str] | None = None
"""Additional knowledge base UUID grants. None means no additional KB grants."""
@@ -114,8 +121,8 @@ class StatePolicy(pydantic.BaseModel):
enable_state: bool = True
"""Whether host-owned state is enabled."""
state_scopes: list[typing.Literal["conversation", "actor", "subject", "runner"]] = (
pydantic.Field(default_factory=lambda: ["conversation", "actor"])
state_scopes: list[typing.Literal['conversation', 'actor', 'subject', 'runner']] = pydantic.Field(
default_factory=lambda: ['conversation', 'actor']
)
"""Enabled state scopes."""
@@ -162,7 +169,7 @@ class AgentConfig(pydantic.BaseModel):
delivery_policy: DeliveryPolicy = pydantic.Field(default_factory=DeliveryPolicy)
"""Delivery policy for this Agent."""
event_types: list[str] = pydantic.Field(default_factory=lambda: ["message.received"])
event_types: list[str] = pydantic.Field(default_factory=lambda: ['message.received'])
"""Event types this Agent handles."""
enabled: bool = True
@@ -185,7 +192,7 @@ class AgentBinding(pydantic.BaseModel):
scope: BindingScope = pydantic.Field(default_factory=BindingScope)
"""Binding scope."""
event_types: list[str] = pydantic.Field(default_factory=lambda: ["message.received"])
event_types: list[str] = pydantic.Field(default_factory=lambda: ['message.received'])
"""Event types this binding handles."""
runner_id: str
src/langbot/pkg/agent/runner/orchestrator.py
+39 -6
View File
@@ -12,6 +12,14 @@ from ...core import app
from .binding_resolver import AgentBindingResolver
from .context_builder import AgentRunContextBuilder, AgentRunContextPayload
from .descriptor import AgentRunnerDescriptor
from .execution_context import (
append_mcp_resource_context_to_event,
build_mcp_resource_context_addition,
build_execution_query,
prepare_box_scope,
prepare_execution_query,
project_mcp_resource_config,
)
from .host_models import AgentBinding, AgentEventEnvelope
from .invoker import AgentRunnerInvoker
from .query_bridge import QueryRunBridge
@@ -73,6 +81,17 @@ class AgentRunOrchestrator:
runner_id = binding.runner_id
descriptor = await self.registry.get(runner_id, bound_plugins)
execution_query = adapter_context.get('_query') if adapter_context else None
if execution_query is None:
execution_query = build_execution_query(event, [])
project_mcp_resource_config(execution_query, binding.runner_config)
execution_event = event
resource_addition = await build_mcp_resource_context_addition(self.ap, execution_query)
if resource_addition:
execution_event = event.model_copy(deep=True)
append_mcp_resource_context_to_event(execution_event, resource_addition)
resources = await self.resource_builder.build_resources_from_binding(
event=event,
binding=binding,
@@ -80,7 +99,7 @@ class AgentRunOrchestrator:
)
context = await self.context_builder.build_context_from_event(
event=event,
event=execution_event,
binding=binding,
descriptor=descriptor,
resources=resources,
@@ -88,19 +107,23 @@ class AgentRunOrchestrator:
session_query_id = None
if adapter_context:
query = adapter_context.get('_query')
if query is not None:
if execution_query is not None:
skill_loader.restore_activated_skills_from_state(
self.ap,
query,
execution_query,
context.get('state', {}),
)
session_query_id = adapter_context.get('query_id')
if query is not None or session_query_id is not None:
if execution_query is not None or session_query_id is not None:
context['context']['available_apis']['prompt_get'] = True
if 'params' in adapter_context:
context['adapter']['extra']['params'] = adapter_context['params']
authorized_skill_names = [
str(skill['skill_name']) for skill in resources.get('skills', []) if skill.get('skill_name')
]
prepare_execution_query(execution_query, event, authorized_skill_names)
state_context = build_state_context(event, binding, descriptor)
run_id = context['run_id']
available_apis = context.get('context', {}).get('available_apis')
@@ -152,6 +175,7 @@ class AgentRunOrchestrator:
'state_scopes': list(binding.state_policy.state_scopes),
},
state_context=state_context,
execution_query=execution_query,
)
event_log_id = await self.journal.write_event_log(
@@ -309,6 +333,9 @@ class AgentRunOrchestrator:
adapter_context = dict(plan.adapter_context)
adapter_context['_query'] = query
# Inbound files and subsequent runner tools must share one Host scope.
prepare_box_scope(query, plan.event)
# Materialize inbound attachments into sandbox before running
await self._materialize_inbound_attachments(query, plan.event)
@@ -392,7 +419,13 @@ class AgentRunOrchestrator:
if target_run_id is None:
return False
steering_item = self._build_steering_item(event, target_run_id, descriptor.id)
execution_event = event
resource_addition = await build_mcp_resource_context_addition(self.ap, query)
if resource_addition:
execution_event = event.model_copy(deep=True)
append_mcp_resource_context_to_event(execution_event, resource_addition)
steering_item = self._build_steering_item(execution_event, target_run_id, descriptor.id)
if not await self._session_registry.enqueue_steering(target_run_id, steering_item):
return False
src/langbot/pkg/agent/runner/query_bridge.py
+3 -3
View File
@@ -8,7 +8,7 @@ import typing
from langbot_plugin.api.entities.builtin.pipeline import query as pipeline_query
from .binding_resolver import AgentBindingResolver
from .config_migration import ConfigMigration
from .config_resolver import RunnerConfigResolver
from .errors import RunnerNotFoundError
from .host_models import AgentBinding, AgentEventEnvelope
from .query_entry_adapter import QueryEntryAdapter
@@ -34,7 +34,7 @@ class QueryRunBridge:
def build_plan(self, query: pipeline_query.Query) -> QueryRunPlan:
"""Build an event-first run plan from a Pipeline Query."""
runner_id = ConfigMigration.resolve_runner_id(query.pipeline_config)
runner_id = RunnerConfigResolver.resolve_runner_id(query.pipeline_config)
if not runner_id:
raise RunnerNotFoundError('no runner configured')
@@ -53,4 +53,4 @@ class QueryRunBridge:
def resolve_runner_id_for_telemetry(self, query: pipeline_query.Query) -> str | None:
"""Resolve runner ID for telemetry/logging without full execution."""
return ConfigMigration.resolve_runner_id(query.pipeline_config)
return RunnerConfigResolver.resolve_runner_id(query.pipeline_config)
src/langbot/pkg/agent/runner/query_entry_adapter.py
+99 -84
View File
@@ -3,6 +3,7 @@
This adapter bridges the current Query entry point with the event-first
Protocol v1 architecture without exposing Query internals to runners.
"""
from __future__ import annotations
import hashlib
@@ -23,12 +24,13 @@ from langbot_plugin.api.entities.builtin.agent_runner.delivery import DeliveryCo
from .host_models import (
AgentConfig,
AgentEventEnvelope,
ResourcePolicy,
StatePolicy,
DeliveryPolicy,
)
from .config_migration import ConfigMigration
from .config_resolver import RunnerConfigResolver
from .resource_policy import ResourcePolicyProjector
from . import events as runner_events
from ...provider.tools.toolmgr import TOOL_SOURCE_REFS_QUERY_KEY
class QueryEntryAdapter:
@@ -83,7 +85,7 @@ class QueryEntryAdapter:
event_id=event.event_id or str(query.query_id),
event_type=event.event_type or runner_events.MESSAGE_RECEIVED,
event_time=event.event_time,
source="host_adapter",
source='host_adapter',
source_event_type=event.source_event_type,
bot_id=query.bot_uuid,
workspace_id=None, # Not available in Query
@@ -105,21 +107,22 @@ class QueryEntryAdapter:
) -> AgentConfig:
"""Project the current Pipeline config container into target Agent config."""
pipeline_config = query.pipeline_config or {}
runner_config = ConfigMigration.resolve_runner_config(pipeline_config, runner_id)
runner_config = RunnerConfigResolver.resolve_runner_config(pipeline_config, runner_id)
agent_id = getattr(query, 'pipeline_uuid', None)
# Build resource policy from current config
resource_policy = ResourcePolicy(
allowed_model_uuids=cls._extract_allowed_models(query),
allowed_tool_names=cls._extract_allowed_tools(query),
allowed_kb_uuids=cls._extract_allowed_kbs(query),
allowed_skill_names=cls._extract_allowed_skills(query),
resource_policy = ResourcePolicyProjector.from_runner_config(
runner_config,
resolved_model_uuids=cls._extract_allowed_models(query),
resolved_tool_names=cls._extract_allowed_tools(query),
resolved_tool_sources=cls._extract_allowed_tool_sources(query),
resolved_kb_uuids=cls._extract_allowed_kbs(query),
resolved_skill_names=cls._extract_allowed_skills(query),
)
# Build state policy
state_policy = StatePolicy(
enable_state=True,
state_scopes=["conversation", "actor", "subject", "runner"],
state_scopes=['conversation', 'actor', 'subject', 'runner'],
)
# Build delivery policy
@@ -181,10 +184,7 @@ class QueryEntryAdapter:
if isinstance(value, (list, tuple)):
return all(cls.is_json_serializable(item) for item in value)
if isinstance(value, dict):
return all(
isinstance(k, str) and cls.is_json_serializable(v)
for k, v in value.items()
)
return all(isinstance(k, str) and cls.is_json_serializable(v) for k, v in value.items())
return False
# Private helper methods
@@ -228,7 +228,7 @@ class QueryEntryAdapter:
event_id=cls._build_scoped_event_id(query, source_event_id, event_time),
event_type=runner_events.MESSAGE_RECEIVED,
event_time=event_time,
source="host_adapter",
source='host_adapter',
source_event_type=source_event_type,
data=event_data,
)
@@ -345,7 +345,7 @@ class QueryEntryAdapter:
actor_name = sender.get_name() if sender and hasattr(sender, 'get_name') else None
return ActorContext(
actor_type="user",
actor_type='user',
actor_id=str(actor_id) if actor_id is not None else None,
actor_name=actor_name,
metadata={},
@@ -371,13 +371,13 @@ class QueryEntryAdapter:
launcher_type_value = getattr(launcher_type, 'value', launcher_type)
return SubjectContext(
subject_type="message",
subject_type='message',
subject_id=str(message_id or query_id or ''),
data={
"launcher_type": launcher_type_value,
"launcher_id": getattr(query, 'launcher_id', None),
"sender_id": str(getattr(query, 'sender_id', '')) if getattr(query, 'sender_id', None) else None,
"bot_uuid": getattr(query, 'bot_uuid', None),
'launcher_type': launcher_type_value,
'launcher_id': getattr(query, 'launcher_id', None),
'sender_id': str(getattr(query, 'sender_id', '')) if getattr(query, 'sender_id', None) else None,
'bot_uuid': getattr(query, 'bot_uuid', None),
},
)
@@ -467,31 +467,39 @@ class QueryEntryAdapter:
if elem_type == 'image_url':
image_url = elem.get('image_url') or {}
add_attachment({
'type': 'image',
'source': 'url',
'url': image_url.get('url') if isinstance(image_url, dict) else str(image_url),
})
add_attachment(
{
'type': 'image',
'source': 'url',
'url': image_url.get('url') if isinstance(image_url, dict) else str(image_url),
}
)
elif elem_type == 'image_base64':
add_attachment({
'type': 'image',
'source': 'base64',
'content': elem.get('image_base64'),
})
add_attachment(
{
'type': 'image',
'source': 'base64',
'content': elem.get('image_base64'),
}
)
elif elem_type == 'file_url':
add_attachment({
'type': 'file',
'source': 'url',
'url': elem.get('file_url'),
'name': elem.get('file_name'),
})
add_attachment(
{
'type': 'file',
'source': 'url',
'url': elem.get('file_url'),
'name': elem.get('file_name'),
}
)
elif elem_type == 'file_base64':
add_attachment({
'type': 'file',
'source': 'base64',
'content': elem.get('file_base64'),
'name': elem.get('file_name'),
})
add_attachment(
{
'type': 'file',
'source': 'base64',
'content': elem.get('file_base64'),
'name': elem.get('file_name'),
}
)
message_chain = getattr(query, 'message_chain', None)
if message_chain:
@@ -505,30 +513,36 @@ class QueryEntryAdapter:
image_id = component.image_id or None
image_url = component.url or None
image_base64 = component.base64 or None
add_attachment({
'type': 'image',
'source': 'message_chain',
'id': image_id,
'url': image_url,
'content': image_base64,
})
add_attachment(
{
'type': 'image',
'source': 'message_chain',
'id': image_id,
'url': image_url,
'content': image_base64,
}
)
elif isinstance(component, platform_message.File):
add_attachment({
'type': 'file',
'source': 'message_chain',
'id': component.id or None,
'name': component.name or None,
'url': component.url or None,
'content': component.base64 or None,
})
add_attachment(
{
'type': 'file',
'source': 'message_chain',
'id': component.id or None,
'name': component.name or None,
'url': component.url or None,
'content': component.base64 or None,
}
)
elif isinstance(component, platform_message.Voice):
add_attachment({
'type': 'voice',
'source': 'message_chain',
'id': component.voice_id or None,
'url': component.url or None,
'content': component.base64 or None,
})
add_attachment(
{
'type': 'voice',
'source': 'message_chain',
'id': component.voice_id or None,
'url': component.url or None,
'content': component.base64 or None,
}
)
return attachments
@@ -557,9 +571,9 @@ class QueryEntryAdapter:
"""Build DeliveryContext from Query."""
message_chain = getattr(query, 'message_chain', None)
return DeliveryContext(
surface="platform",
surface='platform',
reply_target={
"message_id": getattr(message_chain, 'message_id', None),
'message_id': getattr(message_chain, 'message_id', None),
},
supports_streaming=True,
supports_edit=False,
@@ -601,22 +615,24 @@ class QueryEntryAdapter:
) -> list[str] | None:
"""Extract allowed tool names from query."""
use_funcs = getattr(query, 'use_funcs', None)
if not use_funcs:
if use_funcs is None:
return None
try:
tool_names = []
for func in use_funcs:
if isinstance(func, dict):
name = func.get('name')
elif hasattr(func, 'name'):
name = func.name
else:
continue
if name:
tool_names.append(name)
return tool_names if tool_names else None
return ResourcePolicyProjector.extract_tool_names(use_funcs)
except (TypeError, AttributeError):
return []
@classmethod
def _extract_allowed_tool_sources(
cls,
query: pipeline_query.Query,
) -> dict[str, typing.Any] | None:
"""Extract the Host-frozen implementation for each allowed tool."""
variables = getattr(query, 'variables', None)
if not isinstance(variables, dict):
return None
refs = variables.get(TOOL_SOURCE_REFS_QUERY_KEY)
return refs if isinstance(refs, dict) else None
@classmethod
def _extract_allowed_kbs(
@@ -627,10 +643,9 @@ class QueryEntryAdapter:
variables = getattr(query, 'variables', None)
if not variables:
return None
kb_uuids = variables.get('_knowledge_base_uuids')
if kb_uuids:
return kb_uuids
return None
if '_knowledge_base_uuids' not in variables:
return None
return ResourcePolicyProjector.normalize_names(variables.get('_knowledge_base_uuids'))
@classmethod
def _extract_allowed_skills(
src/langbot/pkg/agent/runner/resource_builder.py
+108 -41
View File
@@ -1,4 +1,5 @@
"""Agent resource builder for constructing authorized resources."""
from __future__ import annotations
import typing
@@ -15,6 +16,9 @@ from .context_builder import (
)
from . import config_schema
from .host_models import AgentEventEnvelope, AgentBinding
from .resource_policy import ResourcePolicyProjector
from ...provider.tools.loaders.mcp import MCP_TOOL_LIST_RESOURCES, MCP_TOOL_READ_RESOURCE
from ...provider.tools.toolmgr import ToolSourceRef
class AgentResourceBuilder:
@@ -66,18 +70,12 @@ class AgentResourceBuilder:
manifest_perms = descriptor.permissions
# Build each resource category
models = await self._build_models_from_binding(
manifest_perms, resource_policy, descriptor, runner_config
)
tools = await self._build_tools_from_binding(
manifest_perms, resource_policy, descriptor
)
models = await self._build_models_from_binding(manifest_perms, resource_policy, descriptor, runner_config)
tools = await self._build_tools_from_binding(manifest_perms, resource_policy, descriptor, runner_config)
knowledge_bases = await self._build_knowledge_bases_from_binding(
manifest_perms, resource_policy, descriptor, runner_config
)
skills = self._build_skills_from_binding(
resource_policy, descriptor
)
skills = self._build_skills_from_binding(resource_policy, descriptor)
storage = self._build_storage_from_binding(manifest_perms, binding)
return {
@@ -133,6 +131,7 @@ class AgentResourceBuilder:
manifest_perms: typing.Any,
resource_policy: typing.Any,
descriptor: AgentRunnerDescriptor,
runner_config: dict[str, typing.Any],
) -> list[ToolResource]:
"""Build tools list from binding."""
tools: list[ToolResource] = []
@@ -143,8 +142,43 @@ class AgentResourceBuilder:
if not config_schema.uses_host_tools(descriptor):
return tools
# Get tool names from resource policy
allowed_names = resource_policy.allowed_tool_names
allowed_sources = resource_policy.allowed_tool_sources
if resource_policy.allow_all_tools or allowed_sources is None:
get_catalog = getattr(getattr(self.ap, 'tool_mgr', None), 'get_resolved_tool_catalog', None)
if get_catalog is None:
return tools
try:
catalog = await get_catalog(
include_skill_authoring=True,
include_mcp_resource_tools=True,
)
except Exception as e:
self.ap.logger.warning(f'Failed to resolve visible Host tools: {e}')
return tools
if not resource_policy.allow_all_tools:
selected_names = set(allowed_names or [])
catalog = [item for item in catalog if item.get('name') in selected_names]
allowed_names = ResourcePolicyProjector.extract_tool_names(catalog)
allowed_sources = {
item['name']: ref for item in catalog if (ref := self._catalog_source_ref(item)) is not None
}
if runner_config.get('mcp-resource-agent-read-enabled', True) is not True:
denied_resource_tools = {MCP_TOOL_LIST_RESOURCES, MCP_TOOL_READ_RESOURCE}
allowed_names = [
name
for name in (allowed_names or [])
if not (
name in denied_resource_tools
and allowed_sources
and (source_ref := allowed_sources.get(name)) is not None
and source_ref['source'] == 'mcp'
and source_ref.get('source_id') is None
)
]
tool_operations = [operation for operation in ('detail', 'call') if operation in tool_perms]
# Prefill full tool schema (best-effort) so runners can build LLM tool
@@ -153,20 +187,39 @@ class AgentResourceBuilder:
get_tool_schema = getattr(getattr(self.ap, 'tool_mgr', None), 'get_tool_schema', None)
if allowed_names:
for tool_name in allowed_names:
source_ref = allowed_sources.get(tool_name) if allowed_sources else None
if source_ref is None:
self.ap.logger.warning(f'Tool {tool_name} is not authorized because its Host source is unresolved')
continue
if get_tool_schema is not None:
description, parameters = await get_tool_schema(tool_name)
description, parameters = await get_tool_schema(tool_name, source_ref=source_ref)
else:
description, parameters = None, None
tools.append({
'tool_name': tool_name,
'tool_type': None,
'description': description,
'operations': tool_operations,
'parameters': parameters,
})
tools.append(
{
'tool_name': tool_name,
'tool_type': source_ref['source'],
'description': description,
'operations': tool_operations,
'parameters': parameters,
'source': source_ref['source'],
'source_id': source_ref.get('source_id'),
}
)
return tools
@staticmethod
def _catalog_source_ref(item: dict[str, typing.Any]) -> ToolSourceRef | None:
source = item.get('source')
if not isinstance(source, str) or not source:
return None
source_id = item.get('source_id')
return {
'source': source,
'source_id': source_id if isinstance(source_id, str) and source_id else None,
}
async def _build_knowledge_bases_from_binding(
self,
manifest_perms: typing.Any,
@@ -196,12 +249,16 @@ class AgentResourceBuilder:
try:
kb = await self.ap.rag_mgr.get_knowledge_base_by_uuid(kb_uuid)
if kb:
kb_resources.append({
'kb_id': kb_uuid,
'kb_name': kb.get_name(),
'kb_type': kb.knowledge_base_entity.kb_type if hasattr(kb.knowledge_base_entity, 'kb_type') else None,
'operations': kb_operations,
})
kb_resources.append(
{
'kb_id': kb_uuid,
'kb_name': kb.get_name(),
'kb_type': kb.knowledge_base_entity.kb_type
if hasattr(kb.knowledge_base_entity, 'kb_type')
else None,
'operations': kb_operations,
}
)
except Exception as e:
self.ap.logger.warning(f'Failed to build knowledge base resource {kb_uuid}: {e}')
@@ -233,11 +290,13 @@ class AgentResourceBuilder:
skills: list[SkillResource] = []
for skill_name in names:
skill_data = loaded_skills.get(skill_name) or {}
skills.append({
'skill_name': skill_name,
'display_name': skill_data.get('display_name') or skill_data.get('name') or skill_name,
'description': skill_data.get('description') or None,
})
skills.append(
{
'skill_name': skill_name,
'display_name': skill_data.get('display_name') or skill_data.get('name') or skill_name,
'description': skill_data.get('description') or None,
}
)
return skills
def _build_storage_from_binding(
@@ -285,12 +344,16 @@ class AgentResourceBuilder:
try:
model = await self.ap.model_mgr.get_model_by_uuid(model_uuid)
if model and model.model_entity:
models.append({
'model_id': model_uuid,
'model_type': getattr(model.model_entity, 'model_type', None),
'provider': getattr(model.provider_entity, 'name', None) if hasattr(model, 'provider_entity') else None,
'operations': operations,
})
models.append(
{
'model_id': model_uuid,
'model_type': getattr(model.model_entity, 'model_type', None),
'provider': getattr(model.provider_entity, 'name', None)
if hasattr(model, 'provider_entity')
else None,
'operations': operations,
}
)
seen_model_ids.add(model_uuid)
except Exception as e:
self.ap.logger.warning(f'Failed to build LLM model resource {model_uuid}: {e}')
@@ -308,12 +371,16 @@ class AgentResourceBuilder:
try:
model = await self.ap.model_mgr.get_rerank_model_by_uuid(model_uuid)
if model and model.model_entity:
models.append({
'model_id': model_uuid,
'model_type': getattr(model.model_entity, 'model_type', 'rerank') or 'rerank',
'provider': getattr(model.provider_entity, 'name', None) if hasattr(model, 'provider_entity') else None,
'operations': ['rerank'],
})
models.append(
{
'model_id': model_uuid,
'model_type': getattr(model.model_entity, 'model_type', 'rerank') or 'rerank',
'provider': getattr(model.provider_entity, 'name', None)
if hasattr(model, 'provider_entity')
else None,
'operations': ['rerank'],
}
)
seen_model_ids.add(model_uuid)
except Exception as e:
self.ap.logger.warning(f'Failed to build rerank model resource {model_uuid}: {e}')
src/langbot/pkg/agent/runner/resource_policy.py
+146
View File
@@ -0,0 +1,146 @@
"""Project AgentRunner configuration into Host resource policy."""
from __future__ import annotations
import collections.abc
import typing
from .host_models import ResourcePolicy
class ResourcePolicyProjector:
"""Build one generic resource policy for Pipeline and Agent bindings."""
@classmethod
def from_runner_config(
cls,
runner_config: dict[str, typing.Any] | None,
*,
resolved_model_uuids: collections.abc.Iterable[typing.Any] | None = None,
resolved_tool_names: collections.abc.Iterable[typing.Any] | None = None,
resolved_tool_sources: typing.Mapping[str, typing.Any] | None = None,
resolved_kb_uuids: collections.abc.Iterable[typing.Any] | None = None,
resolved_skill_names: collections.abc.Iterable[typing.Any] | None = None,
) -> ResourcePolicy:
"""Project standard resource fields without depending on a runner ID.
Resolved values are supplied by entry paths that already narrowed the
available resources, such as Pipeline preprocessing. Independent Agent
bindings omit them so the Host can resolve an explicit all-tools grant
against the live tool catalog when the run starts.
"""
config = runner_config if isinstance(runner_config, dict) else {}
selected_tool_names = cls.normalize_names(config.get('tools'))
enable_all_tools = config.get('enable-all-tools', True) is True
if resolved_tool_names is not None:
available_tool_names = cls.normalize_names(resolved_tool_names)
if enable_all_tools:
allowed_tool_names = available_tool_names
else:
selected = set(selected_tool_names)
allowed_tool_names = [name for name in available_tool_names if name in selected]
allow_all_tools = False
elif enable_all_tools:
allowed_tool_names = None
allow_all_tools = True
else:
allowed_tool_names = selected_tool_names
allow_all_tools = False
allowed_tool_sources = cls.normalize_tool_sources(resolved_tool_sources)
if allowed_tool_sources is not None:
allowed = set(allowed_tool_names or [])
allowed_tool_sources = {name: ref for name, ref in allowed_tool_sources.items() if name in allowed}
if resolved_kb_uuids is not None:
allowed_kb_uuids = cls.normalize_names(resolved_kb_uuids)
elif 'knowledge-bases' in config:
allowed_kb_uuids = cls.normalize_names(config.get('knowledge-bases'))
else:
allowed_kb_uuids = None
return ResourcePolicy(
allowed_model_uuids=cls.normalize_optional_names(resolved_model_uuids),
allowed_tool_names=allowed_tool_names,
allowed_tool_sources=allowed_tool_sources,
allow_all_tools=allow_all_tools,
allowed_kb_uuids=allowed_kb_uuids,
allowed_skill_names=cls.normalize_optional_names(resolved_skill_names),
)
@staticmethod
def normalize_names(values: typing.Any) -> list[str]:
"""Return unique, non-empty string identifiers in source order."""
if not isinstance(values, collections.abc.Iterable) or isinstance(values, (str, bytes, dict)):
return []
names: list[str] = []
seen: set[str] = set()
for value in values:
if not isinstance(value, str) or not value or value in seen:
continue
seen.add(value)
names.append(value)
return names
@classmethod
def normalize_optional_names(
cls,
values: collections.abc.Iterable[typing.Any] | None,
) -> list[str] | None:
if values is None:
return None
return cls.normalize_names(values)
@classmethod
def extract_tool_names(cls, tools: collections.abc.Iterable[typing.Any] | None) -> list[str]:
"""Extract tool identifiers from dictionary and SDK object shapes."""
if tools is None:
return []
names: list[str] = []
for tool in tools:
name = tool.get('name') if isinstance(tool, dict) else getattr(tool, 'name', None)
if isinstance(name, str):
names.append(name)
return cls.normalize_names(names)
@staticmethod
def normalize_tool_sources(
values: typing.Mapping[str, typing.Any] | None,
) -> dict[str, dict[str, str | None]] | None:
if values is None:
return None
normalized: dict[str, dict[str, str | None]] = {}
for name, value in values.items():
if not isinstance(name, str) or not name or not isinstance(value, collections.abc.Mapping):
continue
source = value.get('source')
if not isinstance(source, str) or not source:
continue
source_id = value.get('source_id')
normalized[name] = {
'source': source,
'source_id': source_id if isinstance(source_id, str) and source_id else None,
}
return normalized
@classmethod
def filter_tools(
cls,
tools: collections.abc.Iterable[typing.Any],
policy: ResourcePolicy,
) -> list[typing.Any]:
"""Apply a projected policy to an already scoped tool collection."""
tool_list = list(tools)
if policy.allow_all_tools:
return tool_list
allowed_names = set(policy.allowed_tool_names or [])
return [
tool
for tool in tool_list
if (tool.get('name') if isinstance(tool, dict) else getattr(tool, 'name', None)) in allowed_names
]
src/langbot/pkg/agent/runner/run_journal.py
+1 -9
View File
@@ -304,13 +304,10 @@ class AgentRunJournal:
for item in items:
event = item.get('event') if isinstance(item.get('event'), dict) else {}
input_data = item.get('input') if isinstance(item.get('input'), dict) else {}
conversation = item.get('conversation') if isinstance(item.get('conversation'), dict) else {}
actor = item.get('actor') if isinstance(item.get('actor'), dict) else {}
subject = item.get('subject') if isinstance(item.get('subject'), dict) else {}
text = input_data.get('text')
input_summary = text[:1000] if isinstance(text, str) and text else 'Unconsumed steering input dropped'
event_time = None
raw_event_time = event.get('event_time')
if raw_event_time:
@@ -335,12 +332,7 @@ class AgentRunJournal:
actor_name=actor.get('actor_name'),
subject_type=subject.get('subject_type'),
subject_id=subject.get('subject_id'),
input_summary=input_summary,
input_json={
'text': text,
'contents': self._sanitize_contents(input_data.get('contents') or []),
'attachments': self._sanitize_attachments(input_data.get('attachments') or []),
},
input_summary='Unconsumed steering input dropped',
run_id=run_id,
runner_id=runner_id,
event_time=event_time,
src/langbot/pkg/agent/runner/session_registry.py
+31
View File
@@ -1,4 +1,5 @@
"""Agent run session registry for proxy action permission validation."""
from __future__ import annotations
import asyncio
@@ -7,7 +8,10 @@ import typing
import time
import threading
from langbot_plugin.api.entities.builtin.pipeline import query as pipeline_query
from .context_builder import AgentResources
from ...provider.tools.toolmgr import ToolSourceRef
MAX_STEERING_QUEUE_ITEMS = 100
@@ -22,6 +26,7 @@ DEFAULT_RESOURCE_OPERATIONS: dict[str, set[str]] = {
class AgentRunSessionStatus(typing.TypedDict):
"""Status tracking for agent run session."""
started_at: int
last_activity_at: int
@@ -58,13 +63,16 @@ class AgentRunSession(typing.TypedDict):
run_id: Unique run identifier (UUID from AgentRunContext)
runner_id: Runner descriptor ID (plugin:author/name/runner)
query_id: Host entry query ID, only present for query-based adapters
execution_query: Host-only Query used by providers and tool loaders
plugin_identity: Plugin identifier (author/name) of the runner
authorization: Run-scoped authorization snapshot; runtime auth truth
status: Session status tracking
"""
run_id: str
runner_id: str
query_id: int | None
execution_query: pipeline_query.Query | None
plugin_identity: str # author/name
authorization: RunAuthorizationSnapshot
status: AgentRunSessionStatus
@@ -104,6 +112,7 @@ class AgentRunSessionRegistry:
available_apis: dict[str, bool] | None = None,
state_policy: dict[str, typing.Any] | None = None,
state_context: dict[str, typing.Any] | None = None,
execution_query: pipeline_query.Query | None = None,
) -> None:
"""Register a new agent run session.
@@ -120,6 +129,7 @@ class AgentRunSessionRegistry:
available_apis: Run-scoped pull APIs exposed in AgentRunContext
state_policy: State policy from binding (enable_state, state_scopes)
state_context: Context for state API (scope_keys, binding_identity, etc.)
execution_query: Host-only Query used for provider and tool execution
"""
if not isinstance(plugin_identity, str) or not plugin_identity.strip():
raise ValueError('plugin_identity is required for agent run sessions')
@@ -153,6 +163,7 @@ class AgentRunSessionRegistry:
'run_id': run_id,
'runner_id': runner_id,
'query_id': query_id,
'execution_query': execution_query,
'plugin_identity': plugin_identity,
'authorization': authorization,
'status': {
@@ -371,6 +382,26 @@ class AgentRunSessionRegistry:
return False
@staticmethod
def get_tool_source_ref(
session: AgentRunSession,
tool_name: str,
) -> ToolSourceRef | None:
"""Return the implementation identity frozen in this run's grant."""
resources = session['authorization']['resources']
for tool in resources.get('tools', []):
if tool.get('tool_name') != tool_name:
continue
source = tool.get('source')
if not isinstance(source, str) or not source:
return None
source_id = tool.get('source_id')
return {
'source': source,
'source_id': source_id if isinstance(source_id, str) and source_id else None,
}
return None
async def list_active_runs(self) -> list[AgentRunSession]:
"""List all active run sessions.
src/langbot/pkg/api/http/controller/groups/agents.py
+8 -2
View File
@@ -16,7 +16,10 @@ class AgentsRouterGroup(group.RouterGroup):
return self.success(data={'agents': await self.ap.agent_service.get_agents(sort_by, sort_order)})
json_data = await quart.request.json
created = await self.ap.agent_service.create_agent(json_data)
try:
created = await self.ap.agent_service.create_agent(json_data)
except ValueError as exc:
return self.http_status(400, -1, str(exc))
return self.success(data=created)
@self.route('/_/metadata', methods=['GET'], auth_type=group.AuthType.USER_TOKEN_OR_API_KEY)
@@ -33,7 +36,10 @@ class AgentsRouterGroup(group.RouterGroup):
if quart.request.method == 'PUT':
json_data = await quart.request.json
await self.ap.agent_service.update_agent(agent_uuid, json_data)
try:
await self.ap.agent_service.update_agent(agent_uuid, json_data)
except ValueError as exc:
return self.http_status(400, -1, str(exc))
return self.success()
await self.ap.agent_service.delete_agent(agent_uuid)
src/langbot/pkg/api/http/controller/groups/pipelines/pipelines.py
+54 -24
View File
@@ -3,6 +3,7 @@ from __future__ import annotations
import quart
from ... import group
from ......pipeline.extension_preferences import normalize_extension_preferences
@group.group_class('pipelines', '/api/v1/pipelines')
@@ -19,7 +20,10 @@ class PipelinesRouterGroup(group.RouterGroup):
elif quart.request.method == 'POST':
json_data = await quart.request.json
pipeline_uuid = await self.ap.pipeline_service.create_pipeline(json_data)
try:
pipeline_uuid = await self.ap.pipeline_service.create_pipeline(json_data)
except ValueError as exc:
return self.http_status(400, -1, str(exc))
return self.success(data={'uuid': pipeline_uuid})
@@ -41,7 +45,10 @@ class PipelinesRouterGroup(group.RouterGroup):
elif quart.request.method == 'PUT':
json_data = await quart.request.json
await self.ap.pipeline_service.update_pipeline(pipeline_uuid, json_data)
try:
await self.ap.pipeline_service.update_pipeline(pipeline_uuid, json_data)
except ValueError as exc:
return self.http_status(400, -1, str(exc))
return self.success()
elif quart.request.method == 'DELETE':
@@ -81,21 +88,19 @@ class PipelinesRouterGroup(group.RouterGroup):
self.ap.logger.warning('Unable to list skills for pipeline extensions: %s', exc)
available_skills = []
extensions_prefs = pipeline.get('extensions_preferences', {})
extensions_prefs = normalize_extension_preferences(pipeline.get('extensions_preferences'))
return self.success(
data={
'enable_all_plugins': extensions_prefs.get('enable_all_plugins', True),
'enable_all_mcp_servers': extensions_prefs.get('enable_all_mcp_servers', True),
'enable_all_skills': extensions_prefs.get('enable_all_skills', True),
'bound_plugins': extensions_prefs.get('plugins', []),
'enable_all_plugins': extensions_prefs['enable_all_plugins'],
'enable_all_mcp_servers': extensions_prefs['enable_all_mcp_servers'],
'enable_all_skills': extensions_prefs['enable_all_skills'],
'bound_plugins': extensions_prefs['plugins'],
'available_plugins': plugins,
'bound_mcp_servers': extensions_prefs.get('mcp_servers', []),
'bound_mcp_servers': extensions_prefs['mcp_servers'],
'available_mcp_servers': mcp_servers,
'bound_mcp_resources': extensions_prefs.get('mcp_resources', []),
'mcp_resource_agent_read_enabled': extensions_prefs.get(
'mcp_resource_agent_read_enabled', True
),
'bound_skills': extensions_prefs.get('skills', []),
'bound_mcp_resources': extensions_prefs['mcp_resources'],
'mcp_resource_agent_read_enabled': extensions_prefs['mcp_resource_agent_read_enabled'],
'bound_skills': extensions_prefs['skills'],
'available_skills': available_skills,
}
)
@@ -111,16 +116,41 @@ class PipelinesRouterGroup(group.RouterGroup):
bound_mcp_resources = json_data.get('bound_mcp_resources')
mcp_resource_agent_read_enabled = json_data.get('mcp_resource_agent_read_enabled')
await self.ap.pipeline_service.update_pipeline_extensions(
pipeline_uuid,
bound_plugins,
bound_mcp_servers,
enable_all_plugins,
enable_all_mcp_servers,
bound_skills=bound_skills,
enable_all_skills=enable_all_skills,
bound_mcp_resources=bound_mcp_resources,
mcp_resource_agent_read_enabled=mcp_resource_agent_read_enabled,
)
extension_flags = {
'enable_all_plugins': enable_all_plugins,
'enable_all_mcp_servers': enable_all_mcp_servers,
'enable_all_skills': enable_all_skills,
}
for field, value in extension_flags.items():
if field in json_data and not isinstance(value, bool):
return self.http_status(
400,
-1,
f"Pipeline extension field '{field}' must be a boolean",
)
if 'mcp_resource_agent_read_enabled' in json_data and not isinstance(
mcp_resource_agent_read_enabled, bool
):
return self.http_status(
400,
-1,
"Pipeline extension field 'mcp_resource_agent_read_enabled' must be a boolean",
)
try:
await self.ap.pipeline_service.update_pipeline_extensions(
pipeline_uuid,
bound_plugins,
bound_mcp_servers,
enable_all_plugins,
enable_all_mcp_servers,
bound_skills=bound_skills,
enable_all_skills=enable_all_skills,
bound_mcp_resources=bound_mcp_resources,
mcp_resource_agent_read_enabled=mcp_resource_agent_read_enabled,
)
except ValueError as exc:
return self.http_status(400, -1, str(exc))
return self.success()
src/langbot/pkg/api/http/controller/groups/resources/tools.py
+40 -38
View File
@@ -3,59 +3,61 @@ from __future__ import annotations
import quart
from ... import group
from ......pipeline.extension_preferences import normalize_extension_preferences
@group.group_class('tools', '/api/v1/tools')
class ToolsRouterGroup(group.RouterGroup):
async def _get_scoped_tool_catalog(self) -> list[dict] | None:
pipeline_uuid = quart.request.args.get('pipeline_uuid') or quart.request.args.get('pipeline_id')
bound_plugins: list[str] | None = None
bound_mcp_servers: list[str] | None = None
if pipeline_uuid:
pipeline = await self.ap.pipeline_service.get_pipeline(pipeline_uuid)
if pipeline is None:
return None
extensions_prefs = normalize_extension_preferences(pipeline.get('extensions_preferences'))
if not extensions_prefs['enable_all_plugins']:
bound_plugins = [f'{plugin["author"]}/{plugin["name"]}' for plugin in extensions_prefs['plugins']]
if not extensions_prefs['enable_all_mcp_servers']:
bound_mcp_servers = extensions_prefs['mcp_servers']
return await self.ap.tool_mgr.get_resolved_tool_catalog(
bound_plugins,
bound_mcp_servers,
include_skill_authoring=True,
)
async def initialize(self) -> None:
@self.route('', methods=['GET'], auth_type=group.AuthType.USER_TOKEN)
async def _() -> str:
"""获取所有可用工具列表"""
pipeline_uuid = quart.request.args.get('pipeline_uuid') or quart.request.args.get('pipeline_id')
bound_plugins: list[str] | None = None
bound_mcp_servers: list[str] | None = None
catalog = await self._get_scoped_tool_catalog()
if catalog is None:
return self.http_status(404, -1, 'pipeline not found')
return self.success(data={'tools': catalog})
if pipeline_uuid:
pipeline = await self.ap.pipeline_service.get_pipeline(pipeline_uuid)
if pipeline is None:
return self.http_status(404, -1, 'pipeline not found')
extensions_prefs = pipeline.get('extensions_preferences', {}) or {}
if not extensions_prefs.get('enable_all_plugins', True):
bound_plugins = [
f'{plugin.get("author", "")}/{plugin.get("name", "")}'
for plugin in extensions_prefs.get('plugins', [])
if isinstance(plugin, dict) and plugin.get('name')
]
if not extensions_prefs.get('enable_all_mcp_servers', True):
bound_mcp_servers = [
server for server in (extensions_prefs.get('mcp_servers', []) or []) if isinstance(server, str)
]
return self.success(
data={
'tools': await self.ap.tool_mgr.get_tool_catalog(
bound_plugins,
bound_mcp_servers,
include_skill_authoring=True,
)
}
)
@self.route('/<tool_name>', methods=['GET'], auth_type=group.AuthType.USER_TOKEN)
@self.route('/<path:tool_name>', methods=['GET'], auth_type=group.AuthType.USER_TOKEN)
async def _(tool_name: str) -> str:
"""获取特定工具详情"""
tools = await self.ap.tool_mgr.get_all_tools()
catalog = await self._get_scoped_tool_catalog()
if catalog is None:
return self.http_status(404, -1, 'pipeline not found')
for tool in tools:
if tool.name == tool_name:
for tool in catalog:
if tool.get('name') == tool_name:
return self.success(
data={
'tool': {
'name': tool.name,
'description': tool.description,
'human_desc': tool.human_desc,
'parameters': tool.parameters,
'name': tool['name'],
'description': tool.get('description') or '',
'human_desc': tool.get('human_desc') or '',
'parameters': tool.get('parameters') or {},
'source': tool.get('source'),
'source_name': tool.get('source_name'),
'source_id': tool.get('source_id'),
}
}
)
src/langbot/pkg/api/http/service/agent.py
+10 -16
View File
@@ -7,6 +7,7 @@ import typing
import sqlalchemy
from ....core import app
from ....agent.runner.config_resolver import RunnerConfigResolver
from ....entity.persistence import agent as persistence_agent
@@ -82,8 +83,8 @@ class AgentService:
if kind != AGENT_KIND_AGENT:
raise ValueError(f'Unsupported agent kind: {kind}')
config = agent_data.get('config') or await self._get_default_agent_config()
runner_id = self._resolve_runner_id(config)
config = agent_data['config'] if 'config' in agent_data else await self._get_default_agent_config()
config, runner_id, _ = RunnerConfigResolver.resolve_agent_runner_config(config)
new_uuid = str(uuid.uuid4())
values = {
'uuid': new_uuid,
@@ -91,7 +92,7 @@ class AgentService:
'description': agent_data.get('description') or '',
'emoji': agent_data.get('emoji') or '🤖',
'kind': AGENT_KIND_AGENT,
'component_ref': agent_data.get('component_ref') or runner_id,
'component_ref': runner_id,
'config': config,
'enabled': agent_data.get('enabled', True),
'supported_event_patterns': agent_data.get('supported_event_patterns') or AGENT_DEFAULT_EVENT_PATTERNS,
@@ -109,12 +110,14 @@ class AgentService:
return
update_data = agent_data.copy()
for protected_field in ('uuid', 'kind', 'created_at', 'updated_at', 'capability'):
for protected_field in ('uuid', 'kind', 'component_ref', 'created_at', 'updated_at', 'capability'):
update_data.pop(protected_field, None)
if 'config' in update_data:
update_data['component_ref'] = update_data.get('component_ref') or self._resolve_runner_id(
update_data['config']
)
config, runner_id, _ = RunnerConfigResolver.resolve_agent_runner_config(update_data['config'])
update_data['config'] = config
else:
_, runner_id, _ = RunnerConfigResolver.resolve_agent_runner_config(existing_agent.config)
update_data['component_ref'] = runner_id
if 'supported_event_patterns' in update_data and not update_data['supported_event_patterns']:
update_data['supported_event_patterns'] = AGENT_DEFAULT_EVENT_PATTERNS
@@ -169,15 +172,6 @@ class AgentService:
},
}
@staticmethod
def _resolve_runner_id(config: dict[str, typing.Any]) -> str | None:
runner = config.get('runner') if isinstance(config, dict) else None
if isinstance(runner, dict):
runner_id = runner.get('id')
if runner_id:
return runner_id
return None
def _agent_to_product_item(
self,
agent: persistence_agent.Agent,
src/langbot/pkg/api/http/service/pipeline.py
+66 -13
View File
@@ -6,7 +6,12 @@ import sqlalchemy
import typing
from ....core import app
from ....agent.runner.config_resolver import RunnerConfigResolver
from ....entity.persistence import pipeline as persistence_pipeline
from ....pipeline.extension_preferences import (
normalize_extension_preferences,
validate_extension_preferences,
)
default_stage_order = [
@@ -163,6 +168,11 @@ class PipelineService:
return self.ap.persistence_mgr.serialize_model(persistence_pipeline.LegacyPipeline, pipeline)
async def create_pipeline(self, pipeline_data: dict, default: bool = False) -> str:
if 'extensions_preferences' in pipeline_data:
self._validate_extension_preferences(pipeline_data['extensions_preferences'])
if 'config' in pipeline_data:
RunnerConfigResolver.validate_pipeline_config(pipeline_data['config'])
# Check limitation
limitation = self.ap.instance_config.data.get('system', {}).get('limitation', {})
max_pipelines = limitation.get('max_pipelines', -1)
@@ -177,6 +187,7 @@ class PipelineService:
pipeline_data['is_default'] = default
pipeline_data['config'] = await self.get_default_pipeline_config()
RunnerConfigResolver.validate_pipeline_config(pipeline_data['config'])
# Ensure extensions_preferences is set with enable_all_plugins and enable_all_mcp_servers=True by default
if 'extensions_preferences' not in pipeline_data:
@@ -203,6 +214,10 @@ class PipelineService:
pipeline_data = pipeline_data.copy()
for protected_field in ('uuid', 'for_version', 'stages', 'is_default'):
pipeline_data.pop(protected_field, None)
if 'config' in pipeline_data:
RunnerConfigResolver.validate_pipeline_config(pipeline_data['config'])
if 'extensions_preferences' in pipeline_data:
self._validate_extension_preferences(pipeline_data['extensions_preferences'])
await self.ap.persistence_mgr.execute_async(
sqlalchemy.update(persistence_pipeline.LegacyPipeline)
@@ -259,18 +274,7 @@ class PipelineService:
'stages': original_pipeline.stages.copy() if original_pipeline.stages else default_stage_order.copy(),
'config': original_pipeline.config.copy() if original_pipeline.config else {},
'is_default': False,
'extensions_preferences': (
original_pipeline.extensions_preferences.copy()
if original_pipeline.extensions_preferences
else {
'enable_all_plugins': True,
'enable_all_mcp_servers': True,
'plugins': [],
'mcp_servers': [],
'mcp_resources': [],
'mcp_resource_agent_read_enabled': True,
}
),
'extensions_preferences': normalize_extension_preferences(original_pipeline.extensions_preferences),
}
# Insert the new pipeline
@@ -297,6 +301,36 @@ class PipelineService:
mcp_resource_agent_read_enabled: bool | None = None,
) -> None:
"""Update the bound plugins and MCP servers for a pipeline"""
extension_updates: dict[str, typing.Any] = {
'enable_all_plugins': enable_all_plugins,
'enable_all_mcp_servers': enable_all_mcp_servers,
'enable_all_skills': enable_all_skills,
'plugins': bound_plugins,
}
if bound_mcp_servers is not None:
extension_updates['mcp_servers'] = bound_mcp_servers
if bound_skills is not None:
extension_updates['skills'] = bound_skills
if bound_mcp_resources is not None:
extension_updates['mcp_resources'] = bound_mcp_resources
RunnerConfigResolver.validate_mcp_resource_attachments(
bound_mcp_resources,
context='Pipeline extension',
field_name='bound_mcp_resources',
)
if mcp_resource_agent_read_enabled is not None:
extension_updates['mcp_resource_agent_read_enabled'] = mcp_resource_agent_read_enabled
self._validate_extension_preferences(
extension_updates,
context='Pipeline extension',
field_aliases={
'plugins': 'bound_plugins',
'mcp_servers': 'bound_mcp_servers',
'skills': 'bound_skills',
'mcp_resources': 'bound_mcp_resources',
},
)
# Get current pipeline
result = await self.ap.persistence_mgr.execute_async(
sqlalchemy.select(persistence_pipeline.LegacyPipeline).where(
@@ -309,7 +343,7 @@ class PipelineService:
raise ValueError(f'Pipeline {pipeline_uuid} not found')
# Update extensions_preferences
extensions_preferences = pipeline.extensions_preferences or {}
extensions_preferences = normalize_extension_preferences(pipeline.extensions_preferences)
extensions_preferences['enable_all_plugins'] = enable_all_plugins
extensions_preferences['enable_all_mcp_servers'] = enable_all_mcp_servers
extensions_preferences['enable_all_skills'] = enable_all_skills
@@ -333,3 +367,22 @@ class PipelineService:
await self.ap.pipeline_mgr.remove_pipeline(pipeline_uuid)
pipeline = await self.get_pipeline(pipeline_uuid)
await self.ap.pipeline_mgr.load_pipeline(pipeline)
@staticmethod
def _validate_extension_preferences(
value: typing.Any,
*,
context: str = 'Pipeline extensions_preferences',
field_aliases: typing.Mapping[str, str] | None = None,
) -> dict[str, typing.Any]:
validated = validate_extension_preferences(
value,
context=context,
field_aliases=field_aliases,
)
RunnerConfigResolver.validate_mcp_resource_attachments(
validated.get('mcp_resources'),
context=context,
field_name='mcp_resources',
)
return validated
src/langbot/pkg/box/service.py
+46 -84
View File
@@ -4,6 +4,7 @@ import asyncio
import collections
import datetime as _dt
import enum
import hashlib
import json
import os
from typing import TYPE_CHECKING
@@ -13,6 +14,7 @@ import pydantic
from langbot_plugin.box.client import BoxRuntimeClient
from .connector import BoxRuntimeConnector, _get_box_config
from ..telemetry import features as telemetry_features
from ..utils import constants
from langbot_plugin.box.errors import BoxError, BoxValidationError
from langbot_plugin.box.models import (
BUILTIN_PROFILES,
@@ -27,6 +29,8 @@ _INT_ADAPTER = pydantic.TypeAdapter(int)
_UTC = _dt.timezone.utc
_MAX_RECENT_ERRORS = 50
_MIB = 1024 * 1024
_HOST_BOX_SCOPE_VARIABLE = '_host_box_scope'
_BOX_SESSION_ID_PREFIX = 'lb-box-'
def _is_path_under(path: str, root: str) -> bool:
@@ -224,45 +228,53 @@ class BoxService:
return self._serialize_result(result)
def resolve_box_session_id(self, query: pipeline_query.Query) -> str:
"""Resolve the Box session_id from the pipeline's template and query variables.
"""Resolve a Host-owned Box session ID for the current conversation."""
if query is None:
raise BoxValidationError('Box execution requires a Host session context.')
When ``system.limitation.force_box_session_id_template`` is set to a
non-empty value, that template overrides whatever the pipeline
configured. This is the authoritative SaaS guard: it runs on every
``exec`` call, so a tenant cannot escape a single shared sandbox even
by editing the pipeline config directly through the API (which only
gates the web UI).
"""
forced_template = self._forced_box_session_id_template()
if forced_template:
template = forced_template
else:
template = '{launcher_type}_{launcher_id}'
pipeline_config = query.pipeline_config or {}
ai_config = pipeline_config.get('ai', {}) if isinstance(pipeline_config, dict) else {}
runner_selector = ai_config.get('runner', {}) if isinstance(ai_config, dict) else {}
runner_id = runner_selector.get('id') if isinstance(runner_selector, dict) else None
runner_configs = ai_config.get('runner_config', {}) if isinstance(ai_config, dict) else {}
runner_config = runner_configs.get(runner_id, {}) if isinstance(runner_configs, dict) else {}
configured_template = (
runner_config.get('box-session-id-template') if isinstance(runner_config, dict) else None
)
if isinstance(configured_template, str) and configured_template:
template = configured_template
variables = dict(query.variables or {})
variables = getattr(query, 'variables', None)
if isinstance(variables, dict) and _HOST_BOX_SCOPE_VARIABLE in variables:
private_scope = variables[_HOST_BOX_SCOPE_VARIABLE]
if not isinstance(private_scope, str) or not private_scope.strip():
raise BoxValidationError('Box execution requires a Host conversation scope.')
return self._hash_box_session_scope(f'host:{private_scope}')
session = getattr(query, 'session', None)
launcher_type = getattr(query, 'launcher_type', None)
launcher_id = getattr(query, 'launcher_id', None)
if launcher_type is None:
launcher_type = getattr(session, 'launcher_type', None)
if launcher_id is None:
launcher_id = getattr(session, 'launcher_id', None)
if hasattr(launcher_type, 'value'):
launcher_type = launcher_type.value
launcher_id = getattr(query, 'launcher_id', None)
sender_id = getattr(query, 'sender_id', None)
query_id = getattr(query, 'query_id', None)
variables.setdefault('query_id', str(query_id or 'unknown'))
variables.setdefault('launcher_type', str(launcher_type or 'query'))
variables.setdefault('launcher_id', str(launcher_id or query_id or 'unknown'))
variables.setdefault('sender_id', str(sender_id or launcher_id or query_id or 'unknown'))
variables.setdefault('global', 'global')
return template.format_map(collections.defaultdict(lambda: 'unknown', variables))
if launcher_type is None or launcher_id is None or not str(launcher_id).strip():
raise BoxValidationError('Box execution requires a Host session context.')
adapter = getattr(query, 'adapter', None)
adapter_identity = adapter.__class__.__name__ if adapter is not None else None
scope = json.dumps(
{
'instance_id': str(constants.instance_id or '') or None,
'workspace_id': None,
'bot_id': getattr(query, 'bot_uuid', None),
'platform_adapter': adapter_identity,
'target_type': str(launcher_type),
'target_id': str(launcher_id),
'thread_id': None,
},
ensure_ascii=False,
sort_keys=True,
separators=(',', ':'),
)
return self._hash_box_session_scope(f'host:{scope}')
@staticmethod
def _hash_box_session_scope(scope: str) -> str:
digest = hashlib.sha256(scope.encode('utf-8')).hexdigest()
return f'{_BOX_SESSION_ID_PREFIX}{digest}'
def build_skill_extra_mounts(self, query: pipeline_query.Query) -> list[dict]:
"""Build extra_mounts entries for all pipeline-bound skills.
@@ -1128,20 +1140,6 @@ class BoxService:
raw = str(self._local_config().get('image', '') or '').strip()
return raw or None
def _forced_box_session_id_template(self) -> str:
"""Return the SaaS-forced sandbox-scope template, or '' when unset.
Read from ``system.limitation.force_box_session_id_template``. A
non-empty value pins every pipeline to a single sandbox scope
(e.g. ``'{global}'``) and cannot be overridden per-pipeline.
"""
limitation = (
(self.ap.instance_config.data or {}).get('system', {}).get('limitation', {})
if getattr(self.ap, 'instance_config', None) is not None
else {}
)
return str(limitation.get('force_box_session_id_template', '') or '').strip()
def _load_workspace_quota_mb(self) -> int | None:
raw_value = self._local_config().get('workspace_quota_mb')
if raw_value in (None, ''):
@@ -1311,42 +1309,6 @@ class BoxService:
def get_recent_errors(self) -> list[dict]:
return list(self._recent_errors)
def get_system_guidance(self, query_id=None) -> str:
"""Return LLM system-prompt guidance for the exec tool.
All execution-specific prompt text is kept here so that callers
(e.g. LocalAgentRunner) stay free of box domain knowledge.
``query_id`` is the current turn's pipeline query id. When provided,
the guidance ALWAYS advertises the per-query outbox path so the agent
knows how to deliver generated files back to the user — even on turns
where the user sent no inbound attachment (e.g. "generate a QR code"),
which is exactly when the inbound-attachment note never fires. Outbound
collection in the wrapper runs on every turn regardless of inbound
files, so without this the file would be produced and silently dropped.
"""
guidance = (
'When the exec tool is available, use it for exact calculations, statistics, structured data parsing, '
'and code execution instead of estimating mentally. If the user provides numbers, tables, CSV-like text, '
'JSON, or other data and asks for a computed answer, prefer running a short Python script via exec '
'and then answer from the tool result. Unless the user explicitly asks for the script, code, or implementation '
'details, do not include the generated script in the final answer; return the result and a brief explanation only.'
)
if self.default_workspace:
guidance += (
' A default workspace is mounted at /workspace for file tasks. When the user asks to read, create, or '
'modify local files in the working directory, use exec with /workspace paths directly; do not ask the '
'user for directory parameters unless they explicitly need a different directory.'
)
if query_id is not None:
outbox_dir = f'{self.OUTBOX_MOUNT_DIR}/{query_id}'
guidance += (
f' If you produce any file (image, audio, document, etc.) that should be sent back to the user, '
f'write it into {outbox_dir}/ (create the directory if needed). Every file placed there will be '
'delivered to the user automatically; do not paste file contents or base64 into your reply.'
)
return guidance
async def get_status(self) -> dict:
if not self._available:
return {
src/langbot/pkg/core/stages/load_config.py
-38
View File
@@ -110,44 +110,6 @@ class LoadConfigStage(stage.BootingStage):
async def run(self, ap: app.Application):
"""Load config file"""
# # ======= deprecated =======
# if os.path.exists('data/config/command.json'):
# ap.command_cfg = await config.load_json_config(
# 'data/config/command.json',
# 'templates/legacy/command.json',
# completion=False,
# )
# if os.path.exists('data/config/pipeline.json'):
# ap.pipeline_cfg = await config.load_json_config(
# 'data/config/pipeline.json',
# 'templates/legacy/pipeline.json',
# completion=False,
# )
# if os.path.exists('data/config/platform.json'):
# ap.platform_cfg = await config.load_json_config(
# 'data/config/platform.json',
# 'templates/legacy/platform.json',
# completion=False,
# )
# if os.path.exists('data/config/provider.json'):
# ap.provider_cfg = await config.load_json_config(
# 'data/config/provider.json',
# 'templates/legacy/provider.json',
# completion=False,
# )
# if os.path.exists('data/config/system.json'):
# ap.system_cfg = await config.load_json_config(
# 'data/config/system.json',
# 'templates/legacy/system.json',
# completion=False,
# )
# # ======= deprecated =======
ap.instance_config = await config.load_yaml_config('data/config.yaml', 'config.yaml', completion=False)
# Apply environment variable overrides to data/config.yaml
src/langbot/pkg/entity/persistence/metadata.py
-9
View File
@@ -1,15 +1,6 @@
import sqlalchemy
from .base import Base
from ...utils import constants
initial_metadata = [
{
'key': 'database_version',
'value': str(constants.required_database_version),
},
]
class Metadata(Base):
src/langbot/pkg/persistence/alembic/versions/0001_baseline.py
+3 -4
View File
@@ -1,7 +1,7 @@
"""baseline: stamp existing schema (db version 25)
"""baseline: stamp the supported 4.x schema
This is a no-op migration that marks the starting point for Alembic.
All tables already exist via create_all() + legacy DBMigration system.
Current tables already exist via SQLAlchemy metadata create_all().
Revision ID: 0001_baseline
Revises: None
@@ -15,8 +15,7 @@ depends_on = None
def upgrade() -> None:
# No-op: existing schema is already at database_version=25
# This revision serves as the Alembic baseline.
# No-op: this revision serves as the Alembic baseline.
pass
src/langbot/pkg/persistence/alembic/versions/0006_normalize_mcp_remote_mode.py
+2 -2
View File
@@ -10,7 +10,7 @@ ssereadtimeout) live in ``extra_args`` and are left untouched — the
auto-detecting remote transport consumes them regardless.
Revision ID: 0006_normalize_mcp_remote_mode
Revises: 8d3a1f2c4b6e
Revises: 0005_add_llm_context_length
Create Date: 2026-06-21
"""
@@ -18,7 +18,7 @@ import sqlalchemy as sa
from alembic import op
revision = '0006_normalize_mcp_remote_mode'
down_revision = '8d3a1f2c4b6e'
down_revision = '0005_add_llm_context_length'
branch_labels = None
depends_on = None
src/langbot/pkg/persistence/alembic/versions/0007_add_bot_admins.py
+71 -24
View File
@@ -5,6 +5,8 @@ Revises: 0006_normalize_mcp_remote_mode
Create Date: 2026-06-26
"""
import json
import sqlalchemy as sa
from alembic import op
@@ -14,19 +16,64 @@ branch_labels = None
depends_on = None
_BOT_ADMINS = sa.table(
'bot_admins',
sa.column('bot_uuid', sa.String(255)),
sa.column('launcher_type', sa.String(64)),
sa.column('launcher_id', sa.String(255)),
)
def _upsert_admin(conn: sa.Connection, *, bot_uuid: str, launcher_type: str, launcher_id: str) -> None:
values = {
'bot_uuid': bot_uuid,
'launcher_type': launcher_type,
'launcher_id': launcher_id,
}
conflict_columns = [
_BOT_ADMINS.c.bot_uuid,
_BOT_ADMINS.c.launcher_type,
_BOT_ADMINS.c.launcher_id,
]
if conn.dialect.name == 'postgresql':
from sqlalchemy.dialects.postgresql import insert
statement = insert(_BOT_ADMINS).values(**values).on_conflict_do_nothing(index_elements=conflict_columns)
elif conn.dialect.name == 'sqlite':
from sqlalchemy.dialects.sqlite import insert
statement = insert(_BOT_ADMINS).values(**values).on_conflict_do_nothing(index_elements=conflict_columns)
else:
existing = conn.execute(
sa.select(sa.literal(1))
.select_from(_BOT_ADMINS)
.where(
_BOT_ADMINS.c.bot_uuid == bot_uuid,
_BOT_ADMINS.c.launcher_type == launcher_type,
_BOT_ADMINS.c.launcher_id == launcher_id,
)
.limit(1)
).first()
if existing is not None:
return
statement = sa.insert(_BOT_ADMINS).values(**values)
conn.execute(statement)
def upgrade() -> None:
conn = op.get_bind()
if 'bot_admins' in sa.inspect(conn).get_table_names():
return
op.create_table(
'bot_admins',
sa.Column('id', sa.Integer, primary_key=True, autoincrement=True),
sa.Column('bot_uuid', sa.String(255), nullable=False),
sa.Column('launcher_type', sa.String(64), nullable=False),
sa.Column('launcher_id', sa.String(255), nullable=False),
sa.Column('created_at', sa.DateTime, nullable=False, server_default=sa.func.now()),
sa.UniqueConstraint('bot_uuid', 'launcher_type', 'launcher_id', name='uq_bot_admin'),
)
if 'bot_admins' not in sa.inspect(conn).get_table_names():
op.create_table(
'bot_admins',
sa.Column('id', sa.Integer, primary_key=True, autoincrement=True),
sa.Column('bot_uuid', sa.String(255), nullable=False),
sa.Column('launcher_type', sa.String(64), nullable=False),
sa.Column('launcher_id', sa.String(255), nullable=False),
sa.Column('created_at', sa.DateTime, nullable=False, server_default=sa.func.now()),
sa.UniqueConstraint('bot_uuid', 'launcher_type', 'launcher_id', name='uq_bot_admin'),
)
# Migrate old config-based admins into the first bot (best-effort)
inspector = sa.inspect(conn)
@@ -48,28 +95,27 @@ def upgrade() -> None:
if meta_row is None:
return
import json
try:
cfg = json.loads(meta_row[0])
except Exception:
except (TypeError, json.JSONDecodeError):
return
admins = cfg.get('admins', [])
if not isinstance(admins, list):
return
for entry in admins:
if not isinstance(entry, str):
continue
parts = entry.split('_', 1)
if len(parts) != 2:
continue
launcher_type, launcher_id = parts
try:
conn.execute(
sa.text(
'INSERT OR IGNORE INTO bot_admins (bot_uuid, launcher_type, launcher_id) VALUES (:bu, :lt, :li)'
),
{'bu': first_bot_uuid, 'lt': launcher_type, 'li': launcher_id},
)
except Exception:
pass
_upsert_admin(
conn,
bot_uuid=first_bot_uuid,
launcher_type=launcher_type,
launcher_id=launcher_id,
)
# Remove admins key from stored config
if 'admins' in cfg:
@@ -81,4 +127,5 @@ def upgrade() -> None:
def downgrade() -> None:
op.drop_table('bot_admins')
if 'bot_admins' in sa.inspect(op.get_bind()).get_table_names():
op.drop_table('bot_admins')
src/langbot/pkg/persistence/alembic/versions/0009_migrate_routing_to_event_bindings.py → src/langbot/pkg/persistence/alembic/versions/0009_migrate_event_bindings.py
+17 -13
View File
@@ -3,7 +3,7 @@
Bindings keep referencing the original Pipeline UUIDs. This migration never
creates Agent rows or copies Pipeline runner configuration.
Revision ID: 0009_migrate_routing_to_event_bindings
Revision ID: 0009_migrate_event_bindings
Revises: 0008_agent_product_surface
Create Date: 2026-06-26
"""
@@ -14,7 +14,7 @@ import uuid
import sqlalchemy as sa
from alembic import op
revision = '0009_migrate_routing_to_event_bindings'
revision = '0009_migrate_event_bindings'
down_revision = '0008_agent_product_surface'
depends_on = None
@@ -30,24 +30,28 @@ def _column_exists(table_name: str, column_name: str) -> bool:
def _rule_to_filters(rule: dict) -> list[dict] | None:
"""Convert a pipeline_routing_rule to event_binding filters (best effort).
Rules that don't map cleanly (message_content, message_has_element) are
skipped — callers should handle None as "cannot migrate".
"""
"""Convert one legacy Pipeline routing rule to an EBA event filter."""
rule_type = rule.get('type')
operator = rule.get('operator', 'eq')
value = rule.get('value', '')
if rule_type == 'launcher_type':
if value == 'group':
return [{'field': 'group', 'operator': 'neq', 'value': None}]
if value == 'person':
return [{'field': 'group', 'operator': 'eq', 'value': None}]
elif rule_type == 'launcher_id':
return [{'field': 'chat_type', 'operator': operator, 'value': value}]
if rule_type == 'launcher_id':
return [{'field': 'chat_id', 'operator': operator, 'value': value}]
if rule_type == 'message_content':
return [{'field': 'message_text', 'operator': operator, 'value': value}]
if rule_type == 'message_has_element':
element_operator = {
'eq': 'contains',
'neq': 'not_contains',
}.get(operator)
if element_operator is None:
# The legacy matcher treated every other operator as non-matching.
element_operator = 'unsupported_legacy_operator'
return [{'field': 'message_element_types', 'operator': element_operator, 'value': value}]
return None # message_content / message_has_element: no clean mapping
return None
def upgrade() -> None:
src/langbot/pkg/persistence/alembic/versions/0010_merge_mcp_agent_heads.py
+19
View File
@@ -0,0 +1,19 @@
"""merge mcp resource and agent product migration heads
Revision ID: 0010_merge_mcp_agent_heads
Revises: 0008_mcp_resource_prefs, 0009_migrate_event_bindings
Create Date: 2026-06-30
"""
revision = '0010_merge_mcp_agent_heads'
down_revision = ('0008_mcp_resource_prefs', '0009_migrate_event_bindings')
branch_labels = None
depends_on = None
def upgrade() -> None:
pass
def downgrade() -> None:
pass
src/langbot/pkg/persistence/alembic/versions/0010_merge_mcp_resource_agent_heads.py
-19
View File
@@ -1,19 +0,0 @@
"""merge mcp resource and agent product migration heads
Revision ID: 0010_merge_mcp_resource_agent_heads
Revises: 0008_mcp_resource_prefs, 0009_migrate_routing_to_event_bindings
Create Date: 2026-06-30
"""
revision = '0010_merge_mcp_resource_agent_heads'
down_revision = ('0008_mcp_resource_prefs', '0009_migrate_routing_to_event_bindings')
branch_labels = None
depends_on = None
def upgrade() -> None:
pass
def downgrade() -> None:
pass
src/langbot/pkg/persistence/alembic/versions/0011_drop_legacy_bot_routing.py
+2 -2
View File
@@ -1,7 +1,7 @@
"""drop legacy bot pipeline routing columns
Revision ID: 0011_drop_legacy_bot_routing
Revises: 0010_merge_mcp_resource_agent_heads
Revises: 0010_merge_mcp_agent_heads
Create Date: 2026-07-01
"""
@@ -10,7 +10,7 @@ import sqlalchemy as sa
revision = '0011_drop_legacy_bot_routing'
down_revision = '0010_merge_mcp_resource_agent_heads'
down_revision = '0010_merge_mcp_agent_heads'
branch_labels = None
depends_on = None
src/langbot/pkg/persistence/alembic/versions/0012_add_monitoring_tool_calls.py
+67
View File
@@ -0,0 +1,67 @@
"""add monitoring tool calls
Revision ID: 0012_monitoring_tool_calls
Revises: 0011_drop_legacy_bot_routing
Create Date: 2026-07-12
"""
from __future__ import annotations
import sqlalchemy as sa
from alembic import op
revision = '0012_monitoring_tool_calls'
down_revision = '0011_drop_legacy_bot_routing'
branch_labels = None
depends_on = None
_INDEXES = {
'ix_monitoring_tool_calls_timestamp': ['timestamp'],
'ix_monitoring_tool_calls_bot_id': ['bot_id'],
'ix_monitoring_tool_calls_pipeline_id': ['pipeline_id'],
'ix_monitoring_tool_calls_session_id': ['session_id'],
'ix_monitoring_tool_calls_message_id': ['message_id'],
}
def _table_exists() -> bool:
return 'monitoring_tool_calls' in sa.inspect(op.get_bind()).get_table_names()
def _index_names() -> set[str]:
if not _table_exists():
return set()
return {index['name'] for index in sa.inspect(op.get_bind()).get_indexes('monitoring_tool_calls')}
def upgrade() -> None:
if not _table_exists():
op.create_table(
'monitoring_tool_calls',
sa.Column('id', sa.String(255), primary_key=True),
sa.Column('timestamp', sa.DateTime(), nullable=False),
sa.Column('tool_name', sa.String(255), nullable=False),
sa.Column('tool_source', sa.String(50), nullable=False),
sa.Column('duration', sa.Integer(), nullable=False),
sa.Column('status', sa.String(50), nullable=False),
sa.Column('bot_id', sa.String(255), nullable=False),
sa.Column('bot_name', sa.String(255), nullable=False),
sa.Column('pipeline_id', sa.String(255), nullable=False),
sa.Column('pipeline_name', sa.String(255), nullable=False),
sa.Column('session_id', sa.String(255), nullable=True),
sa.Column('message_id', sa.String(255), nullable=True),
sa.Column('arguments', sa.Text(), nullable=True),
sa.Column('result', sa.Text(), nullable=True),
sa.Column('error_message', sa.Text(), nullable=True),
)
existing_indexes = _index_names()
for index_name, columns in _INDEXES.items():
if index_name not in existing_indexes:
op.create_index(index_name, 'monitoring_tool_calls', columns, unique=False)
def downgrade() -> None:
if _table_exists():
op.drop_table('monitoring_tool_calls')
src/langbot/pkg/persistence/mgr.py
+5 -53
View File
@@ -7,15 +7,14 @@ import typing
import sqlalchemy.ext.asyncio as sqlalchemy_asyncio
import sqlalchemy
from . import database, migration
from ..entity.persistence import base, metadata, model as persistence_model
from . import database
from ..entity.persistence import base, model as persistence_model
from ..entity import persistence
from ..core import app
from ..utils import constants, importutil
from . import databases, migrations
from ..utils import importutil
from . import databases
importutil.import_modules_in_pkg(databases)
importutil.import_modules_in_pkg(migrations)
importutil.import_modules_in_pkg(persistence)
@@ -43,40 +42,6 @@ class PersistenceManager:
break
await self.create_tables()
# run migrations
database_version = await self.execute_async(
sqlalchemy.select(metadata.Metadata).where(metadata.Metadata.key == 'database_version')
)
database_version = int(database_version.fetchone()[1])
required_database_version = constants.required_database_version
if database_version < required_database_version:
migrations = migration.preregistered_db_migrations
migrations.sort(key=lambda x: x.number)
last_migration_number = database_version
for migration_cls in migrations:
migration_instance = migration_cls(self.ap)
if (
migration_instance.number > database_version
and migration_instance.number <= required_database_version
):
await migration_instance.upgrade()
await self.execute_async(
sqlalchemy.update(metadata.Metadata)
.where(metadata.Metadata.key == 'database_version')
.values({'value': str(migration_instance.number)})
)
last_migration_number = migration_instance.number
self.ap.logger.info(f'Migration {migration_instance.number} completed.')
self.ap.logger.info(f'Successfully upgraded database to version {last_migration_number}.')
# Run Alembic migrations (new migration system)
await self._run_alembic_migrations()
await self.write_space_model_providers()
@@ -88,19 +53,6 @@ class PersistenceManager:
await conn.commit()
# ======= write initial data =======
# write initial metadata
self.ap.logger.info('Creating initial metadata...')
for item in metadata.initial_metadata:
# check if the item exists
result = await self.execute_async(
sqlalchemy.select(metadata.Metadata).where(metadata.Metadata.key == item['key'])
)
row = result.first()
if row is None:
await self.execute_async(sqlalchemy.insert(metadata.Metadata).values(item))
async def write_space_model_providers(self):
space_models_gateway_api_url = self.ap.instance_config.data.get('space', {}).get(
'models_gateway_api_url', 'https://api.langbot.cloud/v1'
@@ -139,7 +91,7 @@ class PersistenceManager:
# =================================
async def _run_alembic_migrations(self):
"""Run Alembic-based migrations after legacy migrations complete."""
"""Run Alembic migrations for supported 4.x databases."""
from . import alembic_runner
engine = self.get_db_engine()
src/langbot/pkg/persistence/migration.py
-40
View File
@@ -1,40 +0,0 @@
from __future__ import annotations
import typing
import abc
from ..core import app
preregistered_db_migrations: list[typing.Type[DBMigration]] = []
def migration_class(number: int):
"""Migration class decorator"""
def wrapper(cls: typing.Type[DBMigration]) -> typing.Type[DBMigration]:
cls.number = number
preregistered_db_migrations.append(cls)
return cls
return wrapper
class DBMigration(abc.ABC):
"""Database migration"""
number: int
"""Migration number"""
def __init__(self, ap: app.Application):
self.ap = ap
@abc.abstractmethod
async def upgrade(self):
"""Upgrade"""
pass
@abc.abstractmethod
async def downgrade(self):
"""Downgrade"""
pass
src/langbot/pkg/persistence/migrations/README.md
-36
View File
@@ -1,36 +0,0 @@
# Legacy migrations (DEPRECATED — do not add new files here)
This directory holds the **legacy 3.x database migration system**
(`DBMigration` subclasses in `dbmXXX_*.py`, registered via
`@migration.migration_class(N)` and run from `pkg/persistence/mgr.py`).
**This system is frozen. Do not add new `dbmXXX_*.py` migrations.**
The chain is capped at version 25 (`required_database_version = 25` in
`pkg/utils/constants.py`). These files exist only to upgrade pre-existing
3.x databases up to the Alembic baseline (`0001_baseline`). Removing them
would break in-place upgrades from old installations, so they are kept
read-only.
## All new schema changes use Alembic
Migrations now live in `pkg/persistence/alembic/versions/`. To create one:
```bash
uv run python -m langbot.pkg.persistence.alembic_runner autogenerate "description of your change"
```
(requires `data/config.yaml` to exist). Review and edit the generated
script before committing — Alembic migrations run automatically on startup
and must be idempotent and guard against missing tables (the test suite
runs them against empty databases).
### Rules for Alembic revision ids
- Keep the revision id **≤ 32 characters** — PostgreSQL stores
`alembic_version.version_num` as `varchar(32)` and will raise
`StringDataRightTruncationError` on overflow.
- Guard every `op` call against a missing table / missing column
(`inspector.get_table_names()` / `inspector.get_columns()`); fresh
installs create the schema via `create_all()` and stamp the baseline,
so migrations may run against tables that already match or do not exist.
src/langbot/pkg/persistence/migrations/__init__.py
View File
src/langbot/pkg/persistence/migrations/dbm001_migrate_v3_config.py
-218
View File
@@ -1,218 +0,0 @@
from .. import migration
from copy import deepcopy
import uuid
import os
import sqlalchemy
import shutil
from ...config import manager as config_manager
from ...entity.persistence import (
model as persistence_model,
pipeline as persistence_pipeline,
bot as persistence_bot,
)
@migration.migration_class(1)
class DBMigrateV3Config(migration.DBMigration):
"""Migrate v3 config to v4 database"""
async def upgrade(self):
"""Upgrade"""
"""
Migrate all config files under data/config.
After migration, all previous config files are saved under data/legacy/config.
After migration, all config files under data/metadata/ are saved under data/legacy/metadata.
"""
if self.ap.provider_cfg is None:
return
# ======= Migrate model =======
# Only migrate the currently selected model
model_name = self.ap.provider_cfg.data.get('model', 'gpt-4o')
model_requester = 'openai-chat-completions'
model_requester_config = {}
model_api_keys = ['sk-proj-1234567890']
model_abilities = []
model_extra_args = {}
if os.path.exists('data/metadata/llm-models.json'):
_llm_model_meta = await config_manager.load_json_config('data/metadata/llm-models.json', completion=False)
for item in _llm_model_meta.data.get('list', []):
if item.get('name') == model_name:
if 'model_name' in item:
model_name = item['model_name']
if 'requester' in item:
model_requester = item['requester']
if 'token_mgr' in item:
_token_mgr = item['token_mgr']
if _token_mgr in self.ap.provider_cfg.data.get('keys', {}):
model_api_keys = self.ap.provider_cfg.data.get('keys', {})[_token_mgr]
if 'tool_call_supported' in item and item['tool_call_supported']:
model_abilities.append('func_call')
if 'vision_supported' in item and item['vision_supported']:
model_abilities.append('vision')
if (
model_requester in self.ap.provider_cfg.data.get('requester', {})
and 'args' in self.ap.provider_cfg.data.get('requester', {})[model_requester]
):
model_extra_args = self.ap.provider_cfg.data.get('requester', {})[model_requester]['args']
if model_requester in self.ap.provider_cfg.data.get('requester', {}):
model_requester_config = self.ap.provider_cfg.data.get('requester', {})[model_requester]
model_requester_config = {
'base_url': model_requester_config['base-url'],
'timeout': model_requester_config['timeout'],
}
break
model_uuid = str(uuid.uuid4())
llm_model_data = {
'uuid': model_uuid,
'name': model_name,
'description': '由 LangBot v3 迁移而来',
'requester': model_requester,
'requester_config': model_requester_config,
'api_keys': model_api_keys,
'abilities': model_abilities,
'extra_args': model_extra_args,
}
await self.ap.persistence_mgr.execute_async(
sqlalchemy.insert(persistence_model.LLMModel).values(**llm_model_data)
)
# ======= Migrate pipeline config =======
# Modify to default pipeline
default_pipeline = [
self.ap.persistence_mgr.serialize_model(persistence_pipeline.LegacyPipeline, pipeline)
for pipeline in (
await self.ap.persistence_mgr.execute_async(
sqlalchemy.select(persistence_pipeline.LegacyPipeline).where(
persistence_pipeline.LegacyPipeline.is_default == True
)
)
).all()
][0]
pipeline_uuid = str(uuid.uuid4())
pipeline_name = 'ChatPipeline'
if default_pipeline:
pipeline_name = default_pipeline['name']
pipeline_uuid = default_pipeline['uuid']
pipeline_config = default_pipeline['config']
# trigger
pipeline_config['trigger']['group-respond-rules'] = self.ap.pipeline_cfg.data['respond-rules']['default']
pipeline_config['trigger']['access-control'] = self.ap.pipeline_cfg.data['access-control']
pipeline_config['trigger']['ignore-rules'] = self.ap.pipeline_cfg.data['ignore-rules']
# safety
pipeline_config['safety']['content-filter'] = {
'scope': 'all',
'check-sensitive-words': self.ap.pipeline_cfg.data['check-sensitive-words'],
}
pipeline_config['safety']['rate-limit'] = {
'window-length': self.ap.pipeline_cfg.data['rate-limit']['fixwin']['default']['window-size'],
'limitation': self.ap.pipeline_cfg.data['rate-limit']['fixwin']['default']['limit'],
'strategy': self.ap.pipeline_cfg.data['rate-limit']['strategy'],
}
# output
pipeline_config['output']['long-text-processing'] = self.ap.platform_cfg.data['long-text-process']
pipeline_config['output']['force-delay'] = self.ap.platform_cfg.data['force-delay']
pipeline_config['output']['misc'] = {
'hide-exception': self.ap.platform_cfg.data['hide-exception-info'],
'quote-origin': self.ap.platform_cfg.data['quote-origin'],
'at-sender': self.ap.platform_cfg.data['at-sender'],
'track-function-calls': self.ap.platform_cfg.data['track-function-calls'],
}
default_pipeline['description'] = default_pipeline['description'] + ' [已迁移 LangBot v3 配置]'
default_pipeline['config'] = pipeline_config
default_pipeline.pop('created_at')
default_pipeline.pop('updated_at')
await self.ap.persistence_mgr.execute_async(
sqlalchemy.update(persistence_pipeline.LegacyPipeline)
.values(default_pipeline)
.where(persistence_pipeline.LegacyPipeline.uuid == default_pipeline['uuid'])
)
# ======= Migrate bot =======
# Only migrate enabled bots
for adapter in self.ap.platform_cfg.data.get('platform-adapters', []):
if not adapter.get('enable'):
continue
args = deepcopy(adapter)
args.pop('adapter')
args.pop('enable')
bot_data = {
'uuid': str(uuid.uuid4()),
'name': adapter.get('adapter'),
'description': '由 LangBot v3 迁移而来',
'adapter': adapter.get('adapter'),
'adapter_config': args,
'enable': True,
'use_pipeline_uuid': pipeline_uuid,
'use_pipeline_name': pipeline_name,
}
await self.ap.persistence_mgr.execute_async(sqlalchemy.insert(persistence_bot.Bot).values(**bot_data))
# ======= Migrate system settings =======
self.ap.instance_config.data['admins'] = self.ap.system_cfg.data['admin-sessions']
self.ap.instance_config.data['api']['port'] = self.ap.system_cfg.data['http-api']['port']
self.ap.instance_config.data['command'] = {
'prefix': self.ap.command_cfg.data['command-prefix'],
'enable': self.ap.command_cfg.data['command-enable']
if 'command-enable' in self.ap.command_cfg.data
else True,
'privilege': self.ap.command_cfg.data['privilege'],
}
self.ap.instance_config.data['concurrency']['pipeline'] = self.ap.system_cfg.data['pipeline-concurrency']
self.ap.instance_config.data['concurrency']['session'] = self.ap.system_cfg.data['session-concurrency'][
'default'
]
self.ap.instance_config.data['mcp'] = self.ap.provider_cfg.data['mcp']
self.ap.instance_config.data['proxy'] = self.ap.system_cfg.data['network-proxies']
await self.ap.instance_config.dump_config()
# ======= move files =======
# Migrate all config files under data/config
all_legacy_dir_name = [
'config',
# 'metadata',
'prompts',
'scenario',
]
def move_legacy_files(dir_name: str):
if not os.path.exists(f'data/legacy/{dir_name}'):
os.makedirs(f'data/legacy/{dir_name}')
if os.path.exists(f'data/{dir_name}'):
for file in os.listdir(f'data/{dir_name}'):
if file.endswith('.json'):
shutil.move(f'data/{dir_name}/{file}', f'data/legacy/{dir_name}/{file}')
os.rmdir(f'data/{dir_name}')
for dir_name in all_legacy_dir_name:
move_legacy_files(dir_name)
async def downgrade(self):
"""Downgrade"""
src/langbot/pkg/persistence/migrations/dbm002_combine_quote_msg_config.py
-55
View File
@@ -1,55 +0,0 @@
from .. import migration
import sqlalchemy
import json
@migration.migration_class(2)
class DBMigrateCombineQuoteMsgConfig(migration.DBMigration):
"""Combine quote message config"""
async def upgrade(self):
"""Upgrade"""
# Read all pipelines using raw SQL
result = await self.ap.persistence_mgr.execute_async(
sqlalchemy.text('SELECT uuid, config FROM legacy_pipelines')
)
pipelines = result.fetchall()
current_version = self.ap.ver_mgr.get_current_version()
for pipeline_row in pipelines:
uuid = pipeline_row[0]
config = json.loads(pipeline_row[1]) if isinstance(pipeline_row[1], str) else pipeline_row[1]
# Ensure 'trigger' exists
if 'trigger' not in config:
config['trigger'] = {}
# Ensure 'misc' exists in 'trigger'
if 'misc' not in config['trigger']:
config['trigger']['misc'] = {}
# Add 'combine-quote-message' if not exists
if 'combine-quote-message' not in config['trigger']['misc']:
config['trigger']['misc']['combine-quote-message'] = False
# Update using raw SQL with compatibility for both SQLite and PostgreSQL
if self.ap.persistence_mgr.db.name == 'postgresql':
await self.ap.persistence_mgr.execute_async(
sqlalchemy.text(
'UPDATE legacy_pipelines SET config = :config::jsonb, for_version = :for_version WHERE uuid = :uuid'
),
{'config': json.dumps(config), 'for_version': current_version, 'uuid': uuid},
)
else:
await self.ap.persistence_mgr.execute_async(
sqlalchemy.text(
'UPDATE legacy_pipelines SET config = :config, for_version = :for_version WHERE uuid = :uuid'
),
{'config': json.dumps(config), 'for_version': current_version, 'uuid': uuid},
)
async def downgrade(self):
"""Downgrade"""
pass
src/langbot/pkg/persistence/migrations/dbm003_n8n_config.py
-62
View File
@@ -1,62 +0,0 @@
from .. import migration
import sqlalchemy
import json
@migration.migration_class(3)
class DBMigrateN8nConfig(migration.DBMigration):
"""N8n config"""
async def upgrade(self):
"""Upgrade"""
# Read all pipelines using raw SQL
result = await self.ap.persistence_mgr.execute_async(
sqlalchemy.text('SELECT uuid, config FROM legacy_pipelines')
)
pipelines = result.fetchall()
current_version = self.ap.ver_mgr.get_current_version()
for pipeline_row in pipelines:
uuid = pipeline_row[0]
config = json.loads(pipeline_row[1]) if isinstance(pipeline_row[1], str) else pipeline_row[1]
# Ensure 'ai' exists
if 'ai' not in config:
config['ai'] = {}
# Add 'n8n-service-api' if not exists
if 'n8n-service-api' not in config['ai']:
config['ai']['n8n-service-api'] = {
'webhook-url': 'http://your-n8n-webhook-url',
'auth-type': 'none',
'basic-username': '',
'basic-password': '',
'jwt-secret': '',
'jwt-algorithm': 'HS256',
'header-name': '',
'header-value': '',
'timeout': 120,
'output-key': 'response',
}
# Update using raw SQL with compatibility for both SQLite and PostgreSQL
if self.ap.persistence_mgr.db.name == 'postgresql':
await self.ap.persistence_mgr.execute_async(
sqlalchemy.text(
'UPDATE legacy_pipelines SET config = :config::jsonb, for_version = :for_version WHERE uuid = :uuid'
),
{'config': json.dumps(config), 'for_version': current_version, 'uuid': uuid},
)
else:
await self.ap.persistence_mgr.execute_async(
sqlalchemy.text(
'UPDATE legacy_pipelines SET config = :config, for_version = :for_version WHERE uuid = :uuid'
),
{'config': json.dumps(config), 'for_version': current_version, 'uuid': uuid},
)
async def downgrade(self):
"""Downgrade"""
pass
src/langbot/pkg/persistence/migrations/dbm004_rag_kb_uuid.py
-53
View File
@@ -1,53 +0,0 @@
from .. import migration
import sqlalchemy
import json
@migration.migration_class(4)
class DBMigrateRAGKBUUID(migration.DBMigration):
"""RAG知识库UUID"""
async def upgrade(self):
"""升级"""
# Read all pipelines using raw SQL
result = await self.ap.persistence_mgr.execute_async(
sqlalchemy.text('SELECT uuid, config FROM legacy_pipelines')
)
pipelines = result.fetchall()
current_version = self.ap.ver_mgr.get_current_version()
for pipeline_row in pipelines:
uuid = pipeline_row[0]
config = json.loads(pipeline_row[1]) if isinstance(pipeline_row[1], str) else pipeline_row[1]
# Ensure nested structure exists
if 'ai' not in config:
config['ai'] = {}
if 'local-agent' not in config['ai']:
config['ai']['local-agent'] = {}
# Add 'knowledge-base' if not exists
if 'knowledge-base' not in config['ai']['local-agent']:
config['ai']['local-agent']['knowledge-base'] = ''
# Update using raw SQL with compatibility for both SQLite and PostgreSQL
if self.ap.persistence_mgr.db.name == 'postgresql':
await self.ap.persistence_mgr.execute_async(
sqlalchemy.text(
'UPDATE legacy_pipelines SET config = :config::jsonb, for_version = :for_version WHERE uuid = :uuid'
),
{'config': json.dumps(config), 'for_version': current_version, 'uuid': uuid},
)
else:
await self.ap.persistence_mgr.execute_async(
sqlalchemy.text(
'UPDATE legacy_pipelines SET config = :config, for_version = :for_version WHERE uuid = :uuid'
),
{'config': json.dumps(config), 'for_version': current_version, 'uuid': uuid},
)
async def downgrade(self):
"""降级"""
pass
src/langbot/pkg/persistence/migrations/dbm005_pipeline_remove_cot_config.py
-53
View File
@@ -1,53 +0,0 @@
from .. import migration
import sqlalchemy
import json
@migration.migration_class(5)
class DBMigratePipelineRemoveCotConfig(migration.DBMigration):
"""Pipeline remove cot config"""
async def upgrade(self):
"""Upgrade"""
# Read all pipelines using raw SQL
result = await self.ap.persistence_mgr.execute_async(
sqlalchemy.text('SELECT uuid, config FROM legacy_pipelines')
)
pipelines = result.fetchall()
current_version = self.ap.ver_mgr.get_current_version()
for pipeline_row in pipelines:
uuid = pipeline_row[0]
config = json.loads(pipeline_row[1]) if isinstance(pipeline_row[1], str) else pipeline_row[1]
# Ensure nested structure exists
if 'output' not in config:
config['output'] = {}
if 'misc' not in config['output']:
config['output']['misc'] = {}
# Add 'remove-think' if not exists
if 'remove-think' not in config['output']['misc']:
config['output']['misc']['remove-think'] = False
# Update using raw SQL with compatibility for both SQLite and PostgreSQL
if self.ap.persistence_mgr.db.name == 'postgresql':
await self.ap.persistence_mgr.execute_async(
sqlalchemy.text(
'UPDATE legacy_pipelines SET config = :config::jsonb, for_version = :for_version WHERE uuid = :uuid'
),
{'config': json.dumps(config), 'for_version': current_version, 'uuid': uuid},
)
else:
await self.ap.persistence_mgr.execute_async(
sqlalchemy.text(
'UPDATE legacy_pipelines SET config = :config, for_version = :for_version WHERE uuid = :uuid'
),
{'config': json.dumps(config), 'for_version': current_version, 'uuid': uuid},
)
async def downgrade(self):
"""Downgrade"""
pass
src/langbot/pkg/persistence/migrations/dbm006_langflow_api_config.py
-58
View File
@@ -1,58 +0,0 @@
from .. import migration
import sqlalchemy
import json
@migration.migration_class(6)
class DBMigrateLangflowApiConfig(migration.DBMigration):
"""Langflow API config"""
async def upgrade(self):
"""Upgrade"""
# Read all pipelines using raw SQL
result = await self.ap.persistence_mgr.execute_async(
sqlalchemy.text('SELECT uuid, config FROM legacy_pipelines')
)
pipelines = result.fetchall()
current_version = self.ap.ver_mgr.get_current_version()
for pipeline_row in pipelines:
uuid = pipeline_row[0]
config = json.loads(pipeline_row[1]) if isinstance(pipeline_row[1], str) else pipeline_row[1]
# Ensure 'ai' exists
if 'ai' not in config:
config['ai'] = {}
# Add 'langflow-api' if not exists
if 'langflow-api' not in config['ai']:
config['ai']['langflow-api'] = {
'base-url': 'http://localhost:7860',
'api-key': 'your-api-key',
'flow-id': 'your-flow-id',
'input-type': 'chat',
'output-type': 'chat',
'tweaks': '{}',
}
# Update using raw SQL with compatibility for both SQLite and PostgreSQL
if self.ap.persistence_mgr.db.name == 'postgresql':
await self.ap.persistence_mgr.execute_async(
sqlalchemy.text(
'UPDATE legacy_pipelines SET config = :config::jsonb, for_version = :for_version WHERE uuid = :uuid'
),
{'config': json.dumps(config), 'for_version': current_version, 'uuid': uuid},
)
else:
await self.ap.persistence_mgr.execute_async(
sqlalchemy.text(
'UPDATE legacy_pipelines SET config = :config, for_version = :for_version WHERE uuid = :uuid'
),
{'config': json.dumps(config), 'for_version': current_version, 'uuid': uuid},
)
async def downgrade(self):
"""Downgrade"""
pass
src/langbot/pkg/persistence/migrations/dbm007_plugin_install_source.py
-44
View File
@@ -1,44 +0,0 @@
import sqlalchemy
from .. import migration
@migration.migration_class(7)
class DBMigratePluginInstallSource(migration.DBMigration):
"""插件安装来源"""
async def upgrade(self):
"""升级"""
# 查询表结构获取所有列名（异步执行 SQL）
columns = []
if self.ap.persistence_mgr.db.name == 'postgresql':
result = await self.ap.persistence_mgr.execute_async(
sqlalchemy.text(
"SELECT column_name FROM information_schema.columns WHERE table_name = 'plugin_settings';"
)
)
all_result = result.fetchall()
columns = [row[0] for row in all_result]
else:
result = await self.ap.persistence_mgr.execute_async(sqlalchemy.text('PRAGMA table_info(plugin_settings);'))
all_result = result.fetchall()
columns = [row[1] for row in all_result]
# 检查并添加 install_source 列
if 'install_source' not in columns:
await self.ap.persistence_mgr.execute_async(
sqlalchemy.text(
"ALTER TABLE plugin_settings ADD COLUMN install_source VARCHAR(255) NOT NULL DEFAULT 'github'"
)
)
# 检查并添加 install_info 列
if 'install_info' not in columns:
await self.ap.persistence_mgr.execute_async(
sqlalchemy.text("ALTER TABLE plugin_settings ADD COLUMN install_info JSON NOT NULL DEFAULT '{}'")
)
async def downgrade(self):
"""降级"""
pass
src/langbot/pkg/persistence/migrations/dbm008_plugin_config.py
-22
View File
@@ -1,22 +0,0 @@
from .. import migration
@migration.migration_class(8)
class DBMigratePluginConfig(migration.DBMigration):
"""插件配置"""
async def upgrade(self):
"""升级"""
if 'plugin' not in self.ap.instance_config.data:
self.ap.instance_config.data['plugin'] = {
'runtime_ws_url': 'ws://langbot_plugin_runtime:5400/control/ws',
'enable_marketplace': True,
'cloud_service_url': 'https://space.langbot.app',
}
await self.ap.instance_config.dump_config()
async def downgrade(self):
"""降级"""
pass
src/langbot/pkg/persistence/migrations/dbm009_pipeline_extension_preferences.py
-20
View File
@@ -1,20 +0,0 @@
import sqlalchemy
from .. import migration
@migration.migration_class(9)
class DBMigratePipelineExtensionPreferences(migration.DBMigration):
"""Pipeline extension preferences"""
async def upgrade(self):
"""Upgrade"""
sql_text = sqlalchemy.text(
"ALTER TABLE legacy_pipelines ADD COLUMN extensions_preferences JSON NOT NULL DEFAULT '{}'"
)
await self.ap.persistence_mgr.execute_async(sql_text)
async def downgrade(self):
"""Downgrade"""
sql_text = sqlalchemy.text('ALTER TABLE legacy_pipelines DROP COLUMN extensions_preferences')
await self.ap.persistence_mgr.execute_async(sql_text)
src/langbot/pkg/persistence/migrations/dbm010_pipeline_multi_knowledge_base.py
-105
View File
@@ -1,105 +0,0 @@
from .. import migration
import sqlalchemy
import json
@migration.migration_class(10)
class DBMigratePipelineMultiKnowledgeBase(migration.DBMigration):
"""Pipeline support multiple knowledge base binding"""
async def upgrade(self):
"""Upgrade"""
# Read all pipelines using raw SQL
result = await self.ap.persistence_mgr.execute_async(
sqlalchemy.text('SELECT uuid, config FROM legacy_pipelines')
)
pipelines = result.fetchall()
current_version = self.ap.ver_mgr.get_current_version()
for pipeline_row in pipelines:
uuid = pipeline_row[0]
config = json.loads(pipeline_row[1]) if isinstance(pipeline_row[1], str) else pipeline_row[1]
# Convert knowledge-base from string to array
if 'ai' in config and 'local-agent' in config['ai']:
current_kb = config['ai']['local-agent'].get('knowledge-base', '')
# If it's already a list, skip
if isinstance(current_kb, list):
continue
# Convert string to list
if current_kb and current_kb != '__none__':
config['ai']['local-agent']['knowledge-bases'] = [current_kb]
else:
config['ai']['local-agent']['knowledge-bases'] = []
# Remove old field
if 'knowledge-base' in config['ai']['local-agent']:
del config['ai']['local-agent']['knowledge-base']
# Update using raw SQL with compatibility for both SQLite and PostgreSQL
if self.ap.persistence_mgr.db.name == 'postgresql':
await self.ap.persistence_mgr.execute_async(
sqlalchemy.text(
'UPDATE legacy_pipelines SET config = :config::jsonb, for_version = :for_version WHERE uuid = :uuid'
),
{'config': json.dumps(config), 'for_version': current_version, 'uuid': uuid},
)
else:
await self.ap.persistence_mgr.execute_async(
sqlalchemy.text(
'UPDATE legacy_pipelines SET config = :config, for_version = :for_version WHERE uuid = :uuid'
),
{'config': json.dumps(config), 'for_version': current_version, 'uuid': uuid},
)
async def downgrade(self):
"""Downgrade"""
# Read all pipelines using raw SQL
result = await self.ap.persistence_mgr.execute_async(
sqlalchemy.text('SELECT uuid, config FROM legacy_pipelines')
)
pipelines = result.fetchall()
current_version = self.ap.ver_mgr.get_current_version()
for pipeline_row in pipelines:
uuid = pipeline_row[0]
config = json.loads(pipeline_row[1]) if isinstance(pipeline_row[1], str) else pipeline_row[1]
# Convert knowledge-bases from array back to string
if 'ai' in config and 'local-agent' in config['ai']:
current_kbs = config['ai']['local-agent'].get('knowledge-bases', [])
# If it's already a string, skip
if isinstance(current_kbs, str):
continue
# Convert list to string (take first one or empty)
if current_kbs and len(current_kbs) > 0:
config['ai']['local-agent']['knowledge-base'] = current_kbs[0]
else:
config['ai']['local-agent']['knowledge-base'] = ''
# Remove new field
if 'knowledge-bases' in config['ai']['local-agent']:
del config['ai']['local-agent']['knowledge-bases']
# Update using raw SQL with compatibility for both SQLite and PostgreSQL
if self.ap.persistence_mgr.db.name == 'postgresql':
await self.ap.persistence_mgr.execute_async(
sqlalchemy.text(
'UPDATE legacy_pipelines SET config = :config::jsonb, for_version = :for_version WHERE uuid = :uuid'
),
{'config': json.dumps(config), 'for_version': current_version, 'uuid': uuid},
)
else:
await self.ap.persistence_mgr.execute_async(
sqlalchemy.text(
'UPDATE legacy_pipelines SET config = :config, for_version = :for_version WHERE uuid = :uuid'
),
{'config': json.dumps(config), 'for_version': current_version, 'uuid': uuid},
)
src/langbot/pkg/persistence/migrations/dbm011_dify_base_prompt_config.py
-55
View File
@@ -1,55 +0,0 @@
from .. import migration
import sqlalchemy
import json
@migration.migration_class(11)
class DBMigrateDifyApiConfig(migration.DBMigration):
"""Dify base prompt config"""
async def upgrade(self):
"""Upgrade"""
# Read all pipelines using raw SQL
result = await self.ap.persistence_mgr.execute_async(
sqlalchemy.text('SELECT uuid, config FROM legacy_pipelines')
)
pipelines = result.fetchall()
current_version = self.ap.ver_mgr.get_current_version()
for pipeline_row in pipelines:
uuid = pipeline_row[0]
config = json.loads(pipeline_row[1]) if isinstance(pipeline_row[1], str) else pipeline_row[1]
# Ensure nested structure exists
if 'ai' not in config:
config['ai'] = {}
if 'dify-service-api' not in config['ai']:
config['ai']['dify-service-api'] = {}
# Add 'base-prompt' if not exists
if 'base-prompt' not in config['ai']['dify-service-api']:
config['ai']['dify-service-api']['base-prompt'] = (
'When the file content is readable, please read the content of this file. When the file is an image, describe the content of this image.',
)
# Update using raw SQL with compatibility for both SQLite and PostgreSQL
if self.ap.persistence_mgr.db.name == 'postgresql':
await self.ap.persistence_mgr.execute_async(
sqlalchemy.text(
'UPDATE legacy_pipelines SET config = :config::jsonb, for_version = :for_version WHERE uuid = :uuid'
),
{'config': json.dumps(config), 'for_version': current_version, 'uuid': uuid},
)
else:
await self.ap.persistence_mgr.execute_async(
sqlalchemy.text(
'UPDATE legacy_pipelines SET config = :config, for_version = :for_version WHERE uuid = :uuid'
),
{'config': json.dumps(config), 'for_version': current_version, 'uuid': uuid},
)
async def downgrade(self):
"""Downgrade"""
pass
src/langbot/pkg/persistence/migrations/dbm012_pipeline_extensions_enable_all.py
-73
View File
@@ -1,73 +0,0 @@
from .. import migration
import sqlalchemy
import json
@migration.migration_class(12)
class DBMigratePipelineExtensionsEnableAll(migration.DBMigration):
"""Pipeline extensions enable all"""
async def upgrade(self):
"""Upgrade"""
# Read all pipelines using raw SQL
result = await self.ap.persistence_mgr.execute_async(
sqlalchemy.text('SELECT uuid, extensions_preferences FROM legacy_pipelines')
)
pipelines = result.fetchall()
current_version = self.ap.ver_mgr.get_current_version()
for pipeline_row in pipelines:
uuid = pipeline_row[0]
extensions_preferences = (
json.loads(pipeline_row[1]) if isinstance(pipeline_row[1], str) else pipeline_row[1]
)
# Ensure extensions_preferences is a dict
if extensions_preferences is None:
extensions_preferences = {}
# Add 'enable_all_plugins' if not exists
if 'enable_all_plugins' not in extensions_preferences:
if 'plugins' in extensions_preferences:
extensions_preferences['enable_all_plugins'] = False
else:
extensions_preferences['enable_all_plugins'] = True
extensions_preferences['plugins'] = []
# Add 'enable_all_mcp_servers' if not exists
if 'enable_all_mcp_servers' not in extensions_preferences:
if 'mcp_servers' in extensions_preferences:
extensions_preferences['enable_all_mcp_servers'] = False
else:
extensions_preferences['enable_all_mcp_servers'] = True
extensions_preferences['mcp_servers'] = []
# Update using raw SQL with compatibility for both SQLite and PostgreSQL
if self.ap.persistence_mgr.db.name == 'postgresql':
await self.ap.persistence_mgr.execute_async(
sqlalchemy.text(
'UPDATE legacy_pipelines SET extensions_preferences = :extensions_preferences::jsonb, for_version = :for_version WHERE uuid = :uuid'
),
{
'extensions_preferences': json.dumps(extensions_preferences),
'for_version': current_version,
'uuid': uuid,
},
)
else:
await self.ap.persistence_mgr.execute_async(
sqlalchemy.text(
'UPDATE legacy_pipelines SET extensions_preferences = :extensions_preferences, for_version = :for_version WHERE uuid = :uuid'
),
{
'extensions_preferences': json.dumps(extensions_preferences),
'for_version': current_version,
'uuid': uuid,
},
)
async def downgrade(self):
"""Downgrade"""
pass
src/langbot/pkg/persistence/migrations/dbm013_knowledge_base_updated_at.py
-49
View File
@@ -1,49 +0,0 @@
import sqlalchemy
from .. import migration
@migration.migration_class(13)
class DBMigrateKnowledgeBaseUpdatedAt(migration.DBMigration):
"""Add updated_at field to knowledge_bases table"""
async def upgrade(self):
"""Upgrade"""
# Get all column names from the table
columns = []
if self.ap.persistence_mgr.db.name == 'postgresql':
result = await self.ap.persistence_mgr.execute_async(
sqlalchemy.text(
"SELECT column_name FROM information_schema.columns WHERE table_name = 'knowledge_bases';"
)
)
all_result = result.fetchall()
columns = [row[0] for row in all_result]
else:
result = await self.ap.persistence_mgr.execute_async(sqlalchemy.text('PRAGMA table_info(knowledge_bases);'))
all_result = result.fetchall()
columns = [row[1] for row in all_result]
# Check and add updated_at column
if 'updated_at' not in columns:
if self.ap.persistence_mgr.db.name == 'postgresql':
await self.ap.persistence_mgr.execute_async(
sqlalchemy.text(
'ALTER TABLE knowledge_bases ADD COLUMN updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP'
)
)
else:
# SQLite doesn't support DEFAULT CURRENT_TIMESTAMP in ALTER TABLE
# Add column without default first
await self.ap.persistence_mgr.execute_async(
sqlalchemy.text('ALTER TABLE knowledge_bases ADD COLUMN updated_at DATETIME')
)
# Set initial updated_at values to created_at for existing records
await self.ap.persistence_mgr.execute_async(
sqlalchemy.text('UPDATE knowledge_bases SET updated_at = created_at WHERE updated_at IS NULL')
)
async def downgrade(self):
"""Downgrade"""
pass
src/langbot/pkg/persistence/migrations/dbm014_space_account_support.py
-94
View File
@@ -1,94 +0,0 @@
import sqlalchemy
from .. import migration
@migration.migration_class(14)
class DBMigrateSpaceAccountSupport(migration.DBMigration):
"""Add Space account support fields to users table"""
async def upgrade(self):
"""Upgrade"""
# Get all column names from the users table
columns = []
if self.ap.persistence_mgr.db.name == 'postgresql':
result = await self.ap.persistence_mgr.execute_async(
sqlalchemy.text("SELECT column_name FROM information_schema.columns WHERE table_name = 'users';")
)
all_result = result.fetchall()
columns = [row[0] for row in all_result]
else:
result = await self.ap.persistence_mgr.execute_async(sqlalchemy.text('PRAGMA table_info(users);'))
all_result = result.fetchall()
columns = [row[1] for row in all_result]
# Add account_type column
if 'account_type' not in columns:
if self.ap.persistence_mgr.db.name == 'postgresql':
await self.ap.persistence_mgr.execute_async(
sqlalchemy.text("ALTER TABLE users ADD COLUMN account_type VARCHAR(32) DEFAULT 'local' NOT NULL")
)
else:
await self.ap.persistence_mgr.execute_async(
sqlalchemy.text("ALTER TABLE users ADD COLUMN account_type VARCHAR(32) DEFAULT 'local' NOT NULL")
)
# Add space_account_uuid column
if 'space_account_uuid' not in columns:
if self.ap.persistence_mgr.db.name == 'postgresql':
await self.ap.persistence_mgr.execute_async(
sqlalchemy.text('ALTER TABLE users ADD COLUMN space_account_uuid VARCHAR(255)')
)
else:
await self.ap.persistence_mgr.execute_async(
sqlalchemy.text('ALTER TABLE users ADD COLUMN space_account_uuid VARCHAR(255)')
)
# Add space_access_token column
if 'space_access_token' not in columns:
if self.ap.persistence_mgr.db.name == 'postgresql':
await self.ap.persistence_mgr.execute_async(
sqlalchemy.text('ALTER TABLE users ADD COLUMN space_access_token TEXT')
)
else:
await self.ap.persistence_mgr.execute_async(
sqlalchemy.text('ALTER TABLE users ADD COLUMN space_access_token TEXT')
)
# Add space_refresh_token column
if 'space_refresh_token' not in columns:
if self.ap.persistence_mgr.db.name == 'postgresql':
await self.ap.persistence_mgr.execute_async(
sqlalchemy.text('ALTER TABLE users ADD COLUMN space_refresh_token TEXT')
)
else:
await self.ap.persistence_mgr.execute_async(
sqlalchemy.text('ALTER TABLE users ADD COLUMN space_refresh_token TEXT')
)
# Add space_access_token_expires_at column
if 'space_access_token_expires_at' not in columns:
if self.ap.persistence_mgr.db.name == 'postgresql':
await self.ap.persistence_mgr.execute_async(
sqlalchemy.text('ALTER TABLE users ADD COLUMN space_access_token_expires_at TIMESTAMP')
)
else:
await self.ap.persistence_mgr.execute_async(
sqlalchemy.text('ALTER TABLE users ADD COLUMN space_access_token_expires_at DATETIME')
)
# Add space_api_key column
if 'space_api_key' not in columns:
if self.ap.persistence_mgr.db.name == 'postgresql':
await self.ap.persistence_mgr.execute_async(
sqlalchemy.text('ALTER TABLE users ADD COLUMN space_api_key VARCHAR(255)')
)
else:
await self.ap.persistence_mgr.execute_async(
sqlalchemy.text('ALTER TABLE users ADD COLUMN space_api_key VARCHAR(255)')
)
async def downgrade(self):
"""Downgrade"""
pass
src/langbot/pkg/persistence/migrations/dbm015_model_source_tracking.py
-15
View File
@@ -1,15 +0,0 @@
from .. import migration
# this is a deprecated migration
@migration.migration_class(15)
class DBMigrateModelSourceTracking(migration.DBMigration):
"""Add source tracking fields to models tables for Space integration"""
async def upgrade(self):
"""Upgrade"""
pass
async def downgrade(self):
"""Downgrade"""
pass
src/langbot/pkg/persistence/migrations/dbm016_model_provider_refactor.py
-305
View File
@@ -1,305 +0,0 @@
import uuid as uuid_lib
import sqlalchemy
from .. import migration
@migration.migration_class(16)
class DBMigrateModelProviderRefactor(migration.DBMigration):
"""Refactor model structure: create providers from existing models and update references"""
async def upgrade(self):
"""Upgrade"""
# Step 1: Create model_providers table if not exists
await self._create_providers_table()
# Step 2: Migrate existing models to use providers
await self._migrate_llm_models()
await self._migrate_embedding_models()
# Step 3: Remove deprecated columns
await self._cleanup_columns()
async def _create_providers_table(self):
"""Create model_providers table"""
if self.ap.persistence_mgr.db.name == 'postgresql':
await self.ap.persistence_mgr.execute_async(
sqlalchemy.text("""
CREATE TABLE IF NOT EXISTS model_providers (
uuid VARCHAR(255) PRIMARY KEY,
name VARCHAR(255) NOT NULL,
requester VARCHAR(255) NOT NULL,
base_url VARCHAR(512) NOT NULL,
api_keys JSONB NOT NULL DEFAULT '[]',
created_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
updated_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP
)
""")
)
else:
await self.ap.persistence_mgr.execute_async(
sqlalchemy.text("""
CREATE TABLE IF NOT EXISTS model_providers (
uuid VARCHAR(255) PRIMARY KEY,
name VARCHAR(255) NOT NULL,
requester VARCHAR(255) NOT NULL,
base_url VARCHAR(512) NOT NULL,
api_keys JSON NOT NULL DEFAULT '[]',
created_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
updated_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP
)
""")
)
async def _migrate_llm_models(self):
"""Migrate LLM models to use providers"""
llm_columns = await self._get_columns('llm_models')
# Add provider_uuid column if not exists
if 'provider_uuid' not in llm_columns:
await self.ap.persistence_mgr.execute_async(
sqlalchemy.text('ALTER TABLE llm_models ADD COLUMN provider_uuid VARCHAR(255)')
)
# Add prefered_ranking column if not exists
if 'prefered_ranking' not in llm_columns:
await self.ap.persistence_mgr.execute_async(
sqlalchemy.text('ALTER TABLE llm_models ADD COLUMN prefered_ranking INTEGER NOT NULL DEFAULT 0')
)
# Only migrate if old columns exist
if 'requester' not in llm_columns:
return
# Get all LLM models with old structure
result = await self.ap.persistence_mgr.execute_async(
sqlalchemy.text('SELECT uuid, name, requester, requester_config, api_keys FROM llm_models')
)
models = result.fetchall()
# Create providers and update models
provider_cache = {} # (requester, base_url, api_keys_str) -> provider_uuid
for model in models:
model_uuid, model_name, requester, requester_config, api_keys = model
# Extract base_url from requester_config
base_url = ''
if requester_config:
if isinstance(requester_config, str):
import json
requester_config = json.loads(requester_config)
base_url = requester_config.get('base_url', '') or requester_config.get('base-url', '')
# Parse api_keys if it's a string
if isinstance(api_keys, str):
import json
try:
api_keys = json.loads(api_keys)
except Exception:
api_keys = []
if not api_keys:
api_keys = []
# Create cache key
api_keys_str = str(sorted(api_keys)) if api_keys else '[]'
cache_key = (requester, base_url, api_keys_str)
if cache_key in provider_cache:
provider_uuid = provider_cache[cache_key]
else:
# Create new provider
provider_uuid = str(uuid_lib.uuid4())
provider_name = f'{requester}'
if base_url:
# Extract domain for name
try:
from urllib.parse import urlparse
parsed = urlparse(base_url)
provider_name = parsed.netloc or requester
except Exception:
pass
import json
api_keys_json = json.dumps(api_keys) if api_keys else '[]'
await self.ap.persistence_mgr.execute_async(
sqlalchemy.text("""
INSERT INTO model_providers (uuid, name, requester, base_url, api_keys)
VALUES (:uuid, :name, :requester, :base_url, :api_keys)
"""),
{
'uuid': provider_uuid,
'name': provider_name,
'requester': requester,
'base_url': base_url,
'api_keys': api_keys_json,
},
)
provider_cache[cache_key] = provider_uuid
# Update model with provider_uuid
await self.ap.persistence_mgr.execute_async(
sqlalchemy.text('UPDATE llm_models SET provider_uuid = :provider_uuid WHERE uuid = :uuid'),
{'provider_uuid': provider_uuid, 'uuid': model_uuid},
)
async def _migrate_embedding_models(self):
"""Migrate embedding models to use providers"""
embedding_columns = await self._get_columns('embedding_models')
# Add provider_uuid column if not exists
if 'provider_uuid' not in embedding_columns:
await self.ap.persistence_mgr.execute_async(
sqlalchemy.text('ALTER TABLE embedding_models ADD COLUMN provider_uuid VARCHAR(255)')
)
# Add prefered_ranking column if not exists
if 'prefered_ranking' not in embedding_columns:
await self.ap.persistence_mgr.execute_async(
sqlalchemy.text('ALTER TABLE embedding_models ADD COLUMN prefered_ranking INTEGER NOT NULL DEFAULT 0')
)
# Only migrate if old columns exist
if 'requester' not in embedding_columns:
return
# Get all embedding models with old structure
result = await self.ap.persistence_mgr.execute_async(
sqlalchemy.text('SELECT uuid, name, requester, requester_config, api_keys FROM embedding_models')
)
models = result.fetchall()
# Get existing providers
provider_result = await self.ap.persistence_mgr.execute_async(
sqlalchemy.text('SELECT uuid, requester, base_url, api_keys FROM model_providers')
)
existing_providers = provider_result.fetchall()
provider_cache = {}
for p in existing_providers:
p_uuid, p_requester, p_base_url, p_api_keys = p
api_keys_str = str(sorted(p_api_keys)) if p_api_keys else '[]'
provider_cache[(p_requester, p_base_url, api_keys_str)] = p_uuid
for model in models:
model_uuid, model_name, requester, requester_config, api_keys = model
base_url = ''
if requester_config:
if isinstance(requester_config, str):
import json
requester_config = json.loads(requester_config)
base_url = requester_config.get('base_url', '') or requester_config.get('base-url', '')
# Parse api_keys if it's a string
if isinstance(api_keys, str):
import json
try:
api_keys = json.loads(api_keys)
except Exception:
api_keys = []
if not api_keys:
api_keys = []
api_keys_str = str(sorted(api_keys)) if api_keys else '[]'
cache_key = (requester, base_url, api_keys_str)
if cache_key in provider_cache:
provider_uuid = provider_cache[cache_key]
else:
provider_uuid = str(uuid_lib.uuid4())
provider_name = f'{requester}'
if base_url:
try:
from urllib.parse import urlparse
parsed = urlparse(base_url)
provider_name = parsed.netloc or requester
except Exception:
pass
import json
api_keys_json = json.dumps(api_keys) if api_keys else '[]'
await self.ap.persistence_mgr.execute_async(
sqlalchemy.text("""
INSERT INTO model_providers (uuid, name, requester, base_url, api_keys)
VALUES (:uuid, :name, :requester, :base_url, :api_keys)
"""),
{
'uuid': provider_uuid,
'name': provider_name,
'requester': requester,
'base_url': base_url,
'api_keys': api_keys_json,
},
)
provider_cache[cache_key] = provider_uuid
await self.ap.persistence_mgr.execute_async(
sqlalchemy.text('UPDATE embedding_models SET provider_uuid = :provider_uuid WHERE uuid = :uuid'),
{'provider_uuid': provider_uuid, 'uuid': model_uuid},
)
async def _cleanup_columns(self):
"""Remove deprecated columns from model tables"""
llm_columns = await self._get_columns('llm_models')
deprecated_llm_cols = ['requester', 'requester_config', 'api_keys', 'description', 'source', 'space_model_id']
for col in deprecated_llm_cols:
if col in llm_columns:
if self.ap.persistence_mgr.db.name == 'postgresql':
await self.ap.persistence_mgr.execute_async(
sqlalchemy.text(f'ALTER TABLE llm_models DROP COLUMN IF EXISTS {col}')
)
else:
await self.ap.persistence_mgr.execute_async(
sqlalchemy.text(f'ALTER TABLE llm_models DROP COLUMN {col}')
)
embedding_columns = await self._get_columns('embedding_models')
deprecated_embedding_cols = [
'requester',
'requester_config',
'api_keys',
'description',
'source',
'space_model_id',
]
for col in deprecated_embedding_cols:
if col in embedding_columns:
if self.ap.persistence_mgr.db.name == 'postgresql':
await self.ap.persistence_mgr.execute_async(
sqlalchemy.text(f'ALTER TABLE embedding_models DROP COLUMN IF EXISTS {col}')
)
else:
await self.ap.persistence_mgr.execute_async(
sqlalchemy.text(f'ALTER TABLE embedding_models DROP COLUMN {col}')
)
async def _get_columns(self, table_name: str) -> list:
"""Get column names for a table"""
if self.ap.persistence_mgr.db.name == 'postgresql':
result = await self.ap.persistence_mgr.execute_async(
sqlalchemy.text(
f"SELECT column_name FROM information_schema.columns WHERE table_name = '{table_name}';"
)
)
all_result = result.fetchall()
return [row[0] for row in all_result]
else:
result = await self.ap.persistence_mgr.execute_async(sqlalchemy.text(f'PRAGMA table_info({table_name});'))
all_result = result.fetchall()
return [row[1] for row in all_result]
async def downgrade(self):
"""Downgrade"""
pass
src/langbot/pkg/persistence/migrations/dbm017_move_cloud_service_url.py
-25
View File
@@ -1,25 +0,0 @@
from .. import migration
@migration.migration_class(17)
class MoveCloudServiceUrl(migration.DBMigration):
"""迁移云服务 URL 配置"""
async def upgrade(self):
"""升级"""
if 'space' not in self.ap.instance_config.data:
self.ap.instance_config.data['space'] = {
'url': 'https://space.langbot.app',
'models_gateway_api_url': 'https://api.langbot.cloud/v1',
'oauth_authorize_url': 'https://space.langbot.app/auth/authorize',
'disable_models_service': False,
}
if 'plugin' in self.ap.instance_config.data:
self.ap.instance_config.data['plugin'].pop('cloud_service_url', None)
await self.ap.instance_config.dump_config()
async def downgrade(self):
"""降级"""
pass
src/langbot/pkg/persistence/migrations/dbm018_add_emoji_support.py
-58
View File
@@ -1,58 +0,0 @@
import sqlalchemy
from .. import migration
@migration.migration_class(18)
class DBMigrateAddEmojiSupport(migration.DBMigration):
"""Add emoji field to knowledge_bases, external_knowledge_bases and legacy_pipelines tables"""
async def upgrade(self):
"""Upgrade"""
# Add emoji field to knowledge_bases
await self._add_emoji_to_table('knowledge_bases', '📚')
# Add emoji field to external_knowledge_bases
await self._add_emoji_to_table('external_knowledge_bases', '🔗')
# Add emoji field to legacy_pipelines
await self._add_emoji_to_table('legacy_pipelines', '⚙️')
async def _add_emoji_to_table(self, table_name: str, default_emoji: str):
"""Add emoji column to specified table if it doesn't exist"""
# Get all column names from the table
columns = []
if self.ap.persistence_mgr.db.name == 'postgresql':
result = await self.ap.persistence_mgr.execute_async(
sqlalchemy.text(
f"SELECT column_name FROM information_schema.columns WHERE table_name = '{table_name}';"
)
)
all_result = result.fetchall()
columns = [row[0] for row in all_result]
else:
result = await self.ap.persistence_mgr.execute_async(sqlalchemy.text(f'PRAGMA table_info({table_name});'))
all_result = result.fetchall()
columns = [row[1] for row in all_result]
# Check and add emoji column
if 'emoji' not in columns:
if self.ap.persistence_mgr.db.name == 'postgresql':
await self.ap.persistence_mgr.execute_async(
sqlalchemy.text(f"ALTER TABLE {table_name} ADD COLUMN emoji VARCHAR(10) DEFAULT '{default_emoji}'")
)
else:
# SQLite doesn't support DEFAULT with emoji directly in ALTER TABLE
# Add column without default first
await self.ap.persistence_mgr.execute_async(
sqlalchemy.text(f'ALTER TABLE {table_name} ADD COLUMN emoji VARCHAR(10)')
)
# Set default emoji value for existing records
await self.ap.persistence_mgr.execute_async(
sqlalchemy.text(f"UPDATE {table_name} SET emoji = '{default_emoji}' WHERE emoji IS NULL")
)
async def downgrade(self):
"""Downgrade"""
pass
src/langbot/pkg/persistence/migrations/dbm019_monitoring_message_role.py
-24
View File
@@ -1,24 +0,0 @@
import sqlalchemy
from .. import migration
@migration.migration_class(19)
class DBMigrateMonitoringMessageRole(migration.DBMigration):
"""Add role column to monitoring_messages table"""
async def upgrade(self):
"""Upgrade"""
try:
sql_text = sqlalchemy.text("ALTER TABLE monitoring_messages ADD COLUMN role VARCHAR(50) DEFAULT 'user'")
await self.ap.persistence_mgr.execute_async(sql_text)
except Exception:
# Column may already exist
pass
async def downgrade(self):
"""Downgrade"""
try:
sql_text = sqlalchemy.text('ALTER TABLE monitoring_messages DROP COLUMN role')
await self.ap.persistence_mgr.execute_async(sql_text)
except Exception:
pass
src/langbot/pkg/persistence/migrations/dbm020_knowledge_engine_plugin_architecture.py
-161
View File
@@ -1,161 +0,0 @@
import sqlalchemy
from .. import migration
@migration.migration_class(20)
class DBMigrateKnowledgeEnginePluginArchitecture(migration.DBMigration):
"""Migrate to unified Knowledge Engine plugin architecture.
Changes:
- Backup existing knowledge_bases data to knowledge_bases_backup
- Clear knowledge_bases table and add new plugin architecture columns
- Drop old columns (PostgreSQL only; SQLite leaves them unmapped)
- Preserve external_knowledge_bases table as-is for future migration
- Set rag_plugin_migration_needed flag in metadata if old data exists
"""
async def upgrade(self):
"""Upgrade"""
has_internal_data = await self._backup_knowledge_bases()
has_external_data = await self._check_external_knowledge_bases()
await self._clear_knowledge_bases()
await self._add_columns_to_knowledge_bases()
await self._drop_old_columns()
if has_internal_data or has_external_data:
await self._set_migration_flag()
async def _get_table_columns(self, table_name: str) -> list[str]:
"""Get column names from a table (works for both SQLite and PostgreSQL)."""
if self.ap.persistence_mgr.db.name == 'postgresql':
result = await self.ap.persistence_mgr.execute_async(
sqlalchemy.text(
'SELECT column_name FROM information_schema.columns WHERE table_name = :table_name;'
).bindparams(table_name=table_name)
)
return [row[0] for row in result.fetchall()]
else:
# SQLite PRAGMA does not support bind parameters; validate identifier.
if not table_name.isidentifier():
raise ValueError(f'Invalid table name: {table_name}')
result = await self.ap.persistence_mgr.execute_async(sqlalchemy.text(f'PRAGMA table_info({table_name});'))
return [row[1] for row in result.fetchall()]
async def _table_exists(self, table_name: str) -> bool:
"""Check if a table exists."""
if self.ap.persistence_mgr.db.name == 'postgresql':
result = await self.ap.persistence_mgr.execute_async(
sqlalchemy.text(
'SELECT EXISTS (SELECT FROM information_schema.tables WHERE table_name = :table_name);'
).bindparams(table_name=table_name)
)
return result.scalar()
else:
result = await self.ap.persistence_mgr.execute_async(
sqlalchemy.text("SELECT name FROM sqlite_master WHERE type='table' AND name=:table_name;").bindparams(
table_name=table_name
)
)
return result.first() is not None
async def _backup_knowledge_bases(self) -> bool:
"""Backup knowledge_bases data. Returns True if data was backed up."""
result = await self.ap.persistence_mgr.execute_async(sqlalchemy.text('SELECT COUNT(*) FROM knowledge_bases;'))
count = result.scalar()
if count == 0:
return False
# Drop backup table if it already exists (from a previous failed migration)
if await self._table_exists('knowledge_bases_backup'):
await self.ap.persistence_mgr.execute_async(sqlalchemy.text('DROP TABLE knowledge_bases_backup;'))
await self.ap.persistence_mgr.execute_async(
sqlalchemy.text('CREATE TABLE knowledge_bases_backup AS SELECT * FROM knowledge_bases;')
)
self.ap.logger.info(
'Backed up %d knowledge base(s) to knowledge_bases_backup table.',
count,
)
return True
async def _check_external_knowledge_bases(self) -> bool:
"""Check if external_knowledge_bases table exists and has data.
The table is preserved as-is (not dropped) for future migration.
"""
if not await self._table_exists('external_knowledge_bases'):
return False
result = await self.ap.persistence_mgr.execute_async(
sqlalchemy.text('SELECT COUNT(*) FROM external_knowledge_bases;')
)
count = result.scalar()
if count > 0:
self.ap.logger.info(
'Found %d external knowledge base(s) in external_knowledge_bases table. '
'Table preserved for future migration.',
count,
)
return count > 0
async def _clear_knowledge_bases(self):
"""Clear all rows from knowledge_bases table (preserve table structure)."""
await self.ap.persistence_mgr.execute_async(sqlalchemy.text('DELETE FROM knowledge_bases;'))
async def _add_columns_to_knowledge_bases(self):
"""Add new RAG plugin architecture columns to knowledge_bases table."""
columns = await self._get_table_columns('knowledge_bases')
new_columns = {
'knowledge_engine_plugin_id': 'VARCHAR',
'collection_id': 'VARCHAR',
'creation_settings': 'TEXT', # JSON stored as TEXT for SQLite compatibility
'retrieval_settings': 'TEXT',
}
for col_name, col_type in new_columns.items():
if col_name not in columns:
await self.ap.persistence_mgr.execute_async(
sqlalchemy.text(f'ALTER TABLE knowledge_bases ADD COLUMN {col_name} {col_type};')
)
async def _drop_old_columns(self):
"""Drop embedding_model_uuid and top_k columns (PostgreSQL only).
SQLite does not support DROP COLUMN in older versions, so we leave the
columns in place — the SQLAlchemy entity simply won't map them.
"""
if self.ap.persistence_mgr.db.name != 'postgresql':
return
columns = await self._get_table_columns('knowledge_bases')
if 'embedding_model_uuid' in columns:
await self.ap.persistence_mgr.execute_async(
sqlalchemy.text('ALTER TABLE knowledge_bases DROP COLUMN embedding_model_uuid;')
)
if 'top_k' in columns:
await self.ap.persistence_mgr.execute_async(
sqlalchemy.text('ALTER TABLE knowledge_bases DROP COLUMN top_k;')
)
async def _set_migration_flag(self):
"""Set rag_plugin_migration_needed flag in metadata table."""
# Check if the key already exists
result = await self.ap.persistence_mgr.execute_async(
sqlalchemy.text("SELECT value FROM metadata WHERE key = 'rag_plugin_migration_needed';")
)
row = result.first()
if row is not None:
await self.ap.persistence_mgr.execute_async(
sqlalchemy.text("UPDATE metadata SET value = 'true' WHERE key = 'rag_plugin_migration_needed';")
)
else:
await self.ap.persistence_mgr.execute_async(
sqlalchemy.text("INSERT INTO metadata (key, value) VALUES ('rag_plugin_migration_needed', 'true');")
)
self.ap.logger.info('Set rag_plugin_migration_needed=true in metadata.')
async def downgrade(self):
"""Downgrade"""
pass
src/langbot/pkg/persistence/migrations/dbm021_merge_exception_handling.py
-74
View File
@@ -1,74 +0,0 @@
from .. import migration
import sqlalchemy
import json
@migration.migration_class(21)
class DBMigrateMergeExceptionHandling(migration.DBMigration):
"""Merge hide-exception and block-failed-request-output into a single exception-handling select option,
and add failure-hint field.
Conversion logic:
- block-failed-request-output=true -> exception-handling: hide
- hide-exception=true -> exception-handling: show-hint
- hide-exception=false -> exception-handling: show-error
"""
async def upgrade(self):
"""Upgrade"""
result = await self.ap.persistence_mgr.execute_async(
sqlalchemy.text('SELECT uuid, config FROM legacy_pipelines')
)
pipelines = result.fetchall()
current_version = self.ap.ver_mgr.get_current_version()
for pipeline_row in pipelines:
uuid = pipeline_row[0]
config = json.loads(pipeline_row[1]) if isinstance(pipeline_row[1], str) else pipeline_row[1]
if 'output' not in config:
config['output'] = {}
if 'misc' not in config['output']:
config['output']['misc'] = {}
misc = config['output']['misc']
# Determine new exception-handling value from legacy fields
hide_exception = misc.get('hide-exception', True)
block_failed = misc.get('block-failed-request-output', False)
if block_failed:
exception_handling = 'hide'
elif hide_exception:
exception_handling = 'show-hint'
else:
exception_handling = 'show-error'
misc['exception-handling'] = exception_handling
# Add failure-hint with default value
misc['failure-hint'] = 'Request failed.'
# Remove legacy fields
misc.pop('hide-exception', None)
if self.ap.persistence_mgr.db.name == 'postgresql':
await self.ap.persistence_mgr.execute_async(
sqlalchemy.text(
'UPDATE legacy_pipelines SET config = :config::jsonb, for_version = :for_version WHERE uuid = :uuid'
),
{'config': json.dumps(config), 'for_version': current_version, 'uuid': uuid},
)
else:
await self.ap.persistence_mgr.execute_async(
sqlalchemy.text(
'UPDATE legacy_pipelines SET config = :config, for_version = :for_version WHERE uuid = :uuid'
),
{'config': json.dumps(config), 'for_version': current_version, 'uuid': uuid},
)
async def downgrade(self):
"""Downgrade"""
pass
src/langbot/pkg/persistence/migrations/dbm022_monitoring_user_name.py
-73
View File
@@ -1,73 +0,0 @@
import sqlalchemy
from .. import migration
@migration.migration_class(22)
class DBMigrateMonitoringUserId(migration.DBMigration):
"""Add user_id and user_name columns to monitoring_sessions table
This migration adds the missing user_id column and also ensures user_name
column exists (in case migration 21 failed or was skipped).
"""
async def _table_exists(self, table_name: str) -> bool:
"""Check if a table exists (works for both SQLite and PostgreSQL)."""
if self.ap.persistence_mgr.db.name == 'postgresql':
result = await self.ap.persistence_mgr.execute_async(
sqlalchemy.text(
'SELECT EXISTS (SELECT FROM information_schema.tables WHERE table_name = :table_name);'
).bindparams(table_name=table_name)
)
return bool(result.scalar())
else:
result = await self.ap.persistence_mgr.execute_async(
sqlalchemy.text("SELECT name FROM sqlite_master WHERE type='table' AND name=:table_name;").bindparams(
table_name=table_name
)
)
return result.first() is not None
async def _get_table_columns(self, table_name: str) -> list[str]:
"""Get column names from a table (works for both SQLite and PostgreSQL)."""
if self.ap.persistence_mgr.db.name == 'postgresql':
result = await self.ap.persistence_mgr.execute_async(
sqlalchemy.text(
'SELECT column_name FROM information_schema.columns WHERE table_name = :table_name;'
).bindparams(table_name=table_name)
)
return [row[0] for row in result.fetchall()]
else:
if not table_name.isidentifier():
raise ValueError(f'Invalid table name: {table_name}')
result = await self.ap.persistence_mgr.execute_async(sqlalchemy.text(f'PRAGMA table_info({table_name});'))
return [row[1] for row in result.fetchall()]
async def _add_column_if_not_exists(self, table_name: str, column_name: str, column_type: str):
"""Add a column to a table if it does not already exist."""
columns = await self._get_table_columns(table_name)
if column_name in columns:
self.ap.logger.debug('%s column already exists in %s.', column_name, table_name)
return
await self.ap.persistence_mgr.execute_async(
sqlalchemy.text(f'ALTER TABLE {table_name} ADD COLUMN {column_name} {column_type};')
)
self.ap.logger.info('Added %s column to %s table.', column_name, table_name)
async def upgrade(self):
# Check if monitoring_sessions table exists
if not await self._table_exists('monitoring_sessions'):
self.ap.logger.warning('monitoring_sessions table does not exist, skipping migration.')
return
# Add user_id column to monitoring_sessions table
await self._add_column_if_not_exists('monitoring_sessions', 'user_id', 'VARCHAR(255)')
# Add user_name column to monitoring_sessions table (in case migration 21 failed)
await self._add_column_if_not_exists('monitoring_sessions', 'user_name', 'VARCHAR(255)')
# Add user_name column to monitoring_messages table (in case migration 21 failed)
if await self._table_exists('monitoring_messages'):
await self._add_column_if_not_exists('monitoring_messages', 'user_name', 'VARCHAR(255)')
async def downgrade(self):
pass
src/langbot/pkg/persistence/migrations/dbm023_model_fallback_config.py
-102
View File
@@ -1,102 +0,0 @@
from .. import migration
import sqlalchemy
import json
@migration.migration_class(23)
class DBMigrateModelFallbackConfig(migration.DBMigration):
"""Convert model field from plain UUID string to object with primary/fallbacks"""
async def upgrade(self):
"""Upgrade"""
result = await self.ap.persistence_mgr.execute_async(
sqlalchemy.text('SELECT uuid, config FROM legacy_pipelines')
)
pipelines = result.fetchall()
current_version = self.ap.ver_mgr.get_current_version()
for pipeline_row in pipelines:
uuid = pipeline_row[0]
config = json.loads(pipeline_row[1]) if isinstance(pipeline_row[1], str) else pipeline_row[1]
if 'ai' not in config or 'local-agent' not in config['ai']:
continue
local_agent = config['ai']['local-agent']
changed = False
# Convert model from string to object
model_value = local_agent.get('model', '')
if isinstance(model_value, str):
local_agent['model'] = {
'primary': model_value,
'fallbacks': [],
}
changed = True
# Remove leftover fallback-models field if present
if 'fallback-models' in local_agent:
del local_agent['fallback-models']
changed = True
if not changed:
continue
# Update using raw SQL with compatibility for both SQLite and PostgreSQL
if self.ap.persistence_mgr.db.name == 'postgresql':
await self.ap.persistence_mgr.execute_async(
sqlalchemy.text(
'UPDATE legacy_pipelines SET config = :config::jsonb, for_version = :for_version WHERE uuid = :uuid'
),
{'config': json.dumps(config), 'for_version': current_version, 'uuid': uuid},
)
else:
await self.ap.persistence_mgr.execute_async(
sqlalchemy.text(
'UPDATE legacy_pipelines SET config = :config, for_version = :for_version WHERE uuid = :uuid'
),
{'config': json.dumps(config), 'for_version': current_version, 'uuid': uuid},
)
async def downgrade(self):
"""Downgrade"""
result = await self.ap.persistence_mgr.execute_async(
sqlalchemy.text('SELECT uuid, config FROM legacy_pipelines')
)
pipelines = result.fetchall()
current_version = self.ap.ver_mgr.get_current_version()
for pipeline_row in pipelines:
uuid = pipeline_row[0]
config = json.loads(pipeline_row[1]) if isinstance(pipeline_row[1], str) else pipeline_row[1]
if 'ai' not in config or 'local-agent' not in config['ai']:
continue
local_agent = config['ai']['local-agent']
# Convert model from object back to string
model_value = local_agent.get('model', '')
if isinstance(model_value, dict):
local_agent['model'] = model_value.get('primary', '')
else:
continue
# Update using raw SQL with compatibility for both SQLite and PostgreSQL
if self.ap.persistence_mgr.db.name == 'postgresql':
await self.ap.persistence_mgr.execute_async(
sqlalchemy.text(
'UPDATE legacy_pipelines SET config = :config::jsonb, for_version = :for_version WHERE uuid = :uuid'
),
{'config': json.dumps(config), 'for_version': current_version, 'uuid': uuid},
)
else:
await self.ap.persistence_mgr.execute_async(
sqlalchemy.text(
'UPDATE legacy_pipelines SET config = :config, for_version = :for_version WHERE uuid = :uuid'
),
{'config': json.dumps(config), 'for_version': current_version, 'uuid': uuid},
)
src/langbot/pkg/persistence/migrations/dbm024_wecombot_websocket_mode.py
-49
View File
@@ -1,49 +0,0 @@
from .. import migration
import sqlalchemy
import json
@migration.migration_class(24)
class DBMigrateWecomBotWebSocketMode(migration.DBMigration):
"""Add enable-webhook field to existing wecombot adapter configs.
Existing wecombot bots were all using webhook mode, so we set
enable-webhook=true to preserve their behavior after the new
WebSocket long connection mode is introduced as default.
"""
async def upgrade(self):
"""Upgrade"""
result = await self.ap.persistence_mgr.execute_async(
sqlalchemy.text("SELECT uuid, adapter_config FROM bots WHERE adapter = 'wecombot'")
)
bots = result.fetchall()
for bot_row in bots:
bot_uuid = bot_row[0]
adapter_config = json.loads(bot_row[1]) if isinstance(bot_row[1], str) else bot_row[1]
if 'enable-webhook' in adapter_config:
continue
# Determine mode based on existing config: if webhook fields are present, keep webhook mode
has_webhook_config = bool(
adapter_config.get('Token') and adapter_config.get('EncodingAESKey') and adapter_config.get('Corpid')
)
adapter_config['enable-webhook'] = has_webhook_config
if self.ap.persistence_mgr.db.name == 'postgresql':
await self.ap.persistence_mgr.execute_async(
sqlalchemy.text('UPDATE bots SET adapter_config = :config::jsonb WHERE uuid = :uuid'),
{'config': json.dumps(adapter_config), 'uuid': bot_uuid},
)
else:
await self.ap.persistence_mgr.execute_async(
sqlalchemy.text('UPDATE bots SET adapter_config = :config WHERE uuid = :uuid'),
{'config': json.dumps(adapter_config), 'uuid': bot_uuid},
)
async def downgrade(self):
"""Downgrade"""
pass
src/langbot/pkg/persistence/migrations/dbm025_bot_pipeline_routing_rules.py
-15
View File
@@ -1,15 +0,0 @@
import sqlalchemy
from .. import migration
@migration.migration_class(25)
class DBMigrateBotPipelineRoutingRules(migration.DBMigration):
"""Add pipeline_routing_rules column to bots table"""
async def upgrade(self):
sql_text = sqlalchemy.text("ALTER TABLE bots ADD COLUMN pipeline_routing_rules JSON NOT NULL DEFAULT '[]'")
await self.ap.persistence_mgr.execute_async(sql_text)
async def downgrade(self):
sql_text = sqlalchemy.text('ALTER TABLE bots DROP COLUMN pipeline_routing_rules')
await self.ap.persistence_mgr.execute_async(sql_text)
src/langbot/pkg/persistence/migrations/dbm026_monitoring_tool_calls.py
-17
View File
@@ -1,17 +0,0 @@
from langbot.pkg.entity.persistence import monitoring as persistence_monitoring
from .. import migration
@migration.migration_class(26)
class DBMigrateMonitoringToolCalls(migration.DBMigration):
"""Add monitoring_tool_calls table"""
async def upgrade(self):
"""Upgrade"""
async with self.ap.persistence_mgr.get_db_engine().begin() as conn:
await conn.run_sync(persistence_monitoring.MonitoringToolCall.__table__.create, checkfirst=True)
async def downgrade(self):
"""Downgrade"""
async with self.ap.persistence_mgr.get_db_engine().begin() as conn:
await conn.run_sync(persistence_monitoring.MonitoringToolCall.__table__.drop, checkfirst=True)
src/langbot/pkg/pipeline/extension_preferences.py
+156
View File
@@ -0,0 +1,156 @@
"""Validation and fail-closed reads for Pipeline extension preferences."""
from __future__ import annotations
import typing
_BOOLEAN_FIELDS = (
'enable_all_plugins',
'enable_all_mcp_servers',
'enable_all_skills',
'mcp_resource_agent_read_enabled',
)
def _valid_plugin_binding(value: typing.Any) -> bool:
return (
isinstance(value, dict)
and isinstance(value.get('author'), str)
and bool(value['author'])
and isinstance(value.get('name'), str)
and bool(value['name'])
)
def _valid_name(value: typing.Any) -> bool:
return isinstance(value, str) and bool(value)
def normalize_extension_preferences(value: typing.Any) -> dict[str, typing.Any]:
"""Return a safe runtime view of persisted extension preferences.
Missing fields in a valid object retain the established defaults. A
malformed root is treated as an empty allowlist with every extension
capability disabled.
"""
if not isinstance(value, dict):
return {
'enable_all_plugins': False,
'enable_all_mcp_servers': False,
'enable_all_skills': False,
'mcp_resource_agent_read_enabled': False,
'plugins': [],
'mcp_servers': [],
'skills': [],
'mcp_resources': [],
}
normalized = dict(value)
normalized['enable_all_plugins'] = value.get('enable_all_plugins', True) is True
normalized['enable_all_mcp_servers'] = value.get('enable_all_mcp_servers', True) is True
normalized['enable_all_skills'] = value.get('enable_all_skills', True) is True
normalized['mcp_resource_agent_read_enabled'] = (
value.get('mcp_resource_agent_read_enabled', True) is True
)
plugins = value.get('plugins', [])
plugins_are_valid = isinstance(plugins, list) and all(
_valid_plugin_binding(plugin) for plugin in plugins
)
normalized['plugins'] = list(plugins) if plugins_are_valid else []
if not plugins_are_valid:
normalized['enable_all_plugins'] = False
mcp_servers = value.get('mcp_servers', [])
mcp_servers_are_valid = isinstance(mcp_servers, list) and all(
_valid_name(server) for server in mcp_servers
)
normalized['mcp_servers'] = list(mcp_servers) if mcp_servers_are_valid else []
if not mcp_servers_are_valid:
normalized['enable_all_mcp_servers'] = False
skills = value.get('skills', [])
skills_are_valid = isinstance(skills, list) and all(_valid_name(skill) for skill in skills)
normalized['skills'] = list(skills) if skills_are_valid else []
if not skills_are_valid:
normalized['enable_all_skills'] = False
mcp_resources = value.get('mcp_resources', [])
mcp_resources_are_valid = isinstance(mcp_resources, list) and all(
isinstance(resource, dict) for resource in mcp_resources
)
normalized['mcp_resources'] = list(mcp_resources) if mcp_resources_are_valid else []
if not mcp_resources_are_valid:
normalized['mcp_resource_agent_read_enabled'] = False
return normalized
def validate_extension_preferences(
value: typing.Any,
*,
context: str = 'Pipeline extensions_preferences',
field_aliases: typing.Mapping[str, str] | None = None,
) -> dict[str, typing.Any]:
"""Reject extension preferences that cannot be consumed unambiguously."""
if not isinstance(value, dict):
raise ValueError(f'{context} must be an object')
for field_name in _BOOLEAN_FIELDS:
if field_name in value and not isinstance(value[field_name], bool):
raise ValueError(f"{context} field '{field_name}' must be a boolean")
aliases = field_aliases or {}
_validate_list_field(
value,
'plugins',
aliases.get('plugins', 'plugins'),
_valid_plugin_binding,
'a plugin object with non-empty author and name',
context,
)
_validate_list_field(
value,
'mcp_servers',
aliases.get('mcp_servers', 'mcp_servers'),
_valid_name,
'a non-empty string',
context,
)
_validate_list_field(
value,
'skills',
aliases.get('skills', 'skills'),
_valid_name,
'a non-empty string',
context,
)
_validate_list_field(
value,
'mcp_resources',
aliases.get('mcp_resources', 'mcp_resources'),
lambda item: isinstance(item, dict),
'an object',
context,
)
return value
def _validate_list_field(
value: dict[str, typing.Any],
field_name: str,
field_label: str,
item_validator: typing.Callable[[typing.Any], bool],
item_description: str,
context: str,
) -> None:
if field_name not in value:
return
items = value[field_name]
if not isinstance(items, list):
raise ValueError(f"{context} field '{field_label}' must be a list")
for index, item in enumerate(items):
if not item_validator(item):
raise ValueError(
f"{context} field '{field_label}[{index}]' must be {item_description}"
)
src/langbot/pkg/pipeline/pipelinemgr.py
+20 -15
View File
@@ -14,7 +14,8 @@ import langbot_plugin.api.entities.builtin.platform.events as platform_events
import langbot_plugin.api.entities.events as events
from ..utils import importutil
from .config_coercion import coerce_pipeline_config
from ..agent.runner.config_migration import ConfigMigration
from ..agent.runner.config_resolver import RunnerConfigResolver
from .extension_preferences import normalize_extension_preferences
import langbot_plugin.api.entities.builtin.provider.session as provider_session
import langbot_plugin.api.entities.builtin.pipeline.query as pipeline_query
@@ -92,14 +93,16 @@ class RuntimePipeline:
self.stage_containers = stage_containers
# Extract bound plugins and MCP servers from extensions_preferences
extensions_prefs = pipeline_entity.extensions_preferences or {}
self.enable_all_plugins = extensions_prefs.get('enable_all_plugins', True)
self.enable_all_mcp_servers = extensions_prefs.get('enable_all_mcp_servers', True)
extensions_prefs = normalize_extension_preferences(pipeline_entity.extensions_preferences)
self.enable_all_plugins = extensions_prefs.get('enable_all_plugins', True) is True
self.enable_all_mcp_servers = extensions_prefs.get('enable_all_mcp_servers', True) is True
pipeline_config = pipeline_entity.config or {}
runner_config: dict[str, typing.Any] = {}
runner_id = ConfigMigration.resolve_runner_id(pipeline_config) if isinstance(pipeline_config, dict) else None
runner_id = (
RunnerConfigResolver.resolve_runner_id(pipeline_config) if isinstance(pipeline_config, dict) else None
)
if runner_id:
resolved_runner_config = ConfigMigration.resolve_runner_config(pipeline_config, runner_id)
resolved_runner_config = RunnerConfigResolver.resolve_runner_config(pipeline_config, runner_id)
if isinstance(resolved_runner_config, dict):
runner_config = resolved_runner_config
@@ -107,24 +110,26 @@ class RuntimePipeline:
'mcp-resources',
extensions_prefs.get('mcp_resources', []),
)
self.mcp_resource_agent_read_enabled = runner_config.get(
'mcp-resource-agent-read-enabled',
extensions_prefs.get('mcp_resource_agent_read_enabled', True),
self.mcp_resource_agent_read_enabled = (
runner_config.get(
'mcp-resource-agent-read-enabled',
extensions_prefs.get('mcp_resource_agent_read_enabled', True),
)
is True
)
if self.enable_all_plugins:
# None indicates to use all available plugins
self.bound_plugins = None
else:
plugin_list = extensions_prefs.get('plugins', [])
self.bound_plugins = [f'{p["author"]}/{p["name"]}' for p in plugin_list] if plugin_list else []
plugin_list = extensions_prefs['plugins']
self.bound_plugins = [f'{p["author"]}/{p["name"]}' for p in plugin_list]
if self.enable_all_mcp_servers:
# None indicates to use all available MCP servers
self.bound_mcp_servers = None
else:
mcp_server_list = extensions_prefs.get('mcp_servers', [])
self.bound_mcp_servers = mcp_server_list if mcp_server_list else []
self.bound_mcp_servers = extensions_prefs['mcp_servers']
async def run(self, query: pipeline_query.Query):
query.pipeline_config = self.pipeline_entity.config
@@ -296,9 +301,9 @@ class RuntimePipeline:
# Get runner name from pipeline config
runner_name = None
if query.pipeline_config:
from ..agent.runner.config_migration import ConfigMigration
from ..agent.runner.config_resolver import RunnerConfigResolver
runner_name = ConfigMigration.resolve_runner_id(query.pipeline_config)
runner_name = RunnerConfigResolver.resolve_runner_id(query.pipeline_config)
# Record query start and store message_id
message_id = ''
src/langbot/pkg/pipeline/preproc/preproc.py
+48 -17
View File
@@ -11,8 +11,11 @@ import langbot_plugin.api.entities.builtin.pipeline.query as pipeline_query
import langbot_plugin.api.entities.builtin.platform.events as platform_events
from ...agent.runner.descriptor import AgentRunnerDescriptor
from ...agent.runner.config_migration import ConfigMigration
from ...agent.runner.config_resolver import RunnerConfigResolver
from ...agent.runner import config_schema
from ...agent.runner.resource_policy import ResourcePolicyProjector
from ...provider.tools.toolmgr import ToolManager
from ..extension_preferences import normalize_extension_preferences
DEFAULT_PROMPT_CONFIG = [
@@ -77,6 +80,28 @@ class PreProcessor(stage.PipelineStage):
self.ap.logger.warning(f'Fallback model {fallback_uuid} not found, skipping')
return valid_fallbacks
async def _resolve_host_tools(
self,
query: pipeline_query.Query,
runner_config: dict[str, typing.Any],
bound_plugins: list[str] | None,
bound_mcp_servers: list[str] | None,
include_mcp_resource_tools: bool,
) -> list[typing.Any]:
catalog = await self.ap.tool_mgr.get_resolved_tool_catalog(
bound_plugins,
bound_mcp_servers,
include_skill_authoring=True,
include_mcp_resource_tools=include_mcp_resource_tools,
)
policy = ResourcePolicyProjector.from_runner_config(
runner_config,
resolved_tool_names=ResourcePolicyProjector.extract_tool_names(catalog),
)
catalog = ResourcePolicyProjector.filter_tools(catalog, policy)
ToolManager.bind_query_tool_sources(query, catalog)
return ToolManager.tools_from_catalog(catalog)
def _runner_accepts_multimodal_input(self, descriptor: AgentRunnerDescriptor | None) -> bool:
if descriptor is None:
return True
@@ -136,9 +161,7 @@ class PreProcessor(stage.PipelineStage):
strict_thread=True,
)
except Exception as e:
self.ap.logger.warning(
f'Unable to load Transcript history view for conversation {conversation_uuid}: {e}'
)
self.ap.logger.warning(f'Unable to load Transcript history view for conversation {conversation_uuid}: {e}')
return None
return messages or None
@@ -168,14 +191,16 @@ class PreProcessor(stage.PipelineStage):
) -> entities.StageProcessResult:
"""Process"""
# Resolve runner ID from the current ai.runner.id shape.
runner_id = ConfigMigration.resolve_runner_id(query.pipeline_config)
runner_id = RunnerConfigResolver.resolve_runner_id(query.pipeline_config)
# Get runner config from ai.runner_config[runner_id].
runner_config = ConfigMigration.resolve_runner_config(query.pipeline_config, runner_id) if runner_id else {}
runner_config = (
RunnerConfigResolver.resolve_runner_config(query.pipeline_config, runner_id) if runner_id else {}
)
query.variables = query.variables or {}
bound_plugins = query.variables.get('_pipeline_bound_plugins', None)
bound_mcp_servers = query.variables.get('_pipeline_bound_mcp_servers', None)
include_mcp_resource_tools = query.variables.get('_pipeline_mcp_resource_agent_read_enabled', True)
include_mcp_resource_tools = query.variables.get('_pipeline_mcp_resource_agent_read_enabled', True) is True
descriptor = await self._get_runner_descriptor(runner_id, bound_plugins)
session = await self.ap.sess_mgr.get_session(query)
@@ -204,7 +229,7 @@ class PreProcessor(stage.PipelineStage):
# been idle for longer than the configured conversation expire time.
# The idle window is measured from the last preprocess/update time, not
# from the conversation creation time.
conversation_expire_time = ConfigMigration.get_expire_time(query.pipeline_config)
conversation_expire_time = RunnerConfigResolver.get_expire_time(query.pipeline_config)
now = datetime.datetime.now()
if conversation_expire_time is not None and conversation_expire_time > 0:
last_update_time = getattr(conversation, 'update_time', None) or getattr(conversation, 'create_time', None)
@@ -236,10 +261,12 @@ class PreProcessor(stage.PipelineStage):
query.use_llm_model_uuid = llm_model.model_entity.uuid
if uses_host_tools and 'func_call' in (llm_model.model_entity.abilities or []):
query.use_funcs = await self.ap.tool_mgr.get_all_tools(
query.use_funcs = await self._resolve_host_tools(
query,
runner_config,
bound_plugins,
bound_mcp_servers,
include_mcp_resource_tools=include_mcp_resource_tools,
include_mcp_resource_tools,
)
self.ap.logger.debug(f'Bound plugins: {bound_plugins}')
@@ -249,16 +276,20 @@ class PreProcessor(stage.PipelineStage):
# If primary model doesn't support func_call but fallback models exist,
# load tools anyway since fallback models may support them
if uses_host_tools and not query.use_funcs and query.variables.get('_fallback_model_uuids'):
query.use_funcs = await self.ap.tool_mgr.get_all_tools(
query.use_funcs = await self._resolve_host_tools(
query,
runner_config,
bound_plugins,
bound_mcp_servers,
include_mcp_resource_tools=include_mcp_resource_tools,
include_mcp_resource_tools,
)
elif uses_host_tools:
query.use_funcs = await self.ap.tool_mgr.get_all_tools(
query.use_funcs = await self._resolve_host_tools(
query,
runner_config,
bound_plugins,
bound_mcp_servers,
include_mcp_resource_tools=include_mcp_resource_tools,
include_mcp_resource_tools,
)
self.ap.logger.debug(f'Bound plugins: {bound_plugins}')
@@ -372,13 +403,13 @@ class PreProcessor(stage.PipelineStage):
if getattr(self.ap, 'skill_mgr', None) is not None:
pipeline_data = await self.ap.pipeline_service.get_pipeline(query.pipeline_uuid)
extensions_prefs = (pipeline_data or {}).get('extensions_preferences', {})
enable_all_skills = extensions_prefs.get('enable_all_skills', True)
extensions_prefs = normalize_extension_preferences((pipeline_data or {}).get('extensions_preferences'))
enable_all_skills = extensions_prefs['enable_all_skills']
if enable_all_skills:
bound_skills = None # None = all loaded skills are visible
else:
bound_skills = extensions_prefs.get('skills', [])
bound_skills = extensions_prefs['skills']
query.variables['_pipeline_bound_skills'] = bound_skills
src/langbot/pkg/pipeline/process/handlers/chat.py
+5 -3
View File
@@ -12,7 +12,7 @@ from ... import entities
from ... import plugin_diagnostics
import langbot_plugin.api.entities.events as events
from ....agent.runner.config_migration import ConfigMigration
from ....agent.runner.config_resolver import RunnerConfigResolver
from ....agent.runner import config_schema
from ....utils import constants, runner as runner_utils
from ....telemetry import features as telemetry_features
@@ -312,8 +312,10 @@ class ChatMessageHandler(handler.MessageHandler):
if prompt_config:
return prompt_config
runner_id = ConfigMigration.resolve_runner_id(query.pipeline_config)
runner_config = ConfigMigration.resolve_runner_config(query.pipeline_config, runner_id) if runner_id else {}
runner_id = RunnerConfigResolver.resolve_runner_id(query.pipeline_config)
runner_config = (
RunnerConfigResolver.resolve_runner_config(query.pipeline_config, runner_id) if runner_id else {}
)
bound_plugins = query.variables.get('_pipeline_bound_plugins', None)
descriptor = await self._get_runner_descriptor(runner_id, bound_plugins)
return config_schema.extract_prompt_config(descriptor, runner_config, DEFAULT_PROMPT_CONFIG)
src/langbot/pkg/platform/botmgr.py
+40 -16
View File
@@ -15,14 +15,15 @@ from ..discover import engine
from ..entity.persistence import bot as persistence_bot
from ..entity.errors import platform as platform_errors
from ..agent.runner.config_resolver import RunnerConfigResolver
from ..agent.runner.host_models import (
AgentBinding,
AgentEventEnvelope,
BindingScope,
DeliveryPolicy,
ResourcePolicy,
StatePolicy,
)
from ..agent.runner.resource_policy import ResourcePolicyProjector
from .logger import EventLogger
@@ -52,6 +53,8 @@ class SyntheticRouteTestAdapter:
'create_message_card',
'edit_message',
'delete_message',
'add_reaction',
'remove_reaction',
'forward_message',
'set_group_name',
'mute_member',
@@ -985,6 +988,7 @@ class RuntimeBot:
getattr(event, 'message_id', None) or getattr(event, 'feedback_id', None) or f'{event_type}:{uuid.uuid4()}'
)
target_type, target_id, target_metadata = self._infer_reply_target(event)
supported_apis = self._get_adapter_supported_apis(adapter)
conversation_id = None
if target_type and target_id:
@@ -1014,17 +1018,33 @@ class RuntimeBot:
**target_metadata,
},
supports_streaming=False,
supports_edit=False,
supports_reaction=False,
supports_edit='edit_message' in supported_apis,
supports_reaction=bool({'add_reaction', 'remove_reaction'} & set(supported_apis)),
platform_capabilities={
'adapter': adapter.__class__.__name__,
'event_type': event_type,
'supported_apis': supported_apis,
},
),
raw_ref=RawEventRef(ref_id=str(event_id), storage_key=None),
data=self._compact_event_data(event),
)
@staticmethod
def _get_adapter_supported_apis(
adapter: abstract_platform_adapter.AbstractMessagePlatformAdapter,
) -> list[str]:
get_supported_apis = getattr(adapter, 'get_supported_apis', None)
if not callable(get_supported_apis):
return []
try:
declared_apis = get_supported_apis()
except Exception:
return []
if not isinstance(declared_apis, (list, tuple, set)):
return []
return list(dict.fromkeys(api_name for api_name in declared_apis if isinstance(api_name, str) and api_name))
@staticmethod
def _agent_product_to_binding(
agent: dict[str, typing.Any],
@@ -1033,29 +1053,20 @@ class RuntimeBot:
bot_uuid: str,
) -> AgentBinding | None:
config = agent.get('config') if isinstance(agent, dict) else None
if not isinstance(config, dict):
if config is None:
return None
runner = config.get('runner')
runner_id = None
if isinstance(runner, dict):
runner_id = runner.get('id')
runner_id = runner_id or agent.get('component_ref')
_, runner_id, runner_config = RunnerConfigResolver.resolve_agent_runner_config(config)
if not runner_id:
return None
runner_config_map = config.get('runner_config')
runner_config = {}
if isinstance(runner_config_map, dict):
runner_config = runner_config_map.get(runner_id) or {}
return AgentBinding(
binding_id=f'bot:{bot_uuid}:{event_binding.get("id") or uuid.uuid4()}',
scope=BindingScope(scope_type='bot', scope_id=bot_uuid),
event_types=[event_type],
runner_id=runner_id,
runner_config=runner_config,
resource_policy=ResourcePolicy(),
resource_policy=ResourcePolicyProjector.from_runner_config(runner_config),
state_policy=StatePolicy(state_scopes=['conversation', 'actor', 'subject', 'runner']),
delivery_policy=DeliveryPolicy(enable_streaming=False, enable_reply=True),
enabled=True,
@@ -1262,7 +1273,20 @@ class RuntimeBot:
text=f'EBA event {event_type} target agent does not support this event: {target_uuid}',
)
binding = self._agent_product_to_binding(agent, event_binding, event_type, self.bot_entity.uuid)
try:
binding = self._agent_product_to_binding(agent, event_binding, event_type, self.bot_entity.uuid)
except Exception:
return await self._record_event_route_trace(
event_type=event_type,
status='failed',
level='error',
binding=event_binding,
target_type=target_type,
target_uuid=target_uuid,
failure_code='runner_failed',
reason='Agent configuration is invalid',
text=f'Failed to build Agent binding for EBA event {event_type}: {traceback.format_exc()}',
)
if binding is None:
return await self._record_event_route_trace(
event_type=event_type,

Some files were not shown because too many files have changed in this diff Show More