Compare commits

..

4 Commits

Author SHA1 Message Date
fdc310
60579f7b3a refactor(gewechat): remove redundant set_webhook_url and login logic
Remove the unnecessary set_webhook_url method and its caller in botmgr,
instead inject webhook_prefix via adapter config for self-assembly.
Remove login logic from run_async since login is handled elsewhere.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-22 06:15:16 +08:00
fdc310
c4d4c90930 fix: add set_webhook_url call in botmgr for auto callback URL
Pass webhook_full_url to adapters that implement set_webhook_url(),
built from api.webhook_prefix config + bot_uuid.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-22 05:57:32 +08:00
fdc310
1a3ef217d9 fix: resolve pydantic init error and auto-generate webhook callback URL
- Fix AttributeError by removing self.config assignment before super().__init__()
- Remove callback_base_url from adapter config (no longer needed)
- Add set_webhook_url() method, called by botmgr with auto-built URL
- Add gewechat to webhook URL display list in bot.py
- botmgr now passes webhook_prefix + bot_uuid to adapters via set_webhook_url()

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-22 05:52:50 +08:00
fdc310
6e9fcccd7d feat: migrate gewechat adapter to unified webhook architecture
Move gewechat adapter from legacy (self-hosted Quart server) to the
unified webhook entry point at /api/bots/{bot_uuid}. This removes the
need for a dedicated port and aligns with the modern adapter pattern.

Key changes:
- Add set_bot_uuid() and handle_unified_webhook() methods
- Remove self-hosted Quart server from run_async()
- Auto-generate callback URL from callback_base_url + bot_uuid
- Update constructor signature to (config, logger)
- Rename legacy gewechat.yaml to prevent name conflict

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-22 04:59:29 +08:00
644 changed files with 14553 additions and 110947 deletions

View File

@@ -1,5 +1,5 @@
name: 漏洞反馈
description: 【供中文用户】报错或漏洞请使用这个模板创建不使用此模板创建的异常、漏洞相关issue将被直接关闭。由于自己操作不当/不甚了解所用技术栈引起的网络连接问题恕无法解决,请勿提 issue。容器间网络连接问题参考文档 https://link.langbot.app/zh/docs/network
description: 【供中文用户】报错或漏洞请使用这个模板创建不使用此模板创建的异常、漏洞相关issue将被直接关闭。由于自己操作不当/不甚了解所用技术栈引起的网络连接问题恕无法解决,请勿提 issue。容器间网络连接问题参考文档 https://docs.langbot.app/zh/workshop/network-details.html
title: "[Bug]: "
labels: ["bug?"]
body:
@@ -10,15 +10,6 @@ body:
placeholder: 例如v3.3.0、CentOS x64 Python 3.10.3、Docker
validations:
required: true
- type: dropdown
attributes:
label: 部署版本
description: 请选择您使用的 LangBot 部署版本。
options:
- 社区版
- 云服务
validations:
required: true
- type: textarea
attributes:
label: 异常情况

View File

@@ -1,5 +1,5 @@
name: Bug report
description: Report bugs or vulnerabilities using this template. For container network connection issues, refer to the documentation https://link.langbot.app/en/docs/network
description: Report bugs or vulnerabilities using this template. For container network connection issues, refer to the documentation https://docs.langbot.app/en/workshop/network-details.html
title: "[Bug]: "
labels: ["bug?"]
body:
@@ -10,15 +10,6 @@ body:
placeholder: "For example: v3.3.0, CentOS x64 Python 3.10.3, Docker"
validations:
required: true
- type: dropdown
attributes:
label: Deployment version
description: Please select the LangBot deployment version you are using.
options:
- Community Edition
- Cloud Service
validations:
required: true
- type: textarea
attributes:
label: Exception

View File

@@ -43,10 +43,10 @@ jobs:
run: |
cd /tmp/langbot_build_web/web
npm install
npx vite build
npm run build
- name: Package Output
run: |
cp -r /tmp/langbot_build_web/web/dist ./web
cp -r /tmp/langbot_build_web/web/out ./web
- name: Upload Artifact
uses: actions/upload-artifact@v4
with:

View File

@@ -1,25 +0,0 @@
name: Check i18n Keys
on:
push:
branches:
- main
- master
jobs:
check-i18n:
name: Check i18n Key Consistency
runs-on: ubuntu-latest
permissions:
contents: read
steps:
- name: Checkout code
uses: actions/checkout@v4
- name: Setup Node.js
uses: actions/setup-node@v4
with:
node-version: '20'
- name: Check i18n keys against en-US reference
run: node web/scripts/check-i18n.mjs

View File

@@ -29,8 +29,8 @@ jobs:
npm install -g pnpm
pnpm install
pnpm build
mkdir -p ../src/langbot/web/dist
cp -r dist ../src/langbot/web/
mkdir -p ../src/langbot/web/out
cp -r out ../src/langbot/web/
- name: Install the latest version of uv
uses: astral-sh/setup-uv@v6

View File

@@ -4,25 +4,25 @@ on:
pull_request:
types: [opened, ready_for_review, synchronize]
paths:
- 'src/langbot/**'
- 'pkg/**'
- 'tests/**'
- '.github/workflows/run-tests.yml'
- 'pyproject.toml'
- 'uv.lock'
- 'run_tests.sh'
- 'scripts/test-*.sh'
push:
branches:
- master
- develop
- 'feat/**'
# No path filter on push: every push to the branches above runs the
# full unit-test suite. feat/** branches in particular must be tested
# on every push (they accumulate large changes before a PR exists).
paths:
- 'pkg/**'
- 'tests/**'
- '.github/workflows/run-tests.yml'
- 'pyproject.toml'
- 'run_tests.sh'
jobs:
test:
name: Unit Tests
name: Run Unit Tests
runs-on: ubuntu-latest
strategy:
matrix:
@@ -39,13 +39,28 @@ jobs:
python-version: ${{ matrix.python-version }}
- name: Install uv
uses: astral-sh/setup-uv@v4
run: |
curl -LsSf https://astral.sh/uv/install.sh | sh
echo "$HOME/.cargo/bin" >> $GITHUB_PATH
- name: Install dependencies
run: uv sync --dev
run: |
uv sync --dev
- name: Run unit + smoke tests
run: uv run pytest tests/unit_tests/ tests/smoke/ -q --tb=short
- name: Run unit tests
run: |
bash run_tests.sh
- name: Upload coverage to Codecov
if: matrix.python-version == '3.12'
uses: codecov/codecov-action@v5
with:
files: ./coverage.xml
flags: unit-tests
name: unit-tests-coverage
fail_ci_if_error: false
env:
CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}
- name: Test Summary
if: always()
@@ -54,79 +69,3 @@ jobs:
echo "" >> $GITHUB_STEP_SUMMARY
echo "Python Version: ${{ matrix.python-version }}" >> $GITHUB_STEP_SUMMARY
echo "Test Status: ${{ job.status }}" >> $GITHUB_STEP_SUMMARY
integration:
name: Fast Integration Tests
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@v4
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: '3.12'
- name: Install uv
uses: astral-sh/setup-uv@v4
- name: Install dependencies
run: uv sync --dev
- name: Run fast integration tests
run: uv run pytest tests/integration/ -m "not slow" -q --tb=short
- name: Integration Test Summary
if: always()
run: |
echo "## Integration Tests Results" >> $GITHUB_STEP_SUMMARY
echo "" >> $GITHUB_STEP_SUMMARY
echo "Test Status: ${{ job.status }}" >> $GITHUB_STEP_SUMMARY
coverage:
name: Coverage Gate
runs-on: ubuntu-latest
needs: [test, integration]
steps:
- name: Checkout code
uses: actions/checkout@v4
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: '3.12'
- name: Install uv
uses: astral-sh/setup-uv@v4
- name: Install dependencies
run: uv sync --dev
- name: Run coverage (unit + smoke)
run: |
uv run pytest tests/unit_tests/ tests/smoke/ \
--cov=langbot \
--cov-report=xml \
--cov-report=term-missing \
--cov-fail-under=18 \
-q --tb=short
- name: Upload coverage to Codecov
uses: codecov/codecov-action@v5
with:
files: ./coverage.xml
flags: unit-tests
name: coverage-report
fail_ci_if_error: false
env:
CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}
- name: Coverage Summary
if: always()
run: |
echo "## Coverage Results" >> $GITHUB_STEP_SUMMARY
echo "" >> $GITHUB_STEP_SUMMARY
echo "Threshold: 18%" >> $GITHUB_STEP_SUMMARY
echo "Status: ${{ job.status }}" >> $GITHUB_STEP_SUMMARY

View File

@@ -1,78 +0,0 @@
name: Test Migrations
on:
push:
branches:
- main
- master
- dev
paths:
- 'src/langbot/pkg/persistence/**'
- 'src/langbot/pkg/entity/persistence/**'
- 'tests/integration/persistence/**'
pull_request:
types: [opened, synchronize, reopened, ready_for_review]
paths:
- 'src/langbot/pkg/persistence/**'
- 'src/langbot/pkg/entity/persistence/**'
- 'tests/integration/persistence/**'
jobs:
test-migrations-sqlite:
name: Migrations (SQLite)
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@v4
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: '3.12'
- name: Install uv
uses: astral-sh/setup-uv@v4
- name: Install dependencies
run: uv sync --dev
- name: Run SQLite migration tests
run: uv run pytest tests/integration/persistence/test_migrations.py -q --tb=short
test-migrations-postgres:
name: Migrations (PostgreSQL)
runs-on: ubuntu-latest
services:
postgres:
image: postgres:16
env:
POSTGRES_USER: langbot
POSTGRES_PASSWORD: langbot
POSTGRES_DB: langbot_test
ports:
- 5432:5432
options: >-
--health-cmd="pg_isready -U langbot"
--health-interval=5s
--health-timeout=5s
--health-retries=5
steps:
- name: Checkout code
uses: actions/checkout@v4
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: '3.12'
- name: Install uv
uses: astral-sh/setup-uv@v4
- name: Install dependencies
run: uv sync --dev
- name: Run PostgreSQL migration tests
env:
TEST_POSTGRES_URL: postgresql+asyncpg://langbot:langbot@localhost:5432/langbot_test
run: uv run pytest tests/integration/persistence/test_migrations_postgres.py -q --tb=short

4
.gitignore vendored
View File

@@ -47,12 +47,8 @@ plugins.bak
coverage.xml
.coverage
src/langbot/web/
testsdk/
# Build artifacts
/dist
/build
*.egg-info
# Next.js build cache (legacy)
web/.next/

View File

@@ -9,14 +9,16 @@ repos:
# Run the formatter of backend.
- id: ruff-format
- repo: local
- repo: https://github.com/pre-commit/mirrors-prettier
rev: v3.1.0
hooks:
- id: prettier
name: prettier
entry: npx --prefix web prettier --write --ignore-unknown
language: system
types_or: [javascript, jsx, ts, tsx, css, scss]
additional_dependencies:
- prettier@3.1.0
- repo: local
hooks:
- id: lint-staged
name: lint-staged
entry: cd web && pnpm lint-staged

View File

@@ -70,7 +70,7 @@ Plugin Runtime automatically starts each installed plugin and interacts through
- type: must be a specific type, such as feat (new feature), fix (bug fix), docs (documentation), style (code style), refactor (refactoring), perf (performance optimization), etc.
- scope: the scope of the commit, such as the package name, the file name, the function name, the class name, the module name, etc.
- subject: the subject of the commit, such as the description of the commit, the reason for the commit, the impact of the commit, etc.
- LangBot uses [Alembic](https://alembic.sqlalchemy.org/) to manage database migrations, supporting both SQLite and PostgreSQL. Migration files are located in `src/langbot/pkg/persistence/alembic/versions/`. If you changed the definition of database entities (ORM models), generate a new migration script by running `uv run python -m langbot.pkg.persistence.alembic_runner autogenerate "description of your change"` in the project root (requires `data/config.yaml` to exist). Review and edit the generated script before committing. Migrations are executed automatically on LangBot startup. For data migrations (e.g. modifying JSON field content), you need to manually add the migration code in the generated script.
- If you changed the definition of database entities, please update the migration file in `src/langbot/pkg/persistence/migrations/` and update the constants.py file in `src/langbot/pkg/utils/constants.py` with the new migration number.
## Some Principles

View File

@@ -4,7 +4,7 @@ WORKDIR /app
COPY web ./web
RUN cd web && npm install && npx vite build
RUN cd web && npm install && npm run build
FROM python:3.12.7-slim
@@ -12,7 +12,7 @@ WORKDIR /app
COPY . .
COPY --from=node /app/web/dist ./web/dist
COPY --from=node /app/web/out ./web/out
RUN apt update \
&& apt install gcc -y \

View File

@@ -1,36 +0,0 @@
# LangBot Makefile
# Quick developer commands
.PHONY: test test-quick test-integration-fast test-coverage test-all-local lint
# Run all tests (full suite with coverage)
test:
bash run_tests.sh
# Quick self-test for developers (lint + unit + smoke, no real credentials needed)
test-quick:
bash scripts/test-quick.sh
# Fast integration tests (SQLite/API/Pipeline, no external services)
test-integration-fast:
bash scripts/test-integration-fast.sh
# Coverage gate (all tests, enforces minimum threshold)
test-coverage:
bash scripts/test-coverage.sh
# Full local quality gate (quick + integration + coverage)
test-all-local:
bash scripts/test-quick.sh
bash scripts/test-integration-fast.sh
bash scripts/test-coverage.sh
# Run linting only
lint:
ruff check src/langbot/ tests/
ruff format --check src/langbot/ tests/
# Fix linting issues
lint-fix:
ruff check --fix src/langbot/ tests/
ruff format src/langbot/ tests/

View File

@@ -19,9 +19,9 @@ English / [简体中文](README_CN.md) / [繁體中文](README_TW.md) / [日本
[![GitHub stars](https://img.shields.io/github/stars/langbot-app/LangBot?style=social)](https://github.com/langbot-app/LangBot/stargazers)
<a href="https://langbot.app">Website</a>
<a href="https://link.langbot.app/en/docs/features">Features</a>
<a href="https://link.langbot.app/en/docs/guide">Docs</a>
<a href="https://link.langbot.app/en/docs/api">API</a>
<a href="https://docs.langbot.app/en/insight/features">Features</a>
<a href="https://docs.langbot.app/en/insight/guide">Docs</a>
<a href="https://docs.langbot.app/en/tags/readme">API</a>
<a href="https://space.langbot.app/cloud">Cloud</a>
<a href="https://space.langbot.app">Plugin Market</a>
<a href="https://langbot.featurebase.app/roadmap">Roadmap</a>
@@ -45,9 +45,7 @@ LangBot is an **open-source, production-grade platform** for building AI-powered
- **Web Management Panel** — Configure, manage, and monitor your bots through an intuitive browser interface. No YAML editing required.
- **Multi-Pipeline Architecture** — Different bots for different scenarios, with comprehensive monitoring and exception handling.
[→ Learn more about all features](https://link.langbot.app/en/docs/features)
📍 Practical guides: [deploy a multi-platform AI bot in 5 minutes](https://blog.langbot.app/en/blog/deploy-ai-bot-in-5-minutes/), [connect DeepSeek to WeChat, Discord, and Telegram](https://blog.langbot.app/en/blog/connect-deepseek-to-wechat/), [run a Dify Agent in Discord, Telegram, and Slack](https://blog.langbot.app/en/blog/dify-agent-discord-telegram-slack/), and [build an n8n-powered chatbot](https://blog.langbot.app/en/blog/n8n-multi-platform-ai-chatbot/).
[→ Learn more about all features](https://docs.langbot.app/en/insight/features)
---
@@ -78,7 +76,7 @@ docker compose up -d
[![Deploy on Zeabur](https://zeabur.com/button.svg)](https://zeabur.com/en-US/templates/ZKTBDH)
[![Deploy on Railway](https://railway.com/button.svg)](https://railway.app/template/yRrAyL?referralCode=vogKPF)
**More options:** [Docker](https://link.langbot.app/en/docs/docker) · [Manual](https://link.langbot.app/en/docs/manual-deploy) · [BTPanel](https://link.langbot.app/en/docs/bt-panel) · [Kubernetes](./docker/README_K8S.md)
**More options:** [Docker](https://docs.langbot.app/en/deploy/langbot/docker) · [Manual](https://docs.langbot.app/en/deploy/langbot/manual) · [BTPanel](https://docs.langbot.app/en/deploy/langbot/one-click/bt) · [Kubernetes](./docker/README_K8S.md)
---
@@ -86,72 +84,68 @@ docker compose up -d
| Platform | Status | Notes |
|----------|--------|-------|
| Discord | ✅ | Official |
| Telegram | ✅ | Official |
| Slack | ✅ | Official |
| LINE | ✅ | Official |
| QQ | ✅ | Personal & Official API (Channel, DM, Group) |
| Discord | ✅ | |
| Telegram | ✅ | |
| Slack | ✅ | |
| LINE | ✅ | |
| QQ | ✅ | Personal & Official API |
| WeCom | ✅ | Enterprise WeChat, External CS, AI Bot |
| WeChat | ✅ | Personal & Official Account |
| Lark | ✅ | Official |
| DingTalk | ✅ | Official |
| KOOK | ✅ | Official |
| Lark | ✅ | |
| DingTalk | ✅ | |
| KOOK | ✅ | |
| Satori | ✅ | |
| Email | ✅ | Matrix, Satori |
| Matrix | ✅ | Supports multiple bridged platforms such as Signal, WhatsApp, Messenger, iMessage, Mattermost, Google Chat, IRC, XMPP, Zulip, and more |
---
## Supported LLMs & Integrations
| Provider | Type | Status |
| ----------------------------------------------------------------------------------------------------------------- | ------------ | ------ |
| [OpenAI](https://platform.openai.com/) | LLM | ✅ |
| [Anthropic](https://www.anthropic.com/) | LLM | ✅ |
| [DeepSeek](https://www.deepseek.com/) | LLM | ✅ |
| [Google Gemini](https://aistudio.google.com/prompts/new_chat) | LLM | ✅ |
| [xAI](https://x.ai/) | LLM | ✅ |
| [Moonshot](https://www.moonshot.cn/) | LLM | ✅ |
| [Zhipu AI](https://open.bigmodel.cn/) | LLM | ✅ |
| [Ollama](https://ollama.com/) | Local LLM | ✅ |
| [LM Studio](https://lmstudio.ai/) | Local LLM | ✅ |
| [Dify](https://dify.ai) | LLMOps | ✅ |
| [MCP](https://modelcontextprotocol.io/) | Protocol | ✅ |
| [SiliconFlow](https://siliconflow.cn/) | Gateway | ✅ |
| [Aliyun Bailian](https://bailian.console.aliyun.com/) | Gateway | ✅ |
| [Volc Engine Ark](https://console.volcengine.com/ark/region:ark+cn-beijing/model?vendor=Bytedance&view=LIST_VIEW) | Gateway | ✅ |
| [ModelScope](https://modelscope.cn/docs/model-service/API-Inference/intro) | Gateway | ✅ |
| [GiteeAI](https://ai.gitee.com/) | Gateway | ✅ |
| [CompShare](https://www.compshare.cn/?ytag=GPU_YY-gh_langbot) | GPU Platform | ✅ |
| [PPIO](https://ppinfra.com/user/register?invited_by=QJKFYD&utm_source=github_langbot) | GPU Platform | ✅ |
| [ShengSuanYun](https://www.shengsuanyun.com/?from=CH_KYIPP758) | GPU Platform | ✅ |
| [接口 AI](https://jiekou.ai/) | Gateway | ✅ |
| [302.AI](https://share.302.ai/SuTG99) | Gateway | ✅ |
| [Qiniu](https://www.qiniu.com/ai/agent) | Gateway | ✅ |
| Provider | Type | Status |
|----------|------|--------|
| [OpenAI](https://platform.openai.com/) | LLM | ✅ |
| [Anthropic](https://www.anthropic.com/) | LLM | ✅ |
| [DeepSeek](https://www.deepseek.com/) | LLM | ✅ |
| [Google Gemini](https://aistudio.google.com/prompts/new_chat) | LLM | ✅ |
| [xAI](https://x.ai/) | LLM | ✅ |
| [Moonshot](https://www.moonshot.cn/) | LLM | ✅ |
| [Zhipu AI](https://open.bigmodel.cn/) | LLM | ✅ |
| [Ollama](https://ollama.com/) | Local LLM | ✅ |
| [LM Studio](https://lmstudio.ai/) | Local LLM | ✅ |
| [Dify](https://dify.ai) | LLMOps | ✅ |
| [MCP](https://modelcontextprotocol.io/) | Protocol | ✅ |
| [SiliconFlow](https://siliconflow.cn/) | Gateway | ✅ |
| [Aliyun Bailian](https://bailian.console.aliyun.com/) | Gateway | ✅ |
| [Volc Engine Ark](https://console.volcengine.com/ark/region:ark+cn-beijing/model?vendor=Bytedance&view=LIST_VIEW) | Gateway | ✅ |
| [ModelScope](https://modelscope.cn/docs/model-service/API-Inference/intro) | Gateway | ✅ |
| [GiteeAI](https://ai.gitee.com/) | Gateway | ✅ |
| [CompShare](https://www.compshare.cn/?ytag=GPU_YY-gh_langbot) | GPU Platform | ✅ |
| [PPIO](https://ppinfra.com/user/register?invited_by=QJKFYD&utm_source=github_langbot) | GPU Platform | ✅ |
| [ShengSuanYun](https://www.shengsuanyun.com/?from=CH_KYIPP758) | GPU Platform | ✅ |
| [接口 AI](https://jiekou.ai/) | Gateway | ✅ |
| [302.AI](https://share.302.ai/SuTG99) | Gateway | ✅ |
[→ View all integrations](https://link.langbot.app/en/docs/features)
[→ View all integrations](https://docs.langbot.app/en/insight/features)
---
## Why LangBot?
| Use Case | How LangBot Helps |
| --------------------------- | ------------------------------------------------------------------------------------------ |
| **Customer Support** | Deploy AI agents to Slack/Discord/Telegram that answer questions using your knowledge base |
| **Internal Tools** | Connect n8n/Dify workflows to WeCom/DingTalk for automated business processes |
| **Community Management** | Moderate QQ/Discord groups with AI-powered content filtering and interaction |
| **Multi-Platform Presence** | One bot, all platforms. Manage from a single dashboard |
| Use Case | How LangBot Helps |
|----------|-------------------|
| **Customer Support** | Deploy AI agents to Slack/Discord/Telegram that answer questions using your knowledge base |
| **Internal Tools** | Connect n8n/Dify workflows to WeCom/DingTalk for automated business processes |
| **Community Management** | Moderate QQ/Discord groups with AI-powered content filtering and interaction |
| **Multi-Platform Presence** | One bot, all platforms. Manage from a single dashboard |
---
## Live Demo
**Try it now:** https://demo.langbot.dev/
- Email: `demo@langbot.app`
- Password: `langbot123456`
_Note: Public demo environment. Do not enter sensitive information._
*Note: Public demo environment. Do not enter sensitive information.*
---

View File

@@ -21,11 +21,11 @@
[![star](https://gitcode.com/RockChinQ/LangBot/star/badge.svg)](https://gitcode.com/RockChinQ/LangBot)
<a href="https://langbot.app">官网</a>
<a href="https://link.langbot.app/zh/docs/features">特性</a>
<a href="https://link.langbot.app/zh/docs/guide">文档</a>
<a href="https://link.langbot.app/zh/docs/api">API</a>
<a href="https://docs.langbot.app/zh/insight/features.html">特性</a>
<a href="https://docs.langbot.app/zh/insight/guide.html">文档</a>
<a href="https://docs.langbot.app/zh/tags/readme.html">API</a>
<a href="https://space.langbot.app/cloud">Cloud</a>
<a href="https://space.langbot.app">扩展市场</a>
<a href="https://space.langbot.app">插件市场</a>
<a href="https://langbot.featurebase.app/roadmap">路线图</a>
</div>
@@ -34,6 +34,8 @@
---
## 什么是 LangBot
LangBot 是一个**开源的生产级平台**,用于构建 AI 驱动的即时通信机器人。它将大语言模型LLM连接到各种聊天平台帮助你创建能够对话、执行任务、并集成到现有工作流程中的智能 Agent。
### 核心能力
@@ -41,13 +43,11 @@ LangBot 是一个**开源的生产级平台**,用于构建 AI 驱动的即时
- **AI 对话与 Agent** — 多轮对话、工具调用、多模态、流式输出。自带 RAG知识库深度集成 [Dify](https://dify.ai)、[Coze](https://coze.com)、[n8n](https://n8n.io)、[Langflow](https://langflow.org) 等 LLMOps 平台。
- **全平台支持** — 一套代码,覆盖 QQ、微信、企业微信、飞书、钉钉、Discord、Telegram、Slack、LINE、KOOK 等平台。
- **生产就绪** — 访问控制、限速、敏感词过滤、全面监控与异常处理,已被多家企业采用。
- **插件生态** — 数百个插件,跨进程的事件驱动架构,组件扩展,适配 [MCP 协议](https://modelcontextprotocol.io/)。
- **插件生态** — 数百个插件,事件驱动架构,组件扩展,适配 [MCP 协议](https://modelcontextprotocol.io/)。
- **Web 管理面板** — 通过浏览器直观地配置、管理和监控机器人,无需手动编辑配置文件。
- **多流水线架构** — 不同机器人用于不同场景,具备全面的监控和异常处理能力。
[→ 了解更多功能特性](https://link.langbot.app/zh/docs/features)
📍 实践指南:[5 分钟部署多平台 AI 机器人](https://blog.langbot.app/zh/blog/deploy-ai-bot-in-5-minutes/)、[将 DeepSeek 接入微信、企业微信与 Discord](https://blog.langbot.app/zh/blog/connect-deepseek-to-wechat/)、[让 Dify Agent 跑在 Discord、Telegram 和 Slack 上](https://blog.langbot.app/zh/blog/dify-agent-discord-telegram-slack/),以及[用 n8n 构建多平台 AI 聊天机器人](https://blog.langbot.app/zh/blog/n8n-multi-platform-ai-chatbot/)。
[→ 了解更多功能特性](https://docs.langbot.app/zh/insight/features.html)
---
@@ -78,7 +78,7 @@ docker compose up -d
[![Deploy on Zeabur](https://zeabur.com/button.svg)](https://zeabur.com/zh-CN/templates/ZKTBDH)
[![Deploy on Railway](https://railway.com/button.svg)](https://railway.app/template/yRrAyL?referralCode=vogKPF)
**更多方式:** [Docker](https://link.langbot.app/zh/docs/docker) · [手动部署](https://link.langbot.app/zh/docs/manual-deploy) · [宝塔面板](https://link.langbot.app/zh/docs/bt-panel) · [Kubernetes](./docker/README_K8S.md)
**更多方式:** [Docker](https://docs.langbot.app/zh/deploy/langbot/docker.html) · [手动部署](https://docs.langbot.app/zh/deploy/langbot/manual.html) · [宝塔面板](https://docs.langbot.app/zh/deploy/langbot/one-click/bt.html) · [Kubernetes](./docker/README_K8S.md)
---
@@ -89,16 +89,13 @@ docker compose up -d
| QQ | ✅ | 个人号、官方机器人(频道、私聊、群聊) |
| 微信 | ✅ | 个人微信、微信公众号 |
| 企业微信 | ✅ | 应用消息、对外客服、智能机器人 |
| 飞书 | ✅ | 官方 |
| 钉钉 | ✅ | 官方 |
| Satori | ✅ | |
| Discord | ✅ | 官方 |
| Telegram | ✅ | 官方 |
| Slack | ✅ | 官方 |
| LINE | ✅ | 官方 |
| KOOK | ✅ | 官方 |
| Email | ✅ | 只 Matrix、Satori |
| Matrix | ✅ | 支持多种桥接平台,如 Signal、WhatsApp、Messenger、iMessage、Mattermost、Google Chat、IRC、XMPP、Zulip 等 |
| 飞书 | ✅ | |
| 钉钉 | ✅ | |
| Discord | ✅ | |
| Telegram | ✅ | |
| Slack | ✅ | |
| LINE | ✅ | |
| KOOK | ✅ | |
---
@@ -129,9 +126,8 @@ docker compose up -d
| [302.AI](https://share.302.ai/SuTG99) | 聚合平台 | ✅ |
| [小马算力](https://www.tokenpony.cn/453z1) | 聚合平台 | ✅ |
| [百宝箱Tbox](https://www.tbox.cn/open) | 智能体平台 | ✅ |
| [七牛云Qiniu](https://www.qiniu.com/ai/agent) | 聚合平台 | ✅ |
[→ 查看完整集成列表](https://link.langbot.app/zh/docs/features)
[→ 查看完整集成列表](https://docs.langbot.app/zh/insight/features.html)
### TTS语音合成

View File

@@ -19,9 +19,9 @@
[![GitHub stars](https://img.shields.io/github/stars/langbot-app/LangBot?style=social)](https://github.com/langbot-app/LangBot/stargazers)
<a href="https://langbot.app">Inicio</a>
<a href="https://link.langbot.app/en/docs/features">Características</a>
<a href="https://link.langbot.app/en/docs/guide">Documentación</a>
<a href="https://link.langbot.app/en/docs/api">API</a>
<a href="https://docs.langbot.app/en/insight/features.html">Características</a>
<a href="https://docs.langbot.app/en/insight/guide.html">Documentación</a>
<a href="https://docs.langbot.app/en/tags/readme.html">API</a>
<a href="https://space.langbot.app">Mercado de Plugins</a>
<a href="https://langbot.featurebase.app/roadmap">Hoja de Ruta</a>
@@ -44,9 +44,7 @@ LangBot es una **plataforma de código abierto y grado de producción** para con
- **Panel de Gestión Web** — Configure, gestione y monitoree sus bots a través de una interfaz de navegador intuitiva. Sin necesidad de editar YAML.
- **Arquitectura Multi-Pipeline** — Diferentes bots para diferentes escenarios, con monitoreo completo y manejo de excepciones.
[→ Conocer más sobre todas las funcionalidades](https://link.langbot.app/en/docs/features)
📍 Guías prácticas: [desplegar un bot de IA multiplataforma en 5 minutos](https://blog.langbot.app/en/blog/deploy-ai-bot-in-5-minutes/), [conectar DeepSeek a WeChat, Discord y Telegram](https://blog.langbot.app/en/blog/connect-deepseek-to-wechat/), [ejecutar un Dify Agent en Discord, Telegram y Slack](https://blog.langbot.app/en/blog/dify-agent-discord-telegram-slack/) y [crear un chatbot con n8n](https://blog.langbot.app/en/blog/n8n-multi-platform-ai-chatbot/).
[→ Conocer más sobre todas las funcionalidades](https://docs.langbot.app/en/insight/features.html)
---
@@ -77,7 +75,7 @@ docker compose up -d
[![Deploy on Zeabur](https://zeabur.com/button.svg)](https://zeabur.com/en-US/templates/ZKTBDH)
[![Deploy on Railway](https://railway.com/button.svg)](https://railway.app/template/yRrAyL?referralCode=vogKPF)
**Más opciones:** [Docker](https://link.langbot.app/en/docs/docker) · [Manual](https://link.langbot.app/en/docs/manual-deploy) · [BTPanel](https://link.langbot.app/en/docs/bt-panel) · [Kubernetes](./docker/README_K8S.md)
**Más opciones:** [Docker](https://docs.langbot.app/en/deploy/langbot/docker.html) · [Manual](https://docs.langbot.app/en/deploy/langbot/manual.html) · [BTPanel](https://docs.langbot.app/en/deploy/langbot/one-click/bt.html) · [Kubernetes](./docker/README_K8S.md)
---
@@ -85,19 +83,17 @@ docker compose up -d
| Plataforma | Estado | Notas |
|----------|--------|-------|
| Discord | ✅ | Oficial |
| Telegram | ✅ | Oficial |
| Slack | ✅ | Oficial |
| LINE | ✅ | Oficial |
| QQ | ✅ | Personal y API Oficial (Canal, DM, Grupo) |
| Discord | ✅ | |
| Telegram | ✅ | |
| Slack | ✅ | |
| LINE | ✅ | |
| QQ | ✅ | Personal y API Oficial |
| WeCom | ✅ | WeChat Empresarial, CS Externo, AI Bot |
| WeChat | ✅ | Personal y Cuenta Oficial |
| Lark | ✅ | Oficial |
| DingTalk | ✅ | Oficial |
| KOOK | ✅ | Oficial |
| Lark | ✅ | |
| DingTalk | ✅ | |
| KOOK | ✅ | |
| Satori | ✅ | |
| Email | ✅ | Matrix, Satori |
| Matrix | ✅ | Admite varias plataformas puenteadas como Signal, WhatsApp, Messenger, iMessage, Mattermost, Google Chat, IRC, XMPP, Zulip y más |
---
@@ -126,9 +122,8 @@ docker compose up -d
| [ShengSuanYun](https://www.shengsuanyun.com/?from=CH_KYIPP758) | Plataforma GPU | ✅ |
| [接口 AI](https://jiekou.ai/) | Pasarela | ✅ |
| [302.AI](https://share.302.ai/SuTG99) | Pasarela | ✅ |
| [Qiniu](https://www.qiniu.com/ai/agent) | Pasarela | ✅ |
[→ Ver todas las integraciones](https://link.langbot.app/en/docs/features)
[→ Ver todas las integraciones](https://docs.langbot.app/en/insight/features.html)
---

View File

@@ -19,9 +19,9 @@
[![GitHub stars](https://img.shields.io/github/stars/langbot-app/LangBot?style=social)](https://github.com/langbot-app/LangBot/stargazers)
<a href="https://langbot.app">Accueil</a>
<a href="https://link.langbot.app/en/docs/features">Fonctionnalités</a>
<a href="https://link.langbot.app/en/docs/guide">Documentation</a>
<a href="https://link.langbot.app/en/docs/api">API</a>
<a href="https://docs.langbot.app/en/insight/features.html">Fonctionnalités</a>
<a href="https://docs.langbot.app/en/insight/guide.html">Documentation</a>
<a href="https://docs.langbot.app/en/tags/readme.html">API</a>
<a href="https://space.langbot.app">Marché des Plugins</a>
<a href="https://langbot.featurebase.app/roadmap">Feuille de Route</a>
@@ -44,9 +44,7 @@ LangBot est une **plateforme open-source de niveau production** pour créer des
- **Panneau de Gestion Web** — Configurez, gérez et surveillez vos bots via une interface navigateur intuitive. Aucune édition de YAML requise.
- **Architecture Multi-Pipeline** — Différents bots pour différents scénarios, avec surveillance complète et gestion des exceptions.
[→ En savoir plus sur toutes les fonctionnalités](https://link.langbot.app/en/docs/features)
📍 Guides pratiques : [déployer un bot IA multiplateforme en 5 minutes](https://blog.langbot.app/en/blog/deploy-ai-bot-in-5-minutes/), [connecter DeepSeek à WeChat, Discord et Telegram](https://blog.langbot.app/en/blog/connect-deepseek-to-wechat/), [exécuter un Dify Agent dans Discord, Telegram et Slack](https://blog.langbot.app/en/blog/dify-agent-discord-telegram-slack/) et [créer un chatbot avec n8n](https://blog.langbot.app/en/blog/n8n-multi-platform-ai-chatbot/).
[→ En savoir plus sur toutes les fonctionnalités](https://docs.langbot.app/en/insight/features.html)
---
@@ -77,7 +75,7 @@ docker compose up -d
[![Deploy on Zeabur](https://zeabur.com/button.svg)](https://zeabur.com/en-US/templates/ZKTBDH)
[![Deploy on Railway](https://railway.com/button.svg)](https://railway.app/template/yRrAyL?referralCode=vogKPF)
**Plus d'options :** [Docker](https://link.langbot.app/en/docs/docker) · [Manuel](https://link.langbot.app/en/docs/manual-deploy) · [BTPanel](https://link.langbot.app/en/docs/bt-panel) · [Kubernetes](./docker/README_K8S.md)
**Plus d'options :** [Docker](https://docs.langbot.app/en/deploy/langbot/docker.html) · [Manuel](https://docs.langbot.app/en/deploy/langbot/manual.html) · [BTPanel](https://docs.langbot.app/en/deploy/langbot/one-click/bt.html) · [Kubernetes](./docker/README_K8S.md)
---
@@ -85,19 +83,17 @@ docker compose up -d
| Plateforme | Statut | Notes |
|----------|--------|-------|
| Discord | ✅ | Officiel |
| Telegram | ✅ | Officiel |
| Slack | ✅ | Officiel |
| LINE | ✅ | Officiel |
| QQ | ✅ | Personnel & API Officielle (Canal, DM, Groupe) |
| Discord | ✅ | |
| Telegram | ✅ | |
| Slack | ✅ | |
| LINE | ✅ | |
| QQ | ✅ | Personnel & API Officielle |
| WeCom | ✅ | WeChat Entreprise, CS Externe, AI Bot |
| WeChat | ✅ | Personnel & Compte Officiel |
| Lark | ✅ | Officiel |
| DingTalk | ✅ | Officiel |
| KOOK | ✅ | Officiel |
| Lark | ✅ | |
| DingTalk | ✅ | |
| KOOK | ✅ | |
| Satori | ✅ | |
| Email | ✅ | Matrix, Satori |
| Matrix | ✅ | Prend en charge plusieurs plateformes via ponts, comme Signal, WhatsApp, Messenger, iMessage, Mattermost, Google Chat, IRC, XMPP, Zulip, etc. |
---
@@ -126,9 +122,8 @@ docker compose up -d
| [CompShare](https://www.compshare.cn/?ytag=GPU_YY-gh_langbot) | Plateforme GPU | ✅ |
| [PPIO](https://ppinfra.com/user/register?invited_by=QJKFYD&utm_source=github_langbot) | Plateforme GPU | ✅ |
| [ShengSuanYun](https://www.shengsuanyun.com/?from=CH_KYIPP758) | Plateforme GPU | ✅ |
| [Qiniu](https://www.qiniu.com/ai/agent) | Passerelle | ✅ |
[→ Voir toutes les intégrations](https://link.langbot.app/en/docs/features)
[→ Voir toutes les intégrations](https://docs.langbot.app/en/insight/features.html)
---

View File

@@ -19,9 +19,9 @@
[![GitHub stars](https://img.shields.io/github/stars/langbot-app/LangBot?style=social)](https://github.com/langbot-app/LangBot/stargazers)
<a href="https://langbot.app">ホーム</a>
<a href="https://link.langbot.app/ja/docs/features">機能</a>
<a href="https://link.langbot.app/ja/docs/guide">ドキュメント</a>
<a href="https://link.langbot.app/ja/docs/api">API</a>
<a href="https://docs.langbot.app/ja/insight/features.html">機能</a>
<a href="https://docs.langbot.app/ja/insight/guide.html">ドキュメント</a>
<a href="https://docs.langbot.app/ja/tags/readme.html">API</a>
<a href="https://space.langbot.app">プラグインマーケット</a>
<a href="https://langbot.featurebase.app/roadmap">ロードマップ</a>
@@ -44,9 +44,7 @@ LangBot は、AI搭載のインスタントメッセージングボットを構
- **Web管理パネル** — 直感的なブラウザインターフェースからボットの設定、管理、監視が可能。YAML編集は不要。
- **マルチパイプラインアーキテクチャ** — 異なるシナリオに異なるボットを配置し、包括的な監視と例外処理を実現。
[→ すべての機能について詳しく見る](https://link.langbot.app/ja/docs/features)
📍 実践ガイド: [5分でマルチプラットフォームAIボットをデプロイ](https://blog.langbot.app/en/blog/deploy-ai-bot-in-5-minutes/)、[DeepSeekをWeChat・Discord・Telegramに接続](https://blog.langbot.app/en/blog/connect-deepseek-to-wechat/)、[Dify AgentをDiscord・Telegram・Slackで動かす](https://blog.langbot.app/en/blog/dify-agent-discord-telegram-slack/)、[n8n連携チャットボットを構築](https://blog.langbot.app/en/blog/n8n-multi-platform-ai-chatbot/)。
[→ すべての機能について詳しく見る](https://docs.langbot.app/ja/insight/features.html)
---
@@ -77,7 +75,7 @@ docker compose up -d
[![Deploy on Zeabur](https://zeabur.com/button.svg)](https://zeabur.com/en-US/templates/ZKTBDH)
[![Deploy on Railway](https://railway.com/button.svg)](https://railway.app/template/yRrAyL?referralCode=vogKPF)
**その他:** [Docker](https://link.langbot.app/en/docs/docker) · [手動デプロイ](https://link.langbot.app/en/docs/manual-deploy) · [BTPanel](https://link.langbot.app/en/docs/bt-panel) · [Kubernetes](./docker/README_K8S.md)
**その他:** [Docker](https://docs.langbot.app/en/deploy/langbot/docker.html) · [手動デプロイ](https://docs.langbot.app/en/deploy/langbot/manual.html) · [BTPanel](https://docs.langbot.app/en/deploy/langbot/one-click/bt.html) · [Kubernetes](./docker/README_K8S.md)
---
@@ -85,19 +83,17 @@ docker compose up -d
| プラットフォーム | ステータス | 備考 |
|----------|--------|-------|
| Discord | ✅ | 公式 |
| Telegram | ✅ | 公式 |
| Slack | ✅ | 公式 |
| LINE | ✅ | 公式 |
| QQ | ✅ | 個人公式APIチャンネル・DM・グループ |
| Discord | ✅ | |
| Telegram | ✅ | |
| Slack | ✅ | |
| LINE | ✅ | |
| QQ | ✅ | 個人 & 公式API |
| WeCom | ✅ | 企業WeChat、外部CS、AIボット |
| WeChat | ✅ | 個人公式アカウント |
| Lark | ✅ | 公式 |
| DingTalk | ✅ | 公式 |
| KOOK | ✅ | 公式 |
| WeChat | ✅ | 個人 & 公式アカウント |
| Lark | ✅ | |
| DingTalk | ✅ | |
| KOOK | ✅ | |
| Satori | ✅ | |
| Email | ✅ | Matrix、Satori |
| Matrix | ✅ | Signal、WhatsApp、Messenger、iMessage、Mattermost、Google Chat、IRC、XMPP、Zulip など複数のブリッジ先プラットフォームに対応 |
---
@@ -126,9 +122,8 @@ docker compose up -d
| [ShengSuanYun](https://www.shengsuanyun.com/?from=CH_KYIPP758) | GPUプラットフォーム | ✅ |
| [接口 AI](https://jiekou.ai/) | ゲートウェイ | ✅ |
| [302.AI](https://share.302.ai/SuTG99) | ゲートウェイ | ✅ |
| [Qiniu](https://www.qiniu.com/ai/agent) | ゲートウェイ | ✅ |
[→ すべての統合を表示](https://link.langbot.app/en/docs/features)
[→ すべての統合を表示](https://docs.langbot.app/en/insight/features.html)
---

View File

@@ -19,9 +19,9 @@
[![GitHub stars](https://img.shields.io/github/stars/langbot-app/LangBot?style=social)](https://github.com/langbot-app/LangBot/stargazers)
<a href="https://langbot.app">홈</a>
<a href="https://link.langbot.app/en/docs/features">기능</a>
<a href="https://link.langbot.app/en/docs/guide">문서</a>
<a href="https://link.langbot.app/en/docs/api">API</a>
<a href="https://docs.langbot.app/en/insight/features.html">기능</a>
<a href="https://docs.langbot.app/en/insight/guide.html">문서</a>
<a href="https://docs.langbot.app/en/tags/readme.html">API</a>
<a href="https://space.langbot.app">플러그인 마켓</a>
<a href="https://langbot.featurebase.app/roadmap">로드맵</a>
@@ -44,9 +44,7 @@ LangBot은 AI 기반 인스턴트 메시징 봇을 구축하기 위한 **오픈
- **웹 관리 패널** — 직관적인 브라우저 인터페이스로 봇을 구성, 관리 및 모니터링. YAML 편집 불필요.
- **멀티 파이프라인 아키텍처** — 다양한 시나리오에 맞는 다양한 봇 구성, 종합 모니터링 및 예외 처리.
[→ 모든 기능 자세히 보기](https://link.langbot.app/en/docs/features)
📍 실전 가이드: [5분 만에 멀티 플랫폼 AI 봇 배포하기](https://blog.langbot.app/en/blog/deploy-ai-bot-in-5-minutes/), [DeepSeek를 WeChat, Discord, Telegram에 연결하기](https://blog.langbot.app/en/blog/connect-deepseek-to-wechat/), [Dify Agent를 Discord, Telegram, Slack에서 실행하기](https://blog.langbot.app/en/blog/dify-agent-discord-telegram-slack/), [n8n 기반 챗봇 만들기](https://blog.langbot.app/en/blog/n8n-multi-platform-ai-chatbot/).
[→ 모든 기능 자세히 보기](https://docs.langbot.app/en/insight/features.html)
---
@@ -77,7 +75,7 @@ docker compose up -d
[![Deploy on Zeabur](https://zeabur.com/button.svg)](https://zeabur.com/en-US/templates/ZKTBDH)
[![Deploy on Railway](https://railway.com/button.svg)](https://railway.app/template/yRrAyL?referralCode=vogKPF)
**더 많은 옵션:** [Docker](https://link.langbot.app/en/docs/docker) · [수동 배포](https://link.langbot.app/en/docs/manual-deploy) · [BTPanel](https://link.langbot.app/en/docs/bt-panel) · [Kubernetes](./docker/README_K8S.md)
**더 많은 옵션:** [Docker](https://docs.langbot.app/en/deploy/langbot/docker.html) · [수동 배포](https://docs.langbot.app/en/deploy/langbot/manual.html) · [BTPanel](https://docs.langbot.app/en/deploy/langbot/one-click/bt.html) · [Kubernetes](./docker/README_K8S.md)
---
@@ -85,19 +83,17 @@ docker compose up -d
| 플랫폼 | 상태 | 비고 |
|--------|------|------|
| Discord | ✅ | 공식 |
| Telegram | ✅ | 공식 |
| Slack | ✅ | 공식 |
| LINE | ✅ | 공식 |
| QQ | ✅ | 개인 및 공식 API (채널, DM, 그룹) |
| Discord | ✅ | |
| Telegram | ✅ | |
| Slack | ✅ | |
| LINE | ✅ | |
| QQ | ✅ | 개인 및 공식 API |
| WeCom | ✅ | 기업 WeChat, 외부 CS, AI Bot |
| WeChat | ✅ | 개인 및 공식 계정 |
| Lark | ✅ | 공식 |
| DingTalk | ✅ | 공식 |
| KOOK | ✅ | 공식 |
| Lark | ✅ | |
| DingTalk | ✅ | |
| KOOK | ✅ | |
| Satori | ✅ | |
| Email | ✅ | Matrix, Satori |
| Matrix | ✅ | Signal, WhatsApp, Messenger, iMessage, Mattermost, Google Chat, IRC, XMPP, Zulip 등 여러 브리지 플랫폼 지원 |
---
@@ -126,9 +122,8 @@ docker compose up -d
| [ShengSuanYun](https://www.shengsuanyun.com/?from=CH_KYIPP758) | GPU 플랫폼 | ✅ |
| [接口 AI](https://jiekou.ai/) | 게이트웨이 | ✅ |
| [302.AI](https://share.302.ai/SuTG99) | 게이트웨이 | ✅ |
| [Qiniu](https://www.qiniu.com/ai/agent) | 게이트웨이 | ✅ |
[→ 모든 통합 보기](https://link.langbot.app/en/docs/features)
[→ 모든 통합 보기](https://docs.langbot.app/en/insight/features.html)
---

View File

@@ -19,9 +19,9 @@
[![GitHub stars](https://img.shields.io/github/stars/langbot-app/LangBot?style=social)](https://github.com/langbot-app/LangBot/stargazers)
<a href="https://langbot.app">Главная</a>
<a href="https://link.langbot.app/en/docs/features">Возможности</a>
<a href="https://link.langbot.app/en/docs/guide">Документация</a>
<a href="https://link.langbot.app/en/docs/api">API</a>
<a href="https://docs.langbot.app/en/insight/features.html">Возможности</a>
<a href="https://docs.langbot.app/en/insight/guide.html">Документация</a>
<a href="https://docs.langbot.app/en/tags/readme.html">API</a>
<a href="https://space.langbot.app">Магазин плагинов</a>
<a href="https://langbot.featurebase.app/roadmap">Дорожная карта</a>
@@ -44,9 +44,7 @@ LangBot — это **платформа с открытым исходным к
- **Веб-панель управления** — Настраивайте, управляйте и мониторьте ваших ботов через интуитивный браузерный интерфейс. Ручное редактирование YAML не требуется.
- **Мультиконвейерная архитектура** — Разные боты для разных сценариев с комплексным мониторингом и обработкой исключений.
[→ Подробнее обо всех возможностях](https://link.langbot.app/en/docs/features)
📍 Практические руководства: [развернуть мультиплатформенного ИИ-бота за 5 минут](https://blog.langbot.app/en/blog/deploy-ai-bot-in-5-minutes/), [подключить DeepSeek к WeChat, Discord и Telegram](https://blog.langbot.app/en/blog/connect-deepseek-to-wechat/), [запустить Dify Agent в Discord, Telegram и Slack](https://blog.langbot.app/en/blog/dify-agent-discord-telegram-slack/) и [создать чат-бота на n8n](https://blog.langbot.app/en/blog/n8n-multi-platform-ai-chatbot/).
[→ Подробнее обо всех возможностях](https://docs.langbot.app/en/insight/features.html)
---
@@ -77,7 +75,7 @@ docker compose up -d
[![Deploy on Zeabur](https://zeabur.com/button.svg)](https://zeabur.com/en-US/templates/ZKTBDH)
[![Deploy on Railway](https://railway.com/button.svg)](https://railway.app/template/yRrAyL?referralCode=vogKPF)
**Другие варианты:** [Docker](https://link.langbot.app/en/docs/docker) · [Ручная установка](https://link.langbot.app/en/docs/manual-deploy) · [BTPanel](https://link.langbot.app/en/docs/bt-panel) · [Kubernetes](./docker/README_K8S.md)
**Другие варианты:** [Docker](https://docs.langbot.app/en/deploy/langbot/docker.html) · [Ручная установка](https://docs.langbot.app/en/deploy/langbot/manual.html) · [BTPanel](https://docs.langbot.app/en/deploy/langbot/one-click/bt.html) · [Kubernetes](./docker/README_K8S.md)
---
@@ -85,19 +83,17 @@ docker compose up -d
| Платформа | Статус | Примечания |
|-----------|--------|------------|
| Discord | ✅ | Официальный |
| Telegram | ✅ | Официальный |
| Slack | ✅ | Официальный |
| LINE | ✅ | Официальный |
| QQ | ✅ | Личный и официальный API (Канал, ЛС, Группа) |
| Discord | ✅ | |
| Telegram | ✅ | |
| Slack | ✅ | |
| LINE | ✅ | |
| QQ | ✅ | Личный и официальный API |
| WeCom | ✅ | Корпоративный WeChat, внешний CS, AI-бот |
| WeChat | ✅ | Личный и официальный аккаунт |
| Lark | ✅ | Официальный |
| DingTalk | ✅ | Официальный |
| KOOK | ✅ | Официальный |
| Lark | ✅ | |
| DingTalk | ✅ | |
| KOOK | ✅ | |
| Satori | ✅ | |
| Email | ✅ | Matrix, Satori |
| Matrix | ✅ | Поддерживает несколько платформ через мосты, включая Signal, WhatsApp, Messenger, iMessage, Mattermost, Google Chat, IRC, XMPP, Zulip и другие |
---
@@ -126,9 +122,8 @@ docker compose up -d
| [CompShare](https://www.compshare.cn/?ytag=GPU_YY-gh_langbot) | Платформа GPU | ✅ |
| [PPIO](https://ppinfra.com/user/register?invited_by=QJKFYD&utm_source=github_langbot) | Платформа GPU | ✅ |
| [ShengSuanYun](https://www.shengsuanyun.com/?from=CH_KYIPP758) | Платформа GPU | ✅ |
| [Qiniu](https://www.qiniu.com/ai/agent) | Шлюз | ✅ |
[→ Смотреть все интеграции](https://link.langbot.app/en/docs/features)
[→ Смотреть все интеграции](https://docs.langbot.app/en/insight/features.html)
---

View File

@@ -21,9 +21,9 @@
[![star](https://gitcode.com/RockChinQ/LangBot/star/badge.svg)](https://gitcode.com/RockChinQ/LangBot)
<a href="https://langbot.app">官網</a>
<a href="https://link.langbot.app/zh/docs/features">特性</a>
<a href="https://link.langbot.app/zh/docs/guide">文件</a>
<a href="https://link.langbot.app/zh/docs/api">API</a>
<a href="https://docs.langbot.app/zh/insight/features.html">特性</a>
<a href="https://docs.langbot.app/zh/insight/guide.html">文件</a>
<a href="https://docs.langbot.app/zh/tags/readme.html">API</a>
<a href="https://space.langbot.app">外掛市場</a>
<a href="https://langbot.featurebase.app/roadmap">路線圖</a>
@@ -46,9 +46,7 @@ LangBot 是一個**開源的生產級平台**,用於建構 AI 驅動的即時
- **Web 管理面板** — 透過瀏覽器直觀地配置、管理和監控機器人,無需手動編輯設定檔。
- **多流水線架構** — 不同機器人用於不同場景,具備全面的監控和異常處理能力。
[→ 了解更多功能特性](https://link.langbot.app/zh/docs/features)
📍 實踐指南:[5 分鐘部署多平台 AI 機器人](https://blog.langbot.app/zh/blog/deploy-ai-bot-in-5-minutes/)、[將 DeepSeek 接入微信、企業微信與 Discord](https://blog.langbot.app/zh/blog/connect-deepseek-to-wechat/)、[讓 Dify Agent 跑在 Discord、Telegram 和 Slack 上](https://blog.langbot.app/zh/blog/dify-agent-discord-telegram-slack/),以及[用 n8n 建構多平台 AI 聊天機器人](https://blog.langbot.app/zh/blog/n8n-multi-platform-ai-chatbot/)。
[→ 了解更多功能特性](https://docs.langbot.app/zh/insight/features.html)
---
@@ -79,7 +77,7 @@ docker compose up -d
[![Deploy on Zeabur](https://zeabur.com/button.svg)](https://zeabur.com/zh-CN/templates/ZKTBDH)
[![Deploy on Railway](https://railway.com/button.svg)](https://railway.app/template/yRrAyL?referralCode=vogKPF)
**更多方式:** [Docker](https://link.langbot.app/zh/docs/docker) · [手動部署](https://link.langbot.app/zh/docs/manual-deploy) · [寶塔面板](https://link.langbot.app/zh/docs/bt-panel) · [Kubernetes](./docker/README_K8S.md)
**更多方式:** [Docker](https://docs.langbot.app/zh/deploy/langbot/docker.html) · [手動部署](https://docs.langbot.app/zh/deploy/langbot/manual.html) · [寶塔面板](https://docs.langbot.app/zh/deploy/langbot/one-click/bt.html) · [Kubernetes](./docker/README_K8S.md)
---
@@ -87,19 +85,17 @@ docker compose up -d
| 平台 | 狀態 | 備註 |
|------|------|------|
| Discord | ✅ | 官方 |
| Telegram | ✅ | 官方 |
| Slack | ✅ | 官方 |
| LINE | ✅ | 官方 |
| QQ | ✅ | 個人號、官方機器人(頻道、私聊、群聊) |
| 企業微信 | ✅ | 應用訊息、對外客服、智能機器人 |
| 微信 | ✅ | 個人微信、微信公眾號 |
| 飛書 | ✅ | 官方 |
| 釘釘 | ✅ | 官方 |
| KOOK | ✅ | 官方 |
| 企業微信 | ✅ | 應用訊息、對外客服、智能機器人 |
| 飛書 | ✅ | |
| 釘釘 | ✅ | |
| Discord | ✅ | |
| Telegram | ✅ | |
| Slack | ✅ | |
| LINE | ✅ | |
| KOOK | ✅ | |
| Satori | ✅ | |
| Email | ✅ | 只 Matrix、Satori |
| Matrix | ✅ | 支援多種橋接平台,如 Signal、WhatsApp、Messenger、iMessage、Mattermost、Google Chat、IRC、XMPP、Zulip 等 |
---
@@ -128,7 +124,6 @@ docker compose up -d
| [PPIO](https://ppinfra.com/user/register?invited_by=QJKFYD&utm_source=github_langbot) | GPU 平台 | ✅ |
| [接口 AI](https://jiekou.ai/) | 聚合平台 | ✅ |
| [302.AI](https://share.302.ai/SuTG99) | 聚合平台 | ✅ |
| [Qiniu](https://www.qiniu.com/ai/agent) | 聚合平台 | ✅ |
### TTS語音合成
@@ -144,7 +139,7 @@ docker compose up -d
|-----------|------|
| 阿里雲百煉 | [外掛](https://github.com/Thetail001/LangBot_BailianTextToImagePlugin) |
[→ 查看完整整合列表](https://link.langbot.app/zh/docs/features)
[→ 查看完整整合列表](https://docs.langbot.app/zh/insight/features.html)
---

View File

@@ -19,9 +19,9 @@
[![GitHub stars](https://img.shields.io/github/stars/langbot-app/LangBot?style=social)](https://github.com/langbot-app/LangBot/stargazers)
<a href="https://langbot.app">Trang chủ</a>
<a href="https://link.langbot.app/en/docs/features">Tính năng</a>
<a href="https://link.langbot.app/en/docs/guide">Tài liệu</a>
<a href="https://link.langbot.app/en/docs/api">API</a>
<a href="https://docs.langbot.app/en/insight/features.html">Tính năng</a>
<a href="https://docs.langbot.app/en/insight/guide.html">Tài liệu</a>
<a href="https://docs.langbot.app/en/tags/readme.html">API</a>
<a href="https://space.langbot.app">Chợ Plugin</a>
<a href="https://langbot.featurebase.app/roadmap">Lộ trình</a>
@@ -44,9 +44,7 @@ LangBot là một **nền tảng mã nguồn mở, cấp sản xuất** để x
- **Bảng quản lý Web** — Cấu hình, quản lý và giám sát bot thông qua giao diện trình duyệt trực quan. Không cần chỉnh sửa YAML.
- **Kiến trúc đa Pipeline** — Các bot khác nhau cho các kịch bản khác nhau, với giám sát toàn diện và xử lý ngoại lệ.
[→ Tìm hiểu thêm về tất cả tính năng](https://link.langbot.app/en/docs/features)
📍 Hướng dẫn thực hành: [triển khai bot AI đa nền tảng trong 5 phút](https://blog.langbot.app/en/blog/deploy-ai-bot-in-5-minutes/), [kết nối DeepSeek với WeChat, Discord và Telegram](https://blog.langbot.app/en/blog/connect-deepseek-to-wechat/), [chạy Dify Agent trên Discord, Telegram và Slack](https://blog.langbot.app/en/blog/dify-agent-discord-telegram-slack/) và [xây dựng chatbot với n8n](https://blog.langbot.app/en/blog/n8n-multi-platform-ai-chatbot/).
[→ Tìm hiểu thêm về tất cả tính năng](https://docs.langbot.app/en/insight/features.html)
---
@@ -77,7 +75,7 @@ docker compose up -d
[![Deploy on Zeabur](https://zeabur.com/button.svg)](https://zeabur.com/en-US/templates/ZKTBDH)
[![Deploy on Railway](https://railway.com/button.svg)](https://railway.app/template/yRrAyL?referralCode=vogKPF)
**Thêm tùy chọn:** [Docker](https://link.langbot.app/en/docs/docker) · [Thủ công](https://link.langbot.app/en/docs/manual-deploy) · [BTPanel](https://link.langbot.app/en/docs/bt-panel) · [Kubernetes](./docker/README_K8S.md)
**Thêm tùy chọn:** [Docker](https://docs.langbot.app/en/deploy/langbot/docker.html) · [Thủ công](https://docs.langbot.app/en/deploy/langbot/manual.html) · [BTPanel](https://docs.langbot.app/en/deploy/langbot/one-click/bt.html) · [Kubernetes](./docker/README_K8S.md)
---
@@ -85,19 +83,17 @@ docker compose up -d
| Nền tảng | Trạng thái | Ghi chú |
|----------|--------|-------|
| Discord | ✅ | Chính thức |
| Telegram | ✅ | Chính thức |
| Slack | ✅ | Chính thức |
| LINE | ✅ | Chính thức |
| QQ | ✅ | Cá nhân & API chính thức (Kênh, DM, Nhóm) |
| Discord | ✅ | |
| Telegram | ✅ | |
| Slack | ✅ | |
| LINE | ✅ | |
| QQ | ✅ | Cá nhân & API chính thức |
| WeCom | ✅ | WeChat doanh nghiệp, CS bên ngoài, AI Bot |
| WeChat | ✅ | Cá nhân & Tài khoản công khai |
| Lark | ✅ | Chính thức |
| DingTalk | ✅ | Chính thức |
| KOOK | ✅ | Chính thức |
| Lark | ✅ | |
| DingTalk | ✅ | |
| KOOK | ✅ | |
| Satori | ✅ | |
| Email | ✅ | Matrix, Satori |
| Matrix | ✅ | Hỗ trợ nhiều nền tảng qua bridge như Signal, WhatsApp, Messenger, iMessage, Mattermost, Google Chat, IRC, XMPP, Zulip và hơn thế nữa |
---
@@ -126,9 +122,8 @@ docker compose up -d
| [ShengSuanYun](https://www.shengsuanyun.com/?from=CH_KYIPP758) | Nền tảng GPU | ✅ |
| [接口 AI](https://jiekou.ai/) | Cổng | ✅ |
| [302.AI](https://share.302.ai/SuTG99) | Cổng | ✅ |
| [Qiniu](https://www.qiniu.com/ai/agent) | Cổng | ✅ |
[→ Xem tất cả tích hợp](https://link.langbot.app/en/docs/features)
[→ Xem tất cả tích hợp](https://docs.langbot.app/en/insight/features.html)
---

View File

@@ -312,7 +312,7 @@ spec:
### 参考资源
- [LangBot 官方文档](https://docs.langbot.app)
- [Docker 部署文档](https://link.langbot.app/zh/docs/docker)
- [Docker 部署文档](https://docs.langbot.app/zh/deploy/langbot/docker.html)
- [Kubernetes 官方文档](https://kubernetes.io/docs/)
---
@@ -625,5 +625,5 @@ spec:
### References
- [LangBot Official Documentation](https://docs.langbot.app)
- [Docker Deployment Guide](https://link.langbot.app/zh/docs/docker)
- [Docker Deployment Guide](https://docs.langbot.app/zh/deploy/langbot/docker.html)
- [Kubernetes Official Documentation](https://kubernetes.io/docs/)

View File

@@ -18,40 +18,6 @@ services:
networks:
- langbot_network
# The Box sandbox runtime is optional. It is only started when you run
# ``docker compose --profile box up`` (or ``docker compose --profile all
# up``). With Box off, LangBot keeps the dashboard / skills list visible
# (read-only) but disables sandbox tools, skill add/edit and stdio MCP —
# set ``box.enabled: false`` in ``data/config.yaml`` (or
# ``BOX__ENABLED=false`` in the langbot service env below) to match.
langbot_box:
image: rockchin/langbot:latest
container_name: langbot_box
profiles: ["box", "all"]
volumes:
# Keep the source and target path identical because langbot_box uses the
# host Docker socket to create sandbox containers. Override
# LANGBOT_BOX_ROOT with an absolute path if you do not want the default.
- ${LANGBOT_BOX_ROOT:-${PWD}/data/box}:${LANGBOT_BOX_ROOT:-${PWD}/data/box}
# Mount container runtime socket for Box sandbox backend.
# Uncomment the one that matches your container runtime:
# - /var/run/podman/podman.sock:/var/run/podman/podman.sock # Podman
- /var/run/docker.sock:/var/run/docker.sock # Docker
restart: on-failure
environment:
- TZ=Asia/Shanghai
# The Box runtime does NOT read box.local.* from config.yaml or env; it
# receives its configuration from LangBot via the INIT RPC action.
# Do not add LANGBOT_BOX_* / BOX__* here — they would be silently ignored.
# Launched through the same CLI entry point as the plugin runtime
# (`langbot_plugin.cli.__init__ <subcommand>`). WebSocket is the default
# control transport — mirrors `rt`, which also runs with no flag. Pass
# `-s` / `--stdio-control` only for the stdio mode LangBot uses outside
# containers.
command: ["uv", "run", "--no-sync", "-m", "langbot_plugin.cli.__init__", "box"]
networks:
- langbot_network
langbot:
image: rockchin/langbot:latest
container_name: langbot
@@ -60,13 +26,6 @@ services:
restart: on-failure
environment:
- TZ=Asia/Shanghai
# Unified env-override convention: SECTION__SUBSECTION__KEY overrides the
# matching config.yaml field (see LoadConfigStage). These map onto
# box.local.* and are forwarded to the Box runtime via INIT RPC.
- BOX__LOCAL__HOST_ROOT=${LANGBOT_BOX_ROOT:-${PWD}/data/box}
- BOX__LOCAL__DEFAULT_WORKSPACE=default
- BOX__LOCAL__SKILLS_ROOT=skills
- BOX__LOCAL__ALLOWED_MOUNT_ROOTS=${LANGBOT_BOX_ROOT:-${PWD}/data/box}
ports:
- 5300:5300 # For web ui and webhook callback
- 2280-2285:2280-2285 # For platform reverse connection

View File

@@ -1,149 +0,0 @@
# Agent-owned Context 协议设计
本文档描述插件化 AgentRunner 场景下的上下文边界**设计理由**。结论先行LangBot 不应成为最终 agentic context manager它提供 context substrateAgentRunner 或其背后的 runtime 自己决定如何管理历史、压缩、召回和 KV cache。
> 涉及的数据结构(`AgentRunContext`、`ContextAccess`、`AgentRunAPIProxy` 等)唯一定义在 [PROTOCOL_V1.md](./PROTOCOL_V1.md)。本文只讲语义和约束,不重抄 schema。实现进度见 [PROGRESS.md](./PROGRESS.md)。
## 1. 设计原则
### 1.1 Agent 拥有上下文策略
不同 runner 背后的 runtime 差异很大:
- 官方 local-agent 可能依赖 LangBot 的模型、工具、知识库和存储。
- Claude Code SDK / Codex 类 runtime 有自己的 session、transcript、tool loop 和上下文压缩。
- Pi Agent SDK 或外部 agent 平台可能只需要当前事件和一个外部 conversation key。
因此 LangBot 不应强行决定最终传给模型的历史窗口。Host 只提供:当前事件的完整结构化信息、稳定身份和会话引用、可授权读取的 history / event / artifact / state API、可投影给外部 harness 的 scoped context / MCP / skill / resource refs、payload hard cap 和权限 guardrail。
### 1.2 Host 不定义通用历史窗口
历史窗口策略不是 AgentRunner 协议或 Query entry adapter 的核心概念。Host 只提供 history pull API、cursor、hard cap 和权限边界runner 自己决定是否读取、读取多少、如何截断和压缩。
正确的问题不是"LangBot 每轮裁几轮历史给 agent",而是:
- 这类 runner 是否自管 context
- 事件到来时 host 应 inline 哪些最小信息?
- agent 需要更多上下文时通过什么 API 拉取?
- host 如何保证安全、可审计和可分页?
### 1.3 Host 保存事实源Agent 管理 working context
三类数据要分开:
- `EventLog`: Host 保存原始事件、工具调用、投递结果、错误和系统事件。
- `Transcript`: Host 从 EventLog 投影出的对话视图,用于 UI、审计和按需历史读取。
- `Working context`: Agent 本轮实际送进模型或 runtime 的上下文,由 AgentRunner 决定。
LangBot 不提供 host-side inline history window。简单 runner 如果需要历史窗口,应在 runner 内部通过 Host history API 拉取并裁剪。
## 2. Event 到来时传什么
默认 `AgentRunContext`PROTOCOL_V1 §5.2)应尽量小且稳定。默认规则:
- Host MUST NOT inline full history by default.
- Host SHOULD inline only current event / input and context handles.
- Runner owns working-context assembly.
- Runner MAY use Host history / event / artifact / state / storage API when authorized.
- Official runners MUST consume Host infrastructure through the same public API as third-party runners.
### 2.1 必须 inline 的内容
当前 event 的类型/id/时间/source当前输入文本和结构化内容附件/文件/图片的 metadata 和 artifact refactor / subject / conversation / thread / bot / workspacedelivery 能力已授权资源列表context cursors 和可用 API 能力Agent/runner config。这些是 agent 决定下一步所需的最低信息。
### 2.2 默认不 inline 的内容
完整历史消息、大文件全文、大工具结果、全量知识库内容、平台原始 payload 大对象、每轮重新生成的大段 summary。这些会破坏跨进程序列化成本、泄露范围、KV cache 稳定性,也会迫使 host 替 agent 做 context 策略。
### 2.3 不提供 Host Inline History Window
`AgentRunContext` 不包含 `bootstrap` 字段。Host 不下发历史窗口,也不通过 Pipeline 配置决定窗口大小。runner 若需要类似 `recent_tail` 的策略,应在自己的 manifest/config schema 中声明参数,并在 runner 内部通过 history API 读取、裁剪和压缩。Host 只负责权限、分页、hard cap 和事实源。
## 3. ContextAccess 的作用
`ContextAccess`PROTOCOL_V1 §5.8)是 host 交给 agent 的上下文读取入口描述,告诉 agent当前事件位于哪条 conversation / thread、若需要更多历史从哪个 cursor 开始拉、host inline 了什么没 inline 什么、当前 run 有哪些 context API 权限。
## 4. Agent 如何获取更多上下文
所有 API 都走 `AgentRunAPIProxy`PROTOCOL_V1 §8由 host 用 `run_id` 校验。
### 4.1 History
```python
await api.history.page(conversation_id=ctx.context.conversation_id,
before_cursor=ctx.context.latest_cursor,
limit=50, direction="backward", include_artifacts=False)
```
返回:
```python
class HistoryPage(BaseModel):
items: list[TranscriptItem]
next_cursor: str | None
prev_cursor: str | None
has_more: bool
```
约束:`limit` 有 host hard cap默认只能读当前 conversation / thread跨会话读取需 manifest permission + binding policy返回 artifact ref不默认返回大文件内容。
### 4.2 Search
```python
await api.history.search(query="用户之前提到的数据库连接信息",
filters={"conversation_id": ..., "event_types": ["message.received"]},
top_k=10)
```
Search 可先用数据库全文索引,后续接 embedding recall。它是 host 检索能力,不等于 agent 的长期记忆策略。
### 4.3 Event / Artifact / State
- Event API`events.get` / `events.page`用于读取非消息事件、工具事件、系统事件。Agent 不应把所有事件都当成 user/assistant message。
- Artifact API`artifacts.metadata` / `read_range` / `open_stream`)必须校验 artifact 所属 conversation / run / binding校验 MIME / 大小 / 过期 / 权限,大文件按 range/stream 读取,工具大结果也应 artifact 化。
- State API`state.get` / `set`)是可选寄宿能力。自管 runtime 可以完全不用;依附 LangBot 的官方 runner 可以使用,例如 `external.session_id``summary.checkpoint`
### 4.4 大文件与工具协作
大文件、多模态输入和工具产物不要内联进 prompt 或 tool resultmessage/content 里只放小文本和必要摘要;大文件、图片、音频、长工具输出返回 artifact ref`artifact_id``mime_type``size``digest``summary``expires_at``permissions`)。工具之间传递大结果时传 artifact ref不传完整 blob。Host 校验 artifact 是否属于当前 run / scope默认不允许插件直接读任意本地路径临时文件应有 TTL 和清理机制。
### 4.5 External harness context projection
Claude Code、Codex、Kimi Code 这类 runtime 通常已有自己的 session、工具 loop、MCP 加载、上下文压缩和工作目录。LangBot 不应把它们改造成"host prompt assembler",而应提供可审计的事件和资源投影。推荐 projection 形态:
- `agent-context.json`:结构化 JSON包含 `run_id``event``actor``subject``input``delivery``resources``context``state``runtime`
- `LANGBOT_CONTEXT.md`:人类可读摘要。
- `resources`:只包含本次 run 授权后的句柄,不暴露 Host 内部私有对象。
- `skills`:已授权 skill 投影为目标 harness 可读目录(如 Claude Code 的 `.claude/skills/<name>/SKILL.md`)。
- `MCP config`scoped MCP 配置runner adapter 转成目标 harness 的配置文件或 CLI 参数。
- `state pointers`:外部 session id、working directory、checkpoint 等小型 JSON 状态通过 Host state API 保存。
当前 Claude Code runner 使用 schema `langbot.agent_runner.external_harness_context.v1`(现状见 OFFICIAL_RUNNER_PLUGINS §8。这类 projection 是"把 LangBot 事实源和授权资源交给 harness",不是"由 LangBot 决定最终模型上下文"。
## 5. Runner manifest 中的上下文声明
`AgentRunnerContextPolicy`PROTOCOL_V1 §4.5)声明 runner 的上下文能力:`supports_history_pull` / `supports_history_search` / `supports_artifact_pull` / `owns_compaction` / `wants_static_context_refs`。它表示 Host 只给当前事件和 context handlesrunner 自己决定是否拉取历史、是否搜索、何时摘要、如何构造最终 prompt。
## 6. KV cache 友好的上下文管理
支持 Claude Code SDK、Codex、Pi Agent SDK 等 runtime 时,必须避免每轮由 LangBot 重组大块 prompt
- 稳定 session key`workspace/bot/binding/runner/conversation/thread`
- 静态内容使用 `ref + version/hash``ctx.runtime.static_refs`system prompt、resource manifest、tool schema、platform policy。
- 每轮只传 delta当前 event、artifact refs、少量 runtime metadata。
- 历史 append-only不要每轮改写同一段 history 文本。
- Summary checkpoint 稳定:只有压缩发生时产生新 checkpoint。
- 大文件和工具结果 artifact 化。
- Tool/context API schema 稳定,数据通过 API 拉取而非塞入 prompt。
- 对自管 runtime优先让它复用自身 session/cache而不是强制 LangBot 每轮重放 transcript。
- LiteLLM 接入后,模型窗口元信息应作为 resource/runtime metadata 暴露给 runner由 runner 决定预算和压缩策略。
## 7. Host guardrail
Agent 自管 context 不代表无限制访问。LangBot 仍必须控制:每次 run 的 active `run_id`、runner identity、当前 binding 的 resource policy、conversation / actor / subject scope、page size / artifact read size / API rate limit、跨会话读取权限、数据脱敏和敏感变量过滤、审计日志。Host 不负责"最佳上下文策略",但负责"不越权、不爆内存、不不可审计"。
## 8. 官方 runner 与业务编排边界
官方 runner 插件可以把状态寄宿在 LangBot但必须和第三方 runner 一样通过公开 Host API 消费。LangBot core 不内置官方 agent 的业务流程prompt 组装、tool loop、RAG 编排、summary/compaction、"local-agent 专用"状态字段)。
官方 local-agent 应作为"依附 LangBot 基础设施的复杂 runner 参考实现"transcript/history 通过 `api.history` 读取summary/checkpoint/外部 session id/用户偏好通过 `api.state``api.storage` 保存,图片/文件/工具大结果通过 `api.artifacts` 读取,模型/工具/知识库通过 `api.models` / `api.tools` / `api.knowledge` 调用。这样 LangBot 保持为通用 agent host不变成内置 agent 框架。具体迁移要求见 [OFFICIAL_RUNNER_PLUGINS.md](./OFFICIAL_RUNNER_PLUGINS.md)。

View File

@@ -1,97 +0,0 @@
# Event Based Agent 预留设计
> **future design note**不是当前分支实现范围。EventGateway、EventRouter、Event subscription/notification 由其他分支实现;本分支只预留 event-first 入口和 envelope/binding models。实现进度见 [PROGRESS.md](./PROGRESS.md)。
>
> 数据结构唯一定义在 [PROTOCOL_V1.md](./PROTOCOL_V1.md)runner 可见)与 [HOST_SDK_INFRASTRUCTURE.md](./HOST_SDK_INFRASTRUCTURE.md)Host 内部模型);本文只讲 EBA 语义,不重抄 schema。
本文描述未来 EBA 接入时,事件如何进入 LangBot、如何触发 AgentRunner以及如何复用插件化 agent 基础设施。本阶段不实现完整 EventBus / EventRouter / Platform API目标是把协议边界设计对避免当前消息入口继续绑死 Pipeline 和用户文本消息。
## 1. 设计目标
- 消息、撤回、入群、好友申请、定时任务、API 调用都能抽象为 host event。
- EventRouter 可以根据 event type、bot、workspace、conversation、actor、subject 解析 `AgentBinding`
- AgentRunner 通过同一套 orchestrator 被调用。
- 非消息事件不伪造成用户文本消息。
- 平台动作执行通过显式 capability / permission / result type 预留,不混入普通文本回复。
## 2. 事件不是消息
`message.received` 只是事件的一种。协议不应假设:一定有用户文本、一定有 conversation history、一定要返回一条聊天消息、actor 一定等于 sender、subject 一定等于当前消息。
| event_type | actor | subject | input |
| --- | --- | --- | --- |
| `message.received` | 发消息的人 | 当前消息 | 文本、图片、文件等 |
| `message.recalled` | 撤回操作者,未知时为系统 | 被撤回消息 | 通常为空 |
| `group.member_joined` | 新成员或邀请人 | 群/成员关系 | 通常为空 |
| `friend.request_received` | 申请人 | 好友申请 | 验证消息或申请理由 |
| `schedule.triggered` | 系统 | 定时任务 | 任务 payload |
| `api.invoked` | API caller | API request | request payload |
## 3. 稳定事件名
先保留的稳定事件名(作为插件协议的一部分保持稳定):
- `message.received`
- `message.recalled`
- `group.member_joined`
- `friend.request_received`
平台原始事件名只能进入 `ctx.event.source_event_type` / `raw_ref`,不能成为 `ctx.event.event_type` 的公共契约。
## 4. Event Envelope 与 Binding
- 入口事件用 `AgentEventEnvelope`HOST_SDK §4.1)承载;顶层字段使用 LangBot 稳定协议名,平台原始事件名和原始 payload 放 `metadata` / `raw_ref`
- 触发关系用 `AgentBinding`HOST_SDK §4.2表达。EBA 阶段 binding 通过 `event_types``scope``filters` 决定哪些事件触发当前 bot / channel 绑定的 Agent。
目标产品语义:一个 bot / IM channel 在同一时间只绑定一个负责 agentic
处理的 Agent一个 Agent 可以被多个 bot / channel 复用。因此 EBA 主线按
single-agent dispatch 设计,不做默认 fan-out。
Binding scope 示例workspace 全局、bot 级、platform channel 级、conversation / group / thread 级、user / actor 级。旧 Pipeline 可迁移为 `message.received` 的临时 binding source但目标持久配置应是 Agent不是 Pipeline。
Event Source 可包括:`platform_adapter`飞书、QQ、微信、Telegram 等)、`webui``http_api``scheduler``system`。EventRouter 不应写死平台 adapter 的类名。
## 5. EventRouter 调用链
```text
Platform Adapter / WebUI / API
-> Event Gateway normalize payload
-> EventLog append raw event
-> EventRouter resolve one effective AgentBinding
-> AgentRunOrchestrator.run(event, binding)
-> AgentRunContextBuilder.build(event, binding)
-> PluginRuntimeConnector.run_agent()
-> AgentRunResult stream
-> DeliveryController render / platform action
```
约束:必须复用现有 orchestrator不能为 EBA 单独实现另一套 plugin runner 调用协议;非消息事件不能绕过 resource authorizationdelivery 和 platform action 走统一权限模型;外部 harness runner 也通过同一套 envelope/binding/context/result 协议接入,不为 Claude Code / Codex / Kimi 单独发明队列协议。
若未来产品需要 observer agent、多个 agent 并行处理同一事件、或多 runner
裁决,应另行设计 fan-out 合并、delivery 冲突、state 写入冲突、platform
action 审批和 audit 语义。当前 EBA 预留不隐含这些能力。
## 6. 平台动作执行
EBA 后 `action.requested`PROTOCOL_V1 §7.2,当前仅 telemetry 不执行)将用于请求 host 执行平台动作:
```json
{ "type": "action.requested",
"data": { "action": "friend.request.accept",
"target": {"platform": "wechat", "request_id": "..."},
"reason": "policy matched" } }
```
Host 必须校验runner manifest 是否声明 `platform_api` capability、binding 是否授权该 action、actor / bot / workspace 是否允许、是否需要人工审批。EBA 还可能预留 `delivery.requested`(请求投递到某 surface
Delivery 方面event 不一定回复到当前聊天窗口:消息事件通常带 reply target系统事件可能没有默认 reply target需要 runner 返回 `action.requested` 或由 binding 的 delivery policy 决定投递位置(`DeliveryContext` 见 PROTOCOL_V1 §5.7)。
## 7. 与 Context 协议的关系
EBA 事件进入 AgentRunner 时仍遵循 [AGENT_CONTEXT_PROTOCOL.md](./AGENT_CONTEXT_PROTOCOL.md)inline 当前事件、大 payload 用 raw/artifact ref、不默认 inline 完整 history、agent 按需通过 API 拉取、Host 保留 EventLog 和权限 guardrail。非消息事件可以被投影进 Transcript但不能强制伪装为 user messageAgentRunner 根据 event type 自己决定是否纳入模型上下文。
## 8. 未来 EBA 完整落地需要
EventGateway 完整实现、EventRouter 与 BindingResolver 集成、`AgentBinding` 持久模型和 UI、`DeliveryContext` 完整实现、platform action permission model 和执行器、真实平台事件接入。
落地顺序:① 把当前 Pipeline 消息入口适配成 `message.received` event已完成→ ② 增加 `AgentBinding` 抽象,先由 current config 生成(已完成)→ ③ context builder 改为从 event + binding 构造(已完成)→ ④ 引入 EventLog / Transcript已完成→ ⑤ 增加非消息事件的协议测试,不接真实平台 → ⑥ 接入真实 EventRouter 和 platform action。

View File

@@ -1,240 +0,0 @@
# LangBot Host 与 SDK 基础设施设计
本文档描述 LangBot 作为 agent host 的内部能力与分层架构,以及 Host 内部模型。
- SDK ↔ Host 的协议数据结构(`AgentRunContext``AgentRunnerManifest``AgentRunResult``AgentRunAPIProxy` 等)的**唯一定义在** [PROTOCOL_V1.md](./PROTOCOL_V1.md);本文只引用,不重抄。
- 实现进度见 [PROGRESS.md](./PROGRESS.md)。
- 本文定义的 Host 内部模型(`AgentEventEnvelope``AgentBinding``AgentRunnerDescriptor`)不属于 SDK 协议字段。
## 1. 目标
LangBot 要转为 agent host而不是内置 runner 容器:
- 接收 IM、WebUI、API 和未来 EventRouter 产生的事件。
- 根据事件、bot、workspace、scope 解析应该调用的 Agent / agent binding。
- 发现、校验和调用插件提供的 AgentRunner。
- 为每次 run 提供受限资源、状态、存储、上下文引用和生命周期控制。
- 接收 AgentRunner 返回的事件流,投递到 IM、WebUI 或其他 output surface。
## 2. 非目标
- 不把 Pipeline 当作长期架构中心。
- 不要求所有 AgentRunner 依赖 LangBot 的上下文管理。
- 不要求官方 local-agent 的旧行为反向塑造 host 协议。
- 不在 host 中实现通用 agentic prompt assembler。
- 不强制 runner 使用 LangBot state / storage只提供可选、受控的寄宿能力。
- 不实现 EventGateway它是 future integration point由外部 event branch 提供。本分支只定义 host-side envelope/binding models 和 `run(event, binding)` 入口。
## 3. 分层架构
```text
IM / WebUI / API / EventRouter (future)
|
v
Event Gateway (future - external event branch)
|
v
AgentBindingResolver
|
v
AgentRunOrchestrator
|-- AgentRunnerRegistry
|-- AgentResourceBuilder
|-- AgentContextBuilder
|-- AgentRunSessionRegistry
|-- PersistentStateStore / EventLogStore / TranscriptStore / ArtifactStore
v
Plugin Runtime / AgentRunner
|
v
AgentRunResult stream
|
v
Delivery / Renderer / Platform API
```
目标产品模型中Agent 替代 Pipeline 承载 agent 配置bot / IM
channel 绑定一个 Agent一个 Agent 可以被多个 bot / channel 复用。
当前 Pipeline 只应接入在 Query entry adapter 位置:它可以继续产生
`message.received` 并投影出临时 `AgentBinding`,但不应再拥有 runner
选择、上下文裁剪和业务 agent 执行的核心语义。EventGateway 由外部 event
branch 实现。
## 4. LangBot 侧能力
### 4.1 Event GatewayFuture Integration Point
> EventGateway 由外部 event branch 实现,不在本分支范围。本分支只预留 event-first 入口和 envelope/binding models。
Event Gateway 将把入口统一成 host eventIM 平台消息、WebUI debug chat、API 触发、后续非消息事件),输出稳定的 `AgentEventEnvelope`Host 内部模型):
```python
class AgentEventEnvelope(BaseModel):
event_id: str
event_type: str
event_time: int | None
source: str
bot_id: str | None
workspace_id: str | None
conversation_id: str | None
thread_id: str | None
actor: ActorRef | None
subject: SubjectRef | None
input: AgentInput # 见 PROTOCOL_V1 §5.6
delivery: DeliveryContext # 见 PROTOCOL_V1 §5.7
raw_ref: RawEventRef | None
metadata: dict[str, Any] = {}
```
`AgentEventEnvelope` 是 Host 内部入口模型;投影给 runner 的是 `ctx.event`PROTOCOL_V1 §5.4)。原始平台 payload 存为 raw event 或 artifact ref不扩散到 runner 协议顶层。
**当前 adapter source**`QueryEntryAdapter.query_to_event(query)` 从 Query 生成 `AgentEventEnvelope`
### 4.2 AgentBinding
`AgentBinding` 是"什么事件调用哪个 AgentRunner、带什么 Agent 配置"的
Host 内部运行投影(不暴露给 SDK。产品层的持久对象应是 Agent
Agent 携带 runner id、runner config、resource/state/delivery policy并可被
多个 bot / channel 复用。`AgentBinding` 是 EventRouter / 当前
QueryEntryAdapter 在一次运行前解析出的有效绑定。
```python
class AgentBinding(BaseModel):
binding_id: str
enabled: bool
scope: BindingScope
event_types: list[str]
filters: list[EventFilter] = [] # EBA 阶段使用,见 EVENT_BASED_AGENT
runner_id: str
runner_config: dict[str, Any]
resource_policy: ResourcePolicy
state_policy: StatePolicy
delivery_policy: DeliveryPolicy
```
一个 bot / IM channel 在同一时间只应解析出一个负责 agentic 处理的
AgentBinding。若未来需要 observer / fan-out / 多 agent 裁决,必须另行定义
delivery、state、platform action 和 result 合并语义;当前 v1/EBA 主线不隐式支持。
**当前 adapter source**`QueryEntryAdapter.config_to_agent_config(query, runner_id)`
先把 current config 投影为迁移期 `AgentConfig`,再由
`AgentBindingResolver.resolve_one(event, [agent_config])` 解析出唯一
`AgentBinding`。Pipeline 当前只是迁移期 Agent config sourceAI runner config
→ runner_config、extension preference → resource_policy、output settings →
delivery_policy但新设计不再把这些字段命名为 Pipeline 专属概念。
### 4.3 AgentRunnerRegistry
Registry 收集 runner descriptor来自插件 runtime、开发期本地插件
```python
class AgentRunnerDescriptor(BaseModel):
id: str
source: Literal["plugin"]
label: I18nObject
description: I18nObject | None = None
protocol_version: str = "1"
capabilities: AgentRunnerCapabilities # 见 PROTOCOL_V1 §4.3
permissions: AgentRunnerPermissions # 见 PROTOCOL_V1 §4.4
config_schema: list[DynamicFormItemSchema]
plugin: PluginRef | None = None
```
职责:调用 `plugin_connector.list_agent_runners()` 拉取 runner、校验 manifest`kind == AgentRunner``metadata.name/label` 存在、`protocol_version` 兼容、`spec.*` 类型正确)、输出 descriptor、缓存 discovery 结果并提供 `refresh()`。单个插件 manifest 失败只记 warning不影响其它 runner。`plugin:author/name/runner` 是稳定 id 格式;多个 binding 指向同一 runner id 时**不创建多个插件实例**。
Host 内置 runner / adapter 不能作为 `AgentRunnerDescriptor.source` 绕过插件
runtime、`run_id``ctx.resources``AgentRunAPIProxy` 权限链。若需要
开发期调试 adapter应放在 Host 内部测试入口,不进入可选 runner 列表。
刷新触发点:插件安装/卸载/升级/重启后Pipeline metadata 请求时发现缓存为空;可选 TTL优先保证正确性
### 4.4 AgentRunOrchestrator
Orchestrator 是唯一运行入口:
```text
run(event, binding)
-> resolve runner descriptor
-> build resources
-> build context
-> register run session
-> call plugin runtime
-> normalize result stream
-> update state
-> unregister run session
```
它负责:`run_id` 生成和生命周期、timeout/deadline/cancellation、插件异常隔离、result schema 校验和大小限制、`state.updated` 处理、delivery backpressure 和 telemetry。
`run_from_query()` 保留为 Query entry adapter 入口,但内部转换成 event + binding 后走统一 `run()`。约束:`ChatMessageHandler` 不解析 `plugin:*`、不实例化 wrapper、不知道 runner 组件细节;`PipelineService` 从 registry 读取 metadata不直接访问插件 runtime插件是无状态执行单元跨请求持久化状态必须走授权 storage / 外部服务,不能隐式存在 per-pipeline 插件对象里。
### 4.5 Resource Authorization三层裁剪
LangBot 在每次 run 前生成 `ctx.resources`PROTOCOL_V1 §6来自三层约束
1. runner manifest 声明的 `permissions`(最大能力)。
2. binding / resource policy 允许的资源范围。
3. 当前 event / actor / bot / workspace 的实际权限。
这次裁剪结果必须冻结为 run-scoped authorization snapshot并由
`AgentRunSessionRegistry``run_id` 保存。`ctx.resources` 是投影给 runner
看的同一份授权结果;运行期每个 proxy action 只依据该 snapshot 校验 active
run session、caller plugin identity、resource id、scope、payload size、rate
limit 和 deadline。Handler 不应重新执行三层裁剪,否则 build-time 与 runtime
授权逻辑会漂移。
SDK 侧本地校验只用于开发体验host 侧 run authorization snapshot 才是安全边界。
资源裁剪应通用,不写死 local-agent。selector 与资源的映射示例:`model-fallback-selector` → primary/fallback LLM、`llm-model-selector` → LLM、`rerank-model-selector` → rerank 模型、`knowledge-base-multi-selector` → 知识库;新增 selector 时在 resource builder 中统一扩展。
执行/文件/skill/MCP 等能力的接入方向:先由 Host 封装成普通 tool再通过 `ctx.resources.tools` 进入 runnerrunner 不应识别或硬编码执行环境 provider。
### 4.6 State / Storage
LangBot 可提供 host-owned state 让 runner 寄宿状态conversation / actor / subject / runner / binding / workspace state但**不是强制**。Host 只需提供授权开关、scope key、get/set/list/delete API见 PROTOCOL_V1 §8、持久化 backend、审计和清理策略。外部 agent runtime 可维护自己的 session 和 memory。进程内 state store 只能作为过渡实现,不能作为正式生产语义。
### 4.7 EventLog / Transcript / Artifact事实源
- `EventLog`: durable append-only保存原始事件、系统事件、工具调用、投递结果、错误。
- `Transcript`: 从 EventLog 投影出的对话视图,用于 UI、审计和按需历史读取。
- `ArtifactStore`: 保存大文件、多模态输入、工具大结果、平台附件。
三类数据与 working context 的边界、读取约束见 [AGENT_CONTEXT_PROTOCOL.md](./AGENT_CONTEXT_PROTOCOL.md)。AgentRunner 可读取这些能力,但不被迫使用 LangBot 作为唯一记忆系统。
### 4.8 Prompt / Instruction Package占位
当前 Query 入口不把 preprocessing 后的有效 prompt 放进 adapter metadata。目标形态是 Host 保存或生成一个 run-scoped instruction packagerunner 通过 Host API 拉取:
- Host 记录静态绑定 prompt、host hook / user plugin 产生的 instruction fragment、来源和审计信息。
- `ctx.context.available_apis` 增加 `prompt_get` 能力位表示拉取是否可用。
- Runner 拉取后仍由自己决定如何与 history、RAG、tool 结果、memory 和当前输入组装最终 prompt。
- Host 不实现通用 agentic prompt assembler也不把 Query entry adapter prompt 作为长期业务输入契约。
### 4.9 External harness resource projection
Claude Code、Codex、Kimi Code 等外部 harness runner 可能不直接调用 LangBot 的 model/tool loop而是把 LangBot 事件和授权资源投影到自己的 harness 执行。Host 侧仍保持统一边界Host 负责构造 event-first context、资源授权、state/storage、EventLog/Transcript/ArtifactStore 和审计Host 或 binding policy 决定哪些 MCP server、skill、artifact、history/state 句柄可投影给 runnerrunner plugin 把 scoped projection 转成目标 harness 可消费形式;外部 harness 负责自己的 native session、tool loop、压缩、权限模式和 resume。
投影的具体形态context 文件、skill 目录、MCP config、state pointers见 AGENT_CONTEXT_PROTOCOL §4.5Claude Code / Codex 当前实现见 OFFICIAL_RUNNER_PLUGINS §7。发布级隔离要求见 SECURITY_HARDENING。
## 5. SDK 侧协议
SDK 组件入口如下;所有数据结构定义见 PROTOCOL_V1。
```python
class AgentRunner(BaseComponent):
__kind__ = "AgentRunner"
@classmethod
def get_capabilities(cls) -> AgentRunnerCapabilities: ... # PROTOCOL_V1 §4.3
@classmethod
def get_config_schema(cls) -> list[dict]: ...
async def run(self, ctx: AgentRunContext) -> AsyncGenerator[AgentRunResult, None]: ...
# ctx: PROTOCOL_V1 §5.2 ; AgentRunResult: PROTOCOL_V1 §7
```
- Manifest / capabilities / permissions / context policyPROTOCOL_V1 §4。
- `AgentRunContext`PROTOCOL_V1 §5.2。`messages` / `bootstrap` 不是协议字段。
- `AgentRunResult`PROTOCOL_V1 §7。
- `AgentRunAPIProxy`PROTOCOL_V1 §8是 runner 访问 host 能力的唯一入口,所有请求带 `run_id`

View File

@@ -1,147 +0,0 @@
# 官方 AgentRunner 插件迁移计划
本文档描述内置 `RequestRunner` 迁出 LangBot 后,官方 runner 插件如何组织、迁移和验收。它是 [HOST_SDK_INFRASTRUCTURE.md](./HOST_SDK_INFRASTRUCTURE.md) 和 [AGENT_CONTEXT_PROTOCOL.md](./AGENT_CONTEXT_PROTOCOL.md) 的下游落地计划,不是 LangBot 宿主协议的设计前提。验收状态见 [PROGRESS.md](./PROGRESS.md)QA 入口见 [PHASE1_QA_ACCEPTANCE_MATRIX.md](./PHASE1_QA_ACCEPTANCE_MATRIX.md)。
官方 `local-agent` 可以外移,也可以重写。设计重点不是保留旧内置 runner 的内部结构,而是验证一个依附 LangBot host 基础设施的官方 agent 能否完整工作。同时LangBot host 协议必须服务 Claude Code SDK、Codex、Pi Agent SDK、外部 Agent 平台等自管 context/runtime 的 runner不能被官方插件的实现细节绑死。
## 1. 仓库组织
官方 runner 插件与 LangBot 主仓库、SDK 仓库以不同节奏迭代LangBot 主仓库只维护宿主协议和调度SDK 仓库维护 AgentRunner 组件和 runtime 协议,官方 runner 插件承载业务 runner 的具体实现和第三方平台适配。
当前推荐"官方插件可独立发布,必要时共享 SDK helper"。开发期采用本地多目录布局:
```text
langbot-app/
langbot-local-agent/ # plugin:langbot/local-agent/default
manifest.yaml
components/agent_runner/default.{yaml,py}
langbot-agent-runner/ # 外部服务 runner 仓库
claude-code-agent/ codex-agent/ dify-agent/ n8n-agent/ ...
```
后续可聚合进 monorepo也可继续独立发布——这个选择不影响协议设计。重复逻辑优先沉淀到 SDK 或明确的共享 helper 包,不要把宿主私有结构泄漏给插件。旧 `src/langbot/pkg/provider/runners/*` 在官方插件迁移完成前保留作为行为对齐基准,不作为长期运行路径。
## 2. 插件命名和 runner id
| 旧 runner | 官方插件 | runner id |
| --- | --- | --- |
| `local-agent` | `langbot/local-agent` | `plugin:langbot/local-agent/default` |
| `dify-service-api` | `langbot/dify-agent` | `plugin:langbot/dify-agent/default` |
| `n8n-service-api` | `langbot/n8n-agent` | `plugin:langbot/n8n-agent/default` |
| `coze-api` | `langbot/coze-agent` | `plugin:langbot/coze-agent/default` |
| - | `langbot/claude-code-agent` | `plugin:langbot/claude-code-agent/default` |
| - | `langbot/codex-agent` | `plugin:langbot/codex-agent/default` |
| `dashscope-app-api` | `langbot/dashscope-agent` | `plugin:langbot/dashscope-agent/default` |
| `langflow-api` | `langbot/langflow-agent` | `plugin:langbot/langflow-agent/default` |
| `tbox-app-api` | `langbot/tbox-agent` | `plugin:langbot/tbox-agent/default` |
每个插件可后续提供多个 runner但迁移目标的默认 runner 统一叫 `default`
## 3. 迁移批次
- **Batch 1打通协议**`local-agent`(能力最完整基准)、`claude-code-agent` / `codex-agent`(外部 code-agent harness 边界)、`dify-agent`(传统 service API runner
- **Batch 2外部 workflow**`n8n-agent``langflow-agent`webhook/workflow 输入输出、timeout、外部 conversation id
- **Batch 3平台 Agent API**`coze-agent``dashscope-agent``tbox-agent`(平台特有响应格式、引用资料、文件/图片输入)。
## 4. 每个官方插件的组件要求
每个插件至少包含一个 `AgentRunner` 组件manifest 示例:
```yaml
apiVersion: langbot/v1
kind: AgentRunner
metadata:
name: default
label: { en_US: Dify Agent, zh_Hans: Dify Agent }
description:
en_US: Run a Dify application as a LangBot AgentRunner.
zh_Hans: 将 Dify 应用作为 LangBot AgentRunner 运行。
spec:
protocol_version: "1"
config: []
capabilities: # 字段语义见 PROTOCOL_V1 §4.3
streaming: true
event_context: true
stateful_session: true
permissions: # 字段语义见 PROTOCOL_V1 §4.4
storage: ["plugin"]
context: # 字段语义见 PROTOCOL_V1 §4.5
supports_history_pull: true
owns_compaction: true
execution:
python: { path: ./main.py, attr: DefaultAgentRunner }
```
## 5. local-agent 插件方向
`local-agent` 是官方插件中能力最完整的消费者,但不是宿主协议的设计中心。它需要证明:一个主要依附 LangBot host 能力的 agent runner 可以通过公开协议完成模型、工具、知识库、状态、history、artifact、上下文压缩和消息投递。
迁移或重写需覆盖旧内置 runner 的用户可见能力model primary/fallback 选择、prompt、knowledge-bases、rerank-model、rerank-top-k、function calling、streaming、multimodal input、conversation history、monitoring metadata。
责任边界与 Host API 消费方式见 AGENT_CONTEXT_PROTOCOL §8。关键约束
-`ctx.config` 读取静态绑定 `prompt`**不**读取 `ctx.adapter.extra["prompt"]`;不消费 Query entry adapter 生成的历史窗口。
- 通过 `AgentRunAPIProxy.history` 拉取 transcript而不是依赖 host 每轮强塞历史窗口。
- `ctx.input.contents` 保留图片/文件等多模态内容RAG 只替换/插入文本部分,不丢图片/文件。
- 不能绕过 `ctx.resources` 调用未授权模型、工具或知识库。
- manifest 声明自管上下文能力(`context.supports_history_pull/search``owns_compaction` 等)。
### 5.1 Native Execution / Skills 后续接入
本阶段不把 sandbox/skills 做成 AgentRunner 协议字段。后续 sandbox/skills 分支合并后命令执行、文件操作、skill、MCP managed process 应先由 Host 封装成 scoped tools再通过 `ctx.resources.tools` 暴露给 runner。这让 local-agent 只消费授权后的 Host 基础设施,而不是直接持有宿主机执行能力。
## 6. 外部 runner 插件要求
外部平台 runner 迁移遵循:旧配置字段尽量保持同名便于 migration 复制;输出统一转换为 `AgentRunResult`;外部 API timeout 从 runner config 读取;平台 conversation id 存 plugin storage 或 context runtime state不依赖 LangBot 内置 conversation uuid 私有结构;流式按平台能力声明,没有流式就只发 `message.completed`
### 6.1 Code-agent harness runner
Claude Code、Codex、Kimi Code 这类 runner 不一定通过 LangBot 的模型/工具 loop 执行,可以依赖自己的 harness但仍必须遵守 Host 边界:输入来自 `ctx.event` / `ctx.input`,不依赖 Pipeline 私有 `Query`;授权资源投影为 harness 可读的 context 文件、MCP 配置、skill 目录、环境变量或 CLI 参数(投影形态见 AGENT_CONTEXT_PROTOCOL §4.5);外部 session id / workspace / checkpoint 写入 Host state 或 plugin storage插件实例保持无状态CLI / subprocess runner 必须处理 timeout、取消、空输出、非零退出和 stderr 映射harness 的 permission mode / allow-deny / MCP 配置只是一层执行约束Host 仍负责调用前的资源授权、路径策略、secret 过滤和审计(发布级要求见 [SECURITY_HARDENING.md](./SECURITY_HARDENING.md))。
### 6.2 SDK-owned LangBot MCP bridge
外部 harness 不能直接持有进程内的 `plugin_runtime_handler`,因此不能像 `local-agent` 一样直接调用 `AgentRunAPIProxy`。当前轻量方案是由 SDK 提供一层 per-run MCP bridge
- `AgentRunner.create_external_mcp_bridge(ctx)` 是 runner 父类入口。
- Bridge 由 `AgentRunAPIProxy``AgentRunContext` 构造,生命周期只覆盖当前 run。
- Bridge 暴露 SDK 中显式注解的 `AgentRunExternalTools`,而不是导出全部 SDK actionMCP tool schema 由注解和 Pydantic args model 生成。
- stdio MCP proxy 只把外部 harness 的 MCP 调用转发回当前 run 的本地 bridgerun 结束后 bridge 关闭。
第一批工具保持很小当前事件快照、history page、knowledge retrieve、authorized tool call。新增工具必须先进入 SDK-owned annotated surface再由 MCP adapter 自动投影。
## 7. Claude Code / Codex runner 当前形态
`claude-code-agent``codex-agent` 是最小可运行 MVP / dev path用来证明外部 harness runner 可以接入同一套 AgentRunner 协议。本地 smoke 验收记录见 [PROGRESS.md](./PROGRESS.md) 与 [PHASE1_QA_ACCEPTANCE_MATRIX.md](./PHASE1_QA_ACCEPTANCE_MATRIX.md)。
MVP 含义:已验证 event-first context、resource projection、result stream 和
基础 resume state 可以跑通;不表示 Docker 生产部署、发布级执行隔离、
workspace lifecycle、secret projection、团队级 audit 或 runtime sidecar 已完成。
### 7.1 Claude Code runner
- Runner ID`plugin:langbot/claude-code-agent/default`,执行方式:本地 Claude Code CLI print mode默认 `claude -p`)。
- 默认输出 `message.completed` + `run.completed`;默认权限 `permission-mode=plan``max-turns=1``disallowedTools=AskUserQuestion`
- 投影:写入 `agent-context.json`schema `langbot.agent_runner.external_harness_context.v1`)和 `LANGBOT_CONTEXT.md`;可把 `skills-json` 投影到 `.claude/skills/<name>/SKILL.md`;可把 `mcp-config-json` 写成每次 run 的 MCP config 经 `--mcp-config` / `--strict-mcp-config` 传入;可通过 `enable-langbot-mcp=true` 启用 SDK-owned per-run LangBot MCP bridge。
- 状态Claude Code 返回 `session_id` 时通过 `state.updated` 写回 `external.session_id`;工作目录优先用 config 的 `working-directory`,其次用 Host state 的 `external.working_directory`
### 7.2 Codex runner
- Runner ID`plugin:langbot/codex-agent/default`,执行方式:本地 Codex CLI读取 LangBot event context。
- Codex `thread_id` 写回 host-owned state支持 SDK-owned per-run LangBot MCP bridge需要代理的本地环境可通过 config 的 `environment-json` 显式传递非 secret 环境变量。
### 7.3 当前限制
不是发布级安全边界实现;默认只做本地 CLI 调用,不实现完整执行隔离或 workspace 生命周期;不实现 issue-centric 队列、复杂 workflow engine 或长期任务调度Docker 环境只能访问容器内 CLI 和凭据Codex 仅验证协议形态,不代表 Codex 发布级能力或 Kimi runner 已完成。runtime 管控面方向见 [RUNTIME_CONTROL_PLANE_V2.md](./RUNTIME_CONTROL_PLANE_V2.md)。
## 8. 发布和安装策略
最终 LangBot 安装/升级时需保证官方 runner 插件可用可选方案首次启动检测缺失并提示安装打包发行版预装migration 前检查插件存在性。建议顺序:开发阶段用本地路径插件 → 发布前支持 marketplace 安装 → 历史配置 migration 只在官方插件可用时执行 → 迁移期间保留旧内置 runner 文件,直到对应官方插件通过 parity 验收。
## 9. 验收标准
- 每个旧 runner 都有对应官方 AgentRunner 插件,旧配置能无损复制到新 `runner_config[id]`
- LangBot 主聊天路径不再通过 `RequestRunner` 执行业务 runner。
- 官方插件测试覆盖非流式、流式、错误、timeout、配置缺失。
- `local-agent` 能完成模型 fallback、tool calling、知识库检索、多模态输入、静态绑定 prompt 消费、history API 拉取、rerank。
- `claude-code-agent` 或同类 code-agent harness runner 能消费 event-first context、投影 scoped resources、保存 external session state并通过 WebUI Debug Chat smoke。
- 对外行为与旧内置 local-agent runner 一致;代码结构不需要相同。

View File

@@ -1,245 +0,0 @@
# Agent Runner QA 指南
本文档是 agent-runner 插件化下一轮测试的唯一 QA 入口。它合并并取代旧的 Phase 1 验收矩阵与 2026-05-18 / 2026-05-29 两份本地 QA 报告。
目标不是保留完整历史流水账,而是指导测试 agent 用最小但高价值的路径判断当前分支是否仍然健康。
## 1. 测试边界
当前主线验证的是 AgentRunner Protocol v1
```text
event -> binding -> runner.run(ctx) -> result stream
```
本指南验证:
- Host 能通过当前 Query entry adapter 进入 event-first `run(event, binding)` 主链路。
- Runner 来自插件 registry而不是旧内置 runner 分支。
- `local-agent` 能消费 Host 模型、工具、知识库、history、state、artifact 等基础设施。
- 外部 harness runnerClaude Code / Codex能消费 event-first context并把 session / working directory 等指针写回 host-owned state。
- 错误、权限裁剪、无输出、timeout 等路径不会破坏主聊天流程。
本指南不验证:
- Runtime Control Plane v2。
- EventGateway / EventRouter 完整落地。
- 发布级 path isolation、secret filtering、MCP allowlist、资源配额和 workspace cleanup。
- 所有外部服务 runner 的真实凭据联调。
这些属于后续能力或发布门槛,分别见 [RUNTIME_CONTROL_PLANE_V2.md](./RUNTIME_CONTROL_PLANE_V2.md) 与 [SECURITY_HARDENING.md](./SECURITY_HARDENING.md)。
## 2. 状态定义
测试报告只使用以下状态:
| 状态 | 含义 |
| --- | --- |
| PASS | 按步骤执行,用户可见行为和日志证据都满足通过条件。 |
| FAIL | 环境可用,但行为不满足通过条件。 |
| BLOCKED | 凭据、CLI、外部服务、测试数据或本地配置缺失导致无法执行。必须写清阻塞原因。 |
| N/A | 当前 runner 或平台明确不支持该能力。必须引用 manifest、文档或配置说明。 |
不能使用“看起来正常”“大概通过”“基本没问题”等模糊状态。
## 3. 执行顺序
推荐按以下顺序执行,前一层失败时不要继续扩大测试面:
1. Host / SDK / runner 单测。
2. WebUI 登录与 Pipeline Debug Chat 基础 smoke。
3. `local-agent` 高价值场景。
4. Claude Code / Codex 外部 harness smoke。
5. 权限和错误路径补充检查。
6. 汇总 PASS / FAIL / BLOCKED并给出下一步建议。
用户可见流程必须通过 WebUI 或真实消息平台验证。API / curl 只能作为诊断证据,不能单独让 UI case PASS。
## 4. 必跑基线
### 4.1 单测基线
在 LangBot 仓库运行:
```bash
uv run --frozen pytest tests/unit_tests/agent
```
如果本次改动只触及默认配置或 API service也至少补跑相关目标测试例如
```bash
uv run pytest tests/unit_tests/api/test_pipeline_service_defaults.py
```
通过条件:
- agent 单测全 PASS或失败项已确认与本次 agent-runner 路径无关。
- 若失败来自 `context_builder``orchestrator``session_registry``resource_builder``plugin/handler.py` 的 run action 权限路径,不应进入 UI smoke。
### 4.2 环境基线
`langbot-skills` 做环境检查:
```bash
cd "$LANGBOT_SKILLS_REPO"
bin/lbs env doctor
bin/lbs case list
```
`LANGBOT_SKILLS_REPO` 指向当前工作区里的 `langbot-skills` 仓库。优先使用已有 case而不是临时发明测试路径。
推荐首批 case
- `webui-login-state`
- `pipeline-debug-chat`
- `local-agent-basic-debug-chat`
- `local-agent-rag-debug-chat`(改动涉及 RAG / knowledge
- `local-agent-plugin-tool-call-debug-chat`(改动涉及 tool / resource policy
## 5. WebUI 主链路 Smoke
### 5.1 Runner registry
步骤:
1. 打开 WebUI Pipeline 配置页。
2. 查看 AI runner 下拉列表。
3. 选择 `plugin:langbot/local-agent/default`
4. 保存并刷新页面。
通过条件:
- runner 选项来自插件 registry。
- 保存后配置仍为 `ai.runner.id` + `ai.runner_config[id]`
- `runner_config` 表示 Agent/runner config不表示插件实例状态。
- 插件没有循环重启或 metadata 加载失败。
### 5.2 主聊天路径
步骤:
1. 使用绑定 `plugin:langbot/local-agent/default` 的 Pipeline。
2. 在 Debug Chat 发送确定性普通文本。
3. 查看 WebUI 回复和后端日志。
通过条件:
- 用户可见回复正常。
- 后端日志显示走 `AgentRunOrchestrator` / `RUN_AGENT`
- 不走旧内置 local-agent 主执行分支。
- conversation transcript 写入用户消息和助手消息。
## 6. `local-agent` 高价值测试
只保留最能覆盖架构边界的场景。
| ID | 场景 | 操作 | 通过条件 |
| --- | --- | --- | --- |
| LA-01 | 绑定 prompt | 配置 system prompt 后发送文本。 | runner 使用 `ctx.config.prompt`,不读取 `ctx.adapter.extra["prompt"]`;回复体现绑定 prompt。 |
| LA-02 | history API | 连续两轮对话,第二轮引用第一轮 marker。 | runner 通过 Host history API 或自管上下文读取历史,不依赖 inline history window。 |
| LA-03 | 流式 / 非流式 | 分别用支持流式和关闭流式的路径发送文本。 | 流式 UI 不重复、不空白;非流式只输出最终消息。 |
| LA-04 | 工具调用 | 绑定测试工具,发送会触发工具的 prompt。 | `ctx.resources.tools` 只包含授权工具;工具调用 started/completed最终回复包含工具结果。 |
| LA-05 | RAG | 绑定测试知识库,发送命中文档的 prompt。 | `ctx.resources.knowledge_bases` 包含所选知识库runner 通过授权 API 检索;回复使用检索内容。 |
| LA-06 | 多模态 | 发送图片输入。 | `ctx.input.contents` 保留图片;支持视觉模型时正常处理,不支持时受控失败。 |
| LA-07 | fallback / 错误 | 模拟 primary 模型失败或 runner 抛错。 | fallback 或 `run.failed` 行为受控;后续请求不受影响。 |
| LA-08 | 无输出保护 | 测试 runner 完成但不产出消息。 | 不产生空白成功回复;按受控失败或明确缺陷处理。 |
Rerank、remove-think、文件输入等场景只在本次改动直接涉及时补测不作为每轮必跑项。
## 7. 外部 Harness Runner Smoke
这些测试用于验证 Claude Code / Codex 这类自管 runtime 能走同一条 Host 协议路径。若本机没有 CLI、登录态或代理配置标记 BLOCKED不要伪造 PASS。
### 7.1 Claude Code runner
步骤:
1. 确认 `claude` CLI 在 LangBot runtime host 上可执行。
2. 绑定 `plugin:langbot/claude-code-agent/default`
3. 使用保守权限模式和确定性 prompt。
4. 在 Debug Chat 执行一次真实 smoke。
5. 检查 context / skill / MCP projection 和 host-owned state。
通过条件:
- WebUI 可见回复包含预期 sentinel。
- context JSON schema 为 `langbot.agent_runner.external_harness_context.v1` 或当前文档声明的等价 schema。
- context 包含 event、input、delivery、resources、context、state。
- 如启用 skills / MCP投影路径和配置可被 Claude Code 读取。
- `external.session_id` / `external.working_directory` 写入 host-owned state。
- CLI missing、nonzero exit、timeout、empty output 都转成受控 `run.failed`
### 7.2 Codex runner
步骤:
1. 确认 `codex` CLI 在 LangBot runtime host 上可执行。
2. 绑定 `plugin:langbot/codex-agent/default`
3. 如需要代理,使用 Agent/runner config 的 `environment-json` 显式传入。
4. 在 Debug Chat 执行一次真实 smoke。
5. 检查 JSONL 事件、last message、host-owned state。
通过条件:
- WebUI 可见回复包含预期 sentinel。
- Codex JSONL 至少包含 thread/session 起始事件、agent message、turn completed。
- `external.session_id` / `external.working_directory` 写入 host-owned state。
- timeout/cancel 不遗留 orphan CLI 子进程。
- CLI missing、nonzero exit、timeout、empty output 都转成受控 `run.failed`
### 7.3 API 型外部 runner
Dify、n8n、Coze、DashScope、Langflow、Tbox 等外部服务 runner 不作为每轮必跑项。只有在本次改动触及对应 runner 或凭据已经可用时执行 smoke。
通过条件:
- runner 可选,配置可保存。
- 请求成功,或外部服务错误被清晰返回。
- 外部服务凭据缺失时标记 BLOCKED并记录缺失项。
## 8. 权限与隔离补充
以下优先用单测 / targeted fixture 覆盖,不要求每次通过 UI 人工构造恶意 runner。
| 场景 | 推荐证据 |
| --- | --- |
| 未授权模型调用被拒绝 | `plugin/handler.py` run action 权限测试或目标单测。 |
| 未授权工具调用被拒绝 | `ctx.resources.tools` 与 host action 拒绝日志。 |
| 未授权知识库检索被拒绝 | `ctx.resources.knowledge_bases` 与 host action 拒绝日志。 |
| run_id 结束后复用被拒绝 | session registry 注销测试。 |
| 插件身份不匹配被拒绝 | `caller_plugin_identity` mismatch 测试。 |
| storage/state scope 越权被拒绝 | state/storage proxy 单测。 |
如果这些单测失败,不能用 WebUI 正常回复替代。
## 9. 证据要求
每轮测试报告至少记录:
- LangBot commit、SDK commit、相关 runner 插件 commit。
- Pipeline UUID/name、runner id、关键 runner config 摘要。
- WebUI 截图或 Playwright 操作记录。
- 后端日志中对应 query id / run id 的关键行。
- `langbot-skills` case/report 路径。
- 外部 harness runner 的 context 文件、session id、working directory、CLI 错误摘要。
- FAIL/BLOCKED 的复现步骤和归属仓库建议。
报告结论必须回答:
- 是否建议继续进入下一阶段测试。
- 是否存在主聊天路径阻塞。
- 是否只是凭据 / 外部服务 / 本机 CLI 缺失导致 BLOCKED。
- 是否需要进入 [SECURITY_HARDENING.md](./SECURITY_HARDENING.md) 的发布级验收。
## 10. 历史高价值记录
历史报告已合并为本指南,不再保留单独文档。后续若需要追溯,优先查看 `langbot-skills/reports/` 下的原始执行报告。
截至 2026-05-29已有本地 smoke 证明:
- `local-agent` 可以通过 Pipeline Debug Chat 走插件化 `AgentRunOrchestrator` 主链路。
- Claude Code runner 可以通过同一条 `run(event, binding)` 路径执行。
- Claude Code runner 可以读取 LangBot event-first context / skill / MCP 投影,并写回 `external.session_id` / `external.working_directory`
- Codex runner 可以通过同一条路径执行,并把 Codex `thread_id` 写回 host-owned state。
这些记录只证明本地协议闭环可用,不代表发布级 security hardening 已完成。

View File

@@ -1,160 +0,0 @@
# Agent Runner 插件化实现进度
本文档跟踪 Agent Runner 插件化的实现状态,便于快速了解当前进度。
> 本文是 agent-runner 插件化**实现状态的唯一事实源**。协议规范见 [PROTOCOL_V1.md](./PROTOCOL_V1.md)Host 架构见 [HOST_SDK_INFRASTRUCTURE.md](./HOST_SDK_INFRASTRUCTURE.md)。规范类文档不再各自维护"当前状态/✅"段落,状态一律以本文为准。
## 总体进度
**当前阶段**: Phase 3.5 已完成Event-first 基础设施已完成2026-05-29 已通过本地 `local-agent` 与 Claude Code runner smoke。
| Phase | 描述 | 状态 |
|-------|------|------|
| Phase 0 | PoC 验证 | ✅ 完成 |
| Phase 1 | 核心架构Registry、Orchestrator、上下文模型 | ✅ 完成 |
| Phase 2 | 权限、能力声明、资源注入 | ✅ 完成 |
| Phase 3 | 内置 runner 迁移到插件 | ✅ 完成7/7 |
| Phase 3.5 | Event-first 基础设施 | ✅ 完成 |
| Phase 3.6 | 外部 harness runner 协议 smoke | ✅ 完成Claude Code MVP |
| Phase 4 | EBA 事件支持 | 🔲 未开始(已预留 event-first 入口EventGateway 由其他分支实现) |
---
## 详细状态
### SDK 侧 (`langbot-plugin-sdk`)
| 组件 | 状态 | 备注 |
|------|------|------|
| `AgentRunner` 组件 | ✅ | `api/definition/components/agent_runner/runner.py` |
| `AgentRunContext` | ✅ | `api/entities/builtin/agent_runner/context.py` |
| `AgentRunResult` | ✅ | `api/entities/builtin/agent_runner/result.py` |
| `AgentRunnerCapabilities` | ✅ | `api/entities/builtin/agent_runner/capabilities.py` |
| `AgentRunnerPermissions` | ✅ | `api/entities/builtin/agent_runner/permissions.py` |
| EBA 事件模型 (Event/Actor/Subject) | ✅ | `api/entities/builtin/agent_runner/event.py` |
| `LIST_AGENT_RUNNERS` action | ✅ | `runtime/io/handlers/control.py` |
| `RUN_AGENT` action | ✅ | `runtime/io/handlers/control.py` |
| `AgentRunAPIProxy` | ✅ | `api/proxies/agent_run_api.py` |
| Pull API handlers (State/History/Event/Artifact) | ✅ | `runtime/io/handlers/plugin.py` |
| `caller_plugin_identity` injection | ✅ | Pull API handlers inject caller identity |
### LangBot 侧
| 组件 | 状态 | 备注 |
|------|------|------|
| `AgentRunnerRegistry` | ✅ | `pkg/agent/runner/registry.py` |
| `AgentRunOrchestrator` | ✅ | `pkg/agent/runner/orchestrator.py` - event-first `run(event, binding)` |
| `AgentRunnerDescriptor` | ✅ | `pkg/agent/runner/descriptor.py` |
| `AgentResourceBuilder` | ✅ | `pkg/agent/runner/resource_builder.py` |
| `AgentRunContextBuilder` | ✅ | `pkg/agent/runner/context_builder.py` - event-first context |
| `AgentResultNormalizer` | ✅ | `pkg/agent/runner/result_normalizer.py` |
| `ConfigMigration` | ✅ | `pkg/agent/runner/config_migration.py` |
| `QueryEntryAdapter` | ✅ | `pkg/agent/runner/query_entry_adapter.py` - Query → Event + Binding |
| `run_from_query()``run(event, binding)` | ✅ | Pipeline 路径委托到 event-first path |
| `ChatMessageHandler` 集成 | ✅ | 使用 orchestrator 替代 wrapper |
| `PipelineService` 集成 | ✅ | 从 registry 获取 runner metadata |
| Plugin connector | ✅ | `list_agent_runners()` / `run_agent()` |
| `EventLogStore` | ✅ | `pkg/agent/runner/event_log_store.py` |
| `TranscriptStore` | ✅ | `pkg/agent/runner/transcript_store.py` |
| `ArtifactStore` | ✅ | `pkg/agent/runner/artifact_store.py` |
| `PersistentStateStore` | ✅ | `pkg/agent/runner/persistent_state_store.py` |
| History / Event pull APIs | ✅ | Orchestrator + APIProxy |
| Artifact pull APIs | ✅ | Orchestrator + APIProxy |
| State pull APIs | ✅ | Orchestrator + APIProxy |
| `artifact.created` / `state.updated` handling | ✅ | Event-first handlers in orchestrator |
| Pipeline path host capability coverage | ✅ | EventLog/Transcript/ArtifactStore/PersistentStateStore |
| External harness state handoff | ✅ | `external.session_id` / `external.working_directory` 写入 PersistentStateStore |
### 官方插件
> 外部服务插件仓库:`/home/glwuy/langbot-app/langbot-agent-runner/`
> 本地 Local Agent 插件仓库:`/home/glwuy/langbot-app/langbot-local-agent/`
| 插件 | 状态 | 备注 |
|------|------|------|
| `local-agent` | ✅ 已完成 | 核心功能:模型、工具、知识库、流式、会话 |
| `dify-agent` | ✅ 已完成 | 支持 chat/agent/workflow 三种应用类型 |
| `n8n-agent` | ✅ 已完成 | Webhook 调用,支持 basic/jwt/header 认证 |
| `coze-agent` | ✅ 已完成 | 多模态输入,思维链处理 |
| `claude-code-agent` | ✅ MVP smoke 通过 | 本地 Claude Code CLIcontext / skill / MCP 投影host-owned resume state |
| `dashscope-agent` | ✅ 已完成 | 阿里云百炼,支持 agent/workflow 两种模式 |
| `langflow-agent` | ✅ 已完成 | SSE 流式tweaks 配置支持 |
| `tbox-agent` | ✅ 已完成 | 蚂蚁百宝箱,多模态输入 |
**注意**: LangBot 内置 runner`pkg/provider/runners/`)已停用,文件顶部添加了 DEPRECATED 注释。
### 本地验收
| 日期 | 范围 | 状态 | 证据 |
|------|------|------|------|
| 2026-05-29 | `local-agent` Pipeline Debug Chat | ✅ PASS | `langbot-skills/reports/2026-05-29-17-59-00-462-08-00-pipeline-debug-chat.md` |
| 2026-05-29 | `claude-code-agent` Pipeline Debug Chat | ✅ PASS | `langbot-skills/reports/2026-05-29-18-03-31-169-08-00-pipeline-debug-chat.md` |
| 2026-05-29 | Claude Code context / skill / MCP projection | ✅ PASS | `langbot-skills/reports/claude-code-agent-resource-context-20260529.md` |
| 2026-05-29 | Claude Code resume state | ✅ PASS | `langbot-skills/reports/claude-code-agent-real-workdir-20260529.md` |
| 2026-05-29 | `codex-agent` Debug Chat + thread_id resume state | ✅ PASS | 见 [PHASE1_QA_ACCEPTANCE_MATRIX.md](./PHASE1_QA_ACCEPTANCE_MATRIX.md) §10 / `langbot-skills/reports/` |
---
## 未完成但仍属本分支收尾
以下项目属于本分支收尾工作:
- [x] Smoke / manual validation — `local-agent`、Claude Code MVP、Codex MVP 已通过本地 WebUI smoke
- [ ] Docs final QA
- [ ] Claude Code runner 文档、安装和 marketplace 发布准备
---
## 非本分支范围
以下能力由其他分支负责:
| 能力 | 负责分支 | 备注 |
|------|----------|------|
| EventGateway implementation | event branch | 完整事件网关、事件路由、持久化管理 |
| Event subscription / notification | event branch | 事件订阅、推送通知 |
| BindingResolver persistence UI | 其他模块 | 绑定配置的持久化 UI |
| Event router integration | event branch | 与 BindingResolver 集成 |
| Scheduler / background event source | 其他模块 | 定时任务、后台事件源 |
| Security release hardening | 后续 release gate | 路径隔离、权限边界、secret、MCP/skill 投影策略、资源配额、审计 |
| Codex / Kimi runner 全量接入 | 后续 runner 插件工作 | Codex MVP 已打通Codex 发布级能力、Kimi runner 和全量 hardening 仍不扩大到当前协议闭环 |
| Issue-centric 产品模型 / 异步队列 / workflow engine | 后续产品架构 | 不属于当前 agent-runner plugin 协议闭环 |
---
## 待办事项
### 高优先级
- [x] 工具详情 API — SDK `GET_TOOL_DETAIL` action、`AgentRunAPIProxy.get_tool_detail()` 与 Host 侧授权校验已接通
- [x] Pipeline `run_from_query()``run(event, binding)` — 已完成
- [x] EventLog / Transcript / ArtifactStore / PersistentStateStore — 已完成
- [x] History / Event / Artifact / State pull APIs — 已完成
- [x] `caller_plugin_identity` 验证路径 — 已完成
### 低优先级 / 未来
- [ ] EBA 完整集成 — EventGateway、event subscription、event notification 由其他分支实现
- [ ] 平台 API 动作执行 — `action.requested` 结果类型存在但未执行
- [ ] 安全发布级 hardening — 作为生产默认启用前的 release gate不阻塞当前协议闭环
---
## 关键决策记录
| 日期 | 决策 |
|------|------|
| 2026-05-10 | Phase 0 集成测试通过SDK v1 协议验证成功 |
| 2026-05-13 | Phase 3 完成:所有 7 个官方 runner 插件迁移完成 |
| 2026-05-23 | Phase 3.5 完成:`run_from_query()` 委托到 event-first `run(event, binding)`Pipeline path 获得 host capabilities |
| 2026-05-29 | 本地 `local-agent``claude-code-agent` 通过 WebUI smokeClaude Code runner 验证 external harness context 投影和 host-owned resume state |
---
## 相关文档
- [README.md](./README.md) — 总体设计与路由
- [PROTOCOL_V1.md](./PROTOCOL_V1.md) — 协议规范(唯一 schema 事实源)
- [PHASE1_QA_ACCEPTANCE_MATRIX.md](./PHASE1_QA_ACCEPTANCE_MATRIX.md) — Agent Runner QA 指南和下一轮测试入口
- [OFFICIAL_RUNNER_PLUGINS.md](./OFFICIAL_RUNNER_PLUGINS.md) — 官方插件仓库计划
- [SECURITY_HARDENING.md](./SECURITY_HARDENING.md) — 安全发布级 hardening 后续门槛

View File

@@ -1,531 +0,0 @@
# LangBot AgentRunner Protocol v1
本文档是 LangBot Host 与插件 SDK / Runtime / AgentRunner 之间协议合同的**唯一规范来源single source of truth**。
- 本文件描述"稳定接口应是什么",是 normative spec不混入实现进度。实现状态见 [PROGRESS.md](./PROGRESS.md)。
- 本文件之外的任何文档**不得重新定义这里的数据结构**,只能引用,例如"见 PROTOCOL_V1 §4.2"。
- Host 内部模型(`AgentEventEnvelope``AgentBinding`、Descriptor、各 Store不属于 SDK 协议,定义在 [HOST_SDK_INFRASTRUCTURE.md](./HOST_SDK_INFRASTRUCTURE.md)。
## 1. 协议目标
Protocol v1 只解决四件事:
- LangBot 如何发现插件提供的 AgentRunner。
- LangBot 如何把一次事件调用封装成 `AgentRunContext`
- AgentRunner 如何以事件流形式返回运行结果。
- AgentRunner 如何通过受限 API 访问 LangBot host 能力。
Protocol v1 **不定义**
- LangBot 内部如何持久化 `AgentBinding`(见 HOST_SDK
- AgentRunner 内部如何组装 prompt、压缩历史、管理 memory见 [AGENT_CONTEXT_PROTOCOL.md](./AGENT_CONTEXT_PROTOCOL.md))。
- 官方 runner 的具体实现(见 [OFFICIAL_RUNNER_PLUGINS.md](./OFFICIAL_RUNNER_PLUGINS.md))。
- Pipeline 的长期配置模型。
- 发布级安全 hardening 的完整实现(见 [SECURITY_HARDENING.md](./SECURITY_HARDENING.md))。
## 2. 参与方
| 名称 | 职责 |
| --- | --- |
| LangBot Host | 事件入口、绑定解析、权限、资源、存储、生命周期、结果投递。 |
| Plugin Runtime | 加载插件,响应 Host 的 runner discovery 和 run 调用。 |
| AgentRunner | 插件提供的 agent 执行组件。 |
| AgentRunAPIProxy | AgentRunner 访问 Host 能力的受限 API。 |
| AgentBinding | Host 内部的事件到 runner 绑定配置,不直接暴露给 SDK见 HOST_SDK §4.2)。 |
产品层的 `Agent` 替代旧 Pipeline 承载 agent 配置bot / IM channel
绑定一个 Agent一个 Agent 可以被多个 bot / channel 复用。Host 内部的
`AgentBinding` 是一次事件运行前解析出的有效绑定,只影响 Host 构造出的
`ctx.config``ctx.resources``ctx.context``ctx.delivery`。SDK 不需要知道
Agent / binding 的持久化形态。
外部 harness runnerClaude Code、Codex、Kimi Code 等)也是 `AgentRunner`:它们消费 event-first `AgentRunContext`、返回 `AgentRunResult`,并通过 Host 授权的 state/storage/artifact API 保存跨轮次指针。它们内部可以继续使用自己的 session、tool loop、MCP、上下文压缩和权限模型。
## 3. 版本协商
- `AgentRunnerManifest.protocol_version` 声明 runner 实现的协议大版本,当前为 `"1"`
- `AgentRuntimeContext.protocol_version``ctx.runtime.protocol_version`)声明 Host 下发的协议大版本。
- Host 发现 runner 时校验 `protocol_version` 兼容性;不兼容的 runner 不进入可用列表,只记 warning。
- 字段级演进规则:新增可选字段不提升大版本;删除或改语义需要提升大版本。
- 结果流演进Host **必须忽略未知 result type 并记录 warning**(除非该 type 明确要求强校验)。新增 result type 不提升大版本。
## 4. Discovery 协议
### 4.1 LIST_AGENT_RUNNERS
Host 调用 Plugin Runtime 获取当前插件暴露的 runner 列表,请求无额外 payload。返回
```python
class ListAgentRunnersResponse(BaseModel):
runners: list[AgentRunnerManifest]
```
### 4.2 AgentRunnerManifest
```python
class AgentRunnerManifest(BaseModel):
id: str
name: str
label: I18nObject
description: I18nObject | None = None
protocol_version: str = "1"
capabilities: AgentRunnerCapabilities
permissions: AgentRunnerPermissions
context: AgentRunnerContextPolicy
config_schema: list[DynamicFormItemSchema] = []
metadata: dict[str, Any] = {}
```
- `id` 必须稳定,格式 `plugin:author/name/runner`
- `name` 是插件内 runner 名称,例如 `default`
- `config_schema` 只描述绑定配置表单,不代表插件实例状态。
- `metadata` 只放展示、诊断、非稳定扩展信息。
### 4.3 Capabilities
```python
class AgentRunnerCapabilities(BaseModel):
streaming: bool = False
tool_calling: bool = False
knowledge_retrieval: bool = False
multimodal_input: bool = False
event_context: bool = True
platform_api: bool = False
interrupt: bool = False
stateful_session: bool = False
self_managed_context: bool = True
```
语义:
- `streaming`: runner 可以返回 `message.delta`
- `tool_calling`: runner 可能调用 Host tool API。
- `knowledge_retrieval`: runner 可能调用 Host knowledge API。
- `multimodal_input`: runner 可以处理非纯文本 input / artifact。
- `event_context`: runner 理解 event-first 输入。
- `platform_api`: runner 可能请求平台动作。
- `interrupt`: runner 支持取消或中断。
- `stateful_session`: runner 可能维护跨 run 会话状态。
- `self_managed_context`: runner 自己管理 working contextHost 不应默认 inline 历史。
> Capabilities 字段全部是 `bool`。runner 是否寄宿 host-owned state **不在 capabilities 表达**,而通过 `permissions.storage` 声明(见 §4.4),避免出现非 bool 取值。
### 4.4 Permissions
```python
class AgentRunnerPermissions(BaseModel):
models: list[Literal["invoke", "stream", "rerank"]] = []
tools: list[Literal["detail", "call"]] = []
knowledge_bases: list[Literal["list", "retrieve"]] = []
history: list[Literal["page", "search"]] = []
events: list[Literal["get", "page"]] = []
artifacts: list[Literal["metadata", "read"]] = []
storage: list[Literal["plugin", "workspace", "binding"]] = []
files: list[Literal["config", "knowledge"]] = []
platform_api: list[str] = []
```
Manifest permissions 是 runner 需要的**最大能力**。实际可用资源还要经过 Host binding policy 和当前 run scope 裁剪(三层裁剪见 HOST_SDK §4.5)。
### 4.5 Context Policy
```python
class AgentRunnerContextPolicy(BaseModel):
supports_history_pull: bool = True
supports_history_search: bool = False
supports_artifact_pull: bool = True
owns_compaction: bool = True
wants_static_context_refs: bool = True
```
Host 不使用该声明给 runner inline 历史窗口。默认原则:
- Host 不得默认 inline 全量历史。
- Host 只 inline 当前 event / input 和 context handles。
- Runner 拥有 working context assembly。
- Runner 可在授权后通过 Host history / event / artifact / state API 拉取更多上下文。
- 历史窗口策略不属于 Protocol v1 字段,也不属于 Host 通用语义。
context 边界的设计理由见 [AGENT_CONTEXT_PROTOCOL.md](./AGENT_CONTEXT_PROTOCOL.md)。
## 5. Run 协议
### 5.1 RUN_AGENT
Host 调用 Runtime
```python
class AgentRunRequest(BaseModel):
runner_id: str
runner_name: str
context: AgentRunContext
```
Runtime 返回 `AgentRunResult` 异步流。底层 transport 可继续用 `plugin_author` / `plugin_name` / `runner_name` 定位组件,但协议语义以 `runner_id``context` 为准。
### 5.2 AgentRunContext
这是 SDK 看到的**唯一权威 context 定义**。
```python
class AgentRunContext(BaseModel):
run_id: str
trigger: AgentTrigger
event: AgentEventContext
conversation: ConversationContext | None = None
actor: ActorContext | None = None
subject: SubjectContext | None = None
input: AgentInput
delivery: DeliveryContext
resources: AgentResources
context: ContextAccess
state: AgentRunState
runtime: AgentRuntimeContext
config: dict[str, Any] = {}
adapter: AdapterContext | None = None
metadata: dict[str, Any] = {}
```
核心约束:
- `event` 是必选字段Protocol v1 是 event-first。
- `input` 表示当前事件的主输入,不等于历史消息。
- `bootstrap` / `messages` **不是协议字段**Host 不内联历史窗口。
- `adapter` 只放入口 adapter 的非核心元数据runner 不应依赖它做长期能力。
- `config` 是 Agent/runner config不是插件实例状态。
### 5.3 AgentTrigger
```python
class AgentTrigger(BaseModel):
type: str
source: Literal["platform", "webui", "api", "scheduler", "system", "host_adapter"]
timestamp: int | None = None
```
`trigger.type` 应与 `event.event_type` 一致或更粗粒度。例如入口适配器触发消息时:
```json
{ "type": "message.received", "source": "host_adapter" }
```
### 5.4 AgentEventContext
```python
class AgentEventContext(BaseModel):
event_id: str
event_type: str
event_time: int | None = None
source: str
source_event_type: str | None = None
raw_ref: RawEventRef | None = None
data: dict[str, Any] = {}
```
- `event_type` 使用 LangBot 稳定协议名,例如 `message.received`。稳定事件名清单见 [EVENT_BASED_AGENT.md](./EVENT_BASED_AGENT.md)。
- 平台原始事件名放入 `source_event_type`
- 大型原始 payload 必须放入 `raw_ref` 或 artifact不应直接塞入 `data`
### 5.5 Conversation / Actor / Subject
```python
class ConversationContext(BaseModel):
conversation_id: str | None = None
thread_id: str | None = None
launcher_type: str | None = None
launcher_id: str | None = None
bot_id: str | None = None
workspace_id: str | None = None
class ActorContext(BaseModel):
actor_type: str
actor_id: str | None = None
actor_name: str | None = None
metadata: dict[str, Any] = {}
class SubjectContext(BaseModel):
subject_type: str
subject_id: str | None = None
data: dict[str, Any] = {}
```
示例:
- 消息事件actor 是发消息的人subject 是当前消息。
- 入群事件actor 是新成员或邀请人subject 是群/成员关系。
- 定时事件actor 可以是 systemsubject 是 schedule。
### 5.6 AgentInput
```python
class AgentInput(BaseModel):
text: str | None = None
contents: list[ContentElement] = []
attachments: list[ArtifactRef] = []
message_chain: dict[str, Any] | None = None
```
- 文本、多模态、附件都属于当前 event input。
- 大文件、图片、音频、工具大结果应以 artifact ref 传递。
- `message_chain` 是平台兼容字段,不应成为长期稳定依赖。
### 5.7 DeliveryContext
```python
class DeliveryContext(BaseModel):
surface: str
reply_target: dict[str, Any] | None = None
supports_streaming: bool = False
supports_edit: bool = False
supports_reaction: bool = False
max_message_size: int | None = None
platform_capabilities: dict[str, Any] = {}
```
Runner 可参考 delivery 能力决定返回 `message.delta``message.completed``action.requested`
### 5.8 ContextAccess
```python
class ContextAccess(BaseModel):
conversation_id: str | None = None
thread_id: str | None = None
latest_cursor: str | None = None
event_seq: int | None = None
transcript_seq: int | None = None
has_history_before: bool = False
inline_policy: InlineContextPolicy
available_apis: ContextAPICapabilities
class InlineContextPolicy(BaseModel):
mode: Literal["none", "current_event", "recent_tail", "summary_tail"]
delivered_count: int = 0
source_total_count: int | None = None
messages_complete: bool = False
reason: str | None = None
class ContextAPICapabilities(BaseModel):
history_page: bool = False
history_search: bool = False
event_get: bool = False
event_page: bool = False
artifact_metadata: bool = False
artifact_read: bool = False
state: bool = False
storage: bool = False
```
`ContextAccess` 告诉 runnerHost inline 了什么、没 inline 什么、需要更多上下文时走哪些 API。它是 runner 按需读取上下文的入口说明,不是 Host 的业务上下文编排策略。
### 5.9 AgentRuntimeContext
```python
class AgentRuntimeContext(BaseModel):
host: str = "langbot"
protocol_version: str = "1"
langbot_version: str | None = None
trace_id: str
deadline_at: float | None = None
locale: str | None = None
timezone: str | None = None
static_refs: dict[str, StaticContextRef] = {}
metadata: dict[str, Any] = {}
```
`static_refs` 用于 KV cache 友好的静态上下文引用system policy、tool schema、resource manifest 的 hash/version。理由见 AGENT_CONTEXT_PROTOCOL §6。
### 5.10 AgentRunState
```python
class AgentRunState(BaseModel):
conversation: dict[str, Any] = {}
actor: dict[str, Any] = {}
subject: dict[str, Any] = {}
runner: dict[str, Any] = {}
```
State 是可选 host-owned snapshot。Runner 也可以完全自管状态。
## 6. Resources
```python
class AgentResources(BaseModel):
models: list[ModelResource] = []
tools: list[ToolResource] = []
knowledge_bases: list[KnowledgeBaseResource] = []
files: list[FileResource] = []
storage: StorageResource = StorageResource()
platform_capabilities: dict[str, Any] = {}
```
资源列表是本次 run 的授权结果。History / Event / Artifact 访问通过 permissions、`ctx.context.available_apis` 和 Host 侧 run session 校验控制,不作为可枚举 resource list 暴露。Runner 只能通过 `AgentRunAPIProxy` 访问这些能力。
## 7. Result Stream
### 7.1 AgentRunResult
```python
class AgentRunResult(BaseModel):
run_id: str
type: str
data: dict[str, Any] = {}
sequence: int | None = None
timestamp: int | None = None
```
### 7.2 稳定 result types
| type | 说明 | 当前消费 |
| --- | --- | --- |
| `message.delta` | 流式消息片段。 | ✅ |
| `message.completed` | 完整消息。 | ✅ |
| `tool.call.started` | 工具调用开始的可观测事件。 | telemetry |
| `tool.call.completed` | 工具调用完成的可观测事件。 | telemetry |
| `artifact.created` | runner 生成 artifact。 | ✅ |
| `state.updated` | runner 请求更新 host-owned state。 | ✅ |
| `action.requested` | runner 请求 Host 执行平台动作。 | **reserved / 仅 telemetry不执行** |
| `run.completed` | run 正常结束。 | ✅ |
| `run.failed` | run 失败。 | ✅ |
`action.requested` 是为 EBA 和 platform API 预留的协议表面:当前阶段 Host 收到后只记 telemetry**不执行**runner 作者不应依赖其副作用。执行模型见 EVENT_BASED_AGENT §6。
### 7.3 示例
```json
{ "type": "message.delta", "data": { "chunk": { "role": "assistant", "content": "hel" } } }
{ "type": "message.completed", "data": { "message": { "role": "assistant", "content": "hello" } } }
{ "type": "state.updated", "data": { "scope": "conversation", "key": "external.session_id", "value": "abc" } }
{ "type": "action.requested", "data": { "action": "message.edit", "target": {"message_id": "..."}, "payload": {"text": "..."} } }
```
Host 必须校验 `state.updated` 的 scope、key、value 大小和 JSON 可序列化性。
## 8. AgentRunAPIProxy
所有 proxy action 必须携带 `run_id`。Host 必须校验active run session 存在、caller plugin identity 匹配、resource 在本次 `ctx.resources` 中授权、scope 不越界、payload size / rate limit / deadline 合法。
```python
# Model
await api.models.invoke(model_id, messages, tools=None, extra_args=None)
await api.models.stream(model_id, messages, tools=None, extra_args=None)
await api.models.rerank(model_id, query, documents, top_k=None)
# Tool
await api.tools.get_detail(tool_name)
await api.tools.call(tool_name, parameters)
# Knowledge
await api.knowledge.retrieve(kb_id, query_text, top_k=5, filters=None)
# History返回 Transcript projection不返回原始平台 payload
await api.history.page(conversation_id=None, before_cursor=None, after_cursor=None,
limit=50, direction="backward", include_artifacts=False)
await api.history.search(query, filters=None, top_k=10)
# Event返回稳定 event envelope 或受限 raw ref不默认返回大 payload
await api.events.get(event_id)
await api.events.page(before_cursor=None, limit=50)
# Artifact必须支持大小限制、MIME 校验、过期时间和授权范围)
await api.artifacts.metadata(artifact_id)
await api.artifacts.read_range(artifact_id, offset=0, length=65536)
await api.artifacts.open_stream(artifact_id)
# State / Storage
await api.state.get(scope, key); await api.state.set(scope, key, value); await api.state.delete(scope, key)
await api.storage.get(area, key); await api.storage.set(area, key, value)
await api.storage.delete(area, key); await api.storage.list(area, prefix=None)
# Platform受限能力默认不开放需 manifest + binding policy + 用户审批同时允许)
await api.platform.request_action(action, target, payload)
```
`state``storage` 的建议边界:`state` 放小型 JSONconversation / actor / runner / binding`storage` 放 blob 或较大数据插件私有数据、workspace 数据、checkpoint
返回数据结构(如 `HistoryPage`、artifact metadata见 AGENT_CONTEXT_PROTOCOL §4。
## 9. 错误模型
```python
class AgentAPIError(BaseModel):
code: str
message: str
retryable: bool = False
details: dict[str, Any] = {}
```
| code | 说明 |
| --- | --- |
| `unauthorized` | 未授权访问资源或 scope。 |
| `not_found` | 资源不存在或对当前 runner 不可见。 |
| `deadline_exceeded` | 超过 run deadline。 |
| `payload_too_large` | 请求或响应过大。 |
| `rate_limited` | Host 限流。 |
| `invalid_argument` | 参数错误。 |
| `runtime_error` | Host 或下游能力错误。 |
Runner 失败使用 `run.failed`
```json
{ "type": "run.failed", "data": { "code": "runner.error", "message": "failed to call external agent", "retryable": false } }
```
## 10. Timeout 与 Cancellation
- Host 在 `ctx.runtime.deadline_at` 下发总 deadlineSDK proxy 必须用该 deadline 限制单次 action timeout。
- Host 可以取消 active runRuntime 应尽力中断 runner。
- Runner 支持中断时应返回或触发 `run.failed`code 为 `cancelled`
- Host 必须 unregister active run session。
## 11. Security 与 Guardrail协议层
Protocol v1 的安全边界在 Host
- Runner 不能直接访问未授权 model/tool/kb/history/artifact/storage。
- SDK 本地校验只提升开发体验,不能替代 Host 校验。
- 所有 resource id 对 runner 来说都是 opaque。
- 默认只能访问当前 conversation / thread 的 history跨会话、workspace 级访问必须额外授权。
- 大 payload 必须 artifact 化。
- Host 必须记录 run_id、runner_id、action、resource、scope、result。
Host 不负责业务编排:不拼接全量历史、不替 runner 做 prompt assembly、不内置 agent memory / tool loop / 上下文压缩策略。这些由官方或第三方 AgentRunner 插件实现。
对外部 harness runnerHost 在调用前完成 binding/resource policy 裁剪、路径策略、secret 过滤和审计runner plugin 把授权后的 context/resource projection 适配为目标 harness 的形式harness 的 native permission mode、allowed/disallowed tools 只是额外执行约束,不能替代 Host 授权。
> 发布级路径隔离、MCP allowlist、secret redaction、配额、workspace 清理等**不属于** v1 协议闭环,是生产默认启用前的 release gate见 [SECURITY_HARDENING.md](./SECURITY_HARDENING.md)。
## 12. Pipeline Adapter 边界
Pipeline 是当前入口 adapter不是协议中心。目标产品模型中 Agent 会替代
Pipeline 承载 runner config、resource policy 和 delivery policy当前 Query
entry adapter 只是迁移桥。它负责:
-`Query` 构造 `AgentEventContext` 和临时 `AgentBinding`(见 HOST_SDK §4.2)。
- 从当前 Agent/runner config 构造 `ctx.config`
- 将 Query-only 字段放入 `ctx.adapter`,例如 filtered params 放 `ctx.adapter.extra["params"]`
约束:
- adapter **不**定义历史窗口、prompt 组装或 agentic context 策略。
- `ctx.adapter.extra` 只允许承载一次性、JSON-safe、入口相关的非核心元数据例如 `params`;不得承载 `prompt`、history window、RAG 结果、tool schema 或授权资源。
- 静态绑定 prompt 属于 `ctx.config.prompt`。preprocessing / hook 后的动态有效指令不通过 `ctx.adapter.extra` 主动推送;后续如需要保留这类能力,应通过 Host prompt/instruction pull API 暴露(占位见 HOST_SDK §4.8)。
- 新 runner 不应长期依赖 `adapter`,应只依赖 event-first context 和 Host API。
## 13. 已确认约束
- v1 / EBA 主线是 `one event -> one AgentBinding -> one run_id -> one runner`
- 一个 bot / IM channel 在同一时间只绑定一个负责 agentic 处理的 Agent一个 Agent 可以被多个 bot / channel 复用。
- 如果配置层出现多个匹配 AgentBindingBindingResolver 必须按明确规则选出一个或拒绝配置,不应默认 fan-out。
- observer agent、多 runner fan-out、并行裁决、result 合并等能力需要单独设计 delivery、state、platform action 和 audit 语义,不属于当前 v1 契约。
- `AgentRunnerDescriptor.source` 只允许 `plugin`Host 内置 adapter 不能作为 runner source 绕过插件/runtime/proxy 权限链。
- `ctx.resources` 与 proxy action 校验必须来自同一个 run authorization snapshotruntime handler 不应重新执行资源裁剪。
- 外部 harness runner 当前是 MVP / dev path证明协议可接入不代表发布级安全边界或 Docker 生产可用性完成。
## 14. 开放问题
- `AgentBinding` 是否需要进入 SDK 文档作为只读诊断信息,还是完全 Host 内部。
- `TranscriptItem` 的最小字段集如何定义。
- ArtifactStore 是否复用现有 BinaryStorage backend还是引入独立实体。
- State 与 Storage 的边界是否需要更强类型。
- `platform_api` action 的审批模型如何表达。
- Host 侧 scoped MCP / skill / workspace projection 是否需要从 runner config 上移为一等 resource projection API。

View File

@@ -1,147 +0,0 @@
# Agent Runner 插件化文档入口
本文档是 agent-runner 插件化工作的路由页。具体设计拆到独立文档中维护,避免把 LangBot 宿主架构、SDK 协议、上下文管理、EBA 预留和官方 runner 迁移混在同一份 README 里。
## 文档维护原则(单一事实源)
- **协议数据结构schema唯一定义在 [PROTOCOL_V1.md](./PROTOCOL_V1.md)。** 其他文档不得重抄 schema只能引用例如"见 PROTOCOL_V1 §4.2"。
- **实现状态唯一记录在 [PROGRESS.md](./PROGRESS.md)。** 规范类文档不维护"当前状态/✅"段落。
- Host 内部模型(`AgentEventEnvelope``AgentBinding`、Descriptor、各 Store定义在 [HOST_SDK_INFRASTRUCTURE.md](./HOST_SDK_INFRASTRUCTURE.md),不属于 SDK 协议。
- 其余专题文档只讲"为什么/边界/怎么用",避免重复叙述。
## 本分支目标
**本分支目标AgentRunner 外化 / 插件化基础设施**
本分支只做 LangBot 作为 Agent Host 的基础能力建设,为后续用 `Agent`
替代 Pipeline 承载 agent 配置打底:
- LangBot 与 SDK 的稳定协议合同Protocol v1
- Host-side `AgentEventEnvelope` / `AgentBinding` 模型
- `run(event, binding)` event-first 入口
- `QueryEntryAdapter`Query → AgentEventEnvelope + AgentBinding
- EventLog / Transcript / ArtifactStore / PersistentStateStore
- History / Event / Artifact / State pull APIs
- SDK runtime forwarding pull APIs + `caller_plugin_identity` 验证路径
## 本分支不实现
以下能力由其他分支负责,本分支只预留 integration point
- **EventGateway**:完整事件网关实现、事件路由、事件持久化管理
- **Event subscription / Event notification**:事件订阅、推送通知
- **BindingResolver persistence UI**:绑定配置的持久化 UI 和 event router 集成(如由其他模块负责)
- **Scheduler / Background event source**:定时任务、后台事件源
- **Runtime control plane v2**runtime registry、heartbeat、task queue、daemon claim、progress/cancel 和 runtime audit
EventGateway 在本文档中描述为 **future integration point**,由外部 event branch 提供。本分支只定义 host-side envelope/binding models 和 `run(event, binding)` orchestrator 入口。
## 目标产品模型
未来产品层应把 `Agent` 理解为 Pipeline 的替代物:原先 bot 绑定
PipelinePipeline 携带 agent/provider/RAG/tool 等配置;后续应改为 bot 或
IM channel 绑定一个 AgentAgent 携带 runner id、runner config、
resource/state/delivery policy 等 agent 配置。
约束:
- 一个 bot / IM channel 在同一时间只绑定一个负责 agentic 处理的 Agent。
- 一个 Agent 可以被多个 bot / channel 复用,类似旧 Pipeline 可被多个 bot 共享。
- Agent 配置是运行绑定配置,不是插件实例状态;多个 Agent 指向同一
AgentRunner 时不创建多个插件实例。
- 当前 Pipeline path 只是迁移期入口 adapter它把旧 Pipeline 配置投影为临时
`AgentBinding`,不代表目标架构仍由 Pipeline 承载 agent 语义。
## 当前状态
**当前 Pipeline 是入口 adapter不再是 agent runner 设计核心。**
主入口仍可由 Pipeline 触发,但内部已转换成 event-first path`run_from_query()``QueryEntryAdapter``Query` 转换为 `AgentEventEnvelope` + `AgentBinding`,再委托到统一的 `run(event, binding, ...)`。Pipeline path 因此获得了 event-first host capabilitiesEventLog / Transcript / ArtifactStore / PersistentStateStore 写入History / Event / Artifact / State pull API 可用)。
详细实现进度、已验收能力和未完成收尾见 [PROGRESS.md](./PROGRESS.md)。
## 设计文档
| 文档 | 关注点 |
| --- | --- |
| [PROTOCOL_V1.md](./PROTOCOL_V1.md) | **🔒 唯一 schema 事实源**。LangBot Host 与 SDK / Runtime / AgentRunner 的协议合同版本协商、discovery、run context、result stream、proxy actions、错误和 adapter 边界。 |
| [HOST_SDK_INFRASTRUCTURE.md](./HOST_SDK_INFRASTRUCTURE.md) | LangBot 宿主能力与分层架构、Host 内部模型(`AgentEventEnvelope` / `AgentBinding` / Descriptor / 各 Store、runner 发现、绑定、资源授权、状态、存储、生命周期和调用链。 |
| [AGENT_CONTEXT_PROTOCOL.md](./AGENT_CONTEXT_PROTOCOL.md) | Agent-owned context 方向:事件到来时 LangBot 传什么agent 如何按需拉取更多历史 / artifact / state以及如何支持 KV cache 友好的上下文管理。 |
| [EVENT_BASED_AGENT.md](./EVENT_BASED_AGENT.md) | EBA 预留:事件模型、事件来源、触发绑定、非消息事件如何复用 AgentRunner 调度。**标注为 future design note**。 |
| [RUNTIME_CONTROL_PLANE_V2.md](./RUNTIME_CONTROL_PLANE_V2.md) | Agent Platform v2 / runtime 管控面预留Host 新增 runtime registry、heartbeat、task queue、daemon 执行和 audit管理插件构建在这些 Host 能力之上。**标注为 future design note**。 |
| [OFFICIAL_RUNNER_PLUGINS.md](./OFFICIAL_RUNNER_PLUGINS.md) | 官方 runner 插件迁移,包括 local-agent 和外部 runner。它是下游落地计划不是 LangBot 基础能力设计的前置约束。 |
| [PHASE1_QA_ACCEPTANCE_MATRIX.md](./PHASE1_QA_ACCEPTANCE_MATRIX.md) | Agent Runner QA 指南:保留最高价值测试路径,指导 agent 开展下一轮 WebUI / runner smoke 验证。 |
| [SECURITY_HARDENING.md](./SECURITY_HARDENING.md) | 安全发布级 hardening 的后续发布门槛路径隔离、权限边界、secret、资源配额、MCP / skill 投影和审计。 |
| [PROGRESS.md](./PROGRESS.md) | **🔒 唯一状态事实源**。当前实现进度、已验收能力、未完成收尾和非本分支范围。 |
## 工作拆分
### 1. LangBot + SDK 基础设施
目标是把 LangBot 从内置 runner 执行器变成 agent host
- LangBot 与 SDK 的稳定协议合同
- runner manifest / descriptor / registry
- Agent / binding 配置解析
- run orchestration 和生命周期管理
- resource authorization 与 `run_id` 级权限校验
- host-owned state / storage / event log / transcript / artifact 能力
- SDK `AgentRunner``AgentRunContext``AgentRunResult``AgentRunAPIProxy`
协议合同详见 [PROTOCOL_V1.md](./PROTOCOL_V1.md)。
详见 [HOST_SDK_INFRASTRUCTURE.md](./HOST_SDK_INFRASTRUCTURE.md)。
### 2. Agent-owned context
LangBot 不应成为最终 agentic context manager。它应提供事实源、默认上下文引用和按需读取 APIagent 或其背后的 runtime 负责历史剪裁、摘要、召回和 KV cache 策略。
Host 不定义通用历史窗口字段或策略runner 通过 Host pull API 按需拉取历史并自行管理 working context。
详见 [AGENT_CONTEXT_PROTOCOL.md](./AGENT_CONTEXT_PROTOCOL.md)。
### 3. Event Based AgentFuture
消息只是事件的一种。后续 `message.received``message.recalled``group.member_joined``friend.request_received` 等事件都应能通过统一事件 envelope 触发 AgentRunner。
EBA 主线按单 Agent 调度设计EventRouter 对一个 bot / channel / scope
解析出一个有效 AgentBinding再调用一次 `AgentRunOrchestrator.run(event,
binding)`。多 agent fan-out、observer agent 或并行裁决不属于当前目标语义。
**本分支不实现 EBA 完整能力,只预留:**
- event-first envelope (`AgentEventEnvelope`)
- AgentBinding model
- `run(event, binding)` 入口
- QueryEntryAdapter当前 AgentEventEnvelope / AgentBinding 的 Query entry adapter source
详见 [EVENT_BASED_AGENT.md](./EVENT_BASED_AGENT.md)。
### 4. 官方 runner 插件
官方 `local-agent` 和外部 runner 迁移是下游工作。它们需要依附 LangBot 提供的宿主能力,但不应反过来决定宿主协议。
`local-agent` 可以外移,也可以重写。验收重点是它能完整消费 LangBot 的模型、工具、知识库、存储、事件、history API 和 result stream而不是保留旧内置 runner 的内部结构。
详见 [OFFICIAL_RUNNER_PLUGINS.md](./OFFICIAL_RUNNER_PLUGINS.md)。
### 5. Runtime Control Plane v2Future
当前 AgentRunner v1 主线只负责 `event -> binding -> runner.run(ctx) -> result stream`
后续 Agent Platform v2 可以在 Host 侧新增 runtime registry、heartbeat、task queue、daemon claim、progress/cancel 和 runtime audit。
在这些 Host 能力之上,可以构建独立 agent 管控面插件;插件负责 UI、策略和编排体验runtime/task 的事实源仍由 Host 持有。
详见 [RUNTIME_CONTROL_PLANE_V2.md](./RUNTIME_CONTROL_PLANE_V2.md)。
## 已确认决策
- 一个插件可以声明多个 `AgentRunner` 组件,每个组件独立暴露 manifest、配置 schema、能力和权限。
- 插件本身按单实例、无状态执行单元理解;不同绑定不创建多个插件实例。
- Agent / binding 只保存 runner id 和绑定配置,不代表插件实例状态。
- bot / IM channel 绑定一个 AgentAgent 可被多个 bot / channel 复用。
- LangBot 可以提供 host-owned state / storage 能力,让 runner 把状态寄宿在 LangBot但这应该是授权能力不是强制要求。
- 官方 runner 插件是协议消费者,不是协议设计的优先约束。
- Pipeline 是当前入口 adapter不是未来架构中心。
- Event dispatch 主线是 one event -> one AgentBinding -> one run_id -> one runner。
- EventGateway 是 future integration point由外部 event branch 提供。
- Runtime control plane 是 v2 Host capability layer不阻塞当前 AgentRunner v1 主线agent 管控面插件应构建在该 Host 能力层之上。

View File

@@ -1,227 +0,0 @@
# Agent Runtime Control Plane V2
本文档记录后续 Agent Platform / runtime 管控面的设计方向。它是当前讨论中的 **v2 文档**,但这里的 v2 指 Host capability layer / runtime control plane不是 `AgentRunner Protocol v2`,也不属于当前 AgentRunner Protocol v1 插件化主线的交付范围。
> **future design note**。协议数据结构见 [PROTOCOL_V1.md](./PROTOCOL_V1.md),实现进度见 [PROGRESS.md](./PROGRESS.md)。本文只讲 v2 管控面方向,不重抄 schema。
## 1. 结论
当前主线应继续收口 AgentRunner v1
```text
message/event -> binding -> runner.run(ctx) -> result stream
```
Runtime Control Plane v2 在 Host 侧新增 runtime control plane
```text
event -> task -> runtime selection -> daemon claim -> execute -> progress/audit/result
```
在 Runtime Control Plane v2 之上,可以构建独立的 agent 管控面插件。插件负责 UI、策略和编排体验runtime、task、heartbeat、audit 的事实源必须属于 LangBot Host而不是插件私有 storage。
## 2. 不影响 v1 主线
v2 不应改变 AgentRunner v1 的基本契约:
- 现有 `local-agent`、Dify、n8n、Coze 等 runner 仍可按 v1 直接执行。
- 当前 Claude Code / Codex MVP runner 可以继续作为本机 subprocess 开发路径。
- Host v1 已有的 event-first context、resource authorization、history / event / artifact / state / storage pull APIs 继续保留。
- Pipeline 仍只是当前入口 adapter不参与 v2 runtime 管控面的设计中心。
v2 只是在 Host 上新增一层可选能力。需要管控面的 runner 或管理插件可以声明使用它;不需要的 runner 不受影响。
## 3. 当前 Host 能力与缺口
当前 Host 已经具备 v2 的基础设施底座:
- `AgentEventEnvelope` / `AgentBinding`
- run-scoped resource authorization
- EventLog / Transcript / ArtifactStore / PersistentStateStore
- History / Event / Artifact / State / Storage pull APIs
- AgentRunner result stream 和受控错误回流
- Agent/runner config 与 host-owned state
这些能力足够支持一次 `runner.run(ctx)` 内的安全执行,但不足以承担完整 runtime 管控面。
v2 还需要 Host 新增:
- runtime registryruntime id、所属 workspace、所在机器、provider 能力、状态。
- capability discovery`claude` / `codex` / 其它 CLI 是否存在、版本、登录状态、执行隔离能力。
- heartbeat / livenessruntime 在线、忙闲、最后心跳、可用 slot。
- task queueenqueue、claim、start、progress、complete、fail、cancel。
- workspace mappingLangBot workspace / project 如何映射到 runtime 上的真实目录、仓库或挂载。
- secret / env projection按授权向 runtime 投影 token、代理、MCP 配置、技能和环境变量。
- runtime auditstdout、stderr、事件流、产物、失败原因、执行耗时、使用量。
- control API / UI选择 runtime、测试 runtime、查看状态、下线、取消任务、重试任务。
## 4. 角色边界
### 4.1 LangBot Host
Host 是事实源和控制面内核:
- 保存 runtime / task / heartbeat / audit 状态。
- 做权限校验、资源裁剪、workspace 绑定和审计。
- 决定任务是否可被某 runtime claim。
- 将执行结果统一回写到 event / transcript / artifact / state。
Host 不应内置具体 agent CLI 的复杂业务逻辑,也不应把某个官方 runner 的特殊行为提升为通用协议。
### 4.2 Agent 管控面插件
管理插件是 v2 control plane 的产品化管理层:
- 展示 runtime、agent、task、进度、失败、审计。
- 提供策略配置,例如默认 runtime、provider 偏好、并发限制、重试策略。
- 触发 runtime 测试、任务取消、任务重试、手动分配。
管理插件不应把 runtime/task 的事实源放进自己的 plugin storage。它应该调用 Host v2 API。
### 4.3 Runtime daemon / worker
Runtime daemon 负责真实执行:
- 在所在机器上检测 CLI 和版本。
- 管理工作目录、仓库、挂载、临时文件和进程。
- 从 Host claim 任务,执行后上报 progress / complete / fail。
- 将 stdout / stderr / artifacts / session id 回流 Host。
Claude Code、Codex、OpenCode、Gemini CLI 等 provider 适配逻辑应主要落在 daemon / worker 或 provider adapter 中。
## 5. 部署形态
### 5.1 uv / local embedded
用户用 `uv` 或源码直接启动 LangBot 时LangBot 进程所在机器就是 runtime host。
这种模式下可以直接检测用户主机上的 `claude``codex` 等 CLI也可以直接 subprocess 执行。它适合个人开发和本地 smoke但不应作为团队级管控面的唯一形态。
### 5.2 Docker embedded
用户用 Docker 启动 LangBot 时runtime host 是容器,不是宿主机。
因此:
- 只能检测容器内的 `claude``codex`
- 只能使用容器内的 HOME、PATH、凭据和挂载目录。
- 如果镜像未安装 CLI或未挂载认证文件 / workspaceCLI runner 会不可用。
Docker embedded 可以作为高级部署选项,但需要用户显式安装 CLI、挂载工作区和凭据。Host 不应假设 Docker 容器能自动访问宿主机 CLI。
### 5.3 Sidecar daemon
推荐的 v2 形态是 sidecar daemon
```text
LangBot Host (Docker or server)
<-> Runtime daemon on user host / worker host
-> claude / codex / other CLI
```
这种模式下LangBot 可以跑在 Docker 内runtime daemon 跑在宿主机或独立 worker 机器上。daemon 负责检测本机 CLI、持有本机凭据和工作区访问能力。
### 5.4 Remote runtime
团队场景可以使用远端 runtime
- 开发机、构建机、云主机或专用 worker。
- 多个 workspace 可绑定不同 runtime。
- Host 只通过 registry / task queue / heartbeat / audit 进行管理。
### 5.5 API-only agent
Dify、n8n、Coze、DashScope 等 API 型 runner 不依赖本地 CLI。它们可以继续按 v1 直接执行,也可以在未来按需要接入 v2 task/audit。
## 6. 与 Claude Code / Codex MVP runner 的关系
当前 Claude Code / Codex runner 是 v1 runner
```text
runner.run(ctx) -> subprocess("claude" / "codex")
```
它们适合验证 Host context 投影、state resume、result stream 和基础 CLI 调用,但有明确限制:
- 命令只在 LangBot runtime host 上执行。
- Docker 环境只能看到容器内 CLI。
- 没有 runtime registry、heartbeat、task queue、cancel、workspace lifecycle。
- 不提供发布级执行隔离、secret projection、团队级 audit。
v2 不需要删除这些 runner。它们可以继续作为 dev / MVP 路径存在。未来若接入管控面,可以增加 runtime-managed 执行模式:
```text
runner binding -> Host task -> runtime daemon -> provider CLI -> Host result
```
## 7. 最小 v2 API 草案
以下仅记录能力边界,不代表最终 API 命名。
Runtime
- `runtime.register`
- `runtime.heartbeat`
- `runtime.list`
- `runtime.get`
- `runtime.disable`
- `runtime.capabilities.report`
- `runtime.capabilities.probe`
Task
- `task.enqueue`
- `task.claim`
- `task.start`
- `task.progress`
- `task.complete`
- `task.fail`
- `task.cancel`
- `task.retry`
Workspace
- `runtime.workspace.bind`
- `runtime.workspace.unbind`
- `runtime.workspace.resolve`
Audit / artifacts
- `task.log.append`
- `task.artifact.create`
- `task.events.page`
这些 API 应由 Host 提供,并受 workspace、runtime、binding、actor 和 plugin identity 约束。
## 8. 管控面插件可以构建的能力
基于 v2 Host 能力,可以实现一个类似 Multica 的 agent 管控面插件:
- runtime 列表、在线状态、CLI 能力、版本、认证状态。
- agent profile 与 runtime/provider 绑定。
- 任务看板、任务详情、进度流、失败原因、重试和取消。
- workspace 到 runtime 目录 / 仓库的映射管理。
- provider capability 测试,例如 Claude Code / Codex 是否可执行。
- 审计视图输入、输出、工具、artifact、stdout/stderr、session id。
- 策略配置:并发、队列、默认 runtime、fallback runtime、权限模式。
该插件应该是 Host v2 的消费者,而不是 Host v2 的替代品。
## 9. 设计原则
- v1 先稳定v2 可选叠加。
- Host 保存事实源,插件提供管理体验。
- Runtime daemon 执行具体 CLI 和本机资源访问。
- Docker 不假设拥有宿主机 CLI需要 sidecar 或显式挂载。
- Pipeline 不进入 v2 控制面中心。
- 直接 subprocess runner 可保留,但只作为 local/dev/MVP 路径。
- 发布级能力必须经过 Host 权限、审计和资源边界。
## 10. 待定问题
- runtime daemon 与 Host 的认证模型workspace token、device token、还是 scoped PAT。
- task 与 AgentRunner binding 的映射关系:由 binding 直接 enqueue还是由独立 task policy 决定。
- runtime capability schema 的稳定字段provider、version、login status、execution isolation、workspace access、slot。
- secret projection 的边界Host 存储、用户本机存储、或外部 secret manager。
- Docker compose 是否提供官方 sidecar daemon 示例。
- v2 UI 是核心前端的一部分,还是完全由管理插件提供。

View File

@@ -1,74 +0,0 @@
# Agent Runner Security Hardening
本文档记录 agent-runner 插件化进入生产发布前需要补齐的安全与稳定加固项。
## 状态
**当前结论:暂不塞进本阶段 agent-runner plugin 协议闭环。**
本阶段目标是验证 LangBot 可以通过统一的 `run(event, binding)` 协议接入 `local-agent` 与外部 harness runner如 Claude Code runner并能传递事件、上下文、资源句柄、状态和结果流。
安全发布级 hardening 是后续 release gate不应阻塞当前协议闭环但必须作为进入生产默认启用前的验收条件。
> **硬规则**:能执行代码 / 访问工作目录的外部 harness runnerClaude Code、Codex、Kimi Code 等)在本文 Release Gate Checklist 完成前,**不得在生产环境默认启用**。本地 smoke 通过不等于可生产默认开启。
## 责任边界
### LangBot Host 负责
- 资源授权:决定某个 `run_id` / binding 可以访问哪些模型、RAG、MCP、skill、artifact、history、state。
- 资源投影:只把授权后的资源句柄、配置片段或上下文文件传给 runner。
- 路径策略:限制 workspace / context file / artifact 的允许路径和清理策略。
- Secret 策略:过滤环境变量、配置、日志和 transcript 中的 secret。
- 运行约束:配置超时、轮次、并发、配额、输出大小和取消路径。
- 审计记录记录事件、绑定、资源授权、runner 调用、外部 harness session id、关键错误和结果摘要。
### Runner Plugin 负责
- 遵守 LangBot 下发的 Agent/runner config、授权资源和运行约束。
- 将 LangBot 资源投影成目标 runner 可消费的形式,例如 context 文件、MCP 配置、环境变量或 CLI 参数。
- 不把长期状态保存在插件实例内;需要跨轮次保存的外部 session id / working directory 等状态应写入 host-owned state。
- 对外部进程做最小必要封装,包括命令参数构造、超时、取消、输出解析和错误映射。
### 外部 Harness 负责
Claude Code、Codex、Kimi Code 等外部 harness 可以继续使用自身的权限模型、工具 allow / deny 规则、MCP 加载策略、session/resume 机制和沙箱能力。
但外部 harness 不是 LangBot 的唯一安全边界。LangBot 仍必须在调用前完成资源授权、路径限制、secret 过滤和审计记录。
## 当前 MVP 可接受边界
当前阶段可以接受以下前提:
- 由可信管理员配置 runner binding。
- 工作目录和 context 输出目录为显式配置或 host 生成路径。
- 外部 runner 默认使用保守权限,例如 plan / no-write 模式或禁用高风险工具。
- 通过 timeout、max turns、输出长度和进程取消降低失控风险。
- 通过 host-owned state 保存 `external.session_id``external.working_directory` 等 resume 所需指针。
这些前提足够做本地 E2E 与协议验收,不等同于生产发布完成。
## Release Gate Checklist
进入生产默认启用前,需要补齐:
- Path isolationworkspace allowlist、路径规范化、防止 `..` 逃逸、context / artifact 清理。
- Permission boundaryrunner 能力声明、binding 级资源授权、run 级权限校验。
- Secret handling环境变量白名单、配置脱敏、日志和 transcript redaction。
- MCP policyMCP server allowlist、scoped token、tool allow / deny、危险工具审计。
- Skill projection policyskill 来源验证、只读投影、版本和摘要记录。
- Process isolation进程组管理、取消、超时、CPU / 内存 / 输出配额。
- State lifecyclesession id、workspace、artifact 的过期、清理、迁移和审计。
- Audit first-class事件、资源授权、外部命令、session id、结果摘要可追踪。
- UI / Admin control管理员能看到 runner 权限、风险提示、资源绑定和禁用入口。
- Test matrix路径逃逸、secret 泄漏、权限拒绝、timeout、取消、MCP deny、resume、cleanup、audit 完整性。
## 非当前范围
以下内容不属于本阶段协议闭环:
- 完整异步队列与 issue-centric 产品模型。
- 复杂 workflow engine。
- Codex / Kimi runner 全量接入。
- EBA 分支完整迁移和联调。
- 发布级安全 hardening 的完整实现。

View File

@@ -1,595 +0,0 @@
# Box 系统架构深度分析
> 更新日期: 2026-06-02
> 状态更新: 自部署社区版已具备发布条件box 可选、降级完善、无迁移欠债);工具调用循环上限、配额遍历异步化、`host_path` 挂载白名单等已落地。剩余多租户 / 安全硬化项见 [SaaS 阻塞项清单](./box-issues.md)。
> 分支: `feat/sandbox` (LangBot + langbot-plugin-sdk)
> 相关文档: [SaaS 阻塞项](./box-issues.md) | [Session 作用域](./box-session-scope.md) | [Runtime 对比](./box-vs-plugin-runtime.md) | [测试覆盖](./box-test-coverage.md) | [toB 分析](./box-tob-analysis.md)
---
## 1. 全局架构
```
┌──────────────────────────────────────────────────────────────────┐
│ LangBot 主进程 │
│ │
│ LocalAgentRunner ──> ToolManager ──> NativeToolLoader │
│ │ │ │ │
│ │ │ exec / read / write / edit │
│ │ │ glob / grep │
│ │ │ │
│ │ ├──> MCPLoader ──> BoxStdioSession │
│ │ │ (shared 容器, 多 process) │
│ │ │ │
│ │ ├──> SkillToolLoader (activate 工具) │
│ │ │ │
│ │ ├──> SkillAuthoringToolLoader │
│ │ │ │
│ │ └──> PluginToolLoader │
│ │ │
│ BoxService (门面) │
│ ├─ Profile 管理 (locked 字段) │
│ ├─ Host mount 校验 (allowed_mount_roots) │
│ ├─ Workspace quota 检查 │
│ ├─ 输出截断 (head+tail) │
│ ├─ Session ID 模板解析 (resolve_box_session_id) │
│ ├─ 技能挂载组装 (build_skill_extra_mounts) │
│ ├─ 重连循环 (_reconnect_loop, 指数退避) │
│ └─ BoxRuntimeConnector │
│ ├─ 心跳 loop (20s ping) │
│ └─ ActionRPCBoxClient │
│ │ Action RPC (stdio 或 WebSocket) │
│ │
│ SkillManager (skill_mgr) │
│ └─ 从 Box runtime 拉取 skills, 不可用时回落 data/skills │
└──────────────────────────────────────────────────────────────────┘
┌──────────────────────────────────────────────────────────────────┐
│ Box Runtime 进程 (SDK 侧) │
│ │
│ BoxServerHandler (Action RPC 处理, INIT 配置注入) │
│ │ │
│ BoxRuntime (session 管理 / 进程生命周期 / TTL reaper) │
│ │ └─ session.managed_processes: dict[pid, _ManagedProcess]
│ │ │
│ Backend (启动时根据 box.backend 配置选择): │
│ DockerBackend ──┐ │
│ PodmanBackend ──┤── CLISandboxBackend │
│ NsjailBackend ──┘ (本地 CLI 或 fallback 到容器内 CLI) │
│ E2BBackend (云沙箱, 需要 E2B_API_KEY) │
│ │
│ BoxSkillStore │
│ ├─ list / get / create / update / delete │
│ ├─ scan_skill_directory / read_skill_file / write_skill_file │
│ └─ preview_skill_zip / install_skill_zip (zip 或 GitHub) │
│ │
│ aiohttp 单端口服务 (默认 :5410): │
│ /rpc/ws — Action RPC │
│ /v1/sessions/{id}/managed-process/ws — 默认 process │
│ /v1/sessions/{id}/managed-process/{pid}/ws — 指定 process │
└──────────────────────────────────────────────────────────────────┘
┌──────────────────────────────────────────────────────────────────┐
│ 容器 / 沙箱 (Docker/Podman 容器, nsjail sandbox, 或 E2B 远程沙箱) │
│ - 隔离文件系统 / 网络 / PID 命名空间 │
│ - 资源限制 (CPU, 内存, PID 数, 可选 workspace 配额) │
│ - 主挂载 (host_path → mount_path) + 任意条 extra_mounts │
│ └─ Skills 通过 extra_mounts 挂在 /workspace/.skills/<name> │
│ - exec: 用户命令在此执行 │
│ - managed process: 多个长驻进程并存 (MCP Server / 自定义服务) │
└──────────────────────────────────────────────────────────────────┘
```
**核心设计原则**:
- Box Runtime 作为独立进程运行,通过 Action RPC 与 LangBot 主进程通信,两者复用 SDK 的 IO 层Handler → Connection → Controller
- 一个 session_id 对应一个容器/沙箱实例。同一 session 内可并存多条 mount 与多个 managed process
- Skill / 默认 exec / MCP Server 共享同一个 session 容器(详见 [box-session-scope.md](./box-session-scope.md)
---
## 2. LangBot 侧模块
### 2.1 BoxService (`pkg/box/service.py`, 722 行)
应用层门面,协调 Profile、安全校验、配额、连接、Skill 挂载与 Session 模板:
主要公开方法(按定义顺序):
```
BoxService
├─ initialize() 连接 Box Runtime + 默认 workspace 准备
├─ _on_runtime_disconnect(connector) 触发重连
├─ _reconnect_loop(connector) 指数退避重连
├─ available (property) 连接状态
├─ resolve_box_session_id(query) 从 pipeline 模板解析 session_id
├─ build_skill_extra_mounts(query) 组装 pipeline-bound skill 的挂载列表
├─ execute_tool(parameters, query) Agent 调用 exec 时的入口
│ ├─ _apply_profile / build_spec
│ ├─ _validate_host_mount
│ ├─ _enforce_workspace_quota (phase=pre)
│ ├─ client.execute(spec)
│ ├─ _enforce_workspace_quota (phase=post)
│ └─ _truncate (stdout/stderr)
├─ execute_spec_payload(spec_payload, ...) 内部入口(其他 loader 调用)
├─ create_session(spec_payload, ...) 显式创建 session
├─ start_managed_process(session_id, ...) 启动 managed process
├─ get_managed_process(session_id, pid) 查询进程状态pid 默认 'default'
├─ stop_managed_process(session_id, pid) 单独停止某个 managed process
├─ get_managed_process_websocket_url(...) 返回 WS attach URL
├─ list_skills() / get_skill(name) Skill 元数据
├─ create_skill / update_skill / delete_skill Skill CRUD
├─ scan_skill_directory(path) 扫描目录
├─ list_skill_files / read_skill_file / write_skill_file
├─ preview_skill_zip / install_skill_zip zip / GitHub 安装
├─ shutdown() / dispose() 清理RPC SHUTDOWN + 进程终止
├─ get_status() / get_sessions() / get_recent_errors()
└─ get_system_guidance() LLM 系统提示
```
**Profile 系统**: 4 个内置 Profile`default` / `offline_readonly` / `network_basic` / `network_extended``locked` frozenset 字段不可被 LLM 覆盖。参数合并顺序Profile defaults → LLM 请求参数 → locked 强制值。
**输出截断**: 默认 4000 字符上限,保留前 60% + 后 40%,中间插入 `[...truncated...]`
**Skill 挂载合并**: `execute_tool()` 调用时,`build_skill_extra_mounts(query)` 会把当前 pipeline-bound 的所有 skill 的 `package_root` 作为 `extra_mounts` 加入 BoxSpec挂在 `/workspace/.skills/<name>`。LLM 通过 `activate` 工具显式激活某个 skill 后,工具调用才允许引用这个 skill 的虚拟路径。
### 2.2 BoxRuntimeConnector (`pkg/box/connector.py`, 357 行)
管理与 Box Runtime 的通信连接:
- **本地 stdio**: Unix/macOS 默认路径fork `python -m langbot_plugin.cli.__init__ box -s --ws-control-port {port}` 子进程(与 plugin runtime 统一走 `lbp` CLI 入口)
- **本地 subprocess + WS**: Windows 本地asyncio ProactorEventLoop 不支持 stdio pipe
- **远程 WebSocket**: Docker 部署 / `box.runtime.endpoint` 显式配置时,连接 `ws://{host}:{port}/rpc/ws`
- **同步等待**: `asyncio.Event` + `wait_for(timeout=30s)` 模式确认连接
- **心跳**: `_heartbeat_loop()` 每 20s 调用 `ping()`,失败仅 DEBUG 日志(断开检测靠 connection close
- **重连**: `runtime_disconnect_callback` 由 BoxService 提供,触发 `_reconnect_loop`
- **INIT 注入**: 连接建立后立即下发当前 `box.*` 配置子树(剔除 `runtime` 私有字段Runtime 据此初始化 backend
> **历史改进**: 2026-04-16 版本本文档曾列 P0 「Box 无心跳 / 无重连」已修复commit `2dfd9d5d`、`c6882cf`、`5029d9c` 等)。
### 2.3 BoxWorkspaceSession 工具 (`pkg/box/workspace.py`, 413 行)
此文件目前提供两类能力:
1. **路径与命令重写工具函数**`normalize_host_path` / `rewrite_mounted_path` / `unwrap_venv_path` / `rewrite_venv_command` / `infer_workspace_host_path`,被 MCP loader 与 Skill 路径解析共用。
2. **`BoxWorkspaceSession`** — 围绕 BoxService 的轻量包装,专供 MCP-in-Box 场景使用(管理一个共享 session 的 session_id、构建挂载 payload、stage host 文件到共享 workspace
**变化点**: 早期 Skill exec 会为每个 skill 创建独立 BoxWorkspaceSession独占 session当前实现已转为 `extra_mounts` 模式Skill 不再独占容器,只追加挂载。这部分 wrapping 逻辑已从 native loader 移除。
### 2.4 policy.py (`pkg/box/policy.py`, 98 行) — 仍是死代码
三层安全策略设计(`SandboxPolicy` / `ToolPolicy` / `ElevatedPolicy`),全项目无任何导入或调用。详见 [SaaS 阻塞项 S2](./box-issues.md)。
### 2.5 SkillManager (`pkg/skill/manager.py`, 186 行)
```
SkillManager
├─ initialize() 调用 reload_skills()
├─ reload_skills() 先从 Box runtime list_skills()
│ 不可用则回落 data/skills/ 扫描
├─ refresh_skill_from_disk() 单 skill 重新加载
├─ get_skill_by_name(name)
└─ get_managed_skills_root() 返回 Box 视角的 skills_root 路径
```
skill 元数据通过 `parse_frontmatter` 解析 `SKILL.md` 头部(`name` / `description` / `instructions`),不再做整体扫描的代价(典型 < 50 个)。
### 2.6 Skill activation (`pkg/skill/activation.py`, 33 行) + Skill loader 辅助
历史上 skill 通过 LLM 在文本中输出 `[ACTIVATE_SKILL:name]` 标记激活;当前已改为 **Tool Call 机制**
- `SkillToolLoader` (`pkg/provider/tools/loaders/skill.py`, 157 行) 暴露 `activate` 工具,参数为 skill 名
- 工具实现调用 `register_activated_skill(query, skill_data)`,将激活态写入 `query.variables['_activated_skills']`
- 这种 KV-cache-friendly 模式对齐 Claude Code 设计;详见 [box-session-scope.md §4.3](./box-session-scope.md) 的 Tool Call 描述
`activation.py` 现仅保留对外辅助函数pipeline 层调用 loader 的 `register_activated_skill`)。
---
## 3. SDK 侧模块
### 3.1 BoxRuntime (`box/runtime.py`, 599 行)
核心编排器,管理 session 生命周期与 backend 调度:
```
Session 生命周期:
Client EXEC / CREATE_SESSION
_get_or_create_session(spec)
├─ _reap_expired_sessions_locked() 清理 TTL 过期 session
├─ 已存在? → _assert_session_compatible() → 复用
├─ Backend session 失踪? → 重建 (commit c6882cf)
└─ 新建? → backend.start_session(spec) → 创建容器
│ └─ 应用 spec.extra_mounts (多挂载)
execute(spec)
├─ 获取 session lock (每 session 独立)
├─ backend.exec(session, spec) 在容器中执行命令
├─ 更新 last_used_at
└─ 超时? → 销毁 session
Session 保持存活直到:
├─ TTL 过期 (默认 300s下次操作时清理)
├─ 执行超时 (自动销毁)
├─ 客户端 DELETE_SESSION
└─ SHUTDOWN
```
**关键设计**:
- 每 session 有独立 `asyncio.Lock`,同一 session 内的命令串行执行
- 每 session 维护 `managed_processes: dict[process_id, _ManagedProcess]`支持多个长驻进程并存MCP / 自定义)
- 全局 `_lock` 保护 `_sessions` dict 的读写
- 兼容性检查:比较核心 spec 字段,`image` 字段对不支持自定义镜像的 backendnsjail/E2B会跳过
**Backend 选择 (`_select_backend`)**: 优先级
1. 显式 `box.backend` 配置(`docker` / `nsjail` / `e2b`
2. `local` (默认) → Docker / Podman / nsjail CLI 顺序探测
3. `get_status` 调用时若当前 backend 不可用,会尝试重新选择 (commit `e5617c7`)
### 3.2 Backend 系统
#### CLISandboxBackend (`box/backend.py`, 411 行)
Docker / Podman 公共基类:
```
start_session(spec):
1. validate_sandbox_security(spec)
2. docker/podman run -d --rm --name <name>
--network none (可选)
--cpus/--memory/--pids-limit
--read-only + --tmpfs /tmp
-v <host>:<mount>:<mode> 主挂载
-v <extra.host>:<extra.mount>:.. 额外挂载 (extra_mounts)
<image> sh -lc 'while true; do sleep 3600; done'
3. 返回 BoxSessionInfo
exec(session, spec):
docker/podman exec -e KEY=VAL <container>
sh -lc 'mkdir -p <workdir> && cd <workdir> && <cmd>'
start_managed_process(session, spec):
docker/podman exec -i <container>
sh -lc 'mkdir -p <cwd> && cd <cwd> && exec <command> <args>'
返回 asyncio.subprocess.Process (stdin/stdout PIPE)
```
容器以 idle 进程启动,实际命令通过 `docker exec` 执行。`--rm` 确保容器退出时自动清理。
**Windows 支持**: backend 内对 Windows 路径处理与 subprocess 调用做了适配commit `120817a`)。
**孤儿清理**: 启动时枚举 `langbot.box=true` 标签的容器instance_id 不匹配的强制删除。
#### NsjailBackend (`box/nsjail_backend.py`, 552 行)
轻量级 Linux 沙箱(无容器引擎依赖):
- 使用 namespace 隔离user/mount/pid/ipc/uts/cgroup/net
- 挂载宿主 `/usr`/`/lib`/`/bin`/`/sbin` 只读 + 选定 `/etc` 条目
- 每 session 创建独立目录workspace/tmp/home
- 资源限制: cgroup v2 优先fallback 到 rlimit
- **CLI 兼容**: 通过 `shutil.which(self._nsjail_bin)` 检测系统安装版 nsjail不存在时再尝试容器内 nsjailcommit `686fcc0``feed530`
- **无自定义镜像**: 使用宿主 OS`image` 字段固定为 `'host'`,兼容性检查跳过 image
#### E2BBackend (`box/e2b_backend.py`, 429 行)
云沙箱后端commit `75b547f` 引入):
- 通过 `e2b` SDK 与 E2B 平台通信
- 配置:`box.e2b.api_key` / `api_url` / `template`
- 支持 `extra_mounts`commit `0fea9b1` 同步上传文件)
- 无本地容器引擎依赖,适合无 Docker 的部署或 SaaS 多租户场景
- 不支持自定义 image 字段,由 template 控制
### 3.3 Server (`box/server.py`, 508 行)
单端口 aiohttp 服务(默认 5410通过路径区分commit `8c71ec5` 合并端口):
1. **Action RPC** (`/rpc/ws`): `BoxServerHandler` 处理所有 action包括 `INIT` 配置注入、skill store 操作等
2. **WS Relay** (`/v1/sessions/{id}/managed-process/ws``/v1/sessions/{id}/managed-process/{pid}/ws`): 双向桥接 WebSocket ↔ 指定 managed process stdin/stdout
stdio 模式同样会在 5410 启动 aiohttp专门承担 managed process attachAction RPC 走 stdin/stdout。
### 3.4 Client (`box/client.py`, 377 行)
`ActionRPCBoxClient` 封装 `Handler.call_action()` 调用:
- 25+ 方法对应 25+ 个 RPC actionexec / session / managed-process / skill / status / shutdown
- 错误还原: `_translate_action_error()` 通过字符串前缀匹配还原 SDK 侧异常类型
- `execute()` timeout = 300s其他默认 15s
- `BoxRuntimeClient` 是 ABC供后续可能的非 RPC 实现复用
包级别 `__init__.py` 显式导出:`BoxRuntimeClient``ActionRPCBoxClient`commit `df9c722`)。
### 3.5 Actions (`box/actions.py`, 34 行)
`LangBotToBoxAction` 枚举共定义 **25 个** action
| 类别 | Actions |
|------|---------|
| 控制 | `INIT``HEALTH``STATUS``GET_BACKEND_INFO``SHUTDOWN` |
| 执行 | `EXEC` |
| Session | `CREATE_SESSION` / `GET_SESSION` / `GET_SESSIONS` / `DELETE_SESSION` |
| Managed Process | `START_MANAGED_PROCESS` / `GET_MANAGED_PROCESS` / `STOP_MANAGED_PROCESS` |
| Skill | `LIST_SKILLS` / `GET_SKILL` / `CREATE_SKILL` / `UPDATE_SKILL` / `DELETE_SKILL` / `SCAN_SKILL_DIRECTORY` / `LIST_SKILL_FILES` / `READ_SKILL_FILE` / `WRITE_SKILL_FILE` / `PREVIEW_SKILL_ZIP` / `INSTALL_SKILL_ZIP` |
### 3.6 Models (`box/models.py`, 331 行)
核心数据模型:
| 模型 | 用途 |
|------|------|
| `BoxNetworkMode` | `OFF` / `ON` |
| `BoxExecutionStatus` | `COMPLETED` / `TIMED_OUT` |
| `BoxHostMountMode` | `NONE` / `READ_ONLY` / `READ_WRITE` |
| `BoxManagedProcessStatus` | `RUNNING` / `EXITED` |
| `BoxMountSpec` | 单条挂载host_path/mount_path/mode**新增** |
| `BoxSpec` | 执行请求;新增 `extra_mounts: list[BoxMountSpec]``persistent``workspace_quota_mb` |
| `BoxProfile` | 4 个内置 Profile + `locked` frozenset |
| `BoxSessionInfo` | Session 状态(含 backend_name/created_at/last_used_at |
| `BoxManagedProcessSpec` | 长驻进程参数process_id/command/args/env/cwd |
| `BoxManagedProcessInfo` | 进程状态status/exit_code/stderr_preview/attached |
| `BoxExecutionResult` | 执行结果status/exit_code/stdout/stderr/duration_ms |
`BoxSpec` 校验器: `workdir` 默认继承 `mount_path``host_path` 支持 POSIX 和 Windows 路径;设置 `host_path``workdir` 必须在 `mount_path` 下。
### 3.7 BoxSkillStore (`box/skill_store.py`, 647 行)
新增模块commit `4ab3502`),把 skill 持久化收归 Box runtime
```
BoxSkillStore
├─ list_skills() / get_skill(name)
├─ create_skill(data) / update_skill(name, data) / delete_skill(name)
├─ scan_skill_directory(path) 扫描目录返回候选 skill 包列表
├─ list_skill_files(name, path) 浏览 skill 内文件树
├─ read_skill_file(name, path) / write_skill_file(name, path, content)
├─ preview_skill_zip(zip_bytes, ...) 不落盘预览 zip 内容
└─ install_skill_zip(zip_bytes, ...) 解压、校验、复制到 skills_root
└─ 支持 source_subdir / target_suffixcommit 1aa043f
```
GitHub 安装路径HTTP 层(`api/http/service/skill.py`)先 `git clone` 拉取,再走 `install_skill_zip` 或 directory 路径。Skill 文件存放于 `box.local.skills_root`(默认 `skills`,相对 `host_root`),容器内对应 `/workspace/.skills/`
### 3.8 Security (`box/security.py`, 52 行)
`validate_sandbox_security()`: 黑名单校验 host_path阻止挂载 `/etc`/`/proc`/`/sys`/`/dev`/`/root`/`/boot` 及 Docker/Podman socket。
**已知缺陷**: 根路径 `/` 未拦截,用户 home 目录未拦截,是 denylist 而非 allowlist 策略。详见 [SaaS 阻塞项 S5](./box-issues.md)。
### 3.9 Errors (`box/errors.py`, 33 行)
| 异常类型 | 含义 |
|----------|------|
| `BoxError` | 基类 |
| `BoxValidationError` | spec/参数校验失败 |
| `BoxBackendUnavailableError` | 无可用 backend |
| `BoxRuntimeUnavailableError` | Runtime 服务不可用 |
| `BoxSessionConflictError` | session 已存在但 spec 不兼容 |
| `BoxSessionNotFoundError` | session 不存在 |
| `BoxManagedProcessConflictError` | session 已有同名 process |
| `BoxManagedProcessNotFoundError` | process 不存在 |
---
## 4. 工具系统集成
### 4.1 ToolManager 编排 (`toolmgr.py`)
```
ToolManager.initialize()
├─ NativeToolLoader (exec / read / write / edit / glob / grep)
├─ PluginToolLoader (插件工具)
├─ MCPLoader (MCP Server 工具)
├─ SkillToolLoader (activate 工具 — Tool Call 激活)
└─ SkillAuthoringToolLoader (Skill CRUD)
工具调用优先级: native → plugin → mcp → skill → skill_authoring
```
### 4.2 Native Tools (`native.py`, 846 行)
| 工具 | 是否在 Box 中执行 | 是否访问宿主文件系统 |
|------|:---:|:---:|
| `exec` | 是 | 否 |
| `read` | **否** | **是** — 直接 `open()` 宿主文件 |
| `write` | **否** | **是** — 直接 `open()` 宿主文件 |
| `edit` | **否** | **是** — 直接 `open()` 宿主文件 |
| `glob` | **否** | **是** — 直接遍历宿主目录 |
| `grep` | **否** | **是** — 直接读宿主文件 |
**沙箱边界不对称**: 这是刻意的设计权衡 — `read`/`write`/`edit`/`glob`/`grep` 绕过沙箱以获得性能(避免容器 I/O 开销与跨进程拷贝),但意味着 LLM 可以直接读写 `allowed_mount_roots` 下任何文件。Skill 路径经 `_resolve_host_path()` 重写,禁止穿越 `package_root`
**exec 的 Skill 分支**: 命令中引用 `/workspace/.skills/<name>` 的 skill 时:
1. 验证 skill 已激活
2. 单次 exec 只能引用一个 skill 包
3. 若 skill 是 Python 项目(有 `requirements.txt``pyproject.toml`),命令会被 venv bootstrap 包裹(在 skill 挂载点内创建 `.venv`
4. 调用 `box_service.execute_tool()` → 走默认 session_id 与已组装好的 `extra_mounts`**不再为每 skill 起独立 session**
### 4.3 MCP-in-Box (`mcp_stdio.py`, 354 行)
`BoxStdioSessionRuntime` 让 MCP stdio 服务器在 Box 容器中运行,**共享 session、多 process**模式commit `529088e`
```
initialize()
1. 复用/创建共享 session (session_id = _build_box_session_id())
- persistent=True长期保持
2. workspace.execute_raw(install_cmd) 安装依赖 (可选)
3. 将每个 MCP server 文件 stage 到 /workspace/.mcp/<process_id>/
4. workspace.start_managed_process(process_id=<server>)
5. websocket_client(ws_url) 通过 WS relay 连接
6. ClientSession.initialize() MCP 协议握手
```
配置 (`MCPServerBoxConfig`): `network='on'` (MCP 服务器通常需要网络)`host_path_mode='ro'` (默认只读)`startup_timeout_sec=120` (留时间给 pip install)。
每条 MCP server 是同一 session 中的一个 managed process独立的 `process_id`、独立 attach URL互不阻塞。
---
## 5. 启动与生命周期
### 5.1 启动顺序 (`build_app.py`)
```
BuildAppStage.run(ap)
├─ ... (persistence, models, sessions) ...
├─ BoxService(ap)
├─ box_service.initialize()
│ └─ connector.initialize()
│ ├─ [stdio] fork box subprocess
│ ├─ [subprocess+WS] Windows 本地
│ └─ [remote WS] connect URL
│ └─ 启动心跳 _heartbeat_task
├─ ap.box_service = box_service
├─ ToolManager(ap)
├─ tool_mgr.initialize()
│ ├─ NativeToolLoader (检查 box_service.available)
│ ├─ PluginToolLoader
│ ├─ MCPLoader (Box 可用时stdio MCP 走沙箱)
│ └─ SkillAuthoringToolLoader
├─ ap.tool_mgr = tool_mgr
├─ ... (platform, pipeline) ...
├─ SkillManager.initialize() (从 Box runtime 加载 skill 列表)
└─ ... (RAG, HTTP, plugins) ...
```
BoxService 在 ToolManager **之前**初始化。ToolManager 创建 loader 时检查 `box_service.available`
### 5.2 初始化失败处理
```python
try:
await self._runtime_connector.initialize()
self._available = True
except Exception as e:
self._available = False
logger.warning(f"Box runtime unavailable: {e}")
```
**静默降级**: Box 初始化失败不会阻止应用启动,仅导致 6 个 native tool、所有 Skill 工具和 MCP-in-Box 工具不暴露给 LLM。与 Plugin 的行为不同Plugin 失败会抛异常)。
### 5.3 销毁流程
```
app.dispose()
└─ box_service.dispose()
├─ connector.dispose()
│ ├─ cancel _heartbeat_task
│ ├─ cancel _handler_task / _ctrl_task
│ └─ terminate subprocess (SIGTERM)
└─ loop.create_task(client.shutdown())
└─ RPC SHUTDOWN → Box Runtime 清理所有容器
```
Box 额外做了 RPC SHUTDOWN 通知 Runtime 主动清理容器,比 Plugin 的直接杀进程更安全。
---
## 6. 配置
### config.yaml (重构后)
```yaml
box:
enabled: true # 整个 Box 子系统的总开关。设为 false 时:
# - 不连接远程 Box runtime不 fork 本地 stdio 子进程
# - sandbox 工具 (exec/read/write/edit/glob/grep) 不暴露给 LLM
# - skill 添加/编辑 / GitHub 安装 / 文件写入全部拒绝
# - stdio 模式的 MCP server 启动时报错http/sse 模式不受影响)
# - skill 列表/读取保持只读可用
# BOX__ENABLED 环境变量可覆盖(统一约定)
backend: 'local' # 'local' (探测) / 'docker' / 'nsjail' / 'e2b'
# 由 box.backend / BOX__BACKEND 选择后端
runtime:
endpoint: '' # 外部 Runtime 的 WS 基地址 'ws://host:5410'
# 留空 = 本地自管 Runtime
local:
profile: 'default'
image: '' # 覆盖 profile 默认 image
host_root: './data/box' # 工作区挂载根Docker 部署需绝对路径
default_workspace: '' # 默认 '<host_root>/default'
skills_root: 'skills' # Box 管理的 skill 包目录(相对 host_root
allowed_mount_roots: # 默认 ['<host_root>']
- './data/box'
- '/tmp'
workspace_quota_mb: null # 配额覆盖null = 走 profile
e2b:
api_key: '' # 也可走 E2B_API_KEY 环境变量
api_url: '' # 自托管 E2B 时填写
template: '' # 默认 template ID
```
> **重大变更**: 较 2026-04-16 文档配置结构完全重组commit `eefdea4`)。原字段 `box.profile` / `box.runtime_url` / `box.shared_host_root` / `box.allowed_host_mount_roots` 全部迁入 `box.local.*` 子表,新增 `box.backend` 与 `box.e2b.*` 配置组。
### docker-compose.yaml
`langbot_box` 服务受 compose profile 控制,默认 `docker compose up` **不会**启动它。需要 sandbox 时:
```bash
docker compose --profile box up # 启动 langbot + langbot_box + plugin runtime
docker compose --profile all up # 同上
docker compose up # 只起 langbot + plugin runtime (box 关闭)
```
若不起 `langbot_box`,需要同步在 `data/config.yaml` 中设 `box.enabled: false`(或 langbot 容器 env 加 `BOX__ENABLED=false`),否则 LangBot 会一直尝试连接不存在的 Box runtime 并报错。
```yaml
# langbot_box 的关键 volume
volumes:
- ${LANGBOT_BOX_ROOT}:${LANGBOT_BOX_ROOT} # 工作区挂载(源/目标同路径)
- /var/run/docker.sock:/var/run/docker.sock # Docker backend 复用宿主 docker
```
### 关闭/连接失败时的行为矩阵
`box.enabled = false` 与"启用但连接失败"在用户可观察行为上**完全一致**——都通过 `BoxService.available = False` 表达,只是 `get_status` 多返回 `enabled` 字段供前端区分文案。
| 消费方 | Box 可用 | Box 不可用(disabled 或 failed) |
|---|---|---|
| native exec/read/write/edit/glob/grep 工具 | 暴露给 LLM | **不暴露** |
| `activate` / `register_skill` 工具 | 暴露给 LLM | **不暴露** |
| stdio MCP server | 在 Box 内启动 | **`_init_stdio_python_server` 抛 RuntimeError** 拒绝;不退化到宿主 stdio |
| http/sse MCP server | 正常 | 正常(不依赖 Box) |
| Skill 列表/读取 (`list_skills`/`get_skill`/`read_skill_file`) | 走 Box runtime | 走 LangBot 本地 `data/skills/` 只读 fallback |
| Skill 创建/编辑/安装/写文件 | 走 Box runtime | **HTTP 400** + 明确错误信息(`_require_box_for_write`) |
| Pipeline AI 配置中 `box-session-id-template` | 正常生效 | **前端 banner** 提示字段无效 |
| Pipeline 扩展页 `enable_all_skills` / 绑定 skill | 可编辑 | **前端禁用** + banner |
| 仪表盘 Box 状态卡片 | 绿点 / "已连接" | 灰点 / "已禁用"(disabled) 或 红点 / "已断开"(failed) |
> 后端拒写的边界条件:如果 `ap.box_service` **完全没装**(老式 dev mode,没经过 BuildAppStage),`_require_box_for_write` 视作 no-op,保留 `data/skills/` 本地路径——以兼容历史测试与最小化设置。生产环境总会装 `ap.box_service`,因此该 fallback 不会被触发。
### Pipeline 配置 (templates/metadata/pipeline/ai.yaml)
`local-agent.config.box-session-id-template` 控制 session 作用域,预设:
- `{launcher_type}_{launcher_id}` — 每个会话 (推荐,默认)
- `{launcher_type}_{launcher_id}_{sender_id}` — 群聊每个用户
- `{launcher_type}_{launcher_id}_{conversation_id}` — 每个对话上下文
- `{query_id}` — 每条消息(完全隔离)
详见 [box-session-scope.md](./box-session-scope.md)。
### REST API
| 端点 | 方法 | 说明 | 前端 |
|------|------|------|:---:|
| `/api/v1/box/status` | GET | 可用性、Profile、后端信息 | ✅ 监控页 |
| `/api/v1/box/sessions` | GET | 活跃 session 列表 | ❌ |
| `/api/v1/box/errors` | GET | 最近 50 条错误 | ❌ |
| `/api/v1/skills` 等 | GET/POST/PUT/DELETE | Skill CRUD、文件浏览、zip/GitHub 安装、preview | ✅ Skill 管理页 |
前端 `web/src/app/home/monitoring/components/overview-cards/SystemStatusCards.tsx` 已接入 `/api/v1/box/status`,展示 backend 名称、profile 与活跃 session 数。Sessions 与 errors API 仍未接入。

View File

@@ -1,76 +0,0 @@
# Box 系统 — SaaS 发布前阻塞项
> 更新日期: 2026-06-02
> 分支: `feat/sandbox` (LangBot + langbot-plugin-sdk)
> 相关文档: [架构分析](./box-architecture.md) | [Session 作用域](./box-session-scope.md) | [Runtime 对比](./box-vs-plugin-runtime.md) | [测试覆盖](./box-test-coverage.md) | [toB 分析](./box-tob-analysis.md)
## 范围说明
**自部署社区版已具备发布条件**:默认 stdio 模式、box 为可选项box 关闭 / 不可用时后端、前端、工具、skill、stdio-MCP 均能干净降级(清晰报错、不崩溃);配置向后兼容(旧 `data/config.yaml` 可直接启动);无新增 ORM 模型、无迁移欠债市场安装失败不会破坏实例。CI 全绿。
本清单**只保留发布 SaaS / 多租户 / 公网暴露前必须处理的阻塞项**。社区版(可信、单运营者、内网)不受这些项阻塞——它们的风险面在"不可信调用方能直接触达 Box 控制面"或"多租户共享资源"的场景才成立。
## 已解决(社区版发布前)
| 项 | 处理 |
|----|------|
| 工具调用循环无上限 (原 #13) | `localagent.py` 增加 `MAX_TOOL_CALL_ROUNDS=128`,超限优雅终止(`cafef1a3` |
| 配额校验同步遍历阻塞事件循环 (原 #10) | `_enforce_workspace_quota` 改 async工作区遍历走 `asyncio.to_thread``cafef1a3` |
| `host_path` 挂载白名单 (原 #3 的 LangBot 侧) | `pkg/box/service.py` `allowed_mount_roots` 白名单,空列表时拒绝一切宿主挂载 |
| 重复的 `_is_path_under` (原 #12) | 已去重,仅保留一处定义 |
| 重连 / 心跳 / Windows 兼容 / nsjail image 字段 / 前端 Box 状态接入 | 见上一轮 review 记录,均已合入 |
---
## SaaS 阻塞项
### S1. Box 控制面无认证 — Critical
- **位置**: SDK `box/server.py` — Action RPC WS (`/rpc/ws`) 与 managed-process relay (`/v1/sessions/{id}/managed-process/{pid}/ws`)
- **现状**: 两个 WS handler 在 `ws.prepare` 后直接服务,无任何 token / 鉴权box 默认绑定 `0.0.0.0:5410`。任何能触达该端口者可发起 `EXEC`、创建 session、attach 任意 session 的 managed-process stdin/stdout、甚至 `SHUTDOWN`。LangBot→box 的 INIT 也未下发任何凭证。
- **缓解现状**: 默认 `docker-compose.yaml``langbot_box` 未把 5410 发布到宿主(爆炸半径限于内网 bridge但 box 挂载了 `/var/run/docker.sock`,同网络的任意服务(含被攻破的插件)→ 宿主 root。若运营者把 5410 发布到宿主或独立以 `0.0.0.0` 起 box则完全裸奔。
- **要求**: INIT 时下发 token两个 WS 路由按连接校验query/header。这是 SaaS 的**头号**阻塞项。
### S2. 无 exec 授权模型policy.py 死代码) — High
- **位置**: LangBot `pkg/box/policy.py``SandboxPolicy` / `ToolPolicy` / `ElevatedPolicy` 全项目无引用);`pkg/provider/tools/loaders/native.py``pkg/provider/tools/toolmgr.py`
- **现状**: 原生工具(`exec/read/write/edit/glob/grep`)按"box 是否可用"全有或全无地暴露,**无 per-pipeline 的 exec 网关 / 工具白名单 / 沙箱模式 / 权限提升控制**。只要 box 可用,任何使用 local-agent + 函数调用模型的 pipeline 都能跑任意 shell。
- **要求**: 接入 policy.py或等价机制按 pipeline 控制是否暴露 `exec`、可用工具白名单、沙箱网络/只读模式。
### S3. 会话资源无界DoS — High
- **#5 session 数量无上限**: SDK `box/runtime.py` `_get_or_create_session``_sessions` dict 无容量限制——可变 `session_id` 的恶意调用可无限创建容器,耗尽宿主 CPU/内存/PID/磁盘。
- **#8 无定时回收**: 过期 session 仅在 `_get_or_create_session` 时机会性清理,无独立周期任务;一波创建后转静默会永久泄漏容器。
- **要求**: `max_sessions` 上限(拒绝或 LRU加独立周期 reaper如 60s
### S4. 工作区配额无内核级限制TOCTOU — Med-High
- **位置**: LangBot `pkg/box/service.py` `_enforce_workspace_quota`(应用层 read-then-checkSDK 侧 `workspace_quota_mb` 仅记录/透传,无 `--storage-opt size=` 等内核/FS 限额
- **现状**: 执行前后两次检查之间存在竞态窗口;单条命令(`dd`/`fallocate`)可在检查间隙撑爆磁盘,事后检查只能补救。
- **要求**: Docker `--storage-opt size=` 做内核级限制,或 Redis 原子计数预留式配额。
### S5. 挂载校验缺口 — Med-High
- **位置**: SDK `box/security.py` `_BLOCKED_HOST_PATHS_POSIX``box/backend.py``extra_mounts` 处理
- **现状**: ① SDK 黑名单仍不含 `/`(前缀匹配,`host_path="/"` 可通过,挂载整个宿主 fs用户 home、`/usr``/opt``/tmp` 也未拦截。② `validate_sandbox_security` 只校验 `spec.host_path`**从不遍历 `spec.extra_mounts`**——LangBot 侧 `allowed_mount_roots` 也只校验 `host_path`。当前 `extra_mounts` 仅由 `build_skill_extra_mounts` 内部填充agent 不可达),但缺乏纵深防御:一旦 S1 的无认证 RPC 被触达extra_mounts 可挂任意宿主路径,两层都不拦。
- **要求**: SDK 黑名单加入 `/`(或改白名单);`extra_mounts` 在 SDK 与 LangBot 两侧都纳入挂载校验。
### S6. 容器加固缺失 — Med
- **位置**: SDK `box/backend.py``docker run` 组装
- **现状**: 未设置 `--cap-drop=ALL``--security-opt=no-new-privileges`、非 root `--user`;叠加挂载 docker.sock逃逸面偏大。
- **要求**: 默认加上上述加固 flag需回归常用 skill 不被破坏)。
### S7. 全局锁内执行慢操作(扩展性) — Med
- **位置**: SDK `box/runtime.py` `_get_or_create_session``self._lock` 持有期间调用 `backend.start_session()``docker run` / nsjail 启动 / E2B `Sandbox.create`
- **影响**: 冷启动镜像拉取数秒、E2B >1s期间串行阻塞所有并发请求——多租户负载下整个 Box runtime 停顿。降级表现是延迟而非失败。
- **要求**: 锁内只做状态检查与注册,容器创建移到锁外。
### S8. 其他硬化 / 跟进 — Low
- **#9** SDK `box/server.py` 直接读 `runtime._sessions` 私有字段、绕过锁,并发下可能读到不一致状态——应加公共访问方法。
- **#16** `pkg/provider/tools/toolmgr.py` `execute_func_call` 按优先级分发plugin/MCP 若有同名 `exec/read/write/...` 工具会被静默遮蔽——应加命名空间或冲突告警。
- **#4** SDK `box/runtime.py` INIT/handshake 与 backend 实例化的残留竞态(仅"纯远程 WS box 先启动、LangBot 后连"场景成立stdio/compose 路径下 config 经 env 在 spawn 时已就位,无竞态)——应在 INIT 完成前拒绝业务 action。
- **#11** `extra_mounts` 在容器创建时固定SDK `runtime.py` 兼容性检查不含 extra_mounts长生命周期共享 session 后续新激活的 skill 不会挂上(当前缓解:创建时挂上 pipeline 绑定的全部 skill——动态绑定场景需销毁重建或文档说明。
- **#21** 集成测试未进 CI容器实际执行、E2B 真机、managed-process WS attach 仅本地可跑。安全关键路径缺自动化覆盖——SaaS 前建议加 Docker-in-Docker CI stage 或合并前手动 checklist。

View File

@@ -1,402 +0,0 @@
# Box Session Scope Design
> Date: 2026-04-18 (last reviewed 2026-06-02)
> Status (2026-06-02): the self-hosted community edition is release-ready (box optional, clean degradation, no migration debt). Tool-call loop cap, async quota scan, and the host_path mount allowlist have landed. Remaining multi-tenant / security hardening is tracked in [box-issues.md](./box-issues.md).
> Branch: `feat/sandbox` (LangBot + langbot-plugin-sdk)
> Related: [Box Architecture](./box-architecture.md) | [Box vs Plugin Runtime](./box-vs-plugin-runtime.md)
---
## 0. Implementation Status (2026-05-19)
This document was authored as a design proposal. The current `feat/sandbox` branch
has shipped the design largely as written:
| Item | Status | Notes |
|------|--------|-------|
| `BoxMountSpec` + `BoxSpec.extra_mounts` | ✅ Shipped | SDK `box/models.py` |
| Docker / nsjail / E2B backends apply extra mounts | ✅ Shipped | Last gap closed by SDK commit `0fea9b1` (E2B) |
| `box-session-id-template` in `local-agent` pipeline config | ✅ Shipped | `templates/metadata/pipeline/ai.yaml`, default `{launcher_type}_{launcher_id}` |
| `BoxService.resolve_box_session_id(query)` | ✅ Shipped | `pkg/box/service.py:166` |
| `BoxService.build_skill_extra_mounts(query)` | ✅ Shipped | `pkg/box/service.py:189` |
| Skill exec uses unified container + extra mounts | ✅ Shipped | `pkg/provider/tools/loaders/native.py` skill branch |
| MCP-in-Box uses shared persistent session, multi-process | ✅ Shipped (earlier than originally scoped) | SDK commit `529088e`, LangBot `mcp_stdio.py:_build_box_session_id` |
| `BoxManagedProcessSpec.process_id` + multi-process per session | ✅ Shipped | `BoxRuntime` keeps `managed_processes: dict[pid, _ManagedProcess]` |
| Per-tenant / quota integration with templates | ❌ Not started | See [box-tob-analysis.md](./box-tob-analysis.md) |
The "Phase 2 deferred" note in §10 is **out of date** — MCP unification went in on
the same line. Pipeline-scoped (not user-scoped) MCP container is the realized
behavior: each pipeline's MCP servers share one `mcp-<pipeline>` session, and
user exec sessions use the template-derived id.
The remaining open work is multi-tenant overlays (tenant_id in session_id,
quota counters keyed by tenant), tracked in the toB analysis doc rather than here.
---
## 1. Problems
### 1.1 Default exec: per-message containers
Currently, `BoxService.execute_tool()` sets `session_id = str(query.query_id)` — an
auto-incrementing integer per incoming message. Every user message creates a new sandbox
container. Dependencies installed and in-container state are lost between messages.
### 1.2 Three isolated container pools
Default exec, skills, and MCP servers each manage their own containers with
independent session IDs:
| Path | Session ID | Container |
|--------------|-----------------------------------------------|-------------|
| Default exec | `str(query_id)` (per message) | Ephemeral |
| Skill exec | `skill-{launcher}_{id}-{skill_name}` | Per skill |
| MCP stdio | `mcp-{server_uuid}` | Per server |
This means a single logical user interaction can spawn 3+ containers that cannot
share state, see each other's files, or reuse installed dependencies.
### 1.3 Single bind mount limitation
`BoxSpec` currently supports only **one** `host_path``mount_path` bind mount.
This prevents mounting both a default workspace and skill directories into the
same container.
---
## 2. Concept Model
```
Platform Message
→ Query (query_id: int, auto-increment, per message)
→ Session (launcher_type + launcher_id, per chat window)
→ Conversation (uuid, per dialogue context within a Session)
```
| Concept | Key | Example | Scope |
|---------------|-------------------------------------|----------------------------|------------------------------|
| Query | `query_id` | `42` | Single message |
| Session | `launcher_type` + `launcher_id` | `group_123456` | Chat window (group or PM) |
| Conversation | `conversation_id` (UUID) | `a1b2c3d4-...` | Dialogue context within a Session |
| Sender | `sender_id` | `789` | Individual user |
Note: in a **group chat**, all users share the same Session (keyed by `group_id`). The
individual sender is tracked as `sender_id` but does not affect Session/Conversation routing.
---
## 3. Target Scenarios
| # | Scenario | Box Granularity | Desired `session_id` |
|----|--------------------------------|------------------------------------------|---------------------------------------------------------|
| 1 | Personal assistant | 1 Box per user, long-lived | `{launcher_type}_{launcher_id}` |
| 2 | Customer service | 1 Box per customer, cross-pipeline | `{launcher_type}_{launcher_id}` |
| 3 | Internal employee tool | 1 Box per employee | `{launcher_type}_{launcher_id}` |
| 4 | Group chat shared assistant | 1 Box per group | `{launcher_type}_{launcher_id}` |
| 5 | Group chat isolated per user | 1 Box per user within a group | `{launcher_type}_{launcher_id}_{sender_id}` |
| 6 | Teaching (cross-channel) | 1 Box per student across groups/PMs | `{sender_id}` |
| 7 | One-off execution | 1 Box per message (current behavior) | `{query_id}` |
| 8 | Multi-project development | 1 Box per conversation context | `{launcher_type}_{launcher_id}_{conversation_id}` |
No single fixed granularity covers all scenarios. A template-based approach is needed.
---
## 4. Design Overview
Two key changes:
1. **Unified container**: exec, skills, and MCP all share the same container per
session scope. No more separate container pools.
2. **Configurable session scope**: `session_id` is generated from a template with
pipeline variables, configurable per pipeline.
### 4.1 Unified Container with Multiple Mounts
A single container per session scope is created on first use. It has:
- **Primary mount**: default workspace at `/workspace` (from `default_host_workspace`)
- **Skill mounts**: each pipeline-bound skill's `package_root` mounted at
`/workspace/.skills/{skill_name}/`
- **MCP servers**: run as managed processes inside the same container
```
Container (session_id = "group_123456")
/workspace/ ← default workspace (bind mount, rw)
/workspace/.skills/web-search/ ← skill package (bind mount, rw)
/workspace/.skills/data-analysis/ ← skill package (bind mount, rw)
[managed process: mcp-server-a] ← MCP server running inside
[managed process: mcp-server-b] ← MCP server running inside
```
This requires extending `BoxSpec` to support multiple mounts (see §5).
### 4.2 Session ID Template
A new field `box-session-id-template` in the `local-agent` pipeline runner config
controls the session scope:
```yaml
# templates/metadata/pipeline/ai.yaml (under local-agent.config)
- name: box-session-id-template
label:
en_US: Sandbox Scope
zh_Hans: 沙箱作用域
description:
en_US: >-
Determines how sandbox environments are shared. Use variables to
control isolation granularity.
zh_Hans: >-
决定沙箱环境的共享方式。使用变量控制隔离粒度。
type: select
required: false
default: "{launcher_type}_{launcher_id}"
options:
- value: "{launcher_type}_{launcher_id}"
label:
en_US: Per chat (Recommended)
zh_Hans: 每个会话(推荐)
- value: "{launcher_type}_{launcher_id}_{sender_id}"
label:
en_US: Per user in chat
zh_Hans: 会话中每个用户
- value: "{launcher_type}_{launcher_id}_{conversation_id}"
label:
en_US: Per conversation context
zh_Hans: 每个对话上下文
- value: "{query_id}"
label:
en_US: Per message (isolated)
zh_Hans: 每条消息(完全隔离)
```
Available template variables (populated by PreProcessor in `query.variables`):
| Variable | Source | Example |
|---------------------|---------------------------------|----------------------|
| `{launcher_type}` | `query.session.launcher_type` | `person` / `group` |
| `{launcher_id}` | `query.session.launcher_id` | `123456` |
| `{sender_id}` | `query.sender_id` | `789` |
| `{conversation_id}` | `conversation.uuid` | `a1b2c3d4-...` |
| `{query_id}` | `query.query_id` | `42` |
Default `{launcher_type}_{launcher_id}` covers scenarios 14 out of the box.
---
## 5. SDK Changes: Multi-Mount BoxSpec
### 5.1 Model Extension
```python
# box/models.py
class BoxMountSpec(pydantic.BaseModel):
"""A single bind mount specification."""
host_path: str
mount_path: str
mode: BoxHostMountMode = BoxHostMountMode.READ_WRITE
class BoxSpec(pydantic.BaseModel):
# ... existing fields ...
host_path: str | None = None # Primary mount (backward compat)
host_path_mode: BoxHostMountMode = BoxHostMountMode.READ_WRITE
mount_path: str = DEFAULT_BOX_MOUNT_PATH
extra_mounts: list[BoxMountSpec] = [] # NEW: additional mounts
```
`extra_mounts` is additive — the existing `host_path` / `mount_path` pair remains
the primary mount for backward compatibility.
### 5.2 Backend: Apply Extra Mounts
```python
# box/backend.py — CLISandboxBackend.start_session()
# Primary mount (unchanged)
if spec.host_path is not None and spec.host_path_mode != BoxHostMountMode.NONE:
args.extend(['-v', f'{spec.host_path}:{spec.mount_path}:{spec.host_path_mode.value}'])
# Extra mounts (NEW)
for mount in spec.extra_mounts:
if mount.mode != BoxHostMountMode.NONE:
args.extend(['-v', f'{mount.host_path}:{mount.mount_path}:{mount.mode.value}'])
```
Same pattern for nsjail backend.
---
## 6. LangBot Changes
### 6.1 Session ID Resolution
In `BoxService.execute_tool()`:
```python
# Before:
spec_payload.setdefault('session_id', str(query.query_id))
# After:
template = (query.pipeline_config or {}).get('ai', {}) \
.get('local-agent', {}).get('box-session-id-template',
'{launcher_type}_{launcher_id}')
variables = query.variables or {}
session_id = template.format_map(collections.defaultdict(
lambda: 'unknown', variables
))
spec_payload.setdefault('session_id', session_id)
```
### 6.2 Skill Exec: Use Same Container
Currently `native.py:_invoke_exec` creates a separate `BoxWorkspaceSession` per
skill with `host_path=package_root`. Instead:
1. Use the **same session_id** as default exec (from the template).
2. Pass the skill's `package_root` as an **extra mount** at
`/workspace/.skills/{skill_name}/` instead of replacing `/workspace`.
3. The container already has the default workspace at `/workspace`.
```python
# native.py — _invoke_exec, skill branch (REVISED)
# Same session_id as default exec
session_id = resolve_box_session_id(query)
spec_payload = {
'cmd': rewritten_command,
'workdir': rewritten_workdir,
'session_id': session_id,
'extra_mounts': [{
'host_path': package_root,
'mount_path': f'/workspace/.skills/{selected_skill_name}',
'mode': 'rw',
}],
}
result = await self.ap.box_service.execute_spec_payload(spec_payload, query)
```
The virtual path `/workspace/.skills/{name}` no longer needs rewriting at the
command level — it maps directly to the bind mount path inside the container.
### 6.3 MCP: Use Same Container
MCP servers should run inside the same container as exec and skills. Changes:
1. `BoxStdioSessionRuntime` uses the pipeline's session_id template instead of
`mcp-{server_uuid}`.
2. MCP server's working directory is a subdirectory (e.g. `/workspace/.mcp/{name}/`).
3. MCP server's dependencies are mounted or installed into that subdirectory.
4. The MCP server runs as a managed process inside the shared container.
Since MCP servers start at LangBot boot (not per-query), the session must be
created eagerly. The container will be kept alive by the managed process
exemption in TTL reaping (`runtime.py:259`).
**Note**: MCP sessions are pipeline-scoped (not per-launcher), so their session_id
should be a **fixed identifier per pipeline** rather than the user-facing template.
This means one shared MCP container per pipeline, with user exec sessions separate.
Alternatively, in a future iteration, MCP managed processes could be launched
lazily into the user's container on first MCP tool call. This is more complex
but maximizes sharing. For V1, keeping MCP containers at pipeline scope is
simpler and more predictable.
---
## 7. Mount Layout Summary
### Default exec (no skills activated)
```
Container (session_id from template)
/workspace/ ← default_host_workspace (rw)
```
### Exec with activated skills
```
Container (same session_id)
/workspace/ ← default_host_workspace (rw)
/workspace/.skills/web-search/ ← skill package_root (rw)
/workspace/.skills/data-analysis/ ← skill package_root (rw)
```
Extra mounts are **additive** — they are added when the container is first
created (or on the first exec that references a skill). Since Docker bind
mounts are specified at container creation time, skills must be known at
creation time.
**Resolution**: When creating a container, inject `extra_mounts` for **all
pipeline-bound skills** (from `extensions_preferences`), not just the
currently activated one. This way any skill can be activated later without
recreating the container.
### MCP servers (V1: pipeline-scoped)
```
Container (session_id = "mcp-pipeline-{pipeline_uuid}")
/workspace/ ← MCP shared workspace
/workspace/.mcp/server-a/ ← MCP server A files
/workspace/.mcp/server-b/ ← MCP server B files
[managed process: server-a]
[managed process: server-b]
```
---
## 8. Data Migration
Existing pipelines do not have `box-session-id-template`. The backend uses
`.get(..., default)` so missing keys fall back to `{launcher_type}_{launcher_id}`.
This changes behavior from per-message to per-launcher for existing pipelines.
Recommendation: **accept the behavior change** — per-launcher is the more
intuitive default, and the old per-message behavior was rarely desired.
---
## 9. Cloud Quota Implications
| Scope | Typical concurrent containers |
|-----------------------------------------------|-------------------------------|
| `{query_id}` (per message) | Many, short-lived |
| `{launcher_type}_{launcher_id}` (per chat) | = active chat count |
| `{sender_id}` (per user) | = active user count |
| `{conversation_id}` (per conversation) | Between per-chat and per-msg |
With the unified container model, each scope value maps to exactly **one**
container (instead of potentially 3+ per-message). This significantly reduces
resource usage.
Quota enforcement point: `BoxRuntime._get_or_create_session()` in the SDK.
---
## 10. Implementation Phases
### Phase 1: Session scope + skill unification (this PR)
1. **SDK**: Extend `BoxSpec` with `extra_mounts: list[BoxMountSpec]`.
2. **SDK**: Update Docker/nsjail backends to apply extra mounts.
3. **LangBot**: Add `box-session-id-template` to `local-agent` YAML metadata
and default pipeline config JSON.
4. **LangBot**: Update `BoxService.execute_tool()` to use template interpolation.
5. **LangBot**: Update `native.py:_invoke_exec` skill branch to use same
session_id + extra mounts instead of separate `BoxWorkspaceSession`.
6. **LangBot**: On container creation, inject extra mounts for all
pipeline-bound skills.
7. **Frontend**: No code change — `DynamicFormComponent` renders `select` fields.
8. **Tests**: Unit tests for template interpolation and multi-mount specs.
### Phase 2: MCP unification (future)
1. Refactor `BoxStdioSessionRuntime` to use pipeline-scoped shared container.
2. MCP servers become managed processes in the shared container.
3. Support multiple concurrent managed processes per container.
MCP unification is deferred because it requires changes to the managed process
model (currently 1 managed process per session) and has startup ordering
concerns (MCP servers start at boot, before any user query determines
a session_id).

View File

@@ -1,122 +0,0 @@
# Box 系统测试覆盖分析
> 更新日期: 2026-06-02
> 状态更新: 自部署社区版已具备发布条件box 可选、降级完善、无迁移欠债);工具调用循环上限、配额遍历异步化、`host_path` 挂载白名单等已落地。剩余多租户 / 安全硬化项见 [SaaS 阻塞项清单](./box-issues.md)。
> 分支: `feat/sandbox` (LangBot + langbot-plugin-sdk)
---
## 1. 测试文件清单
### LangBot 仓库
| 文件 | 行数 | CI 运行 | 覆盖范围 |
|------|------|---------|---------|
| `tests/unit_tests/box/test_box_connector.py` | 106 | 是 | Connector 传输决策、WS relay URL、dispose、心跳/重连 |
| `tests/unit_tests/box/test_box_service.py` | 1224 | 是 | Service 核心逻辑(最全面) |
| `tests/unit_tests/box/test_workspace.py` | 147 | 是 | WorkspaceSession 路径重写、payload 构建 |
| `tests/unit_tests/provider/test_mcp_box_integration.py` | 707 | 是 | MCP Box 配置、路径重写、payload、shared-session/multi-process、runtime info |
| `tests/unit_tests/provider/test_localagent_sandbox_exec.py` | 444 | 是 | LocalAgent exec 流程、流式、Skill 激活 (Tool Call) |
| `tests/unit_tests/provider/test_tool_manager_native.py` | 249 | 是 | ToolManager 路由、native tool CRUD、路径穿越、6 工具暴露 |
| `tests/unit_tests/provider/test_skill_tools.py` | 582 | 是 | Skill 管理、Tool Call 激活、路径、authoring CRUD |
| `tests/unit_tests/test_skill_service.py` | 396 | 是 | HTTP serviceskill CRUD、zip/GitHub install、文件浏览 |
| `tests/unit_tests/test_paths.py` | 23 | 是 | paths 工具 |
| `tests/unit_tests/test_preproc.py` | 134 | 是 | PreProcessor 注入 session 变量、bound skill 解析 |
| `tests/unit_tests/pipeline/test_chat_handler_logging.py` | 78 | 是 | Chat handler 日志相关回归 |
| `tests/integration_tests/box/test_box_integration.py` | 329 | **否** | 真实容器执行、超时、网络隔离 |
| `tests/integration_tests/box/test_box_mcp_integration.py` | 368 | **否** | Managed process、WS attach、shared-session 清理 |
### SDK 仓库
| 文件 | 行数 | CI 运行 | 覆盖范围 |
|------|------|---------|---------|
| `tests/box/test_backend_selection.py` | 255 | 是 | 显式 backend / local 模式探测顺序 / 配置变更触发 reselect |
| `tests/box/test_nsjail_backend.py` | 452 | 是 | nsjail 可用性、安装版 CLI vs 容器内 CLI、session、arg 构建、资源限制 |
| `tests/box/test_e2b_backend.py` | 482 | 是 | E2B SDK mock、session 生命周期、extra_mounts 同步 |
| `tests/box/test_skill_store.py` | 88 | 是 | zip preview/install、基础 file CRUD |
**总计**: 17 个测试文件, ~6,500 行测试代码; 其中 2 个集成测试(约 700 行)在 CI 中不运行。
> 较 2026-04-16 版增加:`test_skill_service.py`、`test_paths.py`、`test_preproc.py`、`test_chat_handler_logging.py` (LangBot)`test_backend_selection.py`、`test_e2b_backend.py`、`test_skill_store.py` (SDK)。`test_nsjail_backend.py` 增加 CLI 兼容性 case (commit `feed530`)。
---
## 2. 覆盖良好的区域
| 区域 | 质量 | 说明 |
|------|------|------|
| BoxRuntime session 管理 | 优秀 | session 复用、冲突检测、TTL 配置、消失 session 重建 |
| BoxService Profile 系统 | 优秀 | 4 个内置 Profile、locked/unlocked 字段、timeout clamp |
| BoxService host mount 安全 | 优秀 | allowed_mount_roots、disallowed_roots、shared host root |
| BoxService workspace quota | 优秀 | 前置/后置配额检查、超额清理 |
| BoxService 输出截断 | 优秀 | 短/精确边界/长输出、独立 stderr |
| BoxService 可观测性 | 优秀 | 状态报告、error ring buffer、buffer 上限 |
| BoxService session 模板 | 良好 | `resolve_box_session_id` + `build_skill_extra_mounts` 在 service / native / mcp 三处都有覆盖 |
| RPC client/server 协议 | 优秀 | execute/get_sessions/delete/create/conflict error |
| BoxRuntimeConnector | 良好 | local/remote 模式、Docker 平台、relay URL、心跳与重连回调 |
| BoxWorkspaceSession | 良好 | payload 构建、managed process 路径重写、stage host file |
| BoxHostMountMode.NONE | 良好 | 枚举校验、workdir 约束 |
| NsjailBackend | 良好 | 可用性、安装版 vs 容器内、session 生命周期、arg 构建、资源限制 |
| E2BBackend | 良好 | mock SDK、session/extra_mounts 同步 |
| Backend selection | 良好 | 显式 backend 优先级、local 探测顺序、配置变更触发 reselect |
| MCP Box 集成 | 良好 | config model、路径重写、payload、shared-session 多 process |
| Native tool loader | 良好 | 6 工具exec/read/write/edit/glob/grep、路径穿越拦截 |
| LocalAgent exec 流程 | 良好 | 完整 tool call 循环、流式、system prompt 注入、Tool Call 激活 |
| Skill 系统 | 良好 | 加载、Tool Call 激活、marker、路径解析、authoring CRUD、HTTP service |
---
## 3. 覆盖缺失的区域
### 3.1 零测试 / 严重不足
| 区域 | 源文件 | 影响 |
|------|--------|------|
| **`security.py`** | SDK `box/security.py` (52 行) | `validate_sandbox_security()` 无任何测试。阻止 `/etc`/`/proc`/Docker socket 等危险挂载的安全函数从未被验证 |
| **`policy.py`** | `pkg/box/policy.py` (98 行) | 三层安全策略无测试(也是死代码) |
| **`skill_store.py` 边缘场景** | SDK `box/skill_store.py` (647 行) vs 测试 88 行 | GitHub 安装路径、`source_subdir` / `target_suffix` 组合、损坏 zip、文件冲突等场景未覆盖 |
### 3.2 未测试的关键路径
| 区域 | 说明 |
|------|------|
| **Session TTL 过期** | 测试配置了 `session_ttl_sec` 但从未推进时间验证过期清理 |
| **并发 session 访问** | 无并发 exec / 并发创建 / race condition 测试 |
| **Container backend (Docker)** | 仅通过集成测试覆盖CI 不运行),单元测试全用 FakeBackend |
| **E2B 真实 sandbox** | 单测全是 mock未对接真实 E2B API |
| **BoxRuntime shutdown()** | 在 test cleanup 中调用但未验证行为 |
| **BoxServerHandler 错误路径** | 畸形请求、未知 action 类型 |
| **WS relay** | 仅在集成测试中覆盖CI 不运行) |
| **NsjailBackend managed process** | 完全未测试 |
| **MCP stdio 完整生命周期** | 依赖安装 → 进程启动 → 健康检查 → 多 process 并发 → 重试 |
| **BoxService start/stop_managed_process** | 单 process 流转有单测,多 process 互不阻塞主要靠集成测试 |
| **重连指数退避** | connector 单测覆盖回调接线,未实际跑完整重连周期 |
### 3.3 边缘情况缺失
| 区域 | 说明 |
|------|------|
| BoxSpec 校验 | 无效 session_id 格式、超长命令、env 特殊字符 |
| BoxSpec.extra_mounts | 重复 mount_path、与 host_path 冲突、绝对 vs 相对路径 |
| BoxExecutionResult | 仅 COMPLETED 和 TIMED_OUT无 ERROR 状态测试 |
| 多后端 fallback | local 模式探测顺序仅靠 mock无真实 Docker 不可用 → nsjail 真机 fallback 测试 |
| Profile YAML 加载 | 测试用硬编码字符串,未从真实 config.yaml 加载 |
| INIT 配置变更触发 backend 重建 | 单测仅在初始化场景验证 |
---
## 4. 集成测试 vs CI 的差距
CI 仅运行 `tests/unit_tests/`,以下场景**从未在自动化中验证**:
- 真实容器的创建/执行/销毁
- 容器网络隔离(`--network none`
- 容器资源限制生效cpus/memory/pids_limit
- Managed process 的 WS 双向 I/O
- 多 process 同 session 并发 I/O
- 孤儿容器清理
- Session 删除清理容器
- 进程退出检测
- E2B 真实 sandbox 行为
**建议**: 在 CI 中加一个可选的 Docker-in-Docker 集成测试 stage至少覆盖核心执行路径exec / MCP attach / session 销毁)。

View File

@@ -1,167 +0,0 @@
# Box 系统 toB 商业化分析
> 更新日期: 2026-06-02
> 状态更新: 自部署社区版已具备发布条件box 可选、降级完善、无迁移欠债);工具调用循环上限、配额遍历异步化、`host_path` 挂载白名单等已落地。剩余多租户 / 安全硬化项见 [SaaS 阻塞项清单](./box-issues.md)。
> 分支: `feat/sandbox` (LangBot + langbot-plugin-sdk)
---
## 1. 现有优势
| 能力 | toB 价值 | 代码位置 |
|------|---------|---------|
| **沙箱隔离执行** | 企业安全运行不受信代码的基础能力 | SDK `box/backend.py` |
| **多后端支持** | 适配不同企业容器基础设施 (Podman/Docker/nsjail/E2B) | SDK `box/runtime.py` `_select_backend()` |
| **E2B 云沙箱** | SaaS / 无 Docker 部署的兜底执行环境 | SDK `box/e2b_backend.py` |
| **连接自愈** | 心跳 + 自动重连,单点 Box runtime 故障可恢复 | `pkg/box/connector.py` `_heartbeat_loop`, `pkg/box/service.py` `_reconnect_loop` |
| **Profile + locked 字段** | 运维锁定安全边界LLM/用户无法绕过 | `pkg/box/service.py`, SDK `box/models.py` |
| **资源限制** | CPU/内存/PID 数限制防止资源滥用 | SDK `backend.py` `--cpus/--memory/--pids-limit` |
| **Workspace quota** | 磁盘用量控制 | `pkg/box/service.py` `_enforce_workspace_quota` |
| **静默降级** | Box 不可用不影响其他功能,降低部署门槛 | `pkg/box/service.py:78` `_available=False` |
| **孤儿容器清理** | 防止泄漏的容器持续占用资源 | SDK `backend.py` `cleanup_orphaned_containers` |
| **网络隔离** | `--network none` 防止数据外泄 | SDK `backend.py` start_session |
| **只读根文件系统** | `--read-only` 防止容器被持久篡改 | SDK `backend.py` start_session |
| **Host path 白名单** | `allowed_host_mount_roots` 限制可挂载目录 | `pkg/box/service.py` `_validate_host_mount` |
---
## 2. toB 差距分析
### 2.1 安全与合规
| 维度 | 现状 | toB 要求 | 优先级 |
|------|------|---------|--------|
| **WS relay 认证** | 无认证,任何人可 attach | 至少 token 认证 | **P0** |
| **安全策略** | policy.py 是死代码,实际无细粒度控制 | 工具级 allow/deny、沙箱模式控制 | **P0** |
| **审计日志** | 仅内存中 50 条 `_recent_errors` | 持久化审计:谁何时执行了什么、结果如何 | **P0** |
| **Host path 校验** | 黑名单策略,`/` 未拦截 | 白名单策略,默认拒绝 | **P1** |
| **数据驻留** | 无控制 | GDPR / 等保要求的数据隔离 | **P2** |
### 2.2 多租户
| 维度 | 现状 | toB 要求 | 优先级 |
|------|------|---------|--------|
| **租户隔离** | 无租户概念 | BoxSpec/Profile 绑定 tenant_id | **P0** |
| **RBAC** | 仅 token 认证 | admin/operator/viewer 角色权限 | **P0** |
| **资源配额** | 单一 workspace quota | 每租户 CPU 时间/内存/并发/执行次数配额 | **P1** |
| **Session 隔离** | 所有 session 共享 dict | 按租户分区,互不可见 | **P1** |
### 2.3 可靠性
| 维度 | 现状 | toB 要求 | 优先级 |
|------|------|---------|--------|
| **连接恢复** | 已实现20s 心跳 + `_reconnect_loop` 指数退避 | 已满足基本要求 | 已有 |
| **Session 清理** | 机会性(仅新建时触发) | 定时清理 + 独立 reaper | **P1** |
| **水平扩展** | 单 Box Runtime 实例 | 多实例负载均衡(按 tenant 路由) | **P1** |
| **优雅降级** | 已有_available=False | 已满足基本要求 | 已有 |
| **Backend 自愈** | 已实现:`get_status` 时若 backend 不可用会重新选择 | 已满足基本要求 | 已有 |
### 2.4 可观测性
| 维度 | 现状 | toB 要求 | 优先级 |
|------|------|---------|--------|
| **监控指标** | 无 Prometheus metrics | session 数/执行延迟/资源用量/错误率 | **P1** |
| **结构化日志** | Python logging, 无结构化 | JSON 格式日志,含 trace_id/tenant_id | **P1** |
| **前端面板** | 监控页接入 `/api/v1/box/status`backend 名 + 活跃 session 数);`sessions` / `errors` 仍未接入 | 完整状态面板 + 历史错误/审计列表 | **P2** |
---
## 3. SaaS 部署架构建议
### 3.1 方案 A: 共享 Box Runtime Pool (快速上线)
```
LangBot Instance ──> Box Runtime (共享)
├─ tenant_id 标签隔离
├─ Redis 配额计数器
└─ Container labels: langbot.tenant_id=xxx
```
- **优点**: 改动最小,加 tenant_id 到 BoxSpec/labels 即可
- **缺点**: 容器引擎共享,安全隔离弱
### 3.2 方案 B: 每租户 K8s Namespace + gVisor (推荐中期)
```
LangBot ──> K8s API
├─ namespace: tenant-xxx
│ ├─ RuntimeClass: gVisor (runsc)
│ ├─ ResourceQuota
│ └─ NetworkPolicy
└─ namespace: tenant-yyy
└─ ...
```
- **优点**: 强隔离namespace + gVisor原生 K8s 配额
- **缺点**: 需要重写 backend 为 K8s Job部署复杂度高
### 3.3 方案 C: K8s Job 直接编排 (长期)
```
LangBot ──> K8s Job per execution
├─ 每次执行创建 Job
├─ Pod Security Standards
├─ 自动调度和资源分配
└─ Job TTL Controller 自动清理
```
- **优点**: 最强隔离,天然水平扩展
- **缺点**: 冷启动延迟,架构重写
**推荐演进路径**: A → B → C
---
## 4. 配额体系建议
### 三层配额
| 层 | 实现 | 作用 |
|----|------|------|
| **内核层** | Docker `--cpus`/`--memory`/`--storage-opt` | 硬性资源上限,不可绕过 |
| **应用层** | Redis 原子计数器 | 并发 session 数/执行次数/CPU 时间预算 |
| **计费层** | 月度聚合 | 按租户计费session-hours/execution-count |
### Profile 与套餐映射
| 套餐 | Profile | locked 字段 | 配额 |
|------|---------|------------|------|
| Free | `offline_readonly` | network, host_path_mode, rootfs | 10 exec/天, 0.5 CPU, 256MB |
| Pro | `default` | (无) | 100 exec/天, 1 CPU, 512MB |
| Enterprise | `network_extended` | (按需) | 无限, 2 CPU, 1GB, 自定义镜像 |
### TOCTOU 配额修复
当前 `_enforce_workspace_quota` 的 TOCTOU 问题可通过两种方式解决:
1. **预留式配额** (应用层): Redis `INCRBY` 预扣额度 → 执行 → 成功则扣减,失败则回滚
2. **内核级限制** (Docker): `--storage-opt size=500m` 直接限制容器可写层大小
---
## 5. 优先实施路线
### Phase 1 (2-4 周): 安全基线
- [ ] WS relay 加 token 认证
- [ ] 接入或删除 policy.py
- [x] ~~Box 加重连和心跳~~(已完成,见 [box-issues.md 已解决](./box-issues.md)
- [ ] 审计日志持久化(至少写文件/数据库)
- [ ] `security.py``/` 拦截,考虑白名单
- [ ] INIT 与 backend 初始化顺序整理(避免 backend 在配置到达前实例化)
### Phase 2 (4-8 周): 多租户基础
- [ ] BoxSpec 加 `tenant_id` 字段
- [ ] 容器 labels 加 tenant 标识
- [ ] Redis 配额计数器(并发/执行次数/时间)
- [ ] RBAC 基础框架
- [ ] 定时 session reaper
### Phase 3 (8-16 周): 生产就绪
- [ ] Prometheus metrics exporter
- [ ] 前端 Box 状态面板
- [ ] K8s backend 支持 (方案 B)
- [ ] 结构化日志 (JSON, trace_id)
- [ ] 水平扩展支持

View File

@@ -1,222 +0,0 @@
# Box Runtime vs Plugin Runtime: 连接架构对比
> 更新日期: 2026-06-02
> 状态更新: 自部署社区版已具备发布条件box 可选、降级完善、无迁移欠债);工具调用循环上限、配额遍历异步化、`host_path` 挂载白名单等已落地。剩余多租户 / 安全硬化项见 [SaaS 阻塞项清单](./box-issues.md)。
> 分支: `feat/sandbox` (LangBot + langbot-plugin-sdk)
---
## 1. 总体差异
| 维度 | Plugin Runtime | Box Runtime |
|------|---------------|-------------|
| **继承关系** | `PluginRuntimeConnector(ManagedRuntimeConnector)` | `BoxRuntimeConnector`(独立类) |
| **传输分支** | 3 条 (Docker/WS, Win32/subprocess+WS, Unix/stdio) | 3 条 (本地 stdio, Win32/subprocess+WS, 远程 WS) |
| **心跳** | 20s ping loop | 20s ping loop`_heartbeat_loop` |
| **重连** | WS 模式: sleep 3s → re-initialize | 由 BoxService `_reconnect_loop` 处理,指数退避 |
| **Handler 类型** | `RuntimeConnectionHandler` (1132 行, 25+ action) | 基础 `Handler` + `BoxServerHandler`SDK 端 25 action |
| **Client 抽象** | Handler 即 API | 独立 `ActionRPCBoxClient` 封装 Handler |
| **启用/禁用** | `is_enable_plugin` 开关 | 无开关(可用/不可用由初始化结果决定) |
| **初始化失败** | 异常上抛 | 静默降级 `_available=False` |
| **Shutdown** | 直接杀进程 | RPC SHUTDOWN → 清理容器 → 再杀进程 |
---
## 2. 传输决策
### Plugin: 3-路决策
```python
# pkg/plugin/connector.py:106-165
if get_platform() == 'docker' or use_websocket_to_connect_plugin_runtime():
# Docker/WS → ws://langbot_plugin_runtime:5400/control/ws
elif get_platform() == 'win32':
# Windows → 起子进程(无 pipe) + ws://localhost:5400/control/ws
else:
# Unix/Mac → StdioClientController(python -m langbot_plugin.cli rt -s)
```
### Box: 3-路决策
```python
# pkg/box/connector.py
if self._uses_websocket():
if platform.get_platform() == 'win32' and not self.configured_runtime_url:
await self._start_subprocess_then_ws() # subprocess + ws://localhost:5410/rpc/ws
else:
await self._connect_remote_ws() # ws://{host}:5410/rpc/ws
else:
await self._start_local_stdio() # StdioClientController
```
> 历史2026-04-16 版本本文档曾把 Box 描述为 2 路决策(缺 Windows 分支)。现已对齐 Plugin 的 3 路设计。
### 决策矩阵
| 环境 | Plugin | Box |
|------|--------|-----|
| Docker | WS → `:5400` | WS → `:5410/rpc/ws` |
| `--standalone-box` | N/A | WS → `localhost:5410/rpc/ws` |
| Windows 非 Docker | subprocess + WS (`:5400`) | subprocess + WS (`localhost:5410/rpc/ws`) |
| Unix/Mac 非 Docker | stdio | stdio |
| 手动配置 URL | 通过配置项 | WS → 用户配置的 URL |
---
## 3. 连接建立
### 同步模式差异
**Plugin**: `new_connection_callback` 内直接 ping + await handler_task`initialize()` 通过 `create_task()` 异步启动,不阻塞等待连接。
**Box**: 使用 `asyncio.Event` + `wait_for(timeout=30s)` 模式,`initialize()` 同步等待连接成功或超时。
### Box stdio 路径
```
connector._start_local_stdio()
├─ connected = asyncio.Event()
├─ ctrl = StdioClientController(python, ['-m', 'langbot_plugin.cli.__init__', 'box', '-s', '--ws-control-port', N])
├─ _ctrl_task = create_task(ctrl.run(callback))
│ callback:
│ handler = Handler(connection) ← 基础 Handler, 无 disconnect_callback
│ client.set_handler(handler)
│ _handler_task = create_task(handler.run())
│ call_action(PING, {}) ← 握手, timeout=15s
│ connected.set() ← 通知外层
│ await _handler_task ← 阻塞直到断开
└─ await wait_for(connected.wait(), 30s) ← 同步等待
```
### Plugin stdio 路径
```
connector.initialize()
├─ ctrl = StdioClientController(python, ['-m', 'langbot_plugin.cli', 'rt', '-s'])
├─ task = ctrl.run(callback)
│ callback:
│ disconnect_callback:
│ [WS] → runtime_disconnect_callback → 重连
│ [stdio] → 仅日志, 不重连
│ handler = RuntimeConnectionHandler(conn, disconnect_cb, ap)
│ create_task(handler.run())
│ handler.ping() ← 握手, timeout=10s
│ await handler_task ← 阻塞直到断开
├─ create_task(heartbeat_loop()) ← 20s ping loop
└─ create_task(task) ← 不等待连接
```
---
## 4. 心跳与重连
### 心跳
| 维度 | Plugin | Box |
|------|--------|-----|
| 有心跳? | 是 | 是(`connector.py` `_heartbeat_loop` |
| 间隔 | 20s | 20s |
| 失败处理 | 仅 DEBUG 日志,不触发重连 | 仅 DEBUG 日志,依赖 connection close 触发重连 |
| 生命周期 | 整个应用生命周期 | 连接建立后启动;`dispose()` 时 cancel |
### 重连
| 维度 | Plugin | Box |
|------|--------|-----|
| Docker/WS 断开 | `runtime_disconnect_callback` → sleep 3s → re-initialize | `runtime_disconnect_callback``BoxService._reconnect_loop()`(指数退避) |
| WS 连接失败 | 同上 | 同上;初次失败时 `_available=False`,重连成功后恢复 |
| stdio 断开 | 仅日志,不重连 | 接同样回调stdio 重连需重新 fork 子进程 |
| 重连退避 | 固定 3s无 backoff | 指数退避 |
> 历史2026-04-16 版本本文档曾把心跳与重连标记为 Box 缺失。这两项已在 commit `2dfd9d5d` / `c6882cf` / `5029d9c` 等修复(详见 [box-issues.md 已解决](./box-issues.md))。
---
## 5. 共享 IO 层
两者复用同一套 SDK IO 基础设施:
```
Handler ← ABC (runtime/io/handler.py)
├── RuntimeConnectionHandler (Plugin 用, LangBot 侧)
├── ControlConnectionHandler (Plugin 用, SDK 侧)
├── BoxServerHandler (Box 用, SDK 侧)
└── 匿名 Handler 实例 (Box 用, LangBot 侧)
Connection ← ABC
├── StdioConnection (stdio: 16KB chunks, 应用层分帧协议)
└── WebSocketConnection (WS: 64KB chunks, 原生 WS 分帧)
Controller ← ABC
├── StdioClientController (fork 子进程, pipe stdin/stdout)
├── StdioServerController (接管当前进程 stdin/stdout)
├── WebSocketClientController (连接 WS 服务端)
└── WebSocketServerController (监听 WS 端口)
```
共享的核心机制:
- `call_action()` / `call_action_generator()` — RPC 调用/流式调用
- `ActionRequest` / `ActionResponse` — 请求/响应协议
- `seq_id` 关联 — 并发请求复用单连接
- `CommonAction.PING` — 两者都用于初始握手
- 文件传输 (`send_file`) — Plugin 用Box 不用
---
## 6. 端口方案
| 服务 | Plugin | Box |
|------|--------|-----|
| Action RPC (stdio) | stdin/stdout | stdin/stdout |
| Action RPC (WS) | `:5400` | `:5410/rpc/ws` |
| 辅助服务 | debug WS `:5401` | managed process WS relay `:5410/v1/sessions/{id}/managed-process/ws` |
**Box 特点**: 单端口 aiohttp 服务(默认 5410通过路径区分 Action RPC 和 managed process relay。即使在 stdio 模式,也在 `:5410` 启动 aiohttp 用于 managed process attach。Plugin 在 stdio 模式不开额外端口。
---
## 7. 销毁对比
### Plugin
```python
dispose():
if stdio: ctrl.process.terminate()
_dispose_subprocess() # Windows 子进程
heartbeat_task.cancel()
```
### Box
```python
connector.dispose():
_handler_task.cancel()
_ctrl_task.cancel()
_subprocess.terminate()
service.dispose():
connector.dispose()
loop.create_task(client.shutdown()) # RPC SHUTDOWN → 清理所有容器
```
Box 的 RPC SHUTDOWN 确保容器被正确停止不会成为孤儿。Plugin 直接杀进程。
---
## 8. 改进建议
### P0
1. **两者都加 WS 认证**: 至少 token 认证INIT 时下发,连接时校验)
### P1
2. **考虑 Box 继承 ManagedRuntimeConnector**: 复用 `_start_runtime_subprocess` / `_wait_until_ready` / `_dispose_subprocess`,减少重复代码
3. **Plugin 重连加退避**: 固定 3s 无 backoff 可能造成日志洪水,建议向 Box 的指数退避看齐
4. **统一连接管理模式**: Event-based (Box) vs direct-await (Plugin),考虑收敛为一种
### 已完成(自上一轮)
- ~~Box 加重连~~commit `2dfd9d5d`
- ~~Box 加心跳~~20s loop 与 Plugin 一致)
- ~~Box 加 Windows 支持~~commit `120817a` / `fafb7a4`

View File

@@ -1,6 +1,6 @@
[project]
name = "langbot"
version = "4.10.0-beta.1"
version = "4.9.3"
description = "Production-grade platform for building agentic IM bots"
readme = "README.md"
license-files = ["LICENSE"]
@@ -8,7 +8,7 @@ requires-python = ">=3.11,<4.0"
dependencies = [
"aiocqhttp>=1.4.4",
"aiofiles>=24.1.0",
"aiohttp>=3.13.4",
"aiohttp>=3.11.18",
"aioshutil>=1.5",
"aiosqlite>=0.21.0",
"anthropic>=0.51.0",
@@ -16,18 +16,18 @@ dependencies = [
"async-lru>=2.0.5",
"certifi>=2025.4.26",
"colorlog~=6.6.0",
"cryptography>=46.0.7",
"cryptography>=44.0.3",
"dashscope>=1.25.10",
"dingtalk-stream>=0.24.0",
"discord-py>=2.5.2",
"pynacl>=1.5.0", # Required for Discord voice support
"gewechat-client>=0.1.5",
"lark-oapi>=1.5.5",
"lark-oapi>=1.4.15",
"mcp>=1.25.0",
"nakuru-project-idk>=0.0.2.1",
"ollama>=0.4.8",
"openai>1.0.0",
"pillow>=12.2.0",
"pillow>=11.2.1",
"psutil>=7.0.0",
"pycryptodome>=3.22.0",
"pydantic>2.0",
@@ -35,12 +35,10 @@ dependencies = [
"python-telegram-bot>=22.0",
"pyyaml>=6.0.2",
"qq-botpy-rc>=1.2.1.6",
"qrcode>=7.4",
"quart>=0.20.0",
"quart-cors>=0.8.0",
"requests>=2.32.3",
"slack-sdk>=3.35.0",
"alembic>=1.15.0",
"sqlalchemy[asyncio]>=2.0.40",
"sqlmodel>=0.0.24",
"telegramify-markdown>=0.5.1",
@@ -51,7 +49,7 @@ dependencies = [
"pip>=25.1.1",
"ruff>=0.11.9",
"pre-commit>=4.2.0",
"uv>=0.11.6",
"uv>=0.7.11",
"mypy>=1.16.0",
"PyPDF2>=3.0.1",
"python-docx>=1.1.0",
@@ -62,18 +60,13 @@ dependencies = [
"ebooklib>=0.18",
"html2text>=2024.2.26",
"langchain>=0.2.0",
"langchain-core>=1.2.28",
"langsmith>=0.7.31",
"python-multipart>=0.0.26",
"Mako>=1.3.11",
"langchain-text-splitters>=1.1.2",
"langchain-text-splitters>=0.0.1",
"chromadb>=1.0.0,<2.0.0",
"qdrant-client (>=1.15.1,<2.0.0)",
"pyseekdb==1.1.0.post3",
"langbot-plugin==0.4.0",
"langbot-plugin==0.3.3",
"asyncpg>=0.30.0",
"line-bot-sdk>=3.19.0",
"matrix-nio>=0.25.2",
"tboxsdk>=0.0.10",
"boto3>=1.35.0",
"pymilvus>=2.6.4",
@@ -105,9 +98,6 @@ classifiers = [
"Topic :: Communications :: Chat",
]
[tool.uv.sources]
langbot-plugin = { path = "../langbot-plugin-sdk", editable = true }
[project.urls]
Homepage = "https://langbot.app"
Documentation = "https://docs.langbot.app"
@@ -121,13 +111,12 @@ requires = ["setuptools>=61.0", "wheel"]
build-backend = "setuptools.build_meta"
[tool.setuptools]
package-data = { "langbot" = ["templates/**", "pkg/provider/modelmgr/requesters/*", "pkg/platform/sources/*", "web/dist/**", "pkg/persistence/alembic/**"] }
package-data = { "langbot" = ["templates/**", "pkg/provider/modelmgr/requesters/*", "pkg/platform/sources/*", "web/out/**"] }
[dependency-groups]
dev = [
"moto>=5.2.1",
"pre-commit>=4.2.0",
"pytest>=9.0.3",
"pytest>=8.4.1",
"pytest-asyncio>=1.0.0",
"pytest-cov>=7.0.0",
"ruff>=0.11.9",
@@ -226,3 +215,4 @@ skip-magic-trailing-comma = false
# Like Black, automatically detect the appropriate line ending.
line-ending = "auto"

View File

@@ -4,9 +4,6 @@ python_files = test_*.py
python_classes = Test*
python_functions = test_*
# Python path for imports
pythonpath = . tests
# Test paths
testpaths = tests
@@ -25,9 +22,7 @@ markers =
asyncio: mark test as async
unit: mark test as unit test
integration: mark test as integration test
smoke: mark test as smoke test
slow: mark test as slow running
e2e: mark test as end-to-end test (requires real LangBot process)
# Coverage options (when using pytest-cov)
[coverage:run]

View File

@@ -1,65 +0,0 @@
#!/bin/bash
# Coverage gate script
# Runs all tests with coverage, enforcing minimum coverage threshold
# Uses separate pytest invocations to avoid sys.modules pollution between test types
set -euo pipefail
echo "=== LangBot Coverage Gate ==="
echo ""
# Coverage threshold (baseline from current coverage, conservative buffer)
# Current: ~22.14%, threshold: 18%
COVERAGE_THRESHOLD=18
# Create temporary directory for coverage files
COV_DIR=$(mktemp -d)
trap "rm -rf $COV_DIR" EXIT
echo "[1/3] Running unit + smoke tests with coverage..."
uv run pytest tests/unit_tests/ tests/smoke/ \
--cov=langbot \
--cov-report=json:$COV_DIR/unit.json \
--cov-report=term-missing \
-q --tb=short
echo ""
echo "[2/3] Running fast integration tests with coverage..."
uv run pytest tests/integration/ -m "not slow" \
--cov=langbot \
--cov-report=json:$COV_DIR/integration.json \
--cov-report=term-missing \
-q --tb=short
echo ""
echo "[3/3] Combining coverage reports..."
# Use coverage combine if available, otherwise just report total
if command -v coverage &> /dev/null; then
# Combine JSON reports
coverage combine --keep $COV_DIR/unit.json $COV_DIR/integration.json \
--data-file=$COV_DIR/combined.data 2>/dev/null || true
coverage report --data-file=$COV_DIR/combined.data || true
else
echo "Note: coverage combine not available, showing individual reports above"
fi
# Generate final XML report for CI (from last run)
uv run pytest tests/unit_tests/ tests/smoke/ \
--cov=langbot \
--cov-report=xml:coverage.xml \
--cov-report=term \
--cov-fail-under=$COVERAGE_THRESHOLD \
-q 2>/dev/null || {
# If threshold check fails on combined, check unit+smoke baseline
echo ""
echo "Coverage threshold: $COVERAGE_THRESHOLD%"
echo "Note: Full coverage requires running all test types separately"
}
echo ""
echo "=== Coverage Gate Complete ==="
echo ""
echo "Coverage baseline: $COVERAGE_THRESHOLD%"
echo "Coverage report saved to coverage.xml"

View File

@@ -1,16 +0,0 @@
#!/bin/bash
# Fast integration tests
# Runs integration tests excluding slow ones (PostgreSQL, external services)
# Uses fake runner/provider, no real credentials needed
set -euo pipefail
echo "=== LangBot Fast Integration Tests ==="
echo ""
echo "Running integration tests (excluding slow)..."
uv run pytest tests/integration/ -m "not slow" -q --tb=short
echo ""
echo "=== Fast Integration Tests Complete ==="

View File

@@ -1,36 +0,0 @@
#!/bin/bash
# Quick developer self-test command
# Runs linting, unit tests, and smoke tests without requiring real provider keys
# Suitable for local branch validation
set -euo pipefail
echo "=== LangBot Quick Self-Test ==="
echo ""
# 1. Ruff check
echo "[1/3] Running ruff check..."
uv run ruff check src/langbot/ tests/ --output-format=concise || {
echo ""
echo "⚠ Ruff check found issues. Run 'uv run ruff check --fix' to auto-fix."
exit 1
}
echo "✓ Ruff check passed"
echo ""
# 2. Unit tests
echo "[2/3] Running unit tests..."
uv run pytest tests/unit_tests/ -q --tb=short
echo ""
# 3. Smoke tests (if exists)
echo "[3/3] Running smoke tests..."
if [ -d "tests/smoke" ]; then
uv run pytest tests/smoke/ -q --tb=short
else
echo "No smoke tests found, skipping"
fi
echo ""
echo "=== Quick Self-Test Complete ==="

View File

@@ -1,3 +1,3 @@
"""LangBot - Production-grade platform for building agentic IM bots"""
__version__ = '4.10.0-beta.1'
__version__ = '4.9.3'

View File

@@ -5,8 +5,6 @@ import argparse
import sys
import os
from langbot.pkg.utils import paths
# ASCII art banner
asciiart = r"""
_ ___ _
@@ -29,12 +27,6 @@ async def main_entry(loop: asyncio.AbstractEventLoop):
help='Use standalone plugin runtime / 使用独立插件运行时',
default=False,
)
parser.add_argument(
'--standalone-box',
action='store_true',
help='Use standalone box runtime / 使用独立 Box 运行时',
default=False,
)
parser.add_argument('--debug', action='store_true', help='Debug mode / 调试模式', default=False)
args = parser.parse_args()
@@ -43,11 +35,6 @@ async def main_entry(loop: asyncio.AbstractEventLoop):
platform.standalone_runtime = True
if args.standalone_box:
from langbot.pkg.utils import platform
platform.standalone_box = True
if args.debug:
from langbot.pkg.utils import constants
@@ -100,7 +87,7 @@ def main():
# Set up the working directory
# When installed as a package, we need to handle the working directory differently
# We'll create data directory in current working directory if not exists
os.makedirs(paths.get_data_root(), exist_ok=True)
os.makedirs('data', exist_ok=True)
loop = asyncio.new_event_loop()

View File

@@ -182,88 +182,6 @@ class DingTalkClient:
for handler in self._message_handlers[msg_type]:
await handler(event)
async def _parse_quoted_message(self, replied_msg: dict) -> dict:
"""Parse the quoted/replied message and extract its content.
Args:
replied_msg: The repliedMsg object from DingTalk message
Returns:
A dict containing the quoted message info with keys:
- message_id: The original message ID
- msg_type: The message type (text, file, picture, audio, etc.)
- content: The text content (if any)
- file_url: The file download URL (if file type)
- file_name: The file name (if file type)
- picture: The picture base64 (if picture type)
- audio: The audio base64 (if audio type)
"""
quote_info = {
'message_id': replied_msg.get('msgId', ''),
'msg_type': replied_msg.get('msgType', ''),
'sender_id': replied_msg.get('senderId', ''),
}
msg_type = replied_msg.get('msgType', '')
content = replied_msg.get('content', {})
# Handle content as string (JSON) or dict
if isinstance(content, str):
try:
content = json.loads(content)
except (json.JSONDecodeError, TypeError):
content = {}
if msg_type == 'text':
# Text message
if isinstance(content, dict):
quote_info['content'] = content.get('content', '')
else:
quote_info['content'] = str(content)
elif msg_type == 'file':
# File message
download_code = content.get('downloadCode')
file_name = content.get('fileName')
if download_code and file_name:
try:
quote_info['file_url'] = await self.get_file_url(download_code)
quote_info['file_name'] = file_name
except Exception as e:
if self.logger:
await self.logger.error(f'Failed to get quoted file URL: {e}')
elif msg_type == 'picture':
# Picture message
download_code = content.get('downloadCode')
if download_code:
try:
quote_info['picture'] = await self.download_image(download_code)
except Exception as e:
if self.logger:
await self.logger.error(f'Failed to download quoted image: {e}')
elif msg_type == 'audio':
# Audio message
download_code = content.get('downloadCode')
if download_code:
try:
quote_info['audio'] = await self.get_audio_url(download_code)
except Exception as e:
if self.logger:
await self.logger.error(f'Failed to get quoted audio: {e}')
elif msg_type == 'richText':
# Rich text message - extract text content
rich_text = content.get('richText', [])
texts = []
for item in rich_text:
if 'text' in item and item['text'] != '\n':
texts.append(item['text'])
quote_info['content'] = '\n'.join(texts)
return quote_info
async def get_message(self, incoming_message: dingtalk_stream.chatbot.ChatbotMessage):
try:
# print(json.dumps(incoming_message.to_dict(), indent=4, ensure_ascii=False))
@@ -275,15 +193,6 @@ class DingTalkClient:
elif str(incoming_message.conversation_type) == '2':
message_data['conversation_type'] = 'GroupMessage'
# Check for quoted/replied message
raw_data = incoming_message.to_dict()
text_data = raw_data.get('text', {})
if isinstance(text_data, dict) and text_data.get('isReplyMsg'):
replied_msg = text_data.get('repliedMsg', {})
if replied_msg:
quote_info = await self._parse_quoted_message(replied_msg)
message_data['QuotedMessage'] = quote_info
if incoming_message.message_type == 'richText':
data = incoming_message.rich_text_content.to_dict()
@@ -359,52 +268,19 @@ class DingTalkClient:
message_data['Type'] = 'image'
elif incoming_message.message_type == 'audio':
raw_content = incoming_message.to_dict().get('content', {})
# 兼容处理:如果 content 仍为 JSON 字符串则进行解析
if isinstance(raw_content, str):
try:
raw_content = json.loads(raw_content)
except (json.JSONDecodeError, TypeError):
raw_content = {}
if self.logger:
await self.logger.info(f'DingTalk audio raw content: {json.dumps(raw_content, ensure_ascii=False)}')
# 提取钉钉自带的语音转写文字Powered by Qwen
recognition = raw_content.get('recognition', '')
if recognition:
message_data['Content'] = recognition
download_code = raw_content.get('downloadCode')
if download_code:
message_data['Audio'] = await self.get_audio_url(download_code)
message_data['Audio'] = await self.get_audio_url(incoming_message.to_dict()['content']['downloadCode'])
message_data['Type'] = 'audio'
elif incoming_message.message_type == 'file':
# 获取原始数据字典并提取嵌套的文件信息
raw_data = incoming_message.to_dict()
file_info = raw_data.get('content', {})
# 兼容处理:如果 content 仍为 JSON 字符串则进行解析
if isinstance(file_info, str):
try:
file_info = json.loads(file_info)
except (json.JSONDecodeError, TypeError):
file_info = {}
download_code = file_info.get('downloadCode')
file_name = file_info.get('fileName')
if download_code and file_name:
# 转换 downloadCode 为可下载的真实 URL
message_data['File'] = await self.get_file_url(download_code)
message_data['Name'] = file_name
down_list = incoming_message.get_down_list()
if len(down_list) >= 2:
message_data['File'] = await self.get_file_url(down_list[0])
message_data['Name'] = down_list[1]
else:
if self.logger:
await self.logger.error(f'Failed to extract file info from message content: {file_info}')
await self.logger.error(f'get_down_list() returned fewer than 2 elements: {down_list}')
message_data['File'] = None
message_data['Name'] = None
message_data['Type'] = 'file'
copy_message_data = message_data.copy()
@@ -481,12 +357,6 @@ class DingTalkClient:
card_data['config'] = json.dumps({'autoLayout': card_auto_layout})
card_data['content'] = ''
# 将用户的消息内容作为卡片的查询参数,方便后续处理
if incoming_message.message_type == 'text':
card_data['query'] = incoming_message.get_text_list()[0]
else:
card_data['query'] = '...'
card_instance = dingtalk_stream.AICardReplier(self.client, incoming_message)
# print(card_instance)
# 先投放卡片: https://open.dingtalk.com/document/orgapp/create-and-deliver-cards

View File

@@ -47,22 +47,6 @@ class DingTalkEvent(dict):
def conversation(self):
return self.get('conversation_type', '')
@property
def quoted_message(self) -> Optional[Dict[str, Any]]:
"""Get the quoted/replied message info if this is a reply message.
Returns:
A dict containing:
- message_id: The original message ID
- msg_type: The message type (text, file, picture, audio, etc.)
- content: The text content (if any)
- file_url: The file download URL (if file type)
- file_name: The file name (if file type)
- picture: The picture base64 (if picture type)
- audio: The audio base64 (if audio type)
"""
return self.get('QuotedMessage')
def __getattr__(self, key: str) -> Optional[Any]:
"""
允许通过属性访问数据中的任意字段。

View File

@@ -1,3 +0,0 @@
from .client import OpenClawWeixinClient as OpenClawWeixinClient
from .types import ApiError as ApiError
from .types import LoginResult as LoginResult

View File

@@ -1,807 +0,0 @@
"""Async HTTP client for the OpenClaw WeChat API.
Implements the iLink Bot API protocol.
Reference: https://github.com/epiral/weixin-bot
Endpoints: getUpdates (long-poll), sendMessage, getUploadUrl, getConfig, sendTyping.
"""
from __future__ import annotations
import asyncio
import base64
import io
import logging
import os
import struct
import typing
import uuid
from typing import Optional
from urllib.parse import quote
import aiohttp
from .types import (
ApiError,
CDNMedia,
FileItem,
GetConfigResponse,
GetUpdatesResponse,
GetUploadUrlResponse,
ImageItem,
LoginResult,
MessageItem,
QRCodeResponse,
QRStatusResponse,
RefMessage,
TextItem,
VideoItem,
VoiceItem,
WeixinMessage,
)
logger = logging.getLogger('openclaw-weixin-sdk')
DEFAULT_BASE_URL = 'https://ilinkai.weixin.qq.com'
CDN_BASE_URL = 'https://novac2c.cdn.weixin.qq.com/c2c'
CHANNEL_VERSION = '1.0.0'
DEFAULT_API_TIMEOUT = 15
DEFAULT_LONG_POLL_TIMEOUT = 40
DEFAULT_CONFIG_TIMEOUT = 10
DEFAULT_QR_POLL_TIMEOUT = 35
SESSION_EXPIRED_ERRCODE = -14
DEFAULT_BOT_TYPE = '3'
# Maximum text length per message chunk (WeChat limit)
MAX_TEXT_CHUNK_SIZE = 2000
def _random_wechat_uin() -> str:
"""Generate the X-WECHAT-UIN header: random uint32 -> decimal string -> base64."""
rand_bytes = os.urandom(4)
uint32_val = struct.unpack('>I', rand_bytes)[0]
return base64.b64encode(str(uint32_val).encode('utf-8')).decode('utf-8')
def _build_base_info() -> dict:
"""Build the base_info payload included in every API request."""
return {'channel_version': CHANNEL_VERSION}
def _chunk_text(text: str, max_size: int = MAX_TEXT_CHUNK_SIZE) -> list[str]:
"""Split long text into chunks that fit within WeChat's message size limit."""
if len(text) <= max_size:
return [text]
chunks = []
while text:
chunks.append(text[:max_size])
text = text[max_size:]
return chunks
class OpenClawWeixinClient:
"""Async client for the OpenClaw WeChat HTTP JSON API."""
def __init__(self, base_url: str, token: str):
self.base_url = base_url.rstrip('/')
self.token = token
self._session: Optional[aiohttp.ClientSession] = None
async def _get_session(self) -> aiohttp.ClientSession:
if self._session is None or self._session.closed:
self._session = aiohttp.ClientSession()
return self._session
async def close(self):
if self._session and not self._session.closed:
await self._session.close()
def _build_headers(self) -> dict[str, str]:
headers = {
'Content-Type': 'application/json',
'AuthorizationType': 'ilink_bot_token',
'X-WECHAT-UIN': _random_wechat_uin(),
}
if self.token:
headers['Authorization'] = f'Bearer {self.token}'
return headers
async def _post(self, endpoint: str, payload: dict, timeout: float = DEFAULT_API_TIMEOUT) -> dict:
"""Make a POST request and return the JSON response.
Raises ApiError on HTTP errors or when the response contains a non-zero errcode.
"""
payload['base_info'] = _build_base_info()
session = await self._get_session()
url = f'{self.base_url}/{endpoint}'
headers = self._build_headers()
async with session.post(
url, json=payload, headers=headers, timeout=aiohttp.ClientTimeout(total=timeout)
) as resp:
if resp.status != 200:
text = await resp.text()
raise ApiError(
f'OpenClaw API error {resp.status}: {text}',
status=resp.status,
)
data = await resp.json(content_type=None)
# Check for application-level errors in the response body
errcode = data.get('errcode') or data.get('ret')
if errcode and errcode != 0:
raise ApiError(
data.get('errmsg') or f'API errcode {errcode}',
status=200,
code=errcode,
payload=data,
)
return data
async def get_updates(
self, get_updates_buf: str = '', timeout: float = DEFAULT_LONG_POLL_TIMEOUT
) -> GetUpdatesResponse:
"""Long-poll for new messages.
Note: This method does NOT raise ApiError for errcode responses —
it returns them in the GetUpdatesResponse so the caller can handle
session expiry and other errors with full context.
"""
try:
# Bypass the errcode check in _post since get_updates needs
# to return error info (e.g. session expired) to the caller.
payload: dict = {'get_updates_buf': get_updates_buf}
payload['base_info'] = _build_base_info()
session = await self._get_session()
url = f'{self.base_url}/ilink/bot/getupdates'
headers = self._build_headers()
async with session.post(
url,
json=payload,
headers=headers,
timeout=aiohttp.ClientTimeout(total=timeout),
) as resp:
if resp.status != 200:
text = await resp.text()
raise ApiError(
f'OpenClaw API error {resp.status}: {text}',
status=resp.status,
)
data = await resp.json(content_type=None)
except (asyncio.TimeoutError, aiohttp.ServerTimeoutError):
return GetUpdatesResponse(ret=0, msgs=[], get_updates_buf=get_updates_buf)
except ApiError:
raise
except Exception as e:
if 'timeout' in str(e).lower():
return GetUpdatesResponse(ret=0, msgs=[], get_updates_buf=get_updates_buf)
raise
return _parse_get_updates_response(data)
async def send_message(
self,
to_user_id: str,
item_list: list[MessageItem],
context_token: str = '',
) -> None:
"""Send a message to a user."""
items_payload = [_message_item_to_dict(item) for item in item_list]
payload = {
'msg': {
'from_user_id': '',
'to_user_id': to_user_id,
'client_id': f'langbot-{uuid.uuid4().hex[:16]}',
'message_type': WeixinMessage.TYPE_BOT,
'message_state': WeixinMessage.STATE_FINISH,
'item_list': items_payload,
'context_token': context_token or None,
}
}
await self._post('ilink/bot/sendmessage', payload)
async def send_text(self, to_user_id: str, text: str, context_token: str = '') -> None:
"""Send a plain text message, automatically chunking if too long."""
chunks = _chunk_text(text)
for chunk in chunks:
item = MessageItem(type=MessageItem.TEXT, text_item=TextItem(text=chunk))
await self.send_message(to_user_id, [item], context_token)
async def get_config(self, ilink_user_id: str, context_token: str = '') -> GetConfigResponse:
"""Get bot config including typing_ticket."""
data = await self._post(
'ilink/bot/getconfig',
{'ilink_user_id': ilink_user_id, 'context_token': context_token or None},
timeout=DEFAULT_CONFIG_TIMEOUT,
)
return GetConfigResponse(
ret=data.get('ret'),
errmsg=data.get('errmsg'),
typing_ticket=data.get('typing_ticket'),
)
async def send_typing(self, ilink_user_id: str, typing_ticket: str, status: int = 1) -> None:
"""Send typing indicator. status: 1=typing, 2=cancel."""
await self._post(
'ilink/bot/sendtyping',
{
'ilink_user_id': ilink_user_id,
'typing_ticket': typing_ticket,
'status': status,
},
timeout=DEFAULT_CONFIG_TIMEOUT,
)
async def stop_typing(self, ilink_user_id: str, typing_ticket: str) -> None:
"""Cancel the typing indicator for a user."""
await self.send_typing(ilink_user_id, typing_ticket, status=2)
async def download_media(
self,
media: CDNMedia,
) -> bytes:
"""Download and decrypt a file from the WeChat CDN.
Args:
media: CDNMedia object with encrypt_query_param and aes_key.
Returns:
Decrypted file bytes.
"""
from cryptography.hazmat.primitives.ciphers import Cipher, algorithms, modes
from cryptography.hazmat.primitives.padding import PKCS7
if not media.encrypt_query_param:
raise ApiError('CDN media has no encrypt_query_param', status=0)
if not media.aes_key:
raise ApiError('CDN media has no aes_key', status=0)
# Derive 16-byte AES key
# aes_key is base64-encoded; the decoded content may be:
# - raw 16 bytes (direct AES key)
# - 32-char hex string (decode hex to get 16 bytes)
raw = base64.b64decode(media.aes_key)
if len(raw) == 16:
aes_key = raw
elif len(raw) == 32:
# Hex-encoded 16-byte key
aes_key = bytes.fromhex(raw.decode('utf-8'))
else:
raise ApiError(f'Invalid AES key length: {len(raw)} (expected 16 or 32)', status=0)
# Download encrypted bytes from CDN
session = await self._get_session()
cdn_url = f'{CDN_BASE_URL}/download?encrypted_query_param={quote(media.encrypt_query_param, safe="")}'
async with session.get(cdn_url, timeout=aiohttp.ClientTimeout(total=120)) as resp:
if resp.status != 200:
text = await resp.text()
raise ApiError(f'CDN download failed: {resp.status} {text}', status=resp.status)
encrypted = await resp.read()
# Decrypt AES-128-ECB with PKCS7 padding
cipher = Cipher(algorithms.AES(aes_key), modes.ECB())
decryptor = cipher.decryptor()
padded = decryptor.update(encrypted) + decryptor.finalize()
unpadder = PKCS7(128).unpadder()
return unpadder.update(padded) + unpadder.finalize()
async def upload_media(
self,
file_bytes: bytes,
to_user_id: str,
media_type: int,
) -> CDNMedia:
"""Encrypt and upload media to WeChat CDN.
Args:
file_bytes: Raw file bytes to upload.
to_user_id: Recipient user ID.
media_type: 1=IMAGE, 2=VIDEO, 3=FILE, 4=VOICE.
Returns:
CDNMedia with encrypt_query_param and aes_key for use in sendMessage.
"""
import hashlib
from cryptography.hazmat.primitives.ciphers import Cipher, algorithms, modes
from cryptography.hazmat.primitives.padding import PKCS7
# 1. Generate random 16-byte AES key
raw_key = os.urandom(16)
aes_key_hex = raw_key.hex() # 32-char hex string
# 2. Encode key for CDNMedia: base64(hex_string) — same for all media types
# Matches official SDK: Buffer.from(aeskey_hex).toString("base64")
encoded_key = base64.b64encode(aes_key_hex.encode('utf-8')).decode('utf-8')
# 3. Encrypt file with AES-128-ECB + PKCS7
padder = PKCS7(128).padder()
padded = padder.update(file_bytes) + padder.finalize()
cipher = Cipher(algorithms.AES(raw_key), modes.ECB())
encryptor = cipher.encryptor()
encrypted = encryptor.update(padded) + encryptor.finalize()
# 4. Get upload URL
raw_md5 = hashlib.md5(file_bytes).hexdigest()
filekey = os.urandom(16).hex() # 32-char hex, matches official SDK
upload_resp = await self.get_upload_url(
filekey=filekey,
media_type=media_type,
to_user_id=to_user_id,
rawsize=len(file_bytes),
rawfilemd5=raw_md5,
filesize=len(encrypted),
aeskey=aes_key_hex, # hex string, as expected by the API
)
if not upload_resp.upload_param:
raise ApiError('Failed to get upload URL', status=0)
# 5. Upload to CDN
# upload_param is an opaque token from the server — pass it as-is
session = await self._get_session()
cdn_url = f'{CDN_BASE_URL}/upload?encrypted_query_param={quote(upload_resp.upload_param, safe="")}&filekey={quote(filekey, safe="")}'
logger.debug(
'CDN upload: url=%s raw_size=%d encrypted_size=%d md5=%s aeskey=%s',
cdn_url,
len(file_bytes),
len(encrypted),
raw_md5,
encoded_key,
)
async with session.post(
cdn_url,
data=encrypted,
headers={'Content-Type': 'application/octet-stream'},
timeout=aiohttp.ClientTimeout(total=120),
) as resp:
if resp.status != 200:
text = await resp.text()
logger.error('CDN upload failed: status=%d url=%s body=%s', resp.status, cdn_url, text[:500])
raise ApiError(f'CDN upload failed: {resp.status} {text}', status=resp.status)
download_param = resp.headers.get('x-encrypted-param', '')
if not download_param:
raise ApiError('CDN upload succeeded but no x-encrypted-param returned', status=0)
return CDNMedia(
encrypt_query_param=download_param,
aes_key=encoded_key,
encrypt_type=1,
)
async def send_image(
self,
to_user_id: str,
image_bytes: bytes,
context_token: str = '',
) -> None:
"""Upload an image to CDN and send it."""
media = await self.upload_media(image_bytes, to_user_id, media_type=1)
item = MessageItem(
type=MessageItem.IMAGE,
image_item=ImageItem(
media=media,
aeskey=media.aes_key,
),
)
await self.send_message(to_user_id, [item], context_token)
async def send_file(
self,
to_user_id: str,
file_bytes: bytes,
file_name: str,
context_token: str = '',
) -> None:
"""Upload a file to CDN and send it."""
import hashlib
media = await self.upload_media(file_bytes, to_user_id, media_type=3)
item = MessageItem(
type=MessageItem.FILE,
file_item=FileItem(
media=media,
file_name=file_name,
md5=hashlib.md5(file_bytes).hexdigest(),
len=str(len(file_bytes)),
),
)
await self.send_message(to_user_id, [item], context_token)
async def send_voice(
self,
to_user_id: str,
voice_bytes: bytes,
playtime: int = 0,
context_token: str = '',
) -> None:
"""Upload a voice message to CDN and send it."""
media = await self.upload_media(voice_bytes, to_user_id, media_type=4)
item = MessageItem(
type=MessageItem.VOICE,
voice_item=VoiceItem(
media=media,
playtime=playtime,
),
)
await self.send_message(to_user_id, [item], context_token)
async def get_upload_url(
self,
filekey: str,
media_type: int,
to_user_id: str,
rawsize: int,
rawfilemd5: str,
filesize: int,
thumb_rawsize: Optional[int] = None,
thumb_rawfilemd5: Optional[str] = None,
thumb_filesize: Optional[int] = None,
aeskey: Optional[str] = None,
) -> GetUploadUrlResponse:
"""Get a pre-signed CDN upload URL."""
payload: dict = {
'filekey': filekey,
'media_type': media_type,
'to_user_id': to_user_id,
'rawsize': rawsize,
'rawfilemd5': rawfilemd5,
'filesize': filesize,
'no_need_thumb': True,
}
if thumb_rawsize is not None:
payload['thumb_rawsize'] = thumb_rawsize
if thumb_rawfilemd5 is not None:
payload['thumb_rawfilemd5'] = thumb_rawfilemd5
if thumb_filesize is not None:
payload['thumb_filesize'] = thumb_filesize
if aeskey is not None:
payload['aeskey'] = aeskey
data = await self._post('ilink/bot/getuploadurl', payload)
logger.debug('get_upload_url response: %s', data)
return GetUploadUrlResponse(
upload_param=data.get('upload_param'),
thumb_upload_param=data.get('thumb_upload_param'),
)
# -----------------------------------------------------------------------
# QR Code Login
# -----------------------------------------------------------------------
async def fetch_qrcode(self, bot_type: str = DEFAULT_BOT_TYPE) -> QRCodeResponse:
"""Fetch a QR code for WeChat login authorization (GET, no auth needed)."""
session = await self._get_session()
url = f'{self.base_url}/ilink/bot/get_bot_qrcode?bot_type={bot_type}'
async with session.get(url, timeout=aiohttp.ClientTimeout(total=DEFAULT_API_TIMEOUT)) as resp:
if resp.status != 200:
text = await resp.text()
raise ApiError(
f'Failed to fetch QR code: {resp.status} {text}',
status=resp.status,
)
data = await resp.json(content_type=None)
logger.debug(
'fetch_qrcode response: qrcode=%s, img=%s', data.get('qrcode'), bool(data.get('qrcode_img_content'))
)
return QRCodeResponse(
qrcode=data.get('qrcode'),
qrcode_img_content=data.get('qrcode_img_content'),
)
async def _fetch_qr_image_base64(self, url: str) -> str:
"""Generate a QR code image from the URL and return a data URI string.
The qrcode_img_content URL points to an HTML page (not a raw image),
so we generate the QR code locally using the qrcode library.
"""
import qrcode
qr = qrcode.QRCode(error_correction=qrcode.constants.ERROR_CORRECT_L)
qr.add_data(url)
qr.make(fit=True)
img = qr.make_image(fill_color='black', back_color='white')
buf = io.BytesIO()
img.save(buf, format='PNG')
b64 = base64.b64encode(buf.getvalue()).decode('utf-8')
return f'data:image/png;base64,{b64}'
async def poll_qrcode_status(self, qrcode: str) -> QRStatusResponse:
"""Long-poll the QR code scan status (GET with iLink-App-ClientVersion header)."""
session = await self._get_session()
url = f'{self.base_url}/ilink/bot/get_qrcode_status?qrcode={quote(qrcode, safe="")}'
headers = {'iLink-App-ClientVersion': '1'}
try:
async with session.get(
url, headers=headers, timeout=aiohttp.ClientTimeout(total=DEFAULT_QR_POLL_TIMEOUT)
) as resp:
if resp.status != 200:
text = await resp.text()
raise ApiError(
f'Failed to poll QR status: {resp.status} {text}',
status=resp.status,
)
data = await resp.json(content_type=None)
logger.debug('QR status poll response: %s', data)
except (asyncio.TimeoutError, aiohttp.ServerTimeoutError):
return QRStatusResponse(status='wait')
return QRStatusResponse(
status=data.get('status'),
bot_token=data.get('bot_token'),
ilink_bot_id=data.get('ilink_bot_id'),
baseurl=data.get('baseurl'),
ilink_user_id=data.get('ilink_user_id'),
)
async def login(
self,
max_retries: int = 5,
poll_timeout_ms: int = 480_000,
on_qrcode: Optional[typing.Callable[[str, str], typing.Any]] = None,
on_status: Optional[typing.Callable[[str], typing.Any]] = None,
) -> LoginResult:
"""Complete QR code login flow with auto-retry on expiry.
Args:
max_retries: Max number of QR code refreshes on expiry.
poll_timeout_ms: Timeout per QR code in milliseconds.
on_qrcode: Callback(qr_image_base64, qr_url) called each time a
new QR code is fetched. Use this to display the QR code.
on_status: Callback(status_str) called on each status poll change.
Returns:
LoginResult with token, base_url, and account_id.
Raises:
ApiError: On unrecoverable API errors.
Exception: If all retries are exhausted.
"""
last_qr_base64: Optional[str] = None
for attempt in range(max_retries):
qr_resp = await self.fetch_qrcode()
if not qr_resp.qrcode or not qr_resp.qrcode_img_content:
raise ApiError('Failed to get QR code from server', status=0)
# Convert QR image to base64 and notify caller
last_qr_base64 = await self._fetch_qr_image_base64(qr_resp.qrcode_img_content)
if on_qrcode:
try:
result = on_qrcode(last_qr_base64, qr_resp.qrcode_img_content)
if asyncio.iscoroutine(result) or asyncio.isfuture(result):
await result
except Exception as e:
logger.warning('on_qrcode callback error: %s', e)
# Poll until confirmed / expired / timeout
loop = asyncio.get_running_loop()
deadline = loop.time() + poll_timeout_ms / 1000.0
while loop.time() < deadline:
try:
status_resp = await self.poll_qrcode_status(qr_resp.qrcode)
except Exception as e:
logger.error('Error polling QR status: %s', e)
await asyncio.sleep(2)
continue
if on_status:
try:
cb_result = on_status(status_resp.status or 'unknown')
if asyncio.iscoroutine(cb_result) or asyncio.isfuture(cb_result):
await cb_result
except Exception as e:
logger.warning('on_status callback error: %s', e)
if status_resp.status == 'confirmed' and status_resp.bot_token:
new_base_url = status_resp.baseurl or self.base_url
# Update this client instance as well
self.token = status_resp.bot_token
self.base_url = new_base_url.rstrip('/')
return LoginResult(
token=status_resp.bot_token,
base_url=new_base_url,
account_id=status_resp.ilink_bot_id or '',
qr_image_base64=last_qr_base64,
)
if status_resp.status == 'expired':
break # retry with a new QR code
await asyncio.sleep(1)
else:
# While-loop ended without break → poll timeout, treat as expired
pass
remaining = max_retries - attempt - 1
if remaining > 0:
logger.info('QR code expired, refreshing... (%d retries left)', remaining)
else:
raise ApiError('QR code login failed: max retries exceeded', status=0)
# Should not reach here, but just in case
raise ApiError('QR code login failed', status=0)
# ---------------------------------------------------------------------------
# Parsing helpers
# ---------------------------------------------------------------------------
def _parse_cdn_media(data: Optional[dict]) -> Optional[CDNMedia]:
if not data:
return None
return CDNMedia(
encrypt_query_param=data.get('encrypt_query_param'),
aes_key=data.get('aes_key'),
encrypt_type=data.get('encrypt_type'),
)
def _parse_message_item(data: dict) -> MessageItem:
item = MessageItem(
type=data.get('type'),
create_time_ms=data.get('create_time_ms'),
update_time_ms=data.get('update_time_ms'),
is_completed=data.get('is_completed'),
msg_id=data.get('msg_id'),
)
if data.get('text_item'):
item.text_item = TextItem(text=data['text_item'].get('text'))
if data.get('image_item'):
img = data['image_item']
item.image_item = ImageItem(
media=_parse_cdn_media(img.get('media')),
thumb_media=_parse_cdn_media(img.get('thumb_media')),
aeskey=img.get('aeskey'),
url=img.get('url'),
mid_size=img.get('mid_size'),
)
if data.get('voice_item'):
v = data['voice_item']
item.voice_item = VoiceItem(
media=_parse_cdn_media(v.get('media')),
encode_type=v.get('encode_type'),
playtime=v.get('playtime'),
text=v.get('text'),
)
if data.get('file_item'):
f = data['file_item']
item.file_item = FileItem(
media=_parse_cdn_media(f.get('media')),
file_name=f.get('file_name'),
md5=f.get('md5'),
len=f.get('len'),
)
if data.get('video_item'):
vid = data['video_item']
item.video_item = VideoItem(
media=_parse_cdn_media(vid.get('media')),
video_size=vid.get('video_size'),
play_length=vid.get('play_length'),
video_md5=vid.get('video_md5'),
thumb_media=_parse_cdn_media(vid.get('thumb_media')),
)
if data.get('ref_msg'):
ref = data['ref_msg']
item.ref_msg = RefMessage(
title=ref.get('title'),
message_item=_parse_message_item(ref['message_item']) if ref.get('message_item') else None,
)
return item
def _parse_weixin_message(data: dict) -> WeixinMessage:
msg = WeixinMessage(
seq=data.get('seq'),
message_id=data.get('message_id'),
from_user_id=data.get('from_user_id'),
to_user_id=data.get('to_user_id'),
client_id=data.get('client_id'),
create_time_ms=data.get('create_time_ms'),
session_id=data.get('session_id'),
group_id=data.get('group_id'),
message_type=data.get('message_type'),
message_state=data.get('message_state'),
context_token=data.get('context_token'),
)
if data.get('item_list'):
msg.item_list = [_parse_message_item(item) for item in data['item_list']]
return msg
def _parse_get_updates_response(data: dict) -> GetUpdatesResponse:
resp = GetUpdatesResponse(
ret=data.get('ret'),
errcode=data.get('errcode'),
errmsg=data.get('errmsg'),
get_updates_buf=data.get('get_updates_buf'),
longpolling_timeout_ms=data.get('longpolling_timeout_ms'),
)
if data.get('msgs'):
resp.msgs = [_parse_weixin_message(m) for m in data['msgs']]
return resp
def _cdn_media_to_dict(media: Optional[CDNMedia]) -> Optional[dict]:
if not media:
return None
d: dict = {}
if media.encrypt_query_param is not None:
d['encrypt_query_param'] = media.encrypt_query_param
if media.aes_key is not None:
d['aes_key'] = media.aes_key
if media.encrypt_type is not None:
d['encrypt_type'] = media.encrypt_type
return d or None
def _message_item_to_dict(item: MessageItem) -> dict:
d: dict = {'type': item.type}
if item.text_item:
d['text_item'] = {'text': item.text_item.text}
if item.image_item:
img_d: dict = {}
if item.image_item.media:
img_d['media'] = _cdn_media_to_dict(item.image_item.media)
if item.image_item.mid_size is not None:
img_d['mid_size'] = item.image_item.mid_size
d['image_item'] = img_d
if item.voice_item:
voice_d: dict = {}
if item.voice_item.media:
voice_d['media'] = _cdn_media_to_dict(item.voice_item.media)
if item.voice_item.playtime is not None:
voice_d['playtime'] = item.voice_item.playtime
d['voice_item'] = voice_d
if item.file_item:
file_d: dict = {}
if item.file_item.media:
file_d['media'] = _cdn_media_to_dict(item.file_item.media)
if item.file_item.file_name:
file_d['file_name'] = item.file_item.file_name
if item.file_item.len:
file_d['len'] = item.file_item.len
d['file_item'] = file_d
if item.video_item:
vid_d: dict = {}
if item.video_item.media:
vid_d['media'] = _cdn_media_to_dict(item.video_item.media)
if item.video_item.video_size is not None:
vid_d['video_size'] = item.video_item.video_size
d['video_item'] = vid_d
return d

View File

@@ -1,200 +0,0 @@
"""Type definitions for the OpenClaw WeChat API, mirroring the upstream protocol."""
from __future__ import annotations
from dataclasses import dataclass, field
from typing import Any, Optional
SESSION_EXPIRED_ERRCODE = -14
class ApiError(Exception):
"""Structured error raised by the OpenClaw WeChat API."""
def __init__(
self,
message: str,
*,
status: int = 0,
code: int | None = None,
payload: Any = None,
):
super().__init__(message)
self.status = status
self.code = code
self.payload = payload
@property
def is_session_expired(self) -> bool:
return self.code == SESSION_EXPIRED_ERRCODE
@dataclass
class CDNMedia:
encrypt_query_param: Optional[str] = None
aes_key: Optional[str] = None
encrypt_type: Optional[int] = None
@dataclass
class TextItem:
text: Optional[str] = None
@dataclass
class ImageItem:
media: Optional[CDNMedia] = None
thumb_media: Optional[CDNMedia] = None
aeskey: Optional[str] = None
url: Optional[str] = None
mid_size: Optional[int] = None
thumb_size: Optional[int] = None
thumb_height: Optional[int] = None
thumb_width: Optional[int] = None
hd_size: Optional[int] = None
_downloaded_bytes: Optional[bytes] = field(default=None, repr=False)
@dataclass
class VoiceItem:
media: Optional[CDNMedia] = None
encode_type: Optional[int] = None
bits_per_sample: Optional[int] = None
sample_rate: Optional[int] = None
playtime: Optional[int] = None
text: Optional[str] = None
_downloaded_bytes: Optional[bytes] = field(default=None, repr=False)
@dataclass
class FileItem:
media: Optional[CDNMedia] = None
file_name: Optional[str] = None
md5: Optional[str] = None
len: Optional[str] = None
_downloaded_bytes: Optional[bytes] = field(default=None, repr=False)
@dataclass
class VideoItem:
media: Optional[CDNMedia] = None
video_size: Optional[int] = None
play_length: Optional[int] = None
video_md5: Optional[str] = None
thumb_media: Optional[CDNMedia] = None
thumb_size: Optional[int] = None
thumb_height: Optional[int] = None
thumb_width: Optional[int] = None
_downloaded_bytes: Optional[bytes] = field(default=None, repr=False)
@dataclass
class RefMessage:
message_item: Optional[MessageItem] = None
title: Optional[str] = None
@dataclass
class MessageItem:
"""A single content item inside a WeixinMessage."""
# Item types
NONE = 0
TEXT = 1
IMAGE = 2
VOICE = 3
FILE = 4
VIDEO = 5
type: Optional[int] = None
create_time_ms: Optional[int] = None
update_time_ms: Optional[int] = None
is_completed: Optional[bool] = None
msg_id: Optional[str] = None
ref_msg: Optional[RefMessage] = None
text_item: Optional[TextItem] = None
image_item: Optional[ImageItem] = None
voice_item: Optional[VoiceItem] = None
file_item: Optional[FileItem] = None
video_item: Optional[VideoItem] = None
@dataclass
class WeixinMessage:
"""Unified message from getUpdates or for sendMessage."""
# Message types
TYPE_USER = 1
TYPE_BOT = 2
# Message states
STATE_NEW = 0
STATE_GENERATING = 1
STATE_FINISH = 2
seq: Optional[int] = None
message_id: Optional[int] = None
from_user_id: Optional[str] = None
to_user_id: Optional[str] = None
client_id: Optional[str] = None
create_time_ms: Optional[int] = None
update_time_ms: Optional[int] = None
delete_time_ms: Optional[int] = None
session_id: Optional[str] = None
group_id: Optional[str] = None
message_type: Optional[int] = None
message_state: Optional[int] = None
item_list: Optional[list[MessageItem]] = None
context_token: Optional[str] = None
@dataclass
class GetUpdatesResponse:
ret: Optional[int] = None
errcode: Optional[int] = None
errmsg: Optional[str] = None
msgs: list[WeixinMessage] = field(default_factory=list)
get_updates_buf: Optional[str] = None
longpolling_timeout_ms: Optional[int] = None
@dataclass
class GetConfigResponse:
ret: Optional[int] = None
errmsg: Optional[str] = None
typing_ticket: Optional[str] = None
@dataclass
class GetUploadUrlResponse:
upload_param: Optional[str] = None
thumb_upload_param: Optional[str] = None
@dataclass
class QRCodeResponse:
"""Response from get_bot_qrcode endpoint."""
qrcode: Optional[str] = None
qrcode_img_content: Optional[str] = None
@dataclass
class QRStatusResponse:
"""Response from get_qrcode_status endpoint."""
status: Optional[str] = None # "wait" | "scaned" | "confirmed" | "expired"
bot_token: Optional[str] = None
ilink_bot_id: Optional[str] = None
baseurl: Optional[str] = None
ilink_user_id: Optional[str] = None
@dataclass
class LoginResult:
"""Result returned by the login flow."""
token: str
base_url: str
account_id: str
qr_image_base64: Optional[str] = None # data URI of the last QR code shown

View File

@@ -1,10 +1,8 @@
import re
import time
import asyncio
from quart import request
import httpx
from quart import Quart
from typing import Callable, Dict, Any, Optional
from typing import Callable, Dict, Any
import langbot_plugin.api.entities.builtin.platform.events as platform_events
from .qqofficialevent import QQOfficialEvent
import json
@@ -34,8 +32,6 @@ class QQOfficialClient:
self.access_token = ''
self.access_token_expiry_time = None
self.logger = logger
self._msg_seq_counter = 0
self._token_refresh_task: Optional[asyncio.Task] = None
async def check_access_token(self):
"""检查access_token是否存在"""
@@ -54,18 +50,18 @@ class QQOfficialClient:
headers = {
'content-type': 'application/json',
}
response = await client.post(url, json=params, headers=headers)
if response.status_code != 200:
raise Exception(f'Failed to get access_token: HTTP {response.status_code} {response.text}')
response_data = response.json()
access_token = response_data.get('access_token')
expires_in = int(response_data.get('expires_in', 7200))
self.access_token_expiry_time = time.time() + expires_in - 60
if access_token:
self.access_token = access_token
await self.logger.info(f'access_token obtained, expires_in={expires_in}s')
else:
raise Exception('Failed to get access_token: no access_token in response')
try:
response = await client.post(url, json=params, headers=headers)
if response.status_code == 200:
response_data = response.json()
access_token = response_data.get('access_token')
expires_in = int(response_data.get('expires_in', 7200))
self.access_token_expiry_time = time.time() + expires_in - 60
if access_token:
self.access_token = access_token
except Exception as e:
await self.logger.error(f'获取access_token失败: {response_data}')
raise Exception(f'获取access_token失败: {e}')
async def handle_callback_request(self):
"""处理回调请求(独立端口模式,使用全局 request"""
@@ -91,10 +87,10 @@ class QQOfficialClient:
try:
body = await req.get_data()
await self.logger.info(f'Received request, body length: {len(body)}')
print(f'[QQ Official] Received request, body length: {len(body)}')
if not body or len(body) == 0:
await self.logger.info('Received empty body, might be health check or GET request')
print('[QQ Official] Received empty body, might be health check or GET request')
return {'code': 0, 'message': 'ok'}, 200
payload = json.loads(body)
@@ -115,6 +111,7 @@ class QQOfficialClient:
return {'code': 0, 'message': 'success'}
except Exception as e:
print(f'[QQ Official] ERROR: {traceback.format_exc()}')
await self.logger.error(f'Error in handle_callback_request: {traceback.format_exc()}')
return {'error': str(e)}, 400
@@ -142,24 +139,21 @@ class QQOfficialClient:
async def get_message(self, msg: dict) -> Dict[str, Any]:
"""获取消息"""
d = msg.get('d', {})
if not isinstance(d, dict):
return {}
message_data = {
't': msg.get('t', {}),
'user_openid': d.get('author', {}).get('user_openid', {}),
'timestamp': d.get('timestamp', {}),
'd_author_id': d.get('author', {}).get('id', {}),
'content': d.get('content', {}),
'd_id': d.get('id', {}),
'user_openid': msg.get('d', {}).get('author', {}).get('user_openid', {}),
'timestamp': msg.get('d', {}).get('timestamp', {}),
'd_author_id': msg.get('d', {}).get('author', {}).get('id', {}),
'content': msg.get('d', {}).get('content', {}),
'd_id': msg.get('d', {}).get('id', {}),
'id': msg.get('id', {}),
'channel_id': d.get('channel_id', {}),
'username': d.get('author', {}).get('username', {}),
'guild_id': d.get('guild_id', {}),
'member_openid': d.get('author', {}).get('openid', {}),
'group_openid': d.get('group_openid', {}),
'channel_id': msg.get('d', {}).get('channel_id', {}),
'username': msg.get('d', {}).get('author', {}).get('username', {}),
'guild_id': msg.get('d', {}).get('guild_id', {}),
'member_openid': msg.get('d', {}).get('author', {}).get('openid', {}),
'group_openid': msg.get('d', {}).get('group_openid', {}),
}
attachments = d.get('attachments', [])
attachments = msg.get('d', {}).get('attachments', [])
image_attachments = [attachment['url'] for attachment in attachments if await self.is_image(attachment)]
image_attachments_type = [
attachment['content_type'] for attachment in attachments if await self.is_image(attachment)
@@ -198,7 +192,7 @@ class QQOfficialClient:
if response.status_code == 200:
return
else:
await self.logger.error(f'Failed to send private message: {response_data}')
await self.logger.error(f'发送私聊消息失败: {response_data}')
raise ValueError(response)
async def send_group_text_msg(self, group_openid: str, content: str, msg_id: str):
@@ -221,7 +215,7 @@ class QQOfficialClient:
if response.status_code == 200:
return
else:
await self.logger.error(f'Failed to send group message: {response.json()}')
await self.logger.error(f'发送群聊消息失败:{response.json()}')
raise Exception(response.read().decode())
async def send_channle_group_text_msg(self, channel_id: str, content: str, msg_id: str):
@@ -244,7 +238,7 @@ class QQOfficialClient:
if response.status_code == 200:
return True
else:
await self.logger.error(f'Failed to send channel group message: {response.json()}')
await self.logger.error(f'发送频道群聊消息失败: {response.json()}')
raise Exception(response)
async def send_channle_private_text_msg(self, guild_id: str, content: str, msg_id: str):
@@ -267,224 +261,9 @@ class QQOfficialClient:
if response.status_code == 200:
return True
else:
await self.logger.error(f'Failed to send channel private message: {response.json()}')
await self.logger.error(f'发送频道私聊消息失败: {response.json()}')
raise Exception(response)
# ---- 富媒体消息 ----
# 媒体文件类型
MEDIA_TYPE_IMAGE = 1
MEDIA_TYPE_VIDEO = 2
MEDIA_TYPE_VOICE = 3
MEDIA_TYPE_FILE = 4
async def upload_media(
self,
target_type: str,
target_id: str,
file_type: int,
file_url: str = None,
file_data: str = None,
file_name: str = None,
) -> str:
"""上传媒体文件,返回 file_info。
Args:
target_type: 'c2c' | 'group'
target_id: 用户 openid 或群 openid
file_type: 1=图片, 2=视频, 3=语音, 4=文件
file_url: 在线 URL与 file_data 二选一)
file_data: base64 编码的文件数据或 data URL与 file_url 二选一)
file_name: 文件名file_type=4 时必填)
"""
if not await self.check_access_token():
await self.get_access_token()
if target_type == 'c2c':
url = f'{self.base_url}/v2/users/{target_id}/files'
elif target_type == 'group':
url = f'{self.base_url}/v2/groups/{target_id}/files'
else:
raise ValueError(f'Unsupported target_type: {target_type}')
body = {
'file_type': file_type,
'srv_send_msg': False,
}
if file_url:
body['url'] = file_url
elif file_data:
# 处理 data URL 格式: data:image/png;base64,xxxxx
if file_data.startswith('data:'):
match = re.match(r'^data:[^;]+;base64,(.+)$', file_data, re.DOTALL)
if match:
body['file_data'] = match.group(1)
else:
body['file_data'] = file_data
else:
body['file_data'] = file_data
else:
raise ValueError('file_url or file_data is required')
if file_type == self.MEDIA_TYPE_FILE and file_name:
body['file_name'] = file_name
async with httpx.AsyncClient(timeout=120) as client:
headers = {
'Authorization': f'QQBot {self.access_token}',
'Content-Type': 'application/json',
}
response = await client.post(url, headers=headers, json=body)
if response.status_code == 200:
data = response.json()
file_info = data.get('file_info', '')
preview = file_info[:80] + '...' if len(file_info) > 80 else file_info
await self.logger.info(f'Upload media success, file_info={preview}')
return file_info
else:
raise Exception(f'Failed to upload media: HTTP {response.status_code} {response.text}')
async def _send_media_msg(
self,
target_type: str,
target_id: str,
file_info: str,
msg_id: str = None,
content: str = None,
):
"""发送富媒体消息msg_type=7"""
if not await self.check_access_token():
await self.get_access_token()
if target_type == 'c2c':
url = f'{self.base_url}/v2/users/{target_id}/messages'
elif target_type == 'group':
url = f'{self.base_url}/v2/groups/{target_id}/messages'
else:
raise ValueError(f'Unsupported target_type: {target_type}')
self._msg_seq_counter += 1
msg_seq = self._msg_seq_counter
body = {
'msg_type': 7,
'media': {'file_info': file_info},
'msg_seq': msg_seq,
}
if content:
body['content'] = content
if msg_id:
body['msg_id'] = msg_id
async with httpx.AsyncClient(timeout=120) as client:
headers = {
'Authorization': f'QQBot {self.access_token}',
'Content-Type': 'application/json',
}
await self.logger.info(f'Sending rich media: {json.dumps(body, ensure_ascii=False)[:200]}')
response = await client.post(url, headers=headers, json=body)
if response.status_code != 200:
raise Exception(f'Failed to send rich media message: HTTP {response.status_code} {response.text}')
async def send_image_msg(
self,
target_type: str,
target_id: str,
file_url: str = None,
file_data: str = None,
msg_id: str = None,
content: str = None,
):
"""发送图片消息"""
file_info = await self.upload_media(
target_type,
target_id,
self.MEDIA_TYPE_IMAGE,
file_url=file_url,
file_data=file_data,
)
await self._send_media_msg(target_type, target_id, file_info, msg_id, content)
async def send_voice_msg(
self,
target_type: str,
target_id: str,
file_url: str = None,
file_data: str = None,
msg_id: str = None,
):
"""发送语音消息"""
file_info = await self.upload_media(
target_type,
target_id,
self.MEDIA_TYPE_VOICE,
file_url=file_url,
file_data=file_data,
)
await self._send_media_msg(target_type, target_id, file_info, msg_id)
async def send_file_msg(
self,
target_type: str,
target_id: str,
file_url: str = None,
file_data: str = None,
file_name: str = None,
msg_id: str = None,
):
"""发送文件消息(含视频)"""
file_info = await self.upload_media(
target_type,
target_id,
self.MEDIA_TYPE_FILE,
file_url=file_url,
file_data=file_data,
file_name=file_name,
)
await self._send_media_msg(target_type, target_id, file_info, msg_id)
async def send_stream_msg(
self,
user_openid: str,
content: str,
event_id: str,
msg_id: str,
msg_seq: int = 1,
index: int = 0,
stream_msg_id: str = None,
input_state: int = 1,
):
"""发送流式消息C2C 私聊)。
Args:
input_state: 1=生成中, 10=生成结束
"""
if not await self.check_access_token():
await self.get_access_token()
url = f'{self.base_url}/v2/users/{user_openid}/stream_messages'
body = {
'input_mode': 'replace',
'input_state': input_state,
'content_type': 'markdown',
'content_raw': content,
'event_id': event_id,
'msg_id': msg_id,
'msg_seq': msg_seq,
'index': index,
}
if stream_msg_id:
body['stream_msg_id'] = stream_msg_id
async with httpx.AsyncClient(timeout=120) as client:
headers = {
'Authorization': f'QQBot {self.access_token}',
'Content-Type': 'application/json',
}
response = await client.post(url, headers=headers, json=body)
if response.status_code != 200:
raise Exception(f'Failed to send stream message: HTTP {response.status_code} {response.text}')
return response.json()
async def is_token_expired(self):
"""检查token是否过期"""
if self.access_token_expiry_time is None:
@@ -513,325 +292,3 @@ class QQOfficialClient:
'signature': signature,
}
return response
# ---- WebSocket Gateway ----
# Reference: https://bot.q.qq.com/wiki/develop/api-v2/dev-prepare/interface-framework/event-emit.html
INTENT_GUILDS = 1 << 0
INTENT_GUILD_MEMBERS = 1 << 1
INTENT_PUBLIC_GUILD_MESSAGES = 1 << 30
INTENT_DIRECT_MESSAGE = 1 << 12
INTENT_GROUP_AND_C2C = 1 << 25
INTENT_INTERACTION = 1 << 26
FULL_INTENTS = (
INTENT_GUILDS
| INTENT_GUILD_MEMBERS
| INTENT_PUBLIC_GUILD_MESSAGES
| INTENT_DIRECT_MESSAGE
| INTENT_GROUP_AND_C2C
| INTENT_INTERACTION
)
async def get_gateway_url(self) -> str:
"""获取 WebSocket 网关地址"""
if not await self.check_access_token():
await self.get_access_token()
url = f'{self.base_url}/gateway'
async with httpx.AsyncClient() as client:
headers = {
'Authorization': f'QQBot {self.access_token}',
}
response = await client.get(url, headers=headers)
if response.status_code == 200:
data = response.json()
ws_url = data.get('url', '')
if not ws_url:
raise Exception('Gateway URL is empty')
return ws_url
else:
raise Exception(f'Failed to get Gateway URL: HTTP {response.status_code} {response.text}')
async def _background_token_refresh(self):
"""在 token 到期前主动刷新"""
try:
while True:
if self.access_token_expiry_time:
remain = self.access_token_expiry_time - time.time()
if remain > 120:
await asyncio.sleep(remain - 60)
continue
self.access_token = ''
self.access_token_expiry_time = None
if await self.check_access_token():
await asyncio.sleep(60)
else:
await self.get_access_token()
await asyncio.sleep(60)
except asyncio.CancelledError:
pass
async def connect_gateway(
self,
on_event: Callable[[str, dict], Any],
on_ready: Optional[Callable[[], Any]] = None,
on_error: Optional[Callable[[Exception], Any]] = None,
):
"""WebSocket 网关连接,含重连逻辑。持续重连直到达到最大次数或被取消。
Args:
on_event: 收到 op=0 Dispatch 事件时的回调,参数为 (event_type, event_data)
on_ready: 连接就绪 (收到 READY) 时的回调
on_error: 发生错误时的回调
"""
import websockets
session_id = ''
last_seq = 0
reconnect_attempts = 0
max_reconnect_attempts = 100
backoff_delays = [1, 2, 5, 10, 30, 60]
rate_limit_delay = 60
# Cancel previous token refresh task if any
if self._token_refresh_task and not self._token_refresh_task.done():
self._token_refresh_task.cancel()
try:
await self._token_refresh_task
except asyncio.CancelledError:
pass
self._token_refresh_task = None
while reconnect_attempts <= max_reconnect_attempts:
heartbeat_interval = 45000
should_refresh_token = False
ws = None
heartbeat_task = None
# Refresh token if needed
if should_refresh_token:
self.access_token = ''
self.access_token_expiry_time = None
try:
ws_url = await self.get_gateway_url()
await self.logger.info(f'Gateway URL obtained: {ws_url[:60]}...')
except Exception as e:
error_msg = str(e)
await self.logger.error(f'Failed to get gateway URL: {e}')
reconnect_attempts += 1
if '100017' in error_msg or '频率' in error_msg or 'Too many' in error_msg:
delay = rate_limit_delay
else:
delay = backoff_delays[min(reconnect_attempts - 1, len(backoff_delays) - 1)]
await self.logger.info(f'Reconnecting in {delay}s (attempt {reconnect_attempts})')
await asyncio.sleep(delay)
continue
try:
await self.logger.info('Connecting to WebSocket gateway...')
ws = await websockets.connect(ws_url)
await self.logger.info('WebSocket connected')
except Exception as e:
await self.logger.error(f'WebSocket connection failed: {e}')
reconnect_attempts += 1
delay = backoff_delays[min(reconnect_attempts - 1, len(backoff_delays) - 1)]
await self.logger.info(f'Reconnecting in {delay}s (attempt {reconnect_attempts})')
await asyncio.sleep(delay)
continue
try:
async for raw_msg in ws:
try:
payload = json.loads(raw_msg)
except json.JSONDecodeError:
await self.logger.error(f'Failed to parse message: {raw_msg}')
continue
op = payload.get('op')
d = payload.get('d', {})
s = payload.get('s')
t = payload.get('t')
if not isinstance(d, dict):
d = {}
if op == 10: # Hello
heartbeat_interval = d.get('heartbeat_interval', 45000)
await self.logger.info(f'Received Hello, heartbeat_interval={heartbeat_interval}ms')
# Send Identify or Resume
if session_id and last_seq > 0:
resume_payload = {
'op': 6,
'd': {
'token': f'QQBot {self.access_token}',
'session_id': session_id,
'seq': last_seq,
},
}
await ws.send(json.dumps(resume_payload))
await self.logger.info(f'Sent Resume, session_id={session_id}, seq={last_seq}')
else:
identify_payload = {
'op': 2,
'd': {
'token': f'QQBot {self.access_token}',
'intents': self.FULL_INTENTS,
'shard': [0, 1],
},
}
await ws.send(json.dumps(identify_payload))
await self.logger.info(f'Sent Identify, intents={self.FULL_INTENTS}')
# Start heartbeat
async def _heartbeat_loop(conn, interval_ms):
interval_sec = interval_ms / 1000.0
try:
while True:
await asyncio.sleep(interval_sec)
try:
hb_payload = {'op': 1, 'd': last_seq}
await conn.send(json.dumps(hb_payload))
except Exception:
break
except asyncio.CancelledError:
pass
heartbeat_task = asyncio.create_task(_heartbeat_loop(ws, heartbeat_interval))
elif op == 0: # Dispatch
if s is not None:
last_seq = s
if t == 'READY':
session_id = d.get('session_id', '')
reconnect_attempts = 0
await self.logger.info(f'READY, session_id={session_id}')
if on_ready:
try:
result = on_ready()
if asyncio.iscoroutine(result):
await result
except Exception:
pass
# Track token refresh task to avoid leaks
if self._token_refresh_task and not self._token_refresh_task.done():
self._token_refresh_task.cancel()
try:
await self._token_refresh_task
except asyncio.CancelledError:
pass
self._token_refresh_task = asyncio.create_task(self._background_token_refresh())
elif t == 'RESUMED':
reconnect_attempts = 0
await self.logger.info('RESUMED')
else:
await self.logger.debug(f'Received event: {t}, seq={s}')
if on_event:
try:
result = on_event(t, d)
if asyncio.iscoroutine(result):
await result
except Exception:
await self.logger.error(f'Error handling event {t}: {traceback.format_exc()}')
elif op == 11: # Heartbeat ACK
pass
elif op == 7: # Reconnect
await self.logger.info('Received Reconnect directive')
break
elif op == 9: # Invalid Session
can_resume = d.get('can_resume', False)
await self.logger.warning(f'Invalid Session, can_resume={can_resume}')
if not can_resume:
session_id = ''
last_seq = 0
should_refresh_token = True
break
# Connection closed normally (end of async for)
try:
close_code = ws.close_code
close_reason = ws.close_reason or ''
except Exception:
close_code = None
close_reason = ''
await self.logger.info(f'Connection closed, code={close_code}, reason={close_reason}')
if close_code == 4004:
should_refresh_token = True
elif close_code in (4006, 4007, 4009):
session_id = ''
last_seq = 0
should_refresh_token = True
elif close_code == 4008:
reconnect_attempts += 1
delay = rate_limit_delay
await self.logger.info(
f'Rate limited, waiting {delay}s before reconnect (attempt {reconnect_attempts})'
)
await asyncio.sleep(delay)
continue
elif close_code in (4914, 4915):
err = Exception(f'Bot disconnected/banned (close_code={close_code})')
if on_error:
await self._safe_callback(on_error, err)
return
elif close_code in (4900, 4901, 4902, 4903, 4904, 4905, 4906, 4907, 4908, 4909, 4910, 4911, 4912, 4913):
session_id = ''
last_seq = 0
if close_code == 1000:
return
except asyncio.CancelledError:
raise
except Exception:
await self.logger.error(f'Unexpected error in WebSocket loop: {traceback.format_exc()}')
finally:
if heartbeat_task:
heartbeat_task.cancel()
try:
await heartbeat_task
except asyncio.CancelledError:
pass
if ws:
try:
await ws.close()
except Exception:
pass
# If we reach here, we need to reconnect
reconnect_attempts += 1
if reconnect_attempts > max_reconnect_attempts:
await self.logger.error(f'Max reconnect attempts ({max_reconnect_attempts}) reached, stopping')
if on_error:
await self._safe_callback(on_error, Exception('Max reconnect attempts reached'))
return
delay = backoff_delays[min(reconnect_attempts - 1, len(backoff_delays) - 1)]
await self.logger.info(f'Reconnecting in {delay}s (attempt {reconnect_attempts})')
await asyncio.sleep(delay)
async def _safe_callback(self, callback, *args):
"""Safely invoke a callback, handling both sync and async functions."""
try:
result = callback(*args)
if asyncio.iscoroutine(result):
await result
except Exception:
pass
async def connect_gateway_loop(
self,
on_event: Callable[[str, dict], Any],
on_ready: Optional[Callable[[], Any]] = None,
on_error: Optional[Callable[[Exception], Any]] = None,
):
"""持续重连的网关循环。"""
await self.connect_gateway(on_event, on_ready, on_error)

View File

@@ -6,8 +6,7 @@ import traceback
import uuid
import xml.etree.ElementTree as ET
from dataclasses import dataclass, field
import re
from typing import Any, Callable, Optional, Tuple
from typing import Any, Callable, Optional
from urllib.parse import unquote
import httpx
@@ -64,25 +63,16 @@ class StreamSession:
# 缓存最近一次片段,处理重试或超时兜底
last_chunk: Optional[StreamChunk] = None
# 反馈 ID用于接收用户点赞/点踩反馈
feedback_id: Optional[str] = None
class StreamSessionManager:
"""管理 stream 会话的生命周期,并负责队列的生产消费。"""
# Sessions with registered feedback_ids use a longer TTL to survive the
# full like → cancel → dislike feedback flow. Must align with the adapter's
# _stream_to_monitoring_msg TTL (wecombot.py).
_FEEDBACK_SESSION_TTL = 600 # 10 minutes
def __init__(self, logger: EventLogger, ttl: int = 60) -> None:
self.logger = logger
self.ttl = ttl # 超时时间(秒),超过该时间未被访问的会话会被清理由 cleanup
self._sessions: dict[str, StreamSession] = {} # stream_id -> StreamSession 映射
self._msg_index: dict[str, str] = {} # msgid -> stream_id 映射,便于流水线根据消息 ID 找到会话
self._feedback_index: dict[str, str] = {} # feedback_id -> stream_id 映射
def get_stream_id_by_msg(self, msg_id: str) -> Optional[str]:
if not msg_id:
@@ -92,32 +82,6 @@ class StreamSessionManager:
def get_session(self, stream_id: str) -> Optional[StreamSession]:
return self._sessions.get(stream_id)
def get_session_by_feedback_id(self, feedback_id: str) -> Optional[StreamSession]:
"""根据 feedback_id 查找会话。
Args:
feedback_id: 企业微信反馈事件中的反馈 ID。
Returns:
Optional[StreamSession]: 找到的会话实例,未找到返回 None。
"""
if not feedback_id:
return None
stream_id = self._feedback_index.get(feedback_id)
if stream_id:
return self._sessions.get(stream_id)
return None
def register_feedback_id(self, stream_id: str, feedback_id: str) -> None:
"""注册 feedback_id 与 stream_id 的映射。
Args:
stream_id: 企业微信流式会话 ID。
feedback_id: 反馈 ID。
"""
if feedback_id and stream_id:
self._feedback_index[feedback_id] = stream_id
def create_or_get(self, msg_json: dict[str, Any]) -> tuple[StreamSession, bool]:
"""根据企业微信回调创建或获取会话。
@@ -219,17 +183,11 @@ class StreamSessionManager:
session.last_access = time.time()
def cleanup(self) -> None:
"""定期清理过期会话,防止队列与映射无上限累积。
已注册 feedback_id 的会话使用更长的 TTL确保用户在点赞/取消/点踩流程中
不会因为 session 被提前清除而丢失上下文信息。
"""
"""定期清理过期会话,防止队列与映射无上限累积。"""
now = time.time()
expired: list[str] = []
for stream_id, session in self._sessions.items():
# Sessions with registered feedback_ids use a longer TTL
effective_ttl = self._FEEDBACK_SESSION_TTL if session.feedback_id else self.ttl
if now - session.last_access > effective_ttl:
if now - session.last_access > self.ttl:
expired.append(stream_id)
for stream_id in expired:
@@ -239,142 +197,52 @@ class StreamSessionManager:
msg_id = session.msg_id
if msg_id and self._msg_index.get(msg_id) == stream_id:
self._msg_index.pop(msg_id, None)
# Clean up feedback index for expired sessions
if session.feedback_id:
self._feedback_index.pop(session.feedback_id, None)
def _decrypt_file(encrypted_data: bytes, aes_key_str: str) -> bytes:
"""Decrypt AES-256-CBC encrypted file data.
Aligned with the official WeCom AI Bot Python SDK (crypto_utils.py).
Args:
encrypted_data: The raw encrypted bytes.
aes_key_str: Base64-encoded AES key (may lack padding).
Returns:
Decrypted bytes with PKCS#7 padding removed.
"""
if not encrypted_data:
raise ValueError('encrypted_data is empty')
if not aes_key_str:
raise ValueError('aes_key is empty')
# Python's base64.b64decode requires proper padding (length % 4 == 0).
# Node.js Buffer.from tolerates missing '=', so we must pad manually.
remainder = len(aes_key_str) % 4
if remainder != 0:
aes_key_str = aes_key_str + '=' * (4 - remainder)
key = base64.b64decode(aes_key_str)
iv = key[:16]
cipher = AES.new(key, AES.MODE_CBC, iv)
# Ensure encrypted data is aligned to AES block size (16 bytes).
# Node.js setAutoPadding(false) silently handles unaligned data,
# but PyCryptodome will raise an error.
block_size = 16
data_remainder = len(encrypted_data) % block_size
if data_remainder != 0:
encrypted_data = encrypted_data + b'\x00' * (block_size - data_remainder)
decrypted = cipher.decrypt(encrypted_data)
# Remove PKCS#7 padding with validation
if len(decrypted) == 0:
raise ValueError('Decrypted data is empty')
pad_len = decrypted[-1]
if pad_len < 1 or pad_len > 32 or pad_len > len(decrypted):
raise ValueError(f'Invalid PKCS#7 padding value: {pad_len}')
# Verify all padding bytes are consistent
for i in range(len(decrypted) - pad_len, len(decrypted)):
if decrypted[i] != pad_len:
raise ValueError('Invalid PKCS#7 padding: padding bytes mismatch')
return decrypted[: len(decrypted) - pad_len]
def _extract_filename(content_disposition: str) -> Optional[str]:
"""Extract filename from a Content-Disposition header value."""
if not content_disposition:
return None
# RFC 5987: filename*=UTF-8''xxx
utf8_match = re.search(r"filename\*=UTF-8''([^;\s]+)", content_disposition, re.IGNORECASE)
if utf8_match:
return unquote(utf8_match.group(1))
# Standard: filename="xxx" or filename=xxx
match = re.search(r'filename="?([^";\s]+)"?', content_disposition, re.IGNORECASE)
if match:
return unquote(match.group(1))
return None
def _bytes_to_data_uri(data: bytes) -> str:
"""Convert raw bytes to a data URI with auto-detected MIME type."""
if data.startswith(b'\xff\xd8'):
mime_type = 'image/jpeg'
elif data.startswith(b'\x89PNG'):
mime_type = 'image/png'
elif data.startswith((b'GIF87a', b'GIF89a')):
mime_type = 'image/gif'
elif data.startswith(b'BM'):
mime_type = 'image/bmp'
elif data.startswith(b'II*\x00') or data.startswith(b'MM\x00*'):
mime_type = 'image/tiff'
elif data[:4] == b'%PDF':
mime_type = 'application/pdf'
elif data[:4] == b'PK\x03\x04':
mime_type = 'application/zip'
else:
mime_type = 'application/octet-stream'
base64_str = base64.b64encode(data).decode('utf-8')
return f'data:{mime_type};base64,{base64_str}'
async def download_encrypted_file(
download_url: str, aes_key: str, logger: EventLogger
) -> Tuple[Optional[bytes], Optional[str]]:
"""Download an AES-encrypted file from WeChat Work and decrypt it.
async def download_encrypted_file(download_url: str, encoding_aes_key: str, logger: EventLogger) -> Optional[str]:
"""Download an AES-encrypted file from WeChat Work and return as data URI.
Args:
download_url: The encrypted file download URL.
aes_key: The AES key for decryption (base64-encoded, per-message aeskey
or platform EncodingAESKey).
encoding_aes_key: The AES key used for decryption (base64-encoded, without trailing '=').
logger: Logger instance.
Returns:
A tuple of (decrypted_bytes, filename) or (None, None) on failure.
A data URI string (e.g. 'data:image/jpeg;base64,...') or None on failure.
"""
if not download_url:
return None, None
if not aes_key:
await logger.error('download_encrypted_file: aes_key is empty, cannot decrypt')
return None, None
return None
async with httpx.AsyncClient() as client:
response = await client.get(download_url)
if response.status_code != 200:
await logger.error(f'failed to get file: {response.text}')
return None
encrypted_bytes = response.content
filename: Optional[str] = None
try:
async with httpx.AsyncClient(timeout=30.0) as client:
response = await client.get(download_url)
if response.status_code != 200:
await logger.error(f'Failed to download file (HTTP {response.status_code}): {response.text[:200]}')
return None, None
encrypted_bytes = response.content
filename = _extract_filename(response.headers.get('content-disposition', ''))
except Exception:
await logger.error(f'Failed to download file: {traceback.format_exc()}')
return None, None
aes_key = base64.b64decode(encoding_aes_key + '=')
iv = aes_key[:16]
try:
decrypted = _decrypt_file(encrypted_bytes, aes_key)
return decrypted, filename
except Exception:
await logger.error(f'Failed to decrypt file: {traceback.format_exc()}')
return None, None
cipher = AES.new(aes_key, AES.MODE_CBC, iv)
decrypted = cipher.decrypt(encrypted_bytes)
pad_len = decrypted[-1]
decrypted = decrypted[:-pad_len]
if decrypted.startswith(b'\xff\xd8'):
mime_type = 'image/jpeg'
elif decrypted.startswith(b'\x89PNG'):
mime_type = 'image/png'
elif decrypted.startswith((b'GIF87a', b'GIF89a')):
mime_type = 'image/gif'
elif decrypted.startswith(b'BM'):
mime_type = 'image/bmp'
elif decrypted.startswith(b'II*\x00') or decrypted.startswith(b'MM\x00*'):
mime_type = 'image/tiff'
else:
mime_type = 'application/octet-stream'
base64_str = base64.b64encode(decrypted).decode('utf-8')
return f'data:{mime_type};base64,{base64_str}'
async def parse_wecom_bot_message(
@@ -405,22 +273,10 @@ async def parse_wecom_bot_message(
max_inline_file_size = 5 * 1024 * 1024
async def _safe_download(url: str, per_msg_aeskey: str = '') -> Tuple[Optional[bytes], Optional[str]]:
"""Download and decrypt a file, preferring per-message aeskey over platform key."""
async def _safe_download(url: str):
if not url:
return None, None
key = per_msg_aeskey or encoding_aes_key
if not key:
await logger.warning('No AES key available for file decryption, skipping download')
return None, None
return await download_encrypted_file(url, key, logger)
async def _safe_download_as_data_uri(url: str, per_msg_aeskey: str = '') -> Optional[str]:
"""Download, decrypt, and convert to data URI for backward compatibility."""
data, _filename = await _safe_download(url, per_msg_aeskey)
if data:
return _bytes_to_data_uri(data)
return None
return None
return await download_encrypted_file(url, encoding_aes_key, logger)
if msg_type == 'text':
message_data['content'] = msg_json.get('text', {}).get('content')
@@ -429,17 +285,14 @@ async def parse_wecom_bot_message(
'content', ''
)
elif msg_type == 'image':
image_info = msg_json.get('image', {})
picurl = image_info.get('url', '')
per_msg_aeskey = image_info.get('aeskey', '')
base64_data = await _safe_download_as_data_uri(picurl, per_msg_aeskey)
picurl = msg_json.get('image', {}).get('url', '')
base64_data = await _safe_download(picurl)
if base64_data:
message_data['picurl'] = base64_data
message_data['images'] = [base64_data]
elif msg_type == 'voice':
voice_info = msg_json.get('voice', {}) or {}
download_url = voice_info.get('url')
per_msg_aeskey = voice_info.get('aeskey', '')
message_data['voice'] = {
'url': download_url,
'md5sum': voice_info.get('md5sum') or voice_info.get('md5'),
@@ -448,14 +301,13 @@ async def parse_wecom_bot_message(
}
if voice_info.get('content'):
message_data['content'] = voice_info.get('content')
# if (message_data['voice'].get('filesize') or 0) <= max_inline_file_size:
# voice_base64 = await _safe_download_as_data_uri(download_url, per_msg_aeskey)
# if voice_base64:
# message_data['voice']['base64'] = voice_base64
if (message_data['voice'].get('filesize') or 0) <= max_inline_file_size:
voice_base64 = await _safe_download(download_url)
if voice_base64:
message_data['voice']['base64'] = voice_base64
elif msg_type == 'video':
video_info = msg_json.get('video', {}) or {}
download_url = video_info.get('url')
per_msg_aeskey = video_info.get('aeskey', '')
video_data = {
'url': download_url,
'filesize': video_info.get('filesize') or video_info.get('size'),
@@ -463,17 +315,14 @@ async def parse_wecom_bot_message(
'md5sum': video_info.get('md5sum') or video_info.get('md5'),
'filename': video_info.get('filename') or video_info.get('name'),
}
# if (video_data.get('filesize') or 0) <= max_inline_file_size:
# video_base64 = await _safe_download_as_data_uri(download_url, per_msg_aeskey)
# if video_base64:
# video_data['base64'] = video_base64
# 应为需要解密但是目前暂时不能下载到内部进行解密所以先将下载链接拼接aeskey返回给用户由插件去处理该链接的下载和解密逻辑
video_data['download_url'] = download_url + f'?aeskey={per_msg_aeskey}'
if (video_data.get('filesize') or 0) <= max_inline_file_size:
video_base64 = await _safe_download(download_url)
if video_base64:
video_data['base64'] = video_base64
message_data['video'] = video_data
elif msg_type == 'file':
file_info = msg_json.get('file', {}) or {}
download_url = file_info.get('url') or file_info.get('fileurl')
per_msg_aeskey = file_info.get('aeskey', '')
file_data = {
'filename': file_info.get('filename') or file_info.get('name'),
'filesize': file_info.get('filesize') or file_info.get('size'),
@@ -482,15 +331,10 @@ async def parse_wecom_bot_message(
'download_url': download_url,
'extra': file_info,
}
# if (file_data.get('filesize') or 0) <= max_inline_file_size:
# file_bytes, dl_filename = await _safe_download(download_url, per_msg_aeskey)
# if file_bytes:
# file_data['base64'] = _bytes_to_data_uri(file_bytes)
# if dl_filename and not file_data.get('filename'):
# file_data['filename'] = dl_filename
# 应为需要解密但是目前暂时不能下载到内部进行解密所以先将下载链接拼接aeskey返回给用户由插件去处理该链接的下载和解密逻辑
file_data['download_url'] = download_url + f'?aeskey={per_msg_aeskey}'
if (file_data.get('filesize') or 0) <= max_inline_file_size:
file_base64 = await _safe_download(download_url)
if file_base64:
file_data['base64'] = file_base64
message_data['file'] = file_data
elif msg_type == 'link':
message_data['link'] = msg_json.get('link', {})
@@ -511,16 +355,13 @@ async def parse_wecom_bot_message(
if item_type == 'text':
texts.append(item.get('text', {}).get('content', ''))
elif item_type == 'image':
img_info = item.get('image', {})
img_url = img_info.get('url')
img_aeskey = img_info.get('aeskey', '')
base64_data = await _safe_download_as_data_uri(img_url, img_aeskey)
img_url = item.get('image', {}).get('url')
base64_data = await _safe_download(img_url)
if base64_data:
images.append(base64_data)
elif item_type == 'file':
file_info = item.get('file', {}) or {}
download_url = file_info.get('url') or file_info.get('fileurl')
item_aeskey = file_info.get('aeskey', '')
file_data = {
'filename': file_info.get('filename') or file_info.get('name'),
'filesize': file_info.get('filesize') or file_info.get('size'),
@@ -530,16 +371,13 @@ async def parse_wecom_bot_message(
'extra': file_info,
}
if (file_data.get('filesize') or 0) <= max_inline_file_size:
file_bytes, dl_filename = await _safe_download(download_url, item_aeskey)
if file_bytes:
file_data['base64'] = _bytes_to_data_uri(file_bytes)
if dl_filename and not file_data.get('filename'):
file_data['filename'] = dl_filename
file_base64 = await _safe_download(download_url)
if file_base64:
file_data['base64'] = file_base64
files.append(file_data)
elif item_type == 'voice':
voice_info = item.get('voice', {}) or {}
download_url = voice_info.get('url')
item_aeskey = voice_info.get('aeskey', '')
voice_data = {
'url': download_url,
'md5sum': voice_info.get('md5sum') or voice_info.get('md5'),
@@ -549,14 +387,13 @@ async def parse_wecom_bot_message(
if voice_info.get('content'):
texts.append(voice_info.get('content'))
if (voice_data.get('filesize') or 0) <= max_inline_file_size:
voice_base64 = await _safe_download_as_data_uri(download_url, item_aeskey)
voice_base64 = await _safe_download(download_url)
if voice_base64:
voice_data['base64'] = voice_base64
voices.append(voice_data)
elif item_type == 'video':
video_info = item.get('video', {}) or {}
download_url = video_info.get('url')
item_aeskey = video_info.get('aeskey', '')
video_data = {
'url': download_url,
'filesize': video_info.get('filesize') or video_info.get('size'),
@@ -565,7 +402,7 @@ async def parse_wecom_bot_message(
'filename': video_info.get('filename') or video_info.get('name'),
}
if (video_data.get('filesize') or 0) <= max_inline_file_size:
video_base64 = await _safe_download_as_data_uri(download_url, item_aeskey)
video_base64 = await _safe_download(download_url)
if video_base64:
video_data['base64'] = video_base64
videos.append(video_data)
@@ -606,120 +443,6 @@ async def parse_wecom_bot_message(
if msg_json.get('aibotid'):
message_data['aibotid'] = msg_json.get('aibotid', '')
# Handle quote (referenced message) - important for group chat file references
quote_info = msg_json.get('quote')
if quote_info:
quote_data: dict[str, Any] = {}
quote_type = quote_info.get('msgtype', '')
quote_data['msgtype'] = quote_type
if quote_type == 'text':
quote_data['content'] = quote_info.get('text', {}).get('content', '')
elif quote_type == 'image':
img_info = quote_info.get('image', {})
img_url = img_info.get('url', '')
img_aeskey = img_info.get('aeskey', '')
base64_data = await _safe_download_as_data_uri(img_url, img_aeskey)
if base64_data:
quote_data['picurl'] = base64_data
quote_data['images'] = [base64_data]
elif quote_type == 'file':
file_info = quote_info.get('file', {}) or {}
download_url = file_info.get('url') or file_info.get('fileurl')
item_aeskey = file_info.get('aeskey', '')
file_data = {
'filename': file_info.get('filename') or file_info.get('name'),
'filesize': file_info.get('filesize') or file_info.get('size'),
'md5sum': file_info.get('md5sum') or file_info.get('md5'),
'sdkfileid': file_info.get('sdkfileid') or file_info.get('fileid'),
'download_url': download_url,
'extra': file_info,
}
# Same as private chat: append aeskey to download_url for plugin processing
if download_url and item_aeskey:
file_data['download_url'] = download_url + f'?aeskey={item_aeskey}'
quote_data['file'] = file_data
elif quote_type == 'voice':
voice_info = quote_info.get('voice', {}) or {}
download_url = voice_info.get('url')
item_aeskey = voice_info.get('aeskey', '')
voice_data = {
'url': download_url,
'md5sum': voice_info.get('md5sum') or voice_info.get('md5'),
'filesize': voice_info.get('filesize') or voice_info.get('size'),
'sdkfileid': voice_info.get('sdkfileid') or voice_info.get('fileid'),
}
if voice_info.get('content'):
quote_data['content'] = voice_info.get('content')
# Same as private chat: append aeskey to url for plugin processing
if download_url and item_aeskey:
voice_data['url'] = download_url + f'?aeskey={item_aeskey}'
quote_data['voice'] = voice_data
elif quote_type == 'video':
video_info = quote_info.get('video', {}) or {}
download_url = video_info.get('url')
item_aeskey = video_info.get('aeskey', '')
video_data = {
'url': download_url,
'filesize': video_info.get('filesize') or video_info.get('size'),
'sdkfileid': video_info.get('sdkfileid') or video_info.get('fileid'),
'md5sum': video_info.get('md5sum') or video_info.get('md5'),
'filename': video_info.get('filename') or video_info.get('name'),
}
# Same as private chat: append aeskey to download_url for plugin processing
if download_url and item_aeskey:
video_data['download_url'] = download_url + f'?aeskey={item_aeskey}'
quote_data['video'] = video_data
elif quote_type == 'link':
quote_data['link'] = quote_info.get('link', {})
link = quote_data['link']
title = link.get('title', '')
desc = link.get('description') or link.get('digest', '')
quote_data['content'] = '\n'.join(filter(None, [title, desc]))
elif quote_type == 'mixed':
# Handle mixed type in quote (text + images + files etc.)
items = quote_info.get('mixed', {}).get('msg_item', [])
texts = []
images = []
files = []
for item in items:
item_type = item.get('msgtype')
if item_type == 'text':
texts.append(item.get('text', {}).get('content', ''))
elif item_type == 'image':
img_info = item.get('image', {})
img_url = img_info.get('url')
img_aeskey = img_info.get('aeskey', '')
base64_data = await _safe_download_as_data_uri(img_url, img_aeskey)
if base64_data:
images.append(base64_data)
elif item_type == 'file':
file_info = item.get('file', {}) or {}
download_url = file_info.get('url') or file_info.get('fileurl')
item_aeskey = file_info.get('aeskey', '')
file_data = {
'filename': file_info.get('filename') or file_info.get('name'),
'filesize': file_info.get('filesize') or file_info.get('size'),
'md5sum': file_info.get('md5sum') or file_info.get('md5'),
'sdkfileid': file_info.get('sdkfileid') or file_info.get('fileid'),
'download_url': download_url,
'extra': file_info,
}
# Same as private chat: append aeskey to download_url for plugin processing
if download_url and item_aeskey:
file_data['download_url'] = download_url + f'?aeskey={item_aeskey}'
files.append(file_data)
if texts:
quote_data['content'] = ' '.join(texts)
if images:
quote_data['images'] = images
quote_data['picurl'] = images[0]
if files:
quote_data['files'] = files
quote_data['file'] = files[0]
message_data['quote'] = quote_data
return message_data
@@ -760,27 +483,14 @@ class WecomBotClient:
self.stream_sessions = StreamSessionManager(logger=logger)
self.stream_poll_timeout = 0.5
self._feedback_callback: Optional[Callable] = None
def set_feedback_callback(self, callback: Callable) -> None:
"""设置反馈回调函数。
Args:
callback: 反馈回调函数,签名: async def callback(feedback_id, feedback_type, feedback_content, inaccurate_reasons, session)
"""
self._feedback_callback = callback
@staticmethod
def _build_stream_payload(
stream_id: str, content: str, finish: bool, feedback_id: Optional[str] = None
) -> dict[str, Any]:
def _build_stream_payload(stream_id: str, content: str, finish: bool) -> dict[str, Any]:
"""按照企业微信协议拼装返回报文。
Args:
stream_id: 企业微信会话 ID。
content: 推送的文本内容。
finish: 是否为最终片段。
feedback_id: 反馈 ID用于接收用户点赞/点踩反馈。
Returns:
dict[str, Any]: 可直接加密返回的 payload。
@@ -788,16 +498,13 @@ class WecomBotClient:
Example:
组装 `{'msgtype': 'stream', 'stream': {'id': 'sid', ...}}` 结构。
"""
stream_payload = {
'id': stream_id,
'finish': finish,
'content': content,
}
if feedback_id:
stream_payload['feedback'] = {'id': feedback_id}
return {
'msgtype': 'stream',
'stream': stream_payload,
'stream': {
'id': stream_id,
'finish': finish,
'content': content,
},
}
async def _encrypt_and_reply(self, payload: dict[str, Any], nonce: str) -> tuple[Response, int]:
@@ -853,14 +560,9 @@ class WecomBotClient:
"""
session, is_new = self.stream_sessions.create_or_get(msg_json)
feedback_id = str(uuid.uuid4())
session.feedback_id = feedback_id
self.stream_sessions.register_feedback_id(session.stream_id, feedback_id)
message_data = await self.get_message(msg_json)
if message_data:
message_data['stream_id'] = session.stream_id
message_data['feedback_id'] = feedback_id
try:
event = wecombotevent.WecomBotEvent(message_data)
except Exception:
@@ -869,7 +571,7 @@ class WecomBotClient:
if is_new:
asyncio.create_task(self._dispatch_event(event))
payload = self._build_stream_payload(session.stream_id, '', False, feedback_id)
payload = self._build_stream_payload(session.stream_id, '', False)
return await self._encrypt_and_reply(payload, nonce)
async def _handle_post_followup_response(self, msg_json: dict[str, Any], nonce: str) -> tuple[Response, int]:
@@ -994,81 +696,11 @@ class WecomBotClient:
msg_json = json.loads(decrypted_xml)
event = msg_json.get('event', {})
event_type = event.get('eventtype', '')
if event_type == 'feedback_event':
return await self._handle_feedback_event(msg_json, nonce)
if msg_json.get('msgtype') == 'stream':
return await self._handle_post_followup_response(msg_json, nonce)
return await self._handle_post_initial_response(msg_json, nonce)
async def _handle_feedback_event(self, msg_json: dict[str, Any], nonce: str) -> tuple[Response, int]:
"""处理企业微信用户反馈事件(点赞/点踩)。
Args:
msg_json: 解密后的企业微信反馈事件 JSON。
nonce: 企业微信回调参数 nonce。
Returns:
Tuple[Response, int]: Quart Response 及状态码。
Note:
企业微信协议要求:反馈事件目前仅支持回复空包。
"""
try:
feedback_event = msg_json.get('event', {}).get('feedback_event', {})
feedback_id = feedback_event.get('id', '')
feedback_type = feedback_event.get('type', 0)
feedback_content = feedback_event.get('content', '')
inaccurate_reasons = feedback_event.get('inaccurate_reason_list', [])
await self.logger.info(
f'收到用户反馈事件: feedback_id={feedback_id}, type={feedback_type}, '
f'content={feedback_content}, reasons={inaccurate_reasons}'
)
session = self.stream_sessions.get_session_by_feedback_id(feedback_id)
if session:
await self.logger.info(
f'反馈关联到会话: stream_id={session.stream_id}, msg_id={session.msg_id}, user_id={session.user_id}'
)
else:
await self.logger.warning(f'未找到 feedback_id={feedback_id} 对应的会话,仍将记录反馈')
# Dispatch feedback event regardless of session availability
for handler in self._message_handlers.get('feedback', []):
try:
await handler(
feedback_id=feedback_id,
feedback_type=feedback_type,
feedback_content=feedback_content,
inaccurate_reasons=inaccurate_reasons,
session=session,
)
except Exception:
await self.logger.error(traceback.format_exc())
if self._feedback_callback:
try:
await self._feedback_callback(
feedback_id=feedback_id,
feedback_type=feedback_type,
feedback_content=feedback_content,
inaccurate_reasons=inaccurate_reasons,
session=session,
)
except Exception:
await self.logger.error(traceback.format_exc())
except Exception:
await self.logger.error(traceback.format_exc())
return await self._encrypt_and_reply({}, nonce)
async def get_message(self, msg_json):
return await parse_wecom_bot_message(msg_json, self.EnCodingAESKey, self.logger)
@@ -1137,20 +769,8 @@ class WecomBotClient:
return decorator
def on_feedback(self):
def decorator(func: Callable):
if 'feedback' not in self._message_handlers:
self._message_handlers['feedback'] = []
self._message_handlers['feedback'].append(func)
return func
return decorator
async def download_url_to_base64(self, download_url, encoding_aes_key):
data, _filename = await download_encrypted_file(download_url, encoding_aes_key, self.logger)
if data:
return _bytes_to_data_uri(data)
return None
return await download_encrypted_file(download_url, encoding_aes_key, self.logger)
async def run_task(self, host: str, port: int, *args, **kwargs):
"""

View File

@@ -133,24 +133,3 @@ class WecomBotEvent(dict):
AI Bot ID
"""
return self.get('aibotid', '')
@property
def feedback_id(self) -> str:
"""
反馈 ID用于关联用户点赞/点踩反馈
"""
return self.get('feedback_id', '')
@property
def stream_id(self) -> str:
"""
流式消息 ID
"""
return self.get('stream_id', '')
@property
def quote(self):
"""
引用消息信息(群聊中用户引用其他消息时返回)
"""
return self.get('quote', {})

View File

@@ -20,7 +20,7 @@ from typing import Any, Callable, Optional
import aiohttp
from langbot.libs.wecom_ai_bot_api import wecombotevent
from langbot.libs.wecom_ai_bot_api.api import parse_wecom_bot_message, StreamSession
from langbot.libs.wecom_ai_bot_api.api import parse_wecom_bot_message
from langbot.pkg.platform.logger import EventLogger
DEFAULT_WS_URL = 'wss://openws.work.weixin.qq.com'
@@ -96,12 +96,6 @@ class WecomBotWsClient:
self._stream_ids: dict[str, str] = {} # msg_id -> req_id|stream_id
# Dedup: skip sending when content hasn't changed
self._stream_last_content: dict[str, str] = {} # msg_id -> last content sent
# Stream session info for feedback tracking
self._stream_sessions: dict[str, dict] = {} # msg_id -> session info
# Feedback tracking: feedback_id -> session info
self._feedback_sessions: dict[str, dict] = {} # feedback_id -> {msg_id, user_id, chat_id, stream_id, req_id}
# msg_id -> feedback_id (for associating feedback with message)
self._msg_feedback_ids: dict[str, str] = {} # msg_id -> feedback_id
# ── Public API ──────────────────────────────────────────────────
@@ -170,27 +164,12 @@ class WecomBotWsClient:
return decorator
def on_feedback(self) -> Callable:
"""Decorator to register a feedback event handler.
Same interface as WecomBotClient.on_feedback for compatibility.
"""
def decorator(func: Callable):
if 'feedback' not in self._message_handlers:
self._message_handlers['feedback'] = []
self._message_handlers['feedback'].append(func)
return func
return decorator
async def reply_stream(
self,
req_id: str,
stream_id: str,
content: str,
finish: bool = False,
feedback_id: str = '',
) -> Optional[dict]:
"""Send a streaming reply frame.
@@ -199,22 +178,17 @@ class WecomBotWsClient:
stream_id: The stream ID for this streaming session.
content: The content to send (supports Markdown).
finish: Whether this is the final chunk.
feedback_id: Optional feedback ID for receiving user feedback (like/dislike).
Returns:
The ACK frame dict, or None on failure.
"""
stream_payload = {
'id': stream_id,
'finish': finish,
'content': content,
}
if feedback_id:
stream_payload['feedback'] = {'id': feedback_id}
body = {
'msgtype': 'stream',
'stream': stream_payload,
'stream': {
'id': stream_id,
'finish': finish,
'content': content,
},
}
return await self._send_reply(req_id, body)
@@ -279,23 +253,11 @@ class WecomBotWsClient:
# Skip sending if content hasn't changed (e.g. during tool call argument streaming)
if not is_final and content == self._stream_last_content.get(msg_id):
return True
# Generate feedback_id for final chunk
feedback_id = ''
if is_final:
feedback_id = _generate_req_id('feedback')
self._msg_feedback_ids[msg_id] = feedback_id
# Store session info for feedback tracking
session_info = self._stream_sessions.get(msg_id)
if session_info:
self._feedback_sessions[feedback_id] = session_info
await self.reply_stream(req_id, stream_id, content, finish=is_final, feedback_id=feedback_id)
await self.reply_stream(req_id, stream_id, content, finish=is_final)
self._stream_last_content[msg_id] = content
if is_final:
self._stream_ids.pop(msg_id, None)
self._stream_last_content.pop(msg_id, None)
self._stream_sessions.pop(msg_id, None)
return True
except Exception:
await self.logger.error(f'Failed to push stream chunk: {traceback.format_exc()}')
@@ -483,15 +445,6 @@ class WecomBotWsClient:
msg_id = message_data.get('msgid', '')
if msg_id:
self._stream_ids[msg_id] = f'{req_id}|{stream_id}'
# Store session info for feedback tracking
self._stream_sessions[msg_id] = {
'req_id': req_id,
'stream_id': stream_id,
'msg_id': msg_id,
'user_id': message_data.get('userid', ''),
'chat_id': message_data.get('chatid', ''),
'chat_type': message_data.get('type', 'single'),
}
message_data['stream_id'] = stream_id
message_data['req_id'] = req_id
@@ -501,7 +454,7 @@ class WecomBotWsClient:
await self.logger.error(f'Error in message callback: {traceback.format_exc()}')
async def _handle_event_callback(self, frame: dict):
"""Handle an incoming event callback frame (enter_chat, template_card_event, feedback_event, disconnected_event)."""
"""Handle an incoming event callback frame (enter_chat, template_card_event, etc.)."""
try:
body = frame.get('body', {})
req_id = frame.get('headers', {}).get('req_id', '')
@@ -526,54 +479,14 @@ class WecomBotWsClient:
if body.get('chatid'):
message_data['chatid'] = body.get('chatid', '')
if event_type == 'feedback_event':
feedback_event = event_info.get('feedback_event', {})
feedback_id = feedback_event.get('id', '')
feedback_type = feedback_event.get('type', 0)
feedback_content = feedback_event.get('content', '')
inaccurate_reasons = feedback_event.get('inaccurate_reason_list', [])
await self.logger.info(
f'收到用户反馈事件: feedback_id={feedback_id}, type={feedback_type}, '
f'content={feedback_content}, reasons={inaccurate_reasons}'
)
# Look up session by feedback_id
session_info = self._feedback_sessions.get(feedback_id)
session = None
if session_info:
session = StreamSession(
stream_id=session_info.get('stream_id', ''),
msg_id=session_info.get('msg_id', ''),
chat_id=session_info.get('chat_id') or None,
user_id=session_info.get('user_id') or None,
feedback_id=feedback_id,
)
await self.logger.info(
f'反馈关联到会话: stream_id={session.stream_id}, msg_id={session.msg_id}, user_id={session.user_id}'
)
else:
await self.logger.warning(f'未找到 feedback_id={feedback_id} 对应的会话')
for handler in self._message_handlers.get('feedback', []):
try:
await handler(
feedback_id=feedback_id,
feedback_type=feedback_type,
feedback_content=feedback_content,
inaccurate_reasons=inaccurate_reasons,
session=session,
)
except Exception:
await self.logger.error(f'Error in feedback handler: {traceback.format_exc()}')
return
event = wecombotevent.WecomBotEvent(message_data)
# Dispatch to event-specific handlers
if event_type in self._message_handlers:
for handler in self._message_handlers[event_type]:
await handler(event)
# Also dispatch to generic 'event' handlers
if 'event' in self._message_handlers:
for handler in self._message_handlers['event']:
await handler(event)

View File

@@ -4,7 +4,6 @@ import base64
import binascii
import httpx
import traceback
from urllib.parse import quote
from quart import Quart
import xml.etree.ElementTree as ET
from typing import Callable, Dict, Any
@@ -68,31 +67,6 @@ class WecomClient:
await self.logger.error(f'获取accesstoken失败:{response.json()}')
raise Exception(f'未获取access token: {data}')
async def get_user_info(self, userid: str) -> dict:
"""
Get user information by user ID using the application secret.
Args:
userid: The user ID to look up.
Returns:
dict: User information including 'name' field.
"""
if not await self.check_access_token():
self.access_token = await self.get_access_token(self.secret)
url = self.base_url + '/user/get?access_token=' + self.access_token + '&userid=' + quote(userid)
async with httpx.AsyncClient() as client:
response = await client.get(url)
data = response.json()
if data.get('errcode') == 40014 or data.get('errcode') == 42001:
self.access_token = await self.get_access_token(self.secret)
return await self.get_user_info(userid)
if data.get('errcode', 0) != 0:
await self.logger.error(f'获取用户信息失败:{data}')
return {}
return data
async def get_users(self):
if not self.check_access_token_for_contacts():
self.access_token_for_contacts = await self.get_access_token(self.secret_for_contacts)

View File

@@ -1,37 +0,0 @@
"""Agent runner subsystem for LangBot."""
from __future__ import annotations
from .runner.descriptor import AgentRunnerDescriptor
from .runner.id import parse_runner_id, format_runner_id, RunnerIdParts, is_plugin_runner_id
from .runner.errors import (
AgentRunnerError,
RunnerNotFoundError,
RunnerNotAuthorizedError,
RunnerProtocolError,
RunnerExecutionError,
)
from .runner.registry import AgentRunnerRegistry
from .runner.context_builder import AgentRunContextBuilder
from .runner.resource_builder import AgentResourceBuilder
from .runner.result_normalizer import AgentResultNormalizer
from .runner.orchestrator import AgentRunOrchestrator
from .runner.config_migration import ConfigMigration
__all__ = [
'AgentRunnerDescriptor',
'parse_runner_id',
'format_runner_id',
'is_plugin_runner_id',
'RunnerIdParts',
'AgentRunnerError',
'RunnerNotFoundError',
'RunnerNotAuthorizedError',
'RunnerProtocolError',
'RunnerExecutionError',
'AgentRunnerRegistry',
'AgentRunContextBuilder',
'AgentResourceBuilder',
'AgentResultNormalizer',
'AgentRunOrchestrator',
'ConfigMigration',
]

View File

@@ -1,61 +0,0 @@
"""Agent runner modules."""
from __future__ import annotations
from .descriptor import AgentRunnerDescriptor
from .id import parse_runner_id, format_runner_id, RunnerIdParts
from .errors import (
AgentRunnerError,
RunnerNotFoundError,
RunnerNotAuthorizedError,
RunnerProtocolError,
RunnerExecutionError,
)
from .registry import AgentRunnerRegistry
from .context_builder import AgentRunContextBuilder
from .resource_builder import AgentResourceBuilder
from .result_normalizer import AgentResultNormalizer
from .orchestrator import AgentRunOrchestrator
from .config_migration import ConfigMigration
from .binding_resolver import AgentBindingResolver, AgentBindingResolutionError
from .session_registry import (
AgentRunSessionRegistry,
AgentRunSession,
RunAuthorizationSnapshot,
get_session_registry,
)
from .events import (
MESSAGE_RECEIVED,
MESSAGE_RECALLED,
GROUP_MEMBER_JOINED,
FRIEND_REQUEST_RECEIVED,
RESERVED_EVENT_TYPES,
)
__all__ = [
'AgentRunnerDescriptor',
'parse_runner_id',
'format_runner_id',
'RunnerIdParts',
'AgentRunnerError',
'RunnerNotFoundError',
'RunnerNotAuthorizedError',
'RunnerProtocolError',
'RunnerExecutionError',
'AgentRunnerRegistry',
'AgentRunContextBuilder',
'AgentResourceBuilder',
'AgentResultNormalizer',
'AgentRunOrchestrator',
'ConfigMigration',
'AgentBindingResolver',
'AgentBindingResolutionError',
'AgentRunSessionRegistry',
'AgentRunSession',
'RunAuthorizationSnapshot',
'get_session_registry',
'MESSAGE_RECEIVED',
'MESSAGE_RECALLED',
'GROUP_MEMBER_JOINED',
'FRIEND_REQUEST_RECEIVED',
'RESERVED_EVENT_TYPES',
]

View File

@@ -1,300 +0,0 @@
"""Artifact store for managing Host-owned artifacts."""
from __future__ import annotations
import json
import datetime
import typing
import uuid
import base64
import sqlalchemy
from sqlalchemy.ext.asyncio import AsyncEngine, AsyncSession
from sqlalchemy.orm import sessionmaker
from ...entity.persistence.artifact import AgentArtifact
from ...entity.persistence.bstorage import BinaryStorage
class ArtifactStore:
"""Store for AgentArtifact records.
Handles artifact metadata registration and content retrieval.
Actual blob storage is delegated to BinaryStorage or external storage.
All methods are async and use the provided database engine.
"""
engine: AsyncEngine
# Hard limits
MAX_INLINE_READ_BYTES = 1024 * 1024 # 1MB max for inline base64
MAX_RANGE_READ_BYTES = 10 * 1024 * 1024 # 10MB max for range reads
def __init__(self, engine: AsyncEngine):
self.engine = engine
self._session_factory = sessionmaker(
engine, class_=AsyncSession, expire_on_commit=False
)
async def register_artifact(
self,
artifact_id: str | None,
artifact_type: str,
source: str,
storage_key: str | None = None,
storage_type: str = 'binary_storage',
mime_type: str | None = None,
name: str | None = None,
size_bytes: int | None = None,
sha256: str | None = None,
conversation_id: str | None = None,
run_id: str | None = None,
runner_id: str | None = None,
bot_id: str | None = None,
workspace_id: str | None = None,
expires_at: datetime.datetime | None = None,
metadata: dict[str, typing.Any] | None = None,
content: bytes | None = None,
) -> str:
"""Register a new artifact.
If content is provided and storage_key is None, stores content
in BinaryStorage automatically.
Args:
artifact_id: Unique artifact ID (generated if None)
artifact_type: Type of artifact (image, file, voice, tool_result, etc.)
source: Source of artifact (platform, runner, tool, system)
storage_key: Key in BinaryStorage or external reference
storage_type: Storage type (binary_storage, file, url)
mime_type: MIME type
name: Original file name
size_bytes: Size in bytes
sha256: SHA256 hash
conversation_id: Conversation ID
run_id: Run ID that created this
runner_id: Runner ID that created this
bot_id: Bot UUID
workspace_id: Workspace ID
expires_at: Expiration time
metadata: Additional metadata
content: Optional content to store in BinaryStorage
Returns:
The artifact_id
"""
if artifact_id is None:
artifact_id = str(uuid.uuid4())
# If content provided, store in BinaryStorage
if content is not None and storage_key is None:
storage_key = f"artifact:{artifact_id}"
storage_type = 'binary_storage'
if size_bytes is None:
size_bytes = len(content)
async with self._session_factory() as session:
# Store content in BinaryStorage if provided
if content is not None:
binary_storage = BinaryStorage(
unique_key=f'artifact:{artifact_id}',
key=storage_key,
owner_type='artifact',
owner='host',
value=content,
)
session.add(binary_storage)
# Store artifact metadata
artifact = AgentArtifact(
artifact_id=artifact_id,
artifact_type=artifact_type,
mime_type=mime_type,
name=name,
size_bytes=size_bytes,
sha256=sha256,
source=source,
storage_key=storage_key,
storage_type=storage_type,
conversation_id=conversation_id,
run_id=run_id,
runner_id=runner_id,
bot_id=bot_id,
workspace_id=workspace_id,
created_at=datetime.datetime.utcnow(),
expires_at=expires_at,
metadata_json=json.dumps(metadata) if metadata else None,
)
session.add(artifact)
await session.commit()
return artifact_id
async def get_metadata(
self,
artifact_id: str,
) -> dict[str, typing.Any] | None:
"""Get artifact metadata (public fields only, no internal storage info).
Args:
artifact_id: Artifact ID
Returns:
Artifact metadata dict compatible with SDK ArtifactMetadata, or None if not found
"""
async with self._session_factory() as session:
result = await session.execute(
sqlalchemy.select(AgentArtifact).where(
AgentArtifact.artifact_id == artifact_id
)
)
row = result.scalars().first()
if row is None:
return None
return self._row_to_public_dict(row)
async def _get_internal_record(
self,
artifact_id: str,
) -> AgentArtifact | None:
"""Get full artifact record including internal fields.
Used internally by read_artifact to access storage_key/storage_type.
Args:
artifact_id: Artifact ID
Returns:
AgentArtifact ORM instance, or None if not found
"""
async with self._session_factory() as session:
result = await session.execute(
sqlalchemy.select(AgentArtifact).where(
AgentArtifact.artifact_id == artifact_id
)
)
return result.scalars().first()
async def read_artifact(
self,
artifact_id: str,
offset: int = 0,
limit: int | None = None,
) -> dict[str, typing.Any] | None:
"""Read artifact content.
For small artifacts, returns content_base64 directly.
For large artifacts, returns file_key for chunked transfer.
Args:
artifact_id: Artifact ID
offset: Byte offset to start reading from (must be >= 0)
limit: Maximum bytes to read (must be > 0 if provided)
Returns:
ArtifactReadResult dict, or None if not found
Raises:
ValueError: If offset < 0 or limit <= 0
"""
# Validate offset and limit
if offset < 0:
raise ValueError("offset must be >= 0")
if limit is not None and limit <= 0:
raise ValueError("limit must be > 0")
# Get internal record (includes storage_key/storage_type)
record = await self._get_internal_record(artifact_id)
if record is None:
return None
storage_type = record.storage_type or 'binary_storage'
storage_key = record.storage_key
size_bytes = record.size_bytes or 0
# Cap limit at hard limit
if limit is None:
limit = self.MAX_INLINE_READ_BYTES
limit = min(limit, self.MAX_RANGE_READ_BYTES)
# For binary_storage, read content
if storage_type == 'binary_storage' and storage_key:
content = await self._read_binary_storage(storage_key)
if content is None:
return None
# Apply offset and limit
if offset > 0:
content = content[offset:]
if limit and len(content) > limit:
content = content[:limit]
has_more = True
else:
has_more = False
return {
'artifact_id': artifact_id,
'mime_type': record.mime_type,
'size_bytes': size_bytes,
'offset': offset,
'length': len(content),
'content_base64': base64.b64encode(content).decode('utf-8'),
'file_key': None,
'has_more': has_more,
}
# For other storage types, return storage reference
# (caller can use file_key for chunked transfer)
return {
'artifact_id': artifact_id,
'mime_type': record.mime_type,
'size_bytes': size_bytes,
'offset': offset,
'length': None,
'content_base64': None,
'file_key': storage_key,
'has_more': False,
}
async def _read_binary_storage(self, key: str) -> bytes | None:
"""Read content from BinaryStorage.
Uses unique_key for isolation to prevent cross-artifact access.
Args:
key: The unique_key used when storing the artifact
Returns:
Content bytes, or None if not found
"""
async with self._session_factory() as session:
result = await session.execute(
sqlalchemy.select(BinaryStorage).where(BinaryStorage.unique_key == key)
)
row = result.scalars().first()
if row is None:
return None
return row.value
def _row_to_public_dict(self, row: AgentArtifact) -> dict[str, typing.Any]:
"""Convert an AgentArtifact row to public dict.
Returns only fields that match SDK ArtifactMetadata entity.
Host-only fields (bot_id, workspace_id, storage_key, storage_type) are excluded.
"""
return {
'artifact_id': row.artifact_id,
'artifact_type': row.artifact_type,
'mime_type': row.mime_type,
'name': row.name,
'size_bytes': row.size_bytes,
'sha256': row.sha256,
'source': row.source,
'conversation_id': row.conversation_id,
'run_id': row.run_id,
'runner_id': row.runner_id,
'created_at': int(row.created_at.timestamp()) if row.created_at else None,
'expires_at': int(row.expires_at.timestamp()) if row.expires_at else None,
'metadata': json.loads(row.metadata_json) if row.metadata_json else {},
}

View File

@@ -1,63 +0,0 @@
"""Resolve host events to one effective Agent binding."""
from __future__ import annotations
from .host_models import AgentConfig, AgentBinding, AgentEventEnvelope, BindingScope
class AgentBindingResolutionError(Exception):
"""Raised when an event cannot resolve to exactly one Agent binding."""
class AgentBindingResolver:
"""Resolve an event to a single AgentBinding.
The target product model is one bot / IM channel -> one Agent. Fan-out,
observer agents, or multi-runner arbitration require separate delivery and
state semantics and are intentionally not hidden in this resolver.
"""
def resolve_one(
self,
event: AgentEventEnvelope,
agents: list[AgentConfig],
) -> AgentBinding:
"""Resolve exactly one enabled Agent for the event."""
matches = [
agent
for agent in agents
if agent.enabled and event.event_type in agent.event_types
]
if not matches:
raise AgentBindingResolutionError(
f'No Agent binding matches event_type={event.event_type}'
)
if len(matches) > 1:
agent_ids = ', '.join(agent.agent_id or '<anonymous>' for agent in matches)
raise AgentBindingResolutionError(
f'Multiple Agent bindings match event_type={event.event_type}: {agent_ids}'
)
return self._to_binding(matches[0])
def _to_binding(self, agent: AgentConfig) -> AgentBinding:
"""Project product-level Agent config into the run-time binding model."""
scope = BindingScope(
scope_type='agent',
scope_id=agent.agent_id,
)
return AgentBinding(
binding_id=f"agent_{agent.agent_id or 'default'}_{agent.runner_id}",
scope=scope,
event_types=list(agent.event_types),
runner_id=agent.runner_id,
runner_config=agent.runner_config,
resource_policy=agent.resource_policy,
state_policy=agent.state_policy,
delivery_policy=agent.delivery_policy,
enabled=agent.enabled,
agent_id=agent.agent_id,
)

View File

@@ -1,95 +0,0 @@
"""Helpers for the current AgentRunner config shape."""
from __future__ import annotations
import typing
class ConfigMigration:
"""Configuration helper for agent runner IDs.
Responsibilities:
- Resolve runner ID from ai.runner.id
- Extract current Agent/runner config from ai.runner_config
- Keep the current config container shape stable on save
"""
@staticmethod
def resolve_runner_id(pipeline_config: dict[str, typing.Any]) -> str | None:
"""Resolve runner ID from current configuration.
Args:
pipeline_config: Current configuration container
Returns:
Runner ID string, or None if not configured
"""
ai_config = pipeline_config.get('ai', {})
runner_config = ai_config.get('runner', {})
runner_id = runner_config.get('id')
if runner_id:
return runner_id
return None
@staticmethod
def resolve_runner_config(
pipeline_config: dict[str, typing.Any],
runner_id: str,
) -> dict[str, typing.Any]:
"""Resolve Agent/runner configuration from the current container.
Args:
pipeline_config: Current configuration container
runner_id: Resolved runner ID
Returns:
Runner configuration dict (empty if not found)
"""
ai_config = pipeline_config.get('ai', {})
runner_configs = ai_config.get('runner_config', {})
if runner_id in runner_configs:
return runner_configs[runner_id]
return {}
@staticmethod
def get_expire_time(pipeline_config: dict[str, typing.Any]) -> int:
"""Get conversation expire time from configuration.
Args:
pipeline_config: Current configuration container
Returns:
Expire time in seconds (0 means no expiry)
"""
ai_config = pipeline_config.get('ai', {})
runner_config = ai_config.get('runner', {})
return runner_config.get('expire-time', 0)
@staticmethod
def migrate_pipeline_config(pipeline_config: dict[str, typing.Any]) -> dict[str, typing.Any]:
"""Normalize the current config container before saving.
Args:
pipeline_config: Original configuration
Returns:
Configuration with explicit ai.runner and ai.runner_config containers
"""
new_config = dict(pipeline_config)
if 'ai' not in new_config:
return new_config
ai_config = dict(new_config.get('ai', {}))
runner_config = dict(ai_config.get('runner', {}))
runner_configs = dict(ai_config.get('runner_config', {}))
ai_config['runner'] = runner_config
ai_config['runner_config'] = runner_configs
new_config['ai'] = ai_config
return new_config

View File

@@ -1,208 +0,0 @@
"""Helpers for interpreting AgentRunner DynamicForm configuration."""
from __future__ import annotations
import typing
from .descriptor import AgentRunnerDescriptor
LLM_MODEL_SELECTOR_TYPES = {'model-fallback-selector', 'llm-model-selector'}
KB_SELECTOR_TYPES = {'knowledge-base-multi-selector'}
PROMPT_EDITOR_TYPES = {'prompt-editor'}
NONE_SENTINELS = {'', '__none__', '__none'}
def iter_schema_items(
descriptor: AgentRunnerDescriptor | None,
field_types: set[str],
) -> typing.Iterator[dict[str, typing.Any]]:
"""Yield descriptor config schema items whose type is in field_types."""
if descriptor is None:
return
for item in descriptor.config_schema or []:
if not isinstance(item, dict):
continue
if item.get('type') in field_types:
yield item
def has_permission(
descriptor: AgentRunnerDescriptor | None,
name: str,
actions: set[str],
) -> bool:
"""Return whether a runner descriptor requests one of the given actions."""
if descriptor is None:
return False
configured_actions = descriptor.permissions.get(name, [])
return any(action in configured_actions for action in actions)
def uses_host_models(descriptor: AgentRunnerDescriptor | None) -> bool:
"""Return whether LangBot should resolve model resources for this runner."""
return (
has_permission(descriptor, 'models', {'invoke', 'stream', 'list'})
and any(True for _ in iter_schema_items(descriptor, LLM_MODEL_SELECTOR_TYPES))
)
def uses_host_tools(descriptor: AgentRunnerDescriptor | None) -> bool:
"""Return whether LangBot should expose tool resources to this runner."""
return (
descriptor is not None
and descriptor.supports_tool_calling()
and has_permission(descriptor, 'tools', {'list', 'detail', 'call'})
)
def uses_host_knowledge_bases(descriptor: AgentRunnerDescriptor | None) -> bool:
"""Return whether LangBot should expose knowledge-base resources to this runner."""
return (
descriptor is not None
and descriptor.supports_knowledge_retrieval()
and has_permission(descriptor, 'knowledge_bases', {'list', 'retrieve'})
)
def extract_prompt_config(
descriptor: AgentRunnerDescriptor | None,
runner_config: dict[str, typing.Any],
default_prompt: list[dict[str, typing.Any]],
) -> list[dict[str, typing.Any]]:
"""Extract the prompt-editor value selected by the runner schema."""
for item in iter_schema_items(descriptor, PROMPT_EDITOR_TYPES):
field_name = item.get('name')
if field_name and field_name in runner_config:
configured_prompt = runner_config[field_name]
if isinstance(configured_prompt, list):
return configured_prompt
default_value = item.get('default')
if isinstance(default_value, list):
return default_value
return default_prompt
def extract_model_selection(
descriptor: AgentRunnerDescriptor | None,
runner_config: dict[str, typing.Any],
) -> tuple[str, list[str]]:
"""Extract primary/fallback LLM selections from schema-defined fields."""
primary_uuid = ''
fallback_uuids: list[str] = []
for item in iter_schema_items(descriptor, LLM_MODEL_SELECTOR_TYPES):
field_name = item.get('name')
if not field_name:
continue
value = runner_config.get(field_name, item.get('default'))
if item.get('type') == 'model-fallback-selector':
if isinstance(value, str):
primary_uuid = value
elif isinstance(value, dict):
primary_uuid = value.get('primary') or ''
fallbacks = value.get('fallbacks', [])
if isinstance(fallbacks, list):
fallback_uuids = [fallback for fallback in fallbacks if isinstance(fallback, str)]
break
if item.get('type') == 'llm-model-selector' and isinstance(value, str):
primary_uuid = value
break
return primary_uuid, fallback_uuids
def extract_knowledge_base_uuids(
descriptor: AgentRunnerDescriptor | None,
runner_config: dict[str, typing.Any],
) -> list[str]:
"""Extract configured knowledge-base UUIDs from schema-defined fields."""
if not uses_host_knowledge_bases(descriptor):
return []
kb_uuids: list[str] = []
for item in iter_schema_items(descriptor, KB_SELECTOR_TYPES):
field_name = item.get('name')
if not field_name:
continue
value = runner_config.get(field_name, item.get('default', []))
if isinstance(value, list):
kb_uuids.extend(
kb_uuid for kb_uuid in value if isinstance(kb_uuid, str) and kb_uuid not in NONE_SENTINELS
)
return list(dict.fromkeys(kb_uuids))
def iter_config_model_refs(
descriptor: AgentRunnerDescriptor,
runner_config: dict[str, typing.Any],
) -> typing.Iterator[tuple[str, str]]:
"""Yield model references declared by schema-defined model selector fields."""
for item in descriptor.config_schema or []:
if not isinstance(item, dict):
continue
field_name = item.get('name')
field_type = item.get('type')
if not field_name or field_name not in runner_config:
continue
value = runner_config.get(field_name)
if field_type == 'model-fallback-selector':
if isinstance(value, str) and value not in NONE_SENTINELS:
yield 'llm', value
elif isinstance(value, dict):
primary = value.get('primary')
if isinstance(primary, str) and primary not in NONE_SENTINELS:
yield 'llm', primary
fallbacks = value.get('fallbacks', [])
if isinstance(fallbacks, list):
for fallback_uuid in fallbacks:
if isinstance(fallback_uuid, str) and fallback_uuid not in NONE_SENTINELS:
yield 'llm', fallback_uuid
elif field_type == 'llm-model-selector':
if isinstance(value, str) and value not in NONE_SENTINELS:
yield 'llm', value
elif field_type == 'rerank-model-selector':
if isinstance(value, str) and value not in NONE_SENTINELS:
yield 'rerank', value
def set_empty_llm_model_selection(
descriptor: AgentRunnerDescriptor,
runner_config: dict[str, typing.Any],
model_uuid: str,
) -> bool:
"""Set the first empty schema-defined LLM selector to model_uuid."""
for item in iter_schema_items(descriptor, LLM_MODEL_SELECTOR_TYPES):
field_name = item.get('name')
field_type = item.get('type')
if not field_name:
continue
value = runner_config.get(field_name, item.get('default'))
if field_type == 'model-fallback-selector':
if isinstance(value, dict):
primary = value.get('primary') or ''
if primary not in NONE_SENTINELS:
return False
fallbacks = value.get('fallbacks', [])
runner_config[field_name] = {
'primary': model_uuid,
'fallbacks': fallbacks if isinstance(fallbacks, list) else [],
}
return True
if isinstance(value, str) and value not in NONE_SENTINELS:
return False
runner_config[field_name] = {'primary': model_uuid, 'fallbacks': []}
return True
if field_type == 'llm-model-selector':
if isinstance(value, str) and value not in NONE_SENTINELS:
return False
runner_config[field_name] = model_uuid
return True
return False

View File

@@ -1,420 +0,0 @@
"""Agent run context builder for provisioning AgentRunContext envelopes."""
from __future__ import annotations
import uuid
import time
import typing
from ...core import app
from .descriptor import AgentRunnerDescriptor
from .persistent_state_store import get_persistent_state_store
from .host_models import AgentEventEnvelope, AgentBinding
DEFAULT_RUNNER_TIMEOUT_SECONDS = 300
# Internal models for the agent runner context protocol.
class AgentTrigger(typing.TypedDict):
"""Agent trigger information."""
type: str
source: str
timestamp: int | None
class ConversationContext(typing.TypedDict):
"""Conversation context."""
conversation_id: str | None
thread_id: str | None
launcher_type: str | None
launcher_id: str | None
sender_id: str | None
bot_id: str | None
workspace_id: str | None
session_id: str | None
class AgentInput(typing.TypedDict):
"""Agent input."""
text: str | None
contents: list[dict[str, typing.Any]]
message_chain: dict[str, typing.Any] | None
attachments: list[dict[str, typing.Any]]
class AgentRunState(typing.TypedDict):
"""Agent run state with 4 scopes."""
conversation: dict[str, typing.Any]
actor: dict[str, typing.Any]
subject: dict[str, typing.Any]
runner: dict[str, typing.Any]
# Resource payload models matching langbot-plugin-sdk/resources.py.
class ModelResource(typing.TypedDict):
"""Model resource payload."""
model_id: str
model_type: str | None
provider: str | None
class ToolResource(typing.TypedDict):
"""Tool resource payload."""
tool_name: str
tool_type: str | None
description: str | None
class KnowledgeBaseResource(typing.TypedDict):
"""Knowledge base resource payload."""
kb_id: str
kb_name: str | None
kb_type: str | None
class FileResource(typing.TypedDict):
"""File resource payload."""
file_id: str
file_name: str | None
mime_type: str | None
source: str | None
class StorageResource(typing.TypedDict):
"""Storage resource payload."""
plugin_storage: bool
workspace_storage: bool
class AgentResources(typing.TypedDict):
"""Agent resources payload."""
models: list[ModelResource]
tools: list[ToolResource]
knowledge_bases: list[KnowledgeBaseResource]
files: list[FileResource]
storage: StorageResource
platform_capabilities: dict[str, typing.Any]
class AgentRuntimeContext(typing.TypedDict):
"""Agent runtime context."""
langbot_version: str | None
sdk_protocol_version: str
trace_id: str | None
deadline_at: float | None
metadata: dict[str, typing.Any]
class AgentRunContextPayload(typing.TypedDict):
"""AgentRunContext payload passed to an agent runner.
Protocol v1 structure - matches SDK AgentRunContext.
Note: The 'config' field contains the current Agent/runner config
from ai.runner_config[runner_id] while the current Query entry remains
a temporary configuration container. It is not plugin instance config.
"""
run_id: str
trigger: AgentTrigger
conversation: ConversationContext | None
event: dict[str, typing.Any] # REQUIRED for Protocol v1
actor: dict[str, typing.Any] | None
subject: dict[str, typing.Any] | None
input: AgentInput
delivery: dict[str, typing.Any] # REQUIRED for Protocol v1
resources: AgentResources
context: dict[str, typing.Any] # ContextAccess - REQUIRED for Protocol v1
state: AgentRunState
runtime: AgentRuntimeContext
config: dict[str, typing.Any] # Agent/runner config from ai.runner_config[runner_id]
adapter: dict[str, typing.Any] | None # Entry adapter context
metadata: dict[str, typing.Any] # Additional metadata
class AgentRunContextBuilder:
"""Builder for provisioning AgentRunContext.
Responsibilities:
- Generate new run_id (UUID, not query id)
- Set trigger type based on event source
- Build conversation context from event
- Build input from event
- Build state snapshot from PersistentStateStore
- Build runtime context with host info, trace_id, deadline
- Set config from current Agent/runner configuration.
Query adaptation belongs to QueryEntryAdapter, not this builder.
"""
ap: app.Application
def __init__(self, ap: app.Application):
self.ap = ap
async def build_context_from_event(
self,
event: AgentEventEnvelope,
binding: AgentBinding,
descriptor: AgentRunnerDescriptor,
resources: AgentResources,
) -> AgentRunContextPayload:
"""Build AgentRunContext from event-first envelope.
This is the main entry point for Protocol v1.
Does NOT inline full history by default.
Args:
event: Event envelope
binding: Agent binding
descriptor: Runner descriptor
resources: Built resources
Returns:
AgentRunContextPayload for the runner
"""
# Generate new run_id
run_id = str(uuid.uuid4())
# Build trigger from event
trigger: AgentTrigger = {
'type': event.event_type,
'source': event.source,
'timestamp': event.event_time or int(time.time()),
}
# Build conversation context from event
conversation: ConversationContext | None = None
if event.conversation_id:
conversation = {
'session_id': None,
'conversation_id': event.conversation_id,
'thread_id': event.thread_id,
'launcher_type': None, # Will be filled from actor/subject if needed
'launcher_id': None,
'sender_id': event.actor.actor_id if event.actor else None,
'bot_id': event.bot_id,
'workspace_id': event.workspace_id,
}
# Build event context (Protocol v1 event-first)
event_context = {
'event_id': event.event_id,
'event_type': event.event_type,
'event_time': event.event_time,
'source': event.source,
'source_event_type': event.source_event_type,
'raw_ref': event.raw_ref.model_dump(mode='json') if event.raw_ref else None,
'data': event.data,
}
# Build actor context
actor_context = None
if event.actor:
actor_context = {
'actor_type': event.actor.actor_type,
'actor_id': event.actor.actor_id,
'actor_name': event.actor.actor_name,
}
# Build subject context
subject_context = None
if event.subject:
subject_context = {
'subject_type': event.subject.subject_type,
'subject_id': event.subject.subject_id,
'data': event.subject.data,
}
# Build input from event
input: AgentInput = {
'text': event.input.text,
'contents': [c.model_dump(mode='json') if hasattr(c, 'model_dump') else c for c in event.input.contents],
'message_chain': event.input.message_chain,
'attachments': [
a.model_dump(mode='json') if hasattr(a, 'model_dump') else a for a in event.input.attachments
],
}
# Build context access (no history inlined by default for Protocol v1)
# Populate with actual values from stores
context_access = await self._build_context_access(event, descriptor, binding)
# Build state snapshot from persistent state store (event-first Protocol v1)
persistent_state_store = get_persistent_state_store(self.ap.persistence_mgr.get_db_engine())
state: AgentRunState = await persistent_state_store.build_snapshot_from_event(event, binding, descriptor)
# Build runtime context
runtime: AgentRuntimeContext = {
'langbot_version': self.ap.ver_mgr.get_current_version(),
'sdk_protocol_version': descriptor.protocol_version,
'trace_id': run_id,
'deadline_at': self._build_deadline_from_binding(binding),
'metadata': {
'bot_id': event.bot_id,
'workspace_id': event.workspace_id,
'streaming_supported': event.delivery.supports_streaming,
'model_context_window_tokens': None,
# TODO(model-info): populate model_context_window_tokens after
# LiteLLM/model metadata lands. Runners fall back to their
# ctx.config until Host can provide the real window.
},
}
# Build delivery context
delivery_context = {
'surface': event.delivery.surface,
'reply_target': event.delivery.reply_target,
'supports_streaming': event.delivery.supports_streaming,
'supports_edit': event.delivery.supports_edit,
'supports_reaction': event.delivery.supports_reaction,
'max_message_size': event.delivery.max_message_size,
'platform_capabilities': event.delivery.platform_capabilities,
}
# Build adapter context (empty for event-first)
adapter_context = {
'extra': {},
}
# Build full context - Protocol v1 structure
context: AgentRunContextPayload = {
'run_id': run_id,
'trigger': trigger,
'conversation': conversation,
'event': event_context, # REQUIRED
'actor': actor_context,
'subject': subject_context,
'input': input,
'delivery': delivery_context, # REQUIRED
'resources': resources,
'context': context_access, # ContextAccess - REQUIRED
'state': state,
'runtime': runtime,
'config': binding.runner_config,
'adapter': adapter_context,
'metadata': {}, # Additional metadata
}
return context
def _build_deadline_from_binding(self, binding: AgentBinding) -> float | None:
"""Build deadline timestamp from binding timeout config.
Args:
binding: Agent binding with runner_config
Returns:
Deadline timestamp or None
"""
timeout = binding.runner_config.get('timeout', DEFAULT_RUNNER_TIMEOUT_SECONDS)
if timeout is None:
return None
try:
timeout_seconds = float(timeout)
except (TypeError, ValueError):
return None
if timeout_seconds <= 0:
return None
return time.time() + timeout_seconds
async def _build_context_access(
self,
event: AgentEventEnvelope,
descriptor: AgentRunnerDescriptor,
binding: AgentBinding | None = None,
) -> dict[str, typing.Any]:
"""Build ContextAccess with actual values from stores.
Args:
event: Event envelope
descriptor: Runner descriptor
binding: Agent binding (required for state_policy in event-first mode)
Returns:
ContextAccess dict
"""
conversation_id = event.conversation_id
# Check if history APIs are available for this runner
# Based on runner permissions
permissions = descriptor.permissions or {}
history_permissions = permissions.get('history', [])
event_permissions = permissions.get('events', [])
artifact_permissions = permissions.get('artifacts', [])
history_page_enabled = 'page' in history_permissions and conversation_id is not None
history_search_enabled = 'search' in history_permissions and conversation_id is not None
event_get_enabled = 'get' in event_permissions
event_page_enabled = 'page' in event_permissions and conversation_id is not None
artifact_metadata_enabled = 'metadata' in artifact_permissions
artifact_read_enabled = 'read' in artifact_permissions
# Determine state API availability based on binding state_policy.
state_enabled = False
if binding is not None:
state_policy = binding.state_policy
if state_policy.enable_state and state_policy.state_scopes:
state_enabled = True
# Get latest cursor and has_history_before if conversation exists
latest_cursor = None
has_history_before = False
if conversation_id:
try:
from .transcript_store import TranscriptStore
store = TranscriptStore(self.ap.persistence_mgr.get_db_engine())
latest_cursor = await store.get_latest_cursor(conversation_id)
if latest_cursor:
has_history_before = True
except Exception as e:
self.ap.logger.warning(f'Failed to get transcript cursor: {e}')
return {
'conversation_id': conversation_id,
'thread_id': event.thread_id,
'latest_cursor': latest_cursor,
'event_seq': None, # Will be populated when EventLog is written
'transcript_seq': int(latest_cursor) if latest_cursor else None,
'has_history_before': has_history_before,
'inline_policy': {
'mode': 'current_event',
'delivered_count': 0,
'source_total_count': None,
'messages_complete': False,
'reason': 'self_managed_context',
},
'available_apis': {
'history_page': history_page_enabled,
'history_search': history_search_enabled,
'event_get': event_get_enabled,
'event_page': event_page_enabled,
'artifact_metadata': artifact_metadata_enabled,
'artifact_read': artifact_read_enabled,
'state': state_enabled,
'storage': True,
'prompt_get': False,
},
}

View File

@@ -1,72 +0,0 @@
"""Agent runner descriptor."""
from __future__ import annotations
import typing
import pydantic
class AgentRunnerDescriptor(pydantic.BaseModel):
"""Descriptor for an agent runner.
Represents the discovered metadata for a runner, including
its identity, capabilities, permissions, and configuration schema.
"""
id: str
"""Unique runner ID: plugin:author/plugin_name/runner_name"""
source: typing.Literal['plugin']
"""Runner source type"""
label: dict[str, str]
"""Display labels keyed by locale (e.g., en_US, zh_Hans)"""
description: dict[str, str] | None = None
"""Optional description keyed by locale"""
plugin_author: str
"""Plugin author from manifest"""
plugin_name: str
"""Plugin name from manifest"""
runner_name: str
"""AgentRunner component name from manifest"""
plugin_version: str | None = None
"""Optional plugin version"""
protocol_version: str = '1'
"""SDK protocol version, default '1'"""
config_schema: list[dict[str, typing.Any]] = []
"""Configuration schema using DynamicForm format"""
capabilities: dict[str, bool] = {}
"""Runner capabilities: streaming, tool_calling, knowledge_retrieval, etc."""
permissions: dict[str, list[str]] = {}
"""Requested permissions: models, tools, knowledge_bases, storage, files, platform_api"""
raw_manifest: dict[str, typing.Any] = {}
"""Original manifest for reference"""
model_config = pydantic.ConfigDict(
extra='allow',
)
def get_plugin_id(self) -> str:
"""Return plugin identifier as author/name."""
return f'{self.plugin_author}/{self.plugin_name}'
def supports_streaming(self) -> bool:
"""Check if runner supports streaming output."""
return self.capabilities.get('streaming', False)
def supports_tool_calling(self) -> bool:
"""Check if runner supports tool calling."""
return self.capabilities.get('tool_calling', False)
def supports_knowledge_retrieval(self) -> bool:
"""Check if runner supports knowledge retrieval."""
return self.capabilities.get('knowledge_retrieval', False)

View File

@@ -1,37 +0,0 @@
"""Agent runner errors."""
from __future__ import annotations
class AgentRunnerError(Exception):
"""Base error for agent runner operations."""
pass
class RunnerNotFoundError(AgentRunnerError):
"""Runner not found in registry."""
def __init__(self, runner_id: str):
self.runner_id = runner_id
super().__init__(f'Agent runner not found: {runner_id}')
class RunnerNotAuthorizedError(AgentRunnerError):
"""Runner not authorized for this binding."""
def __init__(self, runner_id: str, bound_plugins: list[str] | None):
self.runner_id = runner_id
self.bound_plugins = bound_plugins
super().__init__(f'Agent runner {runner_id} not authorized for bound_plugins={bound_plugins}')
class RunnerProtocolError(AgentRunnerError):
"""Runner protocol version mismatch or invalid manifest."""
def __init__(self, runner_id: str, message: str):
self.runner_id = runner_id
super().__init__(f'Agent runner protocol error for {runner_id}: {message}')
class RunnerExecutionError(AgentRunnerError):
"""Runner execution failed."""
def __init__(self, runner_id: str, message: str, retryable: bool = False):
self.runner_id = runner_id
self.retryable = retryable
super().__init__(f'Agent runner {runner_id} execution failed: {message}')

View File

@@ -1,255 +0,0 @@
"""EventLog store for writing and querying event records."""
from __future__ import annotations
import json
import datetime
import typing
import uuid
import sqlalchemy
from sqlalchemy.ext.asyncio import AsyncEngine, AsyncSession
from sqlalchemy.orm import sessionmaker
from ...entity.persistence.event_log import EventLog
class EventLogStore:
"""Store for EventLog records.
Handles writing events to the event log and querying them.
All methods are async and use the provided database engine.
"""
engine: AsyncEngine
# Hard limits
MAX_INPUT_SUMMARY_LENGTH = 1000
def __init__(self, engine: AsyncEngine):
self.engine = engine
self._session_factory = sessionmaker(
engine, class_=AsyncSession, expire_on_commit=False
)
async def append_event(
self,
event_id: str | None,
event_type: str,
source: str,
bot_id: str | None = None,
workspace_id: str | None = None,
conversation_id: str | None = None,
thread_id: str | None = None,
actor_type: str | None = None,
actor_id: str | None = None,
actor_name: str | None = None,
subject_type: str | None = None,
subject_id: str | None = None,
input_summary: str | None = None,
input_json: dict[str, typing.Any] | None = None,
raw_ref: str | None = None,
run_id: str | None = None,
runner_id: str | None = None,
event_time: datetime.datetime | None = None,
metadata: dict[str, typing.Any] | None = None,
) -> str:
"""Append an event to the event log.
Args:
event_id: Unique event ID (generated if None)
event_type: Event type
source: Event source
bot_id: Bot UUID
workspace_id: Workspace ID
conversation_id: Conversation ID
thread_id: Thread ID
actor_type: Actor type
actor_id: Actor ID
actor_name: Actor display name
subject_type: Subject type
subject_id: Subject ID
input_summary: Brief input summary
input_json: Full input JSON
raw_ref: Reference to raw event payload
run_id: Run ID processing this event
runner_id: Runner ID processing this event
event_time: When the event occurred
metadata: Additional metadata
Returns:
The event_id
"""
if event_id is None:
event_id = str(uuid.uuid4())
# Truncate input summary if too long
if input_summary and len(input_summary) > self.MAX_INPUT_SUMMARY_LENGTH:
input_summary = input_summary[:self.MAX_INPUT_SUMMARY_LENGTH - 3] + "..."
async with self._session_factory() as session:
event = EventLog(
event_id=event_id,
event_type=event_type,
event_time=event_time,
source=source,
bot_id=bot_id,
workspace_id=workspace_id,
conversation_id=conversation_id,
thread_id=thread_id,
actor_type=actor_type,
actor_id=actor_id,
actor_name=actor_name,
subject_type=subject_type,
subject_id=subject_id,
input_summary=input_summary,
input_json=json.dumps(input_json) if input_json else None,
raw_ref=raw_ref,
run_id=run_id,
runner_id=runner_id,
metadata_json=json.dumps(metadata) if metadata else None,
created_at=datetime.datetime.utcnow(),
)
session.add(event)
await session.commit()
return event_id
async def get_event(
self,
event_id: str,
) -> dict[str, typing.Any] | None:
"""Get a single event by ID.
Args:
event_id: Event ID
Returns:
Event record as dict, or None if not found
"""
async with self._session_factory() as session:
result = await session.execute(
sqlalchemy.select(EventLog).where(EventLog.event_id == event_id)
)
row = result.scalars().first()
if row is None:
return None
return self._row_to_dict(row)
async def page_events(
self,
conversation_id: str | None = None,
event_types: list[str] | None = None,
before_seq: int | None = None,
limit: int = 50,
) -> tuple[list[dict[str, typing.Any]], int | None, bool]:
"""Page through event records.
Args:
conversation_id: Filter by conversation ID
event_types: Filter by event types
before_seq: Get events before this sequence number
limit: Maximum items to return (capped at 100)
Returns:
Tuple of (items, next_seq, has_more)
"""
limit = min(limit, 100) # Hard cap
async with self._session_factory() as session:
query = sqlalchemy.select(EventLog)
if conversation_id is not None:
query = query.where(EventLog.conversation_id == conversation_id)
if event_types:
query = query.where(EventLog.event_type.in_(event_types))
if before_seq is not None:
query = query.where(EventLog.id < before_seq)
query = query.order_by(EventLog.id.desc()).limit(limit + 1)
result = await session.execute(query)
rows = result.scalars().all()
items = [self._row_to_dict(row) for row in rows[:limit]]
has_more = len(rows) > limit
next_seq = items[-1]['id'] if items and has_more else None
return items, next_seq, has_more
async def get_latest_cursor(
self,
conversation_id: str,
) -> str | None:
"""Get the latest cursor for a conversation.
Args:
conversation_id: Conversation ID
Returns:
Cursor string (seq number), or None if no events
"""
async with self._session_factory() as session:
result = await session.execute(
sqlalchemy.select(EventLog.id)
.where(EventLog.conversation_id == conversation_id)
.order_by(EventLog.id.desc())
.limit(1)
)
row = result.scalars().first()
if row is None:
return None
return str(row)
async def has_events_before(
self,
conversation_id: str,
seq: int,
) -> bool:
"""Check if there are events before a sequence number.
Args:
conversation_id: Conversation ID
seq: Sequence number
Returns:
True if there are events before
"""
async with self._session_factory() as session:
result = await session.execute(
sqlalchemy.select(sqlalchemy.func.count())
.select_from(EventLog)
.where(
EventLog.conversation_id == conversation_id,
EventLog.id < seq,
)
)
count = result.scalar()
return count > 0
def _row_to_dict(self, row: EventLog) -> dict[str, typing.Any]:
"""Convert an EventLog row to dict."""
return {
'id': row.id,
'event_id': row.event_id,
'event_type': row.event_type,
'event_time': int(row.event_time.timestamp()) if row.event_time else None,
'source': row.source,
'bot_id': row.bot_id,
'workspace_id': row.workspace_id,
'conversation_id': row.conversation_id,
'thread_id': row.thread_id,
'actor_type': row.actor_type,
'actor_id': row.actor_id,
'actor_name': row.actor_name,
'subject_type': row.subject_type,
'subject_id': row.subject_id,
'input_summary': row.input_summary,
'input_json': json.loads(row.input_json) if row.input_json else None,
'raw_ref': row.raw_ref,
'run_id': row.run_id,
'runner_id': row.runner_id,
'created_at': int(row.created_at.timestamp()) if row.created_at else None,
'metadata': json.loads(row.metadata_json) if row.metadata_json else {},
}

View File

@@ -1,25 +0,0 @@
"""Canonical AgentRunner event names reserved for future EBA integration."""
from __future__ import annotations
MESSAGE_RECEIVED = 'message.received'
"""A normal message entered the current Pipeline."""
MESSAGE_RECALLED = 'message.recalled'
"""A platform message was recalled or deleted."""
GROUP_MEMBER_JOINED = 'group.member_joined'
"""A new member joined a group/channel conversation."""
FRIEND_REQUEST_RECEIVED = 'friend.request_received'
"""A new friend/contact request was received."""
RESERVED_EVENT_TYPES = frozenset(
{
MESSAGE_RECEIVED,
MESSAGE_RECALLED,
GROUP_MEMBER_JOINED,
FRIEND_REQUEST_RECEIVED,
}
)

View File

@@ -1,207 +0,0 @@
"""Agent event envelope and binding models for LangBot Host.
These are Host-internal models, not exposed to SDK.
"""
from __future__ import annotations
import typing
import pydantic
from langbot_plugin.api.entities.builtin.agent_runner.event import (
ActorContext,
SubjectContext,
RawEventRef,
)
from langbot_plugin.api.entities.builtin.agent_runner.input import AgentInput
from langbot_plugin.api.entities.builtin.agent_runner.delivery import DeliveryContext
class AgentEventEnvelope(pydantic.BaseModel):
"""Event envelope for LangBot Host event gateway.
This is the unified input model that replaces Query-first approach.
IM / WebUI / API / EventRouter all produce this envelope.
"""
event_id: str
"""Unique event identifier."""
event_type: str
"""Event type (message.received, message.recalled, etc.)."""
event_time: int | None = None
"""Event timestamp (epoch seconds)."""
source: str
"""Event source (platform, webui, api, scheduler, system)."""
source_event_type: str | None = None
"""Original source event type, when available."""
bot_id: str | None = None
"""Bot UUID handling this event."""
workspace_id: str | None = None
"""Workspace ID (for multi-tenant)."""
conversation_id: str | None = None
"""Conversation ID."""
thread_id: str | None = None
"""Thread ID (for platforms supporting threads)."""
actor: ActorContext | None = None
"""Actor (who triggered the event)."""
subject: SubjectContext | None = None
"""Subject (what the event is about)."""
input: AgentInput
"""Event input."""
delivery: DeliveryContext
"""Delivery context."""
raw_ref: RawEventRef | None = None
"""Reference to raw event payload."""
data: dict[str, typing.Any] = pydantic.Field(default_factory=dict)
"""Small structured event payload. Large payloads should be referenced via raw_ref/artifacts."""
# Binding scope types
class BindingScope(pydantic.BaseModel):
"""Scope for agent binding."""
scope_type: typing.Literal["agent", "bot", "workspace", "global"] = "agent"
"""Scope type."""
scope_id: str | None = None
"""Scope identifier (agent_id, bot_uuid, etc.)."""
class ResourcePolicy(pydantic.BaseModel):
"""Resource policy for agent binding.
Controls what resources the runner can access.
"""
allowed_model_uuids: list[str] | None = None
"""Additional model UUID grants. None means no additional model grants."""
allowed_tool_names: list[str] | None = None
"""Additional tool name grants. None means no additional tool grants."""
allowed_kb_uuids: list[str] | None = None
"""Additional knowledge base UUID grants. None means no additional KB grants."""
allow_plugin_storage: bool = True
"""Whether plugin storage is allowed."""
allow_workspace_storage: bool = False
"""Whether workspace storage is allowed."""
class StatePolicy(pydantic.BaseModel):
"""State policy for agent binding.
Controls state management behavior.
"""
enable_state: bool = True
"""Whether host-owned state is enabled."""
state_scopes: list[typing.Literal["conversation", "actor", "subject", "runner"]] = (
pydantic.Field(default_factory=lambda: ["conversation", "actor"])
)
"""Enabled state scopes."""
class DeliveryPolicy(pydantic.BaseModel):
"""Delivery policy for agent binding.
Controls how results are delivered.
"""
enable_streaming: bool = True
"""Whether streaming output is enabled."""
enable_reply: bool = True
"""Whether reply is enabled."""
max_message_size: int | None = None
"""Maximum message size."""
class AgentConfig(pydantic.BaseModel):
"""Host-side Agent configuration.
Product-level Agent is the target replacement for Pipeline-owned agent
config. Current Pipeline entry paths can project their config into this
model during migration.
"""
agent_id: str | None = None
"""Host-side Agent/config identifier."""
runner_id: str
"""Runner ID to invoke."""
runner_config: dict[str, typing.Any] = pydantic.Field(default_factory=dict)
"""Agent/runner binding configuration."""
resource_policy: ResourcePolicy = pydantic.Field(default_factory=ResourcePolicy)
"""Resource policy for this Agent."""
state_policy: StatePolicy = pydantic.Field(default_factory=StatePolicy)
"""State policy for this Agent."""
delivery_policy: DeliveryPolicy = pydantic.Field(default_factory=DeliveryPolicy)
"""Delivery policy for this Agent."""
event_types: list[str] = pydantic.Field(default_factory=lambda: ["message.received"])
"""Event types this Agent handles."""
enabled: bool = True
"""Whether this Agent can be selected by a binding resolver."""
metadata: dict[str, typing.Any] = pydantic.Field(default_factory=dict)
"""Non-protocol diagnostic metadata, such as legacy config source."""
class AgentBinding(pydantic.BaseModel):
"""Binding configuration for mapping events to runners.
This is Host-internal model for event-to-runner binding.
It replaces the old Pipeline runner config role.
"""
binding_id: str
"""Unique binding identifier."""
scope: BindingScope = pydantic.Field(default_factory=BindingScope)
"""Binding scope."""
event_types: list[str] = pydantic.Field(default_factory=lambda: ["message.received"])
"""Event types this binding handles."""
runner_id: str
"""Runner ID to invoke."""
runner_config: dict[str, typing.Any] = pydantic.Field(default_factory=dict)
"""Current Agent/runner configuration."""
resource_policy: ResourcePolicy = pydantic.Field(default_factory=ResourcePolicy)
"""Resource policy."""
state_policy: StatePolicy = pydantic.Field(default_factory=StatePolicy)
"""State policy."""
delivery_policy: DeliveryPolicy = pydantic.Field(default_factory=DeliveryPolicy)
"""Delivery policy."""
enabled: bool = True
"""Whether binding is enabled."""
agent_id: str | None = None
"""Host-side Agent/config identifier for this binding."""

View File

@@ -1,91 +0,0 @@
"""Agent runner ID parsing and formatting."""
from __future__ import annotations
import dataclasses
@dataclasses.dataclass(frozen=True)
class RunnerIdParts:
"""Parsed runner ID components."""
source: str # 'plugin' (future: 'builtin')
plugin_author: str
plugin_name: str
runner_name: str
def to_plugin_id(self) -> str:
"""Return plugin identifier as author/name."""
return f'{self.plugin_author}/{self.plugin_name}'
def parse_runner_id(runner_id: str) -> RunnerIdParts:
"""Parse runner ID string into components.
Args:
runner_id: Runner ID in format 'plugin:author/plugin_name/runner_name'
Returns:
RunnerIdParts with parsed components
Raises:
ValueError: If runner_id format is invalid
"""
if runner_id.startswith('plugin:'):
parts = runner_id[7:].split('/')
if len(parts) != 3:
raise ValueError(
f'Invalid plugin runner ID format: {runner_id}. '
f'Expected: plugin:author/plugin_name/runner_name'
)
plugin_author, plugin_name, runner_name = parts
if not plugin_author or not plugin_name or not runner_name:
raise ValueError(
f'Invalid plugin runner ID: {runner_id}. '
f'author, plugin_name, and runner_name must be non-empty'
)
return RunnerIdParts(
source='plugin',
plugin_author=plugin_author,
plugin_name=plugin_name,
runner_name=runner_name,
)
else:
# Only plugin runner IDs are valid at the protocol boundary.
raise ValueError(
f'Invalid runner ID format: {runner_id}. '
f'Expected: plugin:author/plugin_name/runner_name'
)
def format_runner_id(
source: str,
plugin_author: str,
plugin_name: str,
runner_name: str,
) -> str:
"""Format runner ID from components.
Args:
source: Runner source ('plugin')
plugin_author: Plugin author
plugin_name: Plugin name
runner_name: Runner component name
Returns:
Runner ID string
"""
if source == 'plugin':
return f'plugin:{plugin_author}/{plugin_name}/{runner_name}'
else:
raise ValueError(f'Invalid runner source: {source}')
def is_plugin_runner_id(runner_id: str) -> bool:
"""Check if runner ID is a plugin runner.
Args:
runner_id: Runner ID string
Returns:
True if runner ID starts with 'plugin:'
"""
return runner_id.startswith('plugin:')

View File

@@ -1,886 +0,0 @@
"""Agent run orchestrator for coordinating runner execution."""
from __future__ import annotations
import typing
import traceback
import asyncio
import time
from langbot_plugin.api.entities.builtin.provider import message as provider_message
from langbot_plugin.api.entities.builtin.pipeline import query as pipeline_query
from langbot_plugin.entities.io.errors import ActionCallTimeoutError
from ...core import app
from .descriptor import AgentRunnerDescriptor
from .registry import AgentRunnerRegistry
from .context_builder import AgentRunContextBuilder, AgentRunContextPayload
from .resource_builder import AgentResourceBuilder
from .result_normalizer import AgentResultNormalizer
from .persistent_state_store import get_persistent_state_store, PersistentStateStore
from .session_registry import get_session_registry, AgentRunSessionRegistry
from .config_migration import ConfigMigration
from .host_models import AgentEventEnvelope, AgentBinding
from .query_entry_adapter import QueryEntryAdapter
from .binding_resolver import AgentBindingResolver
from .state_scope import build_state_context
from .errors import (
RunnerNotFoundError,
RunnerExecutionError,
RunnerProtocolError,
)
# Maximum inline artifact content size (1MB)
MAX_ARTIFACT_INLINE_BYTES = 1 * 1024 * 1024
class AgentRunOrchestrator:
"""Orchestrator for agent runner execution.
Responsibilities:
- Resolve runner ID from current Agent/runner config
- Get runner descriptor from registry
- Provision AgentRunContext envelope from Query
- Build AgentResources with permission filtering
- Invoke plugin runtime RUN_AGENT action
- Normalize AgentRunResult to Pipeline messages
- Handle errors, timeouts, protocol errors
- Maintain streaming card behavior
Entry points:
- run(event, binding): Main entry for event-first Protocol v1
- run_from_query(query): current Query entry adapter wrapper
"""
ap: app.Application
registry: AgentRunnerRegistry
context_builder: AgentRunContextBuilder
resource_builder: AgentResourceBuilder
result_normalizer: AgentResultNormalizer
binding_resolver: AgentBindingResolver
# Cached singleton references (set in __init__)
_session_registry: AgentRunSessionRegistry
_persistent_state_store: PersistentStateStore | None
def __init__(
self,
ap: app.Application,
registry: AgentRunnerRegistry,
):
self.ap = ap
self.registry = registry
self.context_builder = AgentRunContextBuilder(ap)
self.resource_builder = AgentResourceBuilder(ap)
self.result_normalizer = AgentResultNormalizer(ap)
self.binding_resolver = AgentBindingResolver()
# Cache singleton references to avoid per-request getter calls
self._session_registry = get_session_registry()
self._persistent_state_store = None # Lazy init on first use
async def run(
self,
event: AgentEventEnvelope,
binding: AgentBinding,
bound_plugins: list[str] | None = None,
adapter_context: dict[str, typing.Any] | None = None,
) -> typing.AsyncGenerator[provider_message.Message | provider_message.MessageChunk, None]:
"""Run agent runner from event-first envelope.
This is the main entry point for Protocol v1.
Event Gateway -> AgentBindingResolver -> run(event, binding).
Args:
event: Event envelope from event gateway
binding: Agent binding
bound_plugins: Optional list of bound plugin identities for authorization
adapter_context: Optional context from an entry adapter
Yields:
Message or MessageChunk for pipeline response
Raises:
RunnerNotFoundError: If runner not found
RunnerNotAuthorizedError: If runner not authorized
RunnerExecutionError: If runner execution failed
"""
runner_id = binding.runner_id
# Get runner descriptor
descriptor = await self.registry.get(runner_id, bound_plugins)
# Build resources from binding
resources = await self.resource_builder.build_resources_from_binding(
event=event,
binding=binding,
descriptor=descriptor,
)
# Build context from event + binding
context = await self.context_builder.build_context_from_event(
event=event,
binding=binding,
descriptor=descriptor,
resources=resources,
)
session_query_id = None
# Merge adapter context if provided
if adapter_context:
session_query_id = adapter_context.get('query_id')
# Merge params into adapter.extra
if 'params' in adapter_context:
context['adapter']['extra']['params'] = adapter_context['params']
# Build state context for State API handlers
state_context = build_state_context(event, binding, descriptor)
# Register session for proxy action permission validation
run_id = context['run_id']
await self._session_registry.register(
run_id=run_id,
runner_id=descriptor.id,
query_id=session_query_id,
plugin_identity=descriptor.get_plugin_id(),
resources=resources,
permissions=descriptor.permissions or {},
conversation_id=event.conversation_id,
state_policy={
'enable_state': binding.state_policy.enable_state,
'state_scopes': list(binding.state_policy.state_scopes),
},
state_context=state_context,
)
# Write incoming event to EventLog
event_log_id = await self._write_event_log(
event=event,
binding=binding,
run_id=run_id,
runner_id=descriptor.id,
)
# Register incoming attachments so input/transcript artifact_refs are resolvable.
await self._register_input_artifacts(
event=event,
run_id=run_id,
runner_id=descriptor.id,
)
# Write user message to Transcript if message.received
if event.event_type == 'message.received' and event.conversation_id:
await self._write_user_transcript(
event=event,
event_log_id=event_log_id,
)
# Track artifact refs for assistant transcript (cleared after each message.completed)
pending_artifact_refs: list[dict[str, typing.Any]] = []
try:
# Run via plugin connector
async for result_dict in self._invoke_runner(descriptor, context):
# Handle artifact.created first - consume before normalizer
if result_dict.get('type') == 'artifact.created':
artifact_ref = await self._handle_artifact_created(
result_dict=result_dict,
event=event,
run_id=run_id,
runner_id=descriptor.id,
)
pending_artifact_refs.append(artifact_ref)
# Pass to normalizer for logging, but don't yield to pipeline
await self.result_normalizer.normalize(result_dict, descriptor)
continue
# Handle state.updated first - consume before normalizer
if result_dict.get('type') == 'state.updated':
await self._handle_state_updated_event(result_dict, event, binding, descriptor)
# Pass to normalizer for logging, but don't yield to pipeline
await self.result_normalizer.normalize(result_dict, descriptor)
continue
# Handle message.completed - write to Transcript
if result_dict.get('type') == 'message.completed' and event.conversation_id:
# Merge pending artifact refs with message's own refs
merged_refs = self._merge_artifact_refs(
pending_artifact_refs,
result_dict,
)
# Clear pending refs after attaching to this message
pending_artifact_refs.clear()
await self._write_assistant_transcript(
result_dict=result_dict,
event=event,
run_id=run_id,
runner_id=descriptor.id,
artifact_refs=merged_refs if merged_refs else None,
)
# Normalize result for other types
result = await self.result_normalizer.normalize(result_dict, descriptor)
if result is not None:
yield result
finally:
# Unregister session after run completes (success or error)
await self._session_registry.unregister(run_id)
async def run_from_query(
self,
query: pipeline_query.Query,
) -> typing.AsyncGenerator[provider_message.Message | provider_message.MessageChunk, None]:
"""Run agent runner from pipeline query.
This is the Query entry adapter wrapper for the query-based flow.
It delegates to the event-first run(event, binding) method.
For the new event-first Protocol v1, use run(event, binding) instead.
Args:
query: Pipeline query with pipeline_config, session, messages, etc.
Yields:
Message or MessageChunk for pipeline response
Raises:
RunnerNotFoundError: If runner not found
RunnerNotAuthorizedError: If runner not authorized
RunnerExecutionError: If runner execution failed
"""
# Resolve runner ID using ConfigMigration
runner_id = ConfigMigration.resolve_runner_id(query.pipeline_config)
if not runner_id:
raise RunnerNotFoundError('no runner configured')
# Convert Query to event-first envelope
event = QueryEntryAdapter.query_to_event(query)
# Project legacy Pipeline config into target Agent config, then resolve
# exactly one effective binding for this event.
agent_config = QueryEntryAdapter.config_to_agent_config(query, runner_id)
binding = self.binding_resolver.resolve_one(event, [agent_config])
# Extract bound plugins for authorization
bound_plugins = query.variables.get('_pipeline_bound_plugins')
# Build adapter context for Query-specific fields
adapter_context = QueryEntryAdapter.build_adapter_context(query, binding)
# Delegate to event-first run()
async for result in self.run(
event,
binding,
bound_plugins=bound_plugins,
adapter_context=adapter_context,
):
yield result
async def _invoke_runner(
self,
descriptor: AgentRunnerDescriptor,
context: AgentRunContextPayload,
) -> typing.AsyncGenerator[dict[str, typing.Any], None]:
"""Invoke runner via plugin connector.
Args:
descriptor: Runner descriptor
context: AgentRunContext dict
Yields:
Raw result dicts from plugin runtime
Raises:
RunnerExecutionError: If plugin system disabled or runtime error
"""
if not self.ap.plugin_connector.is_enable_plugin:
raise RunnerExecutionError(
descriptor.id,
'Plugin system is disabled',
retryable=False,
)
try:
gen = self.ap.plugin_connector.run_agent(
plugin_author=descriptor.plugin_author,
plugin_name=descriptor.plugin_name,
runner_name=descriptor.runner_name,
context=context,
)
while True:
try:
result_dict = await self._next_with_deadline(gen, descriptor, context)
except StopAsyncIteration:
break
yield result_dict
except asyncio.TimeoutError as e:
raise RunnerExecutionError(
descriptor.id,
'Runner timed out (code: runner.timeout)',
retryable=True,
) from e
except ActionCallTimeoutError as e:
raise RunnerExecutionError(
descriptor.id,
f'{e} (code: runner.timeout)',
retryable=True,
) from e
except RunnerExecutionError:
raise
except Exception as e:
# Wrap unexpected errors
self.ap.logger.error(
f'Runner {descriptor.id} unexpected error: {traceback.format_exc()}'
)
raise RunnerExecutionError(
descriptor.id,
str(e),
retryable=False,
)
async def _next_with_deadline(
self,
gen: typing.AsyncGenerator[dict[str, typing.Any], None],
descriptor: AgentRunnerDescriptor,
context: AgentRunContextPayload,
) -> dict[str, typing.Any]:
"""Read the next runner result while enforcing the run deadline."""
remaining = self._remaining_deadline_seconds(context)
if remaining is not None and remaining <= 0:
await self._close_generator(gen, descriptor)
raise asyncio.TimeoutError
try:
if remaining is None:
return await anext(gen)
return await asyncio.wait_for(anext(gen), timeout=remaining)
except StopAsyncIteration:
if self._is_deadline_exhausted(context):
raise asyncio.TimeoutError
raise
except asyncio.TimeoutError:
await self._close_generator(gen, descriptor)
raise
def _remaining_deadline_seconds(
self,
context: AgentRunContextPayload,
) -> float | None:
runtime = context.get('runtime') or {}
deadline_at = runtime.get('deadline_at')
if deadline_at is None:
return None
try:
return float(deadline_at) - time.time()
except (TypeError, ValueError):
return None
def _is_deadline_exhausted(self, context: AgentRunContextPayload) -> bool:
remaining = self._remaining_deadline_seconds(context)
return remaining is not None and remaining <= 0
async def _close_generator(
self,
gen: typing.AsyncGenerator[dict[str, typing.Any], None],
descriptor: AgentRunnerDescriptor,
) -> None:
try:
await gen.aclose()
except Exception as e:
self.ap.logger.warning(f'Failed to close timed-out runner {descriptor.id}: {e}')
def resolve_runner_id_for_telemetry(self, query: pipeline_query.Query) -> str | None:
"""Resolve runner ID for telemetry/logging without full execution.
Args:
query: Pipeline query
Returns:
Runner ID string, or None
"""
return ConfigMigration.resolve_runner_id(query.pipeline_config)
async def _handle_state_updated_event(
self,
result_dict: dict[str, typing.Any],
event: AgentEventEnvelope,
binding: AgentBinding,
descriptor: AgentRunnerDescriptor,
) -> None:
"""Handle state.updated result in event-first mode.
Persists state to database via PersistentStateStore.
Args:
result_dict: Raw result dict with type='state.updated'
event: Event envelope
binding: Agent binding
descriptor: Runner descriptor
"""
data = result_dict.get('data', {})
scope = data.get('scope')
if not scope:
raise RunnerProtocolError(
descriptor.id,
'state.updated missing required field: scope',
)
# Extract key and value
key = data.get('key')
value = data.get('value')
if not key:
raise RunnerProtocolError(
descriptor.id,
'state.updated missing required field: key',
)
# Lazy init persistent state store
if self._persistent_state_store is None:
self._persistent_state_store = get_persistent_state_store(
self.ap.persistence_mgr.get_db_engine()
)
# Apply update to persistent state store
success, error = await self._persistent_state_store.apply_update_from_event(
event=event,
binding=binding,
descriptor=descriptor,
scope=scope,
key=key,
value=value,
logger=self.ap.logger,
)
if success:
self.ap.logger.debug(
f'Runner {descriptor.id} state.updated (event mode): scope={scope}, key={key}'
)
elif error:
self.ap.logger.warning(
f'Runner {descriptor.id} state.updated rejected: {error}'
)
async def _write_event_log(
self,
event: AgentEventEnvelope,
binding: AgentBinding,
run_id: str,
runner_id: str,
) -> str:
"""Write incoming event to EventLog.
Args:
event: Event envelope
binding: Agent binding
run_id: Run ID
runner_id: Runner ID
Returns:
Event log ID
"""
import datetime
from .event_log_store import EventLogStore
store = EventLogStore(self.ap.persistence_mgr.get_db_engine())
# Build input summary
input_summary = None
input_json = None
if event.input:
if event.input.text:
input_summary = event.input.text[:1000]
input_json = {
'text': event.input.text,
'contents': [c.model_dump(mode='json') if hasattr(c, 'model_dump') else c for c in event.input.contents],
'attachments': [a.model_dump(mode='json') if hasattr(a, 'model_dump') else a for a in event.input.attachments],
}
return await store.append_event(
event_id=event.event_id,
event_type=event.event_type,
source=event.source,
bot_id=event.bot_id,
workspace_id=event.workspace_id,
conversation_id=event.conversation_id,
thread_id=event.thread_id,
actor_type=event.actor.actor_type if event.actor else None,
actor_id=event.actor.actor_id if event.actor else None,
actor_name=event.actor.actor_name if event.actor else None,
subject_type=event.subject.subject_type if event.subject else None,
subject_id=event.subject.subject_id if event.subject else None,
input_summary=input_summary,
input_json=input_json,
run_id=run_id,
runner_id=runner_id,
event_time=datetime.datetime.fromtimestamp(event.event_time) if event.event_time else None,
)
async def _register_input_artifacts(
self,
event: AgentEventEnvelope,
run_id: str,
runner_id: str,
) -> None:
"""Register current-event attachments referenced by AgentInput."""
if not event.input or not event.input.attachments:
return
from .artifact_store import ArtifactStore
store = ArtifactStore(self.ap.persistence_mgr.get_db_engine())
for attachment in event.input.attachments:
data = attachment.model_dump(mode='json') if hasattr(attachment, 'model_dump') else attachment
if not isinstance(data, dict):
continue
artifact_id = data.get('artifact_id')
artifact_type = data.get('artifact_type') or 'file'
if not artifact_id:
continue
content, parsed_mime_type = self._decode_attachment_content(data.get('content'))
url = data.get('url')
platform_ref_id = data.get('id')
storage_key = None
storage_type = 'metadata_only'
if content is None:
if url:
storage_key = url
storage_type = 'url'
elif platform_ref_id:
storage_key = platform_ref_id
storage_type = 'platform_ref'
metadata = {
'input_attachment': True,
'input_source': data.get('source') or 'platform',
}
if url:
metadata['url'] = url
if platform_ref_id:
metadata['platform_ref_id'] = platform_ref_id
try:
await store.register_artifact(
artifact_id=artifact_id,
artifact_type=artifact_type,
source='platform',
storage_key=storage_key,
storage_type=storage_type,
mime_type=data.get('mime_type') or parsed_mime_type,
name=data.get('name'),
size_bytes=data.get('size') or (len(content) if content is not None else None),
conversation_id=event.conversation_id,
run_id=run_id,
runner_id=runner_id,
bot_id=event.bot_id,
workspace_id=event.workspace_id,
metadata=metadata,
content=content,
)
except Exception as e:
self.ap.logger.warning(
f'Failed to register input artifact {artifact_id}: {e}'
)
def _decode_attachment_content(
self,
content: typing.Any,
) -> tuple[bytes | None, str | None]:
"""Decode base64 attachment content, including data URLs."""
if not isinstance(content, str) or not content:
return None, None
import base64
import binascii
mime_type = None
payload = content
if content.startswith('data:') and ',' in content:
header, payload = content.split(',', 1)
if ';base64' in header:
mime_type = header[5:].split(';', 1)[0] or None
try:
return base64.b64decode(payload, validate=False), mime_type
except (binascii.Error, ValueError):
return None, mime_type
async def _write_user_transcript(
self,
event: AgentEventEnvelope,
event_log_id: str,
) -> None:
"""Write user message to Transcript.
Args:
event: Event envelope
event_log_id: Event log ID
"""
from .transcript_store import TranscriptStore
store = TranscriptStore(self.ap.persistence_mgr.get_db_engine())
# Build content
content = event.input.text if event.input else None
content_json = None
if event.input:
content_json = {
'role': 'user',
'content': [c.model_dump(mode='json') if hasattr(c, 'model_dump') else c for c in event.input.contents] if event.input.contents else [],
}
# Build artifact refs
artifact_refs = []
if event.input and event.input.attachments:
for a in event.input.attachments:
artifact_refs.append(a.model_dump(mode='json') if hasattr(a, 'model_dump') else a)
await store.append_transcript(
transcript_id=None, # Auto-generate
event_id=event_log_id,
conversation_id=event.conversation_id,
role='user',
content=content,
content_json=content_json,
artifact_refs=artifact_refs if artifact_refs else None,
thread_id=event.thread_id,
item_type='message',
metadata={
'actor_type': event.actor.actor_type if event.actor else None,
'actor_id': event.actor.actor_id if event.actor else None,
},
)
async def _handle_artifact_created(
self,
result_dict: dict[str, typing.Any],
event: AgentEventEnvelope,
run_id: str,
runner_id: str,
) -> dict[str, typing.Any]:
"""Handle artifact.created result - register artifact and write EventLog.
Args:
result_dict: Raw result dict with type='artifact.created'
event: Event envelope
run_id: Current run ID
runner_id: Runner ID
Returns:
Artifact reference dict for Transcript
Raises:
RunnerProtocolError: On validation failures or registration errors
"""
import base64
import uuid
from .artifact_store import ArtifactStore
from .event_log_store import EventLogStore
data = result_dict.get('data', {})
# Validate run_id matches current context
result_run_id = result_dict.get('run_id')
if result_run_id and result_run_id != run_id:
raise RunnerProtocolError(
runner_id,
f'artifact.created run_id mismatch: expected {run_id}, got {result_run_id}',
)
# Extract artifact fields
artifact_id = data.get('artifact_id') or str(uuid.uuid4())
artifact_type = data.get('artifact_type')
if not artifact_type:
raise RunnerProtocolError(
runner_id,
'artifact.created missing required field: artifact_type',
)
mime_type = data.get('mime_type')
name = data.get('name')
size_bytes = data.get('size_bytes')
sha256 = data.get('sha256')
metadata = data.get('metadata')
content_base64 = data.get('content_base64')
# Decode and validate content if provided
content: bytes | None = None
if content_base64:
try:
content = base64.b64decode(content_base64, validate=True)
except Exception as e:
raise RunnerProtocolError(
runner_id,
f'artifact.created invalid base64 content: {e}',
)
# Validate content size
if len(content) > MAX_ARTIFACT_INLINE_BYTES:
raise RunnerProtocolError(
runner_id,
f'artifact.created content size {len(content)} bytes exceeds limit {MAX_ARTIFACT_INLINE_BYTES} bytes',
)
# Register artifact via ArtifactStore
artifact_store = ArtifactStore(self.ap.persistence_mgr.get_db_engine())
try:
registered_id = await artifact_store.register_artifact(
artifact_id=artifact_id,
artifact_type=artifact_type,
source='runner',
mime_type=mime_type,
name=name,
size_bytes=size_bytes,
sha256=sha256,
conversation_id=event.conversation_id,
run_id=run_id,
runner_id=runner_id,
bot_id=event.bot_id,
workspace_id=event.workspace_id,
metadata=metadata,
content=content,
)
except Exception as e:
raise RunnerProtocolError(
runner_id,
f'artifact.created failed to register artifact: {e}',
)
# Write to EventLog
event_log_store = EventLogStore(self.ap.persistence_mgr.get_db_engine())
await event_log_store.append_event(
event_id=str(uuid.uuid4()),
event_type='artifact.created',
source='runner',
bot_id=event.bot_id,
workspace_id=event.workspace_id,
conversation_id=event.conversation_id,
thread_id=event.thread_id,
actor_type=event.actor.actor_type if event.actor else None,
actor_id=event.actor.actor_id if event.actor else None,
actor_name=event.actor.actor_name if event.actor else None,
input_summary=f'Artifact created: {artifact_type}',
input_json={
'artifact_id': registered_id,
'artifact_type': artifact_type,
'mime_type': mime_type,
'name': name,
'size_bytes': size_bytes,
},
run_id=run_id,
runner_id=runner_id,
)
# Return artifact ref for Transcript
return {
'artifact_id': registered_id,
'artifact_type': artifact_type,
'mime_type': mime_type,
'name': name,
}
def _merge_artifact_refs(
self,
pending_refs: list[dict[str, typing.Any]],
result_dict: dict[str, typing.Any],
) -> list[dict[str, typing.Any]]:
"""Merge pending artifact refs with message's own refs, deduplicating by artifact_id.
Args:
pending_refs: Artifact refs accumulated from artifact.created events
result_dict: Result dict that may contain message with artifact_refs
Returns:
Merged and deduplicated list of artifact refs
"""
# Start with pending refs
merged = list(pending_refs)
seen_ids = {ref.get('artifact_id') for ref in pending_refs if ref.get('artifact_id')}
# Extract refs from message data if present
data = result_dict.get('data', {})
message = data.get('message', {})
message_refs = message.get('artifact_refs', [])
if isinstance(message_refs, list):
for ref in message_refs:
if isinstance(ref, dict):
artifact_id = ref.get('artifact_id')
if artifact_id and artifact_id not in seen_ids:
merged.append(ref)
seen_ids.add(artifact_id)
return merged
async def _write_assistant_transcript(
self,
result_dict: dict[str, typing.Any],
event: AgentEventEnvelope,
run_id: str,
runner_id: str,
artifact_refs: list[dict[str, typing.Any]] | None = None,
) -> None:
"""Write assistant message to Transcript.
Args:
result_dict: Result dict from runner
event: Original event envelope
run_id: Run ID
runner_id: Runner ID
artifact_refs: Optional artifact references to include
"""
import uuid
from .transcript_store import TranscriptStore
store = TranscriptStore(self.ap.persistence_mgr.get_db_engine())
data = result_dict.get('data', {})
message = data.get('message', {})
# Build content
content = None
content_json = None
if isinstance(message.get('content'), str):
content = message['content']
content_json = message
elif isinstance(message.get('content'), list):
# Extract text from content list
text_parts = []
for c in message['content']:
if isinstance(c, dict) and c.get('type') == 'text':
text_parts.append(c.get('text', ''))
content = ' '.join(text_parts) if text_parts else None
content_json = message
# Generate a unique event ID for assistant message
assistant_event_id = str(uuid.uuid4())
await store.append_transcript(
transcript_id=str(uuid.uuid4()),
event_id=assistant_event_id,
conversation_id=event.conversation_id,
role='assistant',
content=content,
content_json=content_json,
artifact_refs=artifact_refs,
thread_id=event.thread_id,
item_type='message',
run_id=run_id,
runner_id=runner_id,
metadata={
'run_id': run_id,
'runner_id': runner_id,
},
)

View File

@@ -1,431 +0,0 @@
"""Persistent state store for AgentRunner protocol state.
This module provides a database-backed state store for event-first Protocol v1.
"""
from __future__ import annotations
import typing
import json
import threading
from datetime import datetime
import sqlalchemy
from sqlalchemy.ext.asyncio import AsyncEngine
from sqlalchemy import select, delete, update
from .descriptor import AgentRunnerDescriptor
from .host_models import AgentEventEnvelope, AgentBinding
from .state_scope import (
VALID_STATE_SCOPES,
build_state_scope_key,
get_binding_identity,
normalize_state_key,
)
from ...entity.persistence.agent_runner_state import AgentRunnerState
# Maximum value_json size (256KB)
MAX_VALUE_JSON_BYTES = 256 * 1024
class PersistentStateStore:
"""Database-backed state store for AgentRunner protocol state.
IMPORTANT: This is HOST-OWNED protocol state, NOT plugin instance state.
This store provides:
1. Persistent storage across runs via database
2. Scope isolation by runner_id + binding_identity + scope
3. Policy enforcement (enable_state, state_scopes)
4. JSON value validation and size limits
Used by:
- Event-first Protocol v1 (async methods)
- State API handlers (get/set/delete/list)
"""
def __init__(self, db_engine: AsyncEngine):
self._db_engine = db_engine
def _get_scope_key(
self,
scope: str,
event: AgentEventEnvelope,
binding: AgentBinding,
descriptor: AgentRunnerDescriptor,
) -> str | None:
"""Get scope key for given scope."""
return build_state_scope_key(scope, event, binding, descriptor)
def _check_scope_enabled(self, scope: str, binding: AgentBinding) -> bool:
"""Check if scope is enabled by binding's state_policy."""
state_policy = binding.state_policy
if not state_policy.enable_state:
return False
return scope in state_policy.state_scopes
def _validate_json_value(
self,
value: typing.Any,
logger: typing.Any = None,
) -> tuple[str | None, str | None]:
"""Validate and serialize value to JSON.
Returns:
Tuple of (json_string, error_message). If error_message is not None,
json_string will be None.
"""
try:
json_str = json.dumps(value, ensure_ascii=False)
except (TypeError, ValueError) as e:
return None, f'Value is not JSON-serializable: {e}'
# Check size limit
json_bytes = len(json_str.encode('utf-8'))
if json_bytes > MAX_VALUE_JSON_BYTES:
return None, f'Value size {json_bytes} bytes exceeds limit {MAX_VALUE_JSON_BYTES} bytes'
return json_str, None
# ========== Async DB Operations ==========
async def build_snapshot_from_event(
self,
event: AgentEventEnvelope,
binding: AgentBinding,
descriptor: AgentRunnerDescriptor,
) -> dict[str, dict[str, typing.Any]]:
"""Build state snapshot for all scopes from event and binding.
Reads from database, respects state_policy.
"""
state_policy = binding.state_policy
# If state is disabled, return all empty scopes
if not state_policy.enable_state:
return {
'conversation': {},
'actor': {},
'subject': {},
'runner': {},
}
snapshot: dict[str, dict[str, typing.Any]] = {
'conversation': {},
'actor': {},
'subject': {},
'runner': {},
}
async with self._db_engine.connect() as conn:
for scope in VALID_STATE_SCOPES:
if not self._check_scope_enabled(scope, binding):
continue
scope_key = self._get_scope_key(scope, event, binding, descriptor)
if not scope_key:
continue
# Query all state entries for this scope_key
result = await conn.execute(
select(AgentRunnerState.state_key, AgentRunnerState.value_json)
.where(AgentRunnerState.scope_key == scope_key)
)
rows = result.fetchall()
for row in rows:
key = row.state_key
value_json = row.value_json
if value_json:
try:
snapshot[scope][key] = json.loads(value_json)
except json.JSONDecodeError:
pass # Skip invalid JSON
# Seed external.conversation_id from event.conversation_id if not set
if self._check_scope_enabled('conversation', binding) and event.conversation_id:
if 'external.conversation_id' not in snapshot['conversation']:
snapshot['conversation']['external.conversation_id'] = event.conversation_id
return snapshot
async def apply_update_from_event(
self,
event: AgentEventEnvelope,
binding: AgentBinding,
descriptor: AgentRunnerDescriptor,
scope: str,
key: str,
value: typing.Any,
logger: typing.Any = None,
) -> tuple[bool, str | None]:
"""Apply a state update from event context.
Returns:
Tuple of (success, error_message). If success is False, error_message
contains the reason.
"""
state_policy = binding.state_policy
# Check if state is disabled
if not state_policy.enable_state:
return False, 'State is disabled by binding policy'
# Validate scope
if scope not in VALID_STATE_SCOPES:
return False, f'Invalid scope: {scope}'
# Check if scope is enabled
if not self._check_scope_enabled(scope, binding):
return False, f'Scope "{scope}" not enabled by binding policy'
# Map accepted key aliases
key = normalize_state_key(key)
# Get scope key
scope_key = self._get_scope_key(scope, event, binding, descriptor)
if not scope_key:
return False, f'Missing identity for scope "{scope}"'
# Validate and serialize value
value_json, error = self._validate_json_value(value, logger)
if error:
return False, error
# Build context fields
binding_identity = get_binding_identity(binding)
async with self._db_engine.begin() as conn:
# Check if entry exists
result = await conn.execute(
select(AgentRunnerState.id)
.where(AgentRunnerState.scope_key == scope_key)
.where(AgentRunnerState.state_key == key)
)
existing = result.first()
now = datetime.utcnow()
if existing:
# Update existing entry
await conn.execute(
update(AgentRunnerState)
.where(AgentRunnerState.id == existing.id)
.values(
value_json=value_json,
updated_at=now,
)
)
else:
# Insert new entry
await conn.execute(
sqlalchemy.insert(AgentRunnerState).values(
runner_id=descriptor.id,
binding_identity=binding_identity,
scope=scope,
scope_key=scope_key,
state_key=key,
value_json=value_json,
bot_id=event.bot_id,
workspace_id=event.workspace_id,
conversation_id=event.conversation_id,
thread_id=event.thread_id,
actor_type=event.actor.actor_type if event.actor else None,
actor_id=event.actor.actor_id if event.actor else None,
subject_type=event.subject.subject_type if event.subject else None,
subject_id=event.subject.subject_id if event.subject else None,
created_at=now,
updated_at=now,
)
)
return True, None
async def state_get(
self,
scope_key: str,
state_key: str,
) -> typing.Any:
"""Get a single state value by scope_key and state_key.
Used by State API handlers.
"""
state_key = normalize_state_key(state_key)
async with self._db_engine.connect() as conn:
result = await conn.execute(
select(AgentRunnerState.value_json)
.where(AgentRunnerState.scope_key == scope_key)
.where(AgentRunnerState.state_key == state_key)
)
row = result.first()
if not row or not row.value_json:
return None
try:
return json.loads(row.value_json)
except json.JSONDecodeError:
return None
async def state_set(
self,
scope_key: str,
state_key: str,
value: typing.Any,
runner_id: str,
binding_identity: str,
scope: str,
context: dict[str, typing.Any] | None = None,
logger: typing.Any = None,
) -> tuple[bool, str | None]:
"""Set a state value.
Used by State API handlers.
Context contains optional fields like bot_id, conversation_id, etc.
"""
state_key = normalize_state_key(state_key)
# Validate and serialize value
value_json, error = self._validate_json_value(value, logger)
if error:
return False, error
context = context or {}
async with self._db_engine.begin() as conn:
# Check if entry exists
result = await conn.execute(
select(AgentRunnerState.id)
.where(AgentRunnerState.scope_key == scope_key)
.where(AgentRunnerState.state_key == state_key)
)
existing = result.first()
now = datetime.utcnow()
if existing:
# Update existing entry
await conn.execute(
update(AgentRunnerState)
.where(AgentRunnerState.id == existing.id)
.values(
value_json=value_json,
updated_at=now,
)
)
else:
# Insert new entry
await conn.execute(
sqlalchemy.insert(AgentRunnerState).values(
runner_id=runner_id,
binding_identity=binding_identity,
scope=scope,
scope_key=scope_key,
state_key=state_key,
value_json=value_json,
bot_id=context.get('bot_id'),
workspace_id=context.get('workspace_id'),
conversation_id=context.get('conversation_id'),
thread_id=context.get('thread_id'),
actor_type=context.get('actor_type'),
actor_id=context.get('actor_id'),
subject_type=context.get('subject_type'),
subject_id=context.get('subject_id'),
created_at=now,
updated_at=now,
)
)
return True, None
async def state_delete(
self,
scope_key: str,
state_key: str,
) -> bool:
"""Delete a state value.
Returns True if deleted, False if not found.
"""
state_key = normalize_state_key(state_key)
async with self._db_engine.begin() as conn:
result = await conn.execute(
delete(AgentRunnerState)
.where(AgentRunnerState.scope_key == scope_key)
.where(AgentRunnerState.state_key == state_key)
.returning(AgentRunnerState.id)
)
deleted = result.first()
return deleted is not None
async def state_list(
self,
scope_key: str,
prefix: str | None = None,
limit: int = 100,
) -> tuple[list[str], bool]:
"""List state keys in a scope.
Returns tuple of (keys, has_more).
"""
# Enforce limit cap
limit = min(limit, 100)
async with self._db_engine.connect() as conn:
query = (
select(AgentRunnerState.state_key)
.where(AgentRunnerState.scope_key == scope_key)
.order_by(AgentRunnerState.state_key)
.limit(limit + 1) # Fetch one extra to check has_more
)
if prefix:
prefix = normalize_state_key(prefix)
query = query.where(
AgentRunnerState.state_key.like(f'{prefix}%')
)
result = await conn.execute(query)
rows = result.fetchall()
keys = [row.state_key for row in rows[:limit]]
has_more = len(rows) > limit
return keys, has_more
async def clear_all(self) -> None:
"""Clear all state entries (for testing)."""
async with self._db_engine.begin() as conn:
await conn.execute(delete(AgentRunnerState))
# Global singleton persistent state store
_persistent_state_store: PersistentStateStore | None = None
_persistent_state_store_lock = threading.Lock()
def get_persistent_state_store(db_engine: AsyncEngine | None = None) -> PersistentStateStore:
"""Get the global persistent state store singleton.
Args:
db_engine: Database engine (required on first call)
Returns:
PersistentStateStore singleton
"""
global _persistent_state_store
with _persistent_state_store_lock:
if _persistent_state_store is None:
if db_engine is None:
raise RuntimeError("db_engine required for first call to get_persistent_state_store")
_persistent_state_store = PersistentStateStore(db_engine)
return _persistent_state_store
def reset_persistent_state_store() -> None:
"""Reset the global persistent state store (for testing)."""
global _persistent_state_store
with _persistent_state_store_lock:
_persistent_state_store = None

View File

@@ -1,583 +0,0 @@
"""Query entry adapter for converting Query to event-first envelope.
This adapter bridges the current Query entry point with the event-first
Protocol v1 architecture without exposing Query internals to runners.
"""
from __future__ import annotations
import hashlib
import typing
from langbot_plugin.api.entities.builtin.pipeline import query as pipeline_query
from langbot_plugin.api.entities.builtin.platform import message as platform_message
from langbot_plugin.api.entities.builtin.agent_runner.event import (
AgentEventContext,
ConversationContext,
ActorContext,
SubjectContext,
RawEventRef,
)
from langbot_plugin.api.entities.builtin.agent_runner.input import AgentInput
from langbot_plugin.api.entities.builtin.agent_runner.delivery import DeliveryContext
from .host_models import (
AgentConfig,
AgentEventEnvelope,
ResourcePolicy,
StatePolicy,
DeliveryPolicy,
)
from . import events as runner_events
class QueryEntryAdapter:
"""Adapter for converting Query to event-first envelope.
This adapter is responsible for:
- Converting Query to AgentEventEnvelope
- Projecting current Pipeline config to temporary AgentConfig
- Putting Query-only fields into adapter context
"""
INTERNAL_PREFIX = '_'
SENSITIVE_PATTERNS = ('secret', 'token', 'key', 'password', 'credential', 'api_key', 'apikey')
PERMISSION_VARS = ('_pipeline_bound_plugins', '_authorized', '_permission')
@classmethod
def query_to_event(
cls,
query: pipeline_query.Query,
) -> AgentEventEnvelope:
"""Convert Query to AgentEventEnvelope.
Args:
query: Current entry query
Returns:
AgentEventEnvelope for event-first processing
"""
# Build event context
event = cls._build_event_context(query)
# Build conversation context
conversation = cls._build_conversation_context(query)
# Build actor context
actor = cls._build_actor_context(query)
# Build subject context
subject = cls._build_subject_context(query)
# Build input
input = cls._build_input(query)
# Build delivery context
delivery = cls._build_delivery_context(query)
# Build raw ref
raw_ref = cls._build_raw_ref(query)
return AgentEventEnvelope(
event_id=event.event_id or str(query.query_id),
event_type=event.event_type or runner_events.MESSAGE_RECEIVED,
event_time=event.event_time,
source="host_adapter",
source_event_type=event.source_event_type,
bot_id=query.bot_uuid,
workspace_id=None, # Not available in Query
conversation_id=conversation.conversation_id,
thread_id=conversation.thread_id,
actor=actor,
subject=subject,
input=input,
delivery=delivery,
raw_ref=raw_ref,
data=event.data,
)
@classmethod
def config_to_agent_config(
cls,
query: pipeline_query.Query,
runner_id: str,
) -> AgentConfig:
"""Project the current Pipeline config container into target Agent config."""
pipeline_config = query.pipeline_config or {}
ai_config = pipeline_config.get('ai', {})
runner_config = ai_config.get('runner_config', {}).get(runner_id, {})
agent_id = getattr(query, 'pipeline_uuid', None)
# Build resource policy from current config
resource_policy = ResourcePolicy(
allowed_model_uuids=cls._extract_allowed_models(query),
allowed_tool_names=cls._extract_allowed_tools(query),
allowed_kb_uuids=cls._extract_allowed_kbs(query),
)
# Build state policy
state_policy = StatePolicy(
enable_state=True,
state_scopes=["conversation", "actor", "subject", "runner"],
)
# Build delivery policy
delivery_policy = DeliveryPolicy(
enable_streaming=True,
enable_reply=True,
)
return AgentConfig(
agent_id=agent_id,
runner_id=runner_id,
runner_config=runner_config,
resource_policy=resource_policy,
state_policy=state_policy,
delivery_policy=delivery_policy,
event_types=[runner_events.MESSAGE_RECEIVED],
enabled=True,
metadata={'source': 'pipeline_adapter'},
)
@classmethod
def build_adapter_context(
cls,
query: pipeline_query.Query,
binding: AgentBinding,
) -> dict[str, typing.Any]:
"""Build Query-derived fields for the current entry adapter."""
return {
'params': cls.build_params(query),
'query_id': getattr(query, 'query_id', None),
}
@classmethod
def build_params(cls, query: pipeline_query.Query) -> dict[str, typing.Any]:
"""Build adapter params from Pipeline variables with host filtering."""
params: dict[str, typing.Any] = {}
variables = getattr(query, 'variables', None)
if not variables:
return params
for key, value in variables.items():
if key.startswith(cls.INTERNAL_PREFIX):
continue
key_lower = key.lower()
if any(pattern in key_lower for pattern in cls.SENSITIVE_PATTERNS):
continue
if any(key == perm_var or key.startswith(perm_var) for perm_var in cls.PERMISSION_VARS):
continue
if cls.is_json_serializable(value):
params[key] = value
return params
@classmethod
def is_json_serializable(cls, value: typing.Any) -> bool:
"""Return whether a value can safely cross the adapter boundary as JSON."""
if value is None or isinstance(value, (str, int, float, bool)):
return True
if isinstance(value, (list, tuple)):
return all(cls.is_json_serializable(item) for item in value)
if isinstance(value, dict):
return all(
isinstance(k, str) and cls.is_json_serializable(v)
for k, v in value.items()
)
return False
# Private helper methods
@classmethod
def _build_event_context(
cls,
query: pipeline_query.Query,
) -> AgentEventContext:
"""Build AgentEventContext from Query."""
message_event = getattr(query, 'message_event', None)
event_data: dict[str, typing.Any] = {}
if message_event and hasattr(message_event, 'model_dump'):
try:
event_data = message_event.model_dump(mode='json')
except TypeError:
event_data = message_event.model_dump()
except Exception:
event_data = {}
event_data.pop('source_platform_object', None)
source_event_type = None
if message_event:
source_event_type = getattr(message_event, 'type', None)
message_chain = getattr(query, 'message_chain', None)
message_id = getattr(message_chain, 'message_id', None)
if message_id == -1:
message_id = None
event_time = None
if message_event:
event_time = getattr(message_event, 'time', None)
if isinstance(event_time, (int, float)):
event_time = int(event_time)
source_event_id = str(message_id or query.query_id)
return AgentEventContext(
event_id=cls._build_scoped_event_id(query, source_event_id, event_time),
event_type=runner_events.MESSAGE_RECEIVED,
event_time=event_time,
source="host_adapter",
source_event_type=source_event_type,
data=event_data,
)
@classmethod
def _build_scoped_event_id(
cls,
query: pipeline_query.Query,
source_event_id: str,
event_time: int | None,
) -> str:
"""Build a globally unique host event id from pipeline-local ids."""
launcher_type = getattr(query, 'launcher_type', None)
launcher_type_value = getattr(launcher_type, 'value', launcher_type) if launcher_type is not None else None
scope_parts = [
'host_adapter',
getattr(query, 'pipeline_uuid', None),
getattr(query, 'bot_uuid', None),
launcher_type_value,
getattr(query, 'launcher_id', None),
getattr(query, 'sender_id', None),
source_event_id,
event_time,
]
scoped = '|'.join('' if part is None else str(part) for part in scope_parts)
digest = hashlib.sha256(scoped.encode('utf-8')).hexdigest()[:32]
return f'host:{digest}'
@classmethod
def _build_conversation_context(
cls,
query: pipeline_query.Query,
) -> ConversationContext:
"""Build ConversationContext from Query."""
# Handle launcher_type safely
launcher_type = getattr(query, 'launcher_type', None)
launcher_type_value = None
if launcher_type is not None:
launcher_type_value = getattr(launcher_type, 'value', launcher_type)
# Handle launcher_id
launcher_id = getattr(query, 'launcher_id', None)
# Build session_id from launcher info if available
session_id = None
if launcher_type_value and launcher_id:
session_id = f'{launcher_type_value}_{launcher_id}'
# Handle session and conversation_id
conversation_id = None
session = getattr(query, 'session', None)
if session:
conversation = getattr(session, 'using_conversation', None)
if conversation:
conversation_id = getattr(conversation, 'uuid', None)
if not conversation_id:
variables = getattr(query, 'variables', None) or {}
conversation_id = variables.get('conversation_id') or None
if not conversation_id:
conversation_id = session_id
# Handle sender_id
sender_id = getattr(query, 'sender_id', None)
if sender_id is not None:
sender_id = str(sender_id)
# Handle bot_uuid
bot_uuid = getattr(query, 'bot_uuid', None)
return ConversationContext(
conversation_id=str(conversation_id) if conversation_id is not None else None,
thread_id=None,
launcher_type=launcher_type_value,
launcher_id=launcher_id,
sender_id=sender_id,
bot_id=bot_uuid,
workspace_id=None,
session_id=session_id,
)
@classmethod
def _build_actor_context(
cls,
query: pipeline_query.Query,
) -> ActorContext:
"""Build ActorContext from Query."""
message_event = getattr(query, 'message_event', None)
sender = getattr(message_event, 'sender', None) if message_event else None
sender_id = getattr(query, 'sender_id', None)
actor_id = getattr(sender, 'id', None) if sender else None
if actor_id is None:
actor_id = sender_id
actor_name = sender.get_name() if sender and hasattr(sender, 'get_name') else None
return ActorContext(
actor_type="user",
actor_id=str(actor_id) if actor_id is not None else None,
actor_name=actor_name,
metadata={},
)
@classmethod
def _build_subject_context(
cls,
query: pipeline_query.Query,
) -> SubjectContext:
"""Build SubjectContext from Query."""
message_chain = getattr(query, 'message_chain', None)
message_id = getattr(message_chain, 'message_id', None) if message_chain else None
if message_id == -1:
message_id = None
query_id = getattr(query, 'query_id', None)
# Safely get launcher_type
launcher_type = getattr(query, 'launcher_type', None)
launcher_type_value = None
if launcher_type is not None:
launcher_type_value = getattr(launcher_type, 'value', launcher_type)
return SubjectContext(
subject_type="message",
subject_id=str(message_id or query_id or ''),
data={
"launcher_type": launcher_type_value,
"launcher_id": getattr(query, 'launcher_id', None),
"sender_id": str(getattr(query, 'sender_id', '')) if getattr(query, 'sender_id', None) else None,
"bot_uuid": getattr(query, 'bot_uuid', None),
},
)
@classmethod
def _build_input(
cls,
query: pipeline_query.Query,
) -> AgentInput:
"""Build AgentInput from Query."""
text = None
text_parts: list[str] = []
contents: list[dict[str, typing.Any]] = []
user_message = getattr(query, 'user_message', None)
if user_message:
content = getattr(user_message, 'content', None)
if isinstance(content, list):
for elem in content:
# Handle both real objects and mocks
if hasattr(elem, 'model_dump'):
contents.append(elem.model_dump(mode='json'))
elif isinstance(elem, dict):
contents.append(elem)
else:
# For mocks, extract type and text attributes
elem_type = getattr(elem, 'type', None)
if elem_type == 'text':
elem_text = getattr(elem, 'text', None)
contents.append({'type': 'text', 'text': elem_text})
if elem_text:
text_parts.append(elem_text)
continue
# Extract text for the text field
if hasattr(elem, 'type') and getattr(elem, 'type', None) == 'text':
elem_text = getattr(elem, 'text', None)
if elem_text:
text_parts.append(elem_text)
elif content is not None:
text = str(content)
contents.append({'type': 'text', 'text': text})
if text_parts:
text = ''.join(text_parts)
message_chain_dict = None
message_chain = getattr(query, 'message_chain', None)
if message_chain:
if hasattr(message_chain, 'model_dump'):
message_chain_dict = message_chain.model_dump(mode='json')
attachments = cls._build_attachments(query, contents)
return AgentInput(
text=text,
contents=contents,
message_chain=message_chain_dict,
attachments=attachments,
)
@classmethod
def _build_attachments(
cls,
query: pipeline_query.Query,
contents: list[dict[str, typing.Any]],
) -> list[dict[str, typing.Any]]:
"""Extract attachments from query."""
import uuid
attachments: list[dict[str, typing.Any]] = []
for elem in contents:
elem_type = elem.get('type')
artifact_id = str(uuid.uuid4()) # Generate unique ID
if elem_type == 'image_url':
image_url = elem.get('image_url') or {}
attachments.append({
'artifact_id': artifact_id,
'artifact_type': 'image',
'source': 'url',
'url': image_url.get('url') if isinstance(image_url, dict) else str(image_url),
})
elif elem_type == 'image_base64':
attachments.append({
'artifact_id': artifact_id,
'artifact_type': 'image',
'source': 'base64',
'content': elem.get('image_base64'),
})
elif elem_type == 'file_url':
attachments.append({
'artifact_id': artifact_id,
'artifact_type': 'file',
'source': 'url',
'url': elem.get('file_url'),
'name': elem.get('file_name'),
})
elif elem_type == 'file_base64':
attachments.append({
'artifact_id': artifact_id,
'artifact_type': 'file',
'source': 'base64',
'content': elem.get('file_base64'),
'name': elem.get('file_name'),
})
message_chain = getattr(query, 'message_chain', None)
if message_chain:
try:
for component in message_chain:
artifact_id = str(uuid.uuid4()) # Generate unique ID
if isinstance(component, platform_message.Image):
attachments.append({
'artifact_id': artifact_id,
'artifact_type': 'image',
'source': 'message_chain',
'id': component.image_id or None,
'url': component.url or None,
})
elif isinstance(component, platform_message.File):
attachments.append({
'artifact_id': artifact_id,
'artifact_type': 'file',
'source': 'message_chain',
'id': component.id or None,
'name': component.name or None,
})
elif isinstance(component, platform_message.Voice):
attachments.append({
'artifact_id': artifact_id,
'artifact_type': 'voice',
'source': 'message_chain',
'id': component.voice_id or None,
'url': component.url or None,
})
except TypeError:
# message_chain is not iterable (e.g., a Mock object)
pass
return attachments
@classmethod
def _build_delivery_context(
cls,
query: pipeline_query.Query,
) -> DeliveryContext:
"""Build DeliveryContext from Query."""
message_chain = getattr(query, 'message_chain', None)
return DeliveryContext(
surface="platform",
reply_target={
"message_id": getattr(message_chain, 'message_id', None),
},
supports_streaming=True,
supports_edit=False,
supports_reaction=False,
platform_capabilities={},
)
@classmethod
def _build_raw_ref(
cls,
query: pipeline_query.Query,
) -> RawEventRef | None:
"""Build RawEventRef from Query."""
# For now, we don't store raw event payload
return None
@classmethod
def _extract_allowed_models(
cls,
query: pipeline_query.Query,
) -> list[str] | None:
"""Extract allowed model UUIDs from query."""
model_uuids: list[str] = []
model_uuid = getattr(query, 'use_llm_model_uuid', None)
if model_uuid:
model_uuids.append(model_uuid)
variables = getattr(query, 'variables', None) or {}
for fallback_uuid in variables.get('_fallback_model_uuids', []) or []:
if fallback_uuid and fallback_uuid not in model_uuids:
model_uuids.append(fallback_uuid)
return model_uuids or None
@classmethod
def _extract_allowed_tools(
cls,
query: pipeline_query.Query,
) -> list[str] | None:
"""Extract allowed tool names from query."""
use_funcs = getattr(query, 'use_funcs', None)
if not use_funcs:
return None
try:
tool_names = []
for func in use_funcs:
if isinstance(func, dict):
name = func.get('name')
elif hasattr(func, 'name'):
name = func.name
else:
continue
if name:
tool_names.append(name)
return tool_names if tool_names else None
except (TypeError, AttributeError):
return None
@classmethod
def _extract_allowed_kbs(
cls,
query: pipeline_query.Query,
) -> list[str] | None:
"""Extract allowed knowledge base UUIDs from query."""
variables = getattr(query, 'variables', None)
if not variables:
return None
kb_uuids = variables.get('_knowledge_base_uuids')
if kb_uuids:
return kb_uuids
return None

View File

@@ -1,293 +0,0 @@
"""Agent runner registry for discovering and caching runner descriptors."""
from __future__ import annotations
import typing
import asyncio
from ...core import app
from .descriptor import AgentRunnerDescriptor
from .id import parse_runner_id, format_runner_id
from .errors import RunnerNotFoundError, RunnerNotAuthorizedError
class AgentRunnerRegistry:
"""Registry for discovering and managing agent runners.
Responsibilities:
- Discover runners from plugin runtime via LIST_AGENT_RUNNERS
- Validate runner manifests (kind, metadata, spec)
- Cache discovered runners for performance
- Filter runners by bound plugins
- Handle manifest errors gracefully (log warning, skip runner)
"""
ap: app.Application
_cache: dict[str, AgentRunnerDescriptor] | None
"""Cached runner descriptors keyed by runner ID"""
_cache_lock: asyncio.Lock
"""Lock for cache refresh operations"""
def __init__(self, ap: app.Application):
self.ap = ap
self._cache = None
self._cache_lock = asyncio.Lock()
async def _discover_runners(self) -> dict[str, AgentRunnerDescriptor]:
"""Discover runners from plugin runtime.
Always discovers ALL runners (no bound_plugins filter).
The cache should contain unfiltered discovery results.
Returns:
Dict of runner descriptors keyed by runner ID
"""
if not self.ap.plugin_connector.is_enable_plugin:
return {}
runners: dict[str, AgentRunnerDescriptor] = {}
try:
# Always list all runners (bound_plugins=None)
plugin_runners = await self.ap.plugin_connector.list_agent_runners(None)
for runner_data in plugin_runners:
try:
descriptor = self._validate_and_build_descriptor(runner_data)
if descriptor is not None:
runners[descriptor.id] = descriptor
except Exception as e:
plugin_author = runner_data.get('plugin_author', 'unknown')
plugin_name = runner_data.get('plugin_name', 'unknown')
runner_name = runner_data.get('runner_name', 'unknown')
self.ap.logger.warning(
f'Invalid runner manifest for plugin:{plugin_author}/{plugin_name}/{runner_name}: {e}'
)
continue
except Exception as e:
self.ap.logger.warning(f'Failed to list agent runners from plugin runtime: {e}')
return {}
return runners
def _validate_and_build_descriptor(self, runner_data: dict[str, typing.Any]) -> AgentRunnerDescriptor | None:
"""Validate runner manifest and build descriptor.
Args:
runner_data: Raw runner data from plugin runtime with fields:
- plugin_author, plugin_name, runner_name
- manifest (full component manifest dict)
- protocol_version, capabilities, permissions, config (extracted from spec)
Returns:
AgentRunnerDescriptor if valid, None if invalid
"""
plugin_author = runner_data.get('plugin_author', '')
plugin_name = runner_data.get('plugin_name', '')
runner_name = runner_data.get('runner_name', '')
if not plugin_author or not plugin_name or not runner_name:
return None
manifest = runner_data.get('manifest', {})
# Validate kind
kind = manifest.get('kind', '')
if kind != 'AgentRunner':
return None
# Validate metadata
metadata = manifest.get('metadata', {})
name = metadata.get('name', '')
if not name:
return None
# metadata.label must exist
label = metadata.get('label', {})
if not label:
label = {name: name} # fallback
spec = manifest.get('spec', {})
# SDK now provides these directly extracted from spec. Fall back to
# manifest.spec for older runtimes/tests that return the raw manifest.
protocol_version = runner_data.get('protocol_version') or spec.get('protocol_version', '1')
config_schema = runner_data.get('config') or spec.get('config', [])
capabilities = runner_data.get('capabilities') or spec.get('capabilities', {})
permissions = runner_data.get('permissions') or spec.get('permissions', {})
# Build descriptor
runner_id = format_runner_id(
source='plugin',
plugin_author=plugin_author,
plugin_name=plugin_name,
runner_name=runner_name,
)
return AgentRunnerDescriptor(
id=runner_id,
source='plugin',
label=label,
description=metadata.get('description') or runner_data.get('runner_description'),
plugin_author=plugin_author,
plugin_name=plugin_name,
runner_name=runner_name,
plugin_version=runner_data.get('plugin_version'),
protocol_version=protocol_version,
config_schema=config_schema,
capabilities=capabilities,
permissions=permissions,
raw_manifest=manifest,
)
async def refresh(self) -> None:
"""Refresh runner cache.
Always discovers ALL runners (no bound_plugins filter).
The cache contains unfiltered discovery results.
"""
async with self._cache_lock:
self._cache = await self._discover_runners()
async def list_runners(
self,
bound_plugins: list[str] | None = None,
use_cache: bool = True,
) -> list[AgentRunnerDescriptor]:
"""List available runners.
Args:
bound_plugins: Optional filter for bound plugins (applied locally)
use_cache: Use cached data if available
Returns:
List of runner descriptors
"""
if use_cache and self._cache is not None:
# Filter from cache
return self._filter_runners_by_bound_plugins(self._cache, bound_plugins)
# Discover fresh (always full list)
runners = await self._discover_runners()
# Update cache (full list, unfiltered)
async with self._cache_lock:
self._cache = runners
# Filter locally
return self._filter_runners_by_bound_plugins(runners, bound_plugins)
def _filter_runners_by_bound_plugins(
self,
runners: dict[str, AgentRunnerDescriptor],
bound_plugins: list[str] | None,
) -> list[AgentRunnerDescriptor]:
"""Filter runners by bound plugins.
Args:
runners: Dict of runner descriptors
bound_plugins: Optional filter (None means all plugins allowed)
Returns:
Filtered list of runner descriptors
"""
if bound_plugins is None:
# All plugins allowed
return list(runners.values())
allowed_plugin_ids = set(bound_plugins)
filtered = []
for descriptor in runners.values():
plugin_id = descriptor.get_plugin_id()
if plugin_id in allowed_plugin_ids:
filtered.append(descriptor)
return filtered
async def get(
self,
runner_id: str,
bound_plugins: list[str] | None = None,
) -> AgentRunnerDescriptor:
"""Get a specific runner descriptor.
Args:
runner_id: Runner ID to lookup
bound_plugins: Optional bound plugins filter
Returns:
AgentRunnerDescriptor
Raises:
RunnerNotFoundError: If runner not found
RunnerNotAuthorizedError: If runner not in bound plugins
"""
# Parse and validate runner ID format
try:
parse_runner_id(runner_id)
except ValueError as e:
raise RunnerNotFoundError(runner_id) from e
# Get from cache or discover (always full list)
if self._cache is None:
await self.refresh()
if self._cache is None:
raise RunnerNotFoundError(runner_id)
descriptor = self._cache.get(runner_id)
if descriptor is None:
raise RunnerNotFoundError(runner_id)
# Check authorization
if bound_plugins is not None:
plugin_id = descriptor.get_plugin_id()
if plugin_id not in bound_plugins:
raise RunnerNotAuthorizedError(runner_id, bound_plugins)
return descriptor
async def get_runner_metadata_for_pipeline(self) -> list[dict[str, typing.Any]]:
"""Get runner metadata for pipeline configuration UI.
Returns runner options and their config schemas for the DynamicForm.
"""
# Get all runners (no bound plugin filter for metadata listing)
runners = await self.list_runners(bound_plugins=None)
options = []
stages = []
for descriptor in runners:
config_schema = []
for index, config_item in enumerate(descriptor.config_schema):
item = dict(config_item)
if not item.get('id'):
item_name = item.get('name') or str(index)
item['id'] = f'{descriptor.id}.{item_name}'
config_schema.append(item)
# Add runner option
options.append(
{
'name': descriptor.id,
'label': descriptor.label,
'description': descriptor.description,
}
)
# Add config schema as stage if not empty
if descriptor.config_schema:
stages.append(
{
'name': descriptor.id,
'label': descriptor.label,
'description': descriptor.description,
'config': config_schema,
}
)
return options, stages

View File

@@ -1,268 +0,0 @@
"""Agent resource builder for constructing authorized resources."""
from __future__ import annotations
import typing
from ...core import app
from .descriptor import AgentRunnerDescriptor
from .context_builder import (
AgentResources,
ModelResource,
ToolResource,
KnowledgeBaseResource,
StorageResource,
)
from . import config_schema
from .host_models import AgentEventEnvelope, AgentBinding
class AgentResourceBuilder:
"""Builder for constructing AgentResources with permission filtering.
Responsibilities:
- Apply 3-layer permission filtering:
1. Runner manifest declared permissions
2. Pipeline extensions_preference (bound plugins/MCP servers)
3. Agent/runner config selected resources
- Build models list from authorized models
- Build tools list from bound plugins/MCP servers
- Build knowledge_bases list from config
- Build storage and files permissions summary
Note: This only builds the resource declaration. The actual proxy actions
in handler.py must still validate against ctx.resources at runtime.
Resource field names match the plugin SDK payload:
- ModelResource: model_id, model_type, provider
- ToolResource: tool_name, tool_type, description
- KnowledgeBaseResource: kb_id, kb_name, kb_type
- StorageResource: plugin_storage, workspace_storage
"""
ap: app.Application
def __init__(self, ap: app.Application):
self.ap = ap
async def build_resources_from_binding(
self,
event: AgentEventEnvelope,
binding: AgentBinding,
descriptor: AgentRunnerDescriptor,
) -> AgentResources:
"""Build AgentResources from event and binding.
This is the main entry point for Protocol v1.
Args:
event: Event envelope
binding: Agent binding with resource policy
descriptor: Runner descriptor with permissions and capabilities
Returns:
AgentResources dict with filtered resource lists
"""
# Layer 1: Runner manifest permissions
manifest_perms = descriptor.permissions
# Layer 2: Binding resource policy
resource_policy = binding.resource_policy
# Layer 3: Agent/runner config
runner_config = binding.runner_config
# Build each resource category
models = await self._build_models_from_binding(
manifest_perms, resource_policy, descriptor, runner_config
)
tools = await self._build_tools_from_binding(
manifest_perms, resource_policy, binding
)
knowledge_bases = await self._build_knowledge_bases_from_binding(
manifest_perms, resource_policy, descriptor, runner_config
)
storage = self._build_storage_from_binding(manifest_perms, binding)
return {
'models': models,
'tools': tools,
'knowledge_bases': knowledge_bases,
'files': [], # Files are populated at runtime
'storage': storage,
'platform_capabilities': {}, # Reserved for EBA
}
async def _build_models_from_binding(
self,
manifest_perms: dict[str, list[str]],
resource_policy: typing.Any,
descriptor: AgentRunnerDescriptor,
runner_config: dict[str, typing.Any],
) -> list[ModelResource]:
"""Build models list from binding."""
models: list[ModelResource] = []
seen_model_ids: set[str] = set()
model_perms = manifest_perms.get('models', [])
allow_llm = 'invoke' in model_perms or 'stream' in model_perms
allow_rerank = 'rerank' in model_perms
if not allow_llm and not allow_rerank:
return models
# Get additional model UUID grants from resource policy.
allowed_uuids = resource_policy.allowed_model_uuids
# Add model resources from Agent/runner config schema
await self._append_config_declared_model_resources(
models=models,
seen_model_ids=seen_model_ids,
descriptor=descriptor,
runner_config=runner_config,
include_llm=allow_llm,
include_rerank=allow_rerank,
)
# Add explicitly allowed models
if allowed_uuids and allow_llm:
for model_uuid in allowed_uuids:
await self._append_llm_model_resource(models, seen_model_ids, model_uuid)
return models
async def _build_tools_from_binding(
self,
manifest_perms: dict[str, list[str]],
resource_policy: typing.Any,
binding: AgentBinding,
) -> list[ToolResource]:
"""Build tools list from binding."""
tools: list[ToolResource] = []
# Check manifest permission
tool_perms = manifest_perms.get('tools', [])
if 'detail' not in tool_perms and 'call' not in tool_perms:
return tools
# Get tool names from resource policy
allowed_names = resource_policy.allowed_tool_names
if allowed_names:
for tool_name in allowed_names:
tools.append({
'tool_name': tool_name,
'tool_type': None,
'description': None,
})
return tools
async def _build_knowledge_bases_from_binding(
self,
manifest_perms: dict[str, list[str]],
resource_policy: typing.Any,
descriptor: AgentRunnerDescriptor,
runner_config: dict[str, typing.Any],
) -> list[KnowledgeBaseResource]:
"""Build knowledge bases list from binding."""
kb_resources: list[KnowledgeBaseResource] = []
# Check manifest permission
kb_perms = manifest_perms.get('knowledge_bases', [])
if 'list' not in kb_perms and 'retrieve' not in kb_perms:
return kb_resources
# Get KB UUID grants from schema-defined config fields.
kb_uuids = config_schema.extract_knowledge_base_uuids(descriptor, runner_config)
# Also include resource policy grants.
allowed_uuids = resource_policy.allowed_kb_uuids
if allowed_uuids:
kb_uuids = list(dict.fromkeys([*kb_uuids, *allowed_uuids]))
for kb_uuid in kb_uuids:
try:
kb = await self.ap.rag_mgr.get_knowledge_base_by_uuid(kb_uuid)
if kb:
kb_resources.append({
'kb_id': kb_uuid,
'kb_name': kb.get_name(),
'kb_type': kb.knowledge_base_entity.kb_type if hasattr(kb.knowledge_base_entity, 'kb_type') else None,
})
except Exception as e:
self.ap.logger.warning(f'Failed to build knowledge base resource {kb_uuid}: {e}')
return kb_resources
def _build_storage_from_binding(
self,
manifest_perms: dict[str, list[str]],
binding: AgentBinding,
) -> StorageResource:
"""Build storage permissions from binding."""
storage_perms = manifest_perms.get('storage', [])
resource_policy = binding.resource_policy
return {
'plugin_storage': 'plugin' in storage_perms and resource_policy.allow_plugin_storage,
'workspace_storage': 'workspace' in storage_perms and resource_policy.allow_workspace_storage,
}
async def _append_config_declared_model_resources(
self,
models: list[ModelResource],
seen_model_ids: set[str],
descriptor: AgentRunnerDescriptor,
runner_config: dict[str, typing.Any],
include_llm: bool,
include_rerank: bool,
) -> None:
"""Authorize model-like values selected through DynamicForm fields."""
for model_type, model_uuid in config_schema.iter_config_model_refs(descriptor, runner_config):
if model_type == 'llm' and include_llm:
await self._append_llm_model_resource(models, seen_model_ids, model_uuid)
elif model_type == 'rerank' and include_rerank:
await self._append_rerank_model_resource(models, seen_model_ids, model_uuid)
async def _append_llm_model_resource(
self,
models: list[ModelResource],
seen_model_ids: set[str],
model_uuid: str | None,
) -> None:
"""Append an LLM model resource if it exists and has not been added."""
if not model_uuid or model_uuid == '__none__' or model_uuid in seen_model_ids:
return
try:
model = await self.ap.model_mgr.get_model_by_uuid(model_uuid)
if model and model.model_entity:
models.append({
'model_id': model_uuid,
'model_type': getattr(model.model_entity, 'model_type', None),
'provider': getattr(model.provider_entity, 'name', None) if hasattr(model, 'provider_entity') else None,
})
seen_model_ids.add(model_uuid)
except Exception as e:
self.ap.logger.warning(f'Failed to build LLM model resource {model_uuid}: {e}')
async def _append_rerank_model_resource(
self,
models: list[ModelResource],
seen_model_ids: set[str],
model_uuid: str | None,
) -> None:
"""Append a rerank model resource if it exists and has not been added."""
if not model_uuid or model_uuid == '__none__' or model_uuid in seen_model_ids:
return
try:
model = await self.ap.model_mgr.get_rerank_model_by_uuid(model_uuid)
if model and model.model_entity:
models.append({
'model_id': model_uuid,
'model_type': getattr(model.model_entity, 'model_type', 'rerank') or 'rerank',
'provider': getattr(model.provider_entity, 'name', None) if hasattr(model, 'provider_entity') else None,
})
seen_model_ids.add(model_uuid)
except Exception as e:
self.ap.logger.warning(f'Failed to build rerank model resource {model_uuid}: {e}')

View File

@@ -1,193 +0,0 @@
"""Agent result normalizer for converting AgentRunResult to Pipeline messages."""
from __future__ import annotations
import typing
from langbot_plugin.api.entities.builtin.provider import message as provider_message
from ...core import app
from .descriptor import AgentRunnerDescriptor
from .errors import RunnerExecutionError, RunnerProtocolError
# Maximum size for a single result payload (prevent memory exhaustion)
MAX_RESULT_SIZE_BYTES = 1024 * 1024 # 1 MB
class AgentResultNormalizer:
"""Normalizer for converting AgentRunResult to Pipeline messages.
Responsibilities:
- Accept only supported result types (message.delta, message.completed, etc.)
- Map message.delta -> MessageChunk
- Map message.completed -> Message
- Map run.completed (with message) -> Message
- Handle run.failed as controlled error
- Ignore unknown types with warning
- Validate result size
- Validate message schema
Accepted result types:
- message.delta
- message.completed
- tool.call.started
- tool.call.completed
- state.updated
- run.completed
- run.failed
- action.requested (log only, don't execute)
"""
ap: app.Application
def __init__(self, ap: app.Application):
self.ap = ap
async def normalize(
self,
result_dict: dict[str, typing.Any],
descriptor: AgentRunnerDescriptor,
) -> provider_message.Message | provider_message.MessageChunk | None:
"""Normalize AgentRunResult to Message or MessageChunk.
Args:
result_dict: Raw result dict from plugin runtime
descriptor: Runner descriptor for error context
Returns:
Message, MessageChunk, or None (for non-message events)
Raises:
RunnerExecutionError: On run.failed
RunnerProtocolError: On invalid result format
"""
# Validate result type
result_type = result_dict.get('type')
if not result_type:
raise RunnerProtocolError(descriptor.id, 'Missing result type')
# Validate result size
try:
import json
result_json = json.dumps(result_dict)
if len(result_json) > MAX_RESULT_SIZE_BYTES:
self.ap.logger.warning(
f'Runner {descriptor.id} result too large ({len(result_json)} bytes), truncating'
)
# Truncate content if possible
data = result_dict.get('data', {})
if 'chunk' in data or 'message' in data:
content = data.get('chunk', {}).get('content', '') or data.get('message', {}).get('content', '')
if isinstance(content, str) and len(content) > 10000:
# Keep reasonable length
data['chunk'] = {'role': 'assistant', 'content': content[:10000] + '...[truncated]'}
except Exception as e:
self.ap.logger.warning(f'Failed to validate runner {descriptor.id} result size: {e}')
# Handle each result type
data = result_dict.get('data', {})
if result_type == 'message.delta':
return self._normalize_message_delta(data, descriptor)
elif result_type == 'message.completed':
return self._normalize_message_completed(data, descriptor)
elif result_type == 'tool.call.started':
# Log only, don't yield to pipeline
self.ap.logger.debug(
f'Runner {descriptor.id} tool call started: {data.get("tool_name", "unknown")}'
)
return None
elif result_type == 'tool.call.completed':
# Log only, don't yield to pipeline
self.ap.logger.debug(
f'Runner {descriptor.id} tool call completed: {data.get("tool_name", "unknown")}'
)
return None
elif result_type == 'state.updated':
# Log for telemetry, don't yield to pipeline
# Orchestrator already handles the actual PersistentStateStore update.
scope = data.get('scope', 'unknown')
key = data.get('key', 'unknown')
value_repr = repr(data.get('value', '...'))[:100] # Truncate for log
self.ap.logger.debug(
f'Runner {descriptor.id} state.updated logged: scope={scope}, key={key}, value={value_repr}'
)
return None
elif result_type == 'run.completed':
# May include final message
if 'message' in data:
return self._normalize_message_completed(data, descriptor)
# If no message, it's just completion signal
return None
elif result_type == 'run.failed':
error_msg = data.get('error', 'Unknown error')
error_code = data.get('code', 'unknown')
retryable = data.get('retryable', False)
raise RunnerExecutionError(
descriptor.id,
f'{error_msg} (code: {error_code})',
retryable=retryable,
)
elif result_type == 'action.requested':
# Reserved for EBA - log only, don't execute
self.ap.logger.info(
f'Runner {descriptor.id} requested action (not executed in current phase): '
f'{data.get("action", "unknown")}'
)
return None
elif result_type == 'artifact.created':
# Log for telemetry, consumed by orchestrator
artifact_id = data.get('artifact_id', 'unknown')
artifact_type = data.get('artifact_type', 'unknown')
self.ap.logger.debug(
f'Runner {descriptor.id} artifact.created logged: artifact_id={artifact_id}, type={artifact_type}'
)
return None
else:
# Unknown type - warn and ignore.
self.ap.logger.warning(
f'Runner {descriptor.id} returned unknown result type: {result_type}. '
f'Expected supported types (message.delta, message.completed, run.completed, run.failed, etc.)'
)
return None
def _normalize_message_delta(
self,
data: dict[str, typing.Any],
descriptor: AgentRunnerDescriptor,
) -> provider_message.MessageChunk:
"""Normalize message.delta to MessageChunk."""
chunk_data = data.get('chunk', {})
if not chunk_data:
raise RunnerProtocolError(descriptor.id, 'message.delta missing chunk data')
try:
chunk = provider_message.MessageChunk.model_validate(chunk_data)
return chunk
except Exception as e:
raise RunnerProtocolError(descriptor.id, f'Invalid chunk schema: {e}')
def _normalize_message_completed(
self,
data: dict[str, typing.Any],
descriptor: AgentRunnerDescriptor,
) -> provider_message.Message:
"""Normalize message.completed to Message."""
message_data = data.get('message', {})
if not message_data:
raise RunnerProtocolError(descriptor.id, 'message.completed missing message data')
try:
msg = provider_message.Message.model_validate(message_data)
return msg
except Exception as e:
raise RunnerProtocolError(descriptor.id, f'Invalid message schema: {e}')

View File

@@ -1,263 +0,0 @@
"""Agent run session registry for proxy action permission validation."""
from __future__ import annotations
import asyncio
import copy
import typing
import time
import threading
from .context_builder import AgentResources
class AgentRunSessionStatus(typing.TypedDict):
"""Status tracking for agent run session."""
started_at: int
last_activity_at: int
class RunAuthorizationSnapshot(typing.TypedDict):
"""Frozen authorization data for one active run.
ResourceBuilder creates the authorized resource list once before runner
execution. Runtime proxy handlers must validate against this run-scoped
snapshot instead of recomputing resource policy.
"""
resources: AgentResources
permissions: dict[str, list[str]]
conversation_id: str | None
state_policy: dict[str, typing.Any]
state_context: dict[str, typing.Any]
authorized_ids: dict[str, set[str]]
class AgentRunSession(typing.TypedDict):
"""Session for an active agent runner execution.
Stored in AgentRunSessionRegistry for proxy action permission validation.
Fields:
run_id: Unique run identifier (UUID from AgentRunContext)
runner_id: Runner descriptor ID (plugin:author/name/runner)
query_id: Host entry query ID, only present for query-based adapters
plugin_identity: Plugin identifier (author/name) of the runner
authorization: Run-scoped authorization snapshot; runtime auth truth
status: Session status tracking
"""
run_id: str
runner_id: str
query_id: int | None
plugin_identity: str # author/name
authorization: RunAuthorizationSnapshot
status: AgentRunSessionStatus
class AgentRunSessionRegistry:
"""Registry for active agent run sessions.
Host-owned registry for tracking active AgentRunner executions.
Used by proxy actions in handler.py to validate resource access.
Key: run_id (UUID from AgentRunContext)
Value: AgentRunSession with authorized resources
Thread-safe via asyncio.Lock.
"""
_sessions: dict[str, AgentRunSession]
_lock: asyncio.Lock
def __init__(self):
self._sessions = {}
self._lock = asyncio.Lock()
async def register(
self,
run_id: str,
runner_id: str,
query_id: int | None,
plugin_identity: str,
resources: AgentResources,
conversation_id: str | None = None,
permissions: dict[str, list[str]] | None = None,
state_policy: dict[str, typing.Any] | None = None,
state_context: dict[str, typing.Any] | None = None,
) -> None:
"""Register a new agent run session.
Args:
run_id: Unique run identifier
runner_id: Runner descriptor ID
query_id: Host entry query ID, only present for query-based adapters
plugin_identity: Plugin identifier (author/name)
resources: Authorized resources for this run
conversation_id: Conversation ID for history/event access
permissions: Runner permissions from descriptor (artifacts, history, events, etc.)
state_policy: State policy from binding (enable_state, state_scopes)
state_context: Context for state API (scope_keys, binding_identity, etc.)
"""
now = int(time.time())
# Normalize permissions to empty dict if None
permissions = permissions or {}
# Normalize state_policy to defaults if None
if state_policy is None:
state_policy = {'enable_state': True, 'state_scopes': ['conversation', 'actor']}
# Normalize state_context to empty dict if None
state_context = state_context or {}
resources_snapshot = copy.deepcopy(resources)
authorization: RunAuthorizationSnapshot = {
'resources': resources_snapshot,
'permissions': copy.deepcopy(permissions),
'conversation_id': conversation_id,
'state_policy': copy.deepcopy(state_policy),
'state_context': copy.deepcopy(state_context),
'authorized_ids': self._build_authorized_ids(resources_snapshot),
}
session: AgentRunSession = {
'run_id': run_id,
'runner_id': runner_id,
'query_id': query_id,
'plugin_identity': plugin_identity,
'authorization': authorization,
'status': {
'started_at': now,
'last_activity_at': now,
},
}
async with self._lock:
self._sessions[run_id] = session
def _build_authorized_ids(self, resources: AgentResources) -> dict[str, set[str]]:
"""Pre-compute authorized resource IDs for O(1) lookup."""
return {
'model': {m.get('model_id') for m in resources.get('models', [])},
'tool': {t.get('tool_name') for t in resources.get('tools', [])},
'knowledge_base': {kb.get('kb_id') for kb in resources.get('knowledge_bases', [])},
'file': {f.get('file_id') for f in resources.get('files', [])},
}
async def unregister(self, run_id: str) -> None:
"""Unregister an agent run session.
Args:
run_id: Unique run identifier
"""
async with self._lock:
if run_id in self._sessions:
del self._sessions[run_id]
async def get(self, run_id: str) -> AgentRunSession | None:
"""Get session by run_id.
Args:
run_id: Unique run identifier
Returns:
AgentRunSession if found, None otherwise
"""
async with self._lock:
return self._sessions.get(run_id)
async def update_activity(self, run_id: str) -> None:
"""Update last activity timestamp for session.
Args:
run_id: Unique run identifier
"""
async with self._lock:
if run_id in self._sessions:
self._sessions[run_id]['status']['last_activity_at'] = int(time.time())
def is_resource_allowed(
self,
session: AgentRunSession,
resource_type: str,
resource_id: str,
) -> bool:
"""Check if resource access is allowed for this session.
Uses pre-computed authorized IDs for O(1) lookup.
Args:
session: AgentRunSession to check
resource_type: Resource type ('model', 'tool', 'knowledge_base', 'storage', 'file')
resource_id: Resource identifier (model_id, tool_name, kb_id, 'plugin'/'workspace', file_key)
Returns:
True if resource is authorized, False otherwise
"""
authorization = session['authorization']
authorized_ids = authorization['authorized_ids']
resources = authorization['resources']
if resource_type in ('model', 'tool', 'knowledge_base', 'file'):
return resource_id in authorized_ids.get(resource_type, set())
if resource_type == 'storage':
storage = resources.get('storage', {})
if resource_id == 'plugin':
return storage.get('plugin_storage', False)
elif resource_id == 'workspace':
return storage.get('workspace_storage', False)
return False
return False
async def list_active_runs(self) -> list[AgentRunSession]:
"""List all active run sessions.
Returns:
List of active AgentRunSession dicts
"""
async with self._lock:
return list(self._sessions.values())
async def cleanup_stale_sessions(self, max_age_seconds: int = 3600) -> int:
"""Cleanup sessions that have been inactive for too long.
Args:
max_age_seconds: Maximum inactivity time in seconds (default 1 hour)
Returns:
Number of sessions cleaned up
"""
now = int(time.time())
cleaned = 0
async with self._lock:
stale_run_ids = []
for run_id, session in self._sessions.items():
last_activity = session['status'].get('last_activity_at', 0)
if now - last_activity > max_age_seconds:
stale_run_ids.append(run_id)
for run_id in stale_run_ids:
del self._sessions[run_id]
cleaned += 1
return cleaned
# Global registry instance (singleton)
_global_registry: AgentRunSessionRegistry | None = None
_global_registry_lock = threading.Lock()
def get_session_registry() -> AgentRunSessionRegistry:
"""Get global session registry instance (thread-safe singleton).
Returns:
AgentRunSessionRegistry singleton
"""
global _global_registry
with _global_registry_lock:
if _global_registry is None:
_global_registry = AgentRunSessionRegistry()
return _global_registry

View File

@@ -1,113 +0,0 @@
"""State scope key helpers for AgentRunner host-owned state."""
from __future__ import annotations
import typing
from .descriptor import AgentRunnerDescriptor
from .host_models import AgentBinding, AgentEventEnvelope
VALID_STATE_SCOPES = ('conversation', 'actor', 'subject', 'runner')
STATE_KEY_ALIASES = {
'conversation_id': 'external.conversation_id',
}
def normalize_state_key(key: str) -> str:
"""Map accepted public aliases to protocol state keys."""
return STATE_KEY_ALIASES.get(key, key)
def get_binding_identity(binding: AgentBinding) -> str:
"""Return the stable binding identity used for state isolation."""
if binding.binding_id:
return binding.binding_id
scope = binding.scope
if scope.scope_type and scope.scope_id:
return f'{scope.scope_type}:{scope.scope_id}'
return 'unknown_binding'
def build_state_scope_key(
scope: str,
event: AgentEventEnvelope,
binding: AgentBinding,
descriptor: AgentRunnerDescriptor,
) -> str | None:
"""Build the storage key for one state scope.
Returns None when the event lacks the identity required by that scope.
"""
binding_identity = get_binding_identity(binding)
if scope == 'conversation':
if not event.conversation_id:
return None
parts = [descriptor.id, binding_identity, event.conversation_id]
if event.thread_id:
parts.append(event.thread_id)
return f'conversation:{":".join(parts)}'
if scope == 'actor':
if not event.actor or not event.actor.actor_id:
return None
parts = [
descriptor.id,
binding_identity,
event.actor.actor_type or 'user',
event.actor.actor_id,
]
return f'actor:{":".join(parts)}'
if scope == 'subject':
if not event.subject or not event.subject.subject_id:
return None
parts = [
descriptor.id,
binding_identity,
event.subject.subject_type or 'unknown',
event.subject.subject_id,
]
return f'subject:{":".join(parts)}'
if scope == 'runner':
return f'runner:{descriptor.id}:{binding_identity}'
return None
def build_state_scope_keys(
event: AgentEventEnvelope,
binding: AgentBinding,
descriptor: AgentRunnerDescriptor,
) -> dict[str, str]:
"""Build all available scope keys for an event/binding pair."""
scope_keys: dict[str, str] = {}
for scope in VALID_STATE_SCOPES:
scope_key = build_state_scope_key(scope, event, binding, descriptor)
if scope_key:
scope_keys[scope] = scope_key
return scope_keys
def build_state_context(
event: AgentEventEnvelope,
binding: AgentBinding,
descriptor: AgentRunnerDescriptor,
) -> dict[str, typing.Any]:
"""Build the State API context stored in the run session."""
return {
'scope_keys': build_state_scope_keys(event, binding, descriptor),
'binding_identity': get_binding_identity(binding),
'bot_id': event.bot_id,
'workspace_id': event.workspace_id,
'conversation_id': event.conversation_id,
'thread_id': event.thread_id,
'actor_type': event.actor.actor_type if event.actor else None,
'actor_id': event.actor.actor_id if event.actor else None,
'subject_type': event.subject.subject_type if event.subject else None,
'subject_id': event.subject.subject_id if event.subject else None,
}

View File

@@ -1,290 +0,0 @@
"""Transcript store for writing and querying conversation history."""
from __future__ import annotations
import json
import datetime
import typing
import uuid
import sqlalchemy
from sqlalchemy.ext.asyncio import AsyncEngine, AsyncSession
from sqlalchemy.orm import sessionmaker
from ...entity.persistence.transcript import Transcript
class TranscriptStore:
"""Store for Transcript records.
Handles writing transcript items and querying them for history API.
All methods are async and use the provided database engine.
"""
engine: AsyncEngine
# Hard limits
MAX_CONTENT_LENGTH = 4000
HARD_LIMIT = 100
def __init__(self, engine: AsyncEngine):
self.engine = engine
self._session_factory = sessionmaker(
engine, class_=AsyncSession, expire_on_commit=False
)
async def append_transcript(
self,
transcript_id: str | None,
event_id: str,
conversation_id: str,
role: str,
content: str | None = None,
content_json: dict[str, typing.Any] | None = None,
artifact_refs: list[dict[str, typing.Any]] | None = None,
thread_id: str | None = None,
item_type: str = "message",
run_id: str | None = None,
runner_id: str | None = None,
metadata: dict[str, typing.Any] | None = None,
) -> str:
"""Append a transcript item.
Args:
transcript_id: Unique transcript ID (generated if None)
event_id: Source event ID
conversation_id: Conversation ID
role: Message role (user, assistant, system, tool)
content: Text content
content_json: Full structured content
artifact_refs: Artifact references
thread_id: Thread ID
item_type: Item type
run_id: Run ID that generated this
runner_id: Runner ID that generated this
metadata: Additional metadata
Returns:
The transcript_id
"""
if transcript_id is None:
transcript_id = str(uuid.uuid4())
# Truncate content if too long
if content and len(content) > self.MAX_CONTENT_LENGTH:
content = content[:self.MAX_CONTENT_LENGTH - 3] + "..."
async with self._session_factory() as session:
item = Transcript(
transcript_id=transcript_id,
event_id=event_id,
conversation_id=conversation_id,
thread_id=thread_id,
role=role,
item_type=item_type,
content=content,
content_json=json.dumps(content_json) if content_json else None,
artifact_refs_json=json.dumps(artifact_refs) if artifact_refs else None,
seq=0,
run_id=run_id,
runner_id=runner_id,
created_at=datetime.datetime.utcnow(),
metadata_json=json.dumps(metadata) if metadata else None,
)
session.add(item)
await session.flush()
item.seq = item.id or await self._get_next_seq(conversation_id)
await session.commit()
return transcript_id
async def page_transcript(
self,
conversation_id: str,
before_seq: int | None = None,
after_seq: int | None = None,
limit: int = 50,
direction: str = "backward",
include_artifacts: bool = False,
) -> tuple[list[dict[str, typing.Any]], int | None, int | None, bool]:
"""Page through transcript items.
Args:
conversation_id: Conversation ID
before_seq: Get items before this sequence (backward)
after_seq: Get items after this sequence (forward)
limit: Maximum items to return (capped at 100)
direction: 'backward' (older) or 'forward' (newer)
include_artifacts: Include artifact refs
Returns:
Tuple of (items, next_seq, prev_seq, has_more)
"""
limit = min(limit, self.HARD_LIMIT)
async with self._session_factory() as session:
query = sqlalchemy.select(Transcript).where(
Transcript.conversation_id == conversation_id
)
if direction == "backward" and before_seq is not None:
query = query.where(Transcript.seq < before_seq)
query = query.order_by(Transcript.seq.desc())
elif direction == "forward" and after_seq is not None:
query = query.where(Transcript.seq > after_seq)
query = query.order_by(Transcript.seq.asc())
else:
# Default: most recent items first (backward from latest)
query = query.order_by(Transcript.seq.desc())
query = query.limit(limit + 1)
result = await session.execute(query)
rows = result.scalars().all()
items = [self._row_to_dict(row, include_artifacts) for row in rows[:limit]]
has_more = len(rows) > limit
# Calculate cursors
next_seq = None
prev_seq = None
if direction == "backward":
# Items are in descending order
if items:
next_seq = items[-1].get('seq') if has_more else None
prev_seq = items[0].get('seq')
else:
# Items are in ascending order
if items:
next_seq = items[-1].get('seq') if has_more else None
prev_seq = items[0].get('seq')
return items, next_seq, prev_seq, has_more
async def search_transcript(
self,
conversation_id: str,
query_text: str,
filters: dict[str, typing.Any] | None = None,
top_k: int = 10,
) -> list[dict[str, typing.Any]]:
"""Search transcript items.
Basic implementation using LIKE filtering.
Args:
conversation_id: Conversation ID
query_text: Search query
filters: Optional filters
top_k: Maximum results
Returns:
List of matching items
"""
async with self._session_factory() as session:
query = sqlalchemy.select(Transcript).where(
Transcript.conversation_id == conversation_id,
Transcript.content.ilike(f"%{query_text}%"),
)
# Apply additional filters
if filters:
if 'roles' in filters:
query = query.where(Transcript.role.in_(filters['roles']))
if 'item_types' in filters:
query = query.where(Transcript.item_type.in_(filters['item_types']))
query = query.order_by(Transcript.seq.desc()).limit(top_k)
result = await session.execute(query)
rows = result.scalars().all()
return [self._row_to_dict(row, include_artifacts=True) for row in rows]
async def get_latest_cursor(
self,
conversation_id: str,
) -> str | None:
"""Get the latest cursor for a conversation.
Args:
conversation_id: Conversation ID
Returns:
Cursor string (seq number), or None if no items
"""
async with self._session_factory() as session:
result = await session.execute(
sqlalchemy.select(Transcript.seq)
.where(Transcript.conversation_id == conversation_id)
.order_by(Transcript.seq.desc())
.limit(1)
)
row = result.scalars().first()
if row is None:
return None
return str(row)
async def has_history_before(
self,
conversation_id: str,
seq: int,
) -> bool:
"""Check if there is history before a sequence number.
Args:
conversation_id: Conversation ID
seq: Sequence number
Returns:
True if there are items before
"""
async with self._session_factory() as session:
result = await session.execute(
sqlalchemy.select(sqlalchemy.func.count())
.select_from(Transcript)
.where(
Transcript.conversation_id == conversation_id,
Transcript.seq < seq,
)
)
count = result.scalar()
return count > 0
async def _get_next_seq(self, conversation_id: str) -> int:
"""Fallback next sequence number for stores that cannot expose autoincrement IDs."""
async with self._session_factory() as session:
result = await session.execute(
sqlalchemy.select(sqlalchemy.func.max(Transcript.seq))
.where(Transcript.conversation_id == conversation_id)
)
max_seq = result.scalar()
return (max_seq or 0) + 1
def _row_to_dict(
self,
row: Transcript,
include_artifacts: bool = False,
) -> dict[str, typing.Any]:
"""Convert a Transcript row to dict."""
result = {
'transcript_id': row.transcript_id,
'event_id': row.event_id,
'conversation_id': row.conversation_id,
'thread_id': row.thread_id,
'role': row.role,
'item_type': row.item_type,
'content': row.content,
'content_json': json.loads(row.content_json) if row.content_json else None,
'seq': row.seq,
'cursor': str(row.seq),
'created_at': int(row.created_at.timestamp()) if row.created_at else None,
'metadata': json.loads(row.metadata_json) if row.metadata_json else {},
}
if include_artifacts and row.artifact_refs_json:
result['artifact_refs'] = json.loads(row.artifact_refs_json)
else:
result['artifact_refs'] = []
return result

View File

@@ -1,22 +0,0 @@
from __future__ import annotations
from .. import group
@group.group_class('box', '/api/v1/box')
class BoxRouterGroup(group.RouterGroup):
async def initialize(self) -> None:
@self.route('/status', methods=['GET'], auth_type=group.AuthType.USER_TOKEN)
async def _() -> str:
status = await self.ap.box_service.get_status()
return self.success(data=status)
@self.route('/sessions', methods=['GET'], auth_type=group.AuthType.USER_TOKEN)
async def _() -> str:
sessions = await self.ap.box_service.get_sessions()
return self.success(data=sessions)
@self.route('/errors', methods=['GET'], auth_type=group.AuthType.USER_TOKEN)
async def _() -> str:
errors = self.ap.box_service.get_recent_errors()
return self.success(data=errors)

View File

@@ -1,52 +0,0 @@
from __future__ import annotations
import asyncio
import quart
from .. import group
@group.group_class('extensions', '/api/v1/extensions')
class ExtensionsRouterGroup(group.RouterGroup):
"""Unified API for installed extensions (plugins, MCP servers, skills)."""
async def initialize(self) -> None:
@self.route('', methods=['GET'], auth_type=group.AuthType.USER_TOKEN_OR_API_KEY)
async def _() -> quart.Response:
plugins, mcp_servers, skills = await asyncio.gather(
self.ap.plugin_connector.list_plugins(),
self.ap.mcp_service.get_mcp_servers(contain_runtime_info=True),
self.ap.skill_service.list_skills(),
return_exceptions=True,
)
def _sort_key(item: dict) -> str:
if item['type'] == 'plugin':
return (
item['plugin']
.get('manifest', {})
.get('manifest', {})
.get('metadata', {})
.get('name', '')
.lower()
)
if item['type'] == 'mcp':
return (item['server'].get('name') or '').lower()
if item['type'] == 'skill':
return (item['skill'].get('display_name') or item['skill'].get('name') or '').lower()
return ''
extensions: list[dict] = []
if isinstance(plugins, list):
for plugin in plugins:
extensions.append({'type': 'plugin', 'plugin': plugin})
if isinstance(mcp_servers, list):
for server in mcp_servers:
extensions.append({'type': 'mcp', 'server': server})
if isinstance(skills, list):
for skill in skills:
extensions.append({'type': 'skill', 'skill': skill})
extensions.sort(key=_sort_key)
return self.success(data={'extensions': extensions})

View File

@@ -13,9 +13,9 @@ from .. import group
@group.group_class('files', '/api/v1/files')
class FilesRouterGroup(group.RouterGroup):
async def initialize(self) -> None:
@self.route('/image/<path:image_key>', methods=['GET'], auth_type=group.AuthType.NONE)
@self.route('/image/<image_key>', methods=['GET'], auth_type=group.AuthType.NONE)
async def _(image_key: str) -> quart.Response:
if '..' in image_key or '\\' in image_key:
if '/' in image_key or '\\' in image_key:
return quart.Response(status=404)
if not await self.ap.storage_mgr.storage_provider.exists(image_key):

View File

@@ -456,31 +456,6 @@ class MonitoringRouterGroup(group.RouterGroup):
'platform',
'user_id',
]
elif export_type == 'feedback':
data = await self.ap.monitoring_service.export_feedback(
bot_ids=bot_ids if bot_ids else None,
pipeline_ids=pipeline_ids if pipeline_ids else None,
start_time=start_time,
end_time=end_time,
limit=limit,
)
headers = [
'id',
'timestamp',
'feedback_id',
'feedback_type',
'feedback_content',
'inaccurate_reasons',
'bot_id',
'bot_name',
'pipeline_id',
'pipeline_name',
'session_id',
'message_id',
'stream_id',
'user_id',
'platform',
]
else:
return self.error(message=f'Invalid export type: {export_type}', code=400)
@@ -511,63 +486,3 @@ class MonitoringRouterGroup(group.RouterGroup):
)
return response, 200
@self.route('/feedback/stats', methods=['GET'], auth_type=group.AuthType.USER_TOKEN)
async def get_feedback_stats() -> str:
"""Get feedback statistics"""
# Parse query parameters
bot_ids = quart.request.args.getlist('botId')
pipeline_ids = quart.request.args.getlist('pipelineId')
start_time_str = quart.request.args.get('startTime')
end_time_str = quart.request.args.get('endTime')
# Parse datetime
start_time = parse_iso_datetime(start_time_str)
end_time = parse_iso_datetime(end_time_str)
stats = await self.ap.monitoring_service.get_feedback_stats(
bot_ids=bot_ids if bot_ids else None,
pipeline_ids=pipeline_ids if pipeline_ids else None,
start_time=start_time,
end_time=end_time,
)
return self.success(data=stats)
@self.route('/feedback', methods=['GET'], auth_type=group.AuthType.USER_TOKEN)
async def get_feedback() -> str:
"""Get feedback list"""
# Parse query parameters
bot_ids = quart.request.args.getlist('botId')
pipeline_ids = quart.request.args.getlist('pipelineId')
feedback_type_str = quart.request.args.get('feedbackType')
start_time_str = quart.request.args.get('startTime')
end_time_str = quart.request.args.get('endTime')
limit = int(quart.request.args.get('limit', 100))
offset = int(quart.request.args.get('offset', 0))
# Parse datetime
start_time = parse_iso_datetime(start_time_str)
end_time = parse_iso_datetime(end_time_str)
# Parse feedback type
feedback_type = int(feedback_type_str) if feedback_type_str else None
feedback_list, total = await self.ap.monitoring_service.get_feedback_list(
bot_ids=bot_ids if bot_ids else None,
pipeline_ids=pipeline_ids if pipeline_ids else None,
feedback_type=feedback_type,
start_time=start_time,
end_time=end_time,
limit=limit,
offset=offset,
)
return self.success(
data={
'feedback': feedback_list,
'total': total,
'limit': limit,
'offset': offset,
}
)

View File

@@ -1,384 +0,0 @@
"""Embed widget routes - serve embeddable chat widget for external websites.
All user-facing URLs are keyed by **bot_uuid** (not pipeline_uuid) so that
internal pipeline identifiers are never exposed to end-users. Each handler
resolves the bot_uuid to the owning ``web_page_bot`` RuntimeBot and extracts
the bound pipeline_uuid for internal routing.
"""
import asyncio
import datetime
import json
import logging
import uuid
import hmac
import hashlib
import time
import re
import httpx
import quart
from ... import group
from ......utils import paths
from ......platform.sources.websocket_manager import ws_connection_manager
logger = logging.getLogger(__name__)
# Cache the widget template content
_widget_template_cache: str | None = None
_logo_bytes_cache: bytes | None = None
def _is_valid_uuid(s: str) -> bool:
return bool(re.match(r'^[a-f0-9]{8}-[a-f0-9]{4}-[a-f0-9]{4}-[a-f0-9]{4}-[a-f0-9]{12}$', s))
def _get_widget_template() -> str:
"""Load and cache the widget JS template."""
global _widget_template_cache
if _widget_template_cache is None:
template_path = paths.get_resource_path('templates/embed/widget.js')
with open(template_path, 'r', encoding='utf-8') as f:
_widget_template_cache = f.read()
return _widget_template_cache
def _get_logo_bytes() -> bytes:
"""Load and cache the logo image."""
global _logo_bytes_cache
if _logo_bytes_cache is None:
logo_path = paths.get_resource_path('templates/embed/logo.webp')
with open(logo_path, 'rb') as f:
_logo_bytes_cache = f.read()
return _logo_bytes_cache
@group.group_class('embed', '/api/v1/embed')
class EmbedRouterGroup(group.RouterGroup):
# -- helpers -------------------------------------------------------------
def _resolve_bot(self, bot_uuid: str):
"""Resolve *bot_uuid* to ``(runtime_bot, pipeline_uuid)``.
Returns ``(None, None)`` when the bot does not exist, is not a
``web_page_bot``, is disabled, or has no pipeline bound.
"""
for bot in self.ap.platform_mgr.bots:
if (
bot.bot_entity.uuid == bot_uuid
and bot.bot_entity.adapter == 'web_page_bot'
and bot.bot_entity.enable
and bot.bot_entity.use_pipeline_uuid
):
return bot, bot.bot_entity.use_pipeline_uuid
return None, None
def _get_bot_config(self, bot_uuid: str) -> dict:
for bot in self.ap.platform_mgr.bots:
if bot.bot_entity.uuid == bot_uuid and bot.bot_entity.adapter == 'web_page_bot':
return bot.bot_entity.adapter_config
return {}
async def _verify_session_token(self, request, bot_uuid: str) -> bool:
config = self._get_bot_config(bot_uuid)
secret = config.get('turnstile_secret_key', '')
if not secret:
return True
auth_header = request.headers.get('Authorization', '')
if not auth_header.startswith('Bearer '):
return False
token = auth_header[7:]
try:
ts_str, mac = token.split('.', 1)
ts = float(ts_str)
if time.time() - ts > 86400:
return False
expected_mac = hmac.new(secret.encode(), f'{ts_str}'.encode(), hashlib.sha256).hexdigest()
return hmac.compare_digest(mac, expected_mac)
except Exception:
return False
# -- routes --------------------------------------------------------------
async def initialize(self) -> None:
@self.route('/<bot_uuid>/turnstile/verify', methods=['POST'], auth_type=group.AuthType.NONE)
async def verify_turnstile(bot_uuid: str) -> str:
if not _is_valid_uuid(bot_uuid):
return self.http_status(400, -1, 'Invalid bot_uuid format')
runtime_bot, pipeline_uuid = self._resolve_bot(bot_uuid)
if runtime_bot is None:
return self.http_status(404, -1, 'Bot not found or not available')
try:
data = await quart.request.get_json()
token = data.get('token')
if not token:
return self.http_status(400, -1, 'Token is required')
config = self._get_bot_config(bot_uuid)
secret = config.get('turnstile_secret_key', '')
if not secret:
ts = time.time()
return self.success(data={'token': f'{ts}.dummy'})
async with httpx.AsyncClient() as client:
resp = await client.post(
'https://challenges.cloudflare.com/turnstile/v0/siteverify',
data={'secret': secret, 'response': token},
)
result = resp.json()
if not result.get('success'):
return self.http_status(403, -1, 'Turnstile verification failed')
ts = time.time()
mac = hmac.new(secret.encode(), f'{ts}'.encode(), hashlib.sha256).hexdigest()
session_token = f'{ts}.{mac}'
return self.success(data={'token': session_token})
except Exception as e:
logger.error(f'Turnstile verify failed: {e}', exc_info=True)
return self.http_status(500, -1, 'Internal server error')
@self.route('/<bot_uuid>/widget.js', methods=['GET'], auth_type=group.AuthType.NONE)
async def serve_widget(bot_uuid: str) -> quart.Response:
"""Serve the embed widget JavaScript with injected configuration."""
if not _is_valid_uuid(bot_uuid):
return self.http_status(400, -1, 'Invalid bot_uuid format')
runtime_bot, pipeline_uuid = self._resolve_bot(bot_uuid)
if runtime_bot is None:
return quart.Response(
'// Bot not found or not available', status=404, content_type='application/javascript'
)
try:
template = _get_widget_template()
except FileNotFoundError:
return quart.Response('// Widget template not found', status=404, content_type='application/javascript')
base_url = quart.request.host_url.rstrip('/')
webhook_prefix = self.ap.instance_config.data.get('api', {}).get('webhook_prefix', '')
if webhook_prefix:
base_url = webhook_prefix.rstrip('/')
if not re.match(r'^https?://[a-zA-Z0-9._:/-]+$', base_url):
base_url = quart.request.host_url.rstrip('/')
config = self._get_bot_config(bot_uuid)
site_key = config.get('turnstile_site_key', '')
locale = config.get('language', 'en_US') or 'en_US'
bubble_icon = config.get('bubble_icon', 'logo') or 'logo'
widget_js = template.replace('__LANGBOT_TURNSTILE_SITE_KEY__', site_key)
widget_js = widget_js.replace('__LANGBOT_BOT_UUID__', bot_uuid)
widget_js = widget_js.replace('__LANGBOT_BASE_URL__', base_url)
widget_js = widget_js.replace('__LANGBOT_LOCALE__', locale)
widget_js = widget_js.replace('__LANGBOT_BUBBLE_ICON__', bubble_icon)
response = quart.Response(widget_js, content_type='application/javascript; charset=utf-8')
response.headers['Cache-Control'] = 'public, max-age=300'
return response
@self.route('/logo', methods=['GET'], auth_type=group.AuthType.NONE)
async def serve_logo() -> quart.Response:
"""Serve the LangBot logo for the embed widget."""
try:
logo_data = _get_logo_bytes()
except FileNotFoundError:
return quart.Response('', status=404)
response = quart.Response(logo_data, content_type='image/webp')
response.headers['Cache-Control'] = 'public, max-age=86400'
return response
@self.route('/<bot_uuid>/messages/<session_type>', methods=['GET'], auth_type=group.AuthType.NONE)
async def get_embed_messages(bot_uuid: str, session_type: str) -> str:
if not _is_valid_uuid(bot_uuid):
return self.http_status(400, -1, 'Invalid bot_uuid format')
runtime_bot, pipeline_uuid = self._resolve_bot(bot_uuid)
if runtime_bot is None:
return self.http_status(404, -1, 'Bot not found or not available')
if not await self._verify_session_token(quart.request, bot_uuid):
return self.http_status(403, -1, 'Unauthorized or session expired')
try:
if session_type not in ['person', 'group']:
return self.http_status(400, -1, 'session_type must be person or group')
websocket_adapter = self.ap.platform_mgr.websocket_proxy_bot.adapter
if not websocket_adapter:
return self.http_status(404, -1, 'WebSocket adapter not found')
messages = websocket_adapter.get_websocket_messages(pipeline_uuid, session_type)
return self.success(data={'messages': messages})
except Exception as e:
logger.error(f'Failed to get embed messages: {e}', exc_info=True)
return self.http_status(500, -1, 'Internal server error')
@self.route('/<bot_uuid>/reset/<session_type>', methods=['POST'], auth_type=group.AuthType.NONE)
async def reset_embed_session(bot_uuid: str, session_type: str) -> str:
if not _is_valid_uuid(bot_uuid):
return self.http_status(400, -1, 'Invalid bot_uuid format')
runtime_bot, pipeline_uuid = self._resolve_bot(bot_uuid)
if runtime_bot is None:
return self.http_status(404, -1, 'Bot not found or not available')
if not await self._verify_session_token(quart.request, bot_uuid):
return self.http_status(403, -1, 'Unauthorized or session expired')
try:
if session_type not in ['person', 'group']:
return self.http_status(400, -1, 'session_type must be person or group')
websocket_adapter = self.ap.platform_mgr.websocket_proxy_bot.adapter
if not websocket_adapter:
return self.http_status(404, -1, 'WebSocket adapter not found')
websocket_adapter.reset_session(pipeline_uuid, session_type)
return self.success(data={'message': 'Session reset successfully'})
except Exception as e:
logger.error(f'Failed to reset embed session: {e}', exc_info=True)
return self.http_status(500, -1, 'Internal server error')
@self.route('/<bot_uuid>/feedback', methods=['POST'], auth_type=group.AuthType.NONE)
async def submit_feedback(bot_uuid: str) -> str:
if not _is_valid_uuid(bot_uuid):
return self.http_status(400, -1, 'Invalid bot_uuid format')
runtime_bot, pipeline_uuid = self._resolve_bot(bot_uuid)
if runtime_bot is None:
return self.http_status(404, -1, 'Bot not found or not available')
if not await self._verify_session_token(quart.request, bot_uuid):
return self.http_status(403, -1, 'Unauthorized or session expired')
try:
data = await quart.request.get_json()
message_id = data.get('message_id', '')
feedback_type = data.get('feedback_type')
if feedback_type not in (1, 2, 3):
return self.http_status(400, -1, 'feedback_type must be 1 (like), 2 (dislike), or 3 (cancel)')
feedback_id = f'embed_{uuid.uuid4().hex[:12]}'
await self.ap.monitoring_service.record_feedback(
feedback_id=feedback_id,
feedback_type=feedback_type,
bot_id=runtime_bot.bot_entity.uuid,
bot_name=runtime_bot.bot_entity.name or bot_uuid,
pipeline_id=pipeline_uuid,
message_id=str(message_id),
platform='web_page_bot',
)
return self.success(data={'feedback_id': feedback_id})
except Exception as e:
logger.error(f'Failed to record feedback: {e}', exc_info=True)
return self.http_status(500, -1, 'Internal server error')
# -- Embed WebSocket endpoint ----------------------------------------
@self.quart_app.websocket(self.path + '/<bot_uuid>/ws/connect')
async def embed_websocket_connect(bot_uuid: str):
"""WebSocket connection for embed widget, keyed by bot_uuid."""
if not _is_valid_uuid(bot_uuid):
await quart.websocket.send(json.dumps({'type': 'error', 'message': 'Invalid bot_uuid format'}))
return
runtime_bot, pipeline_uuid = self._resolve_bot(bot_uuid)
if runtime_bot is None:
await quart.websocket.send(json.dumps({'type': 'error', 'message': 'Bot not found or not available'}))
return
session_type = quart.websocket.args.get('session_type', 'person')
if session_type not in ['person', 'group']:
await quart.websocket.send(
json.dumps({'type': 'error', 'message': 'session_type must be person or group'})
)
return
websocket_adapter = self.ap.platform_mgr.websocket_proxy_bot.adapter
if not websocket_adapter:
await quart.websocket.send(json.dumps({'type': 'error', 'message': 'WebSocket adapter not found'}))
return
try:
connection = await ws_connection_manager.add_connection(
websocket=quart.websocket._get_current_object(),
pipeline_uuid=pipeline_uuid,
session_type=session_type,
metadata={'user_agent': quart.websocket.headers.get('User-Agent', '')},
)
await quart.websocket.send(
json.dumps(
{
'type': 'connected',
'connection_id': connection.connection_id,
'bot_uuid': bot_uuid,
'session_type': session_type,
'timestamp': connection.created_at.isoformat(),
}
)
)
logger.debug(
f'Embed WebSocket connected: {connection.connection_id} '
f'(bot={bot_uuid}, pipeline={pipeline_uuid}, session_type={session_type})'
)
receive_task = asyncio.create_task(self._handle_receive(connection, websocket_adapter, runtime_bot))
send_task = asyncio.create_task(self._handle_send(connection))
try:
await asyncio.gather(receive_task, send_task)
except Exception as e:
logger.error(f'Embed WebSocket task error: {e}')
finally:
await ws_connection_manager.remove_connection(connection.connection_id)
except Exception as e:
logger.error(f'Embed WebSocket connection error: {e}', exc_info=True)
try:
await quart.websocket.send(json.dumps({'type': 'error', 'message': 'Internal server error'}))
except Exception:
pass
# -- WebSocket receive/send helpers --------------------------------------
async def _handle_receive(self, connection, websocket_adapter, owner_bot):
try:
while connection.is_active:
message = await quart.websocket.receive()
await ws_connection_manager.update_activity(connection.connection_id)
try:
data = json.loads(message)
message_type = data.get('type', 'message')
if message_type == 'ping':
await connection.send_queue.put(
{'type': 'pong', 'timestamp': datetime.datetime.now().isoformat()}
)
elif message_type == 'message':
await websocket_adapter.handle_websocket_message(connection, data, owner_bot=owner_bot)
elif message_type == 'disconnect':
break
except json.JSONDecodeError:
await connection.send_queue.put({'type': 'error', 'message': 'Invalid JSON format'})
except Exception as e:
logger.error(f'Embed receive error: {e}', exc_info=True)
finally:
connection.is_active = False
async def _handle_send(self, connection):
try:
while connection.is_active:
try:
message = await asyncio.wait_for(connection.send_queue.get(), timeout=1.0)
await quart.websocket.send(json.dumps(message))
except asyncio.TimeoutError:
continue
except Exception as e:
logger.error(f'Embed send error: {e}', exc_info=True)
finally:
connection.is_active = False

View File

@@ -73,21 +73,15 @@ class PipelinesRouterGroup(group.RouterGroup):
plugins = await self.ap.plugin_connector.list_plugins(component_kinds=pipeline_component_kinds)
mcp_servers = await self.ap.mcp_service.get_mcp_servers(contain_runtime_info=True)
# Get available skills
available_skills = await self.ap.skill_service.list_skills()
extensions_prefs = pipeline.get('extensions_preferences', {})
return self.success(
data={
'enable_all_plugins': extensions_prefs.get('enable_all_plugins', True),
'enable_all_mcp_servers': extensions_prefs.get('enable_all_mcp_servers', True),
'enable_all_skills': extensions_prefs.get('enable_all_skills', True),
'bound_plugins': extensions_prefs.get('plugins', []),
'available_plugins': plugins,
'bound_mcp_servers': extensions_prefs.get('mcp_servers', []),
'available_mcp_servers': mcp_servers,
'bound_skills': extensions_prefs.get('skills', []),
'available_skills': available_skills,
}
)
elif quart.request.method == 'PUT':
@@ -95,19 +89,11 @@ class PipelinesRouterGroup(group.RouterGroup):
json_data = await quart.request.json
enable_all_plugins = json_data.get('enable_all_plugins', True)
enable_all_mcp_servers = json_data.get('enable_all_mcp_servers', True)
enable_all_skills = json_data.get('enable_all_skills', True)
bound_plugins = json_data.get('bound_plugins', [])
bound_mcp_servers = json_data.get('bound_mcp_servers', [])
bound_skills = json_data.get('bound_skills', [])
await self.ap.pipeline_service.update_pipeline_extensions(
pipeline_uuid,
bound_plugins,
bound_mcp_servers,
enable_all_plugins,
enable_all_mcp_servers,
bound_skills=bound_skills,
enable_all_skills=enable_all_skills,
pipeline_uuid, bound_plugins, bound_mcp_servers, enable_all_plugins, enable_all_mcp_servers
)
return self.success()

View File

@@ -43,13 +43,6 @@ class WebSocketChatRouterGroup(group.RouterGroup):
await quart.websocket.send(json.dumps({'type': 'error', 'message': 'WebSocket adapter not found'}))
return
# Dashboard pipeline-debug sessions must always run under the
# built-in websocket_proxy_bot identity. We deliberately do NOT
# resolve a web_page_bot owner here — even if one is bound to
# the same pipeline, debug requests must not be attributed to
# it. The embed widget path (`/api/v1/embed/<bot>/ws/connect`)
# is the one that carries the page-bot identity.
# 注册连接
connection = await ws_connection_manager.add_connection(
websocket=quart.websocket._get_current_object(),
@@ -210,9 +203,6 @@ class WebSocketChatRouterGroup(group.RouterGroup):
logger.debug(f'收到消息: {data} from {connection.connection_id}')
# 处理消息不等待响应响应会通过broadcast异步发送
# owner_bot is intentionally NOT passed: the dashboard
# debug WebSocket must always run under the proxy bot,
# never under a coincidentally-bound web_page_bot.
await websocket_adapter.handle_websocket_message(connection, data)
elif message_type == 'disconnect':

View File

@@ -1,6 +1,5 @@
import quart
import mimetypes
import asyncio
from ... import group
from langbot.pkg.utils import importutil
@@ -36,617 +35,3 @@ class AdaptersRouterGroup(group.RouterGroup):
return quart.Response(
importutil.read_resource_file_bytes(icon_path), mimetype=mimetypes.guess_type(icon_path)[0]
)
# In-memory session store for active registrations
_create_app_sessions: dict = {}
_SESSION_TTL = 900 # 15 minutes
def _cleanup_expired_sessions():
"""Remove sessions that have exceeded their TTL."""
import time
now = time.time()
expired = [sid for sid, s in _create_app_sessions.items() if now - s.get('created_at', 0) > _SESSION_TTL]
for sid in expired:
session = _create_app_sessions.pop(sid, None)
if session and session.get('task') and not session['task'].done():
session['task'].cancel()
@self.route('/lark/create-app', methods=['POST'])
async def _() -> str:
"""Start Feishu one-click app registration. Returns session_id + QR code URL."""
import uuid
import time
import lark_oapi as lark
from lark_oapi.scene.registration.errors import AppAccessDeniedError, AppExpiredError
_cleanup_expired_sessions()
session_id = str(uuid.uuid4())
loop = asyncio.get_running_loop()
session = {
'status': 'pending',
'qr_url': None,
'expire_at': None,
'app_id': None,
'app_secret': None,
'error': None,
'created_at': time.time(),
}
_create_app_sessions[session_id] = session
def on_qr_code(info):
# May be called from a background thread by the SDK;
# use call_soon_threadsafe to safely update session state.
def _update():
session['qr_url'] = info['url']
session['expire_at'] = time.time() + 600 # 10 minutes
session['status'] = 'waiting'
loop.call_soon_threadsafe(_update)
async def run_registration():
try:
result = await lark.aregister_app(
on_qr_code=on_qr_code,
source='langbot',
)
session['status'] = 'success'
session['app_id'] = result['client_id']
session['app_secret'] = result['client_secret']
except AppAccessDeniedError:
session['status'] = 'error'
session['error'] = 'User denied authorization'
except AppExpiredError:
session['status'] = 'error'
session['error'] = 'QR code expired'
except Exception as e:
session['status'] = 'error'
session['error'] = str(e)
task = asyncio.create_task(run_registration())
session['task'] = task
# Wait for QR code to be ready (max 10 seconds)
for _ in range(20):
if session['qr_url']:
break
await asyncio.sleep(0.5)
if not session['qr_url']:
task.cancel()
session['status'] = 'error'
session['error'] = 'Timeout waiting for QR code'
return self.http_status(504, -1, 'Timeout waiting for QR code')
return self.success(
data={
'session_id': session_id,
'qr_url': session['qr_url'],
'expire_at': session['expire_at'],
}
)
@self.route('/lark/create-app/status/<session_id>', methods=['GET'])
async def _(session_id: str) -> str:
"""Poll registration status."""
session = _create_app_sessions.get(session_id)
if not session:
return self.http_status(404, -1, 'Session not found')
data = {'status': session['status']}
if session['status'] == 'success':
data['app_id'] = session['app_id']
data['app_secret'] = session['app_secret']
_create_app_sessions.pop(session_id, None)
elif session['status'] == 'error':
data['error'] = session['error']
_create_app_sessions.pop(session_id, None)
return self.success(data=data)
@self.route('/lark/create-app/<session_id>', methods=['DELETE'])
async def _(session_id: str) -> str:
"""Cancel and clean up a registration session."""
session = _create_app_sessions.pop(session_id, None)
if session and session.get('task') and not session['task'].done():
session['task'].cancel()
return self.success(data={})
# -----------------------------------------------------------------------
# WeChat QR Code Login
# -----------------------------------------------------------------------
_weixin_login_sessions: dict = {}
_WEIXIN_SESSION_TTL = 600 # 10 minutes (3 retries × 3 min QR validity)
def _cleanup_expired_weixin_sessions():
import time
now = time.time()
expired = [
sid for sid, s in _weixin_login_sessions.items() if now - s.get('created_at', 0) > _WEIXIN_SESSION_TTL
]
for sid in expired:
session = _weixin_login_sessions.pop(sid, None)
if session and session.get('task') and not session['task'].done():
session['task'].cancel()
@self.route('/weixin/login', methods=['POST'])
async def _() -> str:
"""Start WeChat QR code login. Returns session_id + QR code data URL."""
import uuid
import time
from langbot.libs.openclaw_weixin_api.client import OpenClawWeixinClient, DEFAULT_BASE_URL
_cleanup_expired_weixin_sessions()
session_id = str(uuid.uuid4())
loop = asyncio.get_running_loop()
session = {
'status': 'pending',
'qr_data_url': None,
'expire_at': None,
'token': None,
'base_url': None,
'account_id': None,
'error': None,
'created_at': time.time(),
}
_weixin_login_sessions[session_id] = session
client = OpenClawWeixinClient(
base_url=DEFAULT_BASE_URL,
token='',
)
async def run_login():
try:
def on_qrcode(qr_data_url: str, _qr_url: str):
def _update():
session['qr_data_url'] = qr_data_url
session['expire_at'] = time.time() + 180
session['status'] = 'waiting'
loop.call_soon_threadsafe(_update)
result = await client.login(
max_retries=1,
poll_timeout_ms=180_000,
on_qrcode=on_qrcode,
)
session['status'] = 'success'
session['token'] = result.token
session['base_url'] = result.base_url
session['account_id'] = result.account_id
except Exception as e:
error_message = str(e)
if 'expired' in error_message.lower() or 'max retries exceeded' in error_message.lower():
session['status'] = 'expired'
session['error'] = 'QR code expired'
else:
session['status'] = 'error'
session['error'] = error_message
finally:
await client.close()
task = asyncio.create_task(run_login())
session['task'] = task
# Wait for QR code to be ready (max 10 seconds)
for _ in range(20):
if session['qr_data_url']:
break
await asyncio.sleep(0.5)
if not session['qr_data_url']:
task.cancel()
session['status'] = 'error'
session['error'] = 'Timeout waiting for QR code'
return self.http_status(504, -1, 'Timeout waiting for QR code')
return self.success(
data={
'session_id': session_id,
'qr_data_url': session['qr_data_url'],
'expire_at': session['expire_at'],
}
)
@self.route('/weixin/login/status/<session_id>', methods=['GET'])
async def _(session_id: str) -> str:
"""Poll WeChat login status."""
session = _weixin_login_sessions.get(session_id)
if not session:
return self.http_status(404, -1, 'Session not found')
data = {
'status': session['status'],
'qr_data_url': session['qr_data_url'],
'expire_at': session['expire_at'],
}
if session['status'] == 'success':
data['token'] = session['token']
data['base_url'] = session['base_url']
data['account_id'] = session['account_id']
_weixin_login_sessions.pop(session_id, None)
elif session['status'] == 'error':
data['error'] = session['error']
_weixin_login_sessions.pop(session_id, None)
elif session['status'] == 'expired':
data['error'] = session['error']
_weixin_login_sessions.pop(session_id, None)
return self.success(data=data)
@self.route('/weixin/login/<session_id>', methods=['DELETE'])
async def _(session_id: str) -> str:
"""Cancel and clean up a WeChat login session."""
session = _weixin_login_sessions.pop(session_id, None)
if session and session.get('task') and not session['task'].done():
session['task'].cancel()
return self.success(data={})
# -----------------------------------------------------------------------
# DingTalk Device Flow QR Code Login
# -----------------------------------------------------------------------
_dingtalk_sessions: dict = {}
_DINGTALK_SESSION_TTL = 600 # 10 minutes (QR code validity window)
def _cleanup_expired_dingtalk_sessions():
import time
now = time.time()
expired = [
sid for sid, s in _dingtalk_sessions.items() if now - s.get('created_at', 0) > _DINGTALK_SESSION_TTL
]
for sid in expired:
session = _dingtalk_sessions.pop(sid, None)
if session and session.get('task') and not session['task'].done():
session['task'].cancel()
@self.route('/dingtalk/create-app', methods=['POST'])
async def _() -> str:
"""Start DingTalk one-click app creation via Device Flow. Returns session_id + QR code URL."""
import uuid
import time
import aiohttp
DINGTALK_BASE_URL = 'https://oapi.dingtalk.com'
_cleanup_expired_dingtalk_sessions()
session_id = str(uuid.uuid4())
session = {
'status': 'pending',
'qr_url': None,
'expire_at': None,
'client_id': None,
'client_secret': None,
'error': None,
'created_at': time.time(),
'device_code': None,
'interval': 5,
}
_dingtalk_sessions[session_id] = session
async def run_device_flow():
try:
timeout = aiohttp.ClientTimeout(total=10)
async with aiohttp.ClientSession(timeout=timeout) as http:
# Step 1: Init — get nonce
async with http.post(
f'{DINGTALK_BASE_URL}/app/registration/init',
json={'source': 'langbot'},
) as resp:
try:
data = await resp.json()
except (aiohttp.ContentTypeError, ValueError):
session['status'] = 'error'
session['error'] = 'Invalid response from DingTalk service'
return
if data.get('errcode', -1) != 0:
session['status'] = 'error'
session['error'] = data.get('errmsg', 'Failed to init')
return
nonce = data['nonce']
# Step 2: Begin — get device_code + QR URL
async with http.post(
f'{DINGTALK_BASE_URL}/app/registration/begin',
json={'nonce': nonce},
) as resp:
try:
data = await resp.json()
except (aiohttp.ContentTypeError, ValueError):
session['status'] = 'error'
session['error'] = 'Invalid response from DingTalk service'
return
if data.get('errcode', -1) != 0:
session['status'] = 'error'
session['error'] = data.get('errmsg', 'Failed to begin authorization')
return
device_code = data['device_code']
verification_uri_complete = data.get('verification_uri_complete', '')
expires_in = data.get('expires_in', 7200)
interval = data.get('interval', 5)
session['device_code'] = device_code
session['interval'] = interval
session['qr_url'] = verification_uri_complete
session['expire_at'] = time.time() + 600 # QR code valid for ~10 min
session['status'] = 'waiting'
# Step 3: Poll for authorization result
deadline = time.time() + expires_in
while time.time() < deadline:
await asyncio.sleep(interval)
async with http.post(
f'{DINGTALK_BASE_URL}/app/registration/poll',
json={'device_code': device_code},
) as poll_resp:
try:
poll_data = await poll_resp.json()
except (aiohttp.ContentTypeError, ValueError):
continue
if poll_data.get('errcode', -1) != 0:
session['status'] = 'error'
session['error'] = poll_data.get('errmsg', 'Poll failed')
return
status = poll_data.get('status', '')
if status == 'SUCCESS':
session['status'] = 'success'
session['client_id'] = poll_data.get('client_id', '')
session['client_secret'] = poll_data.get('client_secret', '')
return
elif status == 'FAIL':
session['status'] = 'error'
session['error'] = poll_data.get('fail_reason', 'Authorization failed')
return
elif status == 'EXPIRED':
session['status'] = 'error'
session['error'] = 'QR code expired'
return
# status == 'WAITING': continue polling
# Timeout
session['status'] = 'error'
session['error'] = 'QR code expired'
except asyncio.CancelledError:
return
except Exception as e:
session['status'] = 'error'
session['error'] = str(e)
task = asyncio.create_task(run_device_flow())
session['task'] = task
# Wait for QR code to be ready (max 10 seconds)
for _ in range(20):
if session['qr_url'] or session['error']:
break
await asyncio.sleep(0.5)
if session['error']:
task.cancel()
return self.http_status(502, -1, session['error'])
if not session['qr_url']:
task.cancel()
session['status'] = 'error'
session['error'] = 'Timeout waiting for QR code'
return self.http_status(504, -1, 'Timeout waiting for QR code')
return self.success(
data={
'session_id': session_id,
'qr_url': session['qr_url'],
'expire_at': session['expire_at'],
}
)
@self.route('/dingtalk/create-app/status/<session_id>', methods=['GET'])
async def _(session_id: str) -> str:
"""Poll DingTalk Device Flow status."""
_cleanup_expired_dingtalk_sessions()
session = _dingtalk_sessions.get(session_id)
if not session:
return self.http_status(404, -1, 'Session not found')
data = {'status': session['status']}
if session['status'] == 'success':
data['client_id'] = session['client_id']
data['client_secret'] = session['client_secret']
_dingtalk_sessions.pop(session_id, None)
elif session['status'] == 'error':
data['error'] = session['error']
_dingtalk_sessions.pop(session_id, None)
return self.success(data=data)
@self.route('/dingtalk/create-app/<session_id>', methods=['DELETE'])
async def _(session_id: str) -> str:
"""Cancel and clean up a DingTalk Device Flow session."""
session = _dingtalk_sessions.pop(session_id, None)
if session and session.get('task') and not session['task'].done():
session['task'].cancel()
return self.success(data={})
# -----------------------------------------------------------------------
# WeComBot QR Code One-Click Create
# -----------------------------------------------------------------------
_wecombot_sessions: dict = {}
_WECOMBOT_SESSION_TTL = 300 # 5 minutes (WeCom QR validity window)
def _cleanup_expired_wecombot_sessions():
import time
now = time.time()
expired = [
sid for sid, s in _wecombot_sessions.items() if now - s.get('created_at', 0) > _WECOMBOT_SESSION_TTL
]
for sid in expired:
session = _wecombot_sessions.pop(sid, None)
if session and session.get('task') and not session['task'].done():
session['task'].cancel()
@self.route('/wecombot/create-bot', methods=['POST'])
async def _() -> str:
"""Start WeComBot one-click creation via QR code. Returns session_id + QR code URL."""
import uuid
import time
import aiohttp
WECOM_QC_GENERATE_URL = 'https://work.weixin.qq.com/ai/qc/generate'
WECOM_QC_QUERY_URL = 'https://work.weixin.qq.com/ai/qc/query_result'
_cleanup_expired_wecombot_sessions()
session_id = str(uuid.uuid4())
session = {
'status': 'pending',
'qr_url': None,
'expire_at': None,
'botid': None,
'secret': None,
'error': None,
'created_at': time.time(),
'scode': None,
'task': None,
}
_wecombot_sessions[session_id] = session
async def run_qr_flow():
try:
timeout = aiohttp.ClientTimeout(total=10)
async with aiohttp.ClientSession(timeout=timeout) as http:
# Step 1: Generate QR code
async with http.get(
f'{WECOM_QC_GENERATE_URL}?source=langbot&plat=0',
) as resp:
try:
data = await resp.json()
except (aiohttp.ContentTypeError, ValueError):
session['status'] = 'error'
session['error'] = 'Invalid response from WeCom service'
return
if not data.get('data', {}).get('scode') or not data.get('data', {}).get('auth_url'):
session['status'] = 'error'
session['error'] = data.get('errmsg', 'Failed to generate QR code')
return
scode = data['data']['scode']
auth_url = data['data']['auth_url']
session['scode'] = scode
session['qr_url'] = auth_url
session['expire_at'] = time.time() + _WECOMBOT_SESSION_TTL
session['status'] = 'waiting'
# Step 2: Poll for scan result
deadline = time.time() + _WECOMBOT_SESSION_TTL
while time.time() < deadline:
await asyncio.sleep(3)
async with http.get(
f'{WECOM_QC_QUERY_URL}?scode={scode}',
) as poll_resp:
try:
poll_data = await poll_resp.json()
except (aiohttp.ContentTypeError, ValueError):
continue
status = poll_data.get('data', {}).get('status', '')
if status == 'success':
bot_info = poll_data.get('data', {}).get('bot_info', {})
if bot_info.get('botid') and bot_info.get('secret'):
session['status'] = 'success'
session['botid'] = bot_info['botid']
session['secret'] = bot_info['secret']
return
else:
session['status'] = 'error'
session['error'] = 'Scan succeeded but bot info is incomplete'
return
# Timeout
session['status'] = 'error'
session['error'] = 'QR code expired'
except asyncio.CancelledError:
return
except Exception as e:
session['status'] = 'error'
session['error'] = str(e)
task = asyncio.create_task(run_qr_flow())
session['task'] = task
# Wait for QR code to be ready (max 10 seconds)
for _ in range(20):
if session['qr_url'] or session['error']:
break
await asyncio.sleep(0.5)
if session['error']:
task.cancel()
return self.http_status(502, -1, session['error'])
if not session['qr_url']:
task.cancel()
session['status'] = 'error'
session['error'] = 'Timeout waiting for QR code'
return self.http_status(504, -1, 'Timeout waiting for QR code')
return self.success(
data={
'session_id': session_id,
'qr_url': session['qr_url'],
'expire_at': session['expire_at'],
}
)
@self.route('/wecombot/create-bot/status/<session_id>', methods=['GET'])
async def _(session_id: str) -> str:
"""Poll WeComBot creation status."""
_cleanup_expired_wecombot_sessions()
session = _wecombot_sessions.get(session_id)
if not session:
return self.http_status(404, -1, 'Session not found')
data = {'status': session['status']}
if session['status'] == 'success':
data['botid'] = session['botid']
data['secret'] = session['secret']
_wecombot_sessions.pop(session_id, None)
elif session['status'] == 'error':
data['error'] = session['error']
_wecombot_sessions.pop(session_id, None)
return self.success(data=data)
@self.route('/wecombot/create-bot/<session_id>', methods=['DELETE'])
async def _(session_id: str) -> str:
"""Cancel and clean up a WeComBot creation session."""
session = _wecombot_sessions.pop(session_id, None)
if session and session.get('task') and not session['task'].done():
session['task'].cancel()
return self.success(data={})

View File

@@ -1,153 +1,19 @@
from __future__ import annotations
import base64
import io
import quart
import re
import httpx
import uuid
import os
import zipfile
import yaml
from urllib.parse import urlparse
import posixpath
import sqlalchemy
from .....core import taskmgr
from .....entity.persistence import plugin as persistence_plugin
from .. import group
from langbot_plugin.runtime.plugin.mgr import PluginInstallSource
# Resolve the built-in page SDK JS from the langbot_plugin package
_PAGE_SDK_PATH = None
try:
import langbot_plugin.assets as _assets_pkg
_candidate = os.path.join(os.path.dirname(_assets_pkg.__file__), 'langbot-page-sdk.js')
if os.path.exists(_candidate):
_PAGE_SDK_PATH = _candidate
except Exception:
pass
def _normalize_plugin_asset_path(filepath: str) -> str | None:
filepath = filepath.replace('\\', '/')
if filepath.startswith('/'):
return None
normalized = posixpath.normpath(filepath)
if normalized == '.' or normalized.startswith('../') or normalized == '..':
return None
if normalized.startswith('components/pages/'):
return normalized
return f'assets/{normalized}'
def _get_request_origin() -> str:
"""Return the public request origin, respecting reverse-proxy headers."""
forwarded_proto = quart.request.headers.get('X-Forwarded-Proto', '').split(',')[0].strip()
forwarded_host = quart.request.headers.get('X-Forwarded-Host', '').split(',')[0].strip()
scheme = forwarded_proto or quart.request.scheme
host = forwarded_host or quart.request.host
return f'{scheme}://{host}'
@group.group_class('plugins', '/api/v1/plugins')
class PluginsRouterGroup(group.RouterGroup):
@staticmethod
def _normalize_archive_path(path: str) -> str:
normalized = str(path or '').replace('\\', '/').strip('/')
return posixpath.normpath(normalized) if normalized else ''
@classmethod
def _component_source_path(cls, entry) -> str:
if isinstance(entry, dict):
return cls._normalize_archive_path(entry.get('path') or '')
return cls._normalize_archive_path(str(entry or ''))
@classmethod
def _count_component_configs(cls, component_config, archive_names: list[str]) -> int:
normalized_names = [cls._normalize_archive_path(name) for name in archive_names]
component_files: set[str] = set()
if isinstance(component_config, list):
return len(component_config)
if not isinstance(component_config, dict):
return 1 if component_config else 0
for entry in component_config.get('fromFiles') or []:
source_path = cls._component_source_path(entry)
if source_path and source_path in normalized_names:
component_files.add(source_path)
for entry in component_config.get('fromDirs') or []:
source_dir = cls._component_source_path(entry).rstrip('/')
if not source_dir:
continue
prefix = f'{source_dir}/'
for archive_name in normalized_names:
if not archive_name.startswith(prefix):
continue
if archive_name.lower().endswith(('.yaml', '.yml')):
component_files.add(archive_name)
if component_files:
return len(component_files)
return 1 if any(key in component_config for key in ('path', 'name', 'kind')) else 0
@classmethod
def _count_plugin_components(cls, components, archive_names: list[str]) -> dict[str, int]:
if not isinstance(components, dict):
return {}
component_counts: dict[str, int] = {}
for kind, component_config in components.items():
count = cls._count_component_configs(component_config, archive_names)
if count > 0:
component_counts[str(kind)] = count
return component_counts
@staticmethod
def _parse_github_repo_url(repo_url: str) -> dict | None:
raw_url = str(repo_url or '').strip()
if not raw_url:
return None
if not re.match(r'^[a-zA-Z][a-zA-Z0-9+.-]*://', raw_url):
raw_url = f'https://{raw_url}'
parsed = urlparse(raw_url)
if parsed.netloc.lower() not in ('github.com', 'www.github.com'):
return None
parts = [part for part in parsed.path.strip('/').split('/') if part]
if len(parts) < 2:
return None
owner = parts[0]
repo = parts[1]
if repo.endswith('.git'):
repo = repo[:-4]
if not owner or not repo:
return None
ref = ''
subdir = ''
if len(parts) >= 4 and parts[2] in ('tree', 'blob'):
ref = parts[3]
subdir = '/'.join(parts[4:]).strip('/')
return {
'owner': owner,
'repo': repo,
'ref': ref,
'subdir': subdir,
}
async def _check_extensions_limit(self) -> str | None:
"""Check if extensions limit is reached. Returns error response if limit exceeded, None otherwise."""
limitation = self.ap.instance_config.data.get('system', {}).get('limitation', {})
@@ -161,15 +27,6 @@ class PluginsRouterGroup(group.RouterGroup):
return None
async def initialize(self) -> None:
@self.route('/_sdk/page-sdk.js', methods=['GET'], auth_type=group.AuthType.NONE)
async def _() -> quart.Response:
"""Serve the built-in LangBot page SDK JavaScript."""
if _PAGE_SDK_PATH and os.path.exists(_PAGE_SDK_PATH):
with open(_PAGE_SDK_PATH, 'r') as f:
content = f.read()
return quart.Response(content, mimetype='application/javascript')
return quart.Response('// SDK not found', status=404, mimetype='application/javascript')
@self.route('', methods=['GET'], auth_type=group.AuthType.USER_TOKEN_OR_API_KEY)
async def _() -> str:
plugins = await self.ap.plugin_connector.list_plugins()
@@ -245,15 +102,7 @@ class PluginsRouterGroup(group.RouterGroup):
return self.http_status(404, -1, 'plugin not found')
if quart.request.method == 'GET':
result = await self.ap.persistence_mgr.execute_async(
sqlalchemy.select(persistence_plugin.PluginSetting.config)
.where(persistence_plugin.PluginSetting.plugin_author == author)
.where(persistence_plugin.PluginSetting.plugin_name == plugin_name)
)
persisted_config = result.scalar_one_or_none()
config = persisted_config if persisted_config is not None else plugin['plugin_config']
return self.success(data={'config': config})
return self.success(data={'config': plugin['plugin_config']})
elif quart.request.method == 'PUT':
data = await quart.request.json
@@ -286,62 +135,15 @@ class PluginsRouterGroup(group.RouterGroup):
return quart.Response(icon_data, mimetype=mime_type)
@self.route(
'/<author>/<plugin_name>/assets/<path:filepath>',
'/<author>/<plugin_name>/assets/<filepath>',
methods=['GET'],
auth_type=group.AuthType.NONE,
)
async def _(author: str, plugin_name: str, filepath: str) -> quart.Response:
asset_path = _normalize_plugin_asset_path(filepath)
if asset_path is None:
return quart.Response('Asset not found', status=404)
asset_data = await self.ap.plugin_connector.get_plugin_assets(author, plugin_name, asset_path)
if not asset_data.get('asset_base64'):
return quart.Response('Asset not found', status=404)
asset_data = await self.ap.plugin_connector.get_plugin_assets(author, plugin_name, filepath)
asset_bytes = base64.b64decode(asset_data['asset_base64'])
mime_type = asset_data['mime_type']
resp = quart.Response(asset_bytes, mimetype=mime_type)
# CSP for HTML pages served to sandboxed iframes (opaque origin).
# 'self' doesn't work in sandboxed iframes — use actual server origin.
if mime_type and mime_type.startswith('text/html'):
origin = _get_request_origin()
resp.headers['Content-Security-Policy'] = (
f'default-src {origin}; '
f"script-src {origin} 'unsafe-inline'; "
f"style-src {origin} 'unsafe-inline'; "
f'img-src {origin} data:; '
f'connect-src {origin}; '
"frame-src 'none'; "
"object-src 'none'"
)
return resp
@self.route(
'/<author>/<plugin_name>/page-api',
methods=['POST'],
auth_type=group.AuthType.USER_TOKEN_OR_API_KEY,
)
async def _(author: str, plugin_name: str) -> str:
"""Forward a page API request to the plugin."""
data = await quart.request.json
if not isinstance(data, dict):
return self.http_status(400, -1, 'invalid request body')
page_id = data.get('page_id', '')
endpoint = data.get('endpoint', '')
method = data.get('method', 'POST')
body = data.get('body')
if not isinstance(page_id, str) or not isinstance(endpoint, str) or not isinstance(method, str):
return self.http_status(400, -1, 'invalid page api request')
if not endpoint.startswith('/') or '..' in endpoint:
return self.http_status(400, -1, 'invalid endpoint')
result = await self.ap.plugin_connector.handle_page_api(
author, plugin_name, page_id, endpoint, method.upper(), body
)
if result.get('error'):
return self.http_status(400, -1, result['error'])
return self.success(data=result.get('data'))
return quart.Response(asset_bytes, mimetype=mime_type)
@self.route('/github/releases', methods=['POST'], auth_type=group.AuthType.USER_TOKEN_OR_API_KEY)
async def _() -> str:
@@ -349,37 +151,17 @@ class PluginsRouterGroup(group.RouterGroup):
data = await quart.request.json
repo_url = data.get('repo_url', '')
parsed_repo = self._parse_github_repo_url(repo_url)
if not parsed_repo:
# Parse GitHub repository URL to extract owner and repo
# Supports: https://github.com/owner/repo or github.com/owner/repo
pattern = r'github\.com/([^/]+)/([^/]+?)(?:\.git)?(?:/.*)?$'
match = re.search(pattern, repo_url)
if not match:
return self.http_status(400, -1, 'Invalid GitHub repository URL')
owner = parsed_repo['owner']
repo = parsed_repo['repo']
requested_ref = parsed_repo['ref']
requested_subdir = parsed_repo['subdir']
owner, repo = match.groups()
try:
if requested_ref:
return self.success(
data={
'releases': [
{
'id': 0,
'tag_name': requested_ref,
'name': requested_ref,
'published_at': '',
'prerelease': False,
'draft': False,
'source_type': 'branch',
'archive_url': f'https://api.github.com/repos/{owner}/{repo}/zipball/{requested_ref}',
}
],
'owner': owner,
'repo': repo,
'source_subdir': requested_subdir,
}
)
# Fetch releases from GitHub API
url = f'https://api.github.com/repos/{owner}/{repo}/releases'
async with httpx.AsyncClient(
@@ -405,14 +187,7 @@ class PluginsRouterGroup(group.RouterGroup):
}
)
return self.success(
data={
'releases': formatted_releases,
'owner': owner,
'repo': repo,
'source_subdir': requested_subdir,
}
)
return self.success(data={'releases': formatted_releases, 'owner': owner, 'repo': repo})
except httpx.RequestError as e:
return self.http_status(500, -1, f'Failed to fetch releases: {str(e)}')
@@ -490,8 +265,6 @@ class PluginsRouterGroup(group.RouterGroup):
return self.http_status(400, -1, 'Missing asset_url parameter')
ctx = taskmgr.TaskContext.new()
ctx.metadata['plugin_name'] = f'{owner}/{repo}'
ctx.metadata['install_source'] = 'github'
install_info = {
'asset_url': asset_url,
'owner': owner,
@@ -522,17 +295,12 @@ class PluginsRouterGroup(group.RouterGroup):
data = await quart.request.json
plugin_author = data.get('plugin_author', '')
plugin_name = data.get('plugin_name', '')
ctx = taskmgr.TaskContext.new()
ctx.metadata['plugin_name'] = f'{plugin_author}/{plugin_name}'
ctx.metadata['install_source'] = 'marketplace'
wrapper = self.ap.task_mgr.create_user_task(
self.ap.plugin_connector.install_plugin(PluginInstallSource.MARKETPLACE, data, task_context=ctx),
kind='plugin-operation',
name='plugin-install-marketplace',
label=f'Installing plugin from marketplace {plugin_author}/{plugin_name}',
label=f'Installing plugin from marketplace ...{data}',
context=ctx,
)
@@ -555,74 +323,16 @@ class PluginsRouterGroup(group.RouterGroup):
}
ctx = taskmgr.TaskContext.new()
ctx.metadata['plugin_name'] = file.filename or 'local plugin'
ctx.metadata['install_source'] = 'local'
wrapper = self.ap.task_mgr.create_user_task(
self.ap.plugin_connector.install_plugin(PluginInstallSource.LOCAL, data, task_context=ctx),
kind='plugin-operation',
name='plugin-install-local',
label=f'Installing plugin from local {file.filename}',
label=f'Installing plugin from local ...{file.filename}',
context=ctx,
)
return self.success(data={'task_id': wrapper.id})
@self.route('/install/local/preview', methods=['POST'], auth_type=group.AuthType.USER_TOKEN_OR_API_KEY)
async def _() -> str:
file = (await quart.request.files).get('file')
if file is None:
return self.http_status(400, -1, 'file is required')
file_bytes = file.read()
try:
with zipfile.ZipFile(io.BytesIO(file_bytes)) as zf:
names = [name for name in zf.namelist() if not name.endswith('/')]
manifest_name = next(
(
name
for name in names
if name.replace('\\', '/').strip('/').lower() in ('manifest.yaml', 'manifest.yml')
),
None,
)
if manifest_name is None:
return self.http_status(400, -1, 'manifest.yaml is required')
manifest = yaml.safe_load(zf.read(manifest_name).decode('utf-8')) or {}
requirements: list[str] = []
requirements_name = next(
(name for name in names if name.replace('\\', '/').strip('/').lower() == 'requirements.txt'),
None,
)
if requirements_name is not None:
requirements = [
line.strip()
for line in zf.read(requirements_name).decode('utf-8', errors='ignore').splitlines()
if line.strip() and not line.strip().startswith('#')
]
spec = manifest.get('spec') or {}
components = spec.get('components') or {}
component_counts = self._count_plugin_components(components, names)
component_types = list(component_counts.keys())
return self.success(
data={
'filename': file.filename or 'local plugin',
'size': len(file_bytes),
'manifest': manifest,
'metadata': manifest.get('metadata') or {},
'component_types': component_types,
'component_counts': component_counts,
'requirements': requirements,
'file_count': len(names),
}
)
except zipfile.BadZipFile:
return self.http_status(400, -1, 'invalid .lbpkg file')
except Exception as exc:
return self.http_status(500, -1, f'Failed to preview plugin package: {exc}')
@self.route('/config-files', methods=['POST'], auth_type=group.AuthType.USER_TOKEN)
async def _() -> str:
"""Upload a file for plugin configuration"""

View File

@@ -97,51 +97,3 @@ class EmbeddingModelsRouterGroup(group.RouterGroup):
await self.ap.embedding_models_service.test_embedding_model(model_uuid, json_data)
return self.success()
@group.group_class('models/rerank', '/api/v1/provider/models/rerank')
class RerankModelsRouterGroup(group.RouterGroup):
async def initialize(self) -> None:
@self.route('', methods=['GET', 'POST'], auth_type=group.AuthType.USER_TOKEN_OR_API_KEY)
async def _() -> str:
if quart.request.method == 'GET':
provider_uuid = quart.request.args.get('provider_uuid')
if provider_uuid:
return self.success(
data={
'models': await self.ap.rerank_models_service.get_rerank_models_by_provider(provider_uuid)
}
)
return self.success(data={'models': await self.ap.rerank_models_service.get_rerank_models()})
elif quart.request.method == 'POST':
json_data = await quart.request.json
model_uuid = await self.ap.rerank_models_service.create_rerank_model(json_data)
return self.success(data={'uuid': model_uuid})
@self.route('/<model_uuid>', methods=['GET', 'PUT', 'DELETE'], auth_type=group.AuthType.USER_TOKEN_OR_API_KEY)
async def _(model_uuid: str) -> str:
if quart.request.method == 'GET':
model = await self.ap.rerank_models_service.get_rerank_model(model_uuid)
if model is None:
return self.http_status(404, -1, 'model not found')
return self.success(data={'model': model})
elif quart.request.method == 'PUT':
json_data = await quart.request.json
await self.ap.rerank_models_service.update_rerank_model(model_uuid, json_data)
return self.success()
elif quart.request.method == 'DELETE':
await self.ap.rerank_models_service.delete_rerank_model(model_uuid)
return self.success()
@self.route('/<model_uuid>/test', methods=['POST'], auth_type=group.AuthType.USER_TOKEN_OR_API_KEY)
async def _(model_uuid: str) -> str:
json_data = await quart.request.json
await self.ap.rerank_models_service.test_rerank_model(model_uuid, json_data)
return self.success()

View File

@@ -15,7 +15,6 @@ class ModelProvidersRouterGroup(group.RouterGroup):
counts = await self.ap.provider_service.get_provider_model_counts(provider['uuid'])
provider['llm_count'] = counts['llm_count']
provider['embedding_count'] = counts['embedding_count']
provider['rerank_count'] = counts['rerank_count']
return self.success(data={'providers': providers})
elif quart.request.method == 'POST':
json_data = await quart.request.json
@@ -33,7 +32,6 @@ class ModelProvidersRouterGroup(group.RouterGroup):
counts = await self.ap.provider_service.get_provider_model_counts(provider_uuid)
provider['llm_count'] = counts['llm_count']
provider['embedding_count'] = counts['embedding_count']
provider['rerank_count'] = counts['rerank_count']
return self.success(data={'provider': provider})
elif quart.request.method == 'PUT':
json_data = await quart.request.json
@@ -45,12 +43,3 @@ class ModelProvidersRouterGroup(group.RouterGroup):
return self.success()
except ValueError as e:
return self.http_status(400, -1, str(e))
@self.route('/<provider_uuid>/scan-models', methods=['GET'], auth_type=group.AuthType.USER_TOKEN_OR_API_KEY)
async def _(provider_uuid: str) -> str:
try:
model_type = quart.request.args.get('type')
result = await self.ap.provider_service.scan_provider_models(provider_uuid, model_type)
return self.success(data=result)
except ValueError as e:
return self.http_status(400, -1, str(e))

View File

@@ -31,9 +31,6 @@ class MCPRouterGroup(group.RouterGroup):
@self.route('/servers/<server_name>', methods=['GET', 'PUT', 'DELETE'], auth_type=group.AuthType.USER_TOKEN)
async def _(server_name: str) -> str:
"""获取、更新或删除MCP服务器配置"""
from urllib.parse import unquote
server_name = unquote(server_name)
server_data = await self.ap.mcp_service.get_mcp_server_by_name(server_name)
if server_data is None:
@@ -60,9 +57,6 @@ class MCPRouterGroup(group.RouterGroup):
@self.route('/servers/<server_name>/test', methods=['POST'], auth_type=group.AuthType.USER_TOKEN)
async def _(server_name: str) -> str:
"""测试MCP服务器连接"""
from urllib.parse import unquote
server_name = unquote(server_name)
server_data = await quart.request.json
task_id = await self.ap.mcp_service.test_mcp_server(server_name=server_name, server_data=server_data)
return self.success(data={'task_id': task_id})

View File

@@ -1,45 +0,0 @@
from __future__ import annotations
from ... import group
@group.group_class('tools', '/api/v1/tools')
class ToolsRouterGroup(group.RouterGroup):
async def initialize(self) -> None:
@self.route('', methods=['GET'], auth_type=group.AuthType.USER_TOKEN)
async def _() -> str:
"""获取所有可用工具列表"""
tools = await self.ap.tool_mgr.get_all_tools()
tool_list = []
for tool in tools:
tool_list.append(
{
'name': tool.name,
'description': tool.description,
'human_desc': tool.human_desc,
'parameters': tool.parameters,
}
)
return self.success(data={'tools': tool_list})
@self.route('/<tool_name>', methods=['GET'], auth_type=group.AuthType.USER_TOKEN)
async def _(tool_name: str) -> str:
"""获取特定工具详情"""
tools = await self.ap.tool_mgr.get_all_tools()
for tool in tools:
if tool.name == tool_name:
return self.success(
data={
'tool': {
'name': tool.name,
'description': tool.description,
'human_desc': tool.human_desc,
'parameters': tool.parameters,
}
}
)
return self.http_status(404, -1, f'Tool not found: {tool_name}')

View File

@@ -1,190 +0,0 @@
from __future__ import annotations
import quart
from langbot_plugin.box.errors import BoxError
from .. import group
@group.group_class('skills', '/api/v1/skills')
class SkillsRouterGroup(group.RouterGroup):
"""Skills management API endpoints."""
async def initialize(self) -> None:
@self.route('', methods=['GET', 'POST'], auth_type=group.AuthType.USER_TOKEN_OR_API_KEY)
async def list_or_create_skills() -> quart.Response:
if quart.request.method == 'GET':
try:
skills = await self.ap.skill_service.list_skills()
except (ValueError, BoxError) as exc:
return self.http_status(400, -1, str(exc))
return self.success(data={'skills': skills})
data = await quart.request.json
if 'name' not in data or not data['name']:
return self.http_status(400, -1, 'Missing required field: name')
try:
skill = await self.ap.skill_service.create_skill(data)
return self.success(data={'skill': skill})
except (ValueError, BoxError) as exc:
return self.http_status(400, -1, str(exc))
@self.route('/<skill_name>', methods=['GET', 'PUT', 'DELETE'], auth_type=group.AuthType.USER_TOKEN_OR_API_KEY)
async def get_update_delete_skill(skill_name: str) -> quart.Response:
if quart.request.method == 'GET':
try:
skill = await self.ap.skill_service.get_skill(skill_name)
except (ValueError, BoxError) as exc:
return self.http_status(400, -1, str(exc))
if not skill:
return self.http_status(404, -1, 'Skill not found')
return self.success(data={'skill': skill})
if quart.request.method == 'PUT':
data = await quart.request.json
try:
skill = await self.ap.skill_service.update_skill(skill_name, data)
return self.success(data={'skill': skill})
except (ValueError, BoxError) as exc:
return self.http_status(400, -1, str(exc))
try:
await self.ap.skill_service.delete_skill(skill_name)
return self.success()
except (ValueError, BoxError) as exc:
return self.http_status(400, -1, str(exc))
@self.route('/<skill_name>/files', methods=['GET'], auth_type=group.AuthType.USER_TOKEN_OR_API_KEY)
async def list_skill_files(skill_name: str) -> quart.Response:
"""List files in skill package directory."""
path = quart.request.args.get('path', '.').strip()
include_hidden = quart.request.args.get('include_hidden', 'false').lower() == 'true'
try:
result = await self.ap.skill_service.list_skill_files(
skill_name,
path=path,
include_hidden=include_hidden,
)
return self.success(data=result)
except (ValueError, BoxError) as exc:
return self.http_status(400, -1, str(exc))
@self.route(
'/<skill_name>/files/<path:path>', methods=['GET', 'PUT'], auth_type=group.AuthType.USER_TOKEN_OR_API_KEY
)
async def read_or_write_skill_file(skill_name: str, path: str) -> quart.Response:
"""Read or write a file in skill package."""
if quart.request.method == 'GET':
try:
result = await self.ap.skill_service.read_skill_file(skill_name, path)
return self.success(data=result)
except (ValueError, BoxError) as exc:
return self.http_status(400, -1, str(exc))
# PUT - write file
data = await quart.request.json
content = data.get('content', '')
if content is None:
return self.http_status(400, -1, 'Missing required field: content')
try:
result = await self.ap.skill_service.write_skill_file(skill_name, path, content)
return self.success(data=result)
except (ValueError, BoxError) as exc:
return self.http_status(400, -1, str(exc))
@self.route('/<skill_name>/preview', methods=['GET'], auth_type=group.AuthType.USER_TOKEN_OR_API_KEY)
async def preview_skill(skill_name: str) -> quart.Response:
skill = self.ap.skill_mgr.get_skill_by_name(skill_name)
if not skill:
return self.http_status(404, -1, 'Skill not found')
return self.success(data={'instructions': skill.get('instructions', '')})
@self.route('/install/github', methods=['POST'], auth_type=group.AuthType.USER_TOKEN_OR_API_KEY)
async def install_skill_from_github() -> quart.Response:
data = await quart.request.json
required_fields = ['asset_url', 'owner', 'repo']
for field in required_fields:
if field not in data or not data[field]:
return self.http_status(400, -1, f'Missing required field: {field}')
asset_url = str(data['asset_url']).strip().lower().split('?', 1)[0].split('#', 1)[0]
if not asset_url.endswith('skill.md') and not data.get('release_tag'):
return self.http_status(400, -1, 'Missing required field: release_tag')
try:
skill = await self.ap.skill_service.install_from_github(data)
return self.success(data={'skills': skill})
except (ValueError, BoxError) as exc:
return self.http_status(400, -1, str(exc))
except Exception as exc:
return self.http_status(500, -1, f'Failed to install skill: {exc}')
@self.route('/install/github/preview', methods=['POST'], auth_type=group.AuthType.USER_TOKEN_OR_API_KEY)
async def preview_skill_from_github() -> quart.Response:
data = await quart.request.json
required_fields = ['asset_url', 'owner', 'repo']
for field in required_fields:
if field not in data or not data[field]:
return self.http_status(400, -1, f'Missing required field: {field}')
asset_url = str(data['asset_url']).strip().lower().split('?', 1)[0].split('#', 1)[0]
if not asset_url.endswith('skill.md') and not data.get('release_tag'):
return self.http_status(400, -1, 'Missing required field: release_tag')
try:
preview = await self.ap.skill_service.preview_install_from_github(data)
return self.success(data={'skills': preview})
except (ValueError, BoxError) as exc:
return self.http_status(400, -1, str(exc))
except Exception as exc:
return self.http_status(500, -1, f'Failed to preview skill: {exc}')
@self.route('/install/upload', methods=['POST'], auth_type=group.AuthType.USER_TOKEN_OR_API_KEY)
async def install_skill_from_upload() -> quart.Response:
file = (await quart.request.files).get('file')
if file is None:
return self.http_status(400, -1, 'file is required')
form = await quart.request.form
try:
skill = await self.ap.skill_service.install_from_zip_upload(
file_bytes=file.read(),
filename=file.filename or '',
source_paths=form.getlist('source_paths'),
)
return self.success(data={'skills': skill})
except (ValueError, BoxError) as exc:
return self.http_status(400, -1, str(exc))
except Exception as exc:
return self.http_status(500, -1, f'Failed to install skill: {exc}')
@self.route('/install/upload/preview', methods=['POST'], auth_type=group.AuthType.USER_TOKEN_OR_API_KEY)
async def preview_skill_from_upload() -> quart.Response:
file = (await quart.request.files).get('file')
if file is None:
return self.http_status(400, -1, 'file is required')
try:
preview = await self.ap.skill_service.preview_install_from_zip_upload(
file_bytes=file.read(),
filename=file.filename or '',
)
return self.success(data={'skills': preview})
except (ValueError, BoxError) as exc:
return self.http_status(400, -1, str(exc))
except Exception as exc:
return self.http_status(500, -1, f'Failed to preview skill: {exc}')
@self.route('/scan', methods=['GET'], auth_type=group.AuthType.USER_TOKEN_OR_API_KEY)
async def scan_skill_directory() -> quart.Response:
path = quart.request.args.get('path', '').strip()
if not path:
return self.http_status(400, -1, 'Missing required parameter: path')
try:
result = await self.ap.skill_service.scan_directory_async(path)
return self.success(data=result)
except (ValueError, BoxError) as exc:
return self.http_status(400, -1, str(exc))

View File

@@ -1,11 +1,7 @@
import json
import quart
import sqlalchemy
from .. import group
from .....utils import constants
from .....entity.persistence.metadata import Metadata
@group.group_class('system', '/api/v1/system')
@@ -13,24 +9,6 @@ class SystemRouterGroup(group.RouterGroup):
async def initialize(self) -> None:
@self.route('/info', methods=['GET'], auth_type=group.AuthType.NONE)
async def _() -> str:
# Read wizard_status and wizard_progress from metadata table
wizard_status = 'none'
wizard_progress = None
try:
result = await self.ap.persistence_mgr.execute_async(
sqlalchemy.select(Metadata).where(Metadata.key.in_(['wizard_status', 'wizard_progress']))
)
for row in result:
if row.key == 'wizard_status':
wizard_status = row.value
elif row.key == 'wizard_progress':
try:
wizard_progress = json.loads(row.value)
except (json.JSONDecodeError, TypeError):
wizard_progress = None
except Exception:
pass
return self.success(
data={
'version': constants.semantic_version,
@@ -49,83 +27,17 @@ class SystemRouterGroup(group.RouterGroup):
'disable_models_service', False
),
'limitation': self.ap.instance_config.data.get('system', {}).get('limitation', {}),
'wizard_status': wizard_status,
'wizard_progress': wizard_progress,
}
)
@self.route('/wizard/completed', methods=['POST'], auth_type=group.AuthType.USER_TOKEN)
async def _() -> str:
"""Mark wizard status in metadata table and clear progress.
Accepts JSON body: { "status": "skipped" | "completed" }
"""
data = await quart.request.get_json(silent=True) or {}
status = data.get('status', 'completed')
if status not in ('skipped', 'completed'):
return self.http_status(400, 400, f'Invalid wizard status: {status}')
try:
result = await self.ap.persistence_mgr.execute_async(
sqlalchemy.select(Metadata).where(Metadata.key == 'wizard_status')
)
if result.first():
await self.ap.persistence_mgr.execute_async(
sqlalchemy.update(Metadata).where(Metadata.key == 'wizard_status').values(value=status)
)
else:
await self.ap.persistence_mgr.execute_async(
sqlalchemy.insert(Metadata).values(key='wizard_status', value=status)
)
# Clear wizard progress when wizard is completed/skipped
await self.ap.persistence_mgr.execute_async(
sqlalchemy.delete(Metadata).where(Metadata.key == 'wizard_progress')
)
except Exception as e:
return self.http_status(500, 500, f'Failed to update wizard status: {e}')
return self.success(data={})
@self.route('/wizard/progress', methods=['PUT'], auth_type=group.AuthType.USER_TOKEN)
async def _() -> str:
"""Save wizard progress to metadata table.
Accepts JSON body with wizard state fields:
{ "step": int, "selected_adapter": str|null, "created_bot_uuid": str|null,
"bot_saved": bool, "selected_runner": str|null }
"""
data = await quart.request.get_json(silent=True) or {}
progress_json = json.dumps(data, ensure_ascii=False)
try:
result = await self.ap.persistence_mgr.execute_async(
sqlalchemy.select(Metadata).where(Metadata.key == 'wizard_progress')
)
if result.first():
await self.ap.persistence_mgr.execute_async(
sqlalchemy.update(Metadata).where(Metadata.key == 'wizard_progress').values(value=progress_json)
)
else:
await self.ap.persistence_mgr.execute_async(
sqlalchemy.insert(Metadata).values(key='wizard_progress', value=progress_json)
)
except Exception as e:
return self.http_status(500, 500, f'Failed to save wizard progress: {e}')
return self.success(data={})
@self.route('/tasks', methods=['GET'], auth_type=group.AuthType.USER_TOKEN)
async def _() -> str:
task_type = quart.request.args.get('type')
task_kind = quart.request.args.get('kind')
if task_type == '':
task_type = None
if task_kind == '':
task_kind = None
return self.success(data=self.ap.task_mgr.get_tasks_dict(task_type, task_kind))
return self.success(data=self.ap.task_mgr.get_tasks_dict(task_type))
@self.route('/tasks/<task_id>', methods=['GET'], auth_type=group.AuthType.USER_TOKEN)
async def _(task_id: str) -> str:
@@ -136,9 +48,16 @@ class SystemRouterGroup(group.RouterGroup):
return self.success(data=task.to_dict())
@self.route('/storage-analysis', methods=['GET'], auth_type=group.AuthType.USER_TOKEN)
@self.route('/debug/exec', methods=['POST'], auth_type=group.AuthType.USER_TOKEN)
async def _() -> str:
return self.success(data=await self.ap.maintenance_service.get_storage_analysis())
if not constants.debug_mode:
return self.http_status(403, 403, 'Forbidden')
py_code = await quart.request.data
ap = self.ap
return self.success(data=exec(py_code, {'ap': ap}))
@self.route(
'/debug/plugin/action',

View File

@@ -146,7 +146,6 @@ class UserRouterGroup(group.RouterGroup):
return self.fail(3, str(e))
except ValueError as e:
traceback.print_exc()
self.ap.logger.warning(f'Space OAuth callback failed: {e}')
return self.fail(1, str(e))
except Exception as e:
traceback.print_exc()

View File

@@ -105,29 +105,6 @@ class HTTPController:
):
if os.path.exists(os.path.join(frontend_path, path + '.html')):
path += '.html'
elif not path.startswith('api/'):
# SPA fallback: serve index.html for all non-API, non-static routes
# so that React Router can handle client-side routing (Vite SPA).
# For /home/* sub-routes, first try parent .html files (pre-rendered pages).
if path.startswith('home/'):
segments = path.rstrip('/').split('/')
for i in range(len(segments) - 1, 0, -1):
parent_path = '/'.join(segments[:i]) + '.html'
if os.path.exists(os.path.join(frontend_path, parent_path)):
response = await quart.send_from_directory(
frontend_path, parent_path, mimetype='text/html'
)
response.headers['Cache-Control'] = 'no-cache, no-store, must-revalidate'
response.headers['Pragma'] = 'no-cache'
response.headers['Expires'] = '0'
return response
# Fallback to index.html for SPA client-side routing
response = await quart.send_from_directory(frontend_path, 'index.html', mimetype='text/html')
response.headers['Cache-Control'] = 'no-cache, no-store, must-revalidate'
response.headers['Pragma'] = 'no-cache'
response.headers['Expires'] = '0'
return response
else:
return await quart.send_from_directory(frontend_path, '404.html')

View File

@@ -52,9 +52,6 @@ class ApiKeyService:
async def verify_api_key(self, key: str) -> bool:
"""Verify if an API key is valid"""
if not isinstance(key, str) or not key.startswith('lbk_'):
return False
result = await self.ap.persistence_mgr.execute_async(
sqlalchemy.select(apikey.ApiKey).where(apikey.ApiKey.key == key)
)

View File

@@ -68,6 +68,7 @@ class BotService:
'wecomcs',
'LINE',
'lark',
'gewechat',
]:
webhook_prefix = self.ap.instance_config.data['api'].get('webhook_prefix', 'http://127.0.0.1:5300')
extra_webhook_prefix = self.ap.instance_config.data['api'].get('extra_webhook_prefix', '')
@@ -99,11 +100,11 @@ class BotService:
# TODO: 检查配置信息格式
bot_data['uuid'] = str(uuid.uuid4())
# bind the most recently updated pipeline if any exist
# checkout the default pipeline
result = await self.ap.persistence_mgr.execute_async(
sqlalchemy.select(persistence_pipeline.LegacyPipeline)
.order_by(persistence_pipeline.LegacyPipeline.updated_at.desc())
.limit(1)
sqlalchemy.select(persistence_pipeline.LegacyPipeline).where(
persistence_pipeline.LegacyPipeline.is_default == True
)
)
pipeline = result.first()
if pipeline is not None:
@@ -120,26 +121,24 @@ class BotService:
async def update_bot(self, bot_uuid: str, bot_data: dict) -> None:
"""Update bot"""
update_data = bot_data.copy()
if 'uuid' in update_data:
del update_data['uuid']
if 'uuid' in bot_data:
del bot_data['uuid']
# set use_pipeline_name
if 'use_pipeline_uuid' in update_data:
if 'use_pipeline_uuid' in bot_data:
result = await self.ap.persistence_mgr.execute_async(
sqlalchemy.select(persistence_pipeline.LegacyPipeline).where(
persistence_pipeline.LegacyPipeline.uuid == update_data['use_pipeline_uuid']
persistence_pipeline.LegacyPipeline.uuid == bot_data['use_pipeline_uuid']
)
)
pipeline = result.first()
if pipeline is not None:
update_data['use_pipeline_name'] = pipeline.name
bot_data['use_pipeline_name'] = pipeline.name
else:
raise Exception('Pipeline not found')
await self.ap.persistence_mgr.execute_async(
sqlalchemy.update(persistence_bot.Bot).values(update_data).where(persistence_bot.Bot.uuid == bot_uuid)
sqlalchemy.update(persistence_bot.Bot).values(bot_data).where(persistence_bot.Bot.uuid == bot_uuid)
)
await self.ap.platform_mgr.remove_bot(bot_uuid)

View File

@@ -31,126 +31,15 @@ class KnowledgeService:
if not knowledge_engine_plugin_id:
raise ValueError('knowledge_engine_plugin_id is required')
creation_settings = kb_data.get('creation_settings', {})
retrieval_settings = kb_data.get('retrieval_settings', {})
# Validate required fields based on plugin's creation_schema and retrieval_schema
await self._validate_schema_required_fields(
knowledge_engine_plugin_id,
creation_settings,
retrieval_settings,
)
kb = await self.ap.rag_mgr.create_knowledge_base(
name=kb_data.get('name', 'Untitled'),
knowledge_engine_plugin_id=knowledge_engine_plugin_id,
creation_settings=creation_settings,
retrieval_settings=retrieval_settings,
creation_settings=kb_data.get('creation_settings', {}),
retrieval_settings=kb_data.get('retrieval_settings', {}),
description=kb_data.get('description', ''),
)
return kb.uuid
async def _validate_schema_required_fields(
self,
plugin_id: str,
creation_settings: dict,
retrieval_settings: dict,
) -> None:
"""Validate required fields based on plugin's creation_schema and retrieval_schema.
This is a business-agnostic validation that checks all fields marked as
required in the plugin's schema, regardless of field type.
Args:
plugin_id: Knowledge Engine plugin ID.
creation_settings: User-provided creation settings.
retrieval_settings: User-provided retrieval settings.
Raises:
ValueError: If any required field is missing or empty.
"""
# Validate creation_schema
try:
creation_schema = await self.ap.plugin_connector.get_rag_creation_schema(plugin_id)
self._check_required_fields(creation_schema, creation_settings, 'creation_settings')
except ValueError:
raise
except Exception as e:
self.ap.logger.warning(f'Failed to get creation_schema for validation: {e}')
# Validate retrieval_schema
try:
retrieval_schema = await self.ap.plugin_connector.get_rag_retrieval_schema(plugin_id)
self._check_required_fields(retrieval_schema, retrieval_settings, 'retrieval_settings')
except ValueError:
raise
except Exception as e:
self.ap.logger.warning(f'Failed to get retrieval_schema for validation: {e}')
def _check_required_fields(
self,
schema: dict | list,
settings: dict,
context: str,
) -> None:
"""Check required fields in schema against provided settings.
Args:
schema: Plugin-defined schema (can be list or dict with 'schema' key).
settings: User-provided settings values.
context: Context name for error messages (e.g., 'creation_settings').
Raises:
ValueError: If a required field is missing or empty.
"""
if not schema:
return
# schema can be a list directly, or a dict with 'schema' key
items = schema if isinstance(schema, list) else schema.get('schema', [])
if not items:
return
for item in items:
field_name = item.get('name')
if not field_name:
continue
is_required = item.get('required', False)
if not is_required:
continue
# Check show_if condition - if field is conditionally shown, only validate when condition is met
show_if = item.get('show_if')
if show_if:
depend_field = show_if.get('field')
operator = show_if.get('operator')
expected_value = show_if.get('value')
if depend_field and operator:
depend_value = settings.get(depend_field)
# If show_if condition is not met, skip validation for this field
if operator == 'eq' and depend_value != expected_value:
continue
if operator == 'neq' and depend_value == expected_value:
continue
if operator == 'in' and isinstance(expected_value, list) and depend_value not in expected_value:
continue
value = settings.get(field_name)
# Validate required field has a non-empty value
if value is None or (isinstance(value, str) and value.strip() == ''):
# Get field label for friendly error message
label = item.get('label', {})
field_label = (
label.get('en_US', field_name)
or label.get('zh_Hans', field_name)
or label.get('zh_Hant', field_name)
or field_name
)
raise ValueError(f'{field_label} is required ({context}.{field_name})')
async def update_knowledge_base(self, kb_uuid: str, kb_data: dict) -> None:
"""更新知识库"""
# Filter to only mutable fields

View File

@@ -1,309 +0,0 @@
from __future__ import annotations
import datetime
import os
import re
from pathlib import Path
from typing import Any
import sqlalchemy
from ....core import app
from ....entity.persistence import bstorage as persistence_bstorage
from ....entity.persistence import monitoring as persistence_monitoring
LOG_FILE_PATTERN = re.compile(r'^langbot-(\d{4}-\d{2}-\d{2})\.log(?:\.\d+)?$')
DEFAULT_UPLOAD_FILE_RETENTION_DAYS = 7
DEFAULT_LOG_RETENTION_DAYS = 3
class MaintenanceService:
"""Storage maintenance and diagnostics."""
ap: app.Application
def __init__(self, ap: app.Application) -> None:
self.ap = ap
async def cleanup_expired_files(self) -> dict[str, int]:
cleanup_cfg = self.ap.instance_config.data.get('storage', {}).get('cleanup', {})
upload_retention_days = self._positive_int(
cleanup_cfg.get('uploaded_file_retention_days'),
DEFAULT_UPLOAD_FILE_RETENTION_DAYS,
'storage.cleanup.uploaded_file_retention_days',
)
log_retention_days = self._positive_int(
cleanup_cfg.get('log_retention_days'),
DEFAULT_LOG_RETENTION_DAYS,
'storage.cleanup.log_retention_days',
)
return {
'uploaded_files': await self._cleanup_expired_uploaded_files(upload_retention_days),
'log_files': self._cleanup_expired_log_files(log_retention_days),
}
async def get_storage_analysis(self) -> dict[str, Any]:
cleanup_cfg = self.ap.instance_config.data.get('storage', {}).get('cleanup', {})
upload_retention_days = self._positive_int(
cleanup_cfg.get('uploaded_file_retention_days'),
DEFAULT_UPLOAD_FILE_RETENTION_DAYS,
'storage.cleanup.uploaded_file_retention_days',
)
log_retention_days = self._positive_int(
cleanup_cfg.get('log_retention_days'),
DEFAULT_LOG_RETENTION_DAYS,
'storage.cleanup.log_retention_days',
)
database_cfg = self.ap.instance_config.data.get('database', {})
database_type = database_cfg.get('use', 'sqlite')
database_path = (
Path(database_cfg.get('sqlite', {}).get('path', 'data/langbot.db')) if database_type == 'sqlite' else None
)
roots: list[tuple[str, Path | None]] = [
('database', database_path),
('logs', Path('data/logs')),
('storage', Path('data/storage')),
('vector_store', Path('data/chroma')),
('plugins', Path('data/plugins')),
('mcp', Path('data/mcp')),
('temp', Path('data/temp')),
]
sections = []
for key, path in roots:
sections.append(
{
'key': key,
'path': str(path) if path else '',
'exists': path.exists() if path else False,
'size_bytes': self._path_size(path) if path else 0,
'file_count': self._file_count(path) if path else 0,
}
)
monitoring_counts = await self._monitoring_counts()
binary_storage = await self._binary_storage_stats()
upload_candidates = await self._expired_uploaded_candidates(upload_retention_days)
log_candidates = self._expired_log_candidates(log_retention_days)
return {
'generated_at': datetime.datetime.now(datetime.timezone.utc).isoformat(),
'cleanup_policy': {
'uploaded_file_retention_days': upload_retention_days,
'log_retention_days': log_retention_days,
},
'sections': sections,
'database': {
'type': database_type,
'monitoring_counts': monitoring_counts,
'binary_storage': binary_storage,
},
'cleanup_candidates': {
'uploaded_files': upload_candidates,
'log_files': log_candidates,
},
'tasks': self.ap.task_mgr.get_stats() if self.ap.task_mgr else {},
}
async def _cleanup_expired_uploaded_files(self, retention_days: int) -> int:
provider = self.ap.storage_mgr.storage_provider
provider_name = provider.__class__.__name__
if provider_name == 'LocalStorageProvider':
candidates = self._expired_local_upload_candidates(retention_days, include_paths=True)
deleted = 0
for item in candidates:
try:
os.remove(item['path'])
deleted += 1
except FileNotFoundError:
pass
except Exception as e:
self.ap.logger.warning(f'Failed to delete expired uploaded file {item["key"]}: {e}')
return deleted
if provider_name == 'S3StorageProvider':
return await self._cleanup_expired_s3_uploaded_files(retention_days)
return 0
async def _expired_uploaded_candidates(self, retention_days: int) -> list[dict[str, Any]]:
provider_name = self.ap.storage_mgr.storage_provider.__class__.__name__
if provider_name == 'LocalStorageProvider':
return self._expired_local_upload_candidates(retention_days)
if provider_name == 'S3StorageProvider':
return await self._expired_s3_upload_candidates(retention_days)
return []
async def _cleanup_expired_s3_uploaded_files(self, retention_days: int) -> int:
provider = self.ap.storage_mgr.storage_provider
candidates = await self._expired_s3_upload_candidates(retention_days)
deleted = 0
for item in candidates:
await provider.delete(item['key'])
deleted += 1
return deleted
async def _expired_s3_upload_candidates(self, retention_days: int) -> list[dict[str, Any]]:
provider = self.ap.storage_mgr.storage_provider
cutoff = datetime.datetime.now(datetime.timezone.utc) - datetime.timedelta(days=retention_days)
candidates = []
paginator = provider.s3_client.get_paginator('list_objects_v2')
for page in paginator.paginate(Bucket=provider.bucket_name):
for obj in page.get('Contents', []):
key = obj.get('Key', '')
last_modified = obj.get('LastModified')
if not self._is_uploaded_file_key(key):
continue
if last_modified and last_modified < cutoff:
candidates.append(
{
'key': key,
'size_bytes': obj.get('Size', 0),
'modified_at': last_modified.isoformat(),
}
)
return candidates
def _cleanup_expired_log_files(self, retention_days: int) -> int:
deleted = 0
for item in self._expired_log_candidates(retention_days, include_paths=True):
try:
os.remove(item['path'])
deleted += 1
except FileNotFoundError:
pass
except Exception as e:
self.ap.logger.warning(f'Failed to delete expired log file {item["name"]}: {e}')
return deleted
def _expired_local_upload_candidates(
self, retention_days: int, include_paths: bool = False
) -> list[dict[str, Any]]:
storage_root = Path('data/storage')
if not storage_root.exists():
return []
cutoff = datetime.datetime.now().timestamp() - retention_days * 86400
candidates = []
for entry in storage_root.iterdir():
if not entry.is_file() or not self._is_uploaded_file_key(entry.name):
continue
stat = entry.stat()
if stat.st_mtime >= cutoff:
continue
item = {
'key': entry.name,
'size_bytes': stat.st_size,
'modified_at': datetime.datetime.fromtimestamp(stat.st_mtime, datetime.timezone.utc).isoformat(),
}
if include_paths:
item['path'] = str(entry)
candidates.append(item)
return candidates
def _expired_log_candidates(self, retention_days: int, include_paths: bool = False) -> list[dict[str, Any]]:
log_root = Path('data/logs')
if not log_root.exists():
return []
cutoff_date = datetime.date.today() - datetime.timedelta(days=retention_days - 1)
candidates = []
for entry in log_root.iterdir():
if not entry.is_file():
continue
match = LOG_FILE_PATTERN.match(entry.name)
if not match:
continue
try:
file_date = datetime.date.fromisoformat(match.group(1))
except ValueError:
continue
if file_date >= cutoff_date:
continue
stat = entry.stat()
item = {
'name': entry.name,
'date': file_date.isoformat(),
'size_bytes': stat.st_size,
}
if include_paths:
item['path'] = str(entry)
candidates.append(item)
return candidates
def _is_uploaded_file_key(self, key: str) -> bool:
return '/' not in key and not key.startswith('plugin_config_')
async def _monitoring_counts(self) -> dict[str, int]:
tables = {
'messages': persistence_monitoring.MonitoringMessage.id,
'llm_calls': persistence_monitoring.MonitoringLLMCall.id,
'embedding_calls': persistence_monitoring.MonitoringEmbeddingCall.id,
'errors': persistence_monitoring.MonitoringError.id,
'sessions': persistence_monitoring.MonitoringSession.session_id,
'feedback': persistence_monitoring.MonitoringFeedback.id,
}
counts: dict[str, int] = {}
for key, column in tables.items():
result = await self.ap.persistence_mgr.execute_async(sqlalchemy.select(sqlalchemy.func.count(column)))
counts[key] = result.scalar() or 0
return counts
async def _binary_storage_stats(self) -> dict[str, Any]:
count_result = await self.ap.persistence_mgr.execute_async(
sqlalchemy.select(sqlalchemy.func.count(persistence_bstorage.BinaryStorage.unique_key))
)
size_bytes = None
try:
size_result = await self.ap.persistence_mgr.execute_async(
sqlalchemy.select(sqlalchemy.func.sum(sqlalchemy.func.length(persistence_bstorage.BinaryStorage.value)))
)
size_bytes = size_result.scalar() or 0
except Exception as e:
self.ap.logger.warning(f'Failed to estimate binary storage size: {e}')
return {
'count': count_result.scalar() or 0,
'size_bytes': size_bytes,
}
def _path_size(self, path: Path) -> int:
if not path.exists():
return 0
if path.is_file():
return path.stat().st_size
total = 0
for root, _, files in os.walk(path):
for file_name in files:
file_path = Path(root) / file_name
try:
total += file_path.stat().st_size
except FileNotFoundError:
pass
return total
def _file_count(self, path: Path) -> int:
if not path.exists():
return 0
if path.is_file():
return 1
count = 0
for _, _, files in os.walk(path):
count += len(files)
return count
def _positive_int(self, value: Any, default: int, name: str) -> int:
try:
parsed = int(value)
except (TypeError, ValueError):
self.ap.logger.warning(f'Invalid {name}: {value!r}, using {default}')
return default
if parsed < 1:
self.ap.logger.warning(f'Invalid {name}: {value!r}, using {default}')
return default
return parsed

View File

@@ -9,8 +9,6 @@ from ....core import app
from ....entity.persistence import model as persistence_model
from ....entity.persistence import pipeline as persistence_pipeline
from ....provider.modelmgr import requester as model_requester
from ....agent.runner.config_migration import ConfigMigration
from ....agent.runner import config_schema
def _parse_provider_api_keys(provider_dict: dict) -> dict:
@@ -25,57 +23,12 @@ def _parse_provider_api_keys(provider_dict: dict) -> dict:
return provider_dict
def _runtime_model_data(model_uuid: str, model_data: dict) -> dict:
"""Return model data for rebuilding runtime models after an update.
Update payloads intentionally omit uuid before writing to the database.
Runtime model entities still need the stable uuid so pipeline configs can
resolve the in-memory model immediately after an edit, without requiring a
process restart.
"""
return {**model_data, 'uuid': model_uuid}
class LLMModelsService:
ap: app.Application
def __init__(self, ap: app.Application) -> None:
self.ap = ap
async def _get_runner_descriptor(self, runner_id: str):
registry = getattr(self.ap, 'agent_runner_registry', None)
if registry is None:
return None
try:
return await registry.get(runner_id, bound_plugins=None)
except Exception as e:
logger = getattr(self.ap, 'logger', None)
if logger:
logger.warning(f'Failed to load AgentRunner descriptor while setting default model: {e}')
return None
async def _auto_set_default_pipeline_llm_model(self, pipeline: persistence_pipeline.LegacyPipeline, model_uuid: str):
pipeline_config = pipeline.config
if not isinstance(pipeline_config, dict):
return
runner_id = ConfigMigration.resolve_runner_id(pipeline_config)
if not runner_id:
return
descriptor = await self._get_runner_descriptor(runner_id)
if descriptor is None:
return
ai_config = pipeline_config.setdefault('ai', {})
runner_configs = ai_config.setdefault('runner_config', {})
runner_config = runner_configs.setdefault(runner_id, {})
if not config_schema.set_empty_llm_model_selection(descriptor, runner_config, model_uuid):
return
await self.ap.pipeline_service.update_pipeline(pipeline.uuid, {'config': pipeline_config})
async def get_llm_models(self, include_secret: bool = True) -> list[dict]:
"""Get all LLM models with provider info"""
result = await self.ap.persistence_mgr.execute_async(sqlalchemy.select(persistence_model.LLMModel))
@@ -145,6 +98,7 @@ class LLMModelsService:
self.ap.model_mgr.llm_models.append(runtime_llm_model)
if auto_set_to_default_pipeline:
# set the default pipeline model to this model
result = await self.ap.persistence_mgr.execute_async(
sqlalchemy.select(persistence_pipeline.LegacyPipeline).where(
persistence_pipeline.LegacyPipeline.is_default == True
@@ -152,7 +106,15 @@ class LLMModelsService:
)
pipeline = result.first()
if pipeline is not None:
await self._auto_set_default_pipeline_llm_model(pipeline, model_data['uuid'])
model_config = pipeline.config.get('ai', {}).get('local-agent', {}).get('model', {})
if not model_config.get('primary', ''):
pipeline_config = pipeline.config
pipeline_config['ai']['local-agent']['model'] = {
'primary': model_data['uuid'],
'fallbacks': [],
}
pipeline_data = {'config': pipeline_config}
await self.ap.pipeline_service.update_pipeline(pipeline.uuid, pipeline_data)
return model_data['uuid']
@@ -211,7 +173,7 @@ class LLMModelsService:
raise Exception('provider not found')
runtime_llm_model = await self.ap.model_mgr.load_llm_model_with_provider(
persistence_model.LLMModel(**_runtime_model_data(model_uuid, model_data)),
persistence_model.LLMModel(**model_data),
runtime_provider,
)
self.ap.model_mgr.llm_models.append(runtime_llm_model)
@@ -372,7 +334,7 @@ class EmbeddingModelsService:
raise Exception('provider not found')
runtime_embedding_model = await self.ap.model_mgr.load_embedding_model_with_provider(
persistence_model.EmbeddingModel(**_runtime_model_data(model_uuid, model_data)),
persistence_model.EmbeddingModel(**model_data),
runtime_provider,
)
self.ap.model_mgr.embedding_models.append(runtime_embedding_model)
@@ -405,162 +367,3 @@ class EmbeddingModelsService:
input_text=['Hello, world!'],
extra_args={},
)
class RerankModelsService:
ap: app.Application
def __init__(self, ap: app.Application) -> None:
self.ap = ap
async def get_rerank_models(self) -> list[dict]:
"""Get all rerank models with provider info"""
result = await self.ap.persistence_mgr.execute_async(sqlalchemy.select(persistence_model.RerankModel))
models = result.all()
providers_result = await self.ap.persistence_mgr.execute_async(
sqlalchemy.select(persistence_model.ModelProvider)
)
providers = {p.uuid: p for p in providers_result.all()}
models_list = []
for model in models:
model_dict = self.ap.persistence_mgr.serialize_model(persistence_model.RerankModel, model)
provider = providers.get(model.provider_uuid)
if provider:
provider_dict = self.ap.persistence_mgr.serialize_model(persistence_model.ModelProvider, provider)
model_dict['provider'] = _parse_provider_api_keys(provider_dict)
models_list.append(model_dict)
return models_list
async def get_rerank_models_by_provider(self, provider_uuid: str) -> list[dict]:
"""Get rerank models by provider UUID"""
result = await self.ap.persistence_mgr.execute_async(
sqlalchemy.select(persistence_model.RerankModel).where(
persistence_model.RerankModel.provider_uuid == provider_uuid
)
)
models = result.all()
return [self.ap.persistence_mgr.serialize_model(persistence_model.RerankModel, m) for m in models]
async def create_rerank_model(self, model_data: dict, preserve_uuid: bool = False) -> str:
"""Create a new rerank model"""
if not preserve_uuid:
model_data['uuid'] = str(uuid.uuid4())
if 'provider' in model_data:
provider_data = model_data.pop('provider')
if provider_data.get('uuid'):
model_data['provider_uuid'] = provider_data['uuid']
else:
provider_uuid = await self.ap.provider_service.find_or_create_provider(
requester=provider_data.get('requester', ''),
base_url=provider_data.get('base_url', ''),
api_keys=provider_data.get('api_keys', []),
)
model_data['provider_uuid'] = provider_uuid
await self.ap.persistence_mgr.execute_async(
sqlalchemy.insert(persistence_model.RerankModel).values(**model_data)
)
runtime_provider = self.ap.model_mgr.provider_dict.get(model_data['provider_uuid'])
if runtime_provider is None:
raise Exception('provider not found')
runtime_rerank_model = await self.ap.model_mgr.load_rerank_model_with_provider(
persistence_model.RerankModel(**model_data),
runtime_provider,
)
self.ap.model_mgr.rerank_models.append(runtime_rerank_model)
return model_data['uuid']
async def get_rerank_model(self, model_uuid: str) -> dict | None:
"""Get a single rerank model with provider info"""
result = await self.ap.persistence_mgr.execute_async(
sqlalchemy.select(persistence_model.RerankModel).where(persistence_model.RerankModel.uuid == model_uuid)
)
model = result.first()
if model is None:
return None
model_dict = self.ap.persistence_mgr.serialize_model(persistence_model.RerankModel, model)
provider_result = await self.ap.persistence_mgr.execute_async(
sqlalchemy.select(persistence_model.ModelProvider).where(
persistence_model.ModelProvider.uuid == model.provider_uuid
)
)
provider = provider_result.first()
if provider:
provider_dict = self.ap.persistence_mgr.serialize_model(persistence_model.ModelProvider, provider)
model_dict['provider'] = _parse_provider_api_keys(provider_dict)
return model_dict
async def update_rerank_model(self, model_uuid: str, model_data: dict) -> None:
"""Update an existing rerank model"""
if 'uuid' in model_data:
del model_data['uuid']
if 'provider' in model_data:
provider_data = model_data.pop('provider')
if provider_data.get('uuid'):
model_data['provider_uuid'] = provider_data['uuid']
else:
provider_uuid = await self.ap.provider_service.find_or_create_provider(
requester=provider_data.get('requester', ''),
base_url=provider_data.get('base_url', ''),
api_keys=provider_data.get('api_keys', []),
)
model_data['provider_uuid'] = provider_uuid
await self.ap.persistence_mgr.execute_async(
sqlalchemy.update(persistence_model.RerankModel)
.where(persistence_model.RerankModel.uuid == model_uuid)
.values(**model_data)
)
await self.ap.model_mgr.remove_rerank_model(model_uuid)
runtime_provider = self.ap.model_mgr.provider_dict.get(model_data['provider_uuid'])
if runtime_provider is None:
raise Exception('provider not found')
runtime_rerank_model = await self.ap.model_mgr.load_rerank_model_with_provider(
persistence_model.RerankModel(**_runtime_model_data(model_uuid, model_data)),
runtime_provider,
)
self.ap.model_mgr.rerank_models.append(runtime_rerank_model)
async def delete_rerank_model(self, model_uuid: str) -> None:
"""Delete a rerank model"""
await self.ap.persistence_mgr.execute_async(
sqlalchemy.delete(persistence_model.RerankModel).where(persistence_model.RerankModel.uuid == model_uuid)
)
await self.ap.model_mgr.remove_rerank_model(model_uuid)
async def test_rerank_model(self, model_uuid: str, model_data: dict) -> None:
"""Test a rerank model"""
runtime_rerank_model: model_requester.RuntimeRerankModel | None = None
if model_uuid != '_':
for model in self.ap.model_mgr.rerank_models:
if model.model_entity.uuid == model_uuid:
runtime_rerank_model = model
break
if runtime_rerank_model is None:
raise Exception('model not found')
else:
runtime_rerank_model = await self.ap.model_mgr.init_temporary_runtime_rerank_model(model_data)
await runtime_rerank_model.provider.invoke_rerank(
model=runtime_rerank_model,
query='What is artificial intelligence?',
documents=[
'Artificial intelligence is a branch of computer science.',
'The weather is nice today.',
],
)

Some files were not shown because too many files have changed in this diff Show More