From 04628d93cb4792eca6455eea5327e1513b6dc389 Mon Sep 17 00:00:00 2001 From: RockChinQ <45992437+RockChinQ@users.noreply.github.com> Date: Thu, 25 Jun 2026 04:17:19 -0400 Subject: [PATCH] docs: add architecture guide for agents --- AGENTS.md | 213 +++++++++++++++-------------------------- ARCHITECTURE.md | 250 ++++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 329 insertions(+), 134 deletions(-) create mode 100644 ARCHITECTURE.md diff --git a/AGENTS.md b/AGENTS.md index 86eee323f..a886d2e1a 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -1,160 +1,105 @@ # AGENTS.md -This file guides code agents (Claude Code, GitHub Copilot, OpenAI Codex, etc.) working in the LangBot project. `CLAUDE.md` is a symlink to this file. +This file guides code agents working in the LangBot main repository. `CLAUDE.md` is a symlink to this file. -## Project Overview +Read `ARCHITECTURE.md` before non-trivial backend, frontend, runtime, plugin, Box, MCP, persistence, or cross-repo SDK changes. This file is the working checklist; `ARCHITECTURE.md` is the system map. -LangBot is an open-source, LLM-native instant-messaging bot development platform. It aims to provide an out-of-the-box IM bot development experience with Agent, RAG, MCP and other LLM application capabilities, supporting mainstream global IM platforms and exposing rich APIs for custom development. +## Quick Facts -LangBot has a comprehensive web frontend — almost every operation can be performed through it. +- Python backend: `>=3.11,<4.0`, dependencies managed by `uv`. +- Frontend: `web/` is Vite + React Router 7 + shadcn/ui + Tailwind, managed by `pnpm`. +- Backend framework: Quart served by Hypercorn on `api.port`, default `5300`. +- Frontend dev server: `web/` on `3000`, with `VITE_API_BASE_URL` pointing at the backend. +- Plugin/Box/runtime contracts live in sibling repo `langbot-plugin-sdk`, pinned as `langbot-plugin` in `pyproject.toml`. -- **Python**: `>=3.11,<4.0`, dependencies managed by `uv`. Package version is in `pyproject.toml`. -- **Frontend**: `web/` is a **Vite + React Router 7 + shadcn/ui + Tailwind CSS** SPA, managed by `pnpm`. (Note: this is NOT Next.js — the `dev` script is `vite`.) -- **Backend framework**: Quart (the async flavour of Flask). The HTTP API and the pre-built web UI are both served by the backend on `http://127.0.0.1:5300`. - -## Repository Layout - -``` -LangBot/ -├── main.py # Entrypoint shim -> langbot.__main__.main() -├── pyproject.toml # Python project + deps (uv), pins langbot-plugin== -├── src/langbot/ -│ ├── __main__.py # Real entrypoint, CLI args (--standalone-runtime, --standalone-box, --debug) -│ ├── pkg/ # Core backend package -│ │ ├── api/ # HTTP API controllers + services (Quart) -│ │ ├── core/ # App bootstrap, stages, task manager -│ │ ├── platform/ # IM platform adapters, bot managers, session managers -│ │ ├── provider/ # LLM providers, requesters, tool providers -│ │ ├── pipeline/ # Pipelines, stages, query pool -│ │ ├── plugin/ # Bridge connecting LangBot to the plugin runtime (see below) -│ │ ├── box/ # Code-sandbox subsystem (Docker / nsjail / E2B backends) -│ │ ├── skill/ # Skill subsystem -│ │ ├── rag/ , vector/ # RAG + vector store -│ │ ├── command/ # Built-in commands -│ │ ├── persistence/ # ORM models + Alembic migrations (SQLite & PostgreSQL) -│ │ ├── storage/ # Object/file storage abstractions -│ │ ├── config/, entity/, discover/, utils/, telemetry/, survey/ -│ ├── libs/ # Vendored SDKs (qq_official_api, wecom_api, etc.) -│ └── templates/ # Config/component templates (e.g. templates/config.yaml) -├── web/ # Frontend SPA (Vite + React Router 7 + shadcn + Tailwind) -└── docker/ # docker-compose deployment files -``` - -## Development Environment Setup - -Full guide lives in the wiki: **["开发配置" / Dev Config](https://docs.langbot.app/zh/develop/dev-config)**. Summary: - -### Backend - -```bash -pip install uv -uv sync --dev # uv creates a .venv/ for you; point your editor's interpreter at it -uv run main.py # serves API + web UI on http://127.0.0.1:5300 -``` - -On first run the config file is generated at `data/config.yaml`. DB is SQLite by default (zero setup); PostgreSQL is supported. Migrations run automatically on startup. - -### Frontend - -Requires Node.js + [pnpm](https://pnpm.io/installation). - -```bash -cd web -cp .env.example .env # Windows: copy .env.example .env -pnpm install -pnpm dev # http://127.0.0.1:3000 (npm install / npm run dev also work) -``` - -`pnpm dev` reads `VITE_API_BASE_URL` from `web/.env` so the dev frontend can reach the backend on port `5300`. In production the frontend is pre-built into static files served by the backend on the same origin. - -### Code formatting - -The repo runs lint + format checks in CI. Install the pre-commit hooks so the same checks run locally before each commit: +## Essential Commands ```bash +uv sync --dev +uv run main.py uv run pre-commit install + +cd web +pnpm install +pnpm dev +pnpm build ``` -## Plugin System - -LangBot's plugin system (Plugin SDK, CLI `lbp`, Plugin Runtime, and the shared entity/API definitions) lives in a **separate repository**: [`langbot-plugin-sdk`](https://github.com/langbot-app/langbot-plugin-sdk). LangBot depends on it via the pinned `langbot-plugin` package in `pyproject.toml`. - -### Architecture (what to know inside this repo) - -- Plugins run as independent processes managed by the **Plugin Runtime**. The Runtime supports two control transports: `stdio` and `websocket`. -- When LangBot is started directly by a user (not in a container), it spawns and connects to the Runtime over **stdio** (lightweight/personal use). -- When LangBot runs in a container, it connects to a standalone Runtime over **WebSocket** (production). -- The bridge code lives in `src/langbot/pkg/plugin/` (`connector.py`, `handler.py`). -- Relevant config (`data/config.yaml`): `plugin.runtime_ws_url` (e.g. `ws://langbot_plugin_runtime:5400/control/ws`). Start LangBot with `--standalone-runtime` to make it connect to an externally-launched Runtime over WebSocket instead of spawning one over stdio. - -### Debugging the Plugin Runtime / CLI / SDK - -This is documented in detail in the **SDK repo's `AGENTS.md`** and in the wiki page **["调试插件运行时、CLI、SDK" / Plugin Runtime](https://docs.langbot.app/zh/develop/plugin-runtime)**. The short version: - -- Clone `LangBot` and `langbot-plugin-sdk` as siblings under one parent dir so the editor resolves shared entities. -- Start a standalone Runtime from the SDK repo: `uv run --no-sync lbp rt` (control port `5400`, debug port `5401`). -- To make LangBot use a locally-modified SDK: from the SDK dir, with LangBot's `.venv` active, run `uv pip install .`, then launch LangBot with `uv run --no-sync main.py --standalone-runtime` (keep `--no-sync` so your local SDK isn't overwritten). - -### Debugging the Box (sandbox) runtime - -The Box subsystem (`src/langbot/pkg/box/`) is the code sandbox. It picks the first available backend among **Docker / nsjail / E2B**. The standalone Box runtime is launched via the SDK CLI: `lbp box`. Backend selection details, the `lbp box` flags, and the SDK-side architecture are documented in the SDK repo's `AGENTS.md`. - -Relevant config (`data/config.yaml`, `box:` section): `box.enabled` (master switch — disabling it also disables the native sandbox tools, skill add/edit, and stdio-mode MCP servers), `box.backend` (`'local'` = Docker/nsjail auto-pick, or `'docker'` / `'nsjail'` / `'e2b'`; also settable via `BOX__BACKEND`), and `box.runtime.endpoint` (external Box runtime base URL, e.g. `ws://127.0.0.1:5410`; empty = local auto-managed runtime). Like the plugin runtime, LangBot can connect to an externally-launched Box runtime by setting that endpoint and starting with `--standalone-box`. - -> A common false "No supported sandbox backend (Docker / nsjail / E2B) is available" comes from Docker being installed and running but the current user not being in the `docker` group → `docker info` gets `permission denied` on the socket. Fix: `sudo usermod -aG docker ` and restart the backend in a shell that has the new group. - -## Development Standards - -- LangBot is a global project: **all code comments and docstrings must be in English**, and every user-facing string must support **i18n** (`en_US` + `zh_Hans` at minimum, plus `ja_JP` where the repo already has it). -- LangBot is adopted in both toC and toB scenarios — always consider compatibility and security. -- **Commit message format**: `(): ` - - `type`: one of `feat`, `fix`, `docs`, `style`, `refactor`, `perf`, `test`, `chore`, etc. - - `scope`: the affected package/module/file/class. - - `subject`: concise description of the change. - -### Database migrations (Alembic) - -LangBot uses [Alembic](https://alembic.sqlalchemy.org/) for migrations, supporting both SQLite and PostgreSQL from a single set of scripts. Migration files live in `src/langbot/pkg/persistence/alembic/versions/`. - -If you change ORM model definitions, generate a migration: +Useful focused tests: ```bash -# Run from the project root (requires data/config.yaml to exist) -uv run python -m langbot.pkg.persistence.alembic_runner autogenerate "description of your change" +uv run pytest tests/unit_tests -q +uv run pytest tests/integration -q +uv run pytest tests/integration/persistence -q +uv run pytest tests/manual/mcp_smoke.py + +cd web +pnpm lint +pnpm test:e2e ``` -Review and edit the generated script before committing. Migrations execute automatically on startup. `autogenerate` detects schema changes (add/drop columns, tables, type changes) but **data migrations** (e.g. mutating JSON field contents) must be hand-written into the generated script. `env.py` sets `render_as_batch=True`, so SQLite's ALTER TABLE limits are handled automatically — no need to branch per database. More in the wiki ["开发配置"](https://docs.langbot.app/zh/develop/dev-config#数据库迁移). +Run the narrowest useful test first, then broader checks when confidence is needed. -When writing a migration, follow these rules: +## Where to Look -- **Revision id ≤ 32 characters.** PostgreSQL stores `alembic_version.version_num` as `varchar(32)`; a longer id raises `StringDataRightTruncationError` at runtime. Prefer short, descriptive ids like `0005_add_llm_context_length`. -- **Guard every operation against missing tables/columns.** Fresh installs build the schema via `create_all()` and then stamp the Alembic baseline, so a migration may run against a table that already has the change — or, in tests, against an empty database. Check `inspector.get_table_names()` / `inspector.get_columns(...)` before `add_column` / `drop_column`, mirroring the existing migrations. -- **Keep a single linear head.** Chain `down_revision` to the current head; do not create branches. Run the migration tests after adding one: `uv run pytest tests/integration/persistence/ -q` (the PostgreSQL test needs a running PG via `TEST_POSTGRES_URL`). +- Architecture map: `ARCHITECTURE.md`. +- Dev environment guide: https://docs.langbot.app/zh/develop/dev-config. +- Plugin runtime / CLI / SDK debugging: https://docs.langbot.app/zh/develop/plugin-runtime. +- API-key auth: `docs/API_KEY_AUTH.md`. +- Box deep-dive notes: `docs/review/box-architecture.md` and related files. +- In-repo skills: `skills/` is the single source of truth for LangBot agent skills. +- SDK repo: `../langbot-plugin-sdk/` when changing shared entities, plugin APIs, action protocol, `lbp rt`, or `lbp box`. -> **Legacy migration system (deprecated — do not extend).** The old 3.x migration system under `src/langbot/pkg/persistence/migrations/` (`DBMigration` subclasses in `dbmXXX_*.py`, run from `pkg/persistence/mgr.py`) is **frozen**. Do **not** add new `dbmXXX_*.py` files. The chain is capped at `required_database_version = 25` (`pkg/utils/constants.py`); those files only exist to upgrade pre-existing 3.x databases up to the Alembic baseline and are kept read-only. All new schema changes go through Alembic. +## Cross-Repo SDK Work -## Agent-Facing Surfaces (MCP + Skills) +When changing SDK contracts used by LangBot: -LangBot is built to be **agent-friendly**. Three surfaces let AI agents work -with LangBot, and they MUST be kept in lockstep with the HTTP API: +```bash +# from langbot-plugin-sdk, with LangBot's .venv active +uv pip install . -1. **MCP server** — `src/langbot/pkg/api/mcp/` exposes a curated subset of the - API as MCP tools at `/mcp` (API-key authenticated, including the - `api.global_api_key` from config.yaml). `server.py` defines the tools (they - call the service layer directly); `mount.py` is the ASGI dispatcher. -2. **In-repo skills** — `skills/` is the **single source of truth** for agent - skills (plugin/core/deploy/e2e/MCP-ops). Docs and the landing page link here - rather than embedding their own copies. -3. **API-key auth** — `api.global_api_key` (config.yaml) authenticates the API - and MCP without a login session; see `docs/API_KEY_AUTH.md`. +# from LangBot, preserve the locally installed SDK +uv run --no-sync main.py +``` -> **Maintenance rule (important).** When you add, remove, or change an HTTP API -> endpoint that should be agent-accessible, you MUST update **both** the matching -> MCP tool in `src/langbot/pkg/api/mcp/server.py` **and** the relevant skill under -> `skills/` (especially `skills/skills/langbot-mcp-ops`). The API, the MCP tool -> surface, and the skills are one system — drift between them is a bug. +For standalone runtime debugging: -## Some Principles +```bash +# in langbot-plugin-sdk +uv run --no-sync lbp rt +uv run --no-sync lbp box + +# in LangBot +uv run --no-sync main.py --standalone-runtime +uv run --no-sync main.py --standalone-box +``` + +Config keys to verify in `data/config.yaml` / `src/langbot/templates/config.yaml`: + +- Plugin runtime: `plugin.runtime_ws_url`, default Docker host `langbot_plugin_runtime:5400/control/ws`. +- Box runtime: `box.enabled`, `box.backend`, `box.runtime.endpoint`, Docker host `langbot_box:5410`. +- API/MCP auth: `api.global_api_key`. + +## Change Rules + +- HTTP API changes that should be agent-accessible must update the matching MCP tool in `src/langbot/pkg/api/mcp/server.py` and the relevant skill under `skills/` in the same pass. +- New schema changes use Alembic under `src/langbot/pkg/persistence/alembic/versions/`; do not add legacy `dbmXXX` migrations. +- New platform behavior belongs in platform adapters only for platform translation; pipeline/business logic belongs in `pkg/pipeline/` or services. +- User-facing strings must support i18n (`en_US`, `zh_Hans`; include `ja_JP` where the repo already does). +- Code comments and docstrings must be English. +- Keep compatibility and security in mind; LangBot is used in both self-hosted/community and toB deployments. +- Commit message format: `(): `. + +## Runtime Pitfalls + +- Local stdio Plugin Runtime disconnects do not auto-reconnect; restart LangBot if that path breaks. +- Orphan runtime processes on `5400`/`5401` commonly break plugin debugging. +- Use `uv run --no-sync` after locally installing the SDK, or `uv` may restore the pinned package. +- A false Box “no backend” often means Docker is running but the current user lacks Docker socket permission. +- Do not confuse external MCP servers LangBot connects to (`pkg/provider/tools/loaders/mcp.py`) with LangBot's own `/mcp` server (`pkg/api/mcp/`). +- `CLAUDE.md` is a symlink to this file; edit `AGENTS.md`, not the symlink. + +## Principles - Keep it simple, stupid. - Entities should not be multiplied unnecessarily. diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md new file mode 100644 index 000000000..ded90e7ea --- /dev/null +++ b/ARCHITECTURE.md @@ -0,0 +1,250 @@ +# Architecture + +This document is a map of LangBot's moving parts. It is intentionally more stable than a feature guide and more concrete than the README: when you need to change behavior, start here, then follow the file references into the code. + +For agent-specific working rules, see `AGENTS.md`. For plugin-runtime and Box-runtime implementation details, also read the sibling SDK repo: [`langbot-plugin-sdk`](https://github.com/langbot-app/langbot-plugin-sdk). + +## What LangBot Is + +LangBot is an open-source platform for building production IM bots backed by LLMs, agents, RAG, plugins, MCP tools, and a web management panel. + +At runtime, one LangBot process owns: + +- a Quart/Hypercorn HTTP service and the built web UI on `:5300`; +- messaging-platform adapters such as Discord, Telegram, Slack, WeChat, QQ, WeCom, Lark, DingTalk, KOOK, LINE, Satori, Matrix, and HTTP/WebSocket bots; +- a pipeline engine that turns inbound platform messages into LLM/tool/plugin work and replies; +- persistence, storage, vector database, telemetry, monitoring, and configuration managers; +- bridges to the Plugin Runtime and Box Runtime provided by `langbot-plugin-sdk`; +- an MCP server at `/mcp` exposing a curated agent-facing subset of the service layer. + +## Repository Boundary + +LangBot is not a single-repo system. + +- `LangBot/` is the main product: backend, web UI, platform adapters, pipeline engine, HTTP API, MCP server, RAG, persistence, skills integration, and the bridge code that talks to runtimes. +- `langbot-plugin-sdk/` is published as `langbot-plugin` and pinned in `LangBot/pyproject.toml`. It contains plugin developer APIs, shared entities, `lbp`, the Plugin Runtime (`lbp rt`), and the Box Runtime (`lbp box`). +- Plugins import SDK APIs from `langbot_plugin.*`; the LangBot main process imports the same package for shared entities and runtime protocols. + +This split matters. If a change modifies SDK entities, component APIs, action protocols, `lbp rt`, or `lbp box`, verify the sibling SDK repo and install the local SDK into LangBot's virtualenv when testing cross-repo behavior. + +## Startup Path + +The process entrypoint is small and layered: + +1. `main.py` delegates to `langbot.__main__.main()`. +2. `src/langbot/__main__.py` parses `--standalone-runtime`, `--standalone-box`, and `--debug`, checks dependencies, generates missing config/data files, and calls `pkg.core.boot.main()`. +3. `pkg/core/boot.py` executes startup stages in order: `LoadConfigStage`, `GenKeysStage`, `SetupLoggerStage`, `BuildAppStage`, `ShowNotesStage`. +4. `BuildAppStage` constructs the `Application` object by wiring managers, services, runtime connectors, and controllers. +5. `Application.run()` starts the platform manager, query controller, HTTP controller, telemetry/cleanup loops, and plugin initialization. + +The central runtime object is `pkg/core/app.py::Application`. It is a service locator for long-lived managers. That is not elegant, but it is the current architectural center; most subsystems receive `ap: Application` and collaborate through it. + +## Top-Level Layout + +```text +LangBot/ +├── main.py # Entrypoint shim +├── pyproject.toml # Python package, deps, pinned langbot-plugin +├── src/langbot/ +│ ├── __main__.py # CLI entrypoint and boot handoff +│ ├── pkg/ +│ │ ├── core/ # Application, boot stages, task manager +│ │ ├── api/ # HTTP API + MCP server mount +│ │ ├── platform/ # IM adapters and runtime bot manager +│ │ ├── pipeline/ # Message routing and pipeline stages +│ │ ├── provider/ # LLM runners, model manager, tools +│ │ ├── plugin/ # LangBot-side Plugin Runtime connector/handler +│ │ ├── box/ # LangBot-side Box service/connector +│ │ ├── skill/ # Skill metadata/activation integration +│ │ ├── rag/ , vector/ # Knowledge-base and vector DB integration +│ │ ├── persistence/ # SQLAlchemy/SQLModel, Alembic, legacy migrations +│ │ ├── storage/ # Local/S3 file storage abstraction +│ │ └── config/, entity/, utils/, telemetry/, survey/ +│ ├── libs/ # Vendored third-party platform SDKs +│ └── templates/ # Default config and component metadata +├── web/ # Vite + React Router + shadcn/ui + Tailwind SPA +├── docker/ # Deployment manifests +├── skills/ # In-repo agent skills, single source of truth +└── tests/ # Unit/integration/e2e/manual tests +``` + +## The Runtime Graph + +The most useful mental model is this graph: + +```text +Platform adapter + → RuntimeBot + → MessageAggregator + → QueryPool + → Controller + → RuntimePipeline + → PipelineStage chain + → RequestRunner / ToolManager / PluginRuntimeConnector / BoxService + → response via adapter +``` + +The HTTP and MCP surfaces are parallel entrypoints into the same service layer: + +```text +HTTP client / Web UI + → Quart route group + → api/http/service/* + → Application managers / persistence / runtime connectors + +MCP client + → /mcp mount + → api/mcp/server.py tools + → the same service layer directly +``` + +## Message Flow + +Inbound platform messages enter through adapter-specific SDK callbacks. The common path is: + +1. A platform adapter under `pkg/platform/sources/` converts platform-specific events into SDK message/event entities. +2. `RuntimeBot` in `pkg/platform/botmgr.py` applies pipeline routing rules and either discards the message, pushes it to webhooks, or sends it to the message aggregator. +3. `MessageAggregator` batches/normalizes messages before adding a `Query` to `QueryPool`. +4. `Controller` in `pkg/pipeline/controller.py` selects queries subject to global pipeline concurrency and per-session concurrency. +5. `RuntimePipeline` in `pkg/pipeline/pipelinemgr.py` runs configured pipeline stages using a responsibility-chain style executor that supports generator stages. +6. The chat stage emits plugin events, calls a configured `RequestRunner`, handles streaming/non-streaming responses, records telemetry, and appends conversation history. +7. Output stages send text, cards, chunks, files, or error notices back through the original platform adapter. + +Pipeline components are registered by decorators and package import side effects. When adding a new stage, loader, runner, or adapter, check the corresponding preregistration mechanism instead of inventing a second registry. + +## Platform Layer + +Platform code lives under `pkg/platform/`. + +- `botmgr.py` owns runtime bots, routing rules, event logging, webhook pushing, and adapter lifecycle. +- `sources/` contains adapter implementations. Each adapter subclasses `langbot_plugin.api.definition.abstract.platform.adapter.AbstractMessagePlatformAdapter` from the SDK. +- Platform entities such as `MessageChain`, `Image`, `At`, `Voice`, and events come from `langbot-plugin-sdk`, not from this repo. + +The platform layer should translate between external platform APIs and LangBot's shared message/event model. It should not contain LLM-provider logic or pipeline business logic. + +## Pipeline Layer + +Pipeline code lives under `pkg/pipeline/`. + +Important pieces: + +- `pool.py::QueryPool` stores pending queries and cached in-flight queries for plugin backward-compatible calls. +- `controller.py::Controller` schedules query processing and enforces concurrency. +- `pipelinemgr.py::RuntimePipeline` materializes database pipeline config into a runtime stage chain. +- `process/handlers/chat.py::ChatMessageHandler` is the main LLM conversation handler. +- Stage families include response rules, banned sessions, content filters, preprocessors, rate limits, message truncation, long text handling, response-back, command handling, and wrappers. + +Pipelines are configuration-driven. Prefer adding a stage or extending an existing stage family over hard-coding behavior in platform adapters. + +## Provider, RAG, and Tools + +Provider code lives under `pkg/provider/`. + +- `modelmgr/` manages configured model providers and requesters. +- `runners/` implements request runners such as the local agent runner and external workflow integrations. +- `tools/toolmgr.py` aggregates tools from native tools, plugin tools, external MCP servers, and skill-authoring tools. +- `tools/loaders/mcp.py` is the MCP client side: external MCP servers that LangBot connects to for agent tools. +- RAG lives across `pkg/rag/`, `pkg/vector/`, model services, and plugin KnowledgeEngine actions. + +Do not confuse LangBot's MCP client side with LangBot's own MCP server at `/mcp`; they are different surfaces. + +## Plugin System + +The plugin system crosses the repo boundary. + +In this repo: + +- `pkg/plugin/connector.py` connects LangBot to the Plugin Runtime over stdio or WebSocket. +- `pkg/plugin/handler.py` exposes LangBot actions to the runtime and calls runtime actions for plugin operations. +- `pkg/provider/tools/loaders/plugin.py` exposes plugin Tool components to LLM runners. +- Pipeline handlers emit SDK events such as normal-message events and prompt-processing events. + +In `langbot-plugin-sdk`: + +- `src/langbot_plugin/api/` defines `BasePlugin`, component base classes, message/event entities, contexts, proxies, and manifests. +- `src/langbot_plugin/runtime/` implements `lbp rt`, plugin discovery, dependency installation, process launching, and control/debug connections. +- `src/langbot_plugin/entities/io/` defines the action protocol shared by LangBot, runtime, and plugin processes. + +The Plugin Runtime supports stdio and WebSocket control transports. Direct local LangBot runs usually spawn the runtime over stdio. Containerized/standalone deployments connect over WebSocket using `plugin.runtime_ws_url` and `--standalone-runtime`. + +## Box Runtime and Skills + +Box is the sandbox subsystem used by native agent tools, stdio MCP servers, skill authoring, and managed processes. + +In this repo: + +- `pkg/box/service.py` is the application-facing facade for exec, sessions, managed processes, skill CRUD, status, reconnects, quotas, mounts, and sandbox profiles. +- `pkg/box/connector.py` connects to the Box Runtime over stdio, Windows subprocess+WebSocket, or remote WebSocket. +- `pkg/provider/tools/loaders/native.py`, `mcp_stdio.py`, and skill loaders depend on Box availability. +- `pkg/skill/manager.py` loads skills from the Box runtime, falling back to local `data/skills` when needed. + +In `langbot-plugin-sdk`: + +- `src/langbot_plugin/box/server.py` implements `lbp box` and the WebSocket endpoints on `:5410`. +- `src/langbot_plugin/box/runtime.py` owns sandbox sessions and managed processes. +- `backend.py`, `nsjail_backend.py`, and `e2b_backend.py` implement sandbox backends. +- `skill_store.py` manages skill packages from the Box side. + +Important config keys live under `box:` in `src/langbot/templates/config.yaml`: `box.enabled`, `box.backend`, `box.runtime.endpoint`, and `box.local.*`. Start LangBot with `--standalone-box` when connecting to an externally launched Box runtime. + +## HTTP API, Web UI, and MCP Server + +`pkg/api/http/controller/main.py` builds a Quart app, registers route groups, serves the built SPA, and wraps the ASGI app with the MCP dispatcher. + +- HTTP route groups live under `pkg/api/http/controller/groups/`. +- Service-layer logic lives under `pkg/api/http/service/`. +- The built web UI is served from the frontend build path with SPA fallback. +- The MCP server lives under `pkg/api/mcp/` and is mounted at `/mcp`. + +The MCP server intentionally exposes a curated subset of the API. Tools call service classes directly rather than making HTTP requests back into LangBot. + +Maintenance rule: when adding, removing, or changing an HTTP endpoint that should be agent-accessible, update the matching MCP tool and the relevant in-repo skill under `skills/` in the same pass. + +## Persistence and Configuration + +Persistence is centered on `pkg/persistence/mgr.py`. + +- SQLite is the default database; PostgreSQL is supported. +- Models live under `pkg/entity/persistence/`. +- Fresh schemas are created from metadata, then legacy migrations run up to the frozen 3.x baseline, then Alembic migrations run to head. +- New schema changes should use Alembic under `pkg/persistence/alembic/versions/`; do not extend the frozen legacy migration chain. + +Configuration starts from `src/langbot/templates/config.yaml` and is generated into `data/config.yaml` on first run. Most long-lived managers read from `ap.instance_config.data`. + +## Frontend + +The frontend lives in `web/` and is a Vite SPA using React Router 7, shadcn/ui, Tailwind CSS, and pnpm. It is not Next.js, despite some historical filenames. + +In development, `pnpm dev` serves the UI on `:3000` and reads `VITE_API_BASE_URL` to call the backend on `:5300`. In production, the built frontend is packaged into the Python distribution and served by the backend. + +Keep frontend API behavior aligned with `pkg/api/http/service/` and route groups. User-facing strings must go through the existing i18n setup. + +## Agent-Facing Surfaces + +LangBot is deliberately agent-friendly. The agent-facing surfaces are part of the architecture, not extra docs. + +- `skills/` is the single source of truth for in-repo skills. +- `pkg/api/mcp/server.py` exposes the LangBot MCP server at `/mcp`. +- `api.global_api_key` authenticates API/MCP access without a browser login. +- `AGENTS.md` and `ARCHITECTURE.md` tell coding agents how the repo works. + +When one of these changes, update the others if the behavior or contract changed. API, MCP tools, and skills are one system; drift is a bug. + +## Where to Change Things + +- New HTTP API: add/adjust a service in `pkg/api/http/service/`, a route group in `pkg/api/http/controller/groups/`, tests, and MCP/skills if agent-accessible. +- New platform adapter: add a `pkg/platform/sources/*` adapter, component metadata/templates as needed, i18n, docs, and tests/smoke coverage. +- New pipeline behavior: add or extend a pipeline stage family under `pkg/pipeline/`; avoid putting pipeline rules in adapters. +- New LLM provider/requester: work under `pkg/provider/modelmgr/` and related service/UI surfaces. +- New LLM tool source: extend `pkg/provider/tools/loaders/` and `ToolManager` intentionally. +- New plugin component/API/protocol: change `langbot-plugin-sdk` first or in lockstep, then update LangBot bridge code. +- New Box capability: change both `pkg/box/` and `langbot-plugin-sdk/src/langbot_plugin/box/`, plus config and tests. +- New database schema: add an Alembic migration, not a legacy `dbmXXX` migration. + +## Design Biases + +- Keep platform translation, pipeline orchestration, provider execution, and runtime protocols separate. +- Reuse existing registries and service layers instead of adding parallel paths. +- Prefer small, explicit agent surfaces over exposing every internal API. +- Treat cross-repo contracts with the SDK as public interfaces. +- Test behavior at the narrowest useful layer first, then add integration/e2e coverage for runtime or platform changes.