Compare commits

..

48 Commits

Author SHA1 Message Date
RockChinQ 0551d22689 chore(release): bump version to 4.10.1 2026-06-09 13:32:58 -04:00
RockChinQ 53d4edb609 fix(dify): send 'user' as plain form field in file upload
The multipart tuple form (None, user) is httpx 'files=' syntax for a part
with no filename; placed under 'data=' it expanded into a stray user=None
field, so Dify associated the uploaded file with the wrong user and the
workflow never received the image. Send 'user' as a plain string.
2026-06-09 10:43:55 -04:00
RockChinQ f897987ac1 chore(deps): bump langbot-plugin to 0.4.2 (stable) 2026-06-09 09:52:07 -04:00
Junyan Chin 8e558ad3a1 Feat/saas sandbox adaptation (#2234)
* fix(box): trust Box-reported skill paths when filesystem is not shared

In separated deployments (Docker Compose, k8s sidecar, --standalone-box,
remote runtime.endpoint) the Box runtime owns its own filesystem, so the
skill package_root it reports via list_skills is not resolvable on the
LangBot side. LangBot's reload_skills and build_skill_extra_mounts
validated those paths with os.path.isdir() against its own filesystem,
which silently dropped every skill in such deployments — breaking the
sandbox skill feature for the nsjail/SaaS backend.

Add BoxService.shares_filesystem_with_box, derived from the connector
transport (stdio = shared, WebSocket = separated), with an explicit
override seam for tests/embedders. Gate both isdir() guards on it: keep
local validation in shared-fs stdio mode, trust Box-reported paths
otherwise. The Box runtime only reports skills found on its own
filesystem, so those paths are valid there by construction.

Adds topology-derivation tests (real connector, no mocks) and
skill-retention tests for both shared and separated filesystems.

* build(docker): ship a self-contained nsjail sandbox backend in the image

Compile nsjail 3.6 from source in a dedicated multi-stage build and carry
only the binary plus its runtime libs (libprotobuf32, libnl-route-3-200)
into the final image. This lets the Box runtime isolate sandboxed code via
nsjail user/mount/pid/net namespaces without a host Docker socket — the
prerequisite for running Box on LangBot Cloud (k8s), where mounting
docker.sock would grant node root and is not acceptable for multi-tenant.

The build toolchain (build-essential/bison/flex/protobuf-dev/libnl-dev)
stays in the nsjail-build stage and is not present in the shipped image.

Verified: image builds (583MB), nsjail --help exits 0, libraries resolve,
and the real NsjailBackend executes an isolated command end-to-end on a
v6.1/cgroup2 host matching LangBot Cloud prod (rlimit fallback path, since
container /sys/fs/cgroup is read-only; PID-namespace isolation confirmed).

* feat(box): SaaS guard to force a single global sandbox scope

Add system.limitation.force_box_session_id_template: when non-empty it
overrides every pipeline's box-session-id-template at resolve time, pinning
all queries to one shared sandbox (e.g. {global}). This is the authoritative,
unbypassable guard — it runs on every exec call, so editing the pipeline
config via API cannot escape it. The web UI locks the Sandbox Scope selector
via a combined box_scope_editable flag (box available AND not forced).

* build(deps): pin langbot-plugin==0.4.2b1 (nsjail cgroup container-safety beta)

* fix(web): show forced sandbox scope + make disabled tooltip tap-friendly

When a SaaS deployment pins every pipeline to a fixed sandbox scope via
system.limitation.force_box_session_id_template, the Sandbox Scope selector was
correctly locked but still displayed the pipeline's stored value (e.g. the
per-chat default), misrepresenting the scope that the runtime actually enforces
on every exec. Coerce the displayed/saved value to the forced template so the
locked selector truthfully shows the active scope (e.g. Global).

Also fix the disabled_tooltip being invisible on touch devices: hover-only Radix
tooltips never open without a pointer, so the explanation of why the field is
locked could not be read on mobile. Wrap the info icon so a tap toggles the
tooltip while desktop hover still works.

* feat(web): hide sidebar new-version prompt for edition=cloud

Cloud instances are upgraded centrally by the operator, so surfacing a GitHub
'new version available' badge to tenants is misleading and actionable only by
the operator. Skip the release check entirely when edition=cloud.

* style(web): prettier formatting for DisabledTooltipIcon ternary

* chore(deps): bump langbot-plugin to 0.4.2b2

Picks up the SDK fix that creates a read-write host_path before the
nsjail bind-mount, fixing the SaaS MCP shared-workspace sandbox failure
(exec exit 255 with empty output when host_path didn't exist).

* chore(deps): bump langbot-plugin to 0.4.2b3

Picks up the nsjail /dev-node fix so stdio MCP servers (uvx-launched) can
start under force_global_sandbox instead of failing with 'Connection closed
/ please check URL'.

* fix(web): show real MCP runtime status on installed extensions list

The installed-extensions list badge keyed solely off the enable flag, so a
server that was still CONNECTING (or in ERROR) was shown as 'Connected'.
Reflect the actual runtime_info.status (connecting/connected/error/disabled)
with matching colors, and poll quietly every 3s while any MCP server is
connecting so the badge transitions without a manual refresh.

* chore(deps): bump langbot-plugin to 0.4.2b4

Picks up the 30s start_managed_process timeout so cold uvx MCP bootstraps
don't get torn down mid-install.

* style(web): satisfy prettier — parenthesize nullish-coalescing in ternary

* fix(mcp): isolate transient test sessions from the shared Box session

A config-page 'test' (server_name='_', no persisted UUID) ran in the same
shared 'mcp-shared' Box session as live MCP servers. A failing test (e.g.
empty args) churned that shared session and tore down healthy, already-
connected servers — leaving them stuck after exhausting their retries.

Mark UUID-less sessions as transient, give them their own isolated Box
session ('mcp-test-<uuid>'), and fully delete that session on cleanup so
tests can never disturb live servers and don't leak sessions.

* fix(mcp): tear down transient test session after test completes

A successful config-page test left its isolated 'mcp-test-<uuid>' Box
session running (the lifecycle task blocks until shutdown). Wrap the
transient test coroutine so it always shuts the session down afterward,
preventing isolated test sessions from leaking.
2026-06-09 19:30:17 +08:00
RockChinQ 47fe9bde03 docs(docker): move k8s deployment docs to wiki, drop README_K8S.md
The Kubernetes deployment guide now lives only in the wiki
(docs.langbot.app -> Installation -> Kubernetes). Remove the in-repo
docker/README_K8S.md, repoint the README language variants and the
docker-compose / kubernetes.yaml header comments to the wiki, and keep
kubernetes.yaml self-describing via inline comments.
2026-06-07 11:36:39 -04:00
RockChinQ 5c3a619e2d docs(docker): add Box sandbox runtime to k8s manifest and deploy guide
The k8s manifest was missing the Box runtime that backs the sandbox
tools, the activate skill tool, skill add/edit and stdio MCP. Add a
langbot-box Deployment/Service (port 5410), wire langbot to it via
BOX__RUNTIME__ENDPOINT (explicit Service name since the in-container
default langbot_box uses an underscore, invalid for k8s DNS), and share
the Box workspace root as a node hostPath pinned via podAffinity so the
node Docker daemon resolves bind-mount paths consistently. Document the
component, the shared-FS constraint, security implications and readiness
checks in README_K8S.md (zh + en).
2026-06-07 11:18:27 -04:00
RockChinQ e223edeb45 docs(agents): add --standalone-box flag and box config keys 2026-06-07 08:57:43 -04:00
RockChinQ d2c3146334 docs(agents): refresh AGENTS.md for current architecture and runtime/box debugging 2026-06-07 08:43:30 -04:00
Haoxuan Xing 7d9c8e3065 Merge pull request #2231 from langbot-app/TyperBody-patch-1
Update key capabilities in README.md
2026-06-07 13:08:19 +08:00
Haoxuan Xing f12ed81e1e Update key capabilities in README.md
Added links to Deerflow and Weknora in the capabilities section.
2026-06-07 13:05:46 +08:00
Haoxuan Xing 6d4d19b6d7 Merge pull request #2230 from langbot-app/feat/addweknoradeerflow
Add DeerFlow LangGraph API as a Provider Runner
2026-06-07 12:22:55 +08:00
Typer_Body 07b90f12a2 ruff3 2026-06-07 02:38:05 +08:00
Typer_Body fd896c6974 ruff2 2026-06-07 02:35:10 +08:00
Typer_Body 1fbfa868fb ruff 2026-06-07 02:31:42 +08:00
Typer_Body ad05819c2e readme 2026-06-07 02:26:25 +08:00
Typer_Body 0c6f71738c deerflow 2026-06-07 02:17:40 +08:00
Typer_Body af451e7006 weknora2 2026-06-07 01:14:02 +08:00
Typer_Body 59f20bcc73 weknora 2026-06-07 01:08:39 +08:00
RockChinQ 7eca3cdfca feat(web): show sub-entity name in document title on detail pages
Detail pages (plugin / MCP / pipeline / knowledge base / skill) only showed
the type in the tab title. Drive the /home document title from HomeLayout,
which has the selected entity name via context: '<entity> · <type> · LangBot'
when a sub-entity is open, '<type> · LangBot' otherwise. The top-level hook
now skips /home and only handles login/register/reset-password/wizard.
Type label falls back to a route-derived i18n key on direct page loads.
2026-06-06 12:12:08 -04:00
RockChinQ c40354f838 feat(web): dynamic document title per route
The browser tab title was hard-coded to 'LangBot' in index.html and never
changed. Add a useDocumentTitle hook that maps the active route to an
existing i18n key and sets document.title to '<page> · LangBot', driven by
a new top-level RootLayout route element. Re-runs on navigation and on
language change so the title stays localized. Falls back to the bare app
name for unmapped routes.
2026-06-06 12:07:41 -04:00
RockChinQ 21a5b4658a fix(plugin-market): keep fixed card width regardless of result count
The result grid used auto-fit tracks, so a single search result stretched
to fill the whole row. Switch to fixed responsive column counts (1/2/3/4
across breakpoints), matching langbot-space, so cards keep a consistent
max width no matter how many results are shown.
2026-06-06 11:40:02 -04:00
RockChinQ 073acaa053 feat(plugin-market): move extension count into search box placeholder
Mirror the langbot-space marketplace change: drop the '共 xxx 个扩展'
stats line below the tag filter, surface the count in the search
placeholder ('搜索 xxx 个扩展、能力或场景...') when no query is active,
and show the total at the bottom via allLoadedCount when searching.
Adds searchPlaceholderCount + allLoadedCount to all 8 locales.
2026-06-06 11:33:46 -04:00
RockChinQ 38759b229d feat(plugin-market): show per-format extension counts in type filter
Mirror the LangBot Space marketplace: the advanced-filter type options
(plugin / MCP / skill) now display their live extension count, e.g.
"插件 (74)". Counts are fetched on mount via three lightweight
searchMarketplaceExtensions calls (page_size=1) reading total per type.
The all-formats option intentionally shows no count.
2026-06-06 08:11:59 -04:00
RockChinQ efe32e34ae fix(deps): patch Dependabot vulnerability alerts (Python + web)
Python (pyproject.toml + uv.lock):
- aiohttp 3.13.5->3.14.0, langchain-core 1.3.2->1.4.1, langsmith 0.7.36->0.8.9,
  lxml 6.0.2->6.1.1, Mako 1.3.11->1.3.12, PyJWT 2.11.0->2.13.0,
  python-multipart 0.0.26->0.0.32, urllib3 2.6.3->2.7.0, Pygments 2.19.2->2.20.0,
  idna 3.11->3.18, pip 26.0->26.1.2, python-dotenv 1.2.1->1.2.2,
  requests 2.32.5->2.34.2, starlette 0.52.1->1.2.1, uv 0.11.7->0.11.19

web (package.json + both lockfiles):
- axios ->1.17.0, postcss ->8.5.15, react-router(-dom) ->7.17.0 (direct)
- overrides for transitive: flatted >=3.4.2, follow-redirects >=1.16.0,
  minimatch (3.1.3 / 9.0.7), picomatch (2.3.2 / 4.0.4)
- regenerated both package-lock.json and pnpm-lock.yaml in sync

Verified: uv sync + core imports OK; pnpm --frozen-lockfile + tsc + vite build pass.

Not fixable (no upstream patch yet, tracked separately):
- chromadb (critical, <=1.5.9 is latest) — awaiting upstream release
- PyPDF2 (medium, deprecated; needs migration to pypdf, code change)
2026-06-06 06:06:59 -04:00
Junyan Chin 46db4de11a Update QQ Group link in README_CN.md 2026-06-06 17:20:19 +08:00
RockChinQ 170a6756f4 fix(add-extension): load real icon in install confirm dialog from URL params
When the install confirm dialog is opened via URL query params (e.g. from a
marketplace deep link), installInfo carried no icon, so the icon fell back to
the /resources/icon endpoint which 404s for extensions whose icon is an
external URL (simpleicons / iconify), showing a Package placeholder.

Fetch the icon from the marketplace detail API (mcp/skill/plugin) after opening
the dialog and inject it into installInfo, and reset the icon-failed state when
the resolved URL changes so the <img> retries instead of sticking on the
placeholder.
2026-06-06 04:45:46 -04:00
RockChinQ 7330732f62 fix(ci): bump migration head assertion to 0004, apply prettier
- Update test_migrations / test_migrations_postgres head assertion from
  0003 to 0004 after adding the mcp readme migration.
- Reformat MCPForm.tsx / MCPReadme.tsx to satisfy prettier/prettier.
2026-06-06 03:56:14 -04:00
RockChinQ b08e5ca09a feat(mcp): add Docs/Tools tablist on detail page, tidy sidebar label
Wrap the MCP detail right panel in a compact left-aligned Docs/Tools
tablist (Docs first). Move the tool count into the Tools tab label and
drop the redundant panel title/subtitle; connecting/failed states still
render the status component. Shorten the sidebar 'Installed Extensions'
entry to 'Installed' across all 8 locales, and add tabTools/tabDocs/
noReadme strings.
2026-06-06 03:52:17 -04:00
RockChinQ dff80a0c0a fix(marketplace): use external icon URL when icon field is absolute
Many MCP / skill records store their icon as an absolute external URL
(simpleicons.org / iconify.design) rather than an uploaded file, so the
/resources/icon endpoint 404s and the card icon breaks. Add
resolveMarketplaceIconURL() which prefers an absolute http(s) icon field
and otherwise falls back to the resources endpoint.
2026-06-06 03:52:09 -04:00
RockChinQ f54ae4b91c feat(mcp): persist and display marketplace README
Capture the README markdown from LangBot Space when installing an MCP
server and store it on the mcp_servers record (new readme column +
alembic migration 0004). The detail page can then render docs offline,
independent of the server's runtime/connection state.
2026-06-06 03:52:00 -04:00
RockChinQ e5b3cced1f feat(market): show 24 plugins per page 2026-06-05 11:33:02 -04:00
Junyan Qin 101e04db6d feat(web): add Discord link to sidebar account menu
Add a "Join our Discord" entry to the account dropdown's external-links
group, opening https://discord.gg/wdNEHETs87 in a new tab. lucide-react
has no Discord brand glyph, so include a small inline Discord SVG icon
(brand color). Add the joinDiscord label to all 8 locales.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-04 22:26:55 +08:00
Junyan Qin b79edda3a7 style(web): give extension cards a subtle border
The softened shadow alone left cards with no visible edge against the
page background. Add `border border-border` so each card has a clear,
restrained boundary while keeping the gentle shadow.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-04 21:49:55 +08:00
Junyan Qin a20d3d11e5 style(web): soften extension card shadow and hover effect
Reduce the marketplace card box-shadow (4px/0.2 -> 2px/0.06) and the
hover shadow (8px/0.15 -> 5px/0.08, dark proportional) for a more
restrained, understated look.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-04 21:45:35 +08:00
Junyan Qin 3b4c455813 fix(web): distinct extension-format icons (plugin/mcp/skill)
The format filter used Wrench/AudioWaveform/Book for plugin/mcp/skill,
which collided with the plugin-component icons (Tool/EventListener/
KnowledgeEngine) shown right below. Switch formats to Puzzle/Server/
Sparkles — matching the canonical getTypeIcon used by the detail badges
— across the market filter, installed filter, install-queue map and
install-progress dialog.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-04 21:34:23 +08:00
Junyan Qin c967a2aa82 i18n(market): say "extensions" not "plugins" in the marketplace count
The marketplace now lists plugins, MCPs and skills, so the item count
("Total N plugins") read wrong. Update market.totalPlugins and
market.searchResults to "extensions" across all 8 locales.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-04 21:24:10 +08:00
Junyan Qin 79cc6da96f fix(mcp): surface real cause from TaskGroup ExceptionGroups
MCP connection failures were reported as "unhandled errors in a
TaskGroup (1 sub-exception)" because anyio/the MCP client wrap the real
error in an ExceptionGroup and we interpolated its str() directly. Add
_describe_exception() to recurse into ExceptionGroups and surface the
leaf cause (e.g. "httpx.HTTPStatusError: Client error '410 Gone'") in
both the retry warning and the final error_message.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-04 21:19:18 +08:00
Junyan Qin fee7d48dc3 refactor(web): drop redundant Manual/Scan tabs in model add popover
The model add/scan popover nested a second Manual/Scan tab row inside
the Chat/Embedding/Rerank type tabs. But ProviderCard already opens the
popover from two distinct entry points (Add -> manual, Scan -> scan via
initialMode), so the inner tabs were redundant. Render the manual form
or scan UI directly off `mode` and remove the inner Tabs/TabsList,
leaving a single clean tab row.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-04 18:36:59 +08:00
Junyan Qin 8811fb647f fix(plugin): call _inspect_plugin_package in marketplace install path
Marketplace plugin install referenced self._extract_deps_metadata,
which no longer exists (renamed to _inspect_plugin_package), raising
AttributeError and failing every plugin install from Space. Use the
current method name; it extracts identity + dependency metadata as
the local-install path already does.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-04 18:17:01 +08:00
Junyan Qin 37b017459d fix(modelmgr): upsert Space-managed models instead of insert-only
sync_new_models_from_space() skipped any model whose uuid already
existed. LangBot Space reuses a model's uuid across renames/re-specs
(e.g. the uuid that was claude-opus-4-6 later becomes claude-opus-4-7),
so renamed models never propagated locally — the stale local name was
also sent to the models gateway, causing model_not_found at inference.

Now upsert: create new uuids, and for existing models owned by the
Space provider, update name/abilities/ranking to track Space (models
from other providers are left untouched). Logs added/updated counts.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-04 18:11:26 +08:00
Junyan Qin 4889a3881b chore(release): bump version to 4.10.0
Version-only bump from 4.10.0-beta.3. No release/tag/publish.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-04 17:26:03 +08:00
Junyan Qin fe4f95b9a3 fix(docker): install docker CLI for box backend; bump to 4.10.0-beta.3
The langbot_box service drives sandbox containers through the docker CLI
(CLISandboxBackend shells out to `docker run`/`docker exec`), but the
image shipped without a docker client, so DockerBackend.is_available()
was always false and the Box sandbox backend was unavailable in Docker
deployments — disabling native tools, skill execution and stdio MCP.
Install docker-ce-cli (client only) in the image, arch-aware so
multi-arch builds work.

Also bump langbot-plugin pin to 0.4.1, which disables proxy
auto-detection on internal control-plane WebSocket connections (the
langbot<->plugin_runtime / langbot<->box handshakes were failing on
hosts that inject a proxy into containers).

Bumps version to 4.10.0-beta.3.
2026-06-04 13:20:36 +08:00
Junyan Qin a2817f6524 chore(release): bump version to 4.10.0-beta.2
The 4.10.0-beta.2 release built and tried to publish 4.10.0b1 (the version was
never bumped), which PyPI rejected as a duplicate. Bump pyproject.toml and
__init__.py to 4.10.0-beta.2.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-03 23:24:37 +08:00
Junyan Qin b9560b26ff Revert "chore(tooling): wire CodeGraph MCP server + agent guidance"
This reverts commit 1ad7071aa0.
2026-06-03 23:17:34 +08:00
Junyan Qin 1ad7071aa0 chore(tooling): wire CodeGraph MCP server + agent guidance
Add the codegraph stdio MCP server to .mcp.json and the CodeGraph usage
guidance block to AGENTS.md, so coding agents working in this repo can use the
codegraph_* structural-search tools.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-03 23:14:21 +08:00
huanghuoguoguo 96b041846d Feat/sandbox (#2072)
* feat: add mcp and skills

* feat: add filter

* feat: modify frontend

* feat(box): add sandbox_exec tool loop for local-agent calculations

* feat(box): add host workspace mounting and sandbox_exec guidance

* feat(box): add BoxProfile with resource limits and improved output truncation

  - Implement head+tail output truncation (60/40 split) so LLM sees both
    beginning and final results; add streaming byte-limited reads in backend
    to prevent unbounded memory usage (_MAX_RAW_OUTPUT_BYTES = 1MB)
  - Define BoxProfile model with locked fields and max_timeout_sec clamping
  - Add four built-in profiles: default, offline_readonly, network_basic,
    network_extended with differentiated resource and security constraints
  - Add resource limit fields to BoxSpec (cpus, memory_mb, pids_limit,
    read_only_rootfs) and pass corresponding container CLI flags
    (--cpus, --memory, --pids-limit, --read-only, --tmpfs)
  - Profile loaded from config (box.profile), applied in service layer
    before BoxSpec validation; locked fields cannot be overridden by
    tool-call parameters

* feat(box): add obs

* refactor(box): unify box service lifecycle and local runtime
  management

* refactor(box): remove legacy in-process runtime code and clean up smells

After the architecture settled on always using an independent Box Runtime
service, several pieces of compatibility code and design shortcuts were
left behind. This commit cleans them up:

- Remove `LocalBoxRuntimeClient` and `create_box_runtime_client` from
  production code (moved to test-only helper).
- Remove unused `_clip_bytes` method from backend.
- Remove `__langbot_session_placeholder__` hack by making `BoxSpec.cmd`
  default to empty and validating non-empty only in `runtime.execute()`.
- Extract `get_box_config()` helper to eliminate 5× duplicated config
  access boilerplate.
- Remove `session_id`/`host_path`/`host_path_mode` from the LLM-facing
  tool schema to enforce request-scoped session isolation.
- Fix dual shutdown path: `NativeToolLoader.shutdown()` no longer calls
  `box_service.shutdown()` (handled by `Application.dispose()`).
- Simplify `_assert_session_compatible` with a loop.
- Inline client creation in `BoxRuntimeConnector`.
- Remove redundant `BOX__RUNTIME_URL` env var from docker-compose
  (auto-detected by code).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat: add test

* fix: fix box intergration test

* feat(box/mcp): integrate MCP stdio with Box sandbox — auto-isolation, dep install, security

  ## Summary

  When Podman/Docker is available, all stdio-mode MCP servers now automatically
  run inside Box containers with dependency installation, path rewriting, and
  lifecycle management. When no container runtime exists, LangBot starts normally
  and stdio MCP falls back to host-direct execution.

  ## What changed

  ### MCP stdio → Box integration (mcp.py)
  - Add `MCPServerBoxConfig` pydantic model for structured box configuration
    with validation and defaults (network, host_path_mode, timeouts, resources)
  - Auto-infer `host_path` from command/args with venv detection: recognizes
    `.venv/bin/python` patterns and walks up to the project root
  - Rewrite host paths to container `/workspace` paths transparently
  - Replace venv python commands with container-native `python`
  - Auto-detect `pyproject.toml`/`setup.py`/`requirements.txt` and run
    `pip install` inside the container before starting the MCP server
  - Copy project to `/tmp` before install to handle read-only mounts
  - Add retry with exponential backoff (3 retries, 2s/4s/8s delays)
  - Add Box managed process health monitoring (poll every 5s)
  - Fix session leak: `_cleanup_box_stdio_session()` now runs in `finally`
    block of `_lifecycle_loop`, covering all exit paths
  - Fix retry logic: `_ready_event` is only set after all retries exhaust
    or on success, not on first failure
  - Enhance `get_runtime_info_dict()` with `box_session_id` and `box_enabled`

  ### Box security (security.py — new)
  - `validate_sandbox_security()` blocks dangerous host paths:
    `/etc`, `/proc`, `/sys`, `/dev`, `/root`, `/boot`, `/run`,
    docker.sock, podman socket
  - Called at the start of `CLISandboxBackend.start_session()`

  ### Box models (models.py)
  - Add `BoxHostMountMode.NONE` — skips volume mount entirely
  - Adjust `validate_host_mount_consistency` to allow arbitrary workdir
    when `host_path_mode=NONE`

  ### Box backend (backend.py)
  - Add `validate_sandbox_security()` call in `start_session()`
  - Add `langbot.box.config_hash` label on containers for drift detection
  - Handle `BoxHostMountMode.NONE` — skip `-v` mount arg
  - Add `cleanup_orphaned_containers()` to base class (no-op default) and
    CLI implementation (single batched `rm -f` command)

  ### Box runtime (runtime.py)
  - Call `cleanup_orphaned_containers()` during `initialize()` to remove
    lingering containers from previous runs

  ### Box service (service.py)
  - Graceful degradation: `initialize()` catches runtime errors and sets
    `available=False` instead of crashing LangBot startup
  - Add `available` property and guard on `execute_sandbox_tool()`
  - Add `skip_host_mount_validation` parameter to `build_spec()` and
    `create_session()` — MCP paths are admin-configured and trusted,
    bypassing `allowed_host_mount_roots` restrictions meant for
    LLM-generated sandbox_exec commands

  ### Default behavior
  - stdio MCP servers automatically use Box when `box_service.available`
    is True (Podman/Docker detected); no explicit `box` config needed
  - When no container runtime exists, falls back to host-direct stdio
  - MCP Box defaults: `network=on` (for pip install), `read_only_rootfs=false`
    (for site-packages), `host_path_mode=ro`, `startup_timeout=120s`

  ### Tests
  - `test_box_security.py`: blocked paths, safe paths, subpath rejection
  - `test_mcp_box_integration.py`: config model, path rewriting, venv
    unwrap, host_path inference, payload building, runtime info, box
    availability check
  - `test_box_service.py`: `BoxHostMountMode.NONE` validation tests

* feat(box/mcp): instance-based orphan cleanup, error classification, session API, and integration tests

  ## Changes

  ### Precise orphan container cleanup
  - Runtime generates a unique instance_id on startup
  - Every container gets a `langbot.box.instance_id` label
  - `cleanup_orphaned_containers()` only removes containers from
    previous instances, preserving containers owned by the current one
  - Containers from older versions (no label) are also cleaned up
  - `cleanup_orphaned_containers` added to `BaseSandboxBackend` as
    a no-op default method, removing hasattr duck-typing

  ### Fine-grained MCP error classification
  - New `MCPSessionErrorPhase` enum with 7 phases: session_create,
    dep_install, process_start, relay_connect, mcp_init, runtime,
    tool_call
  - Each phase in `_init_box_stdio_server()` sets the error phase
    before re-raising, enabling precise failure diagnosis
  - `retry_count` tracked across retry attempts
  - `get_runtime_info_dict()` exposes `error_phase` and `retry_count`

  ### GET /v1/sessions/{id} API
  - `BoxRuntime.get_session()` returns session details including
    managed process info when present
  - `handle_get_session` HTTP handler + route in server.py
  - `BoxRuntimeClient.get_session()` abstract method + remote impl

  ### stdio defaults to Box when runtime is available
  - `_uses_box_stdio()` checks `box_service.available` instead of
    requiring explicit `box` key in server_config
  - `BoxService.initialize()` catches runtime errors gracefully,
    sets `available=False` instead of crashing LangBot startup
  - When no container runtime exists, stdio MCP falls back to
    host-direct execution

  ### Code quality (from /simplify review)
  - Extracted `_VENV_DIRS` / `_VENV_BIN_DIRS` module-level constants
  - Removed dead `_box_network_mode()` method and unused `bc` variable
  - Fixed broken import `from ....box.models` → `from ...box.models`
  - Cached `_resolve_host_path()` result — computed once, passed through
  - Config hash now includes `host_path` field
  - Batched orphan cleanup into single `rm -f` command

  ### Session leak fix
  - `_cleanup_box_stdio_session()` now runs in `_lifecycle_loop`'s
    finally block, covering all exit paths (normal shutdown, error,
    retry, final failure)

  ### Integration tests
  - 6 end-to-end tests covering managed process lifecycle, WebSocket
    stdio bidirectional IO, session cleanup verification, single
    session query, process exit detection, and orphan cleanup safety

* refactor: use rpc

* fix: import

* refactor(box): clean up sandbox subsystem code quality and efficiency

  - Fix O(n²) stderr trimming in runtime.py with running length tracker
  - Remove dead code: RESERVED_CONTAINER_PATHS, _subprocess_wait_task,
    unused config_hash computation, unused imports
  - Deduplicate connection callback in BoxRuntimeConnector, parse URL once
  - Use enum comparison instead of stringly-typed spec.network.value check
  - Replace manual _result_to_dict/_session_to_dict with model_dump()
  - Cache NativeToolLoader tool definition and sandbox system guidance
  - Extract _is_path_under() helper to eliminate duplicated path checks
  - Import SANDBOX_EXEC_TOOL_NAME from native.py instead of redefining
  - Add JSON startswith guard in logging_utils to skip futile json.loads
  - Fix ruff lint errors (F401 unused imports, F841 unused variables)

* fix: ruff

* refactor(sandbox): keep box logic out of pipeline and localagent

  - Move sandbox system-prompt guidance from LocalAgentRunner into
    BoxService.get_system_guidance() so all box domain knowledge stays
    in the box module.
  - Remove standalone logging_utils.py; merge format_result_log() into
    MessageHandler base class alongside cut_str().
  - Strip sandbox-specific JSON parsing from log formatting; tool
    results now use generic truncation.
  - Revert TYPE_CHECKING changes in stage.py and runner.py that were
    unrelated to this feature.
  - Skip two test files affected by a pre-existing circular import
    (runner ↔ app) until the import cycle is resolved in a separate PR.

* fix: ruff

* refactor(box): move box runtime to langbot-plugin-sdk

  Extract self-contained box runtime modules (actions, backend, client,
  errors, models, runtime, security, server) to langbot-plugin-sdk and
  update all imports to use `langbot_plugin.box.*`. Keep only service
  and
  connector in LangBot core as they depend on the Application context.

  - Update docker-compose to use `langbot_plugin.box.server` entry
  point
  - Update pyproject.toml to use local SDK via `tool.uv.sources`
  - Remove migrated source files and their unit/integration tests
  - Update remaining test imports to match new module paths

* fix: ruff

* feat: enhance sandbox api

* refactor(box): derive paths from shared host root

* fix(box): tighten sandbox exposure and restore box integration coverage

* refactor(types): remove quoted annotations under postponed evaluation

* feat(box): unify native agent tools around exec/read/write/edit

* chore(sandbox): move MCP loader changes to follow-up branch

* feat(box): add session workspace quota enforcement and SDK quota metadata

* feat(skills): add Agent Skills management system (#1917)

* feat(skills): add Agent Skills management system

Implement comprehensive skills management feature inspired by agentskills spec:

Backend:
- Add Skill and SkillPipelineBinding database entities
- Add database migration (dbm018) for skills tables
- Implement SkillManager for skill loading, matching, and resolution
- Implement SkillService for CRUD operations
- Add skills API endpoints for skill and pipeline binding management
- Integrate skill index injection into pipeline preprocessor
- Add skill activation detection in LocalAgentRunner

Frontend:
- Add Skills page with listing, search, and type filter
- Add SkillDetailDialog for create/edit with preview
- Add SkillCard and SkillForm components
- Add skills API methods to BackendClient
- Add skills entry to sidebar navigation
- Add i18n translations (en-US, zh-Hans)

Features:
- Support skill and workflow types
- Sub-skill composition via {{INVOKE_SKILL: name}} syntax
- Progressive disclosure (index in prompt, full instructions on activation)
- Pipeline-specific skill bindings with priority

* fix: resolve cherry-pick conflicts for agentskills onto sandbox

- Remove non-existent external_kb service import
- Add skill_mgr mock to localagent sandbox_exec tests
- Keep database version at 24 (sandbox branch's latest)

* feat(skills): upgrade to package-backed skills with sandbox execution

  Evolve the skills system from pure prompt-based to package-backed with
  sandbox tool execution support:

  - Add source_type/package_root/entry_file/skill_tools fields to Skill entity
  - SkillManager loads SKILL.md from local package directories
  - SkillToolLoader as 4th dispatch layer in ToolManager (query-scoped)
  - LocalAgent injects skill tools into use_funcs on skill activation
  - BoxService.execute_skill_tool() runs scripts in sandbox (ro mount, env params)
  - Skill tool names auto-namespaced as skill__{skill}__{tool}
  - API validation for package_root allowlist and entry path traversal
  - Frontend source_type toggle, package_root input, skill_tools editor
  - Migration renumbered to 025 with ALTER TABLE fallback for existing DBs
  - Fix unclosed limitation section in i18n files
  - Fix skills API methods misplaced outside BackendClient class

* fix: test info

* feat(skills): switch skills to package-backed storage and add import tooling
  - skills 从 inline/package 双轨收敛成 package-first
  - instructions 改为写入并读取 SKILL.md
  - 新增本地目录扫描和 GitHub 安装 skill
  - 前端把 skills 整合进 plugins 页,新增 SkillsComponent 和 GitHub 导入弹窗
  - skill form 去掉 source_type / type 筛选,改成目录扫描驱动
  - Box skill tool 挂载模式从 ro 改成 rw
  - 测试和中英文文案同步更新

* feat: simplify langbot skill create and import

* refactor(skills): clean up legacy skill API and harden activation flow

* refactor(skills): remove skill dependency expansion and add skill_get

* fix: lint

* fix: delete

* fix(skills): align tool manager loader initialization

* refactor: remove sandbox execute skill

* fix(skills): hide activation markers and isolate skill activation flow

* refactor(skills): switch skill model to filesystem-backed packages

* refactor(skills): switch skill model to filesystem-backed packages

* refactor(skills): unify runtime skill access around filesystem paths

* refactor(skills): unify runtime skill access around filesystem paths

* feat(skills): align rw package design and fix skill activation, visibility, and lint issues

* refactor(skills): replace rich authoring API with import/reload flow and update
  Box design doc

* feat(box): add sandbox_exec tool loop for local-agent calculations

* feat(box): add host workspace mounting and sandbox_exec guidance

* feat(box): add BoxProfile with resource limits and improved output truncation

  - Implement head+tail output truncation (60/40 split) so LLM sees both
    beginning and final results; add streaming byte-limited reads in backend
    to prevent unbounded memory usage (_MAX_RAW_OUTPUT_BYTES = 1MB)
  - Define BoxProfile model with locked fields and max_timeout_sec clamping
  - Add four built-in profiles: default, offline_readonly, network_basic,
    network_extended with differentiated resource and security constraints
  - Add resource limit fields to BoxSpec (cpus, memory_mb, pids_limit,
    read_only_rootfs) and pass corresponding container CLI flags
    (--cpus, --memory, --pids-limit, --read-only, --tmpfs)
  - Profile loaded from config (box.profile), applied in service layer
    before BoxSpec validation; locked fields cannot be overridden by
    tool-call parameters

* feat(box): add obs

* refactor(box): unify box service lifecycle and local runtime
  management

* refactor(box): remove legacy in-process runtime code and clean up smells

After the architecture settled on always using an independent Box Runtime
service, several pieces of compatibility code and design shortcuts were
left behind. This commit cleans them up:

- Remove `LocalBoxRuntimeClient` and `create_box_runtime_client` from
  production code (moved to test-only helper).
- Remove unused `_clip_bytes` method from backend.
- Remove `__langbot_session_placeholder__` hack by making `BoxSpec.cmd`
  default to empty and validating non-empty only in `runtime.execute()`.
- Extract `get_box_config()` helper to eliminate 5× duplicated config
  access boilerplate.
- Remove `session_id`/`host_path`/`host_path_mode` from the LLM-facing
  tool schema to enforce request-scoped session isolation.
- Fix dual shutdown path: `NativeToolLoader.shutdown()` no longer calls
  `box_service.shutdown()` (handled by `Application.dispose()`).
- Simplify `_assert_session_compatible` with a loop.
- Inline client creation in `BoxRuntimeConnector`.
- Remove redundant `BOX__RUNTIME_URL` env var from docker-compose
  (auto-detected by code).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(box/mcp): integrate MCP stdio with Box sandbox — auto-isolation, dep install, security

  ## Summary

  When Podman/Docker is available, all stdio-mode MCP servers now automatically
  run inside Box containers with dependency installation, path rewriting, and
  lifecycle management. When no container runtime exists, LangBot starts normally
  and stdio MCP falls back to host-direct execution.

  ## What changed

  ### MCP stdio → Box integration (mcp.py)
  - Add `MCPServerBoxConfig` pydantic model for structured box configuration
    with validation and defaults (network, host_path_mode, timeouts, resources)
  - Auto-infer `host_path` from command/args with venv detection: recognizes
    `.venv/bin/python` patterns and walks up to the project root
  - Rewrite host paths to container `/workspace` paths transparently
  - Replace venv python commands with container-native `python`
  - Auto-detect `pyproject.toml`/`setup.py`/`requirements.txt` and run
    `pip install` inside the container before starting the MCP server
  - Copy project to `/tmp` before install to handle read-only mounts
  - Add retry with exponential backoff (3 retries, 2s/4s/8s delays)
  - Add Box managed process health monitoring (poll every 5s)
  - Fix session leak: `_cleanup_box_stdio_session()` now runs in `finally`
    block of `_lifecycle_loop`, covering all exit paths
  - Fix retry logic: `_ready_event` is only set after all retries exhaust
    or on success, not on first failure
  - Enhance `get_runtime_info_dict()` with `box_session_id` and `box_enabled`

  ### Box security (security.py — new)
  - `validate_sandbox_security()` blocks dangerous host paths:
    `/etc`, `/proc`, `/sys`, `/dev`, `/root`, `/boot`, `/run`,
    docker.sock, podman socket
  - Called at the start of `CLISandboxBackend.start_session()`

  ### Box models (models.py)
  - Add `BoxHostMountMode.NONE` — skips volume mount entirely
  - Adjust `validate_host_mount_consistency` to allow arbitrary workdir
    when `host_path_mode=NONE`

  ### Box backend (backend.py)
  - Add `validate_sandbox_security()` call in `start_session()`
  - Add `langbot.box.config_hash` label on containers for drift detection
  - Handle `BoxHostMountMode.NONE` — skip `-v` mount arg
  - Add `cleanup_orphaned_containers()` to base class (no-op default) and
    CLI implementation (single batched `rm -f` command)

  ### Box runtime (runtime.py)
  - Call `cleanup_orphaned_containers()` during `initialize()` to remove
    lingering containers from previous runs

  ### Box service (service.py)
  - Graceful degradation: `initialize()` catches runtime errors and sets
    `available=False` instead of crashing LangBot startup
  - Add `available` property and guard on `execute_sandbox_tool()`
  - Add `skip_host_mount_validation` parameter to `build_spec()` and
    `create_session()` — MCP paths are admin-configured and trusted,
    bypassing `allowed_host_mount_roots` restrictions meant for
    LLM-generated sandbox_exec commands

  ### Default behavior
  - stdio MCP servers automatically use Box when `box_service.available`
    is True (Podman/Docker detected); no explicit `box` config needed
  - When no container runtime exists, falls back to host-direct stdio
  - MCP Box defaults: `network=on` (for pip install), `read_only_rootfs=false`
    (for site-packages), `host_path_mode=ro`, `startup_timeout=120s`

  ### Tests
  - `test_box_security.py`: blocked paths, safe paths, subpath rejection
  - `test_mcp_box_integration.py`: config model, path rewriting, venv
    unwrap, host_path inference, payload building, runtime info, box
    availability check
  - `test_box_service.py`: `BoxHostMountMode.NONE` validation tests

* feat(box/mcp): instance-based orphan cleanup, error classification, session API, and integration tests

  ## Changes

  ### Precise orphan container cleanup
  - Runtime generates a unique instance_id on startup
  - Every container gets a `langbot.box.instance_id` label
  - `cleanup_orphaned_containers()` only removes containers from
    previous instances, preserving containers owned by the current one
  - Containers from older versions (no label) are also cleaned up
  - `cleanup_orphaned_containers` added to `BaseSandboxBackend` as
    a no-op default method, removing hasattr duck-typing

  ### Fine-grained MCP error classification
  - New `MCPSessionErrorPhase` enum with 7 phases: session_create,
    dep_install, process_start, relay_connect, mcp_init, runtime,
    tool_call
  - Each phase in `_init_box_stdio_server()` sets the error phase
    before re-raising, enabling precise failure diagnosis
  - `retry_count` tracked across retry attempts
  - `get_runtime_info_dict()` exposes `error_phase` and `retry_count`

  ### GET /v1/sessions/{id} API
  - `BoxRuntime.get_session()` returns session details including
    managed process info when present
  - `handle_get_session` HTTP handler + route in server.py
  - `BoxRuntimeClient.get_session()` abstract method + remote impl

  ### stdio defaults to Box when runtime is available
  - `_uses_box_stdio()` checks `box_service.available` instead of
    requiring explicit `box` key in server_config
  - `BoxService.initialize()` catches runtime errors gracefully,
    sets `available=False` instead of crashing LangBot startup
  - When no container runtime exists, stdio MCP falls back to
    host-direct execution

  ### Code quality (from /simplify review)
  - Extracted `_VENV_DIRS` / `_VENV_BIN_DIRS` module-level constants
  - Removed dead `_box_network_mode()` method and unused `bc` variable
  - Fixed broken import `from ....box.models` → `from ...box.models`
  - Cached `_resolve_host_path()` result — computed once, passed through
  - Config hash now includes `host_path` field
  - Batched orphan cleanup into single `rm -f` command

  ### Session leak fix
  - `_cleanup_box_stdio_session()` now runs in `_lifecycle_loop`'s
    finally block, covering all exit paths (normal shutdown, error,
    retry, final failure)

  ### Integration tests
  - 6 end-to-end tests covering managed process lifecycle, WebSocket
    stdio bidirectional IO, session cleanup verification, single
    session query, process exit detection, and orphan cleanup safety

* refactor: use rpc

* fix: import

* refactor(box): clean up sandbox subsystem code quality and efficiency

  - Fix O(n²) stderr trimming in runtime.py with running length tracker
  - Remove dead code: RESERVED_CONTAINER_PATHS, _subprocess_wait_task,
    unused config_hash computation, unused imports
  - Deduplicate connection callback in BoxRuntimeConnector, parse URL once
  - Use enum comparison instead of stringly-typed spec.network.value check
  - Replace manual _result_to_dict/_session_to_dict with model_dump()
  - Cache NativeToolLoader tool definition and sandbox system guidance
  - Extract _is_path_under() helper to eliminate duplicated path checks
  - Import SANDBOX_EXEC_TOOL_NAME from native.py instead of redefining
  - Add JSON startswith guard in logging_utils to skip futile json.loads
  - Fix ruff lint errors (F401 unused imports, F841 unused variables)

* fix: ruff

* refactor(sandbox): keep box logic out of pipeline and localagent

  - Move sandbox system-prompt guidance from LocalAgentRunner into
    BoxService.get_system_guidance() so all box domain knowledge stays
    in the box module.
  - Remove standalone logging_utils.py; merge format_result_log() into
    MessageHandler base class alongside cut_str().
  - Strip sandbox-specific JSON parsing from log formatting; tool
    results now use generic truncation.
  - Revert TYPE_CHECKING changes in stage.py and runner.py that were
    unrelated to this feature.
  - Skip two test files affected by a pre-existing circular import
    (runner ↔ app) until the import cycle is resolved in a separate PR.

* refactor(box): move box runtime to langbot-plugin-sdk

  Extract self-contained box runtime modules (actions, backend, client,
  errors, models, runtime, security, server) to langbot-plugin-sdk and
  update all imports to use `langbot_plugin.box.*`. Keep only service
  and
  connector in LangBot core as they depend on the Application context.

  - Update docker-compose to use `langbot_plugin.box.server` entry
  point
  - Update pyproject.toml to use local SDK via `tool.uv.sources`
  - Remove migrated source files and their unit/integration tests
  - Update remaining test imports to match new module paths

* fix: ruff

* fix(box): tighten sandbox exposure and restore box integration coverage

* refactor(types): remove quoted annotations under postponed evaluation

* chore(sandbox): move MCP loader changes to follow-up branch

* refactor(plugins): simplify GitHub install flow to default master archive

* revert(api): restore plugin GitHub import flow in plugins controller

* Improve data-root handling and skill install previews

* Add managed skill authoring tools for local agents

* Refactor the skills UI around sidebar detail pages

* Document why managed skill authoring tools bypass box

* fix: lint

* feat(web): refactor plugin/skill install flows and fix skills page

- Fix sidebar skill icon
- Add skills route and error page component
- Refactor plugin GitHub install from dialog modal to inline card
- Add skill install dropdown menu (create/upload/github) in sidebar
- Wire sidebar → skills page communication via pendingSkillInstallAction context
- Add i18n keys for error page and skill install actions

* fix(web): persist sidebar collapsible section open state on navigation

Sections opened via sub-item navigation now retain their expanded state
when the user switches to a different section, instead of collapsing
because the isActive fallback becomes false.

---------

Co-authored-by: youhuanghe <1051233107@qq.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: Junyan Qin <rockchinq@gmail.com>

* feat(sandbox): add MCP box integration on top of sandbox base (#2083)

* refactor(mcp): extract box stdio runtime helper

* refactor(box): introduce reusable workspace session helper

* refactor(box): run Box Runtime as subprocess inside LangBot container

  Remove the separate langbot_box_runtime Docker service. Box Runtime
  now always launches as a local stdio subprocess, regardless of whether
  LangBot runs in Docker or not. The WebSocket transport path is kept
  only for explicit runtime_url configuration (remote deployment).

  This simplifies deployment by eliminating cross-container path mapping
  and network hops. Box Runtime is a pure scheduling process (talks to
  Docker socket / nsjail), it does not execute user code or touch the
  filesystem, so container isolation is unnecessary — unlike Plugin
  Runtime.

* fix(web): prevent first-emission snapshot from swallowing unsaved changes in pipeline editor

When switching runner (e.g. local-agent → n8n), the newly mounted stage's
first emit would re-capture the saved snapshot, erasing the dirty state
caused by the runner change. The save button would incorrectly go dim.

- Skip snapshot re-capture in handleDynamicFormEmit when form is already dirty
- Add mount-time emit to N8nAuthFormComponent (matching DynamicFormComponent)
- Use stable onSubmitRef to prevent useEffect subscription churn
- Add previousInitialValues guard to prevent initialValues echo loops

* style(web): align plugin list header button heights

* docs(review): update Box architecture review documents

Replace old review docs with 5 focused documents:
- box-architecture.md: deep architecture analysis (LangBot + SDK)
- box-issues.md: 22 issues rated P0/P1/P2
- box-test-coverage.md: test coverage analysis
- box-tob-analysis.md: toB commercialization analysis
- box-vs-plugin-runtime.md: Box vs Plugin runtime comparison

* feat(web): improve login error layout and add Terms of Service link

- Improve backend connection error display with bordered container,
  inline icon, and better visual hierarchy
- Extract actual error message from axios response object
- Add Terms of Service link (https://langbot.app/terms) to login footer
- Add termsOfService i18n key for all 7 locales

* refactor(web): replace all hardcoded SVG icons with lucide-react

Unify icon usage across the entire frontend by replacing 67 hardcoded
SVG icons with lucide-react components across ~25 files. This improves
consistency, maintainability, and reduces bundle duplication.

Key replacements:
- Sidebar nav: Zap, LayoutDashboard, Bot, Workflow, BookMarked, etc.
- MCP forms: Loader2, XCircle, Trash2
- Monitoring: Sparkles, MessageSquare, CheckCircle2, RefreshCw, etc.
- Cards: Clock, Star, Workflow, Hexagon, Puzzle, Github, etc.
- Misc: Paperclip, AudioLines, CloudUpload, Layers, Heart, Smile

Zero hardcoded <svg> tags remain in .tsx files.

* fix(web): stop polling plugin tasks when no active installs

The PluginInstallTaskProvider was unconditionally polling
getAsyncTasks every 3s on all /home/* routes. Now it only
syncs once on mount and starts periodic polling only when
there are active (non-terminal) install tasks.

* fix(deps): update langbot-plugin version and add new dependencies

* refactor: use Space API for release checks and stop idle polling

- version.py: switch release list API from GitHub to space.langbot.app,
  remove unused in-place update logic (update_all, compare_version_str),
  translate all comments/logs to English
- PluginInstallTaskContext: only poll when active install tasks exist

* feat(box): add --standalone-box flag and 3-way transport decision for Box runtime

Align Box runtime connection logic with Plugin runtime's pattern:
- Docker: WebSocket to langbot_box container (ws://langbot_box:5411)
- --standalone-box: WebSocket to external Box process (ws://localhost:5411)
- Windows: subprocess + WebSocket (workaround for async stdio limitation)
- Unix/macOS: subprocess + stdio pipe (unchanged)

BoxRuntimeConnector now inherits ManagedRuntimeConnector for subprocess
lifecycle reuse. Add langbot_box service to docker-compose.yaml.

* refactor(box): use single port with path-based routing for Box WS

Update connector to use ws://host:5410/rpc/ws instead of ws://host:5411.
Update review docs to reflect the single-port architecture.

* feat(web): show Box runtime status in plugin debug info popover

Add Box status section to the debug info popover on the plugin list page,
displaying connection status, backend info, profile, active sessions,
and recent error count. Fetched from GET /api/v1/box/status in parallel
with plugin debug info. Includes i18n for all 8 supported languages.

* fix(web): remove ephemeral sandbox count from Box status display

The active_sessions count reflects transient sandbox containers that
expire after 5 minutes of inactivity, making it misleading in the UI.
Keep only connection status, backend, profile, and error count.

* feat(box): configurable sandbox scope and unified skill containers

Replace the per-message session_id with a template-based system
configurable per pipeline via 'Sandbox Scope' in the local-agent panel.
Default scope is per-chat ({launcher_type}_{launcher_id}).

Unify skill exec into the same container as default exec — skills are
mounted at /workspace/.skills/{name}/ via extra_mounts instead of
getting separate containers. All pipeline-bound skills are injected
at container creation time.

- Add box-session-id-template to pipeline metadata (select, 4 options, 8 languages)
- Add resolve_box_session_id() and build_skill_extra_mounts() to BoxService
- Rewrite native.py skill exec path to use execute_tool with shared session
- Update tests for new session_id format
- Add design doc: docs/review/box-session-scope.md

* feat(web): show active sandbox details in Box status popover

Display sandbox count and a detailed list of active sessions including
session ID, image, backend, resources (CPU/memory), network mode, and
last used time. Fetched from GET /api/v1/box/sessions in parallel.
Includes i18n for all 8 supported languages.

* feat(box): add startup and availability logging for sandbox tools

Log Box runtime initialization result (success with profile info, or
failure warning). Log native tool availability status at ToolManager
startup so it's immediately clear whether exec/read/write/edit tools
are registered for the LLM.

* feat(box): support custom sandbox container image via config.yaml

Add 'image' field to box config section. When set, it overrides the
profile default image (python:3.11-slim) for all sandbox containers.
Priority: caller-specified > config.yaml image > profile default.

* feat(box): add heartbeat and reconnection for Box runtime connector

Add 20-second heartbeat ping loop to detect silent Box runtime
disconnections. On disconnect, set available=false and attempt
reconnection after 3 seconds via the disconnect callback chain.

- BoxRuntimeConnector: heartbeat loop, disconnect callback parameter,
  disconnect detection in connection callback and WS failure handler
- BoxService: wire disconnect callback to toggle available state and
  re-initialize the connector on reconnection

* feat(web): move runtime status to dashboard, clean up plugin debug popover

Add SystemStatusCards component to the monitoring dashboard showing
Plugin Runtime and Box Runtime connection status with details (backend,
profile, sandbox count). Remove all Box/session status from the plugin
page debug popover — it now only shows debug URL and key.

Includes i18n for all 8 supported languages.

* refactor(web): compact system status into a single card alongside metrics

Replace the separate two-card row with a single compact 'System Status'
card placed as the 5th column in the metrics grid. Shows green/red dots
for Plugin Runtime and Box Runtime. Click to expand a popover with
connection details (backend, profile, sandbox count).

* feat: show connector error details for Plugin and Box runtime status

Record Box connector error in BoxService and expose it as
'connector_error' in GET /api/v1/box/status when unavailable.
Display error messages in the dashboard System Status popover
for both Plugin Runtime (plugin_connector_error) and Box Runtime
(connector_error) when they are disconnected.

* fix(web): auto-refresh system status and show disconnect errors in real time

Poll Plugin Runtime and Box Runtime status every 30 seconds so the
dashboard reflects disconnections without a manual page refresh.
Also re-fetch when the popover is opened for immediate feedback.

* fix(box): handle RPC failure in get_status/get_sessions gracefully

When the Box runtime disconnects, there is a race between the heartbeat
flipping _available=false and the frontend polling get_status(). If the
poll arrives first, client.get_status() throws a ConnectionClosedError
which propagated as a 500, causing the frontend to show a grey dot
(null status) instead of a red dot with error details.

Now get_status() catches RPC errors and returns available=false with
the exception message as connector_error. get_sessions() returns an
empty list when unavailable or on RPC failure.

* fix(box): add persistent reconnection loop with exponential backoff

The previous disconnect handler only retried once and then gave up.
Now spawns a background task that retries with exponential backoff
(3s, 6s, 12s, ... up to 60s) until the Box runtime is reachable again.
Uses a _reconnecting guard to prevent duplicate loops. Calls
connector.dispose() before each retry to clean up stale tasks.

* fix(box): detect disconnect when handler.run() returns normally

The generic Handler.run() catches ConnectionClosedError and breaks out
of its loop (normal return) instead of raising, because it has no
disconnect_callback. The old code only triggered reconnection in the
except branch, so a clean WebSocket close was never detected.

Now treat handler.run() returning normally (after successful handshake)
as a disconnect event, triggering the reconnection callback.

* fix(web): refresh system status card when clicking Refresh Data button

Pass a refreshKey prop through OverviewCards to SystemStatusCard that
increments on each Refresh Data click, triggering a re-fetch of Plugin
and Box runtime status alongside the monitoring data refresh.

* fix(web): fix system status card stuck in loading state

fetchStatus(showLoading=false) never called setLoading(false), so the
initial loading=true was never cleared. Simplify to always setLoading
in the finally block — the spinner only shows on the very first load
since subsequent fetches complete near-instantly.

* feat(web): show active sandbox details in dashboard Box status popover

Fetch box sessions alongside status and display each active sandbox
in the popover with session ID, image, resources (CPU/memory), and
last used time.

* feat(box): add global sandbox scope option

Add a 'Global (shared by all)' option to the sandbox scope selector.
Uses a constant '{global}' template variable that always resolves to
'global', so all users and chats share one sandbox container.

* refactor(web): replace popover with dialog for system status details

Replace the dropdown popover with a proper Dialog for runtime status
details. Add a small info button on the System Status card that opens
the dialog. Session details now show in a spacious 2-column grid layout
with full image name, backend, CPU/memory, network, mount path, and
created/last-used timestamps.

* fix(web): widen system status dialog and fix scroll border issue

Use max-w-2xl (matching other dialogs) instead of max-w-lg. Move
overflow-y-auto to an inner container with overflow-hidden on
DialogContent to prevent padding bleed at scroll edges.

* feat(web): add tooltips for truncated fields in system status dialog

Wrap session_id, image, and mount path fields with Tooltip components
so hovering over truncated text shows the full value.

* feat: add download button

* feat: successfully install

* feat: delete old filter

* feat: youhua frontend

* fix: align box runtime launch args

* feat: translate

* feat: refactor market

* feat: youhua qianduan

* chore: rename extension zh translation

* feat(extensions): unify extensions endpoint and refresh extensions page UX

- Rename /home/plugins route to /home/extensions and update all sidebar links.
- Add unified GET /api/v1/extensions returning plugins, MCP servers and skills,
  sorted by name; replace the three separate frontend fetches with this single call.
- Migrate the extensions page to shadcn primitives (Tabs/Card/Alert/Badge/Skeleton/
  Switch/Label) and clean up hardcoded color tokens on the extension card.
- Add a localStorage-persisted "Group by type" switch that, when enabled in the
  All Types tab, renders extensions grouped by type with a compact section header.
- Show a spinner while loading and rename the empty-state copy from
  "No plugins installed" to "No extensions installed".
- Rename the "格式 / Formats" filter label to "类型 / Types" across all 8 locales.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(extensions): fallback lucide icon when extension icon is missing

Render a tinted lucide icon (Puzzle / Server / Sparkles) on the extension
card when the icon URL is empty or the image fails to load. Picked icons
distinct from EventListener (AudioWaveform) and KnowledgeEngine (Book) to
avoid visual collision with plugin component badges.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(sidebar): unify installed-extensions list with plugins, MCP and skills

- Render plugins, MCP servers and skills together under the "Installed
  Extensions" sidebar entry, alphabetically sorted to match the list page.
- Resolve per-item routes by extension type (plugin -> /home/extensions,
  mcp -> /home/mcp, skill -> /home/skills) and gate the plugin-only hover
  context menu on extensionType === 'plugin'.
- Lift the "group by type" toggle into SidebarDataContext (still persisted
  in localStorage) so the sidebar groups items with section headers
  whenever the list page has the toggle enabled.
- Show lucide fallback icons (Server / Sparkles / Puzzle) tinted in the
  LangBot blue for MCP, skill, and missing-icon plugin items, overriding
  the SidebarMenuSubButton svg color rule.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(extensions): mobile-friendly layout for extensions and add-extension pages

- Stack the extensions page header vertically on small screens, let the
  filter Tabs scroll horizontally if they overflow, hide the debug
  button label below sm and let the install/debug controls wrap.
- Constrain the debug popover and its inputs to the viewport width so
  they no longer overflow on phone-sized screens.
- Drop the card grid from a fixed 30rem column to a min(100%, 22rem)
  column at base / 28rem at sm, and reduce the gap, so cards render
  cleanly at 360px+ widths in both flat and grouped views.
- Make the add-extension header actions wrap on lg- viewports and the
  install dialog responsive instead of a hard 500px box.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat: change ui

* feat: delete version for mcp and skills

* fix: constrain home page content width

* fix: preserve monitoring card borders under sticky filters

* fix(box): restore sandbox config and shared mcp runtime

* fix(box): harden sandbox session isolation

* fix(skill): remove auto activation setting

* feat(skill): align skill system with Claude Code's Tool Call design

- Replace text marker activation with `activate` tool (Tool Call mechanism)
- Replace 7 authoring tools with 2: `activate` + `register_skill`
- Add builtin skills loading from templates/skills/
- Add create-skill as first builtin skill
- Remove SKILL_ACTIVATION_MARKER and text detection methods
- Tool Result returns SKILL.md content (protects KV Cache)

This aligns with Claude Code's progressive disclosure pattern:
- Metadata (name+description) always visible in tool description
- SKILL.md body loaded on activate via Tool Call
- Bundled resources accessible through virtual path mapping

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* feat(tools): add glob and grep native sandbox tools

Add file discovery and content search capabilities to the sandbox:
- glob: Find files by pattern (supports ** recursive matching)
- grep: Search file contents with regex patterns

Both tools respect skill package paths and include safety limits
(max 100 files for glob, max 200 matches for grep).

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* feat(skill): add skill file browsing capability

- Add API endpoints for listing/reading/writing skill files
- Add FileTree component in SkillForm for directory browsing
- Users can now view scripts/, references/, assets/ directories
- Files can be selected and edited in the instructions textarea
- Add translations for new file browsing features

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* fix(skill): copy builtin skills to data/skills on startup

- Builtin skills (templates/skills/) are now copied to data/skills/
- Users can view and manage builtin skills in the UI
- Rename SkillAuthoringToolLoader to SkillToolLoader

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* fix(skill): improve file browsing and fix path handling

- Fix nested directory display in skill file tree (preserve root entries)
- Fix file content display when clicking files in skill browser
- Add skill manager and tool manager as proper package modules
- Separate fileContent state to allow editing non-SKILL.md files

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* fix(toolmgr): correct skill_tool_loader attribute name

Rename skill_authoring_tool_loader to skill_tool_loader in execute_func_call
and shutdown methods to match the attribute defined in initialize().

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* fix(native): update tool descriptions to use register_skill

Replace references to removed import_skill_from_directory with
register_skill in exec/write/edit tool descriptions.

* feat(toolmgr): enhance tool initialization with backend availability checks

* refactor: remove unused imports and clean up code in various files

* feat: polish extension detail pages

* feat: persist sidebar list expansion

* fix: refine extension ui and backend errors

* fix: align add extension marketplace ui

* feat: manage skills through box runtime

* feat: support github skill installation

* fix: import github skill directories

* feat: install market extensions from card click

* feat(web): improve skill import flow

* feat: polish extension import flow

* fix(mcp): stabilize shared box managed processes

* fix(web): improve backend retry and sidebar scrolling

* docs(review): refresh box architecture review for feat/sandbox

Sync the docs/review/ suite to the current state of the feat/sandbox branch
(both LangBot and langbot-plugin-sdk), ~30 commits ahead of the prior review.

- box-architecture.md: rewrite for the new box.{backend,runtime,local,e2b}
  config schema, add E2B backend, 6 native tools (incl. glob/grep), Skill
  Tool Call activation, shared multi-process MCP container, SkillManager,
  BoxSkillStore (SDK), 25 actions, 9 error types, heartbeat/reconnect
- box-issues.md: move resolved items (reconnect, heartbeat, Windows, nsjail
  image conflict, frontend monitoring card) into a Resolved section; add
  new P0 (INIT/backend ordering), P1 (extra_mounts immutability after
  container creation), P2 (skill_store test gap, integration tests not in CI)
- box-session-scope.md: add §0 Implementation Status — Phase 1 shipped,
  MCP unification landed earlier than originally scoped
- box-test-coverage.md: realign file inventory (4,400 -> 6,500 LOC),
  add 7 new test files including SDK backend_selection/e2b/skill_store
- box-tob-analysis.md: connection recovery now满足基本要求; add E2B and
  backend self-heal to capabilities; tick off Phase 1 reconnect/heartbeat
- box-vs-plugin-runtime.md: heartbeat/reconnect/Windows support now aligned
  with Plugin Runtime; revise remaining gaps (WS auth, shared base class)

* refactor(box): use unified env-override mechanism for box.local config

The box module hand-rolled its own LANGBOT_BOX_LOCAL_* env parsing in two
places (connector._get_box_config and service._local_config), duplicating
logic that LoadConfigStage._apply_env_overrides_to_config already provides
generically via the SECTION__SUBSECTION__KEY convention.

- Drop the bespoke LANGBOT_BOX_LOCAL_* parsing; read box.local straight
  from instance_config (the unified BOX__LOCAL__* overrides are already
  applied before BoxService initializes)
- Harden _load_allowed_mount_roots to accept a comma-separated string,
  since the generic mechanism stores a freshly-created key as a raw
  string when config.yaml has no box.local.allowed_mount_roots entry
- docker-compose: rename the langbot container env vars to
  BOX__LOCAL__* (the canonical convention); remove them entirely from
  the langbot_box container — the Box runtime never reads box.local from
  env/config.yaml, it is configured via the INIT RPC action

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* test: repair stale skill/sandbox tests for feat/sandbox

The skill subsystem moved to Tool-Call activation and a Box-managed
skill store; several tests still asserted removed APIs and a sys.modules
stub leaked across the suite. Full unit suite now green (was 23 failing).

- test_skill_tools: drop TestSkillManagerActivation (text-marker API
  removed); rewrite TestSkillActivationHelper around the current
  skill.activation.register_activated_skill; replace the CRUD
  TestSkillAuthoringToolLoader with TestSkillToolLoader covering the
  current activate/register_skill tools and sandbox-availability gating
- test_tool_manager_native: ToolManager attr is skill_tool_loader (not
  skill_authoring_tool_loader); native loader now exposes 6 tools
  (exec/read/write/edit/glob/grep) and requires initialize() with a
  backend-available get_status()
- test_localagent_sandbox_exec: remove obsolete activation-marker
  leakage tests and their helper providers
- test_model_service / pipeline conftest: give the mocks skill_mgr=None
  so PreProcessor's local-agent skill-binding guard short-circuits
- test_n8nsvapi: stop permanently overwriting sys.modules
  ('langbot.pkg.provider.runner' etc.); save and restore around the
  import so other modules get the real LocalAgentRunner base class

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* ci(tests): run unit tests on every push to feat/** branches

- Add feat/** to push branches so long-lived feature branches are
  tested on every push (they accumulate large changes before a PR)
- Drop the push path filter entirely: every push to master/develop/
  feat/** now runs the full unit suite (the old 'pkg/**' filter never
  matched the real source path 'src/langbot/pkg/**', so backend-only
  pushes silently skipped tests)
- Fix the same broken path glob on the pull_request trigger
  ('pkg/**' -> 'src/langbot/pkg/**')

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(skill): harden mount/reload paths and HTTP errors against stale skill cache

The Box backends behave inconsistently when extra_mounts reference a
missing host directory (nsjail aborts the entire sandbox start, Docker
silently creates a root-owned empty dir on the host, E2B silently skips
the upload). The cache in skill_mgr.skills is only refreshed on
in-process mutations, so out-of-band changes — container rebuilds,
manual rm in the box volume, anything the LangBot API didn't drive —
leave a stale skill that later produces one of those bad mount paths.

- box/service.py: build_skill_extra_mounts now filters skills whose
  package_root is not isdir on the LangBot-visible filesystem and logs
  a warning, instead of passing the bad mount through to the backend
- skill/manager.py: reload_skills (Box path) drops skills whose
  package_root is missing on the LangBot-side filesystem before they
  reach the in-memory cache, with a summary warning
- api/http/controller/groups/skills.py: file/CRUD handlers now also
  catch BoxError (RuntimeError subclass, previously slipping past
  ``except ValueError`` and surfacing as 500); list/get handlers gain
  a try/except so a transient Box RPC failure becomes a clean 400
  instead of a stack trace

Tests added for build_skill_extra_mounts (skip missing, skip empty,
no skill manager) and SkillManager.reload_skills (drop missing on Box
path). Full unit suite: 279 passed.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(box): add box.enabled toggle and gate consumers on availability

Make the Box sandbox runtime optional. When ``box.enabled`` is false in
config (or when an enabled Box fails to connect), every dependent feature
degrades to the same disabled-state UX rather than crashing or silently
falling back to less safe code paths.

Backend:

- config.yaml: new top-level ``box.enabled: true`` flag (default true)
- BoxService:
  - Read box.enabled on construction
  - initialize() short-circuits when disabled — no remote WS connect, no
    stdio subprocess fork
  - _on_runtime_disconnect is a no-op when disabled (no reconnect loop
    on a deliberately-off service)
  - get_status() now exposes ``enabled`` so the frontend can tell
    "disabled in config" from "configured but failed"
- MCP stdio loader (mcp_stdio.uses_box_stdio): requires box_service to
  be available, not just installed
- MCP _init_stdio_python_server: when ap.box_service exists but is
  unavailable, refuse the stdio server with an actionable error instead
  of silently falling through to host-stdio (which bypasses the sandbox
  the operator asked for). Setups without ap.box_service installed at
  all keep the legacy host-stdio fallback for pre-Box dev mode
- SkillService._require_box_for_write: refuses create/update/install/
  write_skill_file when ap.box_service is installed but unavailable.
  Distinguishes disabled vs failed in the error message so the UI can
  surface the right hint. Legacy setups (no ap.box_service) keep the
  local fallback path — that distinction is what keeps the existing
  local-skills tests valid

Tests:
- Box disabled-state behavior (4 cases)
- Skill write refusal in disabled & failed states (7 cases)
- MCP stdio runtime info policy updated to match new refuse-when-down
  behavior

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(web): surface Box disabled/unavailable state across consumers

When Box is disabled in config (``box.enabled = false``) or fails to
connect, every dependent UI surface now degrades visibly:

- ``useBoxStatus`` hook: shared, polled 30s, exposes ``available``,
  ``disabled`` (config-off) and a single ``hint`` key so callers don't
  have to re-derive the three states
- ``BoxUnavailableNotice`` reusable Alert banner driven by that hint
- Dashboard SystemStatusCards: three-state dot + label
  (connected / disabled-gray / disconnected-red); disabled state shows
  the ``boxDisabled`` hint, failed state continues to show the connector
  error. Plugin block kept untouched
- Skills page (create view) and SkillDetailContent (edit view):
  Save button disabled and banner inserted above the form when Box is
  unavailable — matches the backend gate added in the previous commit
- PipelineExtension skill section: ``enable_all_skills`` switch, Add
  Skill button and Remove buttons all gate on Box availability;
  banner inline under the section header
- PipelineFormComponent: banner above the ``local-agent`` stage card
  when Box is unavailable, since that stage carries the sandbox-bound
  ``box-session-id-template`` field
- Box status payload type (``ApiRespBoxStatus.enabled``) and 8 locale
  files updated with ``boxDisabled`` / ``boxUnavailable`` /
  ``boxRequiredHint`` strings

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* docs(box): document the box.enabled toggle and gate behavior matrix

- docker-compose: move ``langbot_box`` under compose profiles
  (``box`` and ``all``) so ``docker compose up`` no longer requires
  the sandbox container. Inline comment explains how to pair the
  profile choice with ``box.enabled`` so the langbot service does not
  thrash trying to reach a runtime that was never started
- docs/review/box-architecture.md:
  - Annotate ``box.enabled`` in the config.yaml example, listing the
    exact side effects (no remote/stdio connect; tools/skills/MCP
    stdio off; reads still work)
  - Replace the bare compose snippet with the actual profile-driven
    invocation and the BOX__ENABLED pairing
  - New "关闭/连接失败时的行为矩阵" section: a single table mapping
    every consumer (native tools, activate/register_skill, stdio MCP,
    skill list/CRUD, pipeline AI config, extensions page, dashboard)
    to its disabled-state behavior, plus the legacy ``ap.box_service``
    distinguisher note

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* refactor(pipeline-form): swap Box banner for field-level disable_if + tooltip

The previous commit hard-coded a BoxUnavailableNotice banner above the
``local-agent`` stage card. That works, but it shouts at the user about
every field in that stage when in reality only one field —
``box-session-id-template`` — depends on the sandbox.

Use the dynamic-form schema's existing variable-injection mechanism
(``__system.*`` references via ``systemContext``) and add a sibling to
``show_if``: ``disable_if`` + ``disabled_tooltip``. The field stays
visible, becomes inert, and an info icon next to its label exposes the
reason on hover. The rest of the AI tab is left untouched.

- entities/form/dynamic.ts: extend IDynamicFormItemSchema with
  ``disable_if: IShowIfCondition`` and ``disabled_tooltip: I18nObject``
- DynamicFormComponent: evaluate disable_if with the same resolver as
  show_if; OR the result into isFieldDisabled; render an Info tooltip
  trigger next to the label when the condition matches
- ai.yaml metadata: attach disable_if (__system.box_available eq false)
  and a localized disabled_tooltip to box-session-id-template
- PipelineFormComponent: drop the BoxUnavailableNotice import and the
  per-stage banner; pass ``systemContext={ box_available: boxAvailable }``
  only for the local-agent stage so other stages aren't paying the
  re-render cost

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(mcp): friendly UI message when stdio MCP refused by Box state

Previously the MCP detail dialog dumped the raw RuntimeError text from
``_init_stdio_python_server`` — English-only, prefixed with "Failed
after 4 attempts", and exposing internal config names. The retry
wrapper also kept retrying a refusal that is deterministically going
to fail again, polluting logs.

Replace the raw text with a structured signal:

- New ``MCPSessionErrorPhase.BOX_UNAVAILABLE`` enum value. The stdio
  refusal path sets it before raising and uses a short opaque
  discriminator (``box_disabled_in_config`` / ``box_unavailable``) as
  the message body — never user-facing
- ``_lifecycle_loop_with_retry`` short-circuits on
  ``BOX_UNAVAILABLE``: surfaces the error immediately, no retries,
  no "Failed after N attempts" prefix. Silences the warning storm
  seen during smoke-testing
- ``MCPServerRuntimeInfo`` (TS type) now declares ``error_phase``,
  ``retry_count``, ``box_session_id``, ``box_enabled`` to match what
  the backend already returns in get_runtime_info_dict()
- Both MCP detail forms (``mcp/components/mcp-form/MCPForm.tsx`` and
  ``plugins/mcp-server/mcp-form/MCPFormDialog.tsx``) detect
  ``error_phase === 'box_unavailable'`` and render a two-line
  localized notice: state line ("Box disabled / unreachable") plus
  remediation line ("enable Box or switch to http/sse")
- 8 locale files (en/zh-Hans/zh-Hant/ja/ru/vi/th/es) get
  ``mcp.boxDisabledStdioRefused``, ``mcp.boxUnavailableStdioRefused``,
  ``mcp.boxStdioRefusedSuggestion``

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(mcp-web): block stdio MCP creation at the form when Box is unavailable

When Box is disabled in config (``box.enabled = false``) or unreachable,
saving a new MCP server in stdio mode produced one that could never
start — the user would only learn that from the runtime error on the
detail page. Stop the user before they save instead.

Both MCP forms (the page-level ``MCPForm.tsx`` and the older dialog
``MCPFormDialog.tsx``) now:

- Disable the ``stdio`` option in the mode select when Box is
  unavailable, with a small "(requires Box)" suffix so the reason is
  obvious. Existing stdio configs still display their current value
- Show ``BoxUnavailableNotice`` inline under the mode select when the
  currently-selected mode is stdio and Box is unavailable, so editing
  a stale stdio config makes the cause visible
- Disable the Save / Submit button while stdio is selected under that
  condition. ``MCPForm`` exposes a new ``onSaveBlockedChange`` prop
  so the parent ``MCPDetailContent`` can disable both its Submit and
  Save buttons. ``MCPFormDialog`` disables its Save button locally
- Refuse the submit handler too (Enter-key path) with a toast carrying
  the same i18n message

i18n: ``mcp.boxRequired`` (short tag in the disabled option) and
``mcp.stdioBlockedByBoxToast`` added to all 8 locales.

Backend runtime gate (``_init_stdio_python_server`` refusal +
``BOX_UNAVAILABLE`` error_phase + retry short-circuit) stays in place
as the last line of defence for API bypass.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(web): prevent plugin config form overflow

* refactor(skill): remove all local-filesystem fallbacks; Box is the sole source

Skills now flow exclusively through the Box runtime. Every read and write
method funnels through ``_box_service()``; when Box is unavailable
(disabled in config, connection failed, or simply not installed) the
operation either returns an empty surface (``list_skills`` → []) or
raises with a clear ``Box runtime ... not initialised / disabled /
unavailable: ...`` message via the new ``_require_box(action)`` helper.

Why: the legacy local-fallback path scanned ``data/skills/``, but Box
manages its own ``box.local.skills_root`` (default ``data/box/skills/``).
The two diverging directories caused stale / phantom skill lists when
Box flapped, and the local-fallback writes silently bypassed all the
sandboxing the operator had configured.

SkillService (``api/http/service/skill.py``):
- New ``_require_box(action)`` returns the box service or raises a
  structured ValueError. ``_require_box_for_write`` kept as alias
- ``list_skills`` → returns [] when Box is down so the UI can render
  the disabled banner cleanly
- ``get_skill`` / ``get_skill_by_name`` → return None
- All read-file / write-file / scan-dir / create / update / delete /
  install / preview methods → ``_require_box`` then box delegate.
  Local fallback bodies (shutil.copytree, tempfile.mkdtemp, preview
  pipelines) removed entirely

SkillManager (``pkg/skill/manager.py``):
- ``reload_skills`` returns early with empty cache when Box is down.
  data/skills/ discovery loop removed
- ``refresh_skill_from_disk`` now just reports cache presence; the
  on-disk re-parse is gone since Box is the only writer

Tests:
- Drop 11 obsolete test_skill_service.py tests that exercised the
  removed local-fallback paths (create/install/file/delete/update)
- Add list-empty + read-refused tests; flip the legacy-allow test to
  legacy-refuses-too
- Rewrite refresh_skill_from_disk test to match the new behaviour

Several helper methods (_managed_skill_path, _resolve_skill_path,
_preview_skill_candidates, _install_preview_candidates, etc.) are now
unreachable; a follow-up commit will prune them so this diff stays
reviewable.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* chore(skill): prune dead local-filesystem helpers left over from Box migration

Follow-up to the Box-only refactor. The previous commit removed the
local-fallback BRANCHES from every public method; this one removes the
HELPERS those branches called, which are now unreachable.

SkillService (service/skill.py): 787 → 449 lines
  Removed: scan_directory (sync), _read_skill_package, _write_skill_md,
  _resolve_create_field, _managed_skill_path,
  _managed_install_root_for_package, _normalize_package_root,
  _resolve_skill_path, _find_skill_entry, _discover_skill_directories,
  _safe_extract_zip, _extract_uploaded_skill_to_temp,
  _download_github_skill_to_temp, _resolve_github_source_root,
  _build_preview_target_dir, _preview_skill_candidates,
  _select_preview_candidates, _install_preview_candidates,
  _preview_source_root, _resolve_installed_skills, plus the
  module-level _FRONTMATTER_FIELDS and _build_skill_md.
  Kept (still needed by the surviving GitHub-import path):
  _download_github_asset, _download_github_skill_directory_as_zip,
  _find_github_skill_archive_entry, _copy_github_skill_directory_to_zip,
  _is_github_skill_md_url, _parse_github_skill_md_url,
  _resolve_github_skill_md_package_name, _validate_github_asset_url,
  _uploaded_skill_target_stem, _validate_skill_name.
  Imports dropped: shutil, tempfile, yaml, ....utils.paths.

SkillManager (skill/manager.py): 187 → 88 lines
  Removed: get_managed_skills_root, _discover_skill_directories,
  _find_skill_entry, _load_skill_file, _normalize_package_root.
  Imports dropped: datetime, parse_frontmatter, paths.

Tests:
  - test_skill_service.py: drop the 3 sync scan_directory tests +
    skill_service fixture + _create_skill_file helper
  - test_skill_tools.py: drop test_load_skill_file_success; rename
    TestSkillManagerPackageLoading → TestSkillManagerCache

Full unit suite: 277 passed, 1 skipped. ``ruff check`` clean.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(skill): re-inject skill index into local-agent system prompt

The contributor's original PR (#1917) appended an ``Available Skills``
index to the system prompt before the LLM saw the user message, so the
LLM could decide whether to activate a skill. ``7145447b`` removed the
text-marker activation flow and, together with it, the entire system
prompt injection — but the Tool Call replacement only put the available
skills inside the ``activate`` tool's description. In practice the LLM
ignores tool descriptions for selection and goes straight to native
tools, so user-visible skill activation silently broke.

Restore the injection, adapted for the Tool Call era:

- SkillManager regains ``get_skill_index(bound_skills)`` and
  ``build_skill_aware_prompt_addition(bound_skills)``. The addendum
  carries only ``name (display_name): description`` for each
  pipeline-visible skill plus one instruction line pointing at the
  ``activate`` tool. No SKILL.md contents — KV cache stays clean
- PreProcessor appends the addendum to the first system message (or
  inserts a new one) of ``query.prompt.messages`` for the local-agent
  runner. Handles plain-string and ContentElement[] bodies. Skips
  cleanly when no skills are visible
- 3 new test_preproc cases: injection happens, bound-skills subset
  honoured, empty addendum touches nothing. 280 passed

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(box): downgrade get_status.available when backend probed unavailable

Until now ``BoxService.get_status`` returned ``available: true`` whenever
the runtime connector was healthy, even if the runtime itself reported
``backend: { available: false }`` (operator selected nsjail without the
binary, Docker daemon crashed mid-session, E2B credentials wrong, ...).
The dashboard / ``useBoxStatus`` hook / skill_service gate consumed the
top-level flag and showed "connected" while every actual call to native
exec or skill management would fail.

The native-tool loader already polled ``status.backend.available``
independently and hid its tools correctly, but every other consumer
(dashboard banner, the disabled-state hint, the LLM-facing message)
disagreed with it.

Combine the two in the payload: ``available = self._available AND
status.backend.available``. When ``backend.available`` is false we now
also surface a ``connector_error`` that names the backend
("Configured sandbox backend \"nsjail\" is unavailable") so the dialog
shows the actionable reason instead of an empty error pane. The
detailed ``backend`` object is preserved unchanged for the dialog.

Internal ``box_service.available`` (used by ``skill_service`` writes,
``mcp_stdio.uses_box_stdio``, the reconnect callback) is intentionally
NOT changed — it still tracks connector health only, so a backend blip
does not trigger spurious reconnect loops.

Tests:
- ``test_get_status_downgrades_available_when_backend_dead`` — exercise
  the new branch (connector OK, backend.available=false → top-level
  available=false, connector_error mentions the backend name)
- ``test_get_status_keeps_available_true_when_backend_ok`` — guard
  against regressing the happy path

Live-verified with ``box.backend: nsjail`` on macOS (no nsjail binary):
``GET /api/v1/box/status`` now returns ``available: false`` with the
named connector_error, instead of the previous misleading
``available: true``.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(web): surface the specific Box failure reason in unavailable banner

When Box is configured but the runtime reports its backend is dead
(e.g. ``box.backend = nsjail`` but the binary is missing, or Docker
daemon crashed), the backend now returns a structured
``connector_error`` like ``Configured sandbox backend "nsjail" is
unavailable``. The previous notice only said "Box sandbox is
unavailable" + a generic "enable Box" hint, hiding the actionable
detail.

- ``useBoxStatus``: derive ``reason`` from ``status.connector_error``.
  Only exposed for the failed-state (``hint === 'boxUnavailable'``),
  since the disabled-by-config message already carries its reason
- ``BoxUnavailableNotice``: insert the reason as a small monospaced
  line between the state message and the action hint. The disabled
  variant is unchanged (operator chose the state)
- Wire ``reason`` through every existing call site (Skills page +
  detail, PipelineExtension, both MCP forms). Old unused ``context``
  prop dropped

Net layout (3 lines, still compact):

  ⚠ Box sandbox is unavailable — sandbox tools, skill add/edit, ...
    Configured sandbox backend "nsjail" is unavailable
    This feature requires the Box runtime. Enable it in config ...

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* test: reconcile master's unit tests with feat/sandbox refactors

The merge from master brought in new unit tests that target pre-refactor
APIs on feat/sandbox. Reconcile each:

- factories/app.py: FakeApp now exposes a Mock skill_mgr (with empty .skills
  dict + inert prompt-addition builder) and a Mock pipeline_service so the
  PreProcessor skill-index injection branch can run end-to-end in tests.

- pipeline/conftest.py: eagerly import langbot.pkg.pipeline.pipelinemgr so
  pipeline.stage is fully initialised before any individual stage test
  (preproc, longtext, ...) tries to lazy-load it. Without this preload,
  running test_preproc.py in isolation hit a circular-import error via the
  stage -> app -> pipelinemgr -> stage chain.

- provider/test_tool_manager.py: ToolManager now probes four loaders
  (native -> plugin -> mcp -> skill). Inject inert native + skill mocks in
  the execute_func_call fixture and assert all four shutdowns fire.

- utils/test_paths.py: drop the three cwd-dependent _check_if_source_install
  cases. The refactor walks Path(__file__).resolve().parents looking for
  pyproject.toml + main.py, so cwd no longer factors in and there's no
  file read to mock-fail. The positive case and caching test still apply.

- utils/test_version.py: delete entirely. is_newer and compare_version_str
  were removed when VersionManager was refactored to use the Space API for
  release checks (1b4107a9); the tests targeted a surface that no longer
  exists.

* refactor(box): launch box runtime via the lbp CLI subcommand

Mirror the plugin runtime: box is now started through the same CLI entry
point (langbot_plugin.cli) instead of the box module directly.

- docker-compose.yaml: langbot_box command runs `langbot_plugin.cli ... box`
  (WebSocket is the default transport, no flag needed — matches `rt`).
- box/connector.py: both subprocess launch sites (_start_local_stdio and
  the Windows _start_subprocess_then_ws path) invoke
  `langbot_plugin.cli.__init__ box`, using `-s` for the stdio transport.
- docs/review: update stale `-m langbot_plugin.box[.server]` references.

Pairs with the SDK change that removes box's direct-launch entry points
(python -m langbot_plugin.box / .box.server) and the legacy --mode flag.

* chore: bump langbot-plugin beta 1

* fix(ci): resolve langbot-plugin from PyPI and clear lint failures

CI on feat/sandbox failed across Unit Tests, Lint and Build Dev Image.
Root causes and fixes:

- pyproject.toml had a [tool.uv.sources] editable override pinning
  langbot-plugin to ../langbot-plugin-sdk. That path only exists in a
  paired local checkout, so `uv sync` failed on every CI runner
  ("Distribution not found"). Remove the override and regenerate uv.lock
  so langbot-plugin==0.4.0b1 resolves from PyPI, matching master.

- tests/integration/api/test_pipelines.py: the pipeline extensions
  endpoint now calls ap.skill_service.list_skills(); add the missing
  skill_service mock to the fake_pipeline_app fixture (the test came
  from master, the endpoint change from feat/sandbox).

- Apply ruff format to three src files and prettier to three web files
  that had committed formatting drift, failing `ruff format --check`
  and `pnpm lint`.

* chore: bump beta version

* docs: remove BOX_BACKEND override reference

* fix(pipelines): stop attributing dashboard debug WS to bound web_page_bot

The dashboard pipeline-debug WebSocket
(/api/v1/pipelines/<uuid>/ws/connect) and the embed widget WebSocket
(/api/v1/embed/<bot_uuid>/ws/connect) already live on separate paths,
but the debug handler ran `_find_owner_bot(pipeline_uuid)` and, when
the same pipeline happened to be bound to a web_page_bot, passed that
bot as `owner_bot` into `handle_websocket_message`. The adapter then
used the page bot's listeners + adapter for the request, so debug
sessions were logged as "page bot" activity in the dashboard.

Debug sessions must always run under the built-in websocket_proxy_bot.
Remove `_find_owner_bot`, drop the `owner_bot` parameter from the
debug-path `_handle_receive`, and call `handle_websocket_message`
without it so the adapter takes its default proxy-bot branch. The
embed handler still resolves and passes its `runtime_bot` for the
page-bot path, so attribution there is unchanged.

* fix(plugin): install marketplace MCP from canonical mode + extra_args

_install_mcp_from_marketplace read the dropped `mcp_data.config` field
and reconstructed mode/extra_args by guessing from the URL — which lost
stdio's command/args/env/box entirely, so stdio MCP installs from the
marketplace always failed.

Use the Space record's canonical `mode` and `extra_args` directly (the
same shape stored in mcp_servers), and gate the install on `mode`
instead of the removed `config`. After a successful install, best-effort
POST to the marketplace install endpoint to bump install_count.

* feat(web): show recommendation lists in plugin market; mixed-type icons

The marketplace recommendation lists (curated rows from Space) were never
mounted in the plugin market page. Wire them in:
- fetch recommendation lists on mount and render them above the extension
  grid, only when no search/filter is active.

Recommendation lists now mix plugins, MCPs and skills, so resolve each
card's icon by type (plugin / mcp / skill marketplace icon URL) instead of
always using the plugin icon endpoint.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(web): auto-open install dialog from one-click deep link

Accept a deep link from LangBot Space's one-click install:
/home/add-extension?install=1&extension_type=<plugin|mcp|skill>&author=&name=&version=
On mount, populate the install info, open the confirm dialog directly, and
strip the params from the URL. Reuses the existing marketplace install flow.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat: push marketplace URL to runtime; fix market client base race

- On connecting to the plugin runtime, push the configured space.url via the
  new SET_RUNTIME_CONFIG action so the runtime downloads plugins from the same
  Space, instead of relying on its own CLOUD_SERVICE_URL env/default. Wrapped
  in try/except so an older SDK without the action degrades gracefully.
- web: the plugin market fetched recommendation lists (and listings) via the
  sync cloud client before its baseURL was resolved from system info, so it
  hit the default space.langbot.app. Await getCloudServiceClient() before the
  initial fetches and for the recommendation list.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(web): don't show MCP "connection failed" while still connecting

The MCP status UI rendered "连接失败" for any non-connected state, so during a
normal connection attempt the subtitle showed "连接失败" while the status pill
below it showed "连接中..." — contradictory.

Only treat an explicit ERROR (or box-unavailable) status as failed; a
CONNECTING or initial/unresolved status now shows "连接中". Applied to the MCP
detail form (subtitle + StatusDisplay) and the MCP server card.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(web): type-aware install dialog + refresh sidebar after install

The marketplace install confirm dialog was hardcoded to "安装插件 / 确定要安装
插件 X 吗" for every type. Make it type-aware (plugin / MCP / skill) and show
more info: type chip, author/name id, and version when present.

Also refresh all sidebar extension lists (plugins, MCP servers, skills) when
an install task completes, so the newly-installed extension appears
immediately regardless of type (previously only refreshPlugins ran).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(web): richer install dialog (icon + name + description), drop redundant type row

The install dialog already states the type in its title, so the "类型" row was
redundant. Replace the info box with the extension's icon (avatar), display
name, author/name id + version, and description — built from the PluginV4 for
in-app installs and from the icon endpoint by type for the one-click deep link.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(web): TDZ crash in add-extension (installIconURL before installInfo)

installIconURL was computed above the useState declaration of installInfo,
causing "Cannot access 'installInfo' before initialization" (500) on the
add-extension page. Move the computation below the state declarations.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(web): redesign install-progress dialog for MCP/skill

The progress dialog showed plugin-only stages (download + dependency install)
for every type. MCP/skill have no such steps, so show a single
"installing → done/failed" row for them (MCP: adding & connecting the server;
skill: installing the package) while keeping the detailed download/deps
stages for plugins.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(web): add missing market.componentName i18n keys

The marketplace component filter (and component badges) used
market.componentName.{Tool,Command,EventListener,KnowledgeEngine,Parser,Page}
but those keys only existed under plugins.componentName, so the market UI
showed raw keys. Add a componentName block to the market namespace (zh-Hans +
en-US; other locales fall back to zh-Hans).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(web): sidebar extensions refresh button + full-name tooltip

- Add a refresh button to the installed-extensions category header in the
  sidebar; it re-fetches plugins + MCP servers + skills and spins while
  loading.
- The sidebar item tooltip now shows the extension's full name (with the
  description below when present), so truncated MCP/extension names are
  readable on hover.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(plugin-market): rename component filter to "插件组件" with hint tooltip + persist filters

- Rename the in-app plugin market component filter label to "插件组件" /
  "Plugin Component"
- Add an Info icon tooltip explaining what plugin components are (Tool /
  Command / EventListener, etc.)
- Persist filter selections (type / component / tags / sort) in localStorage
  so they survive reloads; restored on mount (URL type param still wins)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(plugin-market): restore missing "页面"(Page) component filter option

The market component-filter list on this branch was a diverged rewrite that
dropped the Page component kind master had added. The i18n key
(market.componentName.Page) already existed; re-add the Page entry to the
componentOptions list so plugins providing Page components can be filtered.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* docs(i18n): reword plugin component filter hint

Drop the redundant "插件组件是" lead-in and mention that components extend
LangBot's capabilities; mirror the wording in en-US.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(i18n): backfill missing market/addExtension keys in 6 locales

check-i18n surfaced that market.componentName.*, market.filterByComponentHint
and the addExtension.install* keys existed only in en-US/zh-Hans. Backfill
them for es-ES, ja-JP, ru-RU, th-TH, vi-VN and zh-Hant (reusing each locale's
existing component-name translations) and align the filterByComponent label
with the new "Plugin Component" wording. check-i18n now passes for all locales.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* i18n(plugins): relabel "group by type" as "group by format"

The installed-extensions grouping is by extension format (plugin / MCP / skill),
so rename the toggle label accordingly across all 8 locales (key unchanged).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(plugin-market): cursor-pointer on tag filter trigger

The TagsFilter Select trigger used the default cursor; add cursor-pointer so the
tag filter is clearly clickable.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(sidebar): show edition badge (Community / Cloud) in logo area

Add a small badge next to the LangBot name in the sidebar header that reflects
systemInfo.edition: a neutral "Community" badge for the community edition and a
blue "Cloud" badge for the cloud edition. Adds sidebar.editionCommunity /
sidebar.editionCloud across all 8 locales.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* i18n(sidebar): unify zh-Hans cloud edition label to 云端版

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(sidebar): edition badge - drop hover, use "Cloud" in all locales

The edition badge is not interactive, so remove the hover background on the
cloud badge. Also use the literal "Cloud" label uniformly across all locales
instead of localized variants (云端版/クラウド版/...).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(box): cap tool-call loop and run workspace-quota walk off the event loop

Two robustness fixes that bite under normal sandbox usage (not just attack),
hardening the self-hosted community edition before release:

- localagent: cap the tool-call loop at MAX_TOOL_CALL_ROUNDS (128). A looping
  or adversarial model could otherwise emit tool calls indefinitely (each
  potentially a sandbox exec), producing a non-terminating request and runaway
  cost. The cap is generous enough not to interrupt legitimate multi-step
  agentic workflows.
- box.service: make _enforce_workspace_quota async and run the recursive
  workspace scan via asyncio.to_thread. It ran on every quota-enforced exec and
  a large workspace would block the whole asyncio runtime (all bots/pipelines).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* docs(review): refresh box docs; trim issue list to SaaS blockers only

Community self-hosted edition is release-ready, so the box review docs are
updated to current state (date 2026-06-02 + status note) and box-issues.md is
rewritten to keep only the SaaS / multi-tenant / network-exposed release
blockers (S1-S8): unauthenticated control plane, no per-pipeline exec
authorization, unbounded sessions + no reaper, no kernel-level quota, mount
validation gaps (/ + extra_mounts), missing container hardening, lock-around-
cold-start, and the lower-severity follow-ups. Resolved items (tool-call loop
cap, async quota scan, host_path mount allowlist, _is_path_under dedup) moved to
a short "resolved before community release" record; community-only and
pure-cleanup items dropped.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* chore(deps): pin langbot-plugin to 0.4.0

Track the stable SDK release (0.4.0b1 -> 0.4.0); regenerate uv.lock.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: WangCham <651122857@qq.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: fdc310 <82008029+fdc310@users.noreply.github.com>
Co-authored-by: Junyan Qin <rockchinq@gmail.com>
2026-06-03 11:12:39 +08:00
RockChinQ 4054ba2a76 docs(issue-template): add deployment version selector 2026-06-01 23:31:29 -04:00
Dongchuan Fu c7cb42bd79 feat(lark): add domain configuration options for Lark adapter (#2220) 2026-05-27 15:34:35 +08:00
227 changed files with 28391 additions and 12134 deletions
+9
View File
@@ -10,6 +10,15 @@ body:
placeholder: 例如:v3.3.0、CentOS x64 Python 3.10.3、Docker placeholder: 例如:v3.3.0、CentOS x64 Python 3.10.3、Docker
validations: validations:
required: true required: true
- type: dropdown
attributes:
label: 部署版本
description: 请选择您使用的 LangBot 部署版本。
options:
- 社区版
- 云服务
validations:
required: true
- type: textarea - type: textarea
attributes: attributes:
label: 异常情况 label: 异常情况
+9
View File
@@ -10,6 +10,15 @@ body:
placeholder: "For example: v3.3.0, CentOS x64 Python 3.10.3, Docker" placeholder: "For example: v3.3.0, CentOS x64 Python 3.10.3, Docker"
validations: validations:
required: true required: true
- type: dropdown
attributes:
label: Deployment version
description: Please select the LangBot deployment version you are using.
options:
- Community Edition
- Cloud Service
validations:
required: true
- type: textarea - type: textarea
attributes: attributes:
label: Exception label: Exception
+4 -8
View File
@@ -15,14 +15,10 @@ on:
branches: branches:
- master - master
- develop - develop
paths: - 'feat/**'
- 'src/langbot/**' # No path filter on push: every push to the branches above runs the
- 'tests/**' # full unit-test suite. feat/** branches in particular must be tested
- '.github/workflows/run-tests.yml' # on every push (they accumulate large changes before a PR exists).
- 'pyproject.toml'
- 'uv.lock'
- 'run_tests.sh'
- 'scripts/test-*.sh'
jobs: jobs:
test: test:
+99 -46
View File
@@ -1,81 +1,134 @@
# AGENTS.md # AGENTS.md
This file is for guiding code agents (like Claude Code, GitHub Copilot, OpenAI Codex, etc.) to work in LangBot project. This file guides code agents (Claude Code, GitHub Copilot, OpenAI Codex, etc.) working in the LangBot project. `CLAUDE.md` is a symlink to this file.
## Project Overview ## Project Overview
LangBot is a open-source LLM native instant messaging bot development platform, aiming to provide an out-of-the-box IM robot development experience, with Agent, RAG, MCP and other LLM application functions, supporting global instant messaging platforms, and providing rich API interfaces, supporting custom development. LangBot is an open-source, LLM-native instant-messaging bot development platform. It aims to provide an out-of-the-box IM bot development experience with Agent, RAG, MCP and other LLM application capabilities, supporting mainstream global IM platforms and exposing rich APIs for custom development.
LangBot has a comprehensive frontend, all operations can be performed through the frontend. The project splited into these major parts: LangBot has a comprehensive web frontend almost every operation can be performed through it.
- `./src/langbot`: The main python package of the project, below are the main modules in this package: - **Python**: `>=3.11,<4.0`, dependencies managed by `uv`. Package version is in `pyproject.toml`.
- `./pkg`: The core python package of the project backend. - **Frontend**: `web/` is a **Vite + React Router 7 + shadcn/ui + Tailwind CSS** SPA, managed by `pnpm`. (Note: this is NOT Next.js — the `dev` script is `vite`.)
- `./pkg/platform`: The platform module of the project, containing the logic of message platform adapters, bot managers, message session managers, etc. - **Backend framework**: Quart (the async flavour of Flask). The HTTP API and the pre-built web UI are both served by the backend on `http://127.0.0.1:5300`.
- `./pkg/provider`: The provider module of the project, containing the logic of LLM providers, tool providers, etc.
- `./pkg/pipeline`: The pipeline module of the project, containing the logic of pipelines, stages, query pool, etc.
- `./pkg/api`: The api module of the project, containing the http api controllers and services.
- `./pkg/plugin`: LangBot bridge for connecting with plugin system.
- `./libs`: Some SDKs we previously developed for the project, such as `qq_official_api`, `wecom_api`, etc.
- `./templates`: Templates of config files, components, etc.
- `./web`: Frontend codebase, built with Next.js + **shadcn** + **Tailwind CSS**.
- `./docker`: docker-compose deployment files.
## Backend Development ## Repository Layout
We use `uv` to manage dependencies. ```
LangBot/
├── main.py # Entrypoint shim -> langbot.__main__.main()
├── pyproject.toml # Python project + deps (uv), pins langbot-plugin==<x.y.z>
├── src/langbot/
│ ├── __main__.py # Real entrypoint, CLI args (--standalone-runtime, --standalone-box, --debug)
│ ├── pkg/ # Core backend package
│ │ ├── api/ # HTTP API controllers + services (Quart)
│ │ ├── core/ # App bootstrap, stages, task manager
│ │ ├── platform/ # IM platform adapters, bot managers, session managers
│ │ ├── provider/ # LLM providers, requesters, tool providers
│ │ ├── pipeline/ # Pipelines, stages, query pool
│ │ ├── plugin/ # Bridge connecting LangBot to the plugin runtime (see below)
│ │ ├── box/ # Code-sandbox subsystem (Docker / nsjail / E2B backends)
│ │ ├── skill/ # Skill subsystem
│ │ ├── rag/ , vector/ # RAG + vector store
│ │ ├── command/ # Built-in commands
│ │ ├── persistence/ # ORM models + Alembic migrations (SQLite & PostgreSQL)
│ │ ├── storage/ # Object/file storage abstractions
│ │ ├── config/, entity/, discover/, utils/, telemetry/, survey/
│ ├── libs/ # Vendored SDKs (qq_official_api, wecom_api, etc.)
│ └── templates/ # Config/component templates (e.g. templates/config.yaml)
├── web/ # Frontend SPA (Vite + React Router 7 + shadcn + Tailwind)
└── docker/ # docker-compose deployment files
```
## Development Environment Setup
Full guide lives in the wiki: **["开发配置" / Dev Config](https://docs.langbot.app/zh/develop/dev-config)**. Summary:
### Backend
```bash ```bash
pip install uv pip install uv
uv sync --dev uv sync --dev # uv creates a .venv/ for you; point your editor's interpreter at it
uv run main.py # serves API + web UI on http://127.0.0.1:5300
``` ```
Start the backend and run the project in development mode. On first run the config file is generated at `data/config.yaml`. DB is SQLite by default (zero setup); PostgreSQL is supported. Migrations run automatically on startup.
```bash ### Frontend
uv run main.py
```
Then you can access the project at `http://127.0.0.1:5300`. Requires Node.js + [pnpm](https://pnpm.io/installation).
## Frontend Development
We use `pnpm` to manage dependencies.
```bash ```bash
cd web cd web
cp .env.example .env cp .env.example .env # Windows: copy .env.example .env
pnpm install pnpm install
pnpm dev pnpm dev # http://127.0.0.1:3000 (npm install / npm run dev also work)
``` ```
Then you can access the project at `http://127.0.0.1:3000`. `pnpm dev` reads `VITE_API_BASE_URL` from `web/.env` so the dev frontend can reach the backend on port `5300`. In production the frontend is pre-built into static files served by the backend on the same origin.
## Plugin System Architecture ### Code formatting
LangBot is composed of various internal components such as Large Language Model tools, commands, messaging platform adapters, LLM requesters, and more. To meet extensibility and flexibility requirements, we have implemented a production-grade plugin system. The repo runs lint + format checks in CI. Install the pre-commit hooks so the same checks run locally before each commit:
Each plugin runs in an independent process, managed uniformly by the Plugin Runtime. It has two operating modes: `stdio` and `websocket`. When LangBot is started directly by users (not running in a container), it uses `stdio` mode, which is common for personal users or lightweight environments. When LangBot runs in a container, it uses `websocket` mode, designed specifically for production environments. ```bash
uv run pre-commit install
```
Plugin Runtime automatically starts each installed plugin and interacts through stdio. In plugin development scenarios, developers can use the lbp command-line tool to start plugins and connect to the running Runtime via WebSocket for debugging. ## Plugin System
> Plugin SDK, CLI, Runtime, and entities definitions shared between LangBot and plugins are contained in the [`langbot-plugin-sdk`](https://github.com/langbot-app/langbot-plugin-sdk) repository. LangBot's plugin system (Plugin SDK, CLI `lbp`, Plugin Runtime, and the shared entity/API definitions) lives in a **separate repository**: [`langbot-plugin-sdk`](https://github.com/langbot-app/langbot-plugin-sdk). LangBot depends on it via the pinned `langbot-plugin` package in `pyproject.toml`.
## Some Development Tips and Standards ### Architecture (what to know inside this repo)
- LangBot is a global project, any comments in code should be in English, and user experience should be considered in all aspects. - Plugins run as independent processes managed by the **Plugin Runtime**. The Runtime supports two control transports: `stdio` and `websocket`.
- Thus you should consider the i18n support in all aspects. - When LangBot is started directly by a user (not in a container), it spawns and connects to the Runtime over **stdio** (lightweight/personal use).
- LangBot is widely adopted in both toC and toB scenarios, so you should consider the compatibility and security in all aspects. - When LangBot runs in a container, it connects to a standalone Runtime over **WebSocket** (production).
- If you were asked to make a commit, please follow the commit message format: - The bridge code lives in `src/langbot/pkg/plugin/` (`connector.py`, `handler.py`).
- format: <type>(<scope>): <subject> - Relevant config (`data/config.yaml`): `plugin.runtime_ws_url` (e.g. `ws://langbot_plugin_runtime:5400/control/ws`). Start LangBot with `--standalone-runtime` to make it connect to an externally-launched Runtime over WebSocket instead of spawning one over stdio.
- type: must be a specific type, such as feat (new feature), fix (bug fix), docs (documentation), style (code style), refactor (refactoring), perf (performance optimization), etc.
- scope: the scope of the commit, such as the package name, the file name, the function name, the class name, the module name, etc. ### Debugging the Plugin Runtime / CLI / SDK
- subject: the subject of the commit, such as the description of the commit, the reason for the commit, the impact of the commit, etc.
- LangBot uses [Alembic](https://alembic.sqlalchemy.org/) to manage database migrations, supporting both SQLite and PostgreSQL. Migration files are located in `src/langbot/pkg/persistence/alembic/versions/`. If you changed the definition of database entities (ORM models), generate a new migration script by running `uv run python -m langbot.pkg.persistence.alembic_runner autogenerate "description of your change"` in the project root (requires `data/config.yaml` to exist). Review and edit the generated script before committing. Migrations are executed automatically on LangBot startup. For data migrations (e.g. modifying JSON field content), you need to manually add the migration code in the generated script. This is documented in detail in the **SDK repo's `AGENTS.md`** and in the wiki page **["调试插件运行时、CLI、SDK" / Plugin Runtime](https://docs.langbot.app/zh/develop/plugin-runtime)**. The short version:
- Clone `LangBot` and `langbot-plugin-sdk` as siblings under one parent dir so the editor resolves shared entities.
- Start a standalone Runtime from the SDK repo: `uv run --no-sync lbp rt` (control port `5400`, debug port `5401`).
- To make LangBot use a locally-modified SDK: from the SDK dir, with LangBot's `.venv` active, run `uv pip install .`, then launch LangBot with `uv run --no-sync main.py --standalone-runtime` (keep `--no-sync` so your local SDK isn't overwritten).
### Debugging the Box (sandbox) runtime
The Box subsystem (`src/langbot/pkg/box/`) is the code sandbox. It picks the first available backend among **Docker / nsjail / E2B**. The standalone Box runtime is launched via the SDK CLI: `lbp box`. Backend selection details, the `lbp box` flags, and the SDK-side architecture are documented in the SDK repo's `AGENTS.md`.
Relevant config (`data/config.yaml`, `box:` section): `box.enabled` (master switch — disabling it also disables the native sandbox tools, skill add/edit, and stdio-mode MCP servers), `box.backend` (`'local'` = Docker/nsjail auto-pick, or `'docker'` / `'nsjail'` / `'e2b'`; also settable via `BOX__BACKEND`), and `box.runtime.endpoint` (external Box runtime base URL, e.g. `ws://127.0.0.1:5410`; empty = local auto-managed runtime). Like the plugin runtime, LangBot can connect to an externally-launched Box runtime by setting that endpoint and starting with `--standalone-box`.
> A common false "No supported sandbox backend (Docker / nsjail / E2B) is available" comes from Docker being installed and running but the current user not being in the `docker` group → `docker info` gets `permission denied` on the socket. Fix: `sudo usermod -aG docker <user>` and restart the backend in a shell that has the new group.
## Development Standards
- LangBot is a global project: **all code comments and docstrings must be in English**, and every user-facing string must support **i18n** (`en_US` + `zh_Hans` at minimum, plus `ja_JP` where the repo already has it).
- LangBot is adopted in both toC and toB scenarios — always consider compatibility and security.
- **Commit message format**: `<type>(<scope>): <subject>`
- `type`: one of `feat`, `fix`, `docs`, `style`, `refactor`, `perf`, `test`, `chore`, etc.
- `scope`: the affected package/module/file/class.
- `subject`: concise description of the change.
### Database migrations (Alembic)
LangBot uses [Alembic](https://alembic.sqlalchemy.org/) for migrations, supporting both SQLite and PostgreSQL from a single set of scripts. Migration files live in `src/langbot/pkg/persistence/alembic/versions/`.
If you change ORM model definitions, generate a migration:
```bash
# Run from the project root (requires data/config.yaml to exist)
uv run python -m langbot.pkg.persistence.alembic_runner autogenerate "description of your change"
```
Review and edit the generated script before committing. Migrations execute automatically on startup. `autogenerate` detects schema changes (add/drop columns, tables, type changes) but **data migrations** (e.g. mutating JSON field contents) must be hand-written into the generated script. `env.py` sets `render_as_batch=True`, so SQLite's ALTER TABLE limits are handled automatically — no need to branch per database. More in the wiki ["开发配置"](https://docs.langbot.app/zh/develop/dev-config#数据库迁移).
## Some Principles ## Some Principles
- Keep it simple, stupid. - Keep it simple, stupid.
- Entities should not be multiplied unnecessarily - Entities should not be multiplied unnecessarily.
- 八荣八耻 - 八荣八耻
以瞎猜接口为耻,以认真查询为荣。 以瞎猜接口为耻,以认真查询为荣。
@@ -85,4 +138,4 @@ Plugin Runtime automatically starts each installed plugin and interacts through
以跳过验证为耻,以主动测试为荣。 以跳过验证为耻,以主动测试为荣。
以破坏架构为耻,以遵循规范为荣。 以破坏架构为耻,以遵循规范为荣。
以假装理解为耻,以诚实无知为荣。 以假装理解为耻,以诚实无知为荣。
以盲目修改为耻,以谨慎重构为荣。 以盲目修改为耻,以谨慎重构为荣。
+40 -2
View File
@@ -6,6 +6,25 @@ COPY web ./web
RUN cd web && npm install && npx vite build RUN cd web && npm install && npx vite build
# Build nsjail from source so the image ships a self-contained sandbox backend
# that needs no host Docker socket. Pinned to a release tag for reproducibility.
# Multi-stage keeps the compile toolchain (bison/flex/protobuf-dev/libnl-dev)
# out of the final image; only the nsjail binary and its small runtime libs
# (libprotobuf, libnl-route-3) are carried over.
FROM python:3.12.7-slim AS nsjail-build
ARG NSJAIL_VERSION=3.6
RUN apt-get update \
&& apt-get install -y --no-install-recommends \
ca-certificates git build-essential \
autoconf bison flex libtool pkg-config \
protobuf-compiler libprotobuf-dev libnl-route-3-dev \
&& git clone --depth 1 --branch "${NSJAIL_VERSION}" https://github.com/google/nsjail.git /nsjail \
&& make -C /nsjail \
&& install -m 0755 /nsjail/nsjail /usr/local/bin/nsjail \
&& rm -rf /var/lib/apt/lists/*
FROM python:3.12.7-slim FROM python:3.12.7-slim
WORKDIR /app WORKDIR /app
@@ -14,10 +33,29 @@ COPY . .
COPY --from=node /app/web/dist ./web/dist COPY --from=node /app/web/dist ./web/dist
RUN apt update \ # nsjail binary built in the dedicated stage above. Self-contained sandbox
&& apt install gcc -y \ # backend; lets the Box runtime isolate code without a host Docker socket.
COPY --from=nsjail-build /usr/local/bin/nsjail /usr/local/bin/nsjail
RUN apt-get update \
&& apt-get install -y --no-install-recommends gcc ca-certificates curl gnupg \
# nsjail runtime libraries (the build toolchain stays in the nsjail-build
# stage; only these shared libs are needed to execute the binary).
&& apt-get install -y --no-install-recommends libprotobuf32 libnl-route-3-200 \
# Install the Docker CLI (client only) so the optional langbot_box
# service can drive the mounted host Docker socket and create sandbox
# containers. The same image powers langbot / plugin_runtime / box; only
# box uses the client. Arch-aware via dpkg so multi-arch builds work.
&& install -m 0755 -d /etc/apt/keyrings \
&& curl -fsSL https://download.docker.com/linux/debian/gpg -o /etc/apt/keyrings/docker.asc \
&& chmod a+r /etc/apt/keyrings/docker.asc \
&& echo "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/debian $(. /etc/os-release && echo \"$VERSION_CODENAME\") stable" > /etc/apt/sources.list.d/docker.list \
&& apt-get update \
&& apt-get install -y --no-install-recommends docker-ce-cli \
&& python -m pip install --no-cache-dir uv \ && python -m pip install --no-cache-dir uv \
&& uv sync \ && uv sync \
&& apt-get purge -y --auto-remove curl gnupg \
&& rm -rf /var/lib/apt/lists/* \
&& touch /.dockerenv && touch /.dockerenv
CMD [ "uv", "run", "--no-sync", "main.py" ] CMD [ "uv", "run", "--no-sync", "main.py" ]
+2 -2
View File
@@ -38,7 +38,7 @@ LangBot is an **open-source, production-grade platform** for building AI-powered
### Key Capabilities ### Key Capabilities
- **AI Conversations & Agents** — Multi-turn dialogues, tool calling, multi-modal support, streaming output. Built-in RAG (knowledge base) with deep integration to [Dify](https://dify.ai), [Coze](https://coze.com), [n8n](https://n8n.io), [Langflow](https://langflow.org). - **AI Conversations & Agents** — Multi-turn dialogues, tool calling, multi-modal support, streaming output. Built-in RAG (knowledge base) with deep integration to [Dify](https://dify.ai), [Coze](https://coze.com), [n8n](https://n8n.io), [Langflow](https://langflow.org), [Deerflow](https://deerflow.tech), [Weknora](https://weknora.weixin.qq.com).
- **Universal IM Platform Support** — One codebase for Discord, Telegram, Slack, LINE, QQ, WeChat, WeCom, Lark, DingTalk, KOOK. - **Universal IM Platform Support** — One codebase for Discord, Telegram, Slack, LINE, QQ, WeChat, WeCom, Lark, DingTalk, KOOK.
- **Production-Ready** — Access control, rate limiting, sensitive word filtering, comprehensive monitoring, and exception handling. Trusted by enterprises. - **Production-Ready** — Access control, rate limiting, sensitive word filtering, comprehensive monitoring, and exception handling. Trusted by enterprises.
- **Plugin Ecosystem** — Hundreds of plugins, event-driven architecture, component extensions, and [MCP protocol](https://modelcontextprotocol.io/) support. - **Plugin Ecosystem** — Hundreds of plugins, event-driven architecture, component extensions, and [MCP protocol](https://modelcontextprotocol.io/) support.
@@ -78,7 +78,7 @@ docker compose up -d
[![Deploy on Zeabur](https://zeabur.com/button.svg)](https://zeabur.com/en-US/templates/ZKTBDH) [![Deploy on Zeabur](https://zeabur.com/button.svg)](https://zeabur.com/en-US/templates/ZKTBDH)
[![Deploy on Railway](https://railway.com/button.svg)](https://railway.app/template/yRrAyL?referralCode=vogKPF) [![Deploy on Railway](https://railway.com/button.svg)](https://railway.app/template/yRrAyL?referralCode=vogKPF)
**More options:** [Docker](https://link.langbot.app/en/docs/docker) · [Manual](https://link.langbot.app/en/docs/manual-deploy) · [BTPanel](https://link.langbot.app/en/docs/bt-panel) · [Kubernetes](./docker/README_K8S.md) **More options:** [Docker](https://link.langbot.app/en/docs/docker) · [Manual](https://link.langbot.app/en/docs/manual-deploy) · [BTPanel](https://link.langbot.app/en/docs/bt-panel) · [Kubernetes](https://docs.langbot.app/en/deploy/langbot/kubernetes)
--- ---
+4 -4
View File
@@ -13,7 +13,7 @@
[English](README.md) / 简体中文 / [繁體中文](README_TW.md) / [日本語](README_JP.md) / [Español](README_ES.md) / [Français](README_FR.md) / [한국어](README_KO.md) / [Русский](README_RU.md) / [Tiếng Việt](README_VI.md) [English](README.md) / 简体中文 / [繁體中文](README_TW.md) / [日本語](README_JP.md) / [Español](README_ES.md) / [Français](README_FR.md) / [한국어](README_KO.md) / [Русский](README_RU.md) / [Tiếng Việt](README_VI.md)
[![Discord](https://img.shields.io/discord/1335141740050649118?logo=discord&labelColor=%20%235462eb&logoColor=%20%23f5f5f5&color=%20%235462eb)](https://discord.gg/wdNEHETs87) [![Discord](https://img.shields.io/discord/1335141740050649118?logo=discord&labelColor=%20%235462eb&logoColor=%20%23f5f5f5&color=%20%235462eb)](https://discord.gg/wdNEHETs87)
[![QQ Group](https://img.shields.io/badge/%E7%A4%BE%E5%8C%BAQQ%E7%BE%A4-1030838208-blue)](https://qm.qq.com/q/DxZZcNxM1W) [![QQ Group](https://img.shields.io/badge/%E7%A4%BE%E5%8C%BAQQ%E7%BE%A4-1030838208-blue)](https://qm.qq.com/q/IrlV8QFacU)
[![Ask DeepWiki](https://deepwiki.com/badge.svg)](https://deepwiki.com/langbot-app/LangBot) [![Ask DeepWiki](https://deepwiki.com/badge.svg)](https://deepwiki.com/langbot-app/LangBot)
[![GitHub release (latest by date)](https://img.shields.io/github/v/release/langbot-app/LangBot)](https://github.com/langbot-app/LangBot/releases/latest) [![GitHub release (latest by date)](https://img.shields.io/github/v/release/langbot-app/LangBot)](https://github.com/langbot-app/LangBot/releases/latest)
<img src="https://img.shields.io/badge/python-3.10 ~ 3.13 -blue.svg" alt="python"> <img src="https://img.shields.io/badge/python-3.10 ~ 3.13 -blue.svg" alt="python">
@@ -25,7 +25,7 @@
<a href="https://link.langbot.app/zh/docs/guide">文档</a> <a href="https://link.langbot.app/zh/docs/guide">文档</a>
<a href="https://link.langbot.app/zh/docs/api">API</a> <a href="https://link.langbot.app/zh/docs/api">API</a>
<a href="https://space.langbot.app/cloud">Cloud</a> <a href="https://space.langbot.app/cloud">Cloud</a>
<a href="https://space.langbot.app">插件市场</a> <a href="https://space.langbot.app">扩展市场</a>
<a href="https://langbot.featurebase.app/roadmap">路线图</a> <a href="https://langbot.featurebase.app/roadmap">路线图</a>
</div> </div>
@@ -38,7 +38,7 @@ LangBot 是一个**开源的生产级平台**,用于构建 AI 驱动的即时
### 核心能力 ### 核心能力
- **AI 对话与 Agent** — 多轮对话、工具调用、多模态、流式输出。自带 RAG(知识库),深度集成 [Dify](https://dify.ai)、[Coze](https://coze.com)、[n8n](https://n8n.io)、[Langflow](https://langflow.org) 等 LLMOps 平台。 - **AI 对话与 Agent** — 多轮对话、工具调用、多模态、流式输出。自带 RAG(知识库),深度集成 [Dify](https://dify.ai)、[Coze](https://coze.com)、[n8n](https://n8n.io)、[Langflow](https://langflow.org)、[Deerflow](https://deerflow.tech)、[Weknora](https://weknora.weixin.qq.com)等 LLMOps 平台。
- **全平台支持** — 一套代码,覆盖 QQ、微信、企业微信、飞书、钉钉、Discord、Telegram、Slack、LINE、KOOK 等平台。 - **全平台支持** — 一套代码,覆盖 QQ、微信、企业微信、飞书、钉钉、Discord、Telegram、Slack、LINE、KOOK 等平台。
- **生产就绪** — 访问控制、限速、敏感词过滤、全面监控与异常处理,已被多家企业采用。 - **生产就绪** — 访问控制、限速、敏感词过滤、全面监控与异常处理,已被多家企业采用。
- **插件生态** — 数百个插件,跨进程的事件驱动架构,组件扩展,适配 [MCP 协议](https://modelcontextprotocol.io/)。 - **插件生态** — 数百个插件,跨进程的事件驱动架构,组件扩展,适配 [MCP 协议](https://modelcontextprotocol.io/)。
@@ -78,7 +78,7 @@ docker compose up -d
[![Deploy on Zeabur](https://zeabur.com/button.svg)](https://zeabur.com/zh-CN/templates/ZKTBDH) [![Deploy on Zeabur](https://zeabur.com/button.svg)](https://zeabur.com/zh-CN/templates/ZKTBDH)
[![Deploy on Railway](https://railway.com/button.svg)](https://railway.app/template/yRrAyL?referralCode=vogKPF) [![Deploy on Railway](https://railway.com/button.svg)](https://railway.app/template/yRrAyL?referralCode=vogKPF)
**更多方式:** [Docker](https://link.langbot.app/zh/docs/docker) · [手动部署](https://link.langbot.app/zh/docs/manual-deploy) · [宝塔面板](https://link.langbot.app/zh/docs/bt-panel) · [Kubernetes](./docker/README_K8S.md) **更多方式:** [Docker](https://link.langbot.app/zh/docs/docker) · [手动部署](https://link.langbot.app/zh/docs/manual-deploy) · [宝塔面板](https://link.langbot.app/zh/docs/bt-panel) · [Kubernetes](https://docs.langbot.app/zh/deploy/langbot/kubernetes)
--- ---
+2 -2
View File
@@ -37,7 +37,7 @@ LangBot es una **plataforma de código abierto y grado de producción** para con
### Capacidades Clave ### Capacidades Clave
- **Conversaciones e Agentes IA** — Diálogos de múltiples turnos, llamadas a herramientas, soporte multimodal, salida en streaming. RAG (base de conocimientos) incorporado con integración profunda con [Dify](https://dify.ai), [Coze](https://coze.com), [n8n](https://n8n.io), [Langflow](https://langflow.org). - **Conversaciones e Agentes IA** — Diálogos de múltiples turnos, llamadas a herramientas, soporte multimodal, salida en streaming. RAG (base de conocimientos) incorporado con integración profunda con [Dify](https://dify.ai), [Coze](https://coze.com), [n8n](https://n8n.io), [Langflow](https://langflow.org), [Deerflow](https://deerflow.tech)、[Weknora](https://weknora.weixin.qq.com).
- **Soporte Universal de Plataformas de MI** — Un solo código base para Discord, Telegram, Slack, LINE, QQ, WeChat, WeCom, Lark, DingTalk, KOOK. - **Soporte Universal de Plataformas de MI** — Un solo código base para Discord, Telegram, Slack, LINE, QQ, WeChat, WeCom, Lark, DingTalk, KOOK.
- **Listo para Producción** — Control de acceso, limitación de velocidad, filtrado de palabras sensibles, monitoreo completo y manejo de excepciones. De confianza para empresas. - **Listo para Producción** — Control de acceso, limitación de velocidad, filtrado de palabras sensibles, monitoreo completo y manejo de excepciones. De confianza para empresas.
- **Ecosistema de Plugins** — Cientos de plugins, arquitectura basada en eventos, extensiones de componentes y soporte del [protocolo MCP](https://modelcontextprotocol.io/). - **Ecosistema de Plugins** — Cientos de plugins, arquitectura basada en eventos, extensiones de componentes y soporte del [protocolo MCP](https://modelcontextprotocol.io/).
@@ -77,7 +77,7 @@ docker compose up -d
[![Deploy on Zeabur](https://zeabur.com/button.svg)](https://zeabur.com/en-US/templates/ZKTBDH) [![Deploy on Zeabur](https://zeabur.com/button.svg)](https://zeabur.com/en-US/templates/ZKTBDH)
[![Deploy on Railway](https://railway.com/button.svg)](https://railway.app/template/yRrAyL?referralCode=vogKPF) [![Deploy on Railway](https://railway.com/button.svg)](https://railway.app/template/yRrAyL?referralCode=vogKPF)
**Más opciones:** [Docker](https://link.langbot.app/en/docs/docker) · [Manual](https://link.langbot.app/en/docs/manual-deploy) · [BTPanel](https://link.langbot.app/en/docs/bt-panel) · [Kubernetes](./docker/README_K8S.md) **Más opciones:** [Docker](https://link.langbot.app/en/docs/docker) · [Manual](https://link.langbot.app/en/docs/manual-deploy) · [BTPanel](https://link.langbot.app/en/docs/bt-panel) · [Kubernetes](https://docs.langbot.app/en/deploy/langbot/kubernetes)
--- ---
+2 -2
View File
@@ -37,7 +37,7 @@ LangBot est une **plateforme open-source de niveau production** pour créer des
### Capacités Clés ### Capacités Clés
- **Conversations IA & Agents** — Dialogues multi-tours, appels d'outils, support multimodal, sortie en streaming. RAG (base de connaissances) intégré avec intégration profonde de [Dify](https://dify.ai), [Coze](https://coze.com), [n8n](https://n8n.io), [Langflow](https://langflow.org). - **Conversations IA & Agents** — Dialogues multi-tours, appels d'outils, support multimodal, sortie en streaming. RAG (base de connaissances) intégré avec intégration profonde de [Dify](https://dify.ai), [Coze](https://coze.com), [n8n](https://n8n.io), [Langflow](https://langflow.org), [Deerflow](https://deerflow.tech), [Weknora](https://weknora.weixin.qq.com).
- **Support Universel des Plateformes de MI** — Un seul code pour Discord, Telegram, Slack, LINE, QQ, WeChat, WeCom, Lark, DingTalk, KOOK. - **Support Universel des Plateformes de MI** — Un seul code pour Discord, Telegram, Slack, LINE, QQ, WeChat, WeCom, Lark, DingTalk, KOOK.
- **Prêt pour la Production** — Contrôle d'accès, limitation de débit, filtrage de mots sensibles, surveillance complète et gestion des exceptions. Approuvé par les entreprises. - **Prêt pour la Production** — Contrôle d'accès, limitation de débit, filtrage de mots sensibles, surveillance complète et gestion des exceptions. Approuvé par les entreprises.
- **Écosystème de Plugins** — Des centaines de plugins, architecture événementielle, extensions de composants, et support du [protocole MCP](https://modelcontextprotocol.io/). - **Écosystème de Plugins** — Des centaines de plugins, architecture événementielle, extensions de composants, et support du [protocole MCP](https://modelcontextprotocol.io/).
@@ -77,7 +77,7 @@ docker compose up -d
[![Deploy on Zeabur](https://zeabur.com/button.svg)](https://zeabur.com/en-US/templates/ZKTBDH) [![Deploy on Zeabur](https://zeabur.com/button.svg)](https://zeabur.com/en-US/templates/ZKTBDH)
[![Deploy on Railway](https://railway.com/button.svg)](https://railway.app/template/yRrAyL?referralCode=vogKPF) [![Deploy on Railway](https://railway.com/button.svg)](https://railway.app/template/yRrAyL?referralCode=vogKPF)
**Plus d'options :** [Docker](https://link.langbot.app/en/docs/docker) · [Manuel](https://link.langbot.app/en/docs/manual-deploy) · [BTPanel](https://link.langbot.app/en/docs/bt-panel) · [Kubernetes](./docker/README_K8S.md) **Plus d'options :** [Docker](https://link.langbot.app/en/docs/docker) · [Manuel](https://link.langbot.app/en/docs/manual-deploy) · [BTPanel](https://link.langbot.app/en/docs/bt-panel) · [Kubernetes](https://docs.langbot.app/en/deploy/langbot/kubernetes)
--- ---
+2 -2
View File
@@ -37,7 +37,7 @@ LangBot は、AI搭載のインスタントメッセージングボットを構
### 主な機能 ### 主な機能
- **AI対話とエージェント** — マルチターン対話、ツール呼び出し、マルチモーダル対応、ストリーミング出力。RAG(ナレッジベース)を内蔵し、[Dify](https://dify.ai)、[Coze](https://coze.com)、[n8n](https://n8n.io)、[Langflow](https://langflow.org) と深く統合。 - **AI対話とエージェント** — マルチターン対話、ツール呼び出し、マルチモーダル対応、ストリーミング出力。RAG(ナレッジベース)を内蔵し、[Dify](https://dify.ai)、[Coze](https://coze.com)、[n8n](https://n8n.io)、[Langflow](https://langflow.org)、[Deerflow](https://deerflow.tech)、[Weknora](https://weknora.weixin.qq.com) と深く統合。
- **ユニバーサルIMプラットフォーム対応** — 単一のコードベースで Discord、Telegram、Slack、LINE、QQ、WeChat、WeCom、Lark、DingTalk、KOOK に対応。 - **ユニバーサルIMプラットフォーム対応** — 単一のコードベースで Discord、Telegram、Slack、LINE、QQ、WeChat、WeCom、Lark、DingTalk、KOOK に対応。
- **本番環境対応** — アクセス制御、レート制限、センシティブワードフィルタリング、包括的な監視、例外処理を搭載。エンタープライズの信頼に応える品質。 - **本番環境対応** — アクセス制御、レート制限、センシティブワードフィルタリング、包括的な監視、例外処理を搭載。エンタープライズの信頼に応える品質。
- **プラグインエコシステム** — 数百のプラグイン、イベント駆動アーキテクチャ、コンポーネント拡張、[MCPプロトコル](https://modelcontextprotocol.io/)対応。 - **プラグインエコシステム** — 数百のプラグイン、イベント駆動アーキテクチャ、コンポーネント拡張、[MCPプロトコル](https://modelcontextprotocol.io/)対応。
@@ -77,7 +77,7 @@ docker compose up -d
[![Deploy on Zeabur](https://zeabur.com/button.svg)](https://zeabur.com/en-US/templates/ZKTBDH) [![Deploy on Zeabur](https://zeabur.com/button.svg)](https://zeabur.com/en-US/templates/ZKTBDH)
[![Deploy on Railway](https://railway.com/button.svg)](https://railway.app/template/yRrAyL?referralCode=vogKPF) [![Deploy on Railway](https://railway.com/button.svg)](https://railway.app/template/yRrAyL?referralCode=vogKPF)
**その他:** [Docker](https://link.langbot.app/en/docs/docker) · [手動デプロイ](https://link.langbot.app/en/docs/manual-deploy) · [BTPanel](https://link.langbot.app/en/docs/bt-panel) · [Kubernetes](./docker/README_K8S.md) **その他:** [Docker](https://link.langbot.app/en/docs/docker) · [手動デプロイ](https://link.langbot.app/en/docs/manual-deploy) · [BTPanel](https://link.langbot.app/en/docs/bt-panel) · [Kubernetes](https://docs.langbot.app/en/deploy/langbot/kubernetes)
--- ---
+2 -2
View File
@@ -37,7 +37,7 @@ LangBot은 AI 기반 인스턴트 메시징 봇을 구축하기 위한 **오픈
### 핵심 기능 ### 핵심 기능
- **AI 대화 및 에이전트** — 멀티턴 대화, 도구 호출, 멀티모달 지원, 스트리밍 출력. 내장 RAG(지식 베이스)와 [Dify](https://dify.ai), [Coze](https://coze.com), [n8n](https://n8n.io), [Langflow](https://langflow.org) 심층 통합. - **AI 대화 및 에이전트** — 멀티턴 대화, 도구 호출, 멀티모달 지원, 스트리밍 출력. 내장 RAG(지식 베이스)와 [Dify](https://dify.ai), [Coze](https://coze.com), [n8n](https://n8n.io), [Langflow](https://langflow.org), [Deerflow](https://deerflow.tech), [Weknora](https://weknora.weixin.qq.com) 심층 통합.
- **유니버설 IM 플랫폼 지원** — 단일 코드베이스로 Discord, Telegram, Slack, LINE, QQ, WeChat, WeCom, Lark, DingTalk, KOOK 지원. - **유니버설 IM 플랫폼 지원** — 단일 코드베이스로 Discord, Telegram, Slack, LINE, QQ, WeChat, WeCom, Lark, DingTalk, KOOK 지원.
- **프로덕션 레디** — 접근 제어, 속도 제한, 민감어 필터링, 종합 모니터링 및 예외 처리. 기업 환경에서 검증됨. - **프로덕션 레디** — 접근 제어, 속도 제한, 민감어 필터링, 종합 모니터링 및 예외 처리. 기업 환경에서 검증됨.
- **플러그인 생태계** — 수백 개의 플러그인, 이벤트 기반 아키텍처, 컴포넌트 확장, [MCP 프로토콜](https://modelcontextprotocol.io/) 지원. - **플러그인 생태계** — 수백 개의 플러그인, 이벤트 기반 아키텍처, 컴포넌트 확장, [MCP 프로토콜](https://modelcontextprotocol.io/) 지원.
@@ -77,7 +77,7 @@ docker compose up -d
[![Deploy on Zeabur](https://zeabur.com/button.svg)](https://zeabur.com/en-US/templates/ZKTBDH) [![Deploy on Zeabur](https://zeabur.com/button.svg)](https://zeabur.com/en-US/templates/ZKTBDH)
[![Deploy on Railway](https://railway.com/button.svg)](https://railway.app/template/yRrAyL?referralCode=vogKPF) [![Deploy on Railway](https://railway.com/button.svg)](https://railway.app/template/yRrAyL?referralCode=vogKPF)
**더 많은 옵션:** [Docker](https://link.langbot.app/en/docs/docker) · [수동 배포](https://link.langbot.app/en/docs/manual-deploy) · [BTPanel](https://link.langbot.app/en/docs/bt-panel) · [Kubernetes](./docker/README_K8S.md) **더 많은 옵션:** [Docker](https://link.langbot.app/en/docs/docker) · [수동 배포](https://link.langbot.app/en/docs/manual-deploy) · [BTPanel](https://link.langbot.app/en/docs/bt-panel) · [Kubernetes](https://docs.langbot.app/en/deploy/langbot/kubernetes)
--- ---
+2 -2
View File
@@ -37,7 +37,7 @@ LangBot — это **платформа с открытым исходным к
### Ключевые возможности ### Ключевые возможности
- **ИИ-диалоги и агенты** — Многораундовые диалоги, вызов инструментов, мультимодальная поддержка, потоковый вывод. Встроенная реализация RAG (база знаний) с глубокой интеграцией в [Dify](https://dify.ai), [Coze](https://coze.com), [n8n](https://n8n.io), [Langflow](https://langflow.org). - **ИИ-диалоги и агенты** — Многораундовые диалоги, вызов инструментов, мультимодальная поддержка, потоковый вывод. Встроенная реализация RAG (база знаний) с глубокой интеграцией в [Dify](https://dify.ai), [Coze](https://coze.com), [n8n](https://n8n.io), [Langflow](https://langflow.org), [Deerflow](https://deerflow.tech), [Weknora](https://weknora.weixin.qq.com).
- **Универсальная поддержка IM-платформ** — Единая кодовая база для Discord, Telegram, Slack, LINE, QQ, WeChat, WeCom, Lark, DingTalk, KOOK. - **Универсальная поддержка IM-платформ** — Единая кодовая база для Discord, Telegram, Slack, LINE, QQ, WeChat, WeCom, Lark, DingTalk, KOOK.
- **Готовность к продакшену** — Контроль доступа, ограничение скорости, фильтрация чувствительных слов, комплексный мониторинг и обработка исключений. Проверено в корпоративной среде. - **Готовность к продакшену** — Контроль доступа, ограничение скорости, фильтрация чувствительных слов, комплексный мониторинг и обработка исключений. Проверено в корпоративной среде.
- **Экосистема плагинов** — Сотни плагинов, событийно-ориентированная архитектура, расширения компонентов и поддержка [протокола MCP](https://modelcontextprotocol.io/). - **Экосистема плагинов** — Сотни плагинов, событийно-ориентированная архитектура, расширения компонентов и поддержка [протокола MCP](https://modelcontextprotocol.io/).
@@ -77,7 +77,7 @@ docker compose up -d
[![Deploy on Zeabur](https://zeabur.com/button.svg)](https://zeabur.com/en-US/templates/ZKTBDH) [![Deploy on Zeabur](https://zeabur.com/button.svg)](https://zeabur.com/en-US/templates/ZKTBDH)
[![Deploy on Railway](https://railway.com/button.svg)](https://railway.app/template/yRrAyL?referralCode=vogKPF) [![Deploy on Railway](https://railway.com/button.svg)](https://railway.app/template/yRrAyL?referralCode=vogKPF)
**Другие варианты:** [Docker](https://link.langbot.app/en/docs/docker) · [Ручная установка](https://link.langbot.app/en/docs/manual-deploy) · [BTPanel](https://link.langbot.app/en/docs/bt-panel) · [Kubernetes](./docker/README_K8S.md) **Другие варианты:** [Docker](https://link.langbot.app/en/docs/docker) · [Ручная установка](https://link.langbot.app/en/docs/manual-deploy) · [BTPanel](https://link.langbot.app/en/docs/bt-panel) · [Kubernetes](https://docs.langbot.app/en/deploy/langbot/kubernetes)
--- ---
+2 -2
View File
@@ -39,7 +39,7 @@ LangBot 是一個**開源的生產級平台**,用於建構 AI 驅動的即時
### 核心能力 ### 核心能力
- **AI 對話與 Agent** — 多輪對話、工具調用、多模態、流式輸出。自帶 RAG(知識庫),深度整合 [Dify](https://dify.ai)、[Coze](https://coze.com)、[n8n](https://n8n.io)、[Langflow](https://langflow.org) 等 LLMOps 平台。 - **AI 對話與 Agent** — 多輪對話、工具調用、多模態、流式輸出。自帶 RAG(知識庫),深度整合 [Dify](https://dify.ai)、[Coze](https://coze.com)、[n8n](https://n8n.io)、[Langflow](https://langflow.org)、 [Deerflow](https://deerflow.tech)、[Weknora](https://weknora.weixin.qq.com)等 LLMOps 平台。
- **全平台支援** — 一套程式碼,覆蓋 QQ、微信、企業微信、飛書、釘釘、Discord、Telegram、Slack、LINE、KOOK 等平台。 - **全平台支援** — 一套程式碼,覆蓋 QQ、微信、企業微信、飛書、釘釘、Discord、Telegram、Slack、LINE、KOOK 等平台。
- **生產就緒** — 存取控制、限速、敏感詞過濾、全面監控與異常處理,已被多家企業採用。 - **生產就緒** — 存取控制、限速、敏感詞過濾、全面監控與異常處理,已被多家企業採用。
- **外掛生態** — 數百個外掛,事件驅動架構,組件擴展,適配 [MCP 協議](https://modelcontextprotocol.io/)。 - **外掛生態** — 數百個外掛,事件驅動架構,組件擴展,適配 [MCP 協議](https://modelcontextprotocol.io/)。
@@ -79,7 +79,7 @@ docker compose up -d
[![Deploy on Zeabur](https://zeabur.com/button.svg)](https://zeabur.com/zh-CN/templates/ZKTBDH) [![Deploy on Zeabur](https://zeabur.com/button.svg)](https://zeabur.com/zh-CN/templates/ZKTBDH)
[![Deploy on Railway](https://railway.com/button.svg)](https://railway.app/template/yRrAyL?referralCode=vogKPF) [![Deploy on Railway](https://railway.com/button.svg)](https://railway.app/template/yRrAyL?referralCode=vogKPF)
**更多方式:** [Docker](https://link.langbot.app/zh/docs/docker) · [手動部署](https://link.langbot.app/zh/docs/manual-deploy) · [寶塔面板](https://link.langbot.app/zh/docs/bt-panel) · [Kubernetes](./docker/README_K8S.md) **更多方式:** [Docker](https://link.langbot.app/zh/docs/docker) · [手動部署](https://link.langbot.app/zh/docs/manual-deploy) · [寶塔面板](https://link.langbot.app/zh/docs/bt-panel) · [Kubernetes](https://docs.langbot.app/zh/deploy/langbot/kubernetes)
--- ---
+2 -2
View File
@@ -37,7 +37,7 @@ LangBot là một **nền tảng mã nguồn mở, cấp sản xuất** để x
### Khả năng chính ### Khả năng chính
- **Hội thoại AI & Agent** — Đối thoại nhiều lượt, gọi công cụ, hỗ trợ đa phương thức, đầu ra streaming. RAG (cơ sở kiến thức) tích hợp sẵn với tích hợp sâu vào [Dify](https://dify.ai), [Coze](https://coze.com), [n8n](https://n8n.io), [Langflow](https://langflow.org). - **Hội thoại AI & Agent** — Đối thoại nhiều lượt, gọi công cụ, hỗ trợ đa phương thức, đầu ra streaming. RAG (cơ sở kiến thức) tích hợp sẵn với tích hợp sâu vào [Dify](https://dify.ai), [Coze](https://coze.com), [n8n](https://n8n.io), [Langflow](https://langflow.org), [Deerflow](https://deerflow.tech), [Weknora](https://weknora.weixin.qq.com).
- **Hỗ trợ đa nền tảng IM** — Một mã nguồn cho Discord, Telegram, Slack, LINE, QQ, WeChat, WeCom, Lark, DingTalk, KOOK. - **Hỗ trợ đa nền tảng IM** — Một mã nguồn cho Discord, Telegram, Slack, LINE, QQ, WeChat, WeCom, Lark, DingTalk, KOOK.
- **Sẵn sàng cho sản xuất** — Kiểm soát truy cập, giới hạn tốc độ, lọc từ nhạy cảm, giám sát toàn diện và xử lý ngoại lệ. Được doanh nghiệp tin dùng. - **Sẵn sàng cho sản xuất** — Kiểm soát truy cập, giới hạn tốc độ, lọc từ nhạy cảm, giám sát toàn diện và xử lý ngoại lệ. Được doanh nghiệp tin dùng.
- **Hệ sinh thái Plugin** — Hàng trăm plugin, kiến trúc hướng sự kiện, mở rộng thành phần, và hỗ trợ [giao thức MCP](https://modelcontextprotocol.io/). - **Hệ sinh thái Plugin** — Hàng trăm plugin, kiến trúc hướng sự kiện, mở rộng thành phần, và hỗ trợ [giao thức MCP](https://modelcontextprotocol.io/).
@@ -77,7 +77,7 @@ docker compose up -d
[![Deploy on Zeabur](https://zeabur.com/button.svg)](https://zeabur.com/en-US/templates/ZKTBDH) [![Deploy on Zeabur](https://zeabur.com/button.svg)](https://zeabur.com/en-US/templates/ZKTBDH)
[![Deploy on Railway](https://railway.com/button.svg)](https://railway.app/template/yRrAyL?referralCode=vogKPF) [![Deploy on Railway](https://railway.com/button.svg)](https://railway.app/template/yRrAyL?referralCode=vogKPF)
**Thêm tùy chọn:** [Docker](https://link.langbot.app/en/docs/docker) · [Thủ công](https://link.langbot.app/en/docs/manual-deploy) · [BTPanel](https://link.langbot.app/en/docs/bt-panel) · [Kubernetes](./docker/README_K8S.md) **Thêm tùy chọn:** [Docker](https://link.langbot.app/en/docs/docker) · [Thủ công](https://link.langbot.app/en/docs/manual-deploy) · [BTPanel](https://link.langbot.app/en/docs/bt-panel) · [Kubernetes](https://docs.langbot.app/en/deploy/langbot/kubernetes)
--- ---
-629
View File
@@ -1,629 +0,0 @@
# LangBot Kubernetes 部署指南 / Kubernetes Deployment Guide
[简体中文](#简体中文) | [English](#english)
---
## 简体中文
### 概述
本指南提供了在 Kubernetes 集群中部署 LangBot 的完整步骤。Kubernetes 部署配置基于 `docker-compose.yaml`,适用于生产环境的容器化部署。
### 前置要求
- Kubernetes 集群(版本 1.19+
- `kubectl` 命令行工具已配置并可访问集群
- 集群中有可用的存储类(StorageClass)用于持久化存储(可选但推荐)
- 至少 2 vCPU 和 4GB RAM 的可用资源
### 架构说明
Kubernetes 部署包含以下组件:
1. **langbot**: 主应用服务
- 提供 Web UI(端口 5300
- 处理平台 webhook(端口 2280-2290
- 数据持久化卷
2. **langbot-plugin-runtime**: 插件运行时服务
- WebSocket 通信(端口 5400
- 插件数据持久化卷
3. **持久化存储**:
- `langbot-data`: LangBot 主数据
- `langbot-plugins`: 插件文件
- `langbot-plugin-runtime-data`: 插件运行时数据
### 快速开始
#### 1. 下载部署文件
```bash
# 克隆仓库
git clone https://github.com/langbot-app/LangBot
cd LangBot/docker
# 或直接下载 kubernetes.yaml
wget https://raw.githubusercontent.com/langbot-app/LangBot/main/docker/kubernetes.yaml
```
#### 2. 部署到 Kubernetes
```bash
# 应用所有配置
kubectl apply -f kubernetes.yaml
# 检查部署状态
kubectl get all -n langbot
# 查看 Pod 日志
kubectl logs -n langbot -l app=langbot -f
```
#### 3. 访问 LangBot
默认情况下,LangBot 服务使用 ClusterIP 类型,只能在集群内部访问。您可以选择以下方式之一来访问:
**选项 A: 端口转发(推荐用于测试)**
```bash
kubectl port-forward -n langbot svc/langbot 5300:5300
```
然后访问 http://localhost:5300
**选项 B: NodePort(适用于开发环境)**
编辑 `kubernetes.yaml`,取消注释 NodePort Service 部分,然后:
```bash
kubectl apply -f kubernetes.yaml
# 获取节点 IP
kubectl get nodes -o wide
# 访问 http://<NODE_IP>:30300
```
**选项 C: LoadBalancer(适用于云环境)**
编辑 `kubernetes.yaml`,取消注释 LoadBalancer Service 部分,然后:
```bash
kubectl apply -f kubernetes.yaml
# 获取外部 IP
kubectl get svc -n langbot langbot-loadbalancer
# 访问 http://<EXTERNAL_IP>
```
**选项 D: Ingress(推荐用于生产环境)**
确保集群中已安装 Ingress Controller(如 nginx-ingress),然后:
1. 编辑 `kubernetes.yaml` 中的 Ingress 配置
2. 修改域名为您的实际域名
3. 应用配置:
```bash
kubectl apply -f kubernetes.yaml
# 访问 http://langbot.yourdomain.com
```
### 配置说明
#### 环境变量
`ConfigMap` 中配置环境变量:
```yaml
apiVersion: v1
kind: ConfigMap
metadata:
name: langbot-config
namespace: langbot
data:
TZ: "Asia/Shanghai" # 修改为您的时区
```
#### 存储配置
默认使用动态存储分配。如果您有特定的 StorageClass,请在 PVC 中指定:
```yaml
spec:
storageClassName: your-storage-class-name
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 10Gi
```
#### 资源限制
根据您的需求调整资源限制:
```yaml
resources:
requests:
memory: "1Gi"
cpu: "500m"
limits:
memory: "4Gi"
cpu: "2000m"
```
### 常用操作
#### 查看日志
```bash
# 查看 LangBot 主服务日志
kubectl logs -n langbot -l app=langbot -f
# 查看插件运行时日志
kubectl logs -n langbot -l app=langbot-plugin-runtime -f
```
#### 重启服务
```bash
# 重启 LangBot
kubectl rollout restart deployment/langbot -n langbot
# 重启插件运行时
kubectl rollout restart deployment/langbot-plugin-runtime -n langbot
```
#### 更新镜像
```bash
# 更新到最新版本
kubectl set image deployment/langbot -n langbot langbot=rockchin/langbot:latest
kubectl set image deployment/langbot-plugin-runtime -n langbot langbot-plugin-runtime=rockchin/langbot:latest
# 检查更新状态
kubectl rollout status deployment/langbot -n langbot
```
#### 扩容(不推荐)
注意:由于 LangBot 使用 ReadWriteOnce 的持久化存储,不支持多副本扩容。如需高可用,请考虑使用 ReadWriteMany 存储或其他架构方案。
#### 备份数据
```bash
# 备份 PVC 数据
kubectl exec -n langbot -it <langbot-pod-name> -- tar czf /tmp/backup.tar.gz /app/data
kubectl cp langbot/<langbot-pod-name>:/tmp/backup.tar.gz ./backup.tar.gz
```
### 卸载
```bash
# 删除所有资源(保留 PVC
kubectl delete deployment,service,configmap -n langbot --all
# 删除 PVC(会删除数据)
kubectl delete pvc -n langbot --all
# 删除命名空间
kubectl delete namespace langbot
```
### 故障排查
#### Pod 无法启动
```bash
# 查看 Pod 状态
kubectl get pods -n langbot
# 查看详细信息
kubectl describe pod -n langbot <pod-name>
# 查看事件
kubectl get events -n langbot --sort-by='.lastTimestamp'
```
#### 存储问题
```bash
# 检查 PVC 状态
kubectl get pvc -n langbot
# 检查 PV
kubectl get pv
```
#### 网络访问问题
```bash
# 检查 Service
kubectl get svc -n langbot
# 检查端口转发
kubectl port-forward -n langbot svc/langbot 5300:5300
```
### 生产环境建议
1. **使用特定版本标签**:避免使用 `latest` 标签,使用具体版本号如 `rockchin/langbot:v1.0.0`
2. **配置资源限制**:根据实际负载调整 CPU 和内存限制
3. **使用 Ingress + TLS**:配置 HTTPS 访问和证书管理
4. **配置监控和告警**:集成 Prometheus、Grafana 等监控工具
5. **定期备份**:配置自动备份策略保护数据
6. **使用专用 StorageClass**:为生产环境配置高性能存储
7. **配置亲和性规则**:确保 Pod 调度到合适的节点
### 高级配置
#### 使用 Secrets 管理敏感信息
如果需要配置 API 密钥等敏感信息:
```yaml
apiVersion: v1
kind: Secret
metadata:
name: langbot-secrets
namespace: langbot
type: Opaque
data:
api_key: <base64-encoded-value>
```
然后在 Deployment 中引用:
```yaml
env:
- name: API_KEY
valueFrom:
secretKeyRef:
name: langbot-secrets
key: api_key
```
#### 配置水平自动扩缩容(HPA)
注意:需要确保使用 ReadWriteMany 存储类型
```yaml
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
name: langbot-hpa
namespace: langbot
spec:
scaleTargetRef:
apiVersion: apps/v1
kind: Deployment
name: langbot
minReplicas: 1
maxReplicas: 3
metrics:
- type: Resource
resource:
name: cpu
target:
type: Utilization
averageUtilization: 70
```
### 参考资源
- [LangBot 官方文档](https://docs.langbot.app)
- [Docker 部署文档](https://link.langbot.app/zh/docs/docker)
- [Kubernetes 官方文档](https://kubernetes.io/docs/)
---
## English
### Overview
This guide provides complete steps for deploying LangBot in a Kubernetes cluster. The Kubernetes deployment configuration is based on `docker-compose.yaml` and is suitable for production containerized deployments.
### Prerequisites
- Kubernetes cluster (version 1.19+)
- `kubectl` command-line tool configured with cluster access
- Available StorageClass in the cluster for persistent storage (optional but recommended)
- At least 2 vCPU and 4GB RAM of available resources
### Architecture
The Kubernetes deployment includes the following components:
1. **langbot**: Main application service
- Provides Web UI (port 5300)
- Handles platform webhooks (ports 2280-2290)
- Data persistence volume
2. **langbot-plugin-runtime**: Plugin runtime service
- WebSocket communication (port 5400)
- Plugin data persistence volume
3. **Persistent Storage**:
- `langbot-data`: LangBot main data
- `langbot-plugins`: Plugin files
- `langbot-plugin-runtime-data`: Plugin runtime data
### Quick Start
#### 1. Download Deployment Files
```bash
# Clone repository
git clone https://github.com/langbot-app/LangBot
cd LangBot/docker
# Or download kubernetes.yaml directly
wget https://raw.githubusercontent.com/langbot-app/LangBot/main/docker/kubernetes.yaml
```
#### 2. Deploy to Kubernetes
```bash
# Apply all configurations
kubectl apply -f kubernetes.yaml
# Check deployment status
kubectl get all -n langbot
# View Pod logs
kubectl logs -n langbot -l app=langbot -f
```
#### 3. Access LangBot
By default, LangBot service uses ClusterIP type, accessible only within the cluster. Choose one of the following methods to access:
**Option A: Port Forwarding (Recommended for testing)**
```bash
kubectl port-forward -n langbot svc/langbot 5300:5300
```
Then visit http://localhost:5300
**Option B: NodePort (Suitable for development)**
Edit `kubernetes.yaml`, uncomment the NodePort Service section, then:
```bash
kubectl apply -f kubernetes.yaml
# Get node IP
kubectl get nodes -o wide
# Visit http://<NODE_IP>:30300
```
**Option C: LoadBalancer (Suitable for cloud environments)**
Edit `kubernetes.yaml`, uncomment the LoadBalancer Service section, then:
```bash
kubectl apply -f kubernetes.yaml
# Get external IP
kubectl get svc -n langbot langbot-loadbalancer
# Visit http://<EXTERNAL_IP>
```
**Option D: Ingress (Recommended for production)**
Ensure an Ingress Controller (e.g., nginx-ingress) is installed in the cluster, then:
1. Edit the Ingress configuration in `kubernetes.yaml`
2. Change the domain to your actual domain
3. Apply configuration:
```bash
kubectl apply -f kubernetes.yaml
# Visit http://langbot.yourdomain.com
```
### Configuration
#### Environment Variables
Configure environment variables in ConfigMap:
```yaml
apiVersion: v1
kind: ConfigMap
metadata:
name: langbot-config
namespace: langbot
data:
TZ: "Asia/Shanghai" # Change to your timezone
```
#### Storage Configuration
Uses dynamic storage provisioning by default. If you have a specific StorageClass, specify it in PVC:
```yaml
spec:
storageClassName: your-storage-class-name
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 10Gi
```
#### Resource Limits
Adjust resource limits based on your needs:
```yaml
resources:
requests:
memory: "1Gi"
cpu: "500m"
limits:
memory: "4Gi"
cpu: "2000m"
```
### Common Operations
#### View Logs
```bash
# View LangBot main service logs
kubectl logs -n langbot -l app=langbot -f
# View plugin runtime logs
kubectl logs -n langbot -l app=langbot-plugin-runtime -f
```
#### Restart Services
```bash
# Restart LangBot
kubectl rollout restart deployment/langbot -n langbot
# Restart plugin runtime
kubectl rollout restart deployment/langbot-plugin-runtime -n langbot
```
#### Update Images
```bash
# Update to latest version
kubectl set image deployment/langbot -n langbot langbot=rockchin/langbot:latest
kubectl set image deployment/langbot-plugin-runtime -n langbot langbot-plugin-runtime=rockchin/langbot:latest
# Check update status
kubectl rollout status deployment/langbot -n langbot
```
#### Scaling (Not Recommended)
Note: Due to LangBot using ReadWriteOnce persistent storage, multi-replica scaling is not supported. For high availability, consider using ReadWriteMany storage or alternative architectures.
#### Backup Data
```bash
# Backup PVC data
kubectl exec -n langbot -it <langbot-pod-name> -- tar czf /tmp/backup.tar.gz /app/data
kubectl cp langbot/<langbot-pod-name>:/tmp/backup.tar.gz ./backup.tar.gz
```
### Uninstall
```bash
# Delete all resources (keep PVCs)
kubectl delete deployment,service,configmap -n langbot --all
# Delete PVCs (will delete data)
kubectl delete pvc -n langbot --all
# Delete namespace
kubectl delete namespace langbot
```
### Troubleshooting
#### Pods Not Starting
```bash
# Check Pod status
kubectl get pods -n langbot
# View detailed information
kubectl describe pod -n langbot <pod-name>
# View events
kubectl get events -n langbot --sort-by='.lastTimestamp'
```
#### Storage Issues
```bash
# Check PVC status
kubectl get pvc -n langbot
# Check PV
kubectl get pv
```
#### Network Access Issues
```bash
# Check Service
kubectl get svc -n langbot
# Test port forwarding
kubectl port-forward -n langbot svc/langbot 5300:5300
```
### Production Recommendations
1. **Use specific version tags**: Avoid using `latest` tag, use specific version like `rockchin/langbot:v1.0.0`
2. **Configure resource limits**: Adjust CPU and memory limits based on actual load
3. **Use Ingress + TLS**: Configure HTTPS access and certificate management
4. **Configure monitoring and alerts**: Integrate monitoring tools like Prometheus, Grafana
5. **Regular backups**: Configure automated backup strategy to protect data
6. **Use dedicated StorageClass**: Configure high-performance storage for production
7. **Configure affinity rules**: Ensure Pods are scheduled to appropriate nodes
### Advanced Configuration
#### Using Secrets for Sensitive Information
If you need to configure sensitive information like API keys:
```yaml
apiVersion: v1
kind: Secret
metadata:
name: langbot-secrets
namespace: langbot
type: Opaque
data:
api_key: <base64-encoded-value>
```
Then reference in Deployment:
```yaml
env:
- name: API_KEY
valueFrom:
secretKeyRef:
name: langbot-secrets
key: api_key
```
#### Configure Horizontal Pod Autoscaling (HPA)
Note: Requires ReadWriteMany storage type
```yaml
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
name: langbot-hpa
namespace: langbot
spec:
scaleTargetRef:
apiVersion: apps/v1
kind: Deployment
name: langbot
minReplicas: 1
maxReplicas: 3
metrics:
- type: Resource
resource:
name: cpu
target:
type: Utilization
averageUtilization: 70
```
### References
- [LangBot Official Documentation](https://docs.langbot.app)
- [Docker Deployment Guide](https://link.langbot.app/zh/docs/docker)
- [Kubernetes Official Documentation](https://kubernetes.io/docs/)
+43 -2
View File
@@ -1,5 +1,5 @@
# Docker Compose configuration for LangBot # Docker Compose configuration for LangBot
# For Kubernetes deployment, see kubernetes.yaml and README_K8S.md # For Kubernetes deployment, see kubernetes.yaml and the deployment guide at https://docs.langbot.app
version: "3" version: "3"
services: services:
@@ -18,6 +18,40 @@ services:
networks: networks:
- langbot_network - langbot_network
# The Box sandbox runtime is optional. It is only started when you run
# ``docker compose --profile box up`` (or ``docker compose --profile all
# up``). With Box off, LangBot keeps the dashboard / skills list visible
# (read-only) but disables sandbox tools, skill add/edit and stdio MCP —
# set ``box.enabled: false`` in ``data/config.yaml`` (or
# ``BOX__ENABLED=false`` in the langbot service env below) to match.
langbot_box:
image: rockchin/langbot:latest
container_name: langbot_box
profiles: ["box", "all"]
volumes:
# Keep the source and target path identical because langbot_box uses the
# host Docker socket to create sandbox containers. Override
# LANGBOT_BOX_ROOT with an absolute path if you do not want the default.
- ${LANGBOT_BOX_ROOT:-${PWD}/data/box}:${LANGBOT_BOX_ROOT:-${PWD}/data/box}
# Mount container runtime socket for Box sandbox backend.
# Uncomment the one that matches your container runtime:
# - /var/run/podman/podman.sock:/var/run/podman/podman.sock # Podman
- /var/run/docker.sock:/var/run/docker.sock # Docker
restart: on-failure
environment:
- TZ=Asia/Shanghai
# The Box runtime does NOT read box.local.* from config.yaml or env; it
# receives its configuration from LangBot via the INIT RPC action.
# Do not add LANGBOT_BOX_* / BOX__* here — they would be silently ignored.
# Launched through the same CLI entry point as the plugin runtime
# (`langbot_plugin.cli.__init__ <subcommand>`). WebSocket is the default
# control transport — mirrors `rt`, which also runs with no flag. Pass
# `-s` / `--stdio-control` only for the stdio mode LangBot uses outside
# containers.
command: ["uv", "run", "--no-sync", "-m", "langbot_plugin.cli.__init__", "box"]
networks:
- langbot_network
langbot: langbot:
image: rockchin/langbot:latest image: rockchin/langbot:latest
container_name: langbot container_name: langbot
@@ -26,6 +60,13 @@ services:
restart: on-failure restart: on-failure
environment: environment:
- TZ=Asia/Shanghai - TZ=Asia/Shanghai
# Unified env-override convention: SECTION__SUBSECTION__KEY overrides the
# matching config.yaml field (see LoadConfigStage). These map onto
# box.local.* and are forwarded to the Box runtime via INIT RPC.
- BOX__LOCAL__HOST_ROOT=${LANGBOT_BOX_ROOT:-${PWD}/data/box}
- BOX__LOCAL__DEFAULT_WORKSPACE=default
- BOX__LOCAL__SKILLS_ROOT=skills
- BOX__LOCAL__ALLOWED_MOUNT_ROOTS=${LANGBOT_BOX_ROOT:-${PWD}/data/box}
ports: ports:
- 5300:5300 # For web ui and webhook callback - 5300:5300 # For web ui and webhook callback
- 2280-2285:2280-2285 # For platform reverse connection - 2280-2285:2280-2285 # For platform reverse connection
@@ -34,4 +75,4 @@ services:
networks: networks:
langbot_network: langbot_network:
driver: bridge driver: bridge
+174 -3
View File
@@ -1,6 +1,8 @@
# Kubernetes Deployment for LangBot # Kubernetes Deployment for LangBot
# This file provides Kubernetes deployment manifests for LangBot based on docker-compose.yaml # This file provides Kubernetes deployment manifests for LangBot based on docker-compose.yaml
# #
# Full deployment guide (zh/en/ja): https://docs.langbot.app -> Installation -> Kubernetes
#
# Usage: # Usage:
# kubectl apply -f kubernetes.yaml # kubectl apply -f kubernetes.yaml
# #
@@ -8,13 +10,15 @@
# - A Kubernetes cluster (1.19+) # - A Kubernetes cluster (1.19+)
# - kubectl configured to communicate with your cluster # - kubectl configured to communicate with your cluster
# - (Optional) A StorageClass for dynamic volume provisioning # - (Optional) A StorageClass for dynamic volume provisioning
# - For the Box sandbox runtime: a node with a reachable Docker daemon
# (the box mounts the node's /var/run/docker.sock). See the deployment guide.
# #
# Components: # Components:
# - Namespace: langbot # - Namespace: langbot
# - PersistentVolumeClaims for data persistence # - PersistentVolumeClaims for data persistence
# - Deployments for langbot and langbot_plugin_runtime # - Deployments for langbot, langbot-plugin-runtime, and langbot-box (sandbox)
# - Services for network access # - Services for network access
# - ConfigMap for timezone configuration # - ConfigMap for timezone + runtime endpoints
--- ---
# Namespace # Namespace
@@ -83,6 +87,11 @@ metadata:
data: data:
TZ: "Asia/Shanghai" TZ: "Asia/Shanghai"
PLUGIN__RUNTIME_WS_URL: "ws://langbot-plugin-runtime:5400/control/ws" PLUGIN__RUNTIME_WS_URL: "ws://langbot-plugin-runtime:5400/control/ws"
# Box sandbox runtime endpoint. LangBot connects to the Box runtime over
# WebSocket. The hostname MUST match the langbot-box Service name. Note the
# in-container default ("langbot_box") uses an underscore, which is an
# invalid Kubernetes DNS name — so the endpoint is always set explicitly here.
BOX__RUNTIME__ENDPOINT: "ws://langbot-box:5410"
--- ---
# Deployment for LangBot Plugin Runtime # Deployment for LangBot Plugin Runtime
@@ -169,6 +178,136 @@ spec:
protocol: TCP protocol: TCP
name: runtime name: runtime
---
# Deployment for LangBot Box (sandbox) runtime
#
# The Box runtime backs LangBot's sandbox tools (exec / read / write / edit /
# glob / grep), the `activate` skill tool, skill add/edit, and stdio-mode MCP
# servers. It is OPTIONAL: if you do not deploy it, set `BOX__ENABLED=false` on
# the langbot Deployment (or `box.enabled: false` in config.yaml) so the
# dashboard renders cleanly with sandbox features disabled.
#
# IMPORTANT — how the sandbox actually runs:
# The bundled image ships only the Docker CLI (no dockerd, no nsjail). The Box
# runtime therefore creates sandbox containers by talking to a Docker daemon
# over the mounted socket (`/var/run/docker.sock`). Because that daemon
# resolves bind-mount paths on the NODE filesystem, the Box workspace root
# must be the SAME absolute path inside the box container, inside every
# sandbox container it spawns, AND on the node. That is why this manifest uses
# a hostPath at a fixed absolute path (/app/data/box) and pins langbot + box
# to the same node via podAffinity. A normal PVC will NOT work for the box
# workspace, because the node's dockerd cannot see paths that exist only
# inside the pod's mount namespace.
#
# Security note: mounting the host Docker socket grants the Box runtime (and any
# code executed in the sandbox) effective root on the node. Only deploy Box on
# nodes you trust for this workload, ideally a dedicated node pool. For a
# stronger isolation boundary, switch box.backend to 'e2b' (set E2B_API_KEY) and
# drop the docker.sock mount + hostPath entirely.
apiVersion: apps/v1
kind: Deployment
metadata:
name: langbot-box
namespace: langbot
labels:
app: langbot-box
spec:
replicas: 1
selector:
matchLabels:
app: langbot-box
template:
metadata:
labels:
app: langbot-box
spec:
# Pin to the same node as langbot so they share the hostPath box root.
affinity:
podAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
- labelSelector:
matchLabels:
app: langbot
topologyKey: kubernetes.io/hostname
containers:
- name: langbot-box
image: rockchin/langbot:latest
imagePullPolicy: Always
# Launched through the same CLI entry point as the plugin runtime.
# No flag => WebSocket control transport (default), listening on 5410.
command: ["uv", "run", "--no-sync", "-m", "langbot_plugin.cli.__init__", "box"]
ports:
- containerPort: 5410
name: box-rpc
protocol: TCP
env:
- name: TZ
valueFrom:
configMapKeyRef:
name: langbot-config
key: TZ
# The Box runtime does NOT read box.local.* / BOX__* from its own env;
# it receives its configuration from LangBot via the INIT RPC action.
# Do not add BOX__* here — they would be silently ignored.
volumeMounts:
# Box workspace root — identical path on node, box, and sandbox
# containers (see the IMPORTANT note above).
- name: box-root
mountPath: /app/data/box
# Host Docker socket — the sandbox backend uses it to create containers.
- name: docker-sock
mountPath: /var/run/docker.sock
resources:
requests:
memory: "256Mi"
cpu: "100m"
limits:
memory: "1Gi"
cpu: "1000m"
livenessProbe:
tcpSocket:
port: 5410
initialDelaySeconds: 20
periodSeconds: 10
timeoutSeconds: 5
failureThreshold: 3
readinessProbe:
tcpSocket:
port: 5410
initialDelaySeconds: 10
periodSeconds: 5
timeoutSeconds: 3
failureThreshold: 3
volumes:
- name: box-root
hostPath:
path: /app/data/box
type: DirectoryOrCreate
- name: docker-sock
hostPath:
path: /var/run/docker.sock
type: Socket
restartPolicy: Always
---
# Service for LangBot Box runtime
apiVersion: v1
kind: Service
metadata:
name: langbot-box
namespace: langbot
labels:
app: langbot-box
spec:
type: ClusterIP
selector:
app: langbot-box
ports:
- port: 5410
targetPort: 5410
protocol: TCP
name: box-rpc
--- ---
# Deployment for LangBot # Deployment for LangBot
apiVersion: apps/v1 apiVersion: apps/v1
@@ -213,11 +352,36 @@ spec:
configMapKeyRef: configMapKeyRef:
name: langbot-config name: langbot-config
key: PLUGIN__RUNTIME_WS_URL key: PLUGIN__RUNTIME_WS_URL
# Box (sandbox) runtime endpoint. Connects LangBot to the langbot-box
# Service over WebSocket. Remove this (and the langbot-box Deployment)
# and set BOX__ENABLED=false if you do not want the sandbox.
- name: BOX__RUNTIME__ENDPOINT
valueFrom:
configMapKeyRef:
name: langbot-config
key: BOX__RUNTIME__ENDPOINT
# box.local.* config — forwarded to the Box runtime via INIT RPC. The
# host_root MUST match the box-root hostPath mountPath below AND the box
# Deployment's box-root mountPath, so that skill package paths resolve
# identically on both sides and on the node's Docker daemon.
- name: BOX__LOCAL__HOST_ROOT
value: "/app/data/box"
- name: BOX__LOCAL__DEFAULT_WORKSPACE
value: "default"
- name: BOX__LOCAL__SKILLS_ROOT
value: "skills"
- name: BOX__LOCAL__ALLOWED_MOUNT_ROOTS
value: "/app/data/box"
volumeMounts: volumeMounts:
- name: data - name: data
mountPath: /app/data mountPath: /app/data
- name: plugins - name: plugins
mountPath: /app/plugins mountPath: /app/plugins
# Same node-level box root as the langbot-box Deployment. Mounted over
# the data PVC's /app/data/box subpath so both LangBot and the Box
# runtime (and the node's dockerd) agree on one absolute path.
- name: box-root
mountPath: /app/data/box
resources: resources:
requests: requests:
memory: "1Gi" memory: "1Gi"
@@ -250,6 +414,13 @@ spec:
- name: plugins - name: plugins
persistentVolumeClaim: persistentVolumeClaim:
claimName: langbot-plugins claimName: langbot-plugins
# Node-level box workspace root, shared with the langbot-box Deployment.
# hostPath (not PVC) because the node's Docker daemon must see the same
# absolute path when bind-mounting workspaces into sandbox containers.
- name: box-root
hostPath:
path: /app/data/box
type: DirectoryOrCreate
restartPolicy: Always restartPolicy: Always
--- ---
+595
View File
@@ -0,0 +1,595 @@
# Box 系统架构深度分析
> 更新日期: 2026-06-02
> 状态更新: 自部署社区版已具备发布条件(box 可选、降级完善、无迁移欠债);工具调用循环上限、配额遍历异步化、`host_path` 挂载白名单等已落地。剩余多租户 / 安全硬化项见 [SaaS 阻塞项清单](./box-issues.md)。
> 分支: `feat/sandbox` (LangBot + langbot-plugin-sdk)
> 相关文档: [SaaS 阻塞项](./box-issues.md) | [Session 作用域](./box-session-scope.md) | [Runtime 对比](./box-vs-plugin-runtime.md) | [测试覆盖](./box-test-coverage.md) | [toB 分析](./box-tob-analysis.md)
---
## 1. 全局架构
```
┌──────────────────────────────────────────────────────────────────┐
│ LangBot 主进程 │
│ │
│ LocalAgentRunner ──> ToolManager ──> NativeToolLoader │
│ │ │ │ │
│ │ │ exec / read / write / edit │
│ │ │ glob / grep │
│ │ │ │
│ │ ├──> MCPLoader ──> BoxStdioSession │
│ │ │ (shared 容器, 多 process) │
│ │ │ │
│ │ ├──> SkillToolLoader (activate 工具) │
│ │ │ │
│ │ ├──> SkillAuthoringToolLoader │
│ │ │ │
│ │ └──> PluginToolLoader │
│ │ │
│ BoxService (门面) │
│ ├─ Profile 管理 (locked 字段) │
│ ├─ Host mount 校验 (allowed_mount_roots) │
│ ├─ Workspace quota 检查 │
│ ├─ 输出截断 (head+tail) │
│ ├─ Session ID 模板解析 (resolve_box_session_id) │
│ ├─ 技能挂载组装 (build_skill_extra_mounts) │
│ ├─ 重连循环 (_reconnect_loop, 指数退避) │
│ └─ BoxRuntimeConnector │
│ ├─ 心跳 loop (20s ping) │
│ └─ ActionRPCBoxClient │
│ │ Action RPC (stdio 或 WebSocket) │
│ │
│ SkillManager (skill_mgr) │
│ └─ 从 Box runtime 拉取 skills, 不可用时回落 data/skills │
└──────────────────────────────────────────────────────────────────┘
┌──────────────────────────────────────────────────────────────────┐
│ Box Runtime 进程 (SDK 侧) │
│ │
│ BoxServerHandler (Action RPC 处理, INIT 配置注入) │
│ │ │
│ BoxRuntime (session 管理 / 进程生命周期 / TTL reaper) │
│ │ └─ session.managed_processes: dict[pid, _ManagedProcess]
│ │ │
│ Backend (启动时根据 box.backend 配置选择): │
│ DockerBackend ──┐ │
│ PodmanBackend ──┤── CLISandboxBackend │
│ NsjailBackend ──┘ (本地 CLI 或 fallback 到容器内 CLI) │
│ E2BBackend (云沙箱, 需要 E2B_API_KEY) │
│ │
│ BoxSkillStore │
│ ├─ list / get / create / update / delete │
│ ├─ scan_skill_directory / read_skill_file / write_skill_file │
│ └─ preview_skill_zip / install_skill_zip (zip 或 GitHub) │
│ │
│ aiohttp 单端口服务 (默认 :5410): │
│ /rpc/ws — Action RPC │
│ /v1/sessions/{id}/managed-process/ws — 默认 process │
│ /v1/sessions/{id}/managed-process/{pid}/ws — 指定 process │
└──────────────────────────────────────────────────────────────────┘
┌──────────────────────────────────────────────────────────────────┐
│ 容器 / 沙箱 (Docker/Podman 容器, nsjail sandbox, 或 E2B 远程沙箱) │
│ - 隔离文件系统 / 网络 / PID 命名空间 │
│ - 资源限制 (CPU, 内存, PID 数, 可选 workspace 配额) │
│ - 主挂载 (host_path → mount_path) + 任意条 extra_mounts │
│ └─ Skills 通过 extra_mounts 挂在 /workspace/.skills/<name> │
│ - exec: 用户命令在此执行 │
│ - managed process: 多个长驻进程并存 (MCP Server / 自定义服务) │
└──────────────────────────────────────────────────────────────────┘
```
**核心设计原则**:
- Box Runtime 作为独立进程运行,通过 Action RPC 与 LangBot 主进程通信,两者复用 SDK 的 IO 层(Handler → Connection → Controller
- 一个 session_id 对应一个容器/沙箱实例。同一 session 内可并存多条 mount 与多个 managed process
- Skill / 默认 exec / MCP Server 共享同一个 session 容器(详见 [box-session-scope.md](./box-session-scope.md)
---
## 2. LangBot 侧模块
### 2.1 BoxService (`pkg/box/service.py`, 722 行)
应用层门面,协调 Profile、安全校验、配额、连接、Skill 挂载与 Session 模板:
主要公开方法(按定义顺序):
```
BoxService
├─ initialize() 连接 Box Runtime + 默认 workspace 准备
├─ _on_runtime_disconnect(connector) 触发重连
├─ _reconnect_loop(connector) 指数退避重连
├─ available (property) 连接状态
├─ resolve_box_session_id(query) 从 pipeline 模板解析 session_id
├─ build_skill_extra_mounts(query) 组装 pipeline-bound skill 的挂载列表
├─ execute_tool(parameters, query) Agent 调用 exec 时的入口
│ ├─ _apply_profile / build_spec
│ ├─ _validate_host_mount
│ ├─ _enforce_workspace_quota (phase=pre)
│ ├─ client.execute(spec)
│ ├─ _enforce_workspace_quota (phase=post)
│ └─ _truncate (stdout/stderr)
├─ execute_spec_payload(spec_payload, ...) 内部入口(其他 loader 调用)
├─ create_session(spec_payload, ...) 显式创建 session
├─ start_managed_process(session_id, ...) 启动 managed process
├─ get_managed_process(session_id, pid) 查询进程状态(pid 默认 'default'
├─ stop_managed_process(session_id, pid) 单独停止某个 managed process
├─ get_managed_process_websocket_url(...) 返回 WS attach URL
├─ list_skills() / get_skill(name) Skill 元数据
├─ create_skill / update_skill / delete_skill Skill CRUD
├─ scan_skill_directory(path) 扫描目录
├─ list_skill_files / read_skill_file / write_skill_file
├─ preview_skill_zip / install_skill_zip zip / GitHub 安装
├─ shutdown() / dispose() 清理:RPC SHUTDOWN + 进程终止
├─ get_status() / get_sessions() / get_recent_errors()
└─ get_system_guidance() LLM 系统提示
```
**Profile 系统**: 4 个内置 Profile`default` / `offline_readonly` / `network_basic` / `network_extended`),`locked` frozenset 字段不可被 LLM 覆盖。参数合并顺序:Profile defaults → LLM 请求参数 → locked 强制值。
**输出截断**: 默认 4000 字符上限,保留前 60% + 后 40%,中间插入 `[...truncated...]`
**Skill 挂载合并**: `execute_tool()` 调用时,`build_skill_extra_mounts(query)` 会把当前 pipeline-bound 的所有 skill 的 `package_root` 作为 `extra_mounts` 加入 BoxSpec,挂在 `/workspace/.skills/<name>`。LLM 通过 `activate` 工具显式激活某个 skill 后,工具调用才允许引用这个 skill 的虚拟路径。
### 2.2 BoxRuntimeConnector (`pkg/box/connector.py`, 357 行)
管理与 Box Runtime 的通信连接:
- **本地 stdio**: Unix/macOS 默认路径,fork `python -m langbot_plugin.cli.__init__ box -s --ws-control-port {port}` 子进程(与 plugin runtime 统一走 `lbp` CLI 入口)
- **本地 subprocess + WS**: Windows 本地(asyncio ProactorEventLoop 不支持 stdio pipe
- **远程 WebSocket**: Docker 部署 / `box.runtime.endpoint` 显式配置时,连接 `ws://{host}:{port}/rpc/ws`
- **同步等待**: `asyncio.Event` + `wait_for(timeout=30s)` 模式确认连接
- **心跳**: `_heartbeat_loop()` 每 20s 调用 `ping()`,失败仅 DEBUG 日志(断开检测靠 connection close
- **重连**: `runtime_disconnect_callback` 由 BoxService 提供,触发 `_reconnect_loop`
- **INIT 注入**: 连接建立后立即下发当前 `box.*` 配置子树(剔除 `runtime` 私有字段),Runtime 据此初始化 backend
> **历史改进**: 2026-04-16 版本本文档曾列 P0 「Box 无心跳 / 无重连」,已修复(commit `2dfd9d5d`、`c6882cf`、`5029d9c` 等)。
### 2.3 BoxWorkspaceSession 工具 (`pkg/box/workspace.py`, 413 行)
此文件目前提供两类能力:
1. **路径与命令重写工具函数**`normalize_host_path` / `rewrite_mounted_path` / `unwrap_venv_path` / `rewrite_venv_command` / `infer_workspace_host_path`,被 MCP loader 与 Skill 路径解析共用。
2. **`BoxWorkspaceSession`** — 围绕 BoxService 的轻量包装,专供 MCP-in-Box 场景使用(管理一个共享 session 的 session_id、构建挂载 payload、stage host 文件到共享 workspace)。
**变化点**: 早期 Skill exec 会为每个 skill 创建独立 BoxWorkspaceSession(独占 session);当前实现已转为 `extra_mounts` 模式,Skill 不再独占容器,只追加挂载。这部分 wrapping 逻辑已从 native loader 移除。
### 2.4 policy.py (`pkg/box/policy.py`, 98 行) — 仍是死代码
三层安全策略设计(`SandboxPolicy` / `ToolPolicy` / `ElevatedPolicy`),全项目无任何导入或调用。详见 [SaaS 阻塞项 S2](./box-issues.md)。
### 2.5 SkillManager (`pkg/skill/manager.py`, 186 行)
```
SkillManager
├─ initialize() 调用 reload_skills()
├─ reload_skills() 先从 Box runtime list_skills()
│ 不可用则回落 data/skills/ 扫描
├─ refresh_skill_from_disk() 单 skill 重新加载
├─ get_skill_by_name(name)
└─ get_managed_skills_root() 返回 Box 视角的 skills_root 路径
```
skill 元数据通过 `parse_frontmatter` 解析 `SKILL.md` 头部(`name` / `description` / `instructions`),不再做整体扫描的代价(典型 < 50 个)。
### 2.6 Skill activation (`pkg/skill/activation.py`, 33 行) + Skill loader 辅助
历史上 skill 通过 LLM 在文本中输出 `[ACTIVATE_SKILL:name]` 标记激活;当前已改为 **Tool Call 机制**
- `SkillToolLoader` (`pkg/provider/tools/loaders/skill.py`, 157 行) 暴露 `activate` 工具,参数为 skill 名
- 工具实现调用 `register_activated_skill(query, skill_data)`,将激活态写入 `query.variables['_activated_skills']`
- 这种 KV-cache-friendly 模式对齐 Claude Code 设计;详见 [box-session-scope.md §4.3](./box-session-scope.md) 的 Tool Call 描述
`activation.py` 现仅保留对外辅助函数(pipeline 层调用 loader 的 `register_activated_skill`)。
---
## 3. SDK 侧模块
### 3.1 BoxRuntime (`box/runtime.py`, 599 行)
核心编排器,管理 session 生命周期与 backend 调度:
```
Session 生命周期:
Client EXEC / CREATE_SESSION
_get_or_create_session(spec)
├─ _reap_expired_sessions_locked() 清理 TTL 过期 session
├─ 已存在? → _assert_session_compatible() → 复用
├─ Backend session 失踪? → 重建 (commit c6882cf)
└─ 新建? → backend.start_session(spec) → 创建容器
│ └─ 应用 spec.extra_mounts (多挂载)
execute(spec)
├─ 获取 session lock (每 session 独立)
├─ backend.exec(session, spec) 在容器中执行命令
├─ 更新 last_used_at
└─ 超时? → 销毁 session
Session 保持存活直到:
├─ TTL 过期 (默认 300s,下次操作时清理)
├─ 执行超时 (自动销毁)
├─ 客户端 DELETE_SESSION
└─ SHUTDOWN
```
**关键设计**:
- 每 session 有独立 `asyncio.Lock`,同一 session 内的命令串行执行
- 每 session 维护 `managed_processes: dict[process_id, _ManagedProcess]`,支持多个长驻进程并存(MCP / 自定义)
- 全局 `_lock` 保护 `_sessions` dict 的读写
- 兼容性检查:比较核心 spec 字段,`image` 字段对不支持自定义镜像的 backendnsjail/E2B)会跳过
**Backend 选择 (`_select_backend`)**: 优先级
1. 显式 `box.backend` 配置(`docker` / `nsjail` / `e2b`
2. `local` (默认) → Docker / Podman / nsjail CLI 顺序探测
3. `get_status` 调用时若当前 backend 不可用,会尝试重新选择 (commit `e5617c7`)
### 3.2 Backend 系统
#### CLISandboxBackend (`box/backend.py`, 411 行)
Docker / Podman 公共基类:
```
start_session(spec):
1. validate_sandbox_security(spec)
2. docker/podman run -d --rm --name <name>
--network none (可选)
--cpus/--memory/--pids-limit
--read-only + --tmpfs /tmp
-v <host>:<mount>:<mode> 主挂载
-v <extra.host>:<extra.mount>:.. 额外挂载 (extra_mounts)
<image> sh -lc 'while true; do sleep 3600; done'
3. 返回 BoxSessionInfo
exec(session, spec):
docker/podman exec -e KEY=VAL <container>
sh -lc 'mkdir -p <workdir> && cd <workdir> && <cmd>'
start_managed_process(session, spec):
docker/podman exec -i <container>
sh -lc 'mkdir -p <cwd> && cd <cwd> && exec <command> <args>'
返回 asyncio.subprocess.Process (stdin/stdout PIPE)
```
容器以 idle 进程启动,实际命令通过 `docker exec` 执行。`--rm` 确保容器退出时自动清理。
**Windows 支持**: backend 内对 Windows 路径处理与 subprocess 调用做了适配(commit `120817a`)。
**孤儿清理**: 启动时枚举 `langbot.box=true` 标签的容器,instance_id 不匹配的强制删除。
#### NsjailBackend (`box/nsjail_backend.py`, 552 行)
轻量级 Linux 沙箱(无容器引擎依赖):
- 使用 namespace 隔离(user/mount/pid/ipc/uts/cgroup/net
- 挂载宿主 `/usr`/`/lib`/`/bin`/`/sbin` 只读 + 选定 `/etc` 条目
- 每 session 创建独立目录(workspace/tmp/home
- 资源限制: cgroup v2 优先,fallback 到 rlimit
- **CLI 兼容**: 通过 `shutil.which(self._nsjail_bin)` 检测系统安装版 nsjail;不存在时再尝试容器内 nsjailcommit `686fcc0``feed530`
- **无自定义镜像**: 使用宿主 OS,`image` 字段固定为 `'host'`,兼容性检查跳过 image
#### E2BBackend (`box/e2b_backend.py`, 429 行)
云沙箱后端(commit `75b547f` 引入):
- 通过 `e2b` SDK 与 E2B 平台通信
- 配置:`box.e2b.api_key` / `api_url` / `template`
- 支持 `extra_mounts`commit `0fea9b1` 同步上传文件)
- 无本地容器引擎依赖,适合无 Docker 的部署或 SaaS 多租户场景
- 不支持自定义 image 字段,由 template 控制
### 3.3 Server (`box/server.py`, 508 行)
单端口 aiohttp 服务(默认 5410),通过路径区分(commit `8c71ec5` 合并端口):
1. **Action RPC** (`/rpc/ws`): `BoxServerHandler` 处理所有 action,包括 `INIT` 配置注入、skill store 操作等
2. **WS Relay** (`/v1/sessions/{id}/managed-process/ws``/v1/sessions/{id}/managed-process/{pid}/ws`): 双向桥接 WebSocket ↔ 指定 managed process stdin/stdout
stdio 模式同样会在 5410 启动 aiohttp,专门承担 managed process attachAction RPC 走 stdin/stdout。
### 3.4 Client (`box/client.py`, 377 行)
`ActionRPCBoxClient` 封装 `Handler.call_action()` 调用:
- 25+ 方法对应 25+ 个 RPC actionexec / session / managed-process / skill / status / shutdown
- 错误还原: `_translate_action_error()` 通过字符串前缀匹配还原 SDK 侧异常类型
- `execute()` timeout = 300s,其他默认 15s
- `BoxRuntimeClient` 是 ABC,供后续可能的非 RPC 实现复用
包级别 `__init__.py` 显式导出:`BoxRuntimeClient``ActionRPCBoxClient`commit `df9c722`)。
### 3.5 Actions (`box/actions.py`, 34 行)
`LangBotToBoxAction` 枚举共定义 **25 个** action
| 类别 | Actions |
|------|---------|
| 控制 | `INIT``HEALTH``STATUS``GET_BACKEND_INFO``SHUTDOWN` |
| 执行 | `EXEC` |
| Session | `CREATE_SESSION` / `GET_SESSION` / `GET_SESSIONS` / `DELETE_SESSION` |
| Managed Process | `START_MANAGED_PROCESS` / `GET_MANAGED_PROCESS` / `STOP_MANAGED_PROCESS` |
| Skill | `LIST_SKILLS` / `GET_SKILL` / `CREATE_SKILL` / `UPDATE_SKILL` / `DELETE_SKILL` / `SCAN_SKILL_DIRECTORY` / `LIST_SKILL_FILES` / `READ_SKILL_FILE` / `WRITE_SKILL_FILE` / `PREVIEW_SKILL_ZIP` / `INSTALL_SKILL_ZIP` |
### 3.6 Models (`box/models.py`, 331 行)
核心数据模型:
| 模型 | 用途 |
|------|------|
| `BoxNetworkMode` | `OFF` / `ON` |
| `BoxExecutionStatus` | `COMPLETED` / `TIMED_OUT` |
| `BoxHostMountMode` | `NONE` / `READ_ONLY` / `READ_WRITE` |
| `BoxManagedProcessStatus` | `RUNNING` / `EXITED` |
| `BoxMountSpec` | 单条挂载(host_path/mount_path/mode)— **新增** |
| `BoxSpec` | 执行请求;新增 `extra_mounts: list[BoxMountSpec]``persistent``workspace_quota_mb` |
| `BoxProfile` | 4 个内置 Profile + `locked` frozenset |
| `BoxSessionInfo` | Session 状态(含 backend_name/created_at/last_used_at |
| `BoxManagedProcessSpec` | 长驻进程参数(process_id/command/args/env/cwd |
| `BoxManagedProcessInfo` | 进程状态(status/exit_code/stderr_preview/attached |
| `BoxExecutionResult` | 执行结果(status/exit_code/stdout/stderr/duration_ms |
`BoxSpec` 校验器: `workdir` 默认继承 `mount_path``host_path` 支持 POSIX 和 Windows 路径;设置 `host_path``workdir` 必须在 `mount_path` 下。
### 3.7 BoxSkillStore (`box/skill_store.py`, 647 行)
新增模块(commit `4ab3502`),把 skill 持久化收归 Box runtime
```
BoxSkillStore
├─ list_skills() / get_skill(name)
├─ create_skill(data) / update_skill(name, data) / delete_skill(name)
├─ scan_skill_directory(path) 扫描目录返回候选 skill 包列表
├─ list_skill_files(name, path) 浏览 skill 内文件树
├─ read_skill_file(name, path) / write_skill_file(name, path, content)
├─ preview_skill_zip(zip_bytes, ...) 不落盘预览 zip 内容
└─ install_skill_zip(zip_bytes, ...) 解压、校验、复制到 skills_root
└─ 支持 source_subdir / target_suffixcommit 1aa043f
```
GitHub 安装路径:HTTP 层(`api/http/service/skill.py`)先 `git clone` 拉取,再走 `install_skill_zip` 或 directory 路径。Skill 文件存放于 `box.local.skills_root`(默认 `skills`,相对 `host_root`),容器内对应 `/workspace/.skills/`
### 3.8 Security (`box/security.py`, 52 行)
`validate_sandbox_security()`: 黑名单校验 host_path,阻止挂载 `/etc`/`/proc`/`/sys`/`/dev`/`/root`/`/boot` 及 Docker/Podman socket。
**已知缺陷**: 根路径 `/` 未拦截,用户 home 目录未拦截,是 denylist 而非 allowlist 策略。详见 [SaaS 阻塞项 S5](./box-issues.md)。
### 3.9 Errors (`box/errors.py`, 33 行)
| 异常类型 | 含义 |
|----------|------|
| `BoxError` | 基类 |
| `BoxValidationError` | spec/参数校验失败 |
| `BoxBackendUnavailableError` | 无可用 backend |
| `BoxRuntimeUnavailableError` | Runtime 服务不可用 |
| `BoxSessionConflictError` | session 已存在但 spec 不兼容 |
| `BoxSessionNotFoundError` | session 不存在 |
| `BoxManagedProcessConflictError` | session 已有同名 process |
| `BoxManagedProcessNotFoundError` | process 不存在 |
---
## 4. 工具系统集成
### 4.1 ToolManager 编排 (`toolmgr.py`)
```
ToolManager.initialize()
├─ NativeToolLoader (exec / read / write / edit / glob / grep)
├─ PluginToolLoader (插件工具)
├─ MCPLoader (MCP Server 工具)
├─ SkillToolLoader (activate 工具 — Tool Call 激活)
└─ SkillAuthoringToolLoader (Skill CRUD)
工具调用优先级: native → plugin → mcp → skill → skill_authoring
```
### 4.2 Native Tools (`native.py`, 846 行)
| 工具 | 是否在 Box 中执行 | 是否访问宿主文件系统 |
|------|:---:|:---:|
| `exec` | 是 | 否 |
| `read` | **否** | **是** — 直接 `open()` 宿主文件 |
| `write` | **否** | **是** — 直接 `open()` 宿主文件 |
| `edit` | **否** | **是** — 直接 `open()` 宿主文件 |
| `glob` | **否** | **是** — 直接遍历宿主目录 |
| `grep` | **否** | **是** — 直接读宿主文件 |
**沙箱边界不对称**: 这是刻意的设计权衡 — `read`/`write`/`edit`/`glob`/`grep` 绕过沙箱以获得性能(避免容器 I/O 开销与跨进程拷贝),但意味着 LLM 可以直接读写 `allowed_mount_roots` 下任何文件。Skill 路径经 `_resolve_host_path()` 重写,禁止穿越 `package_root`
**exec 的 Skill 分支**: 命令中引用 `/workspace/.skills/<name>` 的 skill 时:
1. 验证 skill 已激活
2. 单次 exec 只能引用一个 skill 包
3. 若 skill 是 Python 项目(有 `requirements.txt``pyproject.toml`),命令会被 venv bootstrap 包裹(在 skill 挂载点内创建 `.venv`
4. 调用 `box_service.execute_tool()` → 走默认 session_id 与已组装好的 `extra_mounts`**不再为每 skill 起独立 session**
### 4.3 MCP-in-Box (`mcp_stdio.py`, 354 行)
`BoxStdioSessionRuntime` 让 MCP stdio 服务器在 Box 容器中运行,**共享 session、多 process**模式(commit `529088e`):
```
initialize()
1. 复用/创建共享 session (session_id = _build_box_session_id())
- persistent=True,长期保持
2. workspace.execute_raw(install_cmd) 安装依赖 (可选)
3. 将每个 MCP server 文件 stage 到 /workspace/.mcp/<process_id>/
4. workspace.start_managed_process(process_id=<server>)
5. websocket_client(ws_url) 通过 WS relay 连接
6. ClientSession.initialize() MCP 协议握手
```
配置 (`MCPServerBoxConfig`): `network='on'` (MCP 服务器通常需要网络)`host_path_mode='ro'` (默认只读)`startup_timeout_sec=120` (留时间给 pip install)。
每条 MCP server 是同一 session 中的一个 managed process,独立的 `process_id`、独立 attach URL,互不阻塞。
---
## 5. 启动与生命周期
### 5.1 启动顺序 (`build_app.py`)
```
BuildAppStage.run(ap)
├─ ... (persistence, models, sessions) ...
├─ BoxService(ap)
├─ box_service.initialize()
│ └─ connector.initialize()
│ ├─ [stdio] fork box subprocess
│ ├─ [subprocess+WS] Windows 本地
│ └─ [remote WS] connect URL
│ └─ 启动心跳 _heartbeat_task
├─ ap.box_service = box_service
├─ ToolManager(ap)
├─ tool_mgr.initialize()
│ ├─ NativeToolLoader (检查 box_service.available)
│ ├─ PluginToolLoader
│ ├─ MCPLoader (Box 可用时,stdio MCP 走沙箱)
│ └─ SkillAuthoringToolLoader
├─ ap.tool_mgr = tool_mgr
├─ ... (platform, pipeline) ...
├─ SkillManager.initialize() (从 Box runtime 加载 skill 列表)
└─ ... (RAG, HTTP, plugins) ...
```
BoxService 在 ToolManager **之前**初始化。ToolManager 创建 loader 时检查 `box_service.available`
### 5.2 初始化失败处理
```python
try:
await self._runtime_connector.initialize()
self._available = True
except Exception as e:
self._available = False
logger.warning(f"Box runtime unavailable: {e}")
```
**静默降级**: Box 初始化失败不会阻止应用启动,仅导致 6 个 native tool、所有 Skill 工具和 MCP-in-Box 工具不暴露给 LLM。与 Plugin 的行为不同(Plugin 失败会抛异常)。
### 5.3 销毁流程
```
app.dispose()
└─ box_service.dispose()
├─ connector.dispose()
│ ├─ cancel _heartbeat_task
│ ├─ cancel _handler_task / _ctrl_task
│ └─ terminate subprocess (SIGTERM)
└─ loop.create_task(client.shutdown())
└─ RPC SHUTDOWN → Box Runtime 清理所有容器
```
Box 额外做了 RPC SHUTDOWN 通知 Runtime 主动清理容器,比 Plugin 的直接杀进程更安全。
---
## 6. 配置
### config.yaml (重构后)
```yaml
box:
enabled: true # 整个 Box 子系统的总开关。设为 false 时:
# - 不连接远程 Box runtime,不 fork 本地 stdio 子进程
# - sandbox 工具 (exec/read/write/edit/glob/grep) 不暴露给 LLM
# - skill 添加/编辑 / GitHub 安装 / 文件写入全部拒绝
# - stdio 模式的 MCP server 启动时报错(http/sse 模式不受影响)
# - skill 列表/读取保持只读可用
# BOX__ENABLED 环境变量可覆盖(统一约定)
backend: 'local' # 'local' (探测) / 'docker' / 'nsjail' / 'e2b'
# 由 box.backend / BOX__BACKEND 选择后端
runtime:
endpoint: '' # 外部 Runtime 的 WS 基地址 'ws://host:5410'
# 留空 = 本地自管 Runtime
local:
profile: 'default'
image: '' # 覆盖 profile 默认 image
host_root: './data/box' # 工作区挂载根,Docker 部署需绝对路径
default_workspace: '' # 默认 '<host_root>/default'
skills_root: 'skills' # Box 管理的 skill 包目录(相对 host_root
allowed_mount_roots: # 默认 ['<host_root>']
- './data/box'
- '/tmp'
workspace_quota_mb: null # 配额覆盖,null = 走 profile
e2b:
api_key: '' # 也可走 E2B_API_KEY 环境变量
api_url: '' # 自托管 E2B 时填写
template: '' # 默认 template ID
```
> **重大变更**: 较 2026-04-16 文档,配置结构完全重组(commit `eefdea4`)。原字段 `box.profile` / `box.runtime_url` / `box.shared_host_root` / `box.allowed_host_mount_roots` 全部迁入 `box.local.*` 子表,新增 `box.backend` 与 `box.e2b.*` 配置组。
### docker-compose.yaml
`langbot_box` 服务受 compose profile 控制,默认 `docker compose up` **不会**启动它。需要 sandbox 时:
```bash
docker compose --profile box up # 启动 langbot + langbot_box + plugin runtime
docker compose --profile all up # 同上
docker compose up # 只起 langbot + plugin runtime (box 关闭)
```
若不起 `langbot_box`,需要同步在 `data/config.yaml` 中设 `box.enabled: false`(或 langbot 容器 env 加 `BOX__ENABLED=false`),否则 LangBot 会一直尝试连接不存在的 Box runtime 并报错。
```yaml
# langbot_box 的关键 volume
volumes:
- ${LANGBOT_BOX_ROOT}:${LANGBOT_BOX_ROOT} # 工作区挂载(源/目标同路径)
- /var/run/docker.sock:/var/run/docker.sock # Docker backend 复用宿主 docker
```
### 关闭/连接失败时的行为矩阵
`box.enabled = false` 与"启用但连接失败"在用户可观察行为上**完全一致**——都通过 `BoxService.available = False` 表达,只是 `get_status` 多返回 `enabled` 字段供前端区分文案。
| 消费方 | Box 可用 | Box 不可用(disabled 或 failed) |
|---|---|---|
| native exec/read/write/edit/glob/grep 工具 | 暴露给 LLM | **不暴露** |
| `activate` / `register_skill` 工具 | 暴露给 LLM | **不暴露** |
| stdio MCP server | 在 Box 内启动 | **`_init_stdio_python_server` 抛 RuntimeError** 拒绝;不退化到宿主 stdio |
| http/sse MCP server | 正常 | 正常(不依赖 Box) |
| Skill 列表/读取 (`list_skills`/`get_skill`/`read_skill_file`) | 走 Box runtime | 走 LangBot 本地 `data/skills/` 只读 fallback |
| Skill 创建/编辑/安装/写文件 | 走 Box runtime | **HTTP 400** + 明确错误信息(`_require_box_for_write`) |
| Pipeline AI 配置中 `box-session-id-template` | 正常生效 | **前端 banner** 提示字段无效 |
| Pipeline 扩展页 `enable_all_skills` / 绑定 skill | 可编辑 | **前端禁用** + banner |
| 仪表盘 Box 状态卡片 | 绿点 / "已连接" | 灰点 / "已禁用"(disabled) 或 红点 / "已断开"(failed) |
> 后端拒写的边界条件:如果 `ap.box_service` **完全没装**(老式 dev mode,没经过 BuildAppStage),`_require_box_for_write` 视作 no-op,保留 `data/skills/` 本地路径——以兼容历史测试与最小化设置。生产环境总会装 `ap.box_service`,因此该 fallback 不会被触发。
### Pipeline 配置 (templates/metadata/pipeline/ai.yaml)
`local-agent.config.box-session-id-template` 控制 session 作用域,预设:
- `{launcher_type}_{launcher_id}` — 每个会话 (推荐,默认)
- `{launcher_type}_{launcher_id}_{sender_id}` — 群聊每个用户
- `{launcher_type}_{launcher_id}_{conversation_id}` — 每个对话上下文
- `{query_id}` — 每条消息(完全隔离)
详见 [box-session-scope.md](./box-session-scope.md)。
### REST API
| 端点 | 方法 | 说明 | 前端 |
|------|------|------|:---:|
| `/api/v1/box/status` | GET | 可用性、Profile、后端信息 | ✅ 监控页 |
| `/api/v1/box/sessions` | GET | 活跃 session 列表 | ❌ |
| `/api/v1/box/errors` | GET | 最近 50 条错误 | ❌ |
| `/api/v1/skills` 等 | GET/POST/PUT/DELETE | Skill CRUD、文件浏览、zip/GitHub 安装、preview | ✅ Skill 管理页 |
前端 `web/src/app/home/monitoring/components/overview-cards/SystemStatusCards.tsx` 已接入 `/api/v1/box/status`,展示 backend 名称、profile 与活跃 session 数。Sessions 与 errors API 仍未接入。
+76
View File
@@ -0,0 +1,76 @@
# Box 系统 — SaaS 发布前阻塞项
> 更新日期: 2026-06-02
> 分支: `feat/sandbox` (LangBot + langbot-plugin-sdk)
> 相关文档: [架构分析](./box-architecture.md) | [Session 作用域](./box-session-scope.md) | [Runtime 对比](./box-vs-plugin-runtime.md) | [测试覆盖](./box-test-coverage.md) | [toB 分析](./box-tob-analysis.md)
## 范围说明
**自部署社区版已具备发布条件**:默认 stdio 模式、box 为可选项;box 关闭 / 不可用时后端、前端、工具、skill、stdio-MCP 均能干净降级(清晰报错、不崩溃);配置向后兼容(旧 `data/config.yaml` 可直接启动);无新增 ORM 模型、无迁移欠债;市场安装失败不会破坏实例。CI 全绿。
本清单**只保留发布 SaaS / 多租户 / 公网暴露前必须处理的阻塞项**。社区版(可信、单运营者、内网)不受这些项阻塞——它们的风险面在"不可信调用方能直接触达 Box 控制面"或"多租户共享资源"的场景才成立。
## 已解决(社区版发布前)
| 项 | 处理 |
|----|------|
| 工具调用循环无上限 (原 #13) | `localagent.py` 增加 `MAX_TOOL_CALL_ROUNDS=128`,超限优雅终止(`cafef1a3` |
| 配额校验同步遍历阻塞事件循环 (原 #10) | `_enforce_workspace_quota` 改 async,工作区遍历走 `asyncio.to_thread``cafef1a3` |
| `host_path` 挂载白名单 (原 #3 的 LangBot 侧) | `pkg/box/service.py` `allowed_mount_roots` 白名单,空列表时拒绝一切宿主挂载 |
| 重复的 `_is_path_under` (原 #12) | 已去重,仅保留一处定义 |
| 重连 / 心跳 / Windows 兼容 / nsjail image 字段 / 前端 Box 状态接入 | 见上一轮 review 记录,均已合入 |
---
## SaaS 阻塞项
### S1. Box 控制面无认证 — Critical
- **位置**: SDK `box/server.py` — Action RPC WS (`/rpc/ws`) 与 managed-process relay (`/v1/sessions/{id}/managed-process/{pid}/ws`)
- **现状**: 两个 WS handler 在 `ws.prepare` 后直接服务,无任何 token / 鉴权;box 默认绑定 `0.0.0.0:5410`。任何能触达该端口者可发起 `EXEC`、创建 session、attach 任意 session 的 managed-process stdin/stdout、甚至 `SHUTDOWN`。LangBot→box 的 INIT 也未下发任何凭证。
- **缓解现状**: 默认 `docker-compose.yaml``langbot_box` 未把 5410 发布到宿主(爆炸半径限于内网 bridge);但 box 挂载了 `/var/run/docker.sock`,同网络的任意服务(含被攻破的插件)→ 宿主 root。若运营者把 5410 发布到宿主或独立以 `0.0.0.0` 起 box,则完全裸奔。
- **要求**: INIT 时下发 token,两个 WS 路由按连接校验(query/header)。这是 SaaS 的**头号**阻塞项。
### S2. 无 exec 授权模型(policy.py 死代码) — High
- **位置**: LangBot `pkg/box/policy.py``SandboxPolicy` / `ToolPolicy` / `ElevatedPolicy` 全项目无引用);`pkg/provider/tools/loaders/native.py``pkg/provider/tools/toolmgr.py`
- **现状**: 原生工具(`exec/read/write/edit/glob/grep`)按"box 是否可用"全有或全无地暴露,**无 per-pipeline 的 exec 网关 / 工具白名单 / 沙箱模式 / 权限提升控制**。只要 box 可用,任何使用 local-agent + 函数调用模型的 pipeline 都能跑任意 shell。
- **要求**: 接入 policy.py(或等价机制),按 pipeline 控制是否暴露 `exec`、可用工具白名单、沙箱网络/只读模式。
### S3. 会话资源无界(DoS — High
- **#5 session 数量无上限**: SDK `box/runtime.py` `_get_or_create_session``_sessions` dict 无容量限制——可变 `session_id` 的恶意调用可无限创建容器,耗尽宿主 CPU/内存/PID/磁盘。
- **#8 无定时回收**: 过期 session 仅在 `_get_or_create_session` 时机会性清理,无独立周期任务;一波创建后转静默会永久泄漏容器。
- **要求**: `max_sessions` 上限(拒绝或 LRU),加独立周期 reaper(如 60s)。
### S4. 工作区配额无内核级限制(TOCTOU — Med-High
- **位置**: LangBot `pkg/box/service.py` `_enforce_workspace_quota`(应用层 read-then-check);SDK 侧 `workspace_quota_mb` 仅记录/透传,无 `--storage-opt size=` 等内核/FS 限额
- **现状**: 执行前后两次检查之间存在竞态窗口;单条命令(`dd`/`fallocate`)可在检查间隙撑爆磁盘,事后检查只能补救。
- **要求**: Docker `--storage-opt size=` 做内核级限制,或 Redis 原子计数预留式配额。
### S5. 挂载校验缺口 — Med-High
- **位置**: SDK `box/security.py` `_BLOCKED_HOST_PATHS_POSIX``box/backend.py``extra_mounts` 处理
- **现状**: ① SDK 黑名单仍不含 `/`(前缀匹配,`host_path="/"` 可通过,挂载整个宿主 fs);用户 home、`/usr``/opt``/tmp` 也未拦截。② `validate_sandbox_security` 只校验 `spec.host_path`**从不遍历 `spec.extra_mounts`**——LangBot 侧 `allowed_mount_roots` 也只校验 `host_path`。当前 `extra_mounts` 仅由 `build_skill_extra_mounts` 内部填充(agent 不可达),但缺乏纵深防御:一旦 S1 的无认证 RPC 被触达,extra_mounts 可挂任意宿主路径,两层都不拦。
- **要求**: SDK 黑名单加入 `/`(或改白名单);`extra_mounts` 在 SDK 与 LangBot 两侧都纳入挂载校验。
### S6. 容器加固缺失 — Med
- **位置**: SDK `box/backend.py``docker run` 组装
- **现状**: 未设置 `--cap-drop=ALL``--security-opt=no-new-privileges`、非 root `--user`;叠加挂载 docker.sock,逃逸面偏大。
- **要求**: 默认加上上述加固 flag(需回归常用 skill 不被破坏)。
### S7. 全局锁内执行慢操作(扩展性) — Med
- **位置**: SDK `box/runtime.py` `_get_or_create_session``self._lock` 持有期间调用 `backend.start_session()``docker run` / nsjail 启动 / E2B `Sandbox.create`
- **影响**: 冷启动(镜像拉取数秒、E2B >1s)期间串行阻塞所有并发请求——多租户负载下整个 Box runtime 停顿。降级表现是延迟而非失败。
- **要求**: 锁内只做状态检查与注册,容器创建移到锁外。
### S8. 其他硬化 / 跟进 — Low
- **#9** SDK `box/server.py` 直接读 `runtime._sessions` 私有字段、绕过锁,并发下可能读到不一致状态——应加公共访问方法。
- **#16** `pkg/provider/tools/toolmgr.py` `execute_func_call` 按优先级分发,plugin/MCP 若有同名 `exec/read/write/...` 工具会被静默遮蔽——应加命名空间或冲突告警。
- **#4** SDK `box/runtime.py` INIT/handshake 与 backend 实例化的残留竞态(仅"纯远程 WS box 先启动、LangBot 后连"场景成立;stdio/compose 路径下 config 经 env 在 spawn 时已就位,无竞态)——应在 INIT 完成前拒绝业务 action。
- **#11** `extra_mounts` 在容器创建时固定(SDK `runtime.py` 兼容性检查不含 extra_mounts);长生命周期共享 session 后续新激活的 skill 不会挂上(当前缓解:创建时挂上 pipeline 绑定的全部 skill)——动态绑定场景需销毁重建或文档说明。
- **#21** 集成测试未进 CI:容器实际执行、E2B 真机、managed-process WS attach 仅本地可跑。安全关键路径缺自动化覆盖——SaaS 前建议加 Docker-in-Docker CI stage 或合并前手动 checklist。
+402
View File
@@ -0,0 +1,402 @@
# Box Session Scope Design
> Date: 2026-04-18 (last reviewed 2026-06-02)
> Status (2026-06-02): the self-hosted community edition is release-ready (box optional, clean degradation, no migration debt). Tool-call loop cap, async quota scan, and the host_path mount allowlist have landed. Remaining multi-tenant / security hardening is tracked in [box-issues.md](./box-issues.md).
> Branch: `feat/sandbox` (LangBot + langbot-plugin-sdk)
> Related: [Box Architecture](./box-architecture.md) | [Box vs Plugin Runtime](./box-vs-plugin-runtime.md)
---
## 0. Implementation Status (2026-05-19)
This document was authored as a design proposal. The current `feat/sandbox` branch
has shipped the design largely as written:
| Item | Status | Notes |
|------|--------|-------|
| `BoxMountSpec` + `BoxSpec.extra_mounts` | ✅ Shipped | SDK `box/models.py` |
| Docker / nsjail / E2B backends apply extra mounts | ✅ Shipped | Last gap closed by SDK commit `0fea9b1` (E2B) |
| `box-session-id-template` in `local-agent` pipeline config | ✅ Shipped | `templates/metadata/pipeline/ai.yaml`, default `{launcher_type}_{launcher_id}` |
| `BoxService.resolve_box_session_id(query)` | ✅ Shipped | `pkg/box/service.py:166` |
| `BoxService.build_skill_extra_mounts(query)` | ✅ Shipped | `pkg/box/service.py:189` |
| Skill exec uses unified container + extra mounts | ✅ Shipped | `pkg/provider/tools/loaders/native.py` skill branch |
| MCP-in-Box uses shared persistent session, multi-process | ✅ Shipped (earlier than originally scoped) | SDK commit `529088e`, LangBot `mcp_stdio.py:_build_box_session_id` |
| `BoxManagedProcessSpec.process_id` + multi-process per session | ✅ Shipped | `BoxRuntime` keeps `managed_processes: dict[pid, _ManagedProcess]` |
| Per-tenant / quota integration with templates | ❌ Not started | See [box-tob-analysis.md](./box-tob-analysis.md) |
The "Phase 2 deferred" note in §10 is **out of date** — MCP unification went in on
the same line. Pipeline-scoped (not user-scoped) MCP container is the realized
behavior: each pipeline's MCP servers share one `mcp-<pipeline>` session, and
user exec sessions use the template-derived id.
The remaining open work is multi-tenant overlays (tenant_id in session_id,
quota counters keyed by tenant), tracked in the toB analysis doc rather than here.
---
## 1. Problems
### 1.1 Default exec: per-message containers
Currently, `BoxService.execute_tool()` sets `session_id = str(query.query_id)` — an
auto-incrementing integer per incoming message. Every user message creates a new sandbox
container. Dependencies installed and in-container state are lost between messages.
### 1.2 Three isolated container pools
Default exec, skills, and MCP servers each manage their own containers with
independent session IDs:
| Path | Session ID | Container |
|--------------|-----------------------------------------------|-------------|
| Default exec | `str(query_id)` (per message) | Ephemeral |
| Skill exec | `skill-{launcher}_{id}-{skill_name}` | Per skill |
| MCP stdio | `mcp-{server_uuid}` | Per server |
This means a single logical user interaction can spawn 3+ containers that cannot
share state, see each other's files, or reuse installed dependencies.
### 1.3 Single bind mount limitation
`BoxSpec` currently supports only **one** `host_path``mount_path` bind mount.
This prevents mounting both a default workspace and skill directories into the
same container.
---
## 2. Concept Model
```
Platform Message
→ Query (query_id: int, auto-increment, per message)
→ Session (launcher_type + launcher_id, per chat window)
→ Conversation (uuid, per dialogue context within a Session)
```
| Concept | Key | Example | Scope |
|---------------|-------------------------------------|----------------------------|------------------------------|
| Query | `query_id` | `42` | Single message |
| Session | `launcher_type` + `launcher_id` | `group_123456` | Chat window (group or PM) |
| Conversation | `conversation_id` (UUID) | `a1b2c3d4-...` | Dialogue context within a Session |
| Sender | `sender_id` | `789` | Individual user |
Note: in a **group chat**, all users share the same Session (keyed by `group_id`). The
individual sender is tracked as `sender_id` but does not affect Session/Conversation routing.
---
## 3. Target Scenarios
| # | Scenario | Box Granularity | Desired `session_id` |
|----|--------------------------------|------------------------------------------|---------------------------------------------------------|
| 1 | Personal assistant | 1 Box per user, long-lived | `{launcher_type}_{launcher_id}` |
| 2 | Customer service | 1 Box per customer, cross-pipeline | `{launcher_type}_{launcher_id}` |
| 3 | Internal employee tool | 1 Box per employee | `{launcher_type}_{launcher_id}` |
| 4 | Group chat shared assistant | 1 Box per group | `{launcher_type}_{launcher_id}` |
| 5 | Group chat isolated per user | 1 Box per user within a group | `{launcher_type}_{launcher_id}_{sender_id}` |
| 6 | Teaching (cross-channel) | 1 Box per student across groups/PMs | `{sender_id}` |
| 7 | One-off execution | 1 Box per message (current behavior) | `{query_id}` |
| 8 | Multi-project development | 1 Box per conversation context | `{launcher_type}_{launcher_id}_{conversation_id}` |
No single fixed granularity covers all scenarios. A template-based approach is needed.
---
## 4. Design Overview
Two key changes:
1. **Unified container**: exec, skills, and MCP all share the same container per
session scope. No more separate container pools.
2. **Configurable session scope**: `session_id` is generated from a template with
pipeline variables, configurable per pipeline.
### 4.1 Unified Container with Multiple Mounts
A single container per session scope is created on first use. It has:
- **Primary mount**: default workspace at `/workspace` (from `default_host_workspace`)
- **Skill mounts**: each pipeline-bound skill's `package_root` mounted at
`/workspace/.skills/{skill_name}/`
- **MCP servers**: run as managed processes inside the same container
```
Container (session_id = "group_123456")
/workspace/ ← default workspace (bind mount, rw)
/workspace/.skills/web-search/ ← skill package (bind mount, rw)
/workspace/.skills/data-analysis/ ← skill package (bind mount, rw)
[managed process: mcp-server-a] ← MCP server running inside
[managed process: mcp-server-b] ← MCP server running inside
```
This requires extending `BoxSpec` to support multiple mounts (see §5).
### 4.2 Session ID Template
A new field `box-session-id-template` in the `local-agent` pipeline runner config
controls the session scope:
```yaml
# templates/metadata/pipeline/ai.yaml (under local-agent.config)
- name: box-session-id-template
label:
en_US: Sandbox Scope
zh_Hans: 沙箱作用域
description:
en_US: >-
Determines how sandbox environments are shared. Use variables to
control isolation granularity.
zh_Hans: >-
决定沙箱环境的共享方式。使用变量控制隔离粒度。
type: select
required: false
default: "{launcher_type}_{launcher_id}"
options:
- value: "{launcher_type}_{launcher_id}"
label:
en_US: Per chat (Recommended)
zh_Hans: 每个会话(推荐)
- value: "{launcher_type}_{launcher_id}_{sender_id}"
label:
en_US: Per user in chat
zh_Hans: 会话中每个用户
- value: "{launcher_type}_{launcher_id}_{conversation_id}"
label:
en_US: Per conversation context
zh_Hans: 每个对话上下文
- value: "{query_id}"
label:
en_US: Per message (isolated)
zh_Hans: 每条消息(完全隔离)
```
Available template variables (populated by PreProcessor in `query.variables`):
| Variable | Source | Example |
|---------------------|---------------------------------|----------------------|
| `{launcher_type}` | `query.session.launcher_type` | `person` / `group` |
| `{launcher_id}` | `query.session.launcher_id` | `123456` |
| `{sender_id}` | `query.sender_id` | `789` |
| `{conversation_id}` | `conversation.uuid` | `a1b2c3d4-...` |
| `{query_id}` | `query.query_id` | `42` |
Default `{launcher_type}_{launcher_id}` covers scenarios 14 out of the box.
---
## 5. SDK Changes: Multi-Mount BoxSpec
### 5.1 Model Extension
```python
# box/models.py
class BoxMountSpec(pydantic.BaseModel):
"""A single bind mount specification."""
host_path: str
mount_path: str
mode: BoxHostMountMode = BoxHostMountMode.READ_WRITE
class BoxSpec(pydantic.BaseModel):
# ... existing fields ...
host_path: str | None = None # Primary mount (backward compat)
host_path_mode: BoxHostMountMode = BoxHostMountMode.READ_WRITE
mount_path: str = DEFAULT_BOX_MOUNT_PATH
extra_mounts: list[BoxMountSpec] = [] # NEW: additional mounts
```
`extra_mounts` is additive — the existing `host_path` / `mount_path` pair remains
the primary mount for backward compatibility.
### 5.2 Backend: Apply Extra Mounts
```python
# box/backend.py — CLISandboxBackend.start_session()
# Primary mount (unchanged)
if spec.host_path is not None and spec.host_path_mode != BoxHostMountMode.NONE:
args.extend(['-v', f'{spec.host_path}:{spec.mount_path}:{spec.host_path_mode.value}'])
# Extra mounts (NEW)
for mount in spec.extra_mounts:
if mount.mode != BoxHostMountMode.NONE:
args.extend(['-v', f'{mount.host_path}:{mount.mount_path}:{mount.mode.value}'])
```
Same pattern for nsjail backend.
---
## 6. LangBot Changes
### 6.1 Session ID Resolution
In `BoxService.execute_tool()`:
```python
# Before:
spec_payload.setdefault('session_id', str(query.query_id))
# After:
template = (query.pipeline_config or {}).get('ai', {}) \
.get('local-agent', {}).get('box-session-id-template',
'{launcher_type}_{launcher_id}')
variables = query.variables or {}
session_id = template.format_map(collections.defaultdict(
lambda: 'unknown', variables
))
spec_payload.setdefault('session_id', session_id)
```
### 6.2 Skill Exec: Use Same Container
Currently `native.py:_invoke_exec` creates a separate `BoxWorkspaceSession` per
skill with `host_path=package_root`. Instead:
1. Use the **same session_id** as default exec (from the template).
2. Pass the skill's `package_root` as an **extra mount** at
`/workspace/.skills/{skill_name}/` instead of replacing `/workspace`.
3. The container already has the default workspace at `/workspace`.
```python
# native.py — _invoke_exec, skill branch (REVISED)
# Same session_id as default exec
session_id = resolve_box_session_id(query)
spec_payload = {
'cmd': rewritten_command,
'workdir': rewritten_workdir,
'session_id': session_id,
'extra_mounts': [{
'host_path': package_root,
'mount_path': f'/workspace/.skills/{selected_skill_name}',
'mode': 'rw',
}],
}
result = await self.ap.box_service.execute_spec_payload(spec_payload, query)
```
The virtual path `/workspace/.skills/{name}` no longer needs rewriting at the
command level — it maps directly to the bind mount path inside the container.
### 6.3 MCP: Use Same Container
MCP servers should run inside the same container as exec and skills. Changes:
1. `BoxStdioSessionRuntime` uses the pipeline's session_id template instead of
`mcp-{server_uuid}`.
2. MCP server's working directory is a subdirectory (e.g. `/workspace/.mcp/{name}/`).
3. MCP server's dependencies are mounted or installed into that subdirectory.
4. The MCP server runs as a managed process inside the shared container.
Since MCP servers start at LangBot boot (not per-query), the session must be
created eagerly. The container will be kept alive by the managed process
exemption in TTL reaping (`runtime.py:259`).
**Note**: MCP sessions are pipeline-scoped (not per-launcher), so their session_id
should be a **fixed identifier per pipeline** rather than the user-facing template.
This means one shared MCP container per pipeline, with user exec sessions separate.
Alternatively, in a future iteration, MCP managed processes could be launched
lazily into the user's container on first MCP tool call. This is more complex
but maximizes sharing. For V1, keeping MCP containers at pipeline scope is
simpler and more predictable.
---
## 7. Mount Layout Summary
### Default exec (no skills activated)
```
Container (session_id from template)
/workspace/ ← default_host_workspace (rw)
```
### Exec with activated skills
```
Container (same session_id)
/workspace/ ← default_host_workspace (rw)
/workspace/.skills/web-search/ ← skill package_root (rw)
/workspace/.skills/data-analysis/ ← skill package_root (rw)
```
Extra mounts are **additive** — they are added when the container is first
created (or on the first exec that references a skill). Since Docker bind
mounts are specified at container creation time, skills must be known at
creation time.
**Resolution**: When creating a container, inject `extra_mounts` for **all
pipeline-bound skills** (from `extensions_preferences`), not just the
currently activated one. This way any skill can be activated later without
recreating the container.
### MCP servers (V1: pipeline-scoped)
```
Container (session_id = "mcp-pipeline-{pipeline_uuid}")
/workspace/ ← MCP shared workspace
/workspace/.mcp/server-a/ ← MCP server A files
/workspace/.mcp/server-b/ ← MCP server B files
[managed process: server-a]
[managed process: server-b]
```
---
## 8. Data Migration
Existing pipelines do not have `box-session-id-template`. The backend uses
`.get(..., default)` so missing keys fall back to `{launcher_type}_{launcher_id}`.
This changes behavior from per-message to per-launcher for existing pipelines.
Recommendation: **accept the behavior change** — per-launcher is the more
intuitive default, and the old per-message behavior was rarely desired.
---
## 9. Cloud Quota Implications
| Scope | Typical concurrent containers |
|-----------------------------------------------|-------------------------------|
| `{query_id}` (per message) | Many, short-lived |
| `{launcher_type}_{launcher_id}` (per chat) | = active chat count |
| `{sender_id}` (per user) | = active user count |
| `{conversation_id}` (per conversation) | Between per-chat and per-msg |
With the unified container model, each scope value maps to exactly **one**
container (instead of potentially 3+ per-message). This significantly reduces
resource usage.
Quota enforcement point: `BoxRuntime._get_or_create_session()` in the SDK.
---
## 10. Implementation Phases
### Phase 1: Session scope + skill unification (this PR)
1. **SDK**: Extend `BoxSpec` with `extra_mounts: list[BoxMountSpec]`.
2. **SDK**: Update Docker/nsjail backends to apply extra mounts.
3. **LangBot**: Add `box-session-id-template` to `local-agent` YAML metadata
and default pipeline config JSON.
4. **LangBot**: Update `BoxService.execute_tool()` to use template interpolation.
5. **LangBot**: Update `native.py:_invoke_exec` skill branch to use same
session_id + extra mounts instead of separate `BoxWorkspaceSession`.
6. **LangBot**: On container creation, inject extra mounts for all
pipeline-bound skills.
7. **Frontend**: No code change — `DynamicFormComponent` renders `select` fields.
8. **Tests**: Unit tests for template interpolation and multi-mount specs.
### Phase 2: MCP unification (future)
1. Refactor `BoxStdioSessionRuntime` to use pipeline-scoped shared container.
2. MCP servers become managed processes in the shared container.
3. Support multiple concurrent managed processes per container.
MCP unification is deferred because it requires changes to the managed process
model (currently 1 managed process per session) and has startup ordering
concerns (MCP servers start at boot, before any user query determines
a session_id).
+122
View File
@@ -0,0 +1,122 @@
# Box 系统测试覆盖分析
> 更新日期: 2026-06-02
> 状态更新: 自部署社区版已具备发布条件(box 可选、降级完善、无迁移欠债);工具调用循环上限、配额遍历异步化、`host_path` 挂载白名单等已落地。剩余多租户 / 安全硬化项见 [SaaS 阻塞项清单](./box-issues.md)。
> 分支: `feat/sandbox` (LangBot + langbot-plugin-sdk)
---
## 1. 测试文件清单
### LangBot 仓库
| 文件 | 行数 | CI 运行 | 覆盖范围 |
|------|------|---------|---------|
| `tests/unit_tests/box/test_box_connector.py` | 106 | 是 | Connector 传输决策、WS relay URL、dispose、心跳/重连 |
| `tests/unit_tests/box/test_box_service.py` | 1224 | 是 | Service 核心逻辑(最全面) |
| `tests/unit_tests/box/test_workspace.py` | 147 | 是 | WorkspaceSession 路径重写、payload 构建 |
| `tests/unit_tests/provider/test_mcp_box_integration.py` | 707 | 是 | MCP Box 配置、路径重写、payload、shared-session/multi-process、runtime info |
| `tests/unit_tests/provider/test_localagent_sandbox_exec.py` | 444 | 是 | LocalAgent exec 流程、流式、Skill 激活 (Tool Call) |
| `tests/unit_tests/provider/test_tool_manager_native.py` | 249 | 是 | ToolManager 路由、native tool CRUD、路径穿越、6 工具暴露 |
| `tests/unit_tests/provider/test_skill_tools.py` | 582 | 是 | Skill 管理、Tool Call 激活、路径、authoring CRUD |
| `tests/unit_tests/test_skill_service.py` | 396 | 是 | HTTP serviceskill CRUD、zip/GitHub install、文件浏览 |
| `tests/unit_tests/test_paths.py` | 23 | 是 | paths 工具 |
| `tests/unit_tests/test_preproc.py` | 134 | 是 | PreProcessor 注入 session 变量、bound skill 解析 |
| `tests/unit_tests/pipeline/test_chat_handler_logging.py` | 78 | 是 | Chat handler 日志相关回归 |
| `tests/integration_tests/box/test_box_integration.py` | 329 | **否** | 真实容器执行、超时、网络隔离 |
| `tests/integration_tests/box/test_box_mcp_integration.py` | 368 | **否** | Managed process、WS attach、shared-session 清理 |
### SDK 仓库
| 文件 | 行数 | CI 运行 | 覆盖范围 |
|------|------|---------|---------|
| `tests/box/test_backend_selection.py` | 255 | 是 | 显式 backend / local 模式探测顺序 / 配置变更触发 reselect |
| `tests/box/test_nsjail_backend.py` | 452 | 是 | nsjail 可用性、安装版 CLI vs 容器内 CLI、session、arg 构建、资源限制 |
| `tests/box/test_e2b_backend.py` | 482 | 是 | E2B SDK mock、session 生命周期、extra_mounts 同步 |
| `tests/box/test_skill_store.py` | 88 | 是 | zip preview/install、基础 file CRUD |
**总计**: 17 个测试文件, ~6,500 行测试代码; 其中 2 个集成测试(约 700 行)在 CI 中不运行。
> 较 2026-04-16 版增加:`test_skill_service.py`、`test_paths.py`、`test_preproc.py`、`test_chat_handler_logging.py` (LangBot)`test_backend_selection.py`、`test_e2b_backend.py`、`test_skill_store.py` (SDK)。`test_nsjail_backend.py` 增加 CLI 兼容性 case (commit `feed530`)。
---
## 2. 覆盖良好的区域
| 区域 | 质量 | 说明 |
|------|------|------|
| BoxRuntime session 管理 | 优秀 | session 复用、冲突检测、TTL 配置、消失 session 重建 |
| BoxService Profile 系统 | 优秀 | 4 个内置 Profile、locked/unlocked 字段、timeout clamp |
| BoxService host mount 安全 | 优秀 | allowed_mount_roots、disallowed_roots、shared host root |
| BoxService workspace quota | 优秀 | 前置/后置配额检查、超额清理 |
| BoxService 输出截断 | 优秀 | 短/精确边界/长输出、独立 stderr |
| BoxService 可观测性 | 优秀 | 状态报告、error ring buffer、buffer 上限 |
| BoxService session 模板 | 良好 | `resolve_box_session_id` + `build_skill_extra_mounts` 在 service / native / mcp 三处都有覆盖 |
| RPC client/server 协议 | 优秀 | execute/get_sessions/delete/create/conflict error |
| BoxRuntimeConnector | 良好 | local/remote 模式、Docker 平台、relay URL、心跳与重连回调 |
| BoxWorkspaceSession | 良好 | payload 构建、managed process 路径重写、stage host file |
| BoxHostMountMode.NONE | 良好 | 枚举校验、workdir 约束 |
| NsjailBackend | 良好 | 可用性、安装版 vs 容器内、session 生命周期、arg 构建、资源限制 |
| E2BBackend | 良好 | mock SDK、session/extra_mounts 同步 |
| Backend selection | 良好 | 显式 backend 优先级、local 探测顺序、配置变更触发 reselect |
| MCP Box 集成 | 良好 | config model、路径重写、payload、shared-session 多 process |
| Native tool loader | 良好 | 6 工具(exec/read/write/edit/glob/grep)、路径穿越拦截 |
| LocalAgent exec 流程 | 良好 | 完整 tool call 循环、流式、system prompt 注入、Tool Call 激活 |
| Skill 系统 | 良好 | 加载、Tool Call 激活、marker、路径解析、authoring CRUD、HTTP service |
---
## 3. 覆盖缺失的区域
### 3.1 零测试 / 严重不足
| 区域 | 源文件 | 影响 |
|------|--------|------|
| **`security.py`** | SDK `box/security.py` (52 行) | `validate_sandbox_security()` 无任何测试。阻止 `/etc`/`/proc`/Docker socket 等危险挂载的安全函数从未被验证 |
| **`policy.py`** | `pkg/box/policy.py` (98 行) | 三层安全策略无测试(也是死代码) |
| **`skill_store.py` 边缘场景** | SDK `box/skill_store.py` (647 行) vs 测试 88 行 | GitHub 安装路径、`source_subdir` / `target_suffix` 组合、损坏 zip、文件冲突等场景未覆盖 |
### 3.2 未测试的关键路径
| 区域 | 说明 |
|------|------|
| **Session TTL 过期** | 测试配置了 `session_ttl_sec` 但从未推进时间验证过期清理 |
| **并发 session 访问** | 无并发 exec / 并发创建 / race condition 测试 |
| **Container backend (Docker)** | 仅通过集成测试覆盖(CI 不运行),单元测试全用 FakeBackend |
| **E2B 真实 sandbox** | 单测全是 mock,未对接真实 E2B API |
| **BoxRuntime shutdown()** | 在 test cleanup 中调用但未验证行为 |
| **BoxServerHandler 错误路径** | 畸形请求、未知 action 类型 |
| **WS relay** | 仅在集成测试中覆盖(CI 不运行) |
| **NsjailBackend managed process** | 完全未测试 |
| **MCP stdio 完整生命周期** | 依赖安装 → 进程启动 → 健康检查 → 多 process 并发 → 重试 |
| **BoxService start/stop_managed_process** | 单 process 流转有单测,多 process 互不阻塞主要靠集成测试 |
| **重连指数退避** | connector 单测覆盖回调接线,未实际跑完整重连周期 |
### 3.3 边缘情况缺失
| 区域 | 说明 |
|------|------|
| BoxSpec 校验 | 无效 session_id 格式、超长命令、env 特殊字符 |
| BoxSpec.extra_mounts | 重复 mount_path、与 host_path 冲突、绝对 vs 相对路径 |
| BoxExecutionResult | 仅 COMPLETED 和 TIMED_OUT,无 ERROR 状态测试 |
| 多后端 fallback | local 模式探测顺序仅靠 mock,无真实 Docker 不可用 → nsjail 真机 fallback 测试 |
| Profile YAML 加载 | 测试用硬编码字符串,未从真实 config.yaml 加载 |
| INIT 配置变更触发 backend 重建 | 单测仅在初始化场景验证 |
---
## 4. 集成测试 vs CI 的差距
CI 仅运行 `tests/unit_tests/`,以下场景**从未在自动化中验证**:
- 真实容器的创建/执行/销毁
- 容器网络隔离(`--network none`
- 容器资源限制生效(cpus/memory/pids_limit
- Managed process 的 WS 双向 I/O
- 多 process 同 session 并发 I/O
- 孤儿容器清理
- Session 删除清理容器
- 进程退出检测
- E2B 真实 sandbox 行为
**建议**: 在 CI 中加一个可选的 Docker-in-Docker 集成测试 stage,至少覆盖核心执行路径(exec / MCP attach / session 销毁)。
+167
View File
@@ -0,0 +1,167 @@
# Box 系统 toB 商业化分析
> 更新日期: 2026-06-02
> 状态更新: 自部署社区版已具备发布条件(box 可选、降级完善、无迁移欠债);工具调用循环上限、配额遍历异步化、`host_path` 挂载白名单等已落地。剩余多租户 / 安全硬化项见 [SaaS 阻塞项清单](./box-issues.md)。
> 分支: `feat/sandbox` (LangBot + langbot-plugin-sdk)
---
## 1. 现有优势
| 能力 | toB 价值 | 代码位置 |
|------|---------|---------|
| **沙箱隔离执行** | 企业安全运行不受信代码的基础能力 | SDK `box/backend.py` |
| **多后端支持** | 适配不同企业容器基础设施 (Podman/Docker/nsjail/E2B) | SDK `box/runtime.py` `_select_backend()` |
| **E2B 云沙箱** | SaaS / 无 Docker 部署的兜底执行环境 | SDK `box/e2b_backend.py` |
| **连接自愈** | 心跳 + 自动重连,单点 Box runtime 故障可恢复 | `pkg/box/connector.py` `_heartbeat_loop`, `pkg/box/service.py` `_reconnect_loop` |
| **Profile + locked 字段** | 运维锁定安全边界,LLM/用户无法绕过 | `pkg/box/service.py`, SDK `box/models.py` |
| **资源限制** | CPU/内存/PID 数限制防止资源滥用 | SDK `backend.py` `--cpus/--memory/--pids-limit` |
| **Workspace quota** | 磁盘用量控制 | `pkg/box/service.py` `_enforce_workspace_quota` |
| **静默降级** | Box 不可用不影响其他功能,降低部署门槛 | `pkg/box/service.py:78` `_available=False` |
| **孤儿容器清理** | 防止泄漏的容器持续占用资源 | SDK `backend.py` `cleanup_orphaned_containers` |
| **网络隔离** | `--network none` 防止数据外泄 | SDK `backend.py` start_session |
| **只读根文件系统** | `--read-only` 防止容器被持久篡改 | SDK `backend.py` start_session |
| **Host path 白名单** | `allowed_host_mount_roots` 限制可挂载目录 | `pkg/box/service.py` `_validate_host_mount` |
---
## 2. toB 差距分析
### 2.1 安全与合规
| 维度 | 现状 | toB 要求 | 优先级 |
|------|------|---------|--------|
| **WS relay 认证** | 无认证,任何人可 attach | 至少 token 认证 | **P0** |
| **安全策略** | policy.py 是死代码,实际无细粒度控制 | 工具级 allow/deny、沙箱模式控制 | **P0** |
| **审计日志** | 仅内存中 50 条 `_recent_errors` | 持久化审计:谁何时执行了什么、结果如何 | **P0** |
| **Host path 校验** | 黑名单策略,`/` 未拦截 | 白名单策略,默认拒绝 | **P1** |
| **数据驻留** | 无控制 | GDPR / 等保要求的数据隔离 | **P2** |
### 2.2 多租户
| 维度 | 现状 | toB 要求 | 优先级 |
|------|------|---------|--------|
| **租户隔离** | 无租户概念 | BoxSpec/Profile 绑定 tenant_id | **P0** |
| **RBAC** | 仅 token 认证 | admin/operator/viewer 角色权限 | **P0** |
| **资源配额** | 单一 workspace quota | 每租户 CPU 时间/内存/并发/执行次数配额 | **P1** |
| **Session 隔离** | 所有 session 共享 dict | 按租户分区,互不可见 | **P1** |
### 2.3 可靠性
| 维度 | 现状 | toB 要求 | 优先级 |
|------|------|---------|--------|
| **连接恢复** | 已实现:20s 心跳 + `_reconnect_loop` 指数退避 | 已满足基本要求 | 已有 |
| **Session 清理** | 机会性(仅新建时触发) | 定时清理 + 独立 reaper | **P1** |
| **水平扩展** | 单 Box Runtime 实例 | 多实例负载均衡(按 tenant 路由) | **P1** |
| **优雅降级** | 已有(_available=False | 已满足基本要求 | 已有 |
| **Backend 自愈** | 已实现:`get_status` 时若 backend 不可用会重新选择 | 已满足基本要求 | 已有 |
### 2.4 可观测性
| 维度 | 现状 | toB 要求 | 优先级 |
|------|------|---------|--------|
| **监控指标** | 无 Prometheus metrics | session 数/执行延迟/资源用量/错误率 | **P1** |
| **结构化日志** | Python logging, 无结构化 | JSON 格式日志,含 trace_id/tenant_id | **P1** |
| **前端面板** | 监控页接入 `/api/v1/box/status`backend 名 + 活跃 session 数);`sessions` / `errors` 仍未接入 | 完整状态面板 + 历史错误/审计列表 | **P2** |
---
## 3. SaaS 部署架构建议
### 3.1 方案 A: 共享 Box Runtime Pool (快速上线)
```
LangBot Instance ──> Box Runtime (共享)
├─ tenant_id 标签隔离
├─ Redis 配额计数器
└─ Container labels: langbot.tenant_id=xxx
```
- **优点**: 改动最小,加 tenant_id 到 BoxSpec/labels 即可
- **缺点**: 容器引擎共享,安全隔离弱
### 3.2 方案 B: 每租户 K8s Namespace + gVisor (推荐中期)
```
LangBot ──> K8s API
├─ namespace: tenant-xxx
│ ├─ RuntimeClass: gVisor (runsc)
│ ├─ ResourceQuota
│ └─ NetworkPolicy
└─ namespace: tenant-yyy
└─ ...
```
- **优点**: 强隔离(namespace + gVisor),原生 K8s 配额
- **缺点**: 需要重写 backend 为 K8s Job,部署复杂度高
### 3.3 方案 C: K8s Job 直接编排 (长期)
```
LangBot ──> K8s Job per execution
├─ 每次执行创建 Job
├─ Pod Security Standards
├─ 自动调度和资源分配
└─ Job TTL Controller 自动清理
```
- **优点**: 最强隔离,天然水平扩展
- **缺点**: 冷启动延迟,架构重写
**推荐演进路径**: A → B → C
---
## 4. 配额体系建议
### 三层配额
| 层 | 实现 | 作用 |
|----|------|------|
| **内核层** | Docker `--cpus`/`--memory`/`--storage-opt` | 硬性资源上限,不可绕过 |
| **应用层** | Redis 原子计数器 | 并发 session 数/执行次数/CPU 时间预算 |
| **计费层** | 月度聚合 | 按租户计费(session-hours/execution-count |
### Profile 与套餐映射
| 套餐 | Profile | locked 字段 | 配额 |
|------|---------|------------|------|
| Free | `offline_readonly` | network, host_path_mode, rootfs | 10 exec/天, 0.5 CPU, 256MB |
| Pro | `default` | (无) | 100 exec/天, 1 CPU, 512MB |
| Enterprise | `network_extended` | (按需) | 无限, 2 CPU, 1GB, 自定义镜像 |
### TOCTOU 配额修复
当前 `_enforce_workspace_quota` 的 TOCTOU 问题可通过两种方式解决:
1. **预留式配额** (应用层): Redis `INCRBY` 预扣额度 → 执行 → 成功则扣减,失败则回滚
2. **内核级限制** (Docker): `--storage-opt size=500m` 直接限制容器可写层大小
---
## 5. 优先实施路线
### Phase 1 (2-4 周): 安全基线
- [ ] WS relay 加 token 认证
- [ ] 接入或删除 policy.py
- [x] ~~Box 加重连和心跳~~(已完成,见 [box-issues.md 已解决](./box-issues.md)
- [ ] 审计日志持久化(至少写文件/数据库)
- [ ] `security.py``/` 拦截,考虑白名单
- [ ] INIT 与 backend 初始化顺序整理(避免 backend 在配置到达前实例化)
### Phase 2 (4-8 周): 多租户基础
- [ ] BoxSpec 加 `tenant_id` 字段
- [ ] 容器 labels 加 tenant 标识
- [ ] Redis 配额计数器(并发/执行次数/时间)
- [ ] RBAC 基础框架
- [ ] 定时 session reaper
### Phase 3 (8-16 周): 生产就绪
- [ ] Prometheus metrics exporter
- [ ] 前端 Box 状态面板
- [ ] K8s backend 支持 (方案 B)
- [ ] 结构化日志 (JSON, trace_id)
- [ ] 水平扩展支持
+222
View File
@@ -0,0 +1,222 @@
# Box Runtime vs Plugin Runtime: 连接架构对比
> 更新日期: 2026-06-02
> 状态更新: 自部署社区版已具备发布条件(box 可选、降级完善、无迁移欠债);工具调用循环上限、配额遍历异步化、`host_path` 挂载白名单等已落地。剩余多租户 / 安全硬化项见 [SaaS 阻塞项清单](./box-issues.md)。
> 分支: `feat/sandbox` (LangBot + langbot-plugin-sdk)
---
## 1. 总体差异
| 维度 | Plugin Runtime | Box Runtime |
|------|---------------|-------------|
| **继承关系** | `PluginRuntimeConnector(ManagedRuntimeConnector)` | `BoxRuntimeConnector`(独立类) |
| **传输分支** | 3 条 (Docker/WS, Win32/subprocess+WS, Unix/stdio) | 3 条 (本地 stdio, Win32/subprocess+WS, 远程 WS) |
| **心跳** | 20s ping loop | 20s ping loop`_heartbeat_loop` |
| **重连** | WS 模式: sleep 3s → re-initialize | 由 BoxService `_reconnect_loop` 处理,指数退避 |
| **Handler 类型** | `RuntimeConnectionHandler` (1132 行, 25+ action) | 基础 `Handler` + `BoxServerHandler`SDK 端 25 action |
| **Client 抽象** | Handler 即 API | 独立 `ActionRPCBoxClient` 封装 Handler |
| **启用/禁用** | `is_enable_plugin` 开关 | 无开关(可用/不可用由初始化结果决定) |
| **初始化失败** | 异常上抛 | 静默降级 `_available=False` |
| **Shutdown** | 直接杀进程 | RPC SHUTDOWN → 清理容器 → 再杀进程 |
---
## 2. 传输决策
### Plugin: 3-路决策
```python
# pkg/plugin/connector.py:106-165
if get_platform() == 'docker' or use_websocket_to_connect_plugin_runtime():
# Docker/WS → ws://langbot_plugin_runtime:5400/control/ws
elif get_platform() == 'win32':
# Windows → 起子进程(无 pipe) + ws://localhost:5400/control/ws
else:
# Unix/Mac → StdioClientController(python -m langbot_plugin.cli rt -s)
```
### Box: 3-路决策
```python
# pkg/box/connector.py
if self._uses_websocket():
if platform.get_platform() == 'win32' and not self.configured_runtime_url:
await self._start_subprocess_then_ws() # subprocess + ws://localhost:5410/rpc/ws
else:
await self._connect_remote_ws() # ws://{host}:5410/rpc/ws
else:
await self._start_local_stdio() # StdioClientController
```
> 历史:2026-04-16 版本本文档曾把 Box 描述为 2 路决策(缺 Windows 分支)。现已对齐 Plugin 的 3 路设计。
### 决策矩阵
| 环境 | Plugin | Box |
|------|--------|-----|
| Docker | WS → `:5400` | WS → `:5410/rpc/ws` |
| `--standalone-box` | N/A | WS → `localhost:5410/rpc/ws` |
| Windows 非 Docker | subprocess + WS (`:5400`) | subprocess + WS (`localhost:5410/rpc/ws`) |
| Unix/Mac 非 Docker | stdio | stdio |
| 手动配置 URL | 通过配置项 | WS → 用户配置的 URL |
---
## 3. 连接建立
### 同步模式差异
**Plugin**: `new_connection_callback` 内直接 ping + await handler_task`initialize()` 通过 `create_task()` 异步启动,不阻塞等待连接。
**Box**: 使用 `asyncio.Event` + `wait_for(timeout=30s)` 模式,`initialize()` 同步等待连接成功或超时。
### Box stdio 路径
```
connector._start_local_stdio()
├─ connected = asyncio.Event()
├─ ctrl = StdioClientController(python, ['-m', 'langbot_plugin.cli.__init__', 'box', '-s', '--ws-control-port', N])
├─ _ctrl_task = create_task(ctrl.run(callback))
│ callback:
│ handler = Handler(connection) ← 基础 Handler, 无 disconnect_callback
│ client.set_handler(handler)
│ _handler_task = create_task(handler.run())
│ call_action(PING, {}) ← 握手, timeout=15s
│ connected.set() ← 通知外层
│ await _handler_task ← 阻塞直到断开
└─ await wait_for(connected.wait(), 30s) ← 同步等待
```
### Plugin stdio 路径
```
connector.initialize()
├─ ctrl = StdioClientController(python, ['-m', 'langbot_plugin.cli', 'rt', '-s'])
├─ task = ctrl.run(callback)
│ callback:
│ disconnect_callback:
│ [WS] → runtime_disconnect_callback → 重连
│ [stdio] → 仅日志, 不重连
│ handler = RuntimeConnectionHandler(conn, disconnect_cb, ap)
│ create_task(handler.run())
│ handler.ping() ← 握手, timeout=10s
│ await handler_task ← 阻塞直到断开
├─ create_task(heartbeat_loop()) ← 20s ping loop
└─ create_task(task) ← 不等待连接
```
---
## 4. 心跳与重连
### 心跳
| 维度 | Plugin | Box |
|------|--------|-----|
| 有心跳? | 是 | 是(`connector.py` `_heartbeat_loop` |
| 间隔 | 20s | 20s |
| 失败处理 | 仅 DEBUG 日志,不触发重连 | 仅 DEBUG 日志,依赖 connection close 触发重连 |
| 生命周期 | 整个应用生命周期 | 连接建立后启动;`dispose()` 时 cancel |
### 重连
| 维度 | Plugin | Box |
|------|--------|-----|
| Docker/WS 断开 | `runtime_disconnect_callback` → sleep 3s → re-initialize | `runtime_disconnect_callback``BoxService._reconnect_loop()`(指数退避) |
| WS 连接失败 | 同上 | 同上;初次失败时 `_available=False`,重连成功后恢复 |
| stdio 断开 | 仅日志,不重连 | 接同样回调;stdio 重连需重新 fork 子进程 |
| 重连退避 | 固定 3s,无 backoff | 指数退避 |
> 历史:2026-04-16 版本本文档曾把心跳与重连标记为 Box 缺失。这两项已在 commit `2dfd9d5d` / `c6882cf` / `5029d9c` 等修复(详见 [box-issues.md 已解决](./box-issues.md))。
---
## 5. 共享 IO 层
两者复用同一套 SDK IO 基础设施:
```
Handler ← ABC (runtime/io/handler.py)
├── RuntimeConnectionHandler (Plugin 用, LangBot 侧)
├── ControlConnectionHandler (Plugin 用, SDK 侧)
├── BoxServerHandler (Box 用, SDK 侧)
└── 匿名 Handler 实例 (Box 用, LangBot 侧)
Connection ← ABC
├── StdioConnection (stdio: 16KB chunks, 应用层分帧协议)
└── WebSocketConnection (WS: 64KB chunks, 原生 WS 分帧)
Controller ← ABC
├── StdioClientController (fork 子进程, pipe stdin/stdout)
├── StdioServerController (接管当前进程 stdin/stdout)
├── WebSocketClientController (连接 WS 服务端)
└── WebSocketServerController (监听 WS 端口)
```
共享的核心机制:
- `call_action()` / `call_action_generator()` — RPC 调用/流式调用
- `ActionRequest` / `ActionResponse` — 请求/响应协议
- `seq_id` 关联 — 并发请求复用单连接
- `CommonAction.PING` — 两者都用于初始握手
- 文件传输 (`send_file`) — Plugin 用,Box 不用
---
## 6. 端口方案
| 服务 | Plugin | Box |
|------|--------|-----|
| Action RPC (stdio) | stdin/stdout | stdin/stdout |
| Action RPC (WS) | `:5400` | `:5410/rpc/ws` |
| 辅助服务 | debug WS `:5401` | managed process WS relay `:5410/v1/sessions/{id}/managed-process/ws` |
**Box 特点**: 单端口 aiohttp 服务(默认 5410),通过路径区分 Action RPC 和 managed process relay。即使在 stdio 模式,也在 `:5410` 启动 aiohttp 用于 managed process attach。Plugin 在 stdio 模式不开额外端口。
---
## 7. 销毁对比
### Plugin
```python
dispose():
if stdio: ctrl.process.terminate()
_dispose_subprocess() # Windows 子进程
heartbeat_task.cancel()
```
### Box
```python
connector.dispose():
_handler_task.cancel()
_ctrl_task.cancel()
_subprocess.terminate()
service.dispose():
connector.dispose()
loop.create_task(client.shutdown()) # RPC SHUTDOWN → 清理所有容器
```
Box 的 RPC SHUTDOWN 确保容器被正确停止,不会成为孤儿。Plugin 直接杀进程。
---
## 8. 改进建议
### P0
1. **两者都加 WS 认证**: 至少 token 认证(INIT 时下发,连接时校验)
### P1
2. **考虑 Box 继承 ManagedRuntimeConnector**: 复用 `_start_runtime_subprocess` / `_wait_until_ready` / `_dispose_subprocess`,减少重复代码
3. **Plugin 重连加退避**: 固定 3s 无 backoff 可能造成日志洪水,建议向 Box 的指数退避看齐
4. **统一连接管理模式**: Event-based (Box) vs direct-await (Plugin),考虑收敛为一种
### 已完成(自上一轮)
- ~~Box 加重连~~commit `2dfd9d5d`
- ~~Box 加心跳~~20s loop 与 Plugin 一致)
- ~~Box 加 Windows 支持~~commit `120817a` / `fafb7a4`
+12 -13
View File
@@ -1,6 +1,6 @@
[project] [project]
name = "langbot" name = "langbot"
version = "4.9.7" version = "4.10.1"
description = "Production-grade platform for building agentic IM bots" description = "Production-grade platform for building agentic IM bots"
readme = "README.md" readme = "README.md"
license-files = ["LICENSE"] license-files = ["LICENSE"]
@@ -8,7 +8,7 @@ requires-python = ">=3.11,<4.0"
dependencies = [ dependencies = [
"aiocqhttp>=1.4.4", "aiocqhttp>=1.4.4",
"aiofiles>=24.1.0", "aiofiles>=24.1.0",
"aiohttp>=3.13.4", "aiohttp>=3.14.0",
"aioshutil>=1.5", "aioshutil>=1.5",
"aiosqlite>=0.21.0", "aiosqlite>=0.21.0",
"anthropic>=0.51.0", "anthropic>=0.51.0",
@@ -31,27 +31,27 @@ dependencies = [
"psutil>=7.0.0", "psutil>=7.0.0",
"pycryptodome>=3.22.0", "pycryptodome>=3.22.0",
"pydantic>2.0", "pydantic>2.0",
"pyjwt>=2.10.1", "pyjwt>=2.12.0",
"python-telegram-bot>=22.0", "python-telegram-bot>=22.0",
"pyyaml>=6.0.2", "pyyaml>=6.0.2",
"qq-botpy-rc>=1.2.1.6", "qq-botpy-rc>=1.2.1.6",
"qrcode>=7.4", "qrcode>=7.4",
"quart>=0.20.0", "quart>=0.20.0",
"quart-cors>=0.8.0", "quart-cors>=0.8.0",
"requests>=2.32.3", "requests>=2.33.0",
"slack-sdk>=3.35.0", "slack-sdk>=3.35.0",
"alembic>=1.15.0", "alembic>=1.15.0",
"sqlalchemy[asyncio]>=2.0.40", "sqlalchemy[asyncio]>=2.0.40",
"sqlmodel>=0.0.24", "sqlmodel>=0.0.24",
"telegramify-markdown>=0.5.1", "telegramify-markdown>=0.5.1",
"tiktoken>=0.9.0", "tiktoken>=0.9.0",
"urllib3>=2.4.0", "urllib3>=2.7.0",
"websockets>=15.0.1", "websockets>=15.0.1",
"python-socks>=2.7.1", # dingtalk missing dependency "python-socks>=2.7.1", # dingtalk missing dependency
"pip>=25.1.1", "pip>=26.1",
"ruff>=0.11.9", "ruff>=0.11.9",
"pre-commit>=4.2.0", "pre-commit>=4.2.0",
"uv>=0.11.6", "uv>=0.11.15",
"mypy>=1.16.0", "mypy>=1.16.0",
"PyPDF2>=3.0.1", "PyPDF2>=3.0.1",
"python-docx>=1.1.0", "python-docx>=1.1.0",
@@ -62,15 +62,15 @@ dependencies = [
"ebooklib>=0.18", "ebooklib>=0.18",
"html2text>=2024.2.26", "html2text>=2024.2.26",
"langchain>=0.2.0", "langchain>=0.2.0",
"langchain-core>=1.2.28", "langchain-core>=1.3.3",
"langsmith>=0.7.31", "langsmith>=0.8.0",
"python-multipart>=0.0.26", "python-multipart>=0.0.27",
"Mako>=1.3.11", "Mako>=1.3.12",
"langchain-text-splitters>=1.1.2", "langchain-text-splitters>=1.1.2",
"chromadb>=1.0.0,<2.0.0", "chromadb>=1.0.0,<2.0.0",
"qdrant-client (>=1.15.1,<2.0.0)", "qdrant-client (>=1.15.1,<2.0.0)",
"pyseekdb==1.1.0.post3", "pyseekdb==1.1.0.post3",
"langbot-plugin==0.3.11", "langbot-plugin==0.4.2",
"asyncpg>=0.30.0", "asyncpg>=0.30.0",
"line-bot-sdk>=3.19.0", "line-bot-sdk>=3.19.0",
"matrix-nio>=0.25.2", "matrix-nio>=0.25.2",
@@ -223,4 +223,3 @@ skip-magic-trailing-comma = false
# Like Black, automatically detect the appropriate line ending. # Like Black, automatically detect the appropriate line ending.
line-ending = "auto" line-ending = "auto"
-821
View File
@@ -1,821 +0,0 @@
"""Generate the DingTalk human-input card template JSON.
The output is wrapped in the {editorData, widgetInfo, type, mode} envelope
the DingTalk card builder expects on import. editorData is itself a JSON
string (NOT a nested object), matching real exports from the builder.
Run from the repo root: python scripts/build_dingtalk_card_template.py
"""
from __future__ import annotations
import json
from pathlib import Path
OUTPUT = Path('src/langbot/templates/dingtalk_human_input_card.json')
def markdown_block(node_id, variable='content'):
"""A MarkdownBlock whose content is bound to a global variable.
Critical: `content.varType: "markdown"` must be set, otherwise DingTalk
silently fails to render the bound variable (the card body stays blank
even though the variable is supplied via cardParamMap). The working
reference template in I:\\下载\\dingtalk_1782055283543.json confirms
this — its MarkdownBlock has the same varType marker.
isStreaming is left `false` because the adapter writes the variable via
`update_card_data` (the full-card PUT endpoint), not the streaming
`card/streaming` endpoint. Setting `isStreaming: true` here conflicts
with that path and can suppress the rendered body.
"""
return {
'componentName': 'MarkdownBlock',
'id': node_id,
'props': {
'mdVer': 0,
'icon': {'type': 'icon', 'icon': '', 'iconType': 'emoji'},
'content': {
'variable': variable,
'variableType': 'global',
'type': 'variableValue',
'varType': 'markdown',
},
'visible': {
'type': 'dynamicVisible',
'value': True,
'valueType': 'fixed',
'condition': {'op': 'and', 'conditions': []},
},
'isStreaming': False,
'enableLinkStatPoint': False,
'linkStatPoint': {
'type': 'dynamicString',
'content': 'Page_InteractiveCard__Click_markdownOpenlink',
'i18n': False,
},
'linkStatPointParams': [],
'marginTop': 6,
'marginBottom': 6,
'marginLeft': 12,
'marginRight': 12,
},
'title': 'AI 流式富文本',
'hidden': False,
'isLocked': False,
'condition': True,
'conditionGroup': '',
}
def text_block(
node_id,
text,
*,
bold=False,
gravity='left',
font_size=14,
line_height=22,
max_lines=20,
ml=12,
mr=12,
mt=4,
mb=4,
color_token='common_level1_base_color',
style_token='common_body_text_style',
):
return {
'componentName': 'BaseText',
'id': node_id,
'props': {
'text': {'i18n': False, 'type': 'dynamicString', 'content': text},
'hoverText': {'type': 'dynamicString', 'content': '', 'i18n': False},
'iconType': 'iconCode',
'iconFont': {'type': 'icon', 'icon': '', 'iconType': 'ddIcon'},
'icon': {
'type': 'dynamicLink',
'value': '',
'valueType': 'fixed',
'variable': '',
'variableType': 'global',
},
'darkIcon': {
'type': 'dynamicLink',
'value': '',
'valueType': 'fixed',
'variable': '',
'variableType': 'global',
},
'autoWidth': False,
'maxWidth': {
'type': 'dynamicNumber',
'valueType': 'fixed',
'value': 0,
'variable': '',
'variableType': 'global',
},
'fixedWidth': {
'type': 'dynamicNumber',
'valueType': 'fixed',
'value': 0,
'variable': '',
'variableType': 'global',
},
'marginLeft': ml,
'marginRight': mr,
'marginTop': mt,
'marginBottom': mb,
'fontColorType': 'Standard',
'enableHighlight': False,
'maxLine': {
'type': 'dynamicNumber',
'valueType': 'fixed',
'value': max_lines,
'variable': '',
'variableType': 'global',
},
'color': {
'type': 'dynamicColor',
'valueType': 'fixed',
'value': color_token,
'variable': '',
'variableType': 'global',
},
'customLightColor': {
'type': 'dynamicColor',
'valueType': 'fixed',
'value': '#35404b',
'variable': '',
'variableType': 'global',
},
'customDarkColor': {
'type': 'dynamicColor',
'valueType': 'fixed',
'value': '#f6f6f6',
'variable': '',
'variableType': 'global',
},
'gravity': gravity,
'fontSizeType': 'Standard',
'styleType': 'custom',
'styleToken': style_token,
'size': 'middle',
'customFontSize': font_size,
'customFontLineHeight': line_height,
'bold': bold,
'italic': False,
'strikeThrough': False,
'lineHeight': 'normal',
'visible': {
'type': 'dynamicVisible',
'value': True,
'valueType': 'fixed',
'condition': {'op': 'and', 'conditions': []},
},
'autoMaxWidth': False,
'innerOffset': 0,
'enableIcon': False,
'widthMode': 'match_parent',
'margin': -2,
},
'title': '基础文本',
'hidden': False,
'isLocked': False,
'condition': True,
'conditionGroup': '',
}
def button_group(node_id):
return {
'componentName': 'ButtonGroup',
'id': node_id,
'props': {
'dynamicButtons': {'type': 'variableValue', 'variableType': 'global', 'variable': 'btns'},
'marginLeft': 12,
'marginRight': 12,
'marginTop': 6,
'marginBottom': 12,
'visible': {
'type': 'dynamicVisible',
'value': True,
'valueType': 'fixed',
'condition': {'op': 'and', 'conditions': []},
},
'responsiveLayoutWidth': 350,
'buttonsSource': 'variable',
'fixedButtonIds': [],
'fixedButtons': [],
'enableResponsiveLayout': False,
'matchContent': False,
'buttonSpacing': 8,
'margin': -2,
'innerOffset': 0,
},
'title': '按钮组',
'hidden': False,
'isLocked': False,
'condition': True,
'conditionGroup': '',
}
def avatar(node_id, *, name='LangBot', image_variable='bot_avatar'):
"""Avatar component in `userInfo` mode — renders the bot's avatar
image and nickname as a header row above the response content.
Mirrors the layout from `I:\\下载\\dingtalk_1782120006374.json` where
Avatar sits at the top of the done-state AICardContent.
`imageUrl` is bound to a global variable (default `bot_avatar`) so
the adapter can populate it at runtime with a DingTalk media id
(``@xxx``) obtained from the /media/upload endpoint. DingTalk's
Avatar.imageUrl resolver rejects external URLs — it only accepts
DingTalk-hosted media ids, so this binding is the only path to
a custom avatar.
"""
return {
'componentName': 'Avatar',
'id': node_id,
'props': {
'imageUrl': {
'value': '',
'valueType': 'variable',
'type': 'dynamicImage',
'variable': image_variable,
'variableType': 'global',
},
'name': {'i18n': False, 'type': 'dynamicString', 'content': name},
'sizeType': 'Standard',
'size': 'extraSmall',
'customSize': 48,
'marginLeft': 12,
'marginRight': 12,
'marginTop': 6,
'marginBottom': 6,
'visible': {
'type': 'dynamicVisible',
'value': True,
'valueType': 'fixed',
'condition': {'op': 'and', 'conditions': []},
},
'mode': 'userInfo',
'margin': -2,
'innerOffset': 0,
},
'title': '头像',
'hidden': False,
'isLocked': False,
'condition': True,
'conditionGroup': '',
}
def build_editor_data():
component_names = [
'AIPending',
'AICardStatusContainer',
'BaseText',
'AICardContent',
'AICardContainer',
'ButtonGroup',
'MarkdownBlock',
'Avatar',
]
components_map = [
{
'package': '@ali/dxComponent',
'version': '1.0.0',
'exportName': n,
'main': './src/index.tsx',
'destructuring': False,
'subName': '',
'componentName': n,
}
for n in component_names
]
pending_state = {
'componentName': 'AICardStatusContainer',
'id': 'node_status_pending',
'props': {
'status': 1,
'marginLeft': 0,
'marginRight': 0,
'marginTop': 0,
'marginBottom': 0,
'enableExtend': False,
'autoFoldConfig': {
'needFold': True,
'heightLimit': 480,
'foldStatusLocalDataKey': '_cardFoldStatusLocalDataKey',
},
'innerOffset': 0,
'enableCollapse': False,
'margin': -2,
},
'title': '处理中状态',
'hidden': False,
'isLocked': False,
'condition': True,
'conditionGroup': '',
'children': [
{
'componentName': 'AIPending',
'id': 'node_pending_inner',
'props': {
'marginLeft': 0,
'marginRight': 0,
'marginTop': 0,
'marginBottom': 0,
'pendingTip': {'type': 'dynamicString', 'content': '处理中...', 'i18n': False},
'style': 'embed',
'hideIcon': False,
},
'hidden': False,
'title': '',
'isLocked': False,
'condition': True,
'conditionGroup': '',
}
],
}
done_state = {
'componentName': 'AICardStatusContainer',
'id': 'node_status_done',
'props': {
'status': 3,
'marginLeft': 0,
'marginRight': 0,
'marginTop': 0,
'marginBottom': 0,
'enableExtend': False,
'autoFoldConfig': {
'needFold': True,
'heightLimit': 480,
'foldStatusLocalDataKey': '_cardFoldStatusLocalDataKey',
},
'innerOffset': 0,
'enableCollapse': False,
'margin': -2,
},
'title': '完成状态',
'hidden': False,
'isLocked': False,
'condition': True,
'conditionGroup': '',
'children': [
{
'componentName': 'AICardContent',
'id': 'node_done_content',
'props': {
'marginLeft': 0,
'marginRight': 0,
'marginTop': 0,
'marginBottom': 0,
'visible': {
'type': 'dynamicVisible',
'value': True,
'valueType': 'fixed',
'condition': {'op': 'and', 'conditions': []},
},
'innerOffset': 0,
'disabledWhileForward': False,
'statPoint': {'type': 'dynamicString', 'content': '', 'i18n': False},
'statPointParams': [
{'type': 'fixed', 'variable': '', 'value': '', 'name': '', 'variableType': 'global', 'id': '1'}
],
'margin': -2,
'transformToEventChain': False,
'enableStatPoint': False,
},
'hidden': False,
'title': '',
'isLocked': False,
'condition': True,
'conditionGroup': '',
'children': [
avatar('node_avatar', name='LangBot'),
markdown_block('node_text_content', variable='content'),
button_group('node_btn_group'),
],
}
],
}
failed_state = {
'componentName': 'AICardStatusContainer',
'id': 'node_status_failed',
'props': {
'status': 5,
'marginLeft': 0,
'marginRight': 0,
'marginTop': 0,
'marginBottom': 0,
'enableExtend': False,
'autoFoldConfig': {
'needFold': True,
'heightLimit': 480,
'foldStatusLocalDataKey': '_cardFoldStatusLocalDataKey',
},
'innerOffset': 0,
'enableCollapse': False,
'margin': -2,
},
'title': '失败状态',
'hidden': False,
'isLocked': False,
'condition': True,
'conditionGroup': '',
'children': [
{
'componentName': 'AICardContent',
'id': 'node_failed_content',
'props': {
'visible': {
'type': 'dynamicVisible',
'value': True,
'valueType': 'fixed',
'condition': {'op': 'and', 'conditions': []},
},
'marginLeft': 0,
'marginRight': 0,
'marginTop': 0,
'marginBottom': 0,
'innerOffset': 0,
'disabledWhileForward': False,
'statPoint': {'type': 'dynamicString', 'content': '', 'i18n': False},
'statPointParams': [
{'type': 'fixed', 'variable': '', 'value': '', 'name': '', 'variableType': 'global', 'id': '1'}
],
'margin': -2,
'transformToEventChain': False,
'enableStatPoint': False,
},
'hidden': False,
'title': '',
'isLocked': False,
'condition': True,
'conditionGroup': '',
'children': [
text_block(
'node_failed_text',
'操作失败,请稍后重试。',
gravity='center',
mt=10,
mb=10,
ml=10,
mr=10,
max_lines=2,
font_size=15,
)
],
}
],
}
# Empty containers for flowStatus=2 (writing) and flowStatus=4 (doing).
# AICardContainer expects placeholders to exist for every enabled state;
# without them, the renderer can refuse to advance to flowStatus=3 (done)
# and the card body stays empty. They render nothing visible because
# they have no content children, but their presence satisfies the
# state-machine validation.
def _empty_status_container(node_id, status):
return {
'componentName': 'AICardStatusContainer',
'id': node_id,
'props': {
'status': status,
'marginLeft': 0,
'marginRight': 0,
'marginTop': 0,
'marginBottom': 0,
'enableExtend': False,
'autoFoldConfig': {
'needFold': True,
'heightLimit': 480,
'foldStatusLocalDataKey': '_cardFoldStatusLocalDataKey',
},
'innerOffset': 0,
'enableCollapse': False,
'margin': -2,
},
'title': f'状态{status}占位',
'hidden': False,
'isLocked': False,
'condition': True,
'conditionGroup': '',
'children': [
{
'componentName': 'AICardContent',
'id': f'{node_id}_content',
'props': {
'marginLeft': 0,
'marginRight': 0,
'marginTop': 0,
'marginBottom': 0,
'visible': {
'type': 'dynamicVisible',
'value': True,
'valueType': 'fixed',
'condition': {'op': 'and', 'conditions': []},
},
'innerOffset': 0,
'disabledWhileForward': False,
'statPoint': {'type': 'dynamicString', 'content': '', 'i18n': False},
'statPointParams': [
{
'type': 'fixed',
'variable': '',
'value': '',
'name': '',
'variableType': 'global',
'id': '1',
}
],
'margin': -2,
'transformToEventChain': False,
'enableStatPoint': False,
},
'hidden': False,
'title': '',
'isLocked': False,
'condition': True,
'conditionGroup': '',
'children': [],
}
],
}
writing_state = _empty_status_container('node_status_writing', 2)
doing_state = _empty_status_container('node_status_doing', 4)
root = {
'componentName': 'AICardContainer',
'id': 'node_root',
'props': {
'marginLeft': 0,
'marginRight': 0,
'marginTop': 0,
'marginBottom': 0,
'enablePending': True,
# writing/doing must be enabled so AICardContainer recognises
# flowStatus transitions through 2/4 — without this, the
# working reference template (I:\\下载\\dingtalk_1782055283543.json)
# never reaches the done state and the body stays empty.
'enableWriting': True,
'enableDoing': True,
'enableFailed': True,
'summaryContent': {'type': 'variableValue', 'variableType': 'global', 'variable': ''},
'enableTitle': False,
'flowStatusVar': {'type': 'variableValue', 'variableType': 'global', 'variable': 'flowStatus'},
'operationPenalType': 'custom',
'enableFlowAbort': True,
'innerOffset': 0,
'enableGradientBorder': True,
'cardSizeMode': 'adaptive',
'cardSizeHeightMode': 'adaptive',
'cardSizeWidthMode': 'adaptive',
'cardSizeHeight': {
'type': 'dynamicNumber',
'valueType': 'fixed',
'value': 226,
'variable': '',
'variableType': 'global',
},
'hasBackground': False,
'backgroundType': 'Standard',
'standardBackgroundColor': 'gray',
'backgroundColor': '#F6F6F6',
'darkModeBackgroundColor': '#3C3C3C',
'enableEngineUpgrade': False,
'enableExposeStatPoint': False,
'enableDebugTool': False,
},
'hidden': False,
'title': '',
'isLocked': False,
'condition': True,
'conditionGroup': '',
'children': [pending_state, writing_state, doing_state, done_state, failed_state],
}
btns_var = {
'name': 'btns',
'private': False,
'type': 'buttonGroup',
'id': 'btns',
'description': '动态按钮列表(Dify actions',
'editorVarType': 'variables',
'disabled': False,
'schema': [
{
'id': 'btns.text',
'type': 'string',
'name': 'text',
'private': False,
'editorVarType': 'variables',
'disabled': True,
'description': '按钮文案',
},
{
'id': 'btns.color',
'type': 'string',
'name': 'color',
'private': False,
'editorVarType': 'variables',
'disabled': True,
'description': '按钮颜色',
},
{
'id': 'btns.status',
'type': 'string',
'name': 'status',
'private': False,
'editorVarType': 'variables',
'disabled': True,
'description': '按钮状态',
},
{
'id': 'btns.event',
'type': 'dynamicEvent',
'name': 'event',
'private': False,
'editorVarType': 'variables',
'disabled': True,
'description': '按钮点击事件',
'schema': [
{
'id': 'btns.type',
'type': 'string',
'name': 'type',
'private': False,
'editorVarType': 'variables',
'disabled': True,
'description': '事件类型:openLink / sendCardRequest',
},
{
'id': 'btns.params',
'type': 'object',
'name': 'params',
'private': False,
'editorVarType': 'variables',
'disabled': True,
'description': '事件参数',
'schema': [
{
'id': 'btns.url',
'type': 'string',
'name': 'url',
'private': False,
'editorVarType': 'variables',
'disabled': True,
'description': '点击跳转链接(type=openLink',
},
{
'id': 'btns.actionId',
'type': 'string',
'name': 'actionId',
'private': False,
'editorVarType': 'variables',
'disabled': True,
'description': '回传请求 idtype=sendCardRequest',
},
{
'id': 'btns.params',
'type': 'object',
'name': 'params',
'private': False,
'editorVarType': 'variables',
'disabled': True,
'description': '回传请求参数(type=sendCardRequest',
},
],
},
],
},
],
}
return {
'schemaVersion': '3.0.0',
'schema': {
# Match the working reference template — leaving config null lets
# DingTalk pick defaults. Explicit `streaming_mode: true` would
# make the renderer wait for chunks on the streaming endpoint
# (PUT /v1.0/card/streaming), which our adapter does NOT use —
# it pushes content via update_card_data, so streaming_mode=true
# leaves the body empty.
'config': None,
'componentsMap': components_map,
'componentsTree': [root],
'i18n': {},
'version': '1.0.0',
},
'mockData': {
'cardData': {
'flowStatus': 3,
'content': '请审核以下报销申请:\n\n- 申请人:张三\n- 金额:¥1,200\n- 类别:差旅',
'btns': [
{
'text': '通过',
'color': 'blue',
'status': 'normal',
'event': {
'type': 'sendCardRequest',
'params': {'actionId': 'approve', 'params': {'action_id': 'approve'}},
},
},
{
'text': '驳回',
'color': 'gray',
'status': 'normal',
'event': {
'type': 'sendCardRequest',
'params': {'actionId': 'reject', 'params': {'action_id': 'reject'}},
},
},
{
'text': '补充资料',
'color': 'gray',
'status': 'normal',
'event': {
'type': 'sendCardRequest',
'params': {'actionId': 'more_info', 'params': {'action_id': 'more_info'}},
},
},
],
},
'cardPrivateData': {},
'localData': {'flowStatus': '', '_cardFoldStatusLocalDataKey': ''},
'richTextData': {},
},
'renderContext': {'regenerateEnabled': '1', 'regenerateIndex': '2', 'regenerateTotal': '5'},
'editVersion': 0,
'customWidgetInfo': '',
'useCustomWidgetInfo': False,
'variableList': [
{
'id': 'content',
'type': 'markdown',
'name': 'content',
'description': '人工输入提示词(Dify form_content 含可选 node_title 前缀)',
'private': False,
'editorVarType': 'variables',
'disabled': False,
},
{
'id': 'flowStatus',
'type': 'string',
'name': 'flowStatus',
'description': 'AI卡片状态:pending(1)、writing(2)、done(3)、failed(5)',
'private': False,
'editorVarType': 'variables',
'disabled': True,
'visible': False,
},
{
'id': 'bot_avatar',
'type': 'string',
'name': 'bot_avatar',
'description': '机器人头像 DingTalk 媒体 ID@xxx 格式,启动时由 /media/upload 拿到)',
'private': False,
'editorVarType': 'variables',
'disabled': False,
},
btns_var,
],
'formList': [],
'customContextList': [],
'expList': [],
'localList': [],
'hsfList': [],
'lwpList': [],
'pageData': {},
'extension': {
'extendType': 'AI',
# All 5 statuses listed — must mirror the enableX flags on
# AICardContainer. The working reference template's extension
# includes 2 (writing) and 4 (doing); omitting them while
# enableWriting/enableDoing are true makes the renderer reject
# transitions and leaves the card body empty.
'aiStatusList': [3, 1, 5, 2, 4],
'fileTypeList': [],
},
}
def main():
editor_data = build_editor_data()
wrapper = {
'editorData': json.dumps(editor_data, ensure_ascii=False, separators=(',', ':')),
'widgetInfo': '',
'type': 'im',
'mode': 'card',
}
OUTPUT.write_text(json.dumps(wrapper, ensure_ascii=False, indent=2), encoding='utf-8')
print(f'wrote {OUTPUT}')
if __name__ == '__main__':
main()
+1 -1
View File
@@ -1,3 +1,3 @@
"""LangBot - Production-grade platform for building agentic IM bots""" """LangBot - Production-grade platform for building agentic IM bots"""
__version__ = '4.9.7' __version__ = '4.10.1'
+14 -1
View File
@@ -5,6 +5,8 @@ import argparse
import sys import sys
import os import os
from langbot.pkg.utils import paths
# ASCII art banner # ASCII art banner
asciiart = r""" asciiart = r"""
_ ___ _ _ ___ _
@@ -27,6 +29,12 @@ async def main_entry(loop: asyncio.AbstractEventLoop):
help='Use standalone plugin runtime / 使用独立插件运行时', help='Use standalone plugin runtime / 使用独立插件运行时',
default=False, default=False,
) )
parser.add_argument(
'--standalone-box',
action='store_true',
help='Use standalone box runtime / 使用独立 Box 运行时',
default=False,
)
parser.add_argument('--debug', action='store_true', help='Debug mode / 调试模式', default=False) parser.add_argument('--debug', action='store_true', help='Debug mode / 调试模式', default=False)
args = parser.parse_args() args = parser.parse_args()
@@ -35,6 +43,11 @@ async def main_entry(loop: asyncio.AbstractEventLoop):
platform.standalone_runtime = True platform.standalone_runtime = True
if args.standalone_box:
from langbot.pkg.utils import platform
platform.standalone_box = True
if args.debug: if args.debug:
from langbot.pkg.utils import constants from langbot.pkg.utils import constants
@@ -87,7 +100,7 @@ def main():
# Set up the working directory # Set up the working directory
# When installed as a package, we need to handle the working directory differently # When installed as a package, we need to handle the working directory differently
# We'll create data directory in current working directory if not exists # We'll create data directory in current working directory if not exists
os.makedirs('data', exist_ok=True) os.makedirs(paths.get_data_root(), exist_ok=True)
loop = asyncio.new_event_loop() loop = asyncio.new_event_loop()
@@ -0,0 +1,5 @@
from .client import AsyncDeerFlowClient
from .errors import DeerFlowAPIError
from . import stream_utils
__all__ = ['AsyncDeerFlowClient', 'DeerFlowAPIError', 'stream_utils']
+204
View File
@@ -0,0 +1,204 @@
"""DeerFlow LangGraph HTTP API 客户端
参考 astrbot 的 deerflow_api_client 实现,使用 httpx 适配 LangBot 风格。
"""
from __future__ import annotations
import codecs
import json
import typing
from collections.abc import AsyncGenerator
import httpx
from .errors import DeerFlowAPIError
SSE_MAX_BUFFER_CHARS = 1_048_576
def _normalize_sse_newlines(text: str) -> str:
"""规范化 CRLF/CR 为 LF,确保 SSE 块分割稳定"""
return text.replace('\r\n', '\n').replace('\r', '\n')
def _parse_sse_data_lines(data_lines: list[str]) -> typing.Any:
raw_data = '\n'.join(data_lines)
try:
return json.loads(raw_data)
except json.JSONDecodeError:
# 某些 LangGraph 兼容服务端会在单个 SSE 事件中用多个 data 行
# 发送多段 JSON 片段(例如 tuple payload
parsed_lines: list[typing.Any] = []
can_parse_all = True
for line in data_lines:
line = line.strip()
if not line:
continue
try:
parsed_lines.append(json.loads(line))
except json.JSONDecodeError:
can_parse_all = False
break
if can_parse_all and parsed_lines:
return parsed_lines[0] if len(parsed_lines) == 1 else parsed_lines
return raw_data
def _parse_sse_block(block: str) -> dict[str, typing.Any] | None:
if not block.strip():
return None
event_name = 'message'
data_lines: list[str] = []
for line in block.splitlines():
if line.startswith('event:'):
event_name = line[6:].strip()
elif line.startswith('data:'):
data_lines.append(line[5:].lstrip())
if not data_lines:
return None
return {'event': event_name, 'data': _parse_sse_data_lines(data_lines)}
class AsyncDeerFlowClient:
"""DeerFlow LangGraph HTTP API 客户端"""
api_base: str
headers: dict[str, str]
def __init__(
self,
api_base: str = 'http://127.0.0.1:2026',
api_key: str = '',
auth_header: str = '',
) -> None:
self.api_base = api_base.rstrip('/')
self.headers: dict[str, str] = {}
if auth_header:
self.headers['Authorization'] = auth_header
elif api_key:
self.headers['Authorization'] = f'Bearer {api_key}'
async def create_thread(self, timeout: float = 20) -> dict[str, typing.Any]:
"""创建一个新的 LangGraph thread
Returns:
包含 thread_id 等信息的字典
"""
url = f'{self.api_base}/api/langgraph/threads'
payload = {'metadata': {}}
async with httpx.AsyncClient(
trust_env=True,
timeout=timeout,
) as client:
response = await client.post(
url,
headers=self.headers,
json=payload,
)
if response.status_code not in (200, 201):
raise DeerFlowAPIError(
operation='create thread',
status=response.status_code,
body=response.text,
url=url,
)
return response.json()
async def delete_thread(self, thread_id: str, timeout: float = 20) -> None:
"""删除指定 thread"""
url = f'{self.api_base}/api/threads/{thread_id}'
async with httpx.AsyncClient(
trust_env=True,
timeout=timeout,
) as client:
response = await client.delete(url, headers=self.headers)
if response.status_code not in (200, 202, 204, 404):
raise DeerFlowAPIError(
operation='delete thread',
status=response.status_code,
body=response.text,
url=url,
thread_id=thread_id,
)
async def stream_run(
self,
thread_id: str,
payload: dict[str, typing.Any],
timeout: float = 120,
) -> AsyncGenerator[dict[str, typing.Any], None]:
"""运行一次 LangGraph stream 请求,逐事件 yield
Yields:
事件字典 {'event': event_name, 'data': parsed_data}
"""
url = f'{self.api_base}/api/langgraph/threads/{thread_id}/runs/stream'
# 流式请求使用单独的 read timeout 控制
stream_timeout = httpx.Timeout(
connect=min(timeout, 30),
read=timeout,
write=timeout,
pool=timeout,
)
async with httpx.AsyncClient(
trust_env=True,
timeout=stream_timeout,
) as client:
async with client.stream(
'POST',
url,
headers={
**self.headers,
'Accept': 'text/event-stream',
'Content-Type': 'application/json',
},
json=payload,
) as resp:
if resp.status_code != 200:
body = await resp.aread()
raise DeerFlowAPIError(
operation='runs/stream request',
status=resp.status_code,
body=body.decode('utf-8', errors='replace'),
url=url,
thread_id=thread_id,
)
decoder = codecs.getincrementaldecoder('utf-8')('replace')
buffer = ''
async for chunk in resp.aiter_bytes(8192):
buffer += _normalize_sse_newlines(decoder.decode(chunk))
while '\n\n' in buffer:
block, buffer = buffer.split('\n\n', 1)
parsed = _parse_sse_block(block)
if parsed is not None:
yield parsed
if len(buffer) > SSE_MAX_BUFFER_CHARS:
# 缓冲区过大,强制 flush
parsed = _parse_sse_block(buffer)
if parsed is not None:
yield parsed
buffer = ''
# flush 剩余内容
buffer += _normalize_sse_newlines(decoder.decode(b'', final=True))
while '\n\n' in buffer:
block, buffer = buffer.split('\n\n', 1)
parsed = _parse_sse_block(block)
if parsed is not None:
yield parsed
if buffer.strip():
parsed = _parse_sse_block(buffer)
if parsed is not None:
yield parsed
+30
View File
@@ -0,0 +1,30 @@
from __future__ import annotations
class DeerFlowAPIError(Exception):
"""DeerFlow API 请求失败"""
def __init__(
self,
*,
operation: str = '',
status: int = 0,
body: str = '',
url: str = '',
thread_id: str | None = None,
message: str = '',
) -> None:
self.operation = operation
self.status = status
self.body = body
self.url = url
self.thread_id = thread_id
if message:
super().__init__(message)
return
msg = f'DeerFlow {operation} failed: status={status}, url={url}, body={body}'
if thread_id is not None:
msg = f'DeerFlow {operation} failed: thread_id={thread_id}, status={status}, url={url}, body={body}'
super().__init__(msg)
@@ -0,0 +1,212 @@
"""DeerFlow LangGraph 流式响应解析工具
参考 astrbot 实现的 deerflow_stream_utils。
"""
from __future__ import annotations
import typing
from collections.abc import Iterable
def extract_text(content: typing.Any) -> str:
"""从消息 content 中提取纯文本"""
if isinstance(content, str):
return content
if isinstance(content, dict):
if isinstance(content.get('text'), str):
return content['text']
if 'content' in content:
return extract_text(content.get('content'))
if 'kwargs' in content and isinstance(content['kwargs'], dict):
return extract_text(content['kwargs'].get('content'))
if isinstance(content, list):
parts: list[str] = []
for item in content:
if isinstance(item, str):
parts.append(item)
elif isinstance(item, dict):
item_type = item.get('type')
if item_type == 'text' and isinstance(item.get('text'), str):
parts.append(item['text'])
elif 'content' in item:
parts.append(extract_text(item['content']))
return '\n'.join([p for p in parts if p]).strip()
return str(content) if content is not None else ''
def extract_messages_from_values_data(data: typing.Any) -> list[typing.Any]:
"""从 values 事件中提取 messages 列表"""
candidates: list[typing.Any] = []
if isinstance(data, dict):
candidates.append(data)
if isinstance(data.get('values'), dict):
candidates.append(data['values'])
elif isinstance(data, list):
candidates.extend([x for x in data if isinstance(x, dict)])
for item in candidates:
messages = item.get('messages')
if isinstance(messages, list):
return messages
return []
def is_ai_message(message: dict[str, typing.Any]) -> bool:
"""判断是否为 AI/assistant 消息"""
role = str(message.get('role', '')).lower()
if role in {'assistant', 'ai'}:
return True
msg_type = str(message.get('type', '')).lower()
if msg_type in {'ai', 'assistant', 'aimessage', 'aimessagechunk'}:
return True
if 'ai' in msg_type and all(token not in msg_type for token in ('human', 'tool', 'system')):
return True
return False
def extract_latest_ai_text(messages: Iterable[typing.Any]) -> str:
"""获取最近一条 AI 消息的文本内容"""
if isinstance(messages, (list, tuple)):
iterable = reversed(messages)
else:
iterable = reversed(list(messages))
for msg in iterable:
if not isinstance(msg, dict):
continue
if is_ai_message(msg):
text = extract_text(msg.get('content'))
if text:
return text
return ''
def extract_latest_ai_message(messages: Iterable[typing.Any]) -> dict[str, typing.Any] | None:
"""获取最近一条 AI 消息对象"""
if isinstance(messages, (list, tuple)):
iterable = reversed(messages)
else:
iterable = reversed(list(messages))
for msg in iterable:
if not isinstance(msg, dict):
continue
if is_ai_message(msg):
return msg
return None
def is_clarification_tool_message(message: dict[str, typing.Any]) -> bool:
"""判断是否为澄清问题工具消息"""
msg_type = str(message.get('type', '')).lower()
tool_name = str(message.get('name', '')).lower()
return msg_type == 'tool' and tool_name == 'ask_clarification'
def extract_latest_clarification_text(messages: Iterable[typing.Any]) -> str:
"""提取最近的澄清问题文本"""
if isinstance(messages, (list, tuple)):
iterable = reversed(messages)
else:
iterable = reversed(list(messages))
for msg in iterable:
if not isinstance(msg, dict):
continue
if is_clarification_tool_message(msg):
text = extract_text(msg.get('content'))
if text:
return text
return ''
def get_message_id(message: typing.Any) -> str:
"""提取消息 ID"""
if not isinstance(message, dict):
return ''
msg_id = message.get('id')
return msg_id if isinstance(msg_id, str) else ''
def extract_event_message_obj(data: typing.Any) -> dict[str, typing.Any] | None:
"""从事件 data 中提取消息对象"""
msg_obj = data
if isinstance(data, (list, tuple)) and data:
msg_obj = data[0]
if isinstance(msg_obj, dict) and isinstance(msg_obj.get('data'), dict):
msg_obj = msg_obj['data']
return msg_obj if isinstance(msg_obj, dict) else None
def extract_ai_delta_from_event_data(data: typing.Any) -> str:
"""从 messages-tuple 事件中提取 AI delta 文本"""
msg_obj = extract_event_message_obj(data)
if not msg_obj:
return ''
if is_ai_message(msg_obj):
return extract_text(msg_obj.get('content'))
return ''
def extract_clarification_from_event_data(data: typing.Any) -> str:
"""从事件中提取澄清问题"""
msg_obj = extract_event_message_obj(data)
if not msg_obj:
return ''
if is_clarification_tool_message(msg_obj):
return extract_text(msg_obj.get('content'))
return ''
def _iter_custom_event_items(data: typing.Any) -> list[dict[str, typing.Any]]:
items: list[dict[str, typing.Any]] = []
if isinstance(data, dict):
return [data]
if isinstance(data, list):
for item in data:
if isinstance(item, dict):
items.append(item)
elif isinstance(item, (list, tuple)):
for nested in item:
if isinstance(nested, dict):
items.append(nested)
return items
def extract_task_failures_from_custom_event(data: typing.Any) -> list[str]:
"""从 custom 事件中提取子任务失败信息"""
failures: list[str] = []
for item in _iter_custom_event_items(data):
event_type = str(item.get('type', '')).lower()
if event_type not in {'task_failed', 'task_timed_out'}:
continue
task_id = str(item.get('task_id', '')).strip()
error_text = extract_text(item.get('error')).strip()
if task_id and error_text:
failures.append(f'{task_id}: {error_text}')
elif error_text:
failures.append(error_text)
elif task_id:
failures.append(f'{task_id}: unknown error')
else:
failures.append('unknown task failure')
return failures
def build_task_failure_summary(failures: list[str]) -> str:
"""构建任务失败摘要"""
if not failures:
return ''
deduped: list[str] = []
seen: set[str] = set()
for failure in failures:
if failure not in seen:
seen.add(failure)
deduped.append(failure)
if len(deduped) == 1:
return f'DeerFlow subtask failed: {deduped[0]}'
joined = '\n'.join([f'- {item}' for item in deduped[:5]])
return f'DeerFlow subtasks failed:\n{joined}'
+1 -56
View File
@@ -109,61 +109,6 @@ class AsyncDifyServiceClient:
if chunk.startswith('data:'): if chunk.startswith('data:'):
yield json.loads(chunk[5:]) yield json.loads(chunk[5:])
async def workflow_submit(
self,
form_token: str,
workflow_run_id: str,
inputs: dict[str, typing.Any],
user: str,
action: str = '',
timeout: float = 120.0,
) -> typing.AsyncGenerator[dict[str, typing.Any], None]:
"""Submit human input to resume a paused workflow, then stream events.
1. POST /form/human_input/{form_token} to submit the form
2. GET /workflow/{task_id}/events to stream the resumed workflow events
"""
headers = {
'Authorization': f'Bearer {self.api_key}',
'Content-Type': 'application/json',
}
async with httpx.AsyncClient(
base_url=self.base_url,
trust_env=True,
timeout=timeout,
) as client:
# Step 1: Submit the form
payload: dict[str, typing.Any] = {
'inputs': inputs if isinstance(inputs, dict) else {},
'user': user,
'action': action,
}
submit_resp = await client.post(
f'/form/human_input/{form_token}',
headers=headers,
json=payload,
)
if submit_resp.status_code != 200:
raise DifyAPIError(f'{submit_resp.status_code} {submit_resp.text}')
# Step 2: Stream resumed workflow events
async with client.stream(
'GET',
f'/workflow/{workflow_run_id}/events',
headers={'Authorization': f'Bearer {self.api_key}'},
params={'user': user},
) as r:
async for chunk in r.aiter_lines():
if r.status_code != 200:
raise DifyAPIError(f'{r.status_code} {chunk}')
if chunk.strip() == '':
continue
if chunk.startswith('data:'):
yield json.loads(chunk[5:])
async def upload_file( async def upload_file(
self, self,
file: httpx._types.FileTypes, file: httpx._types.FileTypes,
@@ -200,7 +145,7 @@ class AsyncDifyServiceClient:
'file': file, 'file': file,
}, },
data={ data={
'user': (None, user), 'user': user,
}, },
) )
+24 -364
View File
@@ -1,28 +1,17 @@
import asyncio import asyncio
import base64 import base64
import json import json
import logging
import os
import time import time
import typing
import uuid
import urllib.parse import urllib.parse
from typing import Awaitable, Callable, Optional from typing import Callable
import dingtalk_stream # type: ignore import dingtalk_stream # type: ignore
import websockets import websockets
from .EchoHandler import EchoTextHandler from .EchoHandler import EchoTextHandler
from .card_callback import DingTalkCardActionHandler
from .dingtalkevent import DingTalkEvent from .dingtalkevent import DingTalkEvent
import httpx import httpx
import traceback import traceback
_stdout_logger = logging.getLogger('langbot.dingtalk_api')
DINGTALK_OPENAPI_BASE = 'https://api.dingtalk.com'
class DingTalkClient: class DingTalkClient:
def __init__( def __init__(
self, self,
@@ -32,7 +21,6 @@ class DingTalkClient:
robot_code: str, robot_code: str,
markdown_card: bool, markdown_card: bool,
logger: None, logger: None,
card_action_callback: Optional[Callable[[dict], Awaitable[None]]] = None,
): ):
"""初始化 WebSocket 连接并自动启动""" """初始化 WebSocket 连接并自动启动"""
self.credential = dingtalk_stream.Credential(client_id, client_secret) self.credential = dingtalk_stream.Credential(client_id, client_secret)
@@ -42,14 +30,6 @@ class DingTalkClient:
# 在 DingTalkClient 中传入自己作为参数,避免循环导入 # 在 DingTalkClient 中传入自己作为参数,避免循环导入
self.EchoTextHandler = EchoTextHandler(self) self.EchoTextHandler = EchoTextHandler(self)
self.client.register_callback_handler(dingtalk_stream.chatbot.ChatbotMessage.TOPIC, self.EchoTextHandler) self.client.register_callback_handler(dingtalk_stream.chatbot.ChatbotMessage.TOPIC, self.EchoTextHandler)
# STREAM-mode card action button click handler. Forwards parsed payload
# to the adapter so it can resume paused Dify workflows.
self.card_action_callback = card_action_callback
self.card_action_handler = DingTalkCardActionHandler(self.client, self._on_card_action)
self.client.register_callback_handler(
dingtalk_stream.handlers.CallbackHandler.TOPIC_CARD_CALLBACK,
self.card_action_handler,
)
self._message_handlers = { self._message_handlers = {
'example': [], 'example': [],
} }
@@ -59,24 +39,8 @@ class DingTalkClient:
self.access_token_expiry_time = '' self.access_token_expiry_time = ''
self.markdown_card = markdown_card self.markdown_card = markdown_card
self.logger = logger self.logger = logger
# Legacy access_token used by the OLD oapi.dingtalk.com endpoints
# (e.g. /media/upload, which is the only documented way to get an
# `@xxx` media_id usable in card Avatar.imageUrl). The new v1.0
# token doesn't work there — different auth domain.
self.legacy_access_token = ''
self.legacy_access_token_expiry_time: typing.Optional[float] = None
self._stopped = False # Flag to control the event loop self._stopped = False # Flag to control the event loop
async def _on_card_action(self, payload: dict) -> None:
"""Dispatch a parsed card-action payload to the adapter callback."""
if self.card_action_callback is None:
return
try:
await self.card_action_callback(payload)
except Exception:
if self.logger:
await self.logger.error(f'DingTalk card action callback error: {traceback.format_exc()}')
async def get_access_token(self): async def get_access_token(self):
url = 'https://api.dingtalk.com/v1.0/oauth2/accessToken' url = 'https://api.dingtalk.com/v1.0/oauth2/accessToken'
headers = {'Content-Type': 'application/json'} headers = {'Content-Type': 'application/json'}
@@ -465,35 +429,18 @@ class DingTalkClient:
'Content-Type': 'application/json', 'Content-Type': 'application/json',
} }
# For enterprise-internal robots, robotCode == AppKey (client_id).
# The dedicated robot_code field is only required for scenario-group
# robots or third-party robots; fall back to client_id when empty so
# the common single-bot setup keeps working without manual config.
robot_code = self.robot_code or self.key
data = { data = {
'robotCode': robot_code, 'robotCode': self.robot_code,
'userIds': [target_id], 'userIds': [target_id],
'msgKey': 'sampleText', 'msgKey': 'sampleText',
'msgParam': json.dumps({'content': content}), 'msgParam': json.dumps({'content': content}),
} }
_stdout_logger.info(
'DingTalk send_proactive_message_to_one request: robotCode=%s target_id=%s content_len=%d',
robot_code,
target_id,
len(content),
)
try: try:
async with httpx.AsyncClient() as client: async with httpx.AsyncClient() as client:
response = await client.post(url, headers=headers, json=data) response = await client.post(url, headers=headers, json=data)
_stdout_logger.info(
'DingTalk send_proactive_message_to_one response: status=%d body=%s',
response.status_code,
response.text[:500],
)
if response.status_code == 200: if response.status_code == 200:
return return
except Exception: except Exception:
_stdout_logger.exception('DingTalk send_proactive_message_to_one error')
await self.logger.error(f'failed to send proactive massage to person: {traceback.format_exc()}') await self.logger.error(f'failed to send proactive massage to person: {traceback.format_exc()}')
raise Exception(f'failed to send proactive massage to person: {traceback.format_exc()}') raise Exception(f'failed to send proactive massage to person: {traceback.format_exc()}')
@@ -509,7 +456,7 @@ class DingTalkClient:
} }
data = { data = {
'robotCode': self.robot_code or self.key, 'robotCode': self.robot_code,
'openConversationId': target_id, 'openConversationId': target_id,
'msgKey': 'sampleText', 'msgKey': 'sampleText',
'msgParam': json.dumps({'content': content}), 'msgParam': json.dumps({'content': content}),
@@ -530,334 +477,47 @@ class DingTalkClient:
quote_origin: bool = False, quote_origin: bool = False,
card_auto_layout: bool = False, card_auto_layout: bool = False,
): ):
"""Create + deliver the streaming chat card for a chatbot reply. card_data = {}
card_data['config'] = json.dumps({'autoLayout': card_auto_layout})
card_data['content'] = ''
Replaces the old `dingtalk_stream.AICardReplier`-based path. Returns # 将用户的消息内容作为卡片的查询参数,方便后续处理
`(None, out_track_id)` to keep call sites compatible with the
previous `(card_instance, card_instance_id)` shape — the first slot
is unused now that everything is driven by out_track_id.
"""
out_track_id = uuid.uuid4().hex
is_group = str(incoming_message.conversation_type) == '2'
if is_group:
open_space_id = f'dtv1.card//IM_GROUP.{incoming_message.conversation_id}'
else:
open_space_id = f'dtv1.card//IM_ROBOT.{incoming_message.sender_staff_id}'
card_param_map = {'content': ''}
if incoming_message.message_type == 'text': if incoming_message.message_type == 'text':
card_param_map['query'] = incoming_message.get_text_list()[0] card_data['query'] = incoming_message.get_text_list()[0]
else: else:
card_param_map['query'] = '...' card_data['query'] = '...'
await self.create_and_deliver_card( card_instance = dingtalk_stream.AICardReplier(self.client, incoming_message)
card_template_id=temp_card_id, # print(card_instance)
out_track_id=out_track_id, # 先投放卡片: https://open.dingtalk.com/document/orgapp/create-and-deliver-cards
open_space_id=open_space_id, card_instance_id = await card_instance.async_create_and_deliver_card(
is_group=is_group, temp_card_id,
card_param_map=card_param_map, card_data,
card_data_config={'autoLayout': card_auto_layout},
) )
return None, out_track_id return card_instance, card_instance_id
async def send_card_message(self, card_instance, card_instance_id: str, content: str, is_final: bool): async def send_card_message(self, card_instance, card_instance_id: str, content: str, is_final: bool):
"""Stream a single chunk into an existing card's `content` field.""" content_key = 'content'
try: try:
await self.streaming_update_card( await card_instance.async_streaming(
out_track_id=card_instance_id, card_instance_id,
content_key='content', content_key=content_key,
content_value=content, content_value=content,
append=False, append=False,
finished=is_final, finished=is_final,
failed=False, failed=False,
) )
except Exception as e: except Exception as e:
if self.logger: self.logger.exception(e)
self.logger.exception(e) await card_instance.async_streaming(
await self.streaming_update_card( card_instance_id,
out_track_id=card_instance_id, content_key=content_key,
content_key='content',
content_value='', content_value='',
append=False, append=False,
finished=is_final, finished=is_final,
failed=True, failed=True,
) )
async def create_and_deliver_card(
self,
*,
card_template_id: str,
out_track_id: str,
open_space_id: str,
is_group: bool,
card_param_map: Optional[dict] = None,
callback_type: str = 'STREAM',
callback_route_key: Optional[str] = None,
support_forward: bool = True,
dynamic_data_source_configs: Optional[list] = None,
card_data_config: Optional[dict] = None,
at_user_ids: Optional[dict] = None,
recipients: Optional[list] = None,
) -> bool:
"""POST /v1.0/card/instances/createAndDeliver.
Mirrors the SDK's `async_create_and_deliver_card` shape but exposes
the dynamic-data-source config slot so we can register a pull URL
for variable-length button lists.
"""
if not await self.check_access_token():
await self.get_access_token()
cardData: dict = {'cardParamMap': card_param_map or {}}
if card_data_config is not None:
cardData['config'] = json.dumps(card_data_config)
body: dict = {
'cardTemplateId': card_template_id,
'outTrackId': out_track_id,
'cardData': cardData,
'callbackType': callback_type,
'openSpaceId': open_space_id,
'imGroupOpenSpaceModel': {'supportForward': support_forward},
'imRobotOpenSpaceModel': {'supportForward': support_forward},
}
if callback_type == 'HTTP' and callback_route_key:
body['callbackRouteKey'] = callback_route_key
if is_group:
deliver: dict = {'robotCode': self.robot_code or self.key}
if at_user_ids:
deliver['atUserIds'] = at_user_ids
if recipients is not None:
deliver['recipients'] = recipients
body['imGroupOpenDeliverModel'] = deliver
else:
body['imRobotOpenDeliverModel'] = {'spaceType': 'IM_ROBOT'}
if dynamic_data_source_configs:
body['openDynamicDataConfig'] = {'dynamicDataSourceConfigs': dynamic_data_source_configs}
url = f'{DINGTALK_OPENAPI_BASE}/v1.0/card/instances/createAndDeliver'
headers = {
'x-acs-dingtalk-access-token': self.access_token,
'Content-Type': 'application/json',
}
try:
_stdout_logger.info(
'DingTalk createAndDeliver request body: %s',
json.dumps(body, ensure_ascii=False)[:1500],
)
async with httpx.AsyncClient() as client:
response = await client.post(url, headers=headers, json=body, timeout=30.0)
if response.status_code == 200:
_stdout_logger.info(
'DingTalk createAndDeliver response: %s',
response.text[:500],
)
return True
_stdout_logger.error(
'DingTalk createAndDeliver failed: status=%s body=%s',
response.status_code,
response.text,
)
if self.logger:
await self.logger.error(
f'DingTalk createAndDeliver failed: status={response.status_code} body={response.text}'
)
return False
except Exception:
_stdout_logger.exception('DingTalk createAndDeliver error')
if self.logger:
await self.logger.error(f'DingTalk createAndDeliver error: {traceback.format_exc()}')
return False
async def streaming_update_card(
self,
*,
out_track_id: str,
content_key: str,
content_value: str,
append: bool,
finished: bool,
failed: bool = False,
) -> bool:
"""PUT /v1.0/card/streaming.
Replaces `dingtalk_stream.AICardReplier.async_streaming` — same body
shape (outTrackId / guid / key / content / isFull / isFinalize /
isError) per the SDK source.
"""
if not await self.check_access_token():
await self.get_access_token()
body = {
'outTrackId': out_track_id,
'guid': uuid.uuid4().hex,
'key': content_key,
'content': content_value,
'isFull': not append,
'isFinalize': finished,
'isError': failed,
}
url = f'{DINGTALK_OPENAPI_BASE}/v1.0/card/streaming'
headers = {
'x-acs-dingtalk-access-token': self.access_token,
'Content-Type': 'application/json',
}
try:
async with httpx.AsyncClient() as client:
response = await client.put(url, headers=headers, json=body, timeout=30.0)
if response.status_code == 200:
return True
if self.logger:
await self.logger.error(
f'DingTalk card streaming failed: status={response.status_code} body={response.text}'
)
return False
except Exception:
if self.logger:
await self.logger.error(f'DingTalk card streaming error: {traceback.format_exc()}')
return False
async def update_card_data(
self,
*,
out_track_id: str,
card_param_map: Optional[dict] = None,
private_data: Optional[dict] = None,
) -> bool:
"""PUT /v1.0/card/instances — non-streaming card content update."""
if not await self.check_access_token():
await self.get_access_token()
body: dict = {
'outTrackId': out_track_id,
'cardData': {'cardParamMap': card_param_map or {}},
}
if private_data:
body['privateData'] = private_data
url = f'{DINGTALK_OPENAPI_BASE}/v1.0/card/instances'
headers = {
'x-acs-dingtalk-access-token': self.access_token,
'Content-Type': 'application/json',
}
try:
_stdout_logger.info(
'DingTalk update_card_data request: out_track_id=%s body=%s',
out_track_id,
json.dumps(body, ensure_ascii=False)[:500],
)
async with httpx.AsyncClient() as client:
response = await client.put(url, headers=headers, json=body, timeout=30.0)
_stdout_logger.info(
'DingTalk update_card_data response: status=%d body=%s',
response.status_code,
response.text[:300],
)
if response.status_code == 200:
return True
if self.logger:
await self.logger.error(
f'DingTalk update card failed: status={response.status_code} body={response.text}'
)
return False
except Exception:
_stdout_logger.exception('DingTalk update_card_data error')
if self.logger:
await self.logger.error(f'DingTalk update card error: {traceback.format_exc()}')
return False
async def get_legacy_access_token(self) -> Optional[str]:
"""Fetch the LEGACY (oapi.dingtalk.com) access_token. This is a
different auth domain from the v1.0 token cached in
``self.access_token`` — only the legacy token authorises the
``/media/upload`` endpoint that returns an ``@xxx`` media_id
consumable by card components like Avatar.imageUrl.
Returns the token string on success, None on failure. Caches
with a 60s safety margin before the documented 7200s expiry.
"""
now = time.time()
if (
self.legacy_access_token
and self.legacy_access_token_expiry_time
and now < self.legacy_access_token_expiry_time
):
return self.legacy_access_token
url = 'https://oapi.dingtalk.com/gettoken'
try:
async with httpx.AsyncClient() as client:
response = await client.get(url, params={'appkey': self.key, 'appsecret': self.secret}, timeout=15.0)
data = response.json() if response.status_code == 200 else {}
if data.get('errcode') == 0 and data.get('access_token'):
self.legacy_access_token = data['access_token']
expires_in = int(data.get('expires_in', 7200))
self.legacy_access_token_expiry_time = now + expires_in - 60
return self.legacy_access_token
if self.logger:
await self.logger.error(
f'DingTalk legacy gettoken failed: status={response.status_code} body={response.text[:200]}'
)
except Exception:
_stdout_logger.exception('DingTalk legacy gettoken error')
if self.logger:
await self.logger.error(f'DingTalk legacy gettoken error: {traceback.format_exc()}')
return None
async def upload_image_media(self, file_path: str) -> Optional[str]:
"""Upload an image file to DingTalk media storage and return the
``@xxx`` media_id, which can be passed straight into card variables
like Avatar.imageUrl. Endpoint:
POST https://oapi.dingtalk.com/media/upload?access_token=…&type=image
Returns the media_id on success, None on any failure (caller
should handle a None gracefully — DingTalk falls back to a
default avatar when imageUrl is empty/unknown).
"""
if not os.path.exists(file_path):
if self.logger:
await self.logger.error(f'DingTalk upload_image_media: file not found {file_path}')
return None
token = await self.get_legacy_access_token()
if not token:
return None
url = 'https://oapi.dingtalk.com/media/upload'
try:
with open(file_path, 'rb') as f:
file_bytes = f.read()
file_name = os.path.basename(file_path)
# Best-effort content-type guess; DingTalk accepts the major image
# mime types and otherwise infers from the bytes.
ext = os.path.splitext(file_name)[1].lower().lstrip('.')
mime = {'png': 'image/png', 'jpg': 'image/jpeg', 'jpeg': 'image/jpeg', 'gif': 'image/gif'}.get(
ext, 'application/octet-stream'
)
async with httpx.AsyncClient() as client:
response = await client.post(
url,
params={'access_token': token, 'type': 'image'},
files={'media': (file_name, file_bytes, mime)},
timeout=30.0,
)
data = response.json() if response.status_code == 200 else {}
if data.get('errcode') == 0 and data.get('media_id'):
_stdout_logger.info('DingTalk upload_image_media OK: media_id=%s', data['media_id'])
return data['media_id']
if self.logger:
await self.logger.error(
f'DingTalk upload_image_media failed: status={response.status_code} body={response.text[:300]}'
)
except Exception:
_stdout_logger.exception('DingTalk upload_image_media error')
if self.logger:
await self.logger.error(f'DingTalk upload_image_media error: {traceback.format_exc()}')
return None
async def start(self): async def start(self):
"""启动 WebSocket 连接,监听消息""" """启动 WebSocket 连接,监听消息"""
self._stopped = False self._stopped = False
@@ -1,96 +0,0 @@
"""STREAM-mode handler for DingTalk card action button clicks.
DingTalk delivers card-action callbacks over the same WebSocket stream used
for chatbot messages, under the topic `/v1.0/card/instances/callback`. This
module subclasses `dingtalk_stream.CallbackHandler` and forwards the parsed
payload to a coroutine the adapter registers, so the resume-paused-workflow
logic stays in the platform adapter where it belongs.
The `CardCallbackMessage` returned by `from_dict` exposes:
* `card_instance_id` (from `outTrackId`) — the card whose button was clicked
* `user_id` — the clicker's userId
* `content` — parsed JSON; the click params live here. Where exactly inside
`content` they sit depends on the template binding. We probe
the common paths.
* `extension` — parsed JSON; any extra data we set when delivering the card.
"""
from __future__ import annotations
from typing import Awaitable, Callable, Optional
import dingtalk_stream # type: ignore
from dingtalk_stream import AckMessage
from dingtalk_stream.card_callback import CardCallbackMessage
_PARAM_PATHS = (
('params',),
('cardPrivateData', 'params'),
('userPrivateData', 'params'),
)
def _extract_params(content: dict) -> dict:
"""Return the action params dict regardless of where the template put it."""
for path in _PARAM_PATHS:
node = content
for key in path:
if not isinstance(node, dict):
node = None
break
node = node.get(key)
if node is None:
break
if isinstance(node, dict) and node:
return node
return {}
class DingTalkCardActionHandler(dingtalk_stream.CallbackHandler):
def __init__(
self,
dingtalk_stream_client,
on_action: Optional[Callable[[dict], Awaitable[None]]] = None,
):
super().__init__()
self.dingtalk_client = dingtalk_stream_client
self.on_action = on_action
async def process(self, callback: dingtalk_stream.CallbackMessage):
try:
message = CardCallbackMessage.from_dict(callback.data)
params = _extract_params(message.content if isinstance(message.content, dict) else {})
# `CardCallbackMessage.from_dict` does not surface `actionId` (the
# top-level field that ButtonGroup's sendCardRequest event puts
# there). Pull it from the raw callback.data instead.
raw = callback.data if isinstance(callback.data, dict) else {}
action_id = raw.get('actionId') or ''
if not action_id:
# Some templates nest it under actionData / cardPrivateData.
action_data = raw.get('actionData') or {}
if isinstance(action_data, dict):
action_id = action_data.get('actionId') or action_id
if not action_id:
cpd = action_data.get('cardPrivateData') or {}
if isinstance(cpd, dict):
ids = cpd.get('actionIds')
if isinstance(ids, list) and ids:
action_id = str(ids[0])
payload = {
'out_track_id': message.card_instance_id,
'user_id': message.user_id,
'corp_id': message.corp_id,
'action_id': action_id,
'params': params,
'raw_content': message.content,
'extension': message.extension if isinstance(message.extension, dict) else {},
}
if self.on_action is not None:
await self.on_action(payload)
except Exception as e:
self.logger.error(f'DingTalkCardActionHandler.process error: {e}')
return AckMessage.STATUS_OK, 'OK'
+9 -261
View File
@@ -12,70 +12,6 @@ import traceback
from cryptography.hazmat.primitives.asymmetric import ed25519 from cryptography.hazmat.primitives.asymmetric import ed25519
def build_keyboard_from_form(form_data: dict, *, buttons_per_row: int = 2) -> dict:
"""Build a QQ keyboard JSON payload from a Dify human-input form_data.
Each Dify ``action`` becomes a callback button (``action.type=1``)
whose ``data`` is set directly to the Dify ``action_id``. The
INTERACTION_CREATE event carries this back as
``data.resolved.button_data`` so the adapter can match the click to
the originating form.
Layout limits per spec: max 5 rows, max 5 buttons per row. We default
to 2 buttons per row for legibility; oversized button lists wrap
onto additional rows and overflow gets dropped (max 25 visible).
Args:
form_data: Dify ``{"actions": [{"id", "title", "button_style"}, ...]}``.
buttons_per_row: 1..5. Mobile UI looks best at 2.
Returns:
``{"content": {"rows": [{"buttons": [...]}]}}``.
"""
actions = list(form_data.get('actions') or [])[:25] # 5×5 hard cap
buttons_per_row = max(1, min(5, buttons_per_row))
def _button(idx: int, action: dict) -> dict:
action_id = str(action.get('id') or '')
label = str(action.get('title') or action_id or f'选项 {idx + 1}')
style_raw = (action.get('button_style') or '').lower()
# QQ: 0 灰色线框, 1 蓝色线框. Highlight the primary / first action.
if style_raw == 'primary' or (style_raw == '' and idx == 0):
style = 1
else:
style = 0
return {
'id': str(idx + 1),
'render_data': {
'label': label,
# Shown after the user clicks — gives local "已选择" feedback
# without a follow-up message. Style mimics DingTalk/Lark's
# in-card selection state.
'visited_label': f'{label}',
'style': style,
},
'action': {
'type': 1, # callback button
'permission': {'type': 2}, # everyone can click
'data': action_id,
'unsupport_tips': '当前客户端版本不支持此按钮,请升级 QQ',
},
}
rows = []
for row_start in range(0, len(actions), buttons_per_row):
row_actions = actions[row_start : row_start + buttons_per_row]
rows.append(
{
'buttons': [_button(row_start + j, a) for j, a in enumerate(row_actions)],
}
)
if len(rows) >= 5:
break
return {'content': {'rows': rows}}
class QQOfficialClient: class QQOfficialClient:
def __init__(self, secret: str, token: str, app_id: str, logger: None, unified_mode: bool = False): def __init__(self, secret: str, token: str, app_id: str, logger: None, unified_mode: bool = False):
self.unified_mode = unified_mode self.unified_mode = unified_mode
@@ -94,10 +30,6 @@ class QQOfficialClient:
self.token = token self.token = token
self.app_id = app_id self.app_id = app_id
self._message_handlers = {} self._message_handlers = {}
# Single optional handler for INTERACTION_CREATE (button click). We
# don't multiplex like message handlers — only the adapter cares,
# and the click<->resume path needs a single source of truth.
self._interaction_handler: Optional[Callable[[Dict[str, Any], Optional[str]], Any]] = None
self.base_url = 'https://api.sgroup.qq.com' self.base_url = 'https://api.sgroup.qq.com'
self.access_token = '' self.access_token = ''
self.access_token_expiry_time = None self.access_token_expiry_time = None
@@ -175,23 +107,6 @@ class QQOfficialClient:
return response, 200 return response, 200
if payload.get('op') == 0: if payload.get('op') == 0:
# INTERACTION_CREATE (button click) skips ``get_message`` —
# that helper only flattens message-event fields and would
# drop ``data.resolved.button_data`` / ``data.button_id``.
if payload.get('t') == 'INTERACTION_CREATE':
if self._interaction_handler:
try:
d = payload.get('d') or {}
# Top-level ``id`` is the ws/event id used as
# ``event_id`` for passive replies. ``d.id``
# is the interaction id used for ACK. Do not
# confuse the two — QQ rejects misuse with
# 40034025.
ws_event_id = payload.get('id')
await self._interaction_handler(d, ws_event_id)
except Exception:
await self.logger.error(f'Error in interaction handler: {traceback.format_exc()}')
return {'code': 0, 'message': 'success'}
message_data = await self.get_message(payload) message_data = await self.get_message(payload)
if message_data: if message_data:
event = QQOfficialEvent.from_payload(message_data) event = QQOfficialEvent.from_payload(message_data)
@@ -218,21 +133,6 @@ class QQOfficialClient:
return decorator return decorator
def on_interaction(self):
"""Register a single handler for INTERACTION_CREATE events.
The handler receives ``(data_dict, interaction_id)`` — the raw
``d`` payload plus the top-level ``id`` field (the interaction
id, needed for the PUT /interactions/{id} ack and for reuse as
an ``event_id`` on the resumed reply within 30 minutes).
"""
def decorator(func: Callable[[Dict[str, Any], Optional[str]], Any]):
self._interaction_handler = func
return func
return decorator
async def _handle_message(self, event: QQOfficialEvent): async def _handle_message(self, event: QQOfficialEvent):
"""处理消息事件""" """处理消息事件"""
msg_type = event.t msg_type = event.t
@@ -277,20 +177,8 @@ class QQOfficialClient:
content_type = attachment.get('content_type', '') content_type = attachment.get('content_type', '')
return content_type.startswith('image/') return content_type.startswith('image/')
async def send_private_text_msg( async def send_private_text_msg(self, user_openid: str, content: str, msg_id: str):
self, """发送私聊消息"""
user_openid: str,
content: str,
msg_id: Optional[str] = None,
event_id: Optional[str] = None,
msg_seq: int = 1,
):
"""Send a c2c text message.
Either ``msg_id`` (inbound user msg, free passive reply) or
``event_id`` (e.g. INTERACTION_CREATE id, valid 30 min) is
required. Without either, the call costs the proactive-send quota.
"""
if not await self.check_access_token(): if not await self.check_access_token():
await self.get_access_token() await self.get_access_token()
@@ -300,15 +188,11 @@ class QQOfficialClient:
'Authorization': f'QQBot {self.access_token}', 'Authorization': f'QQBot {self.access_token}',
'Content-Type': 'application/json', 'Content-Type': 'application/json',
} }
data: dict[str, Any] = { data = {
'content': content, 'content': content,
'msg_type': 0, 'msg_type': 0,
'msg_seq': msg_seq, 'msg_id': msg_id,
} }
if msg_id:
data['msg_id'] = msg_id
if event_id:
data['event_id'] = event_id
response = await client.post(url, headers=headers, json=data) response = await client.post(url, headers=headers, json=data)
response_data = response.json() response_data = response.json()
if response.status_code == 200: if response.status_code == 200:
@@ -317,19 +201,8 @@ class QQOfficialClient:
await self.logger.error(f'Failed to send private message: {response_data}') await self.logger.error(f'Failed to send private message: {response_data}')
raise ValueError(response) raise ValueError(response)
async def send_group_text_msg( async def send_group_text_msg(self, group_openid: str, content: str, msg_id: str):
self, """发送群聊消息"""
group_openid: str,
content: str,
msg_id: Optional[str] = None,
event_id: Optional[str] = None,
msg_seq: int = 1,
):
"""Send a group text message.
Either ``msg_id`` or ``event_id`` is required (see
:meth:`send_private_text_msg` for the distinction).
"""
if not await self.check_access_token(): if not await self.check_access_token():
await self.get_access_token() await self.get_access_token()
@@ -339,15 +212,11 @@ class QQOfficialClient:
'Authorization': f'QQBot {self.access_token}', 'Authorization': f'QQBot {self.access_token}',
'Content-Type': 'application/json', 'Content-Type': 'application/json',
} }
data: dict[str, Any] = { data = {
'content': content, 'content': content,
'msg_type': 0, 'msg_type': 0,
'msg_seq': msg_seq, 'msg_id': msg_id,
} }
if msg_id:
data['msg_id'] = msg_id
if event_id:
data['event_id'] = event_id
response = await client.post(url, headers=headers, json=data) response = await client.post(url, headers=headers, json=data)
if response.status_code == 200: if response.status_code == 200:
return return
@@ -616,106 +485,6 @@ class QQOfficialClient:
raise Exception(f'Failed to send stream message: HTTP {response.status_code} {response.text}') raise Exception(f'Failed to send stream message: HTTP {response.status_code} {response.text}')
return response.json() return response.json()
async def send_markdown_keyboard(
self,
target_type: str,
target_id: str,
markdown_content: str,
keyboard: dict,
msg_id: Optional[str] = None,
event_id: Optional[str] = None,
msg_seq: int = 1,
) -> dict:
"""Send a ``msg_type=2`` (markdown) message carrying a keyboard.
The keyboard ride-along is the only documented way to attach
buttons in QQ official; pure keyboard-only messages are not
accepted by the server (markdown content is required).
Args:
target_type: 'c2c' (single chat), 'group', 'channel' (text
channel — uses POST /channels/{id}/messages instead of v2).
target_id: openid for c2c/group, channel_id for channel.
markdown_content: Plain markdown text shown above the buttons.
keyboard: ``{'content': {'rows': [{'buttons': [...]}]}}`` per
the official spec. Use :func:`build_keyboard_from_form`
to construct from Dify form_data.
msg_id: Inbound user message id; turns this into a passive
reply (preferred — no monthly quota cost).
event_id: Use ``INTERACTION_CREATE`` event id from a prior
button click to keep within the 30-minute passive window
without an inbound msg_id.
msg_seq: De-dup counter when reusing msg_id.
"""
if not await self.check_access_token():
await self.get_access_token()
if target_type == 'c2c':
url = f'{self.base_url}/v2/users/{target_id}/messages'
elif target_type == 'group':
url = f'{self.base_url}/v2/groups/{target_id}/messages'
elif target_type == 'channel':
url = f'{self.base_url}/channels/{target_id}/messages'
else:
raise ValueError(f'Unsupported target_type for markdown+keyboard: {target_type}')
body: dict[str, Any] = {
'msg_type': 2,
'markdown': {'content': markdown_content},
'keyboard': keyboard,
'msg_seq': msg_seq,
}
if msg_id:
body['msg_id'] = msg_id
if event_id:
body['event_id'] = event_id
async with httpx.AsyncClient(timeout=30) as client:
headers = {
'Authorization': f'QQBot {self.access_token}',
'Content-Type': 'application/json',
}
response = await client.post(url, headers=headers, json=body)
if response.status_code != 200:
await self.logger.error(
f'Failed to send markdown+keyboard: HTTP {response.status_code} {response.text}'
)
raise Exception(f'Failed to send markdown+keyboard: HTTP {response.status_code} {response.text}')
return response.json()
async def ack_interaction(self, interaction_id: str, code: int = 0) -> None:
"""Acknowledge a button-click INTERACTION_CREATE event.
QQ keeps the client in a loading spinner until this ack is
received. Should be called as soon as the click is parsed, before
any heavier downstream work (the actual workflow resume can run
async).
Args:
interaction_id: The ``id`` field from the INTERACTION_CREATE event.
code: 0=success, 1=fail, 2=rate-limited, 3=duplicate, 4=no
permission, 5=admin only. Default 0.
"""
if not interaction_id:
return
if not await self.check_access_token():
await self.get_access_token()
url = f'{self.base_url}/interactions/{interaction_id}'
async with httpx.AsyncClient(timeout=10) as client:
headers = {
'Authorization': f'QQBot {self.access_token}',
'Content-Type': 'application/json',
}
try:
response = await client.put(url, headers=headers, json={'code': code})
if response.status_code >= 400:
await self.logger.warning(
f'ack_interaction non-success: HTTP {response.status_code} {response.text}'
)
except Exception as e:
await self.logger.warning(f'ack_interaction error (non-fatal): {e}')
async def is_token_expired(self): async def is_token_expired(self):
"""检查token是否过期""" """检查token是否过期"""
if self.access_token_expiry_time is None: if self.access_token_expiry_time is None:
@@ -884,12 +653,6 @@ class QQOfficialClient:
d = payload.get('d', {}) d = payload.get('d', {})
s = payload.get('s') s = payload.get('s')
t = payload.get('t') t = payload.get('t')
# Top-level event id, distinct from `d.id`. Per QQ
# spec this is the only value accepted as ``event_id``
# in subsequent passive-reply send-message calls
# (``d.id`` for INTERACTION_CREATE is the interaction
# id, used solely for PUT /interactions/{id} ack).
ws_event_id = payload.get('id')
if not isinstance(d, dict): if not isinstance(d, dict):
d = {} d = {}
@@ -968,22 +731,7 @@ class QQOfficialClient:
else: else:
await self.logger.debug(f'Received event: {t}, seq={s}') await self.logger.debug(f'Received event: {t}, seq={s}')
# INTERACTION_CREATE bypasses the regular if on_event:
# on_event dispatcher so the adapter sees the
# top-level ws_event_id (needed as event_id
# for the resumed reply) — same shape as the
# webhook handler.
if t == 'INTERACTION_CREATE':
if self._interaction_handler:
try:
result = self._interaction_handler(d, ws_event_id)
if asyncio.iscoroutine(result):
await result
except Exception:
await self.logger.error(
f'Error in interaction handler (ws): {traceback.format_exc()}'
)
elif on_event:
try: try:
result = on_event(t, d) result = on_event(t, d)
if asyncio.iscoroutine(result): if asyncio.iscoroutine(result):
+1 -344
View File
@@ -67,16 +67,6 @@ class StreamSession:
# 反馈 ID,用于接收用户点赞/点踩反馈 # 反馈 ID,用于接收用户点赞/点踩反馈
feedback_id: Optional[str] = None feedback_id: Optional[str] = None
# Dify 人工输入暂停态:runner 把 _form_data 传过来时填充。
# 一旦设置,下次企微 followup 请求时返回 button_interaction 模板卡
# 替代 stream chunk。点击按钮会回调 template_card_eventEventKey
# 就是 Dify 的 action_id。
pending_form: Optional[dict] = None
# template_card task_id(企微要求 button_interaction 必填且不可重复)。
# 创建 pending_form 时生成;按钮点击回调里用来反查 session。
pending_form_task_id: Optional[str] = None
class StreamSessionManager: class StreamSessionManager:
"""管理 stream 会话的生命周期,并负责队列的生产消费。""" """管理 stream 会话的生命周期,并负责队列的生产消费。"""
@@ -93,9 +83,6 @@ class StreamSessionManager:
self._sessions: dict[str, StreamSession] = {} # stream_id -> StreamSession 映射 self._sessions: dict[str, StreamSession] = {} # stream_id -> StreamSession 映射
self._msg_index: dict[str, str] = {} # msgid -> stream_id 映射,便于流水线根据消息 ID 找到会话 self._msg_index: dict[str, str] = {} # msgid -> stream_id 映射,便于流水线根据消息 ID 找到会话
self._feedback_index: dict[str, str] = {} # feedback_id -> stream_id 映射 self._feedback_index: dict[str, str] = {} # feedback_id -> stream_id 映射
# task_id (button_interaction template_card 的) -> stream_id 映射,
# 用于按钮点击回调里反查 pending_form。
self._task_index: dict[str, str] = {}
def get_stream_id_by_msg(self, msg_id: str) -> Optional[str]: def get_stream_id_by_msg(self, msg_id: str) -> Optional[str]:
if not msg_id: if not msg_id:
@@ -131,40 +118,6 @@ class StreamSessionManager:
if feedback_id and stream_id: if feedback_id and stream_id:
self._feedback_index[feedback_id] = stream_id self._feedback_index[feedback_id] = stream_id
def set_pending_form(self, stream_id: str, form_data: dict, task_id: str) -> None:
"""把 Dify 人工输入暂停态绑定到 stream session。
下一次企微 followup 请求时,adapter 检测到 pending_form
返回 button_interaction 模板卡而不是 stream chunk。
"""
session = self._sessions.get(stream_id)
if not session:
return
session.pending_form = form_data
session.pending_form_task_id = task_id
if task_id:
self._task_index[task_id] = stream_id
def get_session_by_task_id(self, task_id: str) -> Optional[StreamSession]:
"""按按钮点击回调里的 TaskId 反查 session。"""
if not task_id:
return None
stream_id = self._task_index.get(task_id)
if not stream_id:
return None
return self._sessions.get(stream_id)
def clear_pending_form(self, stream_id: str) -> None:
"""按钮点击消费完后清掉 pending_form,避免重复弹卡。"""
session = self._sessions.get(stream_id)
if not session:
return
task_id = session.pending_form_task_id
session.pending_form = None
session.pending_form_task_id = None
if task_id:
self._task_index.pop(task_id, None)
def create_or_get(self, msg_json: dict[str, Any]) -> tuple[StreamSession, bool]: def create_or_get(self, msg_json: dict[str, Any]) -> tuple[StreamSession, bool]:
"""根据企业微信回调创建或获取会话。 """根据企业微信回调创建或获取会话。
@@ -770,199 +723,8 @@ async def parse_wecom_bot_message(
return message_data return message_data
def _wecom_button_style(action: dict, *, selected: bool = False) -> int:
"""Map Dify button style to WeCom button style."""
if not selected:
return 2
return 1
def build_button_interaction_payload(
form_data: dict,
task_id: str,
*,
source: Optional[dict] = None,
) -> dict[str, Any]:
"""Build a `template_card` (button_interaction) WeCom payload.
Shared by both the webhook-mode client (returns the payload as the
response to a stream-followup callback) and the ws_client (sends it
as a reply frame). Output shape is `{"msgtype": "template_card",
"template_card": {...}}` per the WeCom spec.
Args:
form_data: Dify human-input form data with keys ``actions`` (list of
``{id, title, button_style}``), ``node_title``, ``form_content``.
task_id: Unique per-card identifier. WeCom requires this for
button_interaction. The click callback returns it as TaskId so we
can find the originating session.
source: Optional source header dict ``{icon_url, desc, desc_color}``
shown at the top of the card. WeCom accepts arbitrary HTTPS
URLs for ``icon_url`` (unlike DingTalk Avatar which requires
a uploaded media id), so the LangBot logo URL can be passed
straight through.
Notes:
* ``button.key`` is set directly to the Dify ``action_id``. The click
callback's ``EventKey`` carries this back unchanged (1024-byte limit
per the spec, far more than we ever need).
* WeCom caps the button list at 6. Extra actions are appended to
``sub_title_text`` so users can still reply with the id as text.
* Styles map ``primary``→1 (blue), ``danger``→2 (red), default→0
(gray). First button is auto-promoted to primary when no style.
"""
actions = list(form_data.get('actions') or [])
node_title = (form_data.get('node_title') or '').strip() or '人工介入'
form_content = (form_data.get('form_content') or '').strip()
visible_actions = actions[:6]
overflow = actions[6:]
sub_title_parts: list[str] = []
if form_content:
sub_title_parts.append(form_content)
if overflow:
extra_lines = [f' - {a.get("title") or a.get("id") or ""} (回复 id: {a.get("id") or ""})' for a in overflow]
sub_title_parts.append(f'另有 {len(overflow)} 个选项不在按钮列表中,可直接回复 id:\n' + '\n'.join(extra_lines))
sub_title_text = '\n\n'.join(sub_title_parts) or '请选择一个操作以继续。'
button_list = []
for idx, action in enumerate(visible_actions):
action_id = str(action.get('id') or '')
title = str(action.get('title') or action_id or f'选项 {idx + 1}')
button_list.append(
{
'text': title,
'style': _wecom_button_style(action),
'key': action_id,
}
)
card: dict[str, Any] = {
'card_type': 'button_interaction',
'main_title': {
'title': node_title,
},
'sub_title_text': sub_title_text,
'button_list': button_list,
'task_id': task_id,
}
if source:
card['source'] = source
return {
'msgtype': 'template_card',
'template_card': card,
}
def extract_template_card_action(tce: dict[str, Any]) -> tuple[str, str, str]:
"""Extract task id, clicked button key, and card type from a WeCom callback."""
task_id = tce.get('TaskId') or tce.get('task_id') or tce.get('taskid') or tce.get('taskId') or ''
event_key = (
tce.get('EventKey')
or tce.get('event_key')
or tce.get('eventkey')
or tce.get('eventKey')
or tce.get('key')
or tce.get('Key')
or ''
)
card_type = tce.get('CardType') or tce.get('card_type') or tce.get('cardtype') or tce.get('cardType') or ''
for button_key in ('button', 'Button', 'selected_button', 'selectedButton'):
button = tce.get(button_key)
if isinstance(button, dict):
if not event_key:
event_key = (
button.get('key')
or button.get('Key')
or button.get('event_key')
or button.get('EventKey')
or button.get('id')
or button.get('Id')
or ''
)
break
return str(task_id or ''), str(event_key or ''), str(card_type or '')
def resolve_form_action_title(form_data: dict, action_id: str) -> str:
"""Resolve a Dify form action title from its id."""
clean_action_id = str(action_id or '').strip()
for action in form_data.get('actions') or []:
if str(action.get('id', '')) == clean_action_id:
return str(action.get('title') or clean_action_id)
return clean_action_id
def build_button_interaction_update_card(
form_data: dict,
task_id: str,
action_id: str,
source: Optional[dict[str, Any]] = None,
) -> dict[str, Any]:
"""Build the template_card body used to update a clicked form card."""
node_title = str(form_data.get('node_title') or '').strip() or '人工介入'
form_content = str(form_data.get('form_content') or '').strip()
action_title = resolve_form_action_title(form_data, action_id)
clean_action_id = str(action_id or '').strip()
button_list = []
matched = False
for idx, action in enumerate(list(form_data.get('actions') or [])[:6]):
action_key = str(action.get('id') or '')
button_title = str(action.get('title') or action_key or f'Option {idx + 1}')
button = {
'text': button_title,
'style': _wecom_button_style(action),
'key': action_key,
}
if action_key == clean_action_id:
button['style'] = _wecom_button_style(action, selected=True)
button['text'] = f'已选择:{button_title}'
button['replace_text'] = f'已选择:{button_title}'
matched = True
button_list.append(button)
if clean_action_id and not matched:
button_list.append(
{
'text': action_title or clean_action_id,
'style': 1,
'key': clean_action_id,
'replace_text': f'已选择:{action_title or clean_action_id}',
}
)
card: dict[str, Any] = {
'card_type': 'button_interaction',
'main_title': {
'title': node_title,
},
'sub_title_text': form_content,
'button_list': button_list,
'task_id': task_id,
}
if source:
card['source'] = source
return card
class WecomBotClient: class WecomBotClient:
def __init__( def __init__(self, Token: str, EnCodingAESKey: str, Corpid: str, logger: EventLogger, unified_mode: bool = False):
self,
Token: str,
EnCodingAESKey: str,
Corpid: str,
logger: EventLogger,
unified_mode: bool = False,
):
"""企业微信智能机器人客户端。 """企业微信智能机器人客户端。
Args: Args:
@@ -999,16 +761,6 @@ class WecomBotClient:
self.stream_poll_timeout = 0.5 self.stream_poll_timeout = 0.5
self._feedback_callback: Optional[Callable] = None self._feedback_callback: Optional[Callable] = None
self._card_action_callback: Optional[Callable] = None
# Optional `source` block injected into every interactive template_card
# the client builds. Set via `set_card_source` from the adapter after
# reading config. Format: {icon_url, desc, desc_color}.
self.card_source: Optional[dict] = None
def set_card_source(self, source: Optional[dict]) -> None:
"""Set the `source` header dict injected into every
button_interaction template_card. Pass None to clear."""
self.card_source = source
def set_feedback_callback(self, callback: Callable) -> None: def set_feedback_callback(self, callback: Callable) -> None:
"""设置反馈回调函数。 """设置反馈回调函数。
@@ -1018,19 +770,6 @@ class WecomBotClient:
""" """
self._feedback_callback = callback self._feedback_callback = callback
def set_card_action_callback(self, callback: Callable) -> None:
"""设置按钮卡片点击回调函数。
Signature: ``async def callback(session, action_id, task_id, raw_event) -> None``
``session`` is the StreamSession the card was attached to;
``action_id`` is the Dify action_id reflected back via the
button's ``key`` field; ``task_id`` is the card's task_id
(matches ``session.pending_form_task_id``); ``raw_event`` is the
decoded callback JSON for any extra fields the adapter wants.
"""
self._card_action_callback = callback
@staticmethod @staticmethod
def _build_stream_payload( def _build_stream_payload(
stream_id: str, content: str, finish: bool, feedback_id: Optional[str] = None stream_id: str, content: str, finish: bool, feedback_id: Optional[str] = None
@@ -1061,12 +800,6 @@ class WecomBotClient:
'stream': stream_payload, 'stream': stream_payload,
} }
def _build_button_interaction_payload(self, form_data: dict, task_id: str) -> dict[str, Any]:
"""Class-level shim — delegates to module-level builder and auto-
injects the client's configured `source` block so every card emitted
through this client carries the LangBot header."""
return build_button_interaction_payload(form_data, task_id, source=self.card_source)
async def _encrypt_and_reply(self, payload: dict[str, Any], nonce: str) -> tuple[Response, int]: async def _encrypt_and_reply(self, payload: dict[str, Any], nonce: str) -> tuple[Response, int]:
"""对响应进行加密封装并返回给企业微信。 """对响应进行加密封装并返回给企业微信。
@@ -1159,22 +892,6 @@ class WecomBotClient:
return await self._encrypt_and_reply(self._build_stream_payload('', '', True), nonce) return await self._encrypt_and_reply(self._build_stream_payload('', '', True), nonce)
session = self.stream_sessions.get_session(stream_id) session = self.stream_sessions.get_session(stream_id)
# If a Dify human-input pause arrived during this stream, switch
# the response from `msgtype: stream` to `msgtype: template_card`
# (button_interaction). The session's stream is also marked
# finished so future followups aren't expected (assuming the
# WeCom client treats template_card as the terminal response —
# we'll know from the next callback whether it kept polling).
if session and session.pending_form and session.pending_form_task_id:
await self.logger.info(
f'WeComBot: returning button_interaction for stream_id={stream_id} '
f'task_id={session.pending_form_task_id} actions={len(session.pending_form.get("actions") or [])}'
)
card_payload = self._build_button_interaction_payload(session.pending_form, session.pending_form_task_id)
self.stream_sessions.mark_finished(stream_id)
return await self._encrypt_and_reply(card_payload, nonce)
chunk = await self.stream_sessions.consume(stream_id, timeout=self.stream_poll_timeout) chunk = await self.stream_sessions.consume(stream_id, timeout=self.stream_poll_timeout)
if not chunk: if not chunk:
@@ -1283,48 +1000,11 @@ class WecomBotClient:
if event_type == 'feedback_event': if event_type == 'feedback_event':
return await self._handle_feedback_event(msg_json, nonce) return await self._handle_feedback_event(msg_json, nonce)
# Button click on a button_interaction template_card. The WeCom doc
# calls this `template_card_event`; some routes wrap the button
# event payload inside `event.template_card_event`.
if event_type == 'template_card_event':
return await self._handle_template_card_event(msg_json, nonce)
if msg_json.get('msgtype') == 'stream': if msg_json.get('msgtype') == 'stream':
return await self._handle_post_followup_response(msg_json, nonce) return await self._handle_post_followup_response(msg_json, nonce)
return await self._handle_post_initial_response(msg_json, nonce) return await self._handle_post_initial_response(msg_json, nonce)
async def _handle_template_card_event(self, msg_json: dict[str, Any], nonce: str) -> tuple[Response, int]:
"""Handle a button click on a button_interaction template_card.
WeCom carries the click info in ``event.template_card_event`` with
``TaskId`` matching the card we created and ``EventKey`` carrying
the button's ``key`` (which we set to the Dify ``action_id``).
"""
try:
tce = msg_json.get('event', {}).get('template_card_event', {})
task_id, event_key, card_type = extract_template_card_action(tce)
await self.logger.info(f'收到按钮点击: task_id={task_id} event_key={event_key!r} card_type={card_type}')
session = self.stream_sessions.get_session_by_task_id(task_id)
if session is None:
await self.logger.warning(f'未找到 task_id={task_id} 对应的 session,按钮点击被丢弃')
else:
if self._card_action_callback is not None:
try:
await self._card_action_callback(session, event_key, task_id, msg_json)
except Exception:
await self.logger.error(f'card action callback raised: {traceback.format_exc()}')
# Drop the form so a fresh chunk/followup doesn't re-render
# the same card (and so the task_id can be GC'd).
self.stream_sessions.clear_pending_form(session.stream_id)
except Exception:
await self.logger.error(f'_handle_template_card_event error: {traceback.format_exc()}')
# WeCom expects an empty success ack for event callbacks.
return await self._encrypt_and_reply({}, nonce)
async def _handle_feedback_event(self, msg_json: dict[str, Any], nonce: str) -> tuple[Response, int]: async def _handle_feedback_event(self, msg_json: dict[str, Any], nonce: str) -> tuple[Response, int]:
"""处理企业微信用户反馈事件(点赞/点踩)。 """处理企业微信用户反馈事件(点赞/点踩)。
@@ -1434,29 +1114,6 @@ class WecomBotClient:
self.stream_sessions.mark_finished(stream_id) self.stream_sessions.mark_finished(stream_id)
return True return True
async def push_form_pause(
self, msg_id: str, form_data: dict, task_id: Optional[str] = None
) -> tuple[bool, Optional[str], Optional[str]]:
"""Attach a Dify human-input pause to the active stream session.
On the next WeCom followup poll, the response switches from
``msgtype: stream`` to ``msgtype: template_card`` (button_interaction)
carrying the buttons. ``task_id`` is auto-generated if not provided
and is what the button-click callback uses to look the session back up.
Returns:
``(ok, stream_id, task_id)``. ``ok`` is False if the
adapter's msg_id maps to no stream session (e.g. non-stream mode).
"""
stream_id = self.stream_sessions.get_stream_id_by_msg(msg_id)
if not stream_id:
return False, None, None
if not task_id:
# WeCom requires task_id [A-Za-z0-9_-@], <= 128 bytes, unique per bot.
task_id = f'dify-{uuid.uuid4().hex[:24]}'
self.stream_sessions.set_pending_form(stream_id, form_data, task_id)
return True, stream_id, task_id
async def set_message(self, msg_id: str, content: str): async def set_message(self, msg_id: str, content: str):
"""兼容旧逻辑:若无法流式返回则缓存最终结果。 """兼容旧逻辑:若无法流式返回则缓存最终结果。
+1 -220
View File
@@ -20,13 +20,7 @@ from typing import Any, Callable, Optional
import aiohttp import aiohttp
from langbot.libs.wecom_ai_bot_api import wecombotevent from langbot.libs.wecom_ai_bot_api import wecombotevent
from langbot.libs.wecom_ai_bot_api.api import ( from langbot.libs.wecom_ai_bot_api.api import parse_wecom_bot_message, StreamSession
parse_wecom_bot_message,
StreamSession,
build_button_interaction_payload,
build_button_interaction_update_card,
extract_template_card_action,
)
from langbot.pkg.platform.logger import EventLogger from langbot.pkg.platform.logger import EventLogger
DEFAULT_WS_URL = 'wss://openws.work.weixin.qq.com' DEFAULT_WS_URL = 'wss://openws.work.weixin.qq.com'
@@ -109,22 +103,6 @@ class WecomBotWsClient:
# msg_id -> feedback_id (for associating feedback with message) # msg_id -> feedback_id (for associating feedback with message)
self._msg_feedback_ids: dict[str, str] = {} # msg_id -> feedback_id self._msg_feedback_ids: dict[str, str] = {} # msg_id -> feedback_id
# Dify human-input pause state for ws mode. Keys are task_id (echoed
# back in template_card_event.TaskId so we can rebuild the session
# context on click).
# task_id -> {form_data, msg_id, user_id, chat_id, stream_id, req_id}
self._pending_forms_by_task: dict[str, dict] = {}
# Reverse: msg_id -> task_id (for cleanup when stream finishes).
self._task_id_by_msg: dict[str, str] = {}
# Optional card-action callback registered by the adapter.
# Signature mirrors the http-mode WecomBotClient:
# async def callback(session, action_id, task_id, raw_event) -> None
self._card_action_callback: Optional[Callable] = None
# Optional `source` block injected into every interactive
# template_card the client builds via `push_form_pause`. Set via
# `set_card_source` from the adapter after reading config.
self.card_source: Optional[dict] = None
# ── Public API ────────────────────────────────────────────────── # ── Public API ──────────────────────────────────────────────────
async def connect(self): async def connect(self):
@@ -258,112 +236,6 @@ class WecomBotWsClient:
} }
return await self._send_reply(req_id, body) return await self._send_reply(req_id, body)
async def reply_template_card(self, req_id: str, card_payload: dict[str, Any]) -> Optional[dict]:
"""Send a template_card (button_interaction etc.) reply.
Args:
req_id: The req_id from the original message frame.
card_payload: Body produced by ``build_button_interaction_payload``;
must contain ``msgtype`` and ``template_card`` keys.
Returns:
ACK frame dict, or None on failure.
"""
return await self._send_reply(req_id, card_payload)
async def update_template_card(
self,
req_id: str,
template_card: dict[str, Any],
) -> Optional[dict]:
"""Update an existing template_card via WebSocket.
Uses the ``aibot_respond_update_msg`` command. Must be called
within 5 seconds of receiving the ``template_card_event`` callback,
using the **same req_id** from that callback.
The ``template_card`` dict should contain ``card_type`` and the
new content fields (e.g. ``main_title``, ``button_list`` with
disabled buttons and ``replace_text``).
Returns:
ACK frame dict, or None on failure.
"""
body: dict[str, Any] = {
'response_type': 'update_template_card',
'template_card': template_card,
}
return await self._send_reply(req_id, body, cmd=CMD_RESPOND_UPDATE)
def set_card_action_callback(self, callback: Callable) -> None:
"""Register the button-click handler.
``async def callback(session, action_id, task_id, raw_event) -> None``
— same signature as the http-mode WecomBotClient version so the
adapter can hand both off to the same coroutine.
"""
self._card_action_callback = callback
def set_card_source(self, source: Optional[dict]) -> None:
"""Set the `source` block injected into every interactive
template_card pushed via `push_form_pause`. Pass None to clear."""
self.card_source = source
async def push_form_pause(
self, msg_id: str, form_data: dict, task_id: Optional[str] = None
) -> tuple[bool, Optional[str], Optional[str]]:
"""Attach a Dify human-input pause to the active stream and send
the button_interaction card immediately.
ws mode has no notion of polled "followup" responses — each reply
is a one-shot frame send. So unlike the http path (which defers
card delivery to the next followup), here we just craft the card
and reply with it on the original req_id. The corresponding stream
session is then torn down so subsequent chunks don't re-send.
Returns:
``(ok, stream_id, task_id)``. ``ok=False`` if no active stream
for this msg_id (e.g. message arrived in non-stream mode).
"""
key = self._stream_ids.get(msg_id)
if not key:
return False, None, None
req_id, stream_id = key.split('|', 1)
if not task_id:
task_id = f'dify-{secrets.token_hex(12)}'
session_info = self._stream_sessions.get(msg_id) or {}
self._pending_forms_by_task[task_id] = {
'form_data': form_data,
'msg_id': msg_id,
'user_id': session_info.get('user_id', ''),
'chat_id': session_info.get('chat_id', ''),
'stream_id': stream_id,
'req_id': req_id,
}
self._task_id_by_msg[msg_id] = task_id
card_payload = build_button_interaction_payload(form_data, task_id, source=self.card_source)
try:
await self.reply_template_card(req_id, card_payload)
except Exception:
await self.logger.error(f'Failed to send button_interaction card: {traceback.format_exc()}')
# Roll back the bookkeeping so the next attempt isn't blocked.
self._pending_forms_by_task.pop(task_id, None)
self._task_id_by_msg.pop(msg_id, None)
return False, stream_id, None
# Tear down the stream — WeCom expects either stream chunks OR a
# template_card, not both on the same req_id. Subsequent
# push_stream_chunk calls for this msg_id become no-ops.
self._stream_ids.pop(msg_id, None)
self._stream_last_content.pop(msg_id, None)
# Keep _stream_sessions so the button callback can still resolve
# user/chat context; it gets cleaned up when the click fires.
return True, stream_id, task_id
async def send_message(self, chat_id: str, content: str, msgtype: str = 'markdown') -> Optional[dict]: async def send_message(self, chat_id: str, content: str, msgtype: str = 'markdown') -> Optional[dict]:
"""Proactively send a message to a specified chat. """Proactively send a message to a specified chat.
@@ -386,23 +258,6 @@ class WecomBotWsClient:
body['text'] = {'content': content} body['text'] = {'content': content}
return await self._send_reply(req_id, body, cmd=CMD_SEND_MSG) return await self._send_reply(req_id, body, cmd=CMD_SEND_MSG)
async def send_template_card(self, chat_id: str, card_payload: dict[str, Any]) -> Optional[dict]:
"""Proactively push a template_card to a chat.
Used for the resumed-workflow path (button click → new query):
synthetic events have no inbound req_id to reply against, so we
fall back to proactive ``aibot_send_msg`` instead of reply mode.
Args:
chat_id: userid (single chat) or chatid (group chat).
card_payload: ``{"msgtype": "template_card", "template_card": {...}}``
as produced by :func:`build_button_interaction_payload`.
"""
req_id = _generate_req_id(CMD_SEND_MSG)
body = dict(card_payload)
body['chatid'] = chat_id
return await self._send_reply(req_id, body, cmd=CMD_SEND_MSG)
async def push_stream_chunk(self, msg_id: str, content: str, is_final: bool = False) -> bool: async def push_stream_chunk(self, msg_id: str, content: str, is_final: bool = False) -> bool:
"""Push a streaming chunk for a given message ID. """Push a streaming chunk for a given message ID.
@@ -425,19 +280,6 @@ class WecomBotWsClient:
if not is_final and content == self._stream_last_content.get(msg_id): if not is_final and content == self._stream_last_content.get(msg_id):
return True return True
# Skip empty/whitespace-only chunks — the runner injects a
# zero-width space ('') as a pass-through when workflow_paused
# fires without any preceding LLM output. WeCom renders that
# as an empty bubble that sits before the form card; skip it.
# NOTE: Python str.strip() does NOT strip , so we use
# a regex that treats any character with Unicode category Zs
# (separator space) or Cf (format char like ZWS) as blank.
if not is_final:
import re as _re
if not _re.sub(r'[\s]', '', content):
return True
# Generate feedback_id for final chunk # Generate feedback_id for final chunk
feedback_id = '' feedback_id = ''
if is_final: if is_final:
@@ -726,67 +568,6 @@ class WecomBotWsClient:
await self.logger.error(f'Error in feedback handler: {traceback.format_exc()}') await self.logger.error(f'Error in feedback handler: {traceback.format_exc()}')
return return
if event_type == 'template_card_event':
tce = event_info.get('template_card_event', {})
task_id, event_key, card_type = extract_template_card_action(tce)
await self.logger.info(
f'收到按钮点击 (ws): task_id={task_id} event_key={event_key!r} card_type={card_type}'
)
pending = self._pending_forms_by_task.get(task_id)
if pending is None:
await self.logger.warning(f'未找到 task_id={task_id} 对应的 pending_form (ws),按钮点击被丢弃')
else:
# Update the card in-place to show which button was clicked.
# Must happen within 5s of the event, using the same req_id.
req_id_for_update = frame.get('headers', {}).get('req_id', '')
form_data = pending.get('form_data', {}) or {}
action_title = event_key
node_title = form_data.get('node_title', '') or ''
main_title_text = f'{node_title} - 已选择' if node_title else '已选择'
main_title_desc = f'{action_title}'
update_card = {
'card_type': 'button_interaction',
'main_title': {
'title': main_title_text,
'desc': main_title_desc,
},
'button_list': [],
'task_id': task_id,
}
if self.card_source:
update_card['source'] = self.card_source
update_card = build_button_interaction_update_card(
form_data,
task_id,
event_key,
source=self.card_source,
)
try:
await self.update_template_card(req_id_for_update, update_card)
except Exception:
await self.logger.warning(f'更新卡片失败 (ws): {traceback.format_exc()}')
if self._card_action_callback is not None:
try:
session = StreamSession(
stream_id=pending.get('stream_id', ''),
msg_id=pending.get('msg_id', ''),
chat_id=pending.get('chat_id') or None,
user_id=pending.get('user_id') or None,
)
session.pending_form = pending.get('form_data')
session.pending_form_task_id = task_id
await self._card_action_callback(session, event_key, task_id, body)
except Exception:
await self.logger.error(f'card action callback raised (ws): {traceback.format_exc()}')
# Consume — drop bookkeeping so a stale click can't re-fire.
self._pending_forms_by_task.pop(task_id, None)
msg_id = pending.get('msg_id', '')
if msg_id:
self._task_id_by_msg.pop(msg_id, None)
self._stream_sessions.pop(msg_id, None)
return
event = wecombotevent.WecomBotEvent(message_data) event = wecombotevent.WecomBotEvent(message_data)
if event_type in self._message_handlers: if event_type in self._message_handlers:
+4
View File
@@ -0,0 +1,4 @@
from .client import AsyncWeKnoraClient
from .errors import WeKnoraAPIError
__all__ = ['AsyncWeKnoraClient', 'WeKnoraAPIError']
+180
View File
@@ -0,0 +1,180 @@
from __future__ import annotations
import httpx
import typing
import json
from .errors import WeKnoraAPIError
class AsyncWeKnoraClient:
"""WeKnora API 客户端"""
api_key: str
base_url: str
def __init__(
self,
api_key: str,
base_url: str = 'http://localhost:80/api/v1',
) -> None:
self.api_key = api_key
self.base_url = base_url
async def create_session(
self,
title: str = '',
description: str = '',
timeout: float = 30.0,
) -> str:
"""创建会话,返回 session_id"""
async with httpx.AsyncClient(
base_url=self.base_url,
trust_env=True,
timeout=timeout,
) as client:
payload: dict[str, typing.Any] = {}
if title:
payload['title'] = title
if description:
payload['description'] = description
response = await client.post(
'/sessions',
headers={
'X-API-Key': self.api_key,
'Content-Type': 'application/json',
},
json=payload,
)
if response.status_code not in (200, 201):
raise WeKnoraAPIError(f'{response.status_code} {response.text}')
data = response.json()
return data['data']['id']
async def agent_chat(
self,
session_id: str,
query: str,
user: str,
agent_id: str = '',
knowledge_base_ids: list[str] | None = None,
web_search_enabled: bool = False,
timeout: float = 120.0,
) -> typing.AsyncGenerator[dict[str, typing.Any], None]:
"""
Agent 智能对话(SSE 流式)
响应事件类型:
- agent_query: Agent 开始处理
- thinking: 思考过程
- tool_call: 工具调用
- tool_result: 工具结果
- references: 知识库引用
- answer: 回答内容
- reflection: 反思
- session_title: 会话标题
- error: 错误
"""
if knowledge_base_ids is None:
knowledge_base_ids = []
async with httpx.AsyncClient(
base_url=self.base_url,
trust_env=True,
timeout=timeout,
) as client:
payload: dict[str, typing.Any] = {
'query': query,
'agent_enabled': True,
'channel': 'im',
}
if agent_id:
payload['agent_id'] = agent_id
if knowledge_base_ids:
payload['knowledge_base_ids'] = knowledge_base_ids
if web_search_enabled:
payload['web_search_enabled'] = True
async with client.stream(
'POST',
f'/agent-chat/{session_id}',
headers={
'X-API-Key': self.api_key,
'Content-Type': 'application/json',
},
json=payload,
) as r:
async for chunk in r.aiter_lines():
if r.status_code != 200:
raise WeKnoraAPIError(f'{r.status_code} {chunk}')
if chunk.strip() == '':
continue
if chunk.startswith('data:'):
try:
data = json.loads(chunk[5:].strip())
except json.JSONDecodeError:
continue
yield data
# 收到 error 事件后主动结束流,避免上层未 raise 时持续等待
if data.get('response_type') == 'error':
return
async def knowledge_chat(
self,
session_id: str,
query: str,
user: str,
agent_id: str = 'builtin-quick-answer',
knowledge_base_ids: list[str] | None = None,
timeout: float = 120.0,
) -> typing.AsyncGenerator[dict[str, typing.Any], None]:
"""
知识库 RAG 问答(SSE 流式)
响应事件类型:
- references: 知识库引用
- answer: 回答内容
"""
if knowledge_base_ids is None:
knowledge_base_ids = []
async with httpx.AsyncClient(
base_url=self.base_url,
trust_env=True,
timeout=timeout,
) as client:
payload: dict[str, typing.Any] = {
'query': query,
'channel': 'im',
}
if agent_id:
payload['agent_id'] = agent_id
if knowledge_base_ids:
payload['knowledge_base_ids'] = knowledge_base_ids
async with client.stream(
'POST',
f'/knowledge-chat/{session_id}',
headers={
'X-API-Key': self.api_key,
'Content-Type': 'application/json',
},
json=payload,
) as r:
async for chunk in r.aiter_lines():
if r.status_code != 200:
raise WeKnoraAPIError(f'{r.status_code} {chunk}')
if chunk.strip() == '':
continue
if chunk.startswith('data:'):
try:
data = json.loads(chunk[5:].strip())
except json.JSONDecodeError:
continue
yield data
# 收到 error 事件后主动结束流,避免上层未 raise 时持续等待
if data.get('response_type') == 'error':
return
+6
View File
@@ -0,0 +1,6 @@
class WeKnoraAPIError(Exception):
"""WeKnora API 请求失败"""
def __init__(self, message: str = ''):
self.message = message
super().__init__(self.message)
@@ -0,0 +1,22 @@
from __future__ import annotations
from .. import group
@group.group_class('box', '/api/v1/box')
class BoxRouterGroup(group.RouterGroup):
async def initialize(self) -> None:
@self.route('/status', methods=['GET'], auth_type=group.AuthType.USER_TOKEN)
async def _() -> str:
status = await self.ap.box_service.get_status()
return self.success(data=status)
@self.route('/sessions', methods=['GET'], auth_type=group.AuthType.USER_TOKEN)
async def _() -> str:
sessions = await self.ap.box_service.get_sessions()
return self.success(data=sessions)
@self.route('/errors', methods=['GET'], auth_type=group.AuthType.USER_TOKEN)
async def _() -> str:
errors = self.ap.box_service.get_recent_errors()
return self.success(data=errors)
@@ -0,0 +1,52 @@
from __future__ import annotations
import asyncio
import quart
from .. import group
@group.group_class('extensions', '/api/v1/extensions')
class ExtensionsRouterGroup(group.RouterGroup):
"""Unified API for installed extensions (plugins, MCP servers, skills)."""
async def initialize(self) -> None:
@self.route('', methods=['GET'], auth_type=group.AuthType.USER_TOKEN_OR_API_KEY)
async def _() -> quart.Response:
plugins, mcp_servers, skills = await asyncio.gather(
self.ap.plugin_connector.list_plugins(),
self.ap.mcp_service.get_mcp_servers(contain_runtime_info=True),
self.ap.skill_service.list_skills(),
return_exceptions=True,
)
def _sort_key(item: dict) -> str:
if item['type'] == 'plugin':
return (
item['plugin']
.get('manifest', {})
.get('manifest', {})
.get('metadata', {})
.get('name', '')
.lower()
)
if item['type'] == 'mcp':
return (item['server'].get('name') or '').lower()
if item['type'] == 'skill':
return (item['skill'].get('display_name') or item['skill'].get('name') or '').lower()
return ''
extensions: list[dict] = []
if isinstance(plugins, list):
for plugin in plugins:
extensions.append({'type': 'plugin', 'plugin': plugin})
if isinstance(mcp_servers, list):
for server in mcp_servers:
extensions.append({'type': 'mcp', 'server': server})
if isinstance(skills, list):
for skill in skills:
extensions.append({'type': 'skill', 'skill': skill})
extensions.sort(key=_sort_key)
return self.success(data={'extensions': extensions})
@@ -73,15 +73,21 @@ class PipelinesRouterGroup(group.RouterGroup):
plugins = await self.ap.plugin_connector.list_plugins(component_kinds=pipeline_component_kinds) plugins = await self.ap.plugin_connector.list_plugins(component_kinds=pipeline_component_kinds)
mcp_servers = await self.ap.mcp_service.get_mcp_servers(contain_runtime_info=True) mcp_servers = await self.ap.mcp_service.get_mcp_servers(contain_runtime_info=True)
# Get available skills
available_skills = await self.ap.skill_service.list_skills()
extensions_prefs = pipeline.get('extensions_preferences', {}) extensions_prefs = pipeline.get('extensions_preferences', {})
return self.success( return self.success(
data={ data={
'enable_all_plugins': extensions_prefs.get('enable_all_plugins', True), 'enable_all_plugins': extensions_prefs.get('enable_all_plugins', True),
'enable_all_mcp_servers': extensions_prefs.get('enable_all_mcp_servers', True), 'enable_all_mcp_servers': extensions_prefs.get('enable_all_mcp_servers', True),
'enable_all_skills': extensions_prefs.get('enable_all_skills', True),
'bound_plugins': extensions_prefs.get('plugins', []), 'bound_plugins': extensions_prefs.get('plugins', []),
'available_plugins': plugins, 'available_plugins': plugins,
'bound_mcp_servers': extensions_prefs.get('mcp_servers', []), 'bound_mcp_servers': extensions_prefs.get('mcp_servers', []),
'available_mcp_servers': mcp_servers, 'available_mcp_servers': mcp_servers,
'bound_skills': extensions_prefs.get('skills', []),
'available_skills': available_skills,
} }
) )
elif quart.request.method == 'PUT': elif quart.request.method == 'PUT':
@@ -89,11 +95,19 @@ class PipelinesRouterGroup(group.RouterGroup):
json_data = await quart.request.json json_data = await quart.request.json
enable_all_plugins = json_data.get('enable_all_plugins', True) enable_all_plugins = json_data.get('enable_all_plugins', True)
enable_all_mcp_servers = json_data.get('enable_all_mcp_servers', True) enable_all_mcp_servers = json_data.get('enable_all_mcp_servers', True)
enable_all_skills = json_data.get('enable_all_skills', True)
bound_plugins = json_data.get('bound_plugins', []) bound_plugins = json_data.get('bound_plugins', [])
bound_mcp_servers = json_data.get('bound_mcp_servers', []) bound_mcp_servers = json_data.get('bound_mcp_servers', [])
bound_skills = json_data.get('bound_skills', [])
await self.ap.pipeline_service.update_pipeline_extensions( await self.ap.pipeline_service.update_pipeline_extensions(
pipeline_uuid, bound_plugins, bound_mcp_servers, enable_all_plugins, enable_all_mcp_servers pipeline_uuid,
bound_plugins,
bound_mcp_servers,
enable_all_plugins,
enable_all_mcp_servers,
bound_skills=bound_skills,
enable_all_skills=enable_all_skills,
) )
return self.success() return self.success()
@@ -43,8 +43,12 @@ class WebSocketChatRouterGroup(group.RouterGroup):
await quart.websocket.send(json.dumps({'type': 'error', 'message': 'WebSocket adapter not found'})) await quart.websocket.send(json.dumps({'type': 'error', 'message': 'WebSocket adapter not found'}))
return return
# Find the owning bot for this pipeline (e.g. a web_page_bot) # Dashboard pipeline-debug sessions must always run under the
owner_bot = self._find_owner_bot(pipeline_uuid) # built-in websocket_proxy_bot identity. We deliberately do NOT
# resolve a web_page_bot owner here — even if one is bound to
# the same pipeline, debug requests must not be attributed to
# it. The embed widget path (`/api/v1/embed/<bot>/ws/connect`)
# is the one that carries the page-bot identity.
# 注册连接 # 注册连接
connection = await ws_connection_manager.add_connection( connection = await ws_connection_manager.add_connection(
@@ -73,7 +77,7 @@ class WebSocketChatRouterGroup(group.RouterGroup):
) )
# 创建接收和发送任务 # 创建接收和发送任务
receive_task = asyncio.create_task(self._handle_receive(connection, websocket_adapter, owner_bot)) receive_task = asyncio.create_task(self._handle_receive(connection, websocket_adapter))
send_task = asyncio.create_task(self._handle_send(connection)) send_task = asyncio.create_task(self._handle_send(connection))
# 等待任务完成 # 等待任务完成
@@ -181,14 +185,7 @@ class WebSocketChatRouterGroup(group.RouterGroup):
except Exception as e: except Exception as e:
return self.http_status(500, -1, f'Internal server error: {str(e)}') return self.http_status(500, -1, f'Internal server error: {str(e)}')
def _find_owner_bot(self, pipeline_uuid: str): async def _handle_receive(self, connection, websocket_adapter):
"""Find a user-created bot (e.g. web_page_bot) that owns this pipeline."""
for bot in self.ap.platform_mgr.bots:
if bot.bot_entity.adapter == 'web_page_bot' and bot.bot_entity.use_pipeline_uuid == pipeline_uuid:
return bot
return None
async def _handle_receive(self, connection, websocket_adapter, owner_bot=None):
"""处理接收消息的任务""" """处理接收消息的任务"""
try: try:
while connection.is_active: while connection.is_active:
@@ -213,7 +210,10 @@ class WebSocketChatRouterGroup(group.RouterGroup):
logger.debug(f'收到消息: {data} from {connection.connection_id}') logger.debug(f'收到消息: {data} from {connection.connection_id}')
# 处理消息(不等待响应,响应会通过broadcast异步发送) # 处理消息(不等待响应,响应会通过broadcast异步发送)
await websocket_adapter.handle_websocket_message(connection, data, owner_bot=owner_bot) # owner_bot is intentionally NOT passed: the dashboard
# debug WebSocket must always run under the proxy bot,
# never under a coincidentally-bound web_page_bot.
await websocket_adapter.handle_websocket_message(connection, data)
elif message_type == 'disconnect': elif message_type == 'disconnect':
# 客户端主动断开 # 客户端主动断开
@@ -5,29 +5,6 @@ from ... import group
from langbot.pkg.utils import importutil from langbot.pkg.utils import importutil
def _decrypt_qqofficial_secret(encrypted_b64: str, key: bytes) -> str:
"""Decrypt the AppSecret returned by the QQ Official QR binding endpoint.
The base64 payload is laid out as `nonce (12 B) | ciphertext | tag (16 B)`.
`key` is the 32-byte AES-256 key locally generated when the bind task
was created and submitted as `key` to `q.qq.com/lite/create_bind_task`.
"""
import base64
from cryptography.hazmat.primitives.ciphers.aead import AESGCM
try:
raw = base64.b64decode(encrypted_b64)
except Exception as exc:
raise ValueError('Malformed encrypted credential') from exc
if len(key) != 32 or len(raw) <= 28:
raise ValueError('Invalid encrypted credential layout')
nonce, ciphertext, tag = raw[:12], raw[12:-16], raw[-16:]
try:
return AESGCM(key).decrypt(nonce, ciphertext + tag, None).decode('utf-8')
except Exception as exc:
raise ValueError('Failed to decrypt credential') from exc
@group.group_class('adapters', '/api/v1/platform/adapters') @group.group_class('adapters', '/api/v1/platform/adapters')
class AdaptersRouterGroup(group.RouterGroup): class AdaptersRouterGroup(group.RouterGroup):
async def initialize(self) -> None: async def initialize(self) -> None:
@@ -60,15 +37,6 @@ class AdaptersRouterGroup(group.RouterGroup):
importutil.read_resource_file_bytes(icon_path), mimetype=mimetypes.guess_type(icon_path)[0] importutil.read_resource_file_bytes(icon_path), mimetype=mimetypes.guess_type(icon_path)[0]
) )
@self.route('/dingtalk/human-input-card-template', methods=['GET'], auth_type=group.AuthType.NONE)
async def _() -> quart.Response:
filename = 'dingtalk_human_input_card.json'
response = quart.Response(
importutil.read_resource_file_bytes(f'templates/{filename}'), mimetype='application/json'
)
response.headers['Content-Disposition'] = f'attachment; filename={filename}'
return response
# In-memory session store for active registrations # In-memory session store for active registrations
_create_app_sessions: dict = {} _create_app_sessions: dict = {}
_SESSION_TTL = 900 # 15 minutes _SESSION_TTL = 900 # 15 minutes
@@ -682,220 +650,3 @@ class AdaptersRouterGroup(group.RouterGroup):
if session and session.get('task') and not session['task'].done(): if session and session.get('task') and not session['task'].done():
session['task'].cancel() session['task'].cancel()
return self.success(data={}) return self.success(data={})
# -----------------------------------------------------------------------
# QQ Official QR Binding
# -----------------------------------------------------------------------
_qqofficial_sessions: dict = {}
_QQOFFICIAL_SESSION_TTL = 300 # 5 minutes (QQ bind QR validity window)
def _cleanup_expired_qqofficial_sessions():
import time
now = time.time()
expired = [
sid for sid, s in _qqofficial_sessions.items() if now - s.get('created_at', 0) > _QQOFFICIAL_SESSION_TTL
]
for sid in expired:
session = _qqofficial_sessions.pop(sid, None)
if session and session.get('task') and not session['task'].done():
session['task'].cancel()
@self.route('/qqofficial/bind', methods=['POST'])
async def _() -> str:
"""Start QQ Official QR binding. Returns session_id + QR URL.
Flow: generate a local AES-256 key, register it with
`q.qq.com/lite/create_bind_task`, then poll
`q.qq.com/lite/poll_bind_result` until the user authorizes the
bind inside the QQ Bot Assistant on mobile QQ. The encrypted
AppSecret returned by the poll endpoint is decrypted with the
same key. The key never leaves this process.
"""
import uuid
import time
import secrets
import base64
import aiohttp
QQ_BIND_BASE = 'https://q.qq.com'
_cleanup_expired_qqofficial_sessions()
bind_key_bytes = secrets.token_bytes(32)
bind_key = base64.b64encode(bind_key_bytes).decode('ascii')
session_id = str(uuid.uuid4())
session = {
'status': 'pending',
'qr_url': None,
'expire_at': None,
'appid': None,
'secret': None,
'user_openid': None,
'error': None,
'created_at': time.time(),
'task_id': None,
'bind_key_bytes': bind_key_bytes,
'interval': 2,
}
_qqofficial_sessions[session_id] = session
async def run_qr_binding():
try:
timeout = aiohttp.ClientTimeout(total=10)
async with aiohttp.ClientSession(timeout=timeout) as http:
# Step 1: create_bind_task — register our AES key, get task_id
async with http.post(
f'{QQ_BIND_BASE}/lite/create_bind_task',
json={'key': bind_key},
headers={'Accept': 'application/json'},
) as resp:
try:
data = await resp.json(content_type=None)
except (aiohttp.ContentTypeError, ValueError):
session['status'] = 'error'
session['error'] = 'Invalid response from QQ bind service'
return
if int(data.get('retcode', -1)) != 0:
session['status'] = 'error'
session['error'] = (
data.get('msg') or data.get('message') or 'Failed to create bind task'
)
return
task_id = str((data.get('data') or {}).get('task_id') or '').strip()
if not task_id:
session['status'] = 'error'
session['error'] = 'Missing task_id in QQ response'
return
# The QR encodes a URL that mobile QQ opens inside the QQ Bot Assistant.
# `source=langbot` is a courtesy attribution parameter so Tencent
# can see LangBot adoption metrics, matching the convention used by
# other third-party integrations (e.g. hermes-agent uses `source=hermes`).
qr_url = f'{QQ_BIND_BASE}/qqbot/openclaw/connect.html?task_id={task_id}&_wv=2&source=langbot'
session['task_id'] = task_id
session['qr_url'] = qr_url
session['expire_at'] = time.time() + _QQOFFICIAL_SESSION_TTL
session['status'] = 'waiting'
# Step 2: poll_bind_result until completed (status=2) or expired (3).
deadline = time.time() + _QQOFFICIAL_SESSION_TTL
while time.time() < deadline:
await asyncio.sleep(session['interval'])
async with http.post(
f'{QQ_BIND_BASE}/lite/poll_bind_result',
json={'task_id': task_id},
headers={'Accept': 'application/json'},
) as poll_resp:
try:
poll_data = await poll_resp.json(content_type=None)
except (aiohttp.ContentTypeError, ValueError):
continue
if int(poll_data.get('retcode', -1)) != 0:
session['status'] = 'error'
session['error'] = poll_data.get('msg') or poll_data.get('message') or 'Poll failed'
return
payload = poll_data.get('data') or {}
try:
raw_status = int(payload.get('status', 0))
except (TypeError, ValueError):
raw_status = 0
if raw_status == 2:
appid = str(payload.get('bot_appid') or '').strip()
encrypted = str(payload.get('bot_encrypt_secret') or '').strip()
if not appid or not encrypted:
session['status'] = 'error'
session['error'] = 'Incomplete credential payload'
return
try:
session['secret'] = _decrypt_qqofficial_secret(
encrypted,
bind_key_bytes,
)
except ValueError as exc:
session['status'] = 'error'
session['error'] = str(exc)
return
session['appid'] = appid
# The scanner's OpenID is returned alongside the credentials —
# surfaced to the dashboard for audit / "bound by" display.
session['user_openid'] = str(payload.get('user_openid') or '').strip() or None
session['status'] = 'success'
return
if raw_status == 3:
session['status'] = 'expired'
session['error'] = 'QR code expired'
return
# status 0 / 1: still pending, continue polling
session['status'] = 'expired'
session['error'] = 'QR code expired'
except asyncio.CancelledError:
return
except Exception as e:
session['status'] = 'error'
session['error'] = str(e)
task = asyncio.create_task(run_qr_binding())
session['task'] = task
# Wait up to 10s for the QR URL to be ready before responding.
for _ in range(20):
if session['qr_url'] or session['error']:
break
await asyncio.sleep(0.5)
if session['error']:
task.cancel()
return self.http_status(502, -1, session['error'])
if not session['qr_url']:
task.cancel()
session['status'] = 'error'
session['error'] = 'Timeout waiting for QR code'
return self.http_status(504, -1, 'Timeout waiting for QR code')
return self.success(
data={
'session_id': session_id,
'qr_url': session['qr_url'],
'expire_at': session['expire_at'],
}
)
@self.route('/qqofficial/bind/status/<session_id>', methods=['GET'])
async def _(session_id: str) -> str:
"""Poll QQ Official QR binding status."""
_cleanup_expired_qqofficial_sessions()
session = _qqofficial_sessions.get(session_id)
if not session:
return self.http_status(404, -1, 'Session not found')
data = {'status': session['status']}
if session['status'] == 'success':
data['appid'] = session['appid']
data['secret'] = session['secret']
if session.get('user_openid'):
data['user_openid'] = session['user_openid']
_qqofficial_sessions.pop(session_id, None)
elif session['status'] in ('error', 'expired'):
data['error'] = session['error']
_qqofficial_sessions.pop(session_id, None)
return self.success(data=data)
@self.route('/qqofficial/bind/<session_id>', methods=['DELETE'])
async def _(session_id: str) -> str:
"""Cancel and clean up a QQ Official QR binding session."""
session = _qqofficial_sessions.pop(session_id, None)
if session and session.get('task') and not session['task'].done():
session['task'].cancel()
return self.success(data={})
@@ -1,11 +1,15 @@
from __future__ import annotations from __future__ import annotations
import base64 import base64
import io
import quart import quart
import re import re
import httpx import httpx
import uuid import uuid
import os import os
import zipfile
import yaml
from urllib.parse import urlparse
import posixpath import posixpath
import sqlalchemy import sqlalchemy
@@ -53,6 +57,97 @@ def _get_request_origin() -> str:
@group.group_class('plugins', '/api/v1/plugins') @group.group_class('plugins', '/api/v1/plugins')
class PluginsRouterGroup(group.RouterGroup): class PluginsRouterGroup(group.RouterGroup):
@staticmethod
def _normalize_archive_path(path: str) -> str:
normalized = str(path or '').replace('\\', '/').strip('/')
return posixpath.normpath(normalized) if normalized else ''
@classmethod
def _component_source_path(cls, entry) -> str:
if isinstance(entry, dict):
return cls._normalize_archive_path(entry.get('path') or '')
return cls._normalize_archive_path(str(entry or ''))
@classmethod
def _count_component_configs(cls, component_config, archive_names: list[str]) -> int:
normalized_names = [cls._normalize_archive_path(name) for name in archive_names]
component_files: set[str] = set()
if isinstance(component_config, list):
return len(component_config)
if not isinstance(component_config, dict):
return 1 if component_config else 0
for entry in component_config.get('fromFiles') or []:
source_path = cls._component_source_path(entry)
if source_path and source_path in normalized_names:
component_files.add(source_path)
for entry in component_config.get('fromDirs') or []:
source_dir = cls._component_source_path(entry).rstrip('/')
if not source_dir:
continue
prefix = f'{source_dir}/'
for archive_name in normalized_names:
if not archive_name.startswith(prefix):
continue
if archive_name.lower().endswith(('.yaml', '.yml')):
component_files.add(archive_name)
if component_files:
return len(component_files)
return 1 if any(key in component_config for key in ('path', 'name', 'kind')) else 0
@classmethod
def _count_plugin_components(cls, components, archive_names: list[str]) -> dict[str, int]:
if not isinstance(components, dict):
return {}
component_counts: dict[str, int] = {}
for kind, component_config in components.items():
count = cls._count_component_configs(component_config, archive_names)
if count > 0:
component_counts[str(kind)] = count
return component_counts
@staticmethod
def _parse_github_repo_url(repo_url: str) -> dict | None:
raw_url = str(repo_url or '').strip()
if not raw_url:
return None
if not re.match(r'^[a-zA-Z][a-zA-Z0-9+.-]*://', raw_url):
raw_url = f'https://{raw_url}'
parsed = urlparse(raw_url)
if parsed.netloc.lower() not in ('github.com', 'www.github.com'):
return None
parts = [part for part in parsed.path.strip('/').split('/') if part]
if len(parts) < 2:
return None
owner = parts[0]
repo = parts[1]
if repo.endswith('.git'):
repo = repo[:-4]
if not owner or not repo:
return None
ref = ''
subdir = ''
if len(parts) >= 4 and parts[2] in ('tree', 'blob'):
ref = parts[3]
subdir = '/'.join(parts[4:]).strip('/')
return {
'owner': owner,
'repo': repo,
'ref': ref,
'subdir': subdir,
}
async def _check_extensions_limit(self) -> str | None: async def _check_extensions_limit(self) -> str | None:
"""Check if extensions limit is reached. Returns error response if limit exceeded, None otherwise.""" """Check if extensions limit is reached. Returns error response if limit exceeded, None otherwise."""
limitation = self.ap.instance_config.data.get('system', {}).get('limitation', {}) limitation = self.ap.instance_config.data.get('system', {}).get('limitation', {})
@@ -254,17 +349,37 @@ class PluginsRouterGroup(group.RouterGroup):
data = await quart.request.json data = await quart.request.json
repo_url = data.get('repo_url', '') repo_url = data.get('repo_url', '')
# Parse GitHub repository URL to extract owner and repo parsed_repo = self._parse_github_repo_url(repo_url)
# Supports: https://github.com/owner/repo or github.com/owner/repo if not parsed_repo:
pattern = r'github\.com/([^/]+)/([^/]+?)(?:\.git)?(?:/.*)?$'
match = re.search(pattern, repo_url)
if not match:
return self.http_status(400, -1, 'Invalid GitHub repository URL') return self.http_status(400, -1, 'Invalid GitHub repository URL')
owner, repo = match.groups() owner = parsed_repo['owner']
repo = parsed_repo['repo']
requested_ref = parsed_repo['ref']
requested_subdir = parsed_repo['subdir']
try: try:
if requested_ref:
return self.success(
data={
'releases': [
{
'id': 0,
'tag_name': requested_ref,
'name': requested_ref,
'published_at': '',
'prerelease': False,
'draft': False,
'source_type': 'branch',
'archive_url': f'https://api.github.com/repos/{owner}/{repo}/zipball/{requested_ref}',
}
],
'owner': owner,
'repo': repo,
'source_subdir': requested_subdir,
}
)
# Fetch releases from GitHub API # Fetch releases from GitHub API
url = f'https://api.github.com/repos/{owner}/{repo}/releases' url = f'https://api.github.com/repos/{owner}/{repo}/releases'
async with httpx.AsyncClient( async with httpx.AsyncClient(
@@ -290,7 +405,14 @@ class PluginsRouterGroup(group.RouterGroup):
} }
) )
return self.success(data={'releases': formatted_releases, 'owner': owner, 'repo': repo}) return self.success(
data={
'releases': formatted_releases,
'owner': owner,
'repo': repo,
'source_subdir': requested_subdir,
}
)
except httpx.RequestError as e: except httpx.RequestError as e:
return self.http_status(500, -1, f'Failed to fetch releases: {str(e)}') return self.http_status(500, -1, f'Failed to fetch releases: {str(e)}')
@@ -445,6 +567,62 @@ class PluginsRouterGroup(group.RouterGroup):
return self.success(data={'task_id': wrapper.id}) return self.success(data={'task_id': wrapper.id})
@self.route('/install/local/preview', methods=['POST'], auth_type=group.AuthType.USER_TOKEN_OR_API_KEY)
async def _() -> str:
file = (await quart.request.files).get('file')
if file is None:
return self.http_status(400, -1, 'file is required')
file_bytes = file.read()
try:
with zipfile.ZipFile(io.BytesIO(file_bytes)) as zf:
names = [name for name in zf.namelist() if not name.endswith('/')]
manifest_name = next(
(
name
for name in names
if name.replace('\\', '/').strip('/').lower() in ('manifest.yaml', 'manifest.yml')
),
None,
)
if manifest_name is None:
return self.http_status(400, -1, 'manifest.yaml is required')
manifest = yaml.safe_load(zf.read(manifest_name).decode('utf-8')) or {}
requirements: list[str] = []
requirements_name = next(
(name for name in names if name.replace('\\', '/').strip('/').lower() == 'requirements.txt'),
None,
)
if requirements_name is not None:
requirements = [
line.strip()
for line in zf.read(requirements_name).decode('utf-8', errors='ignore').splitlines()
if line.strip() and not line.strip().startswith('#')
]
spec = manifest.get('spec') or {}
components = spec.get('components') or {}
component_counts = self._count_plugin_components(components, names)
component_types = list(component_counts.keys())
return self.success(
data={
'filename': file.filename or 'local plugin',
'size': len(file_bytes),
'manifest': manifest,
'metadata': manifest.get('metadata') or {},
'component_types': component_types,
'component_counts': component_counts,
'requirements': requirements,
'file_count': len(names),
}
)
except zipfile.BadZipFile:
return self.http_status(400, -1, 'invalid .lbpkg file')
except Exception as exc:
return self.http_status(500, -1, f'Failed to preview plugin package: {exc}')
@self.route('/config-files', methods=['POST'], auth_type=group.AuthType.USER_TOKEN) @self.route('/config-files', methods=['POST'], auth_type=group.AuthType.USER_TOKEN)
async def _() -> str: async def _() -> str:
"""Upload a file for plugin configuration""" """Upload a file for plugin configuration"""
@@ -31,6 +31,9 @@ class MCPRouterGroup(group.RouterGroup):
@self.route('/servers/<server_name>', methods=['GET', 'PUT', 'DELETE'], auth_type=group.AuthType.USER_TOKEN) @self.route('/servers/<server_name>', methods=['GET', 'PUT', 'DELETE'], auth_type=group.AuthType.USER_TOKEN)
async def _(server_name: str) -> str: async def _(server_name: str) -> str:
"""获取、更新或删除MCP服务器配置""" """获取、更新或删除MCP服务器配置"""
from urllib.parse import unquote
server_name = unquote(server_name)
server_data = await self.ap.mcp_service.get_mcp_server_by_name(server_name) server_data = await self.ap.mcp_service.get_mcp_server_by_name(server_name)
if server_data is None: if server_data is None:
@@ -57,6 +60,9 @@ class MCPRouterGroup(group.RouterGroup):
@self.route('/servers/<server_name>/test', methods=['POST'], auth_type=group.AuthType.USER_TOKEN) @self.route('/servers/<server_name>/test', methods=['POST'], auth_type=group.AuthType.USER_TOKEN)
async def _(server_name: str) -> str: async def _(server_name: str) -> str:
"""测试MCP服务器连接""" """测试MCP服务器连接"""
from urllib.parse import unquote
server_name = unquote(server_name)
server_data = await quart.request.json server_data = await quart.request.json
task_id = await self.ap.mcp_service.test_mcp_server(server_name=server_name, server_data=server_data) task_id = await self.ap.mcp_service.test_mcp_server(server_name=server_name, server_data=server_data)
return self.success(data={'task_id': task_id}) return self.success(data={'task_id': task_id})
@@ -0,0 +1,190 @@
from __future__ import annotations
import quart
from langbot_plugin.box.errors import BoxError
from .. import group
@group.group_class('skills', '/api/v1/skills')
class SkillsRouterGroup(group.RouterGroup):
"""Skills management API endpoints."""
async def initialize(self) -> None:
@self.route('', methods=['GET', 'POST'], auth_type=group.AuthType.USER_TOKEN_OR_API_KEY)
async def list_or_create_skills() -> quart.Response:
if quart.request.method == 'GET':
try:
skills = await self.ap.skill_service.list_skills()
except (ValueError, BoxError) as exc:
return self.http_status(400, -1, str(exc))
return self.success(data={'skills': skills})
data = await quart.request.json
if 'name' not in data or not data['name']:
return self.http_status(400, -1, 'Missing required field: name')
try:
skill = await self.ap.skill_service.create_skill(data)
return self.success(data={'skill': skill})
except (ValueError, BoxError) as exc:
return self.http_status(400, -1, str(exc))
@self.route('/<skill_name>', methods=['GET', 'PUT', 'DELETE'], auth_type=group.AuthType.USER_TOKEN_OR_API_KEY)
async def get_update_delete_skill(skill_name: str) -> quart.Response:
if quart.request.method == 'GET':
try:
skill = await self.ap.skill_service.get_skill(skill_name)
except (ValueError, BoxError) as exc:
return self.http_status(400, -1, str(exc))
if not skill:
return self.http_status(404, -1, 'Skill not found')
return self.success(data={'skill': skill})
if quart.request.method == 'PUT':
data = await quart.request.json
try:
skill = await self.ap.skill_service.update_skill(skill_name, data)
return self.success(data={'skill': skill})
except (ValueError, BoxError) as exc:
return self.http_status(400, -1, str(exc))
try:
await self.ap.skill_service.delete_skill(skill_name)
return self.success()
except (ValueError, BoxError) as exc:
return self.http_status(400, -1, str(exc))
@self.route('/<skill_name>/files', methods=['GET'], auth_type=group.AuthType.USER_TOKEN_OR_API_KEY)
async def list_skill_files(skill_name: str) -> quart.Response:
"""List files in skill package directory."""
path = quart.request.args.get('path', '.').strip()
include_hidden = quart.request.args.get('include_hidden', 'false').lower() == 'true'
try:
result = await self.ap.skill_service.list_skill_files(
skill_name,
path=path,
include_hidden=include_hidden,
)
return self.success(data=result)
except (ValueError, BoxError) as exc:
return self.http_status(400, -1, str(exc))
@self.route(
'/<skill_name>/files/<path:path>', methods=['GET', 'PUT'], auth_type=group.AuthType.USER_TOKEN_OR_API_KEY
)
async def read_or_write_skill_file(skill_name: str, path: str) -> quart.Response:
"""Read or write a file in skill package."""
if quart.request.method == 'GET':
try:
result = await self.ap.skill_service.read_skill_file(skill_name, path)
return self.success(data=result)
except (ValueError, BoxError) as exc:
return self.http_status(400, -1, str(exc))
# PUT - write file
data = await quart.request.json
content = data.get('content', '')
if content is None:
return self.http_status(400, -1, 'Missing required field: content')
try:
result = await self.ap.skill_service.write_skill_file(skill_name, path, content)
return self.success(data=result)
except (ValueError, BoxError) as exc:
return self.http_status(400, -1, str(exc))
@self.route('/<skill_name>/preview', methods=['GET'], auth_type=group.AuthType.USER_TOKEN_OR_API_KEY)
async def preview_skill(skill_name: str) -> quart.Response:
skill = self.ap.skill_mgr.get_skill_by_name(skill_name)
if not skill:
return self.http_status(404, -1, 'Skill not found')
return self.success(data={'instructions': skill.get('instructions', '')})
@self.route('/install/github', methods=['POST'], auth_type=group.AuthType.USER_TOKEN_OR_API_KEY)
async def install_skill_from_github() -> quart.Response:
data = await quart.request.json
required_fields = ['asset_url', 'owner', 'repo']
for field in required_fields:
if field not in data or not data[field]:
return self.http_status(400, -1, f'Missing required field: {field}')
asset_url = str(data['asset_url']).strip().lower().split('?', 1)[0].split('#', 1)[0]
if not asset_url.endswith('skill.md') and not data.get('release_tag'):
return self.http_status(400, -1, 'Missing required field: release_tag')
try:
skill = await self.ap.skill_service.install_from_github(data)
return self.success(data={'skills': skill})
except (ValueError, BoxError) as exc:
return self.http_status(400, -1, str(exc))
except Exception as exc:
return self.http_status(500, -1, f'Failed to install skill: {exc}')
@self.route('/install/github/preview', methods=['POST'], auth_type=group.AuthType.USER_TOKEN_OR_API_KEY)
async def preview_skill_from_github() -> quart.Response:
data = await quart.request.json
required_fields = ['asset_url', 'owner', 'repo']
for field in required_fields:
if field not in data or not data[field]:
return self.http_status(400, -1, f'Missing required field: {field}')
asset_url = str(data['asset_url']).strip().lower().split('?', 1)[0].split('#', 1)[0]
if not asset_url.endswith('skill.md') and not data.get('release_tag'):
return self.http_status(400, -1, 'Missing required field: release_tag')
try:
preview = await self.ap.skill_service.preview_install_from_github(data)
return self.success(data={'skills': preview})
except (ValueError, BoxError) as exc:
return self.http_status(400, -1, str(exc))
except Exception as exc:
return self.http_status(500, -1, f'Failed to preview skill: {exc}')
@self.route('/install/upload', methods=['POST'], auth_type=group.AuthType.USER_TOKEN_OR_API_KEY)
async def install_skill_from_upload() -> quart.Response:
file = (await quart.request.files).get('file')
if file is None:
return self.http_status(400, -1, 'file is required')
form = await quart.request.form
try:
skill = await self.ap.skill_service.install_from_zip_upload(
file_bytes=file.read(),
filename=file.filename or '',
source_paths=form.getlist('source_paths'),
)
return self.success(data={'skills': skill})
except (ValueError, BoxError) as exc:
return self.http_status(400, -1, str(exc))
except Exception as exc:
return self.http_status(500, -1, f'Failed to install skill: {exc}')
@self.route('/install/upload/preview', methods=['POST'], auth_type=group.AuthType.USER_TOKEN_OR_API_KEY)
async def preview_skill_from_upload() -> quart.Response:
file = (await quart.request.files).get('file')
if file is None:
return self.http_status(400, -1, 'file is required')
try:
preview = await self.ap.skill_service.preview_install_from_zip_upload(
file_bytes=file.read(),
filename=file.filename or '',
)
return self.success(data={'skills': preview})
except (ValueError, BoxError) as exc:
return self.http_status(400, -1, str(exc))
except Exception as exc:
return self.http_status(500, -1, f'Failed to preview skill: {exc}')
@self.route('/scan', methods=['GET'], auth_type=group.AuthType.USER_TOKEN_OR_API_KEY)
async def scan_skill_directory() -> quart.Response:
path = quart.request.args.get('path', '').strip()
if not path:
return self.http_status(400, -1, 'Missing required parameter: path')
try:
result = await self.ap.skill_service.scan_directory_async(path)
return self.success(data=result)
except (ValueError, BoxError) as exc:
return self.http_status(400, -1, str(exc))
+18 -1
View File
@@ -152,7 +152,24 @@ class MCPService:
coroutine = runtime_mcp_session.refresh() coroutine = runtime_mcp_session.refresh()
else: else:
runtime_mcp_session = await self.ap.tool_mgr.mcp_tool_loader.load_mcp_server(server_config=server_data) runtime_mcp_session = await self.ap.tool_mgr.mcp_tool_loader.load_mcp_server(server_config=server_data)
coroutine = runtime_mcp_session.start()
# A transient test owns an isolated Box session. Always tear it down
# after the test completes (success or failure) so it does not leak.
test_session = runtime_mcp_session
async def _run_and_cleanup() -> None:
try:
await test_session.start()
finally:
try:
await test_session.shutdown()
except Exception as exc:
self.ap.logger.warning(
f'Failed to tear down transient MCP test session '
f'{test_session.server_name}: {type(exc).__name__}: {exc}'
)
coroutine = _run_and_cleanup()
ctx = taskmgr.TaskContext.new() ctx = taskmgr.TaskContext.new()
wrapper = self.ap.task_mgr.create_user_task( wrapper = self.ap.task_mgr.create_user_task(
@@ -215,6 +215,8 @@ class PipelineService:
bound_mcp_servers: list[str] = None, bound_mcp_servers: list[str] = None,
enable_all_plugins: bool = True, enable_all_plugins: bool = True,
enable_all_mcp_servers: bool = True, enable_all_mcp_servers: bool = True,
bound_skills: list[str] = None,
enable_all_skills: bool = True,
) -> None: ) -> None:
"""Update the bound plugins and MCP servers for a pipeline""" """Update the bound plugins and MCP servers for a pipeline"""
# Get current pipeline # Get current pipeline
@@ -232,9 +234,12 @@ class PipelineService:
extensions_preferences = pipeline.extensions_preferences or {} extensions_preferences = pipeline.extensions_preferences or {}
extensions_preferences['enable_all_plugins'] = enable_all_plugins extensions_preferences['enable_all_plugins'] = enable_all_plugins
extensions_preferences['enable_all_mcp_servers'] = enable_all_mcp_servers extensions_preferences['enable_all_mcp_servers'] = enable_all_mcp_servers
extensions_preferences['enable_all_skills'] = enable_all_skills
extensions_preferences['plugins'] = bound_plugins extensions_preferences['plugins'] = bound_plugins
if bound_mcp_servers is not None: if bound_mcp_servers is not None:
extensions_preferences['mcp_servers'] = bound_mcp_servers extensions_preferences['mcp_servers'] = bound_mcp_servers
if bound_skills is not None:
extensions_preferences['skills'] = bound_skills
await self.ap.persistence_mgr.execute_async( await self.ap.persistence_mgr.execute_async(
sqlalchemy.update(persistence_pipeline.LegacyPipeline) sqlalchemy.update(persistence_pipeline.LegacyPipeline)
+428
View File
@@ -0,0 +1,428 @@
from __future__ import annotations
import io
import inspect
import os
import posixpath
import zipfile
from typing import Optional
from urllib.parse import quote, unquote, urlparse
import httpx
from ....core import app
from ....skill.utils import parse_frontmatter
_PUBLIC_SKILL_FIELDS = (
'name',
'display_name',
'description',
'instructions',
'package_root',
'created_at',
'updated_at',
)
_GITHUB_ASSET_HOSTS = {
'github.com',
'api.github.com',
'objects.githubusercontent.com',
'githubusercontent.com',
'raw.githubusercontent.com',
'codeload.github.com',
}
class SkillService:
"""Filesystem-backed skill management service."""
ap: app.Application
def __init__(self, ap: app.Application) -> None:
self.ap = ap
def _box_service(self):
box_service = getattr(self.ap, 'box_service', None)
if box_service is not None and getattr(box_service, 'available', False):
return box_service
return None
def _require_box(self, action: str):
"""Return the Box service or raise if it is not available.
Box is the only source of truth for skills. Every read and write
operation goes through it there is no local-filesystem fallback.
"""
box_service = self._box_service()
if box_service is not None:
return box_service
ap_box = getattr(self.ap, 'box_service', None)
if ap_box is None:
reason = 'not initialised'
elif not getattr(ap_box, 'enabled', True):
reason = 'disabled in config (box.enabled = false)'
else:
connector_error = getattr(ap_box, '_connector_error', '') or 'currently unavailable'
reason = f'unavailable: {connector_error}'
raise ValueError(
f'{action} requires the Box runtime, which is {reason}. '
f'Enable Box in config.yaml (box.enabled = true) and ensure the '
f'runtime is reachable before retrying.'
)
def _require_box_for_write(self, action: str) -> None:
"""Backwards-compatible alias preserved for clarity at call sites."""
self._require_box(action)
@staticmethod
def _serialize_skill(skill: dict) -> dict:
return {field: skill.get(field) for field in _PUBLIC_SKILL_FIELDS if field in skill}
async def list_skills(self) -> list[dict]:
# When Box is unavailable, surface an empty list rather than raising —
# the skills page should render cleanly, and the UI separately renders
# a "Box disabled / unavailable" banner via useBoxStatus.
box_service = self._box_service()
if box_service is None:
return []
return [self._serialize_skill(skill) for skill in await box_service.list_skills()]
async def get_skill(self, skill_name: str) -> Optional[dict]:
box_service = self._box_service()
if box_service is None:
return None
skill = await box_service.get_skill(skill_name)
return self._serialize_skill(skill) if skill else None
async def get_skill_by_name(self, name: str) -> Optional[dict]:
return await self.get_skill(name)
async def create_skill(self, data: dict) -> dict:
box_service = self._require_box('Creating a skill')
created = await box_service.create_skill(data)
await self._reload_skills()
return self._serialize_skill(created)
async def update_skill(self, skill_name: str, data: dict) -> dict:
box_service = self._require_box('Editing a skill')
updated = await box_service.update_skill(skill_name, data)
await self._reload_skills()
return self._serialize_skill(updated)
async def delete_skill(self, skill_name: str) -> bool:
box_service = self._require_box('Deleting a skill')
await box_service.delete_skill(skill_name)
await self._reload_skills()
return True
async def list_skill_files(
self,
skill_name: str,
path: str = '.',
include_hidden: bool = False,
max_entries: int = 200,
) -> dict:
box_service = self._require_box('Browsing skill files')
return await box_service.list_skill_files(skill_name, path, include_hidden, max_entries)
async def read_skill_file(self, skill_name: str, path: str) -> dict:
box_service = self._require_box('Reading a skill file')
return await box_service.read_skill_file(skill_name, path)
async def write_skill_file(self, skill_name: str, path: str, content: str) -> dict:
box_service = self._require_box('Editing skill files')
result = await box_service.write_skill_file(skill_name, path, content)
await self._reload_skills()
return result
async def install_from_github(self, data: dict) -> list[dict]:
box_service = self._require_box('Installing a skill from GitHub')
owner = str(data['owner']).strip()
repo = str(data['repo']).strip()
release_tag = str(data.get('release_tag', '')).strip()
raw_asset_url = str(data['asset_url']).strip()
if self._is_github_skill_md_url(raw_asset_url):
return await self._install_github_skill_md(raw_asset_url, owner=owner, repo=repo, data=data)
asset_url = self._validate_github_asset_url(raw_asset_url, owner=owner, repo=repo, release_tag=release_tag)
source_subdir = str(data.get('source_subdir', '') or '').strip()
zip_bytes = await self._download_github_asset(asset_url)
filename = f'{repo}-{release_tag.lstrip("v").replace("/", "-") or "source"}.zip'
installed = await box_service.install_skill_zip(
zip_bytes,
filename,
source_paths=data.get('source_paths') or [],
source_path=str(data.get('source_path', '') or ''),
source_subdir=source_subdir,
)
await self._reload_skills()
return [self._serialize_skill(skill) for skill in installed]
async def preview_install_from_github(self, data: dict) -> list[dict]:
box_service = self._require_box('Previewing a skill from GitHub')
owner = str(data['owner']).strip()
repo = str(data['repo']).strip()
release_tag = str(data.get('release_tag', '')).strip()
raw_asset_url = str(data['asset_url']).strip()
if self._is_github_skill_md_url(raw_asset_url):
return await self._preview_github_skill_md(raw_asset_url, owner=owner, repo=repo)
asset_url = self._validate_github_asset_url(raw_asset_url, owner=owner, repo=repo, release_tag=release_tag)
source_subdir = str(data.get('source_subdir', '') or '').strip()
zip_bytes = await self._download_github_asset(asset_url)
return await box_service.preview_skill_zip(
zip_bytes,
f'{repo}-{release_tag.lstrip("v").replace("/", "-") or "source"}.zip',
source_subdir=source_subdir,
)
async def install_from_zip_upload(
self,
*,
file_bytes: bytes,
filename: str,
source_paths: list[str] | None = None,
source_path: str = '',
) -> list[dict]:
box_service = self._require_box('Installing a skill from upload')
installed = await box_service.install_skill_zip(
file_bytes,
filename,
source_paths=source_paths or [],
source_path=source_path,
)
await self._reload_skills()
return [self._serialize_skill(skill) for skill in installed]
async def preview_install_from_zip_upload(self, *, file_bytes: bytes, filename: str) -> list[dict]:
box_service = self._require_box('Previewing a skill upload')
return await box_service.preview_skill_zip(file_bytes, filename)
async def _install_github_skill_md(self, asset_url: str, *, owner: str, repo: str, data: dict) -> list[dict]:
box_service = self._require_box('Installing a skill from GitHub')
zip_bytes, filename, _package_name = await self._download_github_skill_directory_as_zip(
asset_url,
owner=owner,
repo=repo,
)
installed = await box_service.install_skill_zip(
zip_bytes,
filename,
source_paths=data.get('source_paths') or [],
source_path=str(data.get('source_path', '') or ''),
target_suffix='',
)
await self._reload_skills()
return [self._serialize_skill(skill) for skill in installed]
async def _preview_github_skill_md(self, asset_url: str, *, owner: str, repo: str) -> list[dict]:
box_service = self._require_box('Previewing a skill from GitHub')
zip_bytes, _filename, package_name = await self._download_github_skill_directory_as_zip(
asset_url,
owner=owner,
repo=repo,
)
return await box_service.preview_skill_zip(zip_bytes, f'{package_name}.zip', target_suffix='')
async def reload_skills(self) -> list[dict]:
await self._reload_skills()
return await self.list_skills()
async def scan_directory_async(self, path: str) -> dict:
box_service = self._require_box('Scanning a skill directory')
return await box_service.scan_skill_directory(path)
async def _reload_skills(self) -> None:
skill_mgr = getattr(self.ap, 'skill_mgr', None)
reload_skills = getattr(skill_mgr, 'reload_skills', None)
if not callable(reload_skills):
return
result = reload_skills()
if inspect.isawaitable(result):
await result
async def _download_github_asset(self, asset_url: str) -> bytes:
async with httpx.AsyncClient(follow_redirects=True, timeout=120) as client:
resp = await client.get(asset_url)
resp.raise_for_status()
return resp.content
async def _download_github_skill_directory_as_zip(
self, asset_url: str, *, owner: str, repo: str
) -> tuple[bytes, str, str]:
info = self._parse_github_skill_md_url(asset_url, owner=owner, repo=repo)
archive_url = f'https://codeload.github.com/{owner}/{repo}/zip/{quote(info["ref"], safe="/")}'
archive_bytes = await self._download_github_asset(archive_url)
try:
source_archive = zipfile.ZipFile(io.BytesIO(archive_bytes), 'r')
except zipfile.BadZipFile as exc:
raise ValueError('GitHub repository archive must be a valid .zip archive') from exc
with source_archive as source_zip:
skill_entry = self._find_github_skill_archive_entry(source_zip, info['file_path'])
try:
skill_md_content = source_zip.read(skill_entry).decode('utf-8')
except UnicodeDecodeError as exc:
raise ValueError('GitHub SKILL.md must be valid UTF-8 text') from exc
package_name = self._resolve_github_skill_md_package_name(skill_md_content, info['package_name'])
source_skill_dir = posixpath.dirname(posixpath.normpath(skill_entry.filename))
buffer = io.BytesIO()
with zipfile.ZipFile(buffer, 'w', zipfile.ZIP_DEFLATED) as target_zip:
self._copy_github_skill_directory_to_zip(source_zip, target_zip, source_skill_dir, package_name)
return buffer.getvalue(), f'{package_name}.zip', package_name
def _find_github_skill_archive_entry(self, archive: zipfile.ZipFile, file_path: str) -> zipfile.ZipInfo:
normalized_file_path = posixpath.normpath(file_path).lower()
for member in archive.infolist():
if member.is_dir():
continue
normalized_member = posixpath.normpath(member.filename)
path_parts = normalized_member.split('/', 1)
if len(path_parts) != 2:
continue
archive_relative_path = path_parts[1].lower()
if archive_relative_path == normalized_file_path:
return member
raise ValueError(f'GitHub archive does not contain requested SKILL.md: {file_path}')
def _copy_github_skill_directory_to_zip(
self,
source_zip: zipfile.ZipFile,
target_zip: zipfile.ZipFile,
source_skill_dir: str,
package_name: str,
) -> None:
normalized_source_dir = posixpath.normpath(source_skill_dir)
source_prefix = f'{normalized_source_dir}/'
copied_files = 0
for member in source_zip.infolist():
normalized_member = posixpath.normpath(member.filename)
if normalized_member != normalized_source_dir and not normalized_member.startswith(source_prefix):
continue
relative_path = posixpath.relpath(normalized_member, normalized_source_dir)
if relative_path in ('', '.'):
continue
if relative_path.startswith('../') or relative_path == '..' or posixpath.isabs(relative_path):
raise ValueError(f'GitHub archive contains an unsafe skill path: {member.filename}')
target_name = f'{package_name}/{relative_path}'
if member.is_dir() and not target_name.endswith('/'):
target_name = f'{target_name}/'
target_info = zipfile.ZipInfo(target_name, date_time=member.date_time)
target_info.external_attr = member.external_attr
target_info.compress_type = zipfile.ZIP_DEFLATED
if member.is_dir():
target_zip.writestr(target_info, b'')
continue
target_zip.writestr(target_info, source_zip.read(member))
copied_files += 1
if copied_files == 0:
raise ValueError('GitHub skill directory is empty')
def _uploaded_skill_target_stem(self, filename: str) -> str:
stem = os.path.splitext(os.path.basename(str(filename or '').strip()))[0]
safe_stem = ''.join(ch if ch.isalnum() or ch in ('-', '_') else '-' for ch in stem).strip('-_')
if not safe_stem:
safe_stem = 'uploaded-skill'
return safe_stem
@staticmethod
def _is_github_skill_md_url(asset_url: str) -> bool:
parsed = urlparse(str(asset_url or '').strip())
normalized_path = posixpath.normpath(parsed.path or '/')
return normalized_path.lower().endswith('/skill.md')
def _parse_github_skill_md_url(self, asset_url: str, *, owner: str, repo: str) -> dict:
parsed = urlparse(str(asset_url or '').strip())
if parsed.scheme != 'https' or not parsed.netloc:
raise ValueError('asset_url must be a valid HTTPS GitHub SKILL.md URL')
host = parsed.netloc.lower()
path_parts = [unquote(part) for part in (parsed.path or '').split('/') if part]
if host == 'github.com':
if (
len(path_parts) < 5
or path_parts[0] != owner
or path_parts[1] != repo
or path_parts[2]
not in (
'blob',
'raw',
)
):
raise ValueError('GitHub SKILL.md URL must point to the requested owner/repo blob path')
ref = path_parts[3]
file_path = '/'.join(path_parts[4:])
elif host == 'raw.githubusercontent.com':
if len(path_parts) < 4 or path_parts[0] != owner or path_parts[1] != repo:
raise ValueError('GitHub SKILL.md URL must point to the requested owner/repo raw path')
ref = path_parts[2]
file_path = '/'.join(path_parts[3:])
else:
raise ValueError('asset_url must point to a GitHub SKILL.md file')
normalized_file_path = posixpath.normpath(file_path)
normalized_file_path_lower = normalized_file_path.lower()
if normalized_file_path_lower != 'skill.md' and not normalized_file_path_lower.endswith('/skill.md'):
raise ValueError('GitHub skill import requires a URL ending with SKILL.md')
parent_dir = posixpath.basename(posixpath.dirname(normalized_file_path)) or repo
return {
'ref': ref,
'file_path': normalized_file_path,
'package_name': self._uploaded_skill_target_stem(parent_dir),
}
def _resolve_github_skill_md_package_name(self, content: str, fallback: str) -> str:
metadata, _instructions = parse_frontmatter(content)
candidate = str(metadata.get('name') or fallback or '').strip()
try:
return self._validate_skill_name(candidate)
except ValueError:
return self._validate_skill_name(fallback)
@staticmethod
def _validate_github_asset_url(asset_url: str, *, owner: str, repo: str, release_tag: str) -> str:
parsed = urlparse(str(asset_url).strip())
if parsed.scheme != 'https' or not parsed.netloc:
raise ValueError('asset_url must be a valid HTTPS GitHub asset URL')
host = parsed.netloc.lower()
if host not in _GITHUB_ASSET_HOSTS:
raise ValueError('asset_url must point to a GitHub-hosted release asset or archive')
normalized_path = posixpath.normpath(parsed.path or '/')
allowed_prefixes = [
f'/repos/{owner}/{repo}/',
f'/{owner}/{repo}/',
]
if not any(normalized_path.startswith(prefix) for prefix in allowed_prefixes):
raise ValueError('asset_url does not match the requested owner/repo')
if release_tag and release_tag not in parsed.path and release_tag not in parsed.query:
raise ValueError('asset_url does not match the requested release_tag')
return parsed.geturl()
@staticmethod
def _validate_skill_name(name: str) -> str:
name = str(name or '').strip()
if not name:
raise ValueError('Skill name is required')
if not name.replace('-', '').replace('_', '').isalnum():
raise ValueError('Skill name can only contain letters, numbers, hyphens and underscores')
if len(name) > 64:
raise ValueError('Skill name cannot exceed 64 characters')
return name
+5
View File
@@ -0,0 +1,5 @@
"""LangBot Box runtime package."""
from .workspace import BoxWorkspaceSession
__all__ = ['BoxWorkspaceSession']
+364
View File
@@ -0,0 +1,364 @@
from __future__ import annotations
import asyncio
import json
import os
import sys
import typing
from typing import TYPE_CHECKING
from urllib.parse import urlparse
from langbot_plugin.entities.io.actions.enums import CommonAction
from langbot_plugin.runtime.io.handler import Handler
from langbot_plugin.runtime.io.connection import Connection
from langbot_plugin.box.client import ActionRPCBoxClient
from langbot_plugin.box.errors import BoxRuntimeUnavailableError
from langbot_plugin.box.actions import LangBotToBoxAction
from ..utils import platform
from ..utils.managed_runtime import ManagedRuntimeConnector
if TYPE_CHECKING:
from ..core import app as core_app
# Default Docker Compose service name for the standalone Box container.
_DOCKER_BOX_HOST = 'langbot_box'
_DEFAULT_PORT = 5410
_HEARTBEAT_INTERVAL_SEC = 20
# Top-level keys under ``box`` that are LangBot-internal and should not be
# forwarded to the Box runtime.
_INTERNAL_BOX_CONFIG_KEYS = frozenset({'runtime'})
def _get_box_config(ap) -> dict:
"""Return the 'box' section from instance config.
Environment-variable overrides are handled uniformly by
``LoadConfigStage._apply_env_overrides_to_config`` using the
``SECTION__SUBSECTION__KEY`` convention (e.g. ``BOX__LOCAL__HOST_ROOT``,
``BOX__LOCAL__ALLOWED_MOUNT_ROOTS="/a,/b"``) before this is read, so no
box-specific env parsing is needed here.
"""
instance_config = getattr(ap, 'instance_config', None)
config_data = getattr(instance_config, 'data', {}) if instance_config is not None else {}
return dict(config_data.get('box', {}) or {})
def _get_runtime_endpoint(box_cfg: dict) -> str:
runtime_cfg = box_cfg.get('runtime') or {}
return str(runtime_cfg.get('endpoint', '')).strip()
def _filter_config_for_runtime(box_cfg: dict) -> dict:
return {k: v for k, v in box_cfg.items() if k not in _INTERNAL_BOX_CONFIG_KEYS}
def resolve_box_ws_relay_url(ap: core_app.Application) -> str:
"""Derive the WS relay base URL used for managed-process attach.
The WS relay serves the ``/v1/sessions/{id}/managed-process/ws`` endpoint
on the *relay* port (default 5410).
"""
box_cfg = _get_box_config(ap)
# Explicit runtime endpoint takes precedence. The config value is a base
# URL; endpoint-specific paths are appended by the SDK client.
endpoint = _get_runtime_endpoint(box_cfg)
if endpoint:
parsed = urlparse(endpoint)
scheme = parsed.scheme or 'ws'
if scheme == 'ws':
scheme = 'http'
elif scheme == 'wss':
scheme = 'https'
host = parsed.hostname or '127.0.0.1'
port = parsed.port or _DEFAULT_PORT
return f'{scheme}://{host}:{port}'
# In Docker, relay lives on the box runtime container.
if platform.get_platform() == 'docker':
return f'http://{_DOCKER_BOX_HOST}:{_DEFAULT_PORT}'
return f'http://127.0.0.1:{_DEFAULT_PORT}'
class BoxRuntimeConnector(ManagedRuntimeConnector):
"""Connect to the Box runtime via action RPC.
Transport decision (mirrors Plugin runtime logic):
1. Docker / --standalone-box / explicit runtime.endpoint -> WebSocket to external Box process
2. Windows (non-Docker) -> subprocess + WebSocket (Windows lacks async stdio pipe)
3. Unix / macOS -> subprocess + stdio pipe
"""
def __init__(
self,
ap: core_app.Application,
runtime_disconnect_callback: typing.Callable[
['BoxRuntimeConnector'], typing.Coroutine[typing.Any, typing.Any, None]
]
| None = None,
):
super().__init__(ap)
self.runtime_disconnect_callback = runtime_disconnect_callback
self.configured_runtime_endpoint = self._load_configured_runtime_endpoint()
self.ws_relay_base_url = resolve_box_ws_relay_url(ap)
self.client = ActionRPCBoxClient(logger=ap.logger)
self._handler: Handler | None = None
self._handler_task: asyncio.Task | None = None
self._ctrl_task: asyncio.Task | None = None
self._heartbeat_task: asyncio.Task | None = None
# Parse the relay URL once for reuse.
parsed = urlparse(self.ws_relay_base_url)
self._relay_host = parsed.hostname or '127.0.0.1'
self._relay_port = parsed.port or _DEFAULT_PORT
self._filtered_box_config = _filter_config_for_runtime(_get_box_config(ap))
def uses_websocket(self) -> bool:
"""Whether the connector should use WebSocket to reach the Box runtime.
True when:
- Running inside Docker (Box runtime is a separate container)
- The ``--standalone-box`` CLI flag was passed
- An explicit ``runtime.endpoint`` was configured
When this is True the Box runtime lives in a separate process with its
own filesystem view (container, pod sidecar, or remote host), so paths
it reports (e.g. skill ``package_root``) are NOT resolvable on the
LangBot side. When False, Box runs as a stdio child process that shares
LangBot's filesystem.
"""
return bool(
self.configured_runtime_endpoint
or platform.get_platform() == 'docker'
or platform.use_websocket_to_connect_box_runtime()
)
# Backwards-compatible private alias.
def _uses_websocket(self) -> bool:
return self.uses_websocket()
async def initialize(self) -> None:
if self._uses_websocket():
if platform.get_platform() == 'win32' and not self.configured_runtime_endpoint:
await self._start_subprocess_then_ws()
else:
await self._connect_remote_ws()
else:
await self._start_local_stdio()
# Start heartbeat after successful connection
if self._heartbeat_task is None:
self._heartbeat_task = asyncio.create_task(self._heartbeat_loop())
# -- heartbeat -----------------------------------------------------------
async def _heartbeat_loop(self) -> None:
"""Periodically ping the Box runtime to detect silent disconnections."""
while True:
await asyncio.sleep(_HEARTBEAT_INTERVAL_SEC)
try:
await self.ping()
self.ap.logger.debug('Heartbeat to Box runtime success.')
except Exception as e:
self.ap.logger.debug(f'Failed to heartbeat to Box runtime: {e}')
async def ping(self) -> None:
if self._handler is None:
raise BoxRuntimeUnavailableError('Box runtime is not connected')
await self._handler.call_action(CommonAction.PING, {})
# -- transport paths -----------------------------------------------------
async def _start_local_stdio(self) -> None:
"""Launch box server as subprocess and connect via stdio (Unix/macOS)."""
from langbot_plugin.runtime.io.controllers.stdio.client import StdioClientController
self.ap.logger.info('Use stdio to connect to box runtime')
python_path = sys.executable
env = os.environ.copy()
if self._filtered_box_config:
env['LANGBOT_BOX_CONFIG'] = json.dumps(self._filtered_box_config)
connected = asyncio.Event()
connect_error: list[Exception] = []
ctrl = StdioClientController(
command=python_path,
# Launched through the same CLI entry point as the plugin runtime
# (cli.__init__ <subcommand>); `-s` selects the stdio transport,
# mirroring `rt -s`.
args=['-m', 'langbot_plugin.cli.__init__', 'box', '-s', '--ws-control-port', str(self._relay_port)],
env=env,
)
self._ctrl_task = asyncio.create_task(
ctrl.run(self._make_connection_callback('stdio', connected, connect_error))
)
try:
await asyncio.wait_for(connected.wait(), timeout=30.0)
except asyncio.TimeoutError:
raise BoxRuntimeUnavailableError('box runtime subprocess did not connect in time')
if connect_error:
raise BoxRuntimeUnavailableError(f'box runtime connection failed: {connect_error[0]}')
self._subprocess = ctrl.process
async def _start_subprocess_then_ws(self) -> None:
"""Launch box server as detached subprocess, then connect via WS (Windows)."""
self.ap.logger.info('(windows) Use cmd to launch box runtime and communicate via ws')
env = os.environ.copy()
if self._filtered_box_config:
env['LANGBOT_BOX_CONFIG'] = json.dumps(self._filtered_box_config)
python_path = sys.executable
# Launched through the same CLI entry point as the plugin runtime
# (cli.__init__ <subcommand>); no flag => WebSocket transport.
self.runtime_subprocess = await asyncio.create_subprocess_exec(
python_path,
'-m',
'langbot_plugin.cli.__init__',
'box',
'--ws-control-port',
str(self._relay_port),
env=env,
)
self.runtime_subprocess_task = asyncio.create_task(self.runtime_subprocess.wait())
ws_url = f'ws://localhost:{self._relay_port}/rpc/ws'
await self._connect_ws(ws_url, '(windows) WebSocket')
async def _connect_remote_ws(self) -> None:
"""Connect to a remote (or Docker) box server via WebSocket."""
ws_url = self._resolve_rpc_ws_url()
self.ap.logger.info(f'Use WebSocket to connect to box runtime ({ws_url})')
await self._connect_ws(ws_url, 'WebSocket')
# -- helpers -------------------------------------------------------------
def _resolve_rpc_ws_url(self) -> str:
"""Determine the action-RPC WebSocket URL.
All endpoints share a single port; action RPC is at ``/rpc/ws``.
"""
if self.configured_runtime_endpoint:
base = self.configured_runtime_endpoint.rstrip('/')
parsed = urlparse(base)
scheme = parsed.scheme or 'ws'
if scheme in ('http', 'https'):
scheme = 'wss' if scheme == 'https' else 'ws'
host = parsed.hostname or '127.0.0.1'
port = parsed.port or _DEFAULT_PORT
return f'{scheme}://{host}:{port}/rpc/ws'
if platform.get_platform() == 'docker':
return f'ws://{_DOCKER_BOX_HOST}:{_DEFAULT_PORT}/rpc/ws'
return f'ws://localhost:{self._relay_port}/rpc/ws'
async def _connect_ws(self, ws_url: str, transport_name: str) -> None:
"""Shared WebSocket connection procedure."""
from langbot_plugin.runtime.io.controllers.ws.client import WebSocketClientController
connected = asyncio.Event()
connect_error: list[Exception] = []
async def on_connect_failed(ctrl, exc):
if exc is not None:
self.ap.logger.error(f'Failed to connect to Box runtime ({ws_url}): {exc}')
else:
self.ap.logger.error(f'Failed to connect to Box runtime ({ws_url}), trying to reconnect...')
connect_error.append(exc or BoxRuntimeUnavailableError('ws connection failed'))
connected.set()
if self.runtime_disconnect_callback is not None:
await self.runtime_disconnect_callback(self)
ctrl = WebSocketClientController(ws_url=ws_url, make_connection_failed_callback=on_connect_failed)
self._ctrl_task = asyncio.create_task(
ctrl.run(self._make_connection_callback(transport_name, connected, connect_error))
)
try:
await asyncio.wait_for(connected.wait(), timeout=30.0)
except asyncio.TimeoutError:
raise BoxRuntimeUnavailableError(f'box runtime ws connection timed out ({ws_url})')
if connect_error:
raise BoxRuntimeUnavailableError(f'box runtime connection failed: {connect_error[0]}')
def _make_connection_callback(
self,
transport_name: str,
connected: asyncio.Event,
connect_error: list[Exception],
):
async def new_connection_callback(connection: Connection) -> None:
handler = Handler(connection)
self._handler = handler
self.client.set_handler(handler)
self._handler_task = asyncio.create_task(handler.run())
try:
await handler.call_action(CommonAction.PING, {})
if self._filtered_box_config:
await handler.call_action(LangBotToBoxAction.INIT, self._filtered_box_config)
self.ap.logger.debug('Sent box configuration to Box runtime via INIT.')
self.ap.logger.info(f'Connected to Box runtime via {transport_name}.')
connected.set()
await self._handler_task
except Exception as exc:
if not connected.is_set():
connect_error.append(exc)
connected.set()
return
# If we reach here, handler.run() returned normally (connection
# closed) or raised after the initial handshake succeeded.
# Either way, treat it as a disconnect.
if connected.is_set():
if self._uses_websocket():
self.ap.logger.error('Disconnected from Box runtime, trying to reconnect...')
if self.runtime_disconnect_callback is not None:
await self.runtime_disconnect_callback(self)
else:
self.ap.logger.error(
'Disconnected from Box runtime via stdio. '
'Cannot automatically reconnect — please restart LangBot.'
)
return new_connection_callback
# -- lifecycle -----------------------------------------------------------
def dispose(self) -> None:
if self._heartbeat_task is not None:
self._heartbeat_task.cancel()
self._heartbeat_task = None
if self._handler_task is not None:
self._handler_task.cancel()
self._handler_task = None
if self._ctrl_task is not None:
self._ctrl_task.cancel()
self._ctrl_task = None
# stdio-managed subprocess (stored as self._subprocess by _start_local_stdio)
if hasattr(self, '_subprocess') and self._subprocess is not None and self._subprocess.returncode is None:
self.ap.logger.info('Terminating managed box runtime process...')
self._subprocess.terminate()
# Subprocess launched by ManagedRuntimeConnector._start_runtime_subprocess (Windows path)
self._dispose_subprocess()
# -- config helpers ------------------------------------------------------
def _load_configured_runtime_endpoint(self) -> str:
return _get_runtime_endpoint(_get_box_config(self.ap))
+98
View File
@@ -0,0 +1,98 @@
"""Three-layer security policy for LangBot Box.
The design separates concerns into three independent layers, aligned with
OpenCode / OpenClaw patterns:
1. **SandboxPolicy** *where* tools run (host vs sandbox).
2. **ToolPolicy** *which* tools are allowed (allow/deny lists).
3. **ElevatedPolicy** *whether* a single exec call may temporarily
escape the default sandbox boundary.
These three layers are orthogonal:
- ToolPolicy is a hard boundary; ``elevated`` cannot bypass a denied tool.
- SandboxPolicy decides the default execution location.
- ElevatedPolicy only affects ``exec`` and only when the framework allows it.
"""
from __future__ import annotations
import enum
from typing import Sequence
# ── Layer 1: Sandbox Policy ──────────────────────────────────────────
class SandboxMode(str, enum.Enum):
"""Determines when agent execution is routed through the sandbox."""
OFF = 'off'
"""Sandbox disabled; all exec runs on the host."""
NON_DEFAULT = 'non_default'
"""Only non-default sessions are sandboxed (e.g. sub-agents, MCP)."""
ALL = 'all'
"""Every agent exec call is routed through the sandbox."""
class SandboxPolicy:
"""Decides whether a given execution context should use the sandbox."""
def __init__(self, mode: SandboxMode = SandboxMode.ALL):
self.mode = mode
def should_sandbox(self, *, is_default_session: bool = True) -> bool:
if self.mode == SandboxMode.OFF:
return False
if self.mode == SandboxMode.ALL:
return True
# NON_DEFAULT: sandbox everything except the default session
return not is_default_session
# ── Layer 2: Tool Policy ─────────────────────────────────────────────
class ToolPolicy:
"""Controls which tools are available to the current agent/session.
Rules:
- ``deny`` always takes precedence over ``allow``.
- An empty ``allow`` list means "all tools allowed" (no allowlist filter).
- ``elevated`` cannot bypass a denied tool.
"""
def __init__(
self,
allow: Sequence[str] = (),
deny: Sequence[str] = (),
):
self._allow: frozenset[str] = frozenset(allow)
self._deny: frozenset[str] = frozenset(deny)
def is_tool_allowed(self, tool_name: str) -> bool:
if tool_name in self._deny:
return False
if self._allow and tool_name not in self._allow:
return False
return True
# ── Layer 3: Elevated Policy ─────────────────────────────────────────
class ElevatedPolicy:
"""Controls whether ``exec`` may request temporary privilege escalation.
``elevated`` only applies to the ``exec`` tool. It means "run this
command outside the default sandbox boundary" (e.g. with network, or
on the host). The framework decides whether to honor the request.
"""
def __init__(self, *, allow_elevated: bool = False, require_approval: bool = True):
self.allow_elevated = allow_elevated
self.require_approval = require_approval
def is_elevation_permitted(self) -> bool:
return self.allow_elevated
+865
View File
@@ -0,0 +1,865 @@
from __future__ import annotations
import asyncio
import collections
import datetime as _dt
import enum
import json
import os
from typing import TYPE_CHECKING
import pydantic
from langbot_plugin.box.client import BoxRuntimeClient
from .connector import BoxRuntimeConnector, _get_box_config
from langbot_plugin.box.errors import BoxError, BoxValidationError
from langbot_plugin.box.models import (
BUILTIN_PROFILES,
BoxExecutionResult,
BoxManagedProcessInfo,
BoxManagedProcessSpec,
BoxProfile,
BoxSpec,
)
_INT_ADAPTER = pydantic.TypeAdapter(int)
_UTC = _dt.timezone.utc
_MAX_RECENT_ERRORS = 50
_MIB = 1024 * 1024
def _is_path_under(path: str, root: str) -> bool:
"""Check whether *path* equals *root* or is a child of *root*."""
return path == root or path.startswith(f'{root}{os.sep}')
if TYPE_CHECKING:
from ..core import app as core_app
import langbot_plugin.api.entities.builtin.pipeline.query as pipeline_query
class BoxService:
def __init__(
self,
ap: core_app.Application,
client: BoxRuntimeClient | None = None,
output_limit_chars: int = 4000,
):
self.ap = ap
self._enabled = self._load_enabled()
self._runtime_connector: BoxRuntimeConnector | None = None
if client is None:
# Always construct a connector — its __init__ is side-effect free
# (no I/O, no subprocess). When ``box.enabled = false`` we simply
# skip ``connector.initialize()`` so no connection is attempted.
self._runtime_connector = BoxRuntimeConnector(ap, runtime_disconnect_callback=self._on_runtime_disconnect)
client = self._runtime_connector.client
self.client = client
self.output_limit_chars = output_limit_chars
self.host_root = self._load_host_root()
self.allowed_mount_roots = self._load_allowed_mount_roots()
self.default_workspace = self._load_default_workspace()
self.profile = self._load_profile()
self.custom_image = self._load_custom_image()
self.workspace_quota_mb = self._load_workspace_quota_mb()
self._recent_errors: collections.deque[dict] = collections.deque(maxlen=_MAX_RECENT_ERRORS)
self._shutdown_task = None
self._available = False
self._connector_error: str = ''
self._reconnecting = False
# Optional explicit override for shares_filesystem_with_box. None means
# "derive from the connector transport". Set by tests / embedders that
# know the real LangBot<->Box filesystem topology.
self._shares_filesystem_with_box_override: bool | None = None
@property
def enabled(self) -> bool:
"""Whether Box is enabled in config. False means the operator has
deliberately turned the sandbox off via ``box.enabled = false``.
Disabled and "enabled but unavailable" are reported as the same
``available = False`` to consumers, but distinguished in get_status."""
return self._enabled
async def initialize(self):
self._ensure_default_workspace()
if not self._enabled:
# Disabled by config: do NOT connect to a remote runtime, do NOT
# fork a stdio subprocess. Every consumer of box_service should
# gate on ``available`` and degrade gracefully.
self._available = False
self._connector_error = 'Box runtime is disabled in config (box.enabled = false)'
self.ap.logger.info(
'Box runtime disabled by config; sandbox features (exec/read/write/edit, '
'skill add/edit, stdio MCP) will be unavailable.'
)
return
try:
if self._runtime_connector is not None:
await self._runtime_connector.initialize()
else:
await self.client.initialize()
self._available = True
self._connector_error = ''
self.ap.logger.info(
f'LangBot Box runtime initialized: profile={self.profile.name} '
f'default_workspace={self.default_workspace or "(none)"}'
)
except Exception as exc:
self.ap.logger.warning(f'LangBot Box runtime unavailable, sandbox features disabled: {exc}')
self._available = False
self._connector_error = str(exc)
async def _on_runtime_disconnect(self, connector: BoxRuntimeConnector) -> None:
"""Called by the connector when the Box runtime connection drops.
Spawns a background reconnection loop so the caller is not blocked.
Skipped entirely when Box is disabled by config that path should
never have connected in the first place.
"""
if not self._enabled:
return
if self._reconnecting:
return # Another reconnect loop is already running
self._reconnecting = True
self._available = False
self._connector_error = 'Disconnected from Box runtime'
self.ap.logger.warning('Box runtime disconnected, sandbox features temporarily disabled.')
asyncio.create_task(self._reconnect_loop(connector))
async def _reconnect_loop(self, connector: BoxRuntimeConnector) -> None:
"""Retry reconnection with exponential backoff (3s → 60s max)."""
delay = 3
max_delay = 60
try:
while True:
self.ap.logger.info(f'Attempting to reconnect to Box runtime in {delay}s...')
await asyncio.sleep(delay)
try:
connector.dispose()
await connector.initialize()
self._available = True
self._connector_error = ''
self.ap.logger.info('Box runtime reconnected, sandbox features restored.')
return
except Exception as exc:
self._connector_error = str(exc)
self.ap.logger.warning(f'Box runtime reconnection failed: {exc}')
delay = min(delay * 2, max_delay)
finally:
self._reconnecting = False
@property
def available(self) -> bool:
return self._available
@property
def shares_filesystem_with_box(self) -> bool:
"""Whether LangBot and the Box runtime share a filesystem view.
This is True only when Box runs as a local stdio child process of
LangBot (same container/host). In that case paths the Box runtime
reports notably skill ``package_root`` resolve identically on the
LangBot side, so LangBot may validate them against its own filesystem.
It is False for every separated deployment (Docker Compose, k8s
sidecar, ``--standalone-box``, or an explicit ``runtime.endpoint``),
where the Box runtime owns its own filesystem and LangBot must trust
the paths it reports rather than checking them locally.
When Box is wired up with an injected client (tests, custom embeds)
there is no connector to introspect; we conservatively report False so
LangBot never wrongly drops Box-reported skills. An explicit override
can be set via ``_shares_filesystem_with_box`` (used by tests and any
embedder that knows the real topology).
"""
if self._shares_filesystem_with_box_override is not None:
return self._shares_filesystem_with_box_override
if self._runtime_connector is None:
return False
return not self._runtime_connector.uses_websocket()
async def execute_spec_payload(
self,
spec_payload: dict,
query: pipeline_query.Query,
*,
skip_host_mount_validation: bool = False,
) -> dict:
if not self._available:
raise BoxError('Box runtime is not available. Install and start Docker to use sandbox features.')
try:
spec = self.build_spec(spec_payload, skip_host_mount_validation=skip_host_mount_validation)
except BoxError as exc:
self._record_error(exc, query)
raise
self.ap.logger.info(
'LangBot Box request: '
f'query_id={query.query_id} '
f'spec={json.dumps(self._summarize_spec(spec), ensure_ascii=False)}'
)
try:
await self._enforce_workspace_quota(spec, phase='before execution')
except BoxError as exc:
self._record_error(exc, query)
raise
try:
result = await self.client.execute(spec)
except BoxError as exc:
self._record_error(exc, query)
raise
try:
await self._enforce_workspace_quota(spec, phase='after execution')
except BoxError as exc:
await self._cleanup_exceeded_session(spec)
self._record_error(exc, query)
raise
self.ap.logger.info(
'LangBot Box result: '
f'query_id={query.query_id} '
f'summary={json.dumps(self._summarize_result(result), ensure_ascii=False)}'
)
return self._serialize_result(result)
def resolve_box_session_id(self, query: pipeline_query.Query) -> str:
"""Resolve the Box session_id from the pipeline's template and query variables.
When ``system.limitation.force_box_session_id_template`` is set to a
non-empty value, that template overrides whatever the pipeline
configured. This is the authoritative SaaS guard: it runs on every
``exec`` call, so a tenant cannot escape a single shared sandbox even
by editing the pipeline config directly through the API (which only
gates the web UI).
"""
forced_template = self._forced_box_session_id_template()
if forced_template:
template = forced_template
else:
template = (
(query.pipeline_config or {})
.get('ai', {})
.get('local-agent', {})
.get('box-session-id-template', '{launcher_type}_{launcher_id}')
)
variables = dict(query.variables or {})
launcher_type = getattr(query, 'launcher_type', None)
if hasattr(launcher_type, 'value'):
launcher_type = launcher_type.value
launcher_id = getattr(query, 'launcher_id', None)
sender_id = getattr(query, 'sender_id', None)
query_id = getattr(query, 'query_id', None)
variables.setdefault('query_id', str(query_id or 'unknown'))
variables.setdefault('launcher_type', str(launcher_type or 'query'))
variables.setdefault('launcher_id', str(launcher_id or query_id or 'unknown'))
variables.setdefault('sender_id', str(sender_id or launcher_id or query_id or 'unknown'))
variables.setdefault('global', 'global')
return template.format_map(collections.defaultdict(lambda: 'unknown', variables))
def build_skill_extra_mounts(self, query: pipeline_query.Query) -> list[dict]:
"""Build extra_mounts entries for all pipeline-bound skills.
This ensures that when a container is first created it already has
all skill packages mounted, regardless of which skill is currently
activated.
Path validation is filesystem-topology dependent. When LangBot and the
Box runtime share a filesystem (local stdio mode), a skill whose
``package_root`` is missing or no longer a directory is skipped with a
warning instead of being passed through to the backend. Without that
guard the three backends behave inconsistently on a stale mount: nsjail
refuses to start the sandbox (failing every exec in the session),
Docker silently auto-creates a root-owned empty directory on the host,
and E2B silently skips the upload none of which surfaces an
actionable error.
When Box runs as a separate process (Docker Compose, k8s sidecar,
``--standalone-box``, or a remote ``runtime.endpoint``), the
``package_root`` reported by ``list_skills`` is the Box runtime's own
filesystem path and is NOT resolvable on the LangBot side. Validating
it locally would wrongly drop every skill, so LangBot trusts the path
and lets the Box runtime resolve it. The Box runtime only ever reports
skills it discovered on its own filesystem, so the path is valid there
by construction.
"""
skill_mgr = getattr(self.ap, 'skill_mgr', None)
if skill_mgr is None:
return []
from ..provider.tools.loaders import skill as skill_loader
validate_locally = self.shares_filesystem_with_box
visible_skills = skill_loader.get_visible_skills(self.ap, query)
mounts: list[dict] = []
for skill_name, skill_data in visible_skills.items():
package_root = str(skill_data.get('package_root', '') or '').strip()
if not package_root:
continue
if validate_locally and not os.path.isdir(package_root):
self.ap.logger.warning(
f'Skill "{skill_name}" package_root missing on filesystem '
f'({package_root}); skipping mount to prevent sandbox failures. '
f'The skill cache may be stale — consider reloading skills.'
)
continue
mounts.append(
{
'host_path': package_root,
'mount_path': f'/workspace/.skills/{skill_name}',
'mode': 'rw',
}
)
return mounts
async def execute_tool(self, parameters: dict, query: pipeline_query.Query) -> dict:
"""Execute an agent-facing ``exec`` tool call.
Translates the agent-facing ``command`` field to the internal
``BoxSpec.cmd`` field and injects the session id from the query.
"""
spec_payload: dict = {'cmd': parameters['command']}
# Pass through allowed agent-facing fields
for key in ('workdir', 'timeout_sec', 'env'):
if key in parameters:
spec_payload[key] = parameters[key]
# Inject context the agent must not control
spec_payload.setdefault('session_id', self.resolve_box_session_id(query))
# Mount all pipeline-bound skills so they are available in the container
if 'extra_mounts' not in spec_payload:
spec_payload['extra_mounts'] = self.build_skill_extra_mounts(query)
return await self.execute_spec_payload(spec_payload, query)
async def shutdown(self):
await self.client.shutdown()
def dispose(self):
if self._runtime_connector is not None:
self._runtime_connector.dispose()
loop = getattr(self.ap, 'event_loop', None)
if loop is not None and not loop.is_closed() and (self._shutdown_task is None or self._shutdown_task.done()):
self._shutdown_task = loop.create_task(self.shutdown())
async def get_sessions(self) -> list[dict]:
if not self._available:
return []
try:
return await self.client.get_sessions()
except Exception:
return []
def build_spec(self, spec_payload: dict, skip_host_mount_validation: bool = False) -> BoxSpec:
spec_payload = dict(spec_payload)
spec_payload.setdefault('env', {})
if spec_payload.get('host_path') in (None, '') and self.default_workspace is not None:
spec_payload['host_path'] = self.default_workspace
if spec_payload.get('workspace_quota_mb') in (None, '') and self.workspace_quota_mb is not None:
spec_payload['workspace_quota_mb'] = self.workspace_quota_mb
# Global custom image overrides profile default (but not caller-specified image)
if self.custom_image and 'image' not in spec_payload:
spec_payload['image'] = self.custom_image
self._apply_profile(spec_payload)
try:
spec = BoxSpec.model_validate(spec_payload)
except pydantic.ValidationError as exc:
first_error = exc.errors()[0]
raise BoxValidationError(first_error.get('msg', 'invalid box arguments')) from exc
if not skip_host_mount_validation:
self._validate_host_mount(spec)
return spec
async def create_session(self, spec_payload: dict, *, skip_host_mount_validation: bool = False) -> dict:
spec = self.build_spec(spec_payload, skip_host_mount_validation=skip_host_mount_validation)
return await self.client.create_session(spec)
async def start_managed_process(self, session_id: str, process_payload: dict) -> BoxManagedProcessInfo:
process_spec = BoxManagedProcessSpec.model_validate(process_payload)
return await self.client.start_managed_process(session_id, process_spec)
async def get_managed_process(self, session_id: str, process_id: str = 'default') -> BoxManagedProcessInfo:
return await self.client.get_managed_process(session_id, process_id)
async def stop_managed_process(self, session_id: str, process_id: str = 'default') -> None:
return await self.client.stop_managed_process(session_id, process_id)
def get_managed_process_websocket_url(self, session_id: str, process_id: str = 'default') -> str:
getter = getattr(self.client, 'get_managed_process_websocket_url', None)
if getter is None:
raise BoxValidationError('box runtime client does not support managed process websocket attach')
ws_relay_base_url = (
self._runtime_connector.ws_relay_base_url
if self._runtime_connector is not None
else 'http://127.0.0.1:5410'
)
return getter(session_id, ws_relay_base_url, process_id)
async def list_skills(self) -> list[dict]:
return await self.client.list_skills()
async def get_skill(self, name: str) -> dict | None:
return await self.client.get_skill(name)
async def create_skill(self, skill: dict) -> dict:
return await self.client.create_skill(skill)
async def update_skill(self, name: str, skill: dict) -> dict:
return await self.client.update_skill(name, skill)
async def delete_skill(self, name: str) -> None:
await self.client.delete_skill(name)
async def scan_skill_directory(self, path: str) -> dict:
return await self.client.scan_skill_directory(path)
async def list_skill_files(
self,
name: str,
path: str = '.',
include_hidden: bool = False,
max_entries: int = 200,
) -> dict:
return await self.client.list_skill_files(name, path, include_hidden, max_entries)
async def read_skill_file(self, name: str, path: str) -> dict:
return await self.client.read_skill_file(name, path)
async def write_skill_file(self, name: str, path: str, content: str) -> dict:
return await self.client.write_skill_file(name, path, content)
async def preview_skill_zip(
self,
file_bytes: bytes,
filename: str,
source_subdir: str = '',
target_suffix: str = 'upload',
) -> list[dict]:
return await self.client.preview_skill_zip(file_bytes, filename, source_subdir, target_suffix)
async def install_skill_zip(
self,
file_bytes: bytes,
filename: str,
source_paths: list[str] | None = None,
source_path: str = '',
source_subdir: str = '',
target_suffix: str = 'upload',
) -> list[dict]:
return await self.client.install_skill_zip(
file_bytes,
filename,
source_paths,
source_path,
source_subdir,
target_suffix,
)
def _serialize_result(self, result: BoxExecutionResult) -> dict:
stdout, stdout_truncated = self._truncate(result.stdout)
stderr, stderr_truncated = self._truncate(result.stderr)
return {
'session_id': result.session_id,
'backend': result.backend_name,
'status': result.status.value,
'ok': result.ok,
'exit_code': result.exit_code,
'stdout': stdout,
'stderr': stderr,
'stdout_truncated': stdout_truncated,
'stderr_truncated': stderr_truncated,
'duration_ms': result.duration_ms,
}
def _truncate(self, text: str) -> tuple[str, bool]:
if len(text) <= self.output_limit_chars:
return text, False
if self.output_limit_chars <= 0:
return '', True
head_size = 0
tail_size = 0
notice = ''
# Recompute once the omitted count is known so the final payload
# stays within output_limit_chars even after adding the notice.
for _ in range(4):
omitted = max(len(text) - head_size - tail_size, 0)
notice = f'\n\n... [{omitted} characters truncated] ...\n\n'
available = self.output_limit_chars - len(notice)
if available <= 0:
return notice[: self.output_limit_chars], True
new_head_size = int(available * 0.6)
new_tail_size = available - new_head_size
if new_head_size == head_size and new_tail_size == tail_size:
break
head_size = new_head_size
tail_size = new_tail_size
head = text[:head_size]
tail = text[-tail_size:] if tail_size else ''
truncated = f'{head}{notice}{tail}'
return truncated[: self.output_limit_chars], True
def _summarize_spec(self, spec: BoxSpec) -> dict:
cmd = spec.cmd.strip()
if len(cmd) > 400:
cmd = f'{cmd[:397]}...'
return {
'session_id': spec.session_id,
'workdir': spec.workdir,
'mount_path': spec.mount_path,
'timeout_sec': spec.timeout_sec,
'network': spec.network.value,
'image': spec.image,
'host_path': spec.host_path,
'host_path_mode': spec.host_path_mode.value,
'cpus': spec.cpus,
'memory_mb': spec.memory_mb,
'pids_limit': spec.pids_limit,
'read_only_rootfs': spec.read_only_rootfs,
'workspace_quota_mb': spec.workspace_quota_mb,
'env_keys': sorted(spec.env.keys()),
'cmd': cmd,
}
def _summarize_result(self, result: BoxExecutionResult) -> dict:
stdout_preview = result.stdout[:200]
stderr_preview = result.stderr[:200]
if len(result.stdout) > 200:
stdout_preview = f'{stdout_preview}...'
if len(result.stderr) > 200:
stderr_preview = f'{stderr_preview}...'
return {
'session_id': result.session_id,
'backend': result.backend_name,
'status': result.status.value,
'exit_code': result.exit_code,
'duration_ms': result.duration_ms,
'stdout_preview': stdout_preview,
'stderr_preview': stderr_preview,
}
def _local_config(self) -> dict:
"""Return ``box.local`` from instance config.
Environment overrides are applied uniformly by
``LoadConfigStage._apply_env_overrides_to_config`` (e.g.
``BOX__LOCAL__HOST_ROOT``) before this is read, so no box-specific
env parsing happens here.
"""
return dict(_get_box_config(self.ap).get('local') or {})
def _load_allowed_mount_roots(self) -> list[str]:
configured_roots = self._local_config().get('allowed_mount_roots', [])
# The unified env-override mechanism stores a brand-new key as a raw
# string when the key is absent from config.yaml. Accept a
# comma-separated string as well as a list so that
# ``BOX__LOCAL__ALLOWED_MOUNT_ROOTS="/a,/b"`` keeps working even when
# the config file has no ``box.local.allowed_mount_roots`` entry.
if isinstance(configured_roots, str):
configured_roots = [item.strip() for item in configured_roots.split(',') if item.strip()]
normalized_roots: list[str] = []
for root in configured_roots:
root_value = str(root).strip()
if not root_value:
continue
normalized_roots.append(os.path.realpath(os.path.abspath(root_value)))
if not normalized_roots and self.host_root is not None:
normalized_roots.append(self.host_root)
return normalized_roots
def _load_host_root(self) -> str | None:
host_root = str(self._local_config().get('host_root', '')).strip()
if not host_root:
return None
return os.path.realpath(os.path.abspath(host_root))
def _load_default_workspace(self) -> str | None:
default_workspace = str(self._local_config().get('default_workspace', '')).strip()
if not default_workspace:
if self.host_root is None:
return None
default_workspace = os.path.join(self.host_root, 'default')
elif not os.path.isabs(default_workspace) and self.host_root is not None:
default_workspace = os.path.join(self.host_root, default_workspace)
return os.path.realpath(os.path.abspath(default_workspace))
def get_skills_root(self) -> str | None:
skills_root = str(self._local_config().get('skills_root', '') or 'skills').strip()
if not skills_root:
skills_root = 'skills'
if not os.path.isabs(skills_root) and self.host_root is not None:
skills_root = os.path.join(self.host_root, skills_root)
return os.path.realpath(os.path.abspath(skills_root))
def _load_enabled(self) -> bool:
"""Read ``box.enabled`` (top-level, not ``box.local.*``). Default True
disabling is opt-in. Accepts bool, ``'true'``/``'false'`` strings,
and the standard env-overridden truthy values that
``LoadConfigStage._apply_env_overrides_to_config`` produces."""
raw = _get_box_config(self.ap).get('enabled', True)
if isinstance(raw, bool):
return raw
return str(raw).strip().lower() not in ('false', '0', 'no', 'off', '')
def _load_custom_image(self) -> str | None:
raw = str(self._local_config().get('image', '') or '').strip()
return raw or None
def _forced_box_session_id_template(self) -> str:
"""Return the SaaS-forced sandbox-scope template, or '' when unset.
Read from ``system.limitation.force_box_session_id_template``. A
non-empty value pins every pipeline to a single sandbox scope
(e.g. ``'{global}'``) and cannot be overridden per-pipeline.
"""
limitation = (
(self.ap.instance_config.data or {}).get('system', {}).get('limitation', {})
if getattr(self.ap, 'instance_config', None) is not None
else {}
)
return str(limitation.get('force_box_session_id_template', '') or '').strip()
def _load_workspace_quota_mb(self) -> int | None:
raw_value = self._local_config().get('workspace_quota_mb')
if raw_value in (None, ''):
return None
try:
value = _INT_ADAPTER.validate_python(raw_value)
except pydantic.ValidationError as exc:
raise BoxValidationError('workspace_quota_mb must be an integer greater than or equal to 0') from exc
if value < 0:
raise BoxValidationError('workspace_quota_mb must be greater than or equal to 0')
return value
def _ensure_default_workspace(self):
if self.default_workspace is None:
return
if os.path.isdir(self.default_workspace):
return
if os.path.exists(self.default_workspace):
raise BoxValidationError('box.local.default_workspace must point to a directory on the host')
if not self.allowed_mount_roots:
raise BoxValidationError(
'box.local.default_workspace cannot be created because no allowed_mount_roots are configured'
)
for allowed_root in self.allowed_mount_roots:
if _is_path_under(self.default_workspace, allowed_root):
os.makedirs(self.default_workspace, exist_ok=True)
return
allowed_roots = ', '.join(self.allowed_mount_roots)
raise BoxValidationError(f'box.local.default_workspace is outside allowed_mount_roots: {allowed_roots}')
def _validate_host_mount(self, spec: BoxSpec):
if spec.host_path is None:
return
host_path = os.path.realpath(spec.host_path)
if not os.path.isdir(host_path):
raise BoxValidationError('host_path must point to an existing directory on the host')
if not self.allowed_mount_roots:
raise BoxValidationError('host_path mounting is disabled because no allowed_mount_roots are configured')
for allowed_root in self.allowed_mount_roots:
if _is_path_under(host_path, allowed_root):
return
allowed_roots = ', '.join(self.allowed_mount_roots)
raise BoxValidationError(f'host_path is outside allowed_mount_roots: {allowed_roots}')
def _load_profile(self) -> BoxProfile:
profile_name = str(self._local_config().get('profile', 'default')).strip() or 'default'
profile = BUILTIN_PROFILES.get(profile_name)
if profile is None:
available = ', '.join(sorted(BUILTIN_PROFILES))
raise BoxValidationError(f"unknown box profile '{profile_name}', available profiles: {available}")
return profile
def _apply_profile(self, params: dict):
"""Merge profile defaults into *params* in-place, enforce locked fields and clamp timeout."""
profile = self.profile
_PROFILE_FIELDS = (
'image',
'network',
'timeout_sec',
'host_path_mode',
'cpus',
'memory_mb',
'pids_limit',
'read_only_rootfs',
'workspace_quota_mb',
)
for field in _PROFILE_FIELDS:
profile_value = getattr(profile, field)
raw_value = profile_value.value if isinstance(profile_value, enum.Enum) else profile_value
if field in profile.locked:
params[field] = raw_value
elif field not in params:
params[field] = raw_value
timeout = params.get('timeout_sec')
try:
normalized_timeout = _INT_ADAPTER.validate_python(timeout)
except pydantic.ValidationError:
return
if normalized_timeout > profile.max_timeout_sec:
params['timeout_sec'] = profile.max_timeout_sec
def _get_workspace_size_bytes(self, root: str) -> int:
total = 0
def _walk(path: str):
nonlocal total
try:
with os.scandir(path) as entries:
for entry in entries:
try:
if entry.is_symlink():
total += entry.stat(follow_symlinks=False).st_size
continue
if entry.is_dir(follow_symlinks=False):
_walk(entry.path)
continue
total += entry.stat(follow_symlinks=False).st_size
except FileNotFoundError:
continue
except FileNotFoundError:
return
_walk(root)
return total
async def _enforce_workspace_quota(self, spec: BoxSpec, *, phase: str) -> None:
if spec.host_path is None or spec.workspace_quota_mb <= 0:
return
host_path = os.path.realpath(spec.host_path)
if not os.path.isdir(host_path):
return
# Walk the workspace off the event loop — this runs on every
# quota-enforced exec, and a large tree would otherwise block the whole
# asyncio runtime (all bots/pipelines) for the duration of the scan.
used_bytes = await asyncio.to_thread(self._get_workspace_size_bytes, host_path)
limit_bytes = spec.workspace_quota_mb * _MIB
if used_bytes <= limit_bytes:
return
raise BoxValidationError(
f'workspace quota exceeded {phase}: '
f'used={used_bytes} bytes limit={limit_bytes} bytes '
f'host_path={host_path} session_id={spec.session_id}'
)
async def _cleanup_exceeded_session(self, spec: BoxSpec) -> None:
try:
await self.client.delete_session(spec.session_id)
except Exception as exc:
self.ap.logger.warning(
'Failed to clean up Box session after workspace quota was exceeded: '
f'session_id={spec.session_id} error={exc}'
)
# ── Observability ─────────────────────────────────────────────────
def _record_error(self, exc: Exception, query: pipeline_query.Query):
self._recent_errors.append(
{
'timestamp': _dt.datetime.now(_UTC).isoformat(),
'type': type(exc).__name__,
'message': str(exc),
'query_id': str(query.query_id),
}
)
def get_recent_errors(self) -> list[dict]:
return list(self._recent_errors)
def get_system_guidance(self) -> str:
"""Return LLM system-prompt guidance for the exec tool.
All execution-specific prompt text is kept here so that callers
(e.g. LocalAgentRunner) stay free of box domain knowledge.
"""
guidance = (
'When the exec tool is available, use it for exact calculations, statistics, structured data parsing, '
'and code execution instead of estimating mentally. If the user provides numbers, tables, CSV-like text, '
'JSON, or other data and asks for a computed answer, prefer running a short Python script via exec '
'and then answer from the tool result. Unless the user explicitly asks for the script, code, or implementation '
'details, do not include the generated script in the final answer; return the result and a brief explanation only.'
)
if self.default_workspace:
guidance += (
' A default workspace is mounted at /workspace for file tasks. When the user asks to read, create, or '
'modify local files in the working directory, use exec with /workspace paths directly; do not ask the '
'user for directory parameters unless they explicitly need a different directory.'
)
return guidance
async def get_status(self) -> dict:
if not self._available:
return {
'available': False,
'enabled': self._enabled,
'profile': self.profile.name,
'recent_error_count': len(self._recent_errors),
'connector_error': self._connector_error,
}
try:
runtime_status = await self.client.get_status()
except Exception as exc:
# RPC failed — the runtime likely just disconnected and the
# heartbeat hasn't flipped _available yet.
return {
'available': False,
'enabled': self._enabled,
'profile': self.profile.name,
'recent_error_count': len(self._recent_errors),
'connector_error': str(exc),
}
# Backend state can be unavailable even when the connector is healthy
# (operator selected nsjail but the binary is missing, Docker daemon
# went down after the runtime started, E2B credentials wrong, ...).
# Report the combined state in the top-level ``available`` so the
# frontend banner / ``useBoxStatus`` hook / native-tool gate all
# agree on "actually usable" rather than "connector alive". The
# detailed ``backend`` object stays in the payload so the dialog
# can still show which backend was tried.
backend_info = runtime_status.get('backend') if isinstance(runtime_status, dict) else None
backend_ok = bool(backend_info and backend_info.get('available', False))
payload = {
**runtime_status,
'available': backend_ok,
'enabled': self._enabled,
'profile': self.profile.name,
'recent_error_count': len(self._recent_errors),
}
if not backend_ok and 'connector_error' not in payload:
backend_name = backend_info.get('name') if backend_info else None
if backend_name:
payload['connector_error'] = f'Configured sandbox backend "{backend_name}" is unavailable'
else:
payload['connector_error'] = 'No supported sandbox backend (Docker / nsjail / E2B) is available'
return payload
+413
View File
@@ -0,0 +1,413 @@
"""Reusable workspace/session helpers built on top of Box.
This module is the middle layer between the raw Box runtime primitives and
application-specific flows such as skills or MCP stdio.
It intentionally stays generic:
- path and virtualenv rewriting are workspace concerns
- Python project detection/bootstrap are workspace concerns
- session exec / managed-process helpers are workspace concerns
Higher layers add their own semantics on top, for example:
- skills choose a stable per-skill session id and use repeated exec
- MCP stdio chooses how to prepare dependencies and attaches to a managed process
"""
from __future__ import annotations
import os
import textwrap
from typing import Any
PYTHON_MANIFEST_FILES = (
'requirements.txt',
'pyproject.toml',
'setup.py',
'setup.cfg',
)
_VENV_DIRS = frozenset({'.venv', 'venv', 'env', '.env'})
_VENV_BIN_DIRS = frozenset({'bin', 'Scripts'})
def normalize_host_path(path: str | None) -> str:
if path is None:
return ''
stripped = str(path).strip()
if not stripped:
return ''
return os.path.realpath(os.path.abspath(stripped))
def rewrite_mounted_path(path: str, host_path: str | None, *, mount_path: str = '/workspace') -> str:
"""Translate a host path into the path visible inside the sandbox mount."""
if not host_path or not path:
return path
normalized_host = os.path.realpath(host_path)
normalized_path = os.path.realpath(path)
if normalized_path.startswith(normalized_host + '/'):
return mount_path + normalized_path[len(normalized_host) :]
if normalized_path == normalized_host:
return mount_path
return path
def unwrap_venv_path(directory: str) -> str:
"""Collapse ``.../.venv/bin`` style paths back to the project root."""
parts = directory.replace('\\', '/').split('/')
for i in range(len(parts) - 1, 0, -1):
if parts[i] in _VENV_BIN_DIRS and i >= 1:
venv_dir = parts[i - 1]
if venv_dir in _VENV_DIRS:
project_root = '/'.join(parts[: i - 1])
return project_root if project_root else '/'
return directory
def infer_workspace_host_path(command: str, args: list[str] | None = None) -> str | None:
"""Infer the project/workspace root from absolute command/arg paths."""
candidates: list[str] = []
for part in [command, *(args or [])]:
if not os.path.isabs(part):
continue
if os.path.exists(part):
directory = os.path.dirname(part)
candidates.append(os.path.realpath(unwrap_venv_path(directory)))
if not candidates:
return None
common = os.path.commonpath(candidates)
return common if common != '/' else None
def rewrite_venv_command(command: str, host_path: str | None, *, mount_path: str = '/workspace') -> str:
"""Rewrite host venv interpreters to plain ``python`` inside the sandbox.
Once a project is mounted into the sandbox, host virtualenv paths are no
longer valid. For those paths we intentionally drop down to ``python`` and
let the sandbox-side environment/bootstrap decide what interpreter to use.
"""
if not host_path or not command:
return command
normalized_host = os.path.realpath(host_path)
normalized_command = os.path.realpath(command)
if not normalized_command.startswith(normalized_host + '/'):
return command
rel = normalized_command[len(normalized_host) + 1 :]
parts = rel.replace('\\', '/').split('/')
if len(parts) >= 3 and parts[0] in _VENV_DIRS and parts[1] in _VENV_BIN_DIRS and parts[2].startswith('python'):
return 'python'
return rewrite_mounted_path(normalized_command, host_path, mount_path=mount_path)
def list_python_manifest_files(host_path: str | None) -> list[str]:
normalized_root = normalize_host_path(host_path)
if not normalized_root:
return []
return [filename for filename in PYTHON_MANIFEST_FILES if os.path.isfile(os.path.join(normalized_root, filename))]
def classify_python_workspace(host_path: str | None) -> str | None:
"""Return the generic Python workspace shape, without app-specific policy."""
manifest_files = set(list_python_manifest_files(host_path))
if not manifest_files:
return None
if {'pyproject.toml', 'setup.py', 'setup.cfg'} & manifest_files:
return 'package'
if 'requirements.txt' in manifest_files:
return 'requirements'
return None
def should_prepare_python_env(host_path: str | None) -> bool:
normalized_root = normalize_host_path(host_path)
if not normalized_root:
return False
if os.path.isdir(os.path.join(normalized_root, '.venv')):
return True
return bool(list_python_manifest_files(normalized_root))
def wrap_python_command_with_env(command: str, *, mount_path: str = '/workspace') -> str:
"""Wrap a command with a reusable sandbox-local Python env bootstrap.
This is the generic "workspace is a Python project" path used by mutable
workspaces such as skills. Read-only installation strategies stay in the
higher-level caller because they are application policy, not workspace
semantics.
"""
bootstrap = textwrap.dedent(
f"""
set -e
_LB_VENV_DIR="{mount_path}/.venv"
_LB_META_DIR="{mount_path}/.langbot"
_LB_META_FILE="$_LB_META_DIR/python-env.json"
_LB_LOCK_DIR="$_LB_META_DIR/python-env.lock"
_LB_TMP_DIR="{mount_path}/.tmp"
_LB_PIP_CACHE_DIR="{mount_path}/.cache/pip"
mkdir -p "$_LB_META_DIR" "$_LB_TMP_DIR" "$_LB_PIP_CACHE_DIR"
export TMPDIR="$_LB_TMP_DIR"
export TEMP="$_LB_TMP_DIR"
export TMP="$_LB_TMP_DIR"
export PIP_CACHE_DIR="$_LB_PIP_CACHE_DIR"
_lb_python_meta() {{
python - <<'PY'
import hashlib
import json
import os
import sys
root = "{mount_path}"
digest = hashlib.sha256()
manifest_files = []
for rel in ("requirements.txt", "pyproject.toml", "setup.py", "setup.cfg"):
path = os.path.join(root, rel)
if not os.path.isfile(path):
continue
manifest_files.append(rel)
with open(path, "rb") as handle:
digest.update(rel.encode("utf-8"))
digest.update(b"\\0")
digest.update(handle.read())
digest.update(b"\\0")
print(
json.dumps(
{{
"python_executable": sys.executable,
"python_version": list(sys.version_info[:3]),
"manifest_files": manifest_files,
"manifest_sha256": digest.hexdigest(),
}},
sort_keys=True,
)
)
PY
}}
_LB_CURRENT_META="$(_lb_python_meta)"
_LB_NEEDS_BOOTSTRAP=0
if [ ! -x "$_LB_VENV_DIR/bin/python" ]; then
_LB_NEEDS_BOOTSTRAP=1
elif [ ! -f "$_LB_META_FILE" ]; then
_LB_NEEDS_BOOTSTRAP=1
elif [ "$(cat "$_LB_META_FILE")" != "$_LB_CURRENT_META" ]; then
_LB_NEEDS_BOOTSTRAP=1
fi
if [ "$_LB_NEEDS_BOOTSTRAP" -eq 1 ]; then
_LB_LOCK_WAIT=0
while ! mkdir "$_LB_LOCK_DIR" 2>/dev/null; do
if [ "$_LB_LOCK_WAIT" -ge 120 ]; then
echo "Timed out waiting for Python environment lock: $_LB_LOCK_DIR" >&2
exit 1
fi
sleep 1
_LB_LOCK_WAIT=$((_LB_LOCK_WAIT + 1))
done
_lb_cleanup_lock() {{
rmdir "$_LB_LOCK_DIR" >/dev/null 2>&1 || true
}}
trap _lb_cleanup_lock EXIT INT TERM
_LB_CURRENT_META="$(_lb_python_meta)"
_LB_NEEDS_BOOTSTRAP=0
if [ ! -x "$_LB_VENV_DIR/bin/python" ]; then
_LB_NEEDS_BOOTSTRAP=1
elif [ ! -f "$_LB_META_FILE" ]; then
_LB_NEEDS_BOOTSTRAP=1
elif [ "$(cat "$_LB_META_FILE")" != "$_LB_CURRENT_META" ]; then
_LB_NEEDS_BOOTSTRAP=1
fi
if [ "$_LB_NEEDS_BOOTSTRAP" -eq 1 ]; then
rm -rf "$_LB_VENV_DIR"
python -m venv "$_LB_VENV_DIR"
. "$_LB_VENV_DIR/bin/activate"
python -m pip install --upgrade pip setuptools wheel
if [ -f "{mount_path}/requirements.txt" ]; then
python -m pip install -r "{mount_path}/requirements.txt"
elif [ -f "{mount_path}/pyproject.toml" ] || [ -f "{mount_path}/setup.py" ] || [ -f "{mount_path}/setup.cfg" ]; then
python -m pip install "{mount_path}"
fi
printf '%s' "$_LB_CURRENT_META" > "$_LB_META_FILE"
fi
fi
export VIRTUAL_ENV="$_LB_VENV_DIR"
export PATH="$_LB_VENV_DIR/bin:$PATH"
{command}
"""
).strip()
return bootstrap + '\n'
class BoxWorkspaceSession:
"""High-level handle for one reusable workspace-backed Box session.
The Box runtime already understands sessions and managed processes. This
wrapper adds LangBot's workspace-centric view on top: a mounted host path,
a stable ``session_id``, optional environment defaults, and convenience
helpers for exec or long-running processes inside that workspace.
"""
def __init__(
self,
box_service,
session_id: str,
*,
host_path: str | None = None,
host_path_mode: str = 'rw',
workdir: str = '/workspace',
env: dict[str, str] | None = None,
mount_path: str = '/workspace',
network: str | None = None,
read_only_rootfs: bool | None = None,
image: str | None = None,
cpus: float | None = None,
memory_mb: int | None = None,
pids_limit: int | None = None,
persistent: bool = False,
):
self.box_service = box_service
self.session_id = session_id
self.host_path = host_path
self.host_path_mode = host_path_mode
self.workdir = workdir
self.env = dict(env or {})
self.mount_path = mount_path
self.network = network
self.read_only_rootfs = read_only_rootfs
self.image = image
self.cpus = cpus
self.memory_mb = memory_mb
self.pids_limit = pids_limit
self.persistent = persistent
def rewrite_path(self, path: str) -> str:
return rewrite_mounted_path(path, self.host_path, mount_path=self.mount_path)
def rewrite_venv_command(self, command: str) -> str:
return rewrite_venv_command(command, self.host_path, mount_path=self.mount_path)
def build_session_payload(self) -> dict[str, Any]:
# Keep this payload generic so callers can reuse the same workspace
# handle for plain exec, file-producing tasks, or managed processes.
payload: dict[str, Any] = {
'session_id': self.session_id,
'workdir': self.workdir,
'env': self.env,
'persistent': self.persistent,
}
if self.network is not None:
payload['network'] = self.network
if self.read_only_rootfs is not None:
payload['read_only_rootfs'] = self.read_only_rootfs
if self.host_path:
payload['host_path'] = self.host_path
payload['host_path_mode'] = self.host_path_mode
for key in ('image', 'cpus', 'memory_mb', 'pids_limit'):
value = getattr(self, key)
if value is not None:
payload[key] = value
return payload
def build_exec_payload(
self,
cmd: str,
*,
workdir: str | None = None,
env: dict[str, str] | None = None,
timeout_sec: int | None = None,
) -> dict[str, Any]:
# Exec payloads inherit the session-level workspace config, then layer
# per-call command/workdir/env overrides on top.
payload = self.build_session_payload()
payload['cmd'] = cmd
payload['workdir'] = workdir or self.workdir
if timeout_sec is not None:
payload['timeout_sec'] = timeout_sec
resolved_env = self.env if env is None else env
if resolved_env:
payload['env'] = resolved_env
elif 'env' in payload and not payload['env']:
payload.pop('env')
return payload
async def execute_raw(
self,
cmd: str,
*,
workdir: str | None = None,
env: dict[str, str] | None = None,
timeout_sec: int | None = None,
):
payload = self.build_exec_payload(cmd, workdir=workdir, env=env, timeout_sec=timeout_sec)
return await self.box_service.client.execute(self.box_service.build_spec(payload))
async def execute_for_query(
self,
query,
cmd: str,
*,
workdir: str | None = None,
env: dict[str, str] | None = None,
timeout_sec: int | None = None,
) -> dict:
payload = self.build_exec_payload(cmd, workdir=workdir, env=env, timeout_sec=timeout_sec)
return await self.box_service.execute_spec_payload(payload, query)
async def create_session(self):
return await self.box_service.create_session(self.build_session_payload())
def build_process_payload(
self,
command: str,
args: list[str] | None = None,
*,
env: dict[str, str] | None = None,
cwd: str = '/workspace',
) -> dict[str, Any]:
# Managed processes run inside the same workspace model as one-shot
# execs, so path/venv rewriting is shared here.
normalized_command = command
normalized_args = list(args or [])
normalized_cwd = cwd
if self.host_path:
normalized_command = self.rewrite_venv_command(command)
normalized_args = [self.rewrite_path(arg) for arg in normalized_args]
normalized_cwd = self.rewrite_path(cwd)
return {
'command': normalized_command,
'args': normalized_args,
'env': dict(env or {}),
'cwd': normalized_cwd,
}
async def start_managed_process(
self,
command: str,
args: list[str] | None = None,
*,
process_id: str = 'default',
env: dict[str, str] | None = None,
cwd: str = '/workspace',
):
payload = self.build_process_payload(command, args, env=env, cwd=cwd)
payload['process_id'] = process_id
return await self.box_service.start_managed_process(self.session_id, payload)
async def get_managed_process(self, process_id: str = 'default'):
return await self.box_service.get_managed_process(self.session_id, process_id)
async def stop_managed_process(self, process_id: str = 'default') -> None:
await self.box_service.stop_managed_process(self.session_id, process_id)
def get_managed_process_websocket_url(self, process_id: str = 'default') -> str:
return self.box_service.get_managed_process_websocket_url(self.session_id, process_id)
async def cleanup(self) -> None:
await self.box_service.client.delete_session(self.session_id)
+12 -2
View File
@@ -9,6 +9,7 @@ from ..platform import botmgr as im_mgr
from ..platform.webhook_pusher import WebhookPusher from ..platform.webhook_pusher import WebhookPusher
from ..provider.session import sessionmgr as llm_session_mgr from ..provider.session import sessionmgr as llm_session_mgr
from ..provider.modelmgr import modelmgr as llm_model_mgr from ..provider.modelmgr import modelmgr as llm_model_mgr
from ..box import service as box_service_module
from langbot.pkg.provider.tools import toolmgr as llm_tool_mgr from langbot.pkg.provider.tools import toolmgr as llm_tool_mgr
from ..config import manager as config_mgr from ..config import manager as config_mgr
@@ -31,8 +32,8 @@ from ..api.http.service import mcp as mcp_service
from ..api.http.service import apikey as apikey_service from ..api.http.service import apikey as apikey_service
from ..api.http.service import webhook as webhook_service from ..api.http.service import webhook as webhook_service
from ..api.http.service import monitoring as monitoring_service from ..api.http.service import monitoring as monitoring_service
from ..api.http.service import skill as skill_service
from ..api.http.service import maintenance as maintenance_service from ..api.http.service import maintenance as maintenance_service
from ..discover import engine as discover_engine from ..discover import engine as discover_engine
from ..storage import mgr as storagemgr from ..storage import mgr as storagemgr
from ..utils import logcache from ..utils import logcache
@@ -43,6 +44,7 @@ from ..rag.service import RAGRuntimeService
from ..vector import mgr as vectordb_mgr from ..vector import mgr as vectordb_mgr
from ..telemetry import telemetry as telemetry_module from ..telemetry import telemetry as telemetry_module
from ..survey import manager as survey_module from ..survey import manager as survey_module
from ..skill import manager as skill_mgr
class Application: class Application:
@@ -70,6 +72,7 @@ class Application:
# TODO move to pipeline # TODO move to pipeline
tool_mgr: llm_tool_mgr.ToolManager = None tool_mgr: llm_tool_mgr.ToolManager = None
box_service: box_service_module.BoxService = None
# ======= Config manager ======= # ======= Config manager =======
@@ -156,6 +159,10 @@ class Application:
monitoring_service: monitoring_service.MonitoringService = None monitoring_service: monitoring_service.MonitoringService = None
skill_service: skill_service.SkillService = None
skill_mgr: skill_mgr.SkillManager = None
maintenance_service: maintenance_service.MaintenanceService = None maintenance_service: maintenance_service.MaintenanceService = None
def __init__(self): def __init__(self):
@@ -301,7 +308,10 @@ class Application:
return parsed return parsed
def dispose(self): def dispose(self):
self.plugin_connector.dispose() if self.plugin_connector is not None:
self.plugin_connector.dispose()
if self.box_service is not None:
self.box_service.dispose()
async def print_web_access_info(self): async def print_web_access_info(self):
"""Print access webui tips""" """Print access webui tips"""
+2
View File
@@ -62,4 +62,6 @@ async def main(loop: asyncio.AbstractEventLoop):
app_inst = await make_app(loop) app_inst = await make_app(loop)
await app_inst.run() await app_inst.run()
except Exception: except Exception:
if app_inst is not None:
app_inst.dispose()
traceback.print_exc() traceback.print_exc()
@@ -0,0 +1,27 @@
from __future__ import annotations
from .. import migration
@migration.migration_class('weknora-api-config', 42)
class WeKnoraAPICfgMigration(migration.Migration):
"""WeKnora API 配置迁移"""
async def need_migrate(self) -> bool:
"""判断当前环境是否需要运行此迁移"""
return 'weknora-api' not in self.ap.provider_cfg.data
async def run(self):
"""执行迁移"""
self.ap.provider_cfg.data['weknora-api'] = {
'base-url': 'http://localhost:8080/api/v1',
'app-type': 'agent',
'api-key': '',
'agent-id': 'builtin-smart-reasoning',
'knowledge-base-ids': [],
'web-search-enabled': False,
'timeout': 120,
'base-prompt': '请回答用户的问题。',
}
await self.ap.provider_cfg.dump_config()
@@ -0,0 +1,30 @@
from __future__ import annotations
from .. import migration
@migration.migration_class('deerflow-api-config', 43)
class DeerFlowAPICfgMigration(migration.Migration):
"""DeerFlow API 配置迁移"""
async def need_migrate(self) -> bool:
"""判断当前环境是否需要运行此迁移"""
return 'deerflow-api' not in self.ap.provider_cfg.data
async def run(self):
"""执行迁移"""
self.ap.provider_cfg.data['deerflow-api'] = {
'api-base': 'http://127.0.0.1:2026',
'api-key': '',
'auth-header': '',
'assistant-id': 'lead_agent',
'model-name': '',
'thinking-enabled': False,
'plan-mode': False,
'subagent-enabled': False,
'max-concurrent-subagents': 3,
'timeout': 300,
'recursion-limit': 1000,
}
await self.ap.provider_cfg.dump_config()
+15
View File
@@ -6,6 +6,7 @@ from .. import stage, app
from ...utils import version, proxy from ...utils import version, proxy
from ...pipeline import pool, controller, pipelinemgr from ...pipeline import pool, controller, pipelinemgr
from ...pipeline import aggregator as message_aggregator from ...pipeline import aggregator as message_aggregator
from ...box import service as box_service
from ...plugin import connector as plugin_connector from ...plugin import connector as plugin_connector
from ...command import cmdmgr from ...command import cmdmgr
from ...provider.session import sessionmgr as llm_session_mgr from ...provider.session import sessionmgr as llm_session_mgr
@@ -28,6 +29,8 @@ from ...api.http.service import mcp as mcp_service
from ...api.http.service import apikey as apikey_service from ...api.http.service import apikey as apikey_service
from ...api.http.service import webhook as webhook_service from ...api.http.service import webhook as webhook_service
from ...api.http.service import monitoring as monitoring_service from ...api.http.service import monitoring as monitoring_service
from ...api.http.service import skill as skill_service
from ...skill import manager as skill_mgr
from ...api.http.service import maintenance as maintenance_service from ...api.http.service import maintenance as maintenance_service
from ...discover import engine as discover_engine from ...discover import engine as discover_engine
from ...storage import mgr as storagemgr from ...storage import mgr as storagemgr
@@ -86,6 +89,9 @@ class BuildAppStage(stage.BootingStage):
webhook_service_inst = webhook_service.WebhookService(ap) webhook_service_inst = webhook_service.WebhookService(ap)
ap.webhook_service = webhook_service_inst ap.webhook_service = webhook_service_inst
skill_service_inst = skill_service.SkillService(ap)
ap.skill_service = skill_service_inst
proxy_mgr = proxy.ProxyManager(ap) proxy_mgr = proxy.ProxyManager(ap)
await proxy_mgr.initialize() await proxy_mgr.initialize()
ap.proxy_mgr = proxy_mgr ap.proxy_mgr = proxy_mgr
@@ -129,6 +135,10 @@ class BuildAppStage(stage.BootingStage):
await llm_session_mgr_inst.initialize() await llm_session_mgr_inst.initialize()
ap.sess_mgr = llm_session_mgr_inst ap.sess_mgr = llm_session_mgr_inst
box_service_inst = box_service.BoxService(ap)
await box_service_inst.initialize()
ap.box_service = box_service_inst
llm_tool_mgr_inst = llm_tool_mgr.ToolManager(ap) llm_tool_mgr_inst = llm_tool_mgr.ToolManager(ap)
await llm_tool_mgr_inst.initialize() await llm_tool_mgr_inst.initialize()
ap.tool_mgr = llm_tool_mgr_inst ap.tool_mgr = llm_tool_mgr_inst
@@ -149,6 +159,11 @@ class BuildAppStage(stage.BootingStage):
msg_aggregator_inst = message_aggregator.MessageAggregator(ap) msg_aggregator_inst = message_aggregator.MessageAggregator(ap)
ap.msg_aggregator = msg_aggregator_inst ap.msg_aggregator = msg_aggregator_inst
# Initialize skill manager
skill_mgr_inst = skill_mgr.SkillManager(ap)
await skill_mgr_inst.initialize()
ap.skill_mgr = skill_mgr_inst
rag_mgr_inst = rag_mgr.RAGManager(ap) rag_mgr_inst = rag_mgr.RAGManager(ap)
await rag_mgr_inst.initialize() await rag_mgr_inst.initialize()
ap.rag_mgr = rag_mgr_inst ap.rag_mgr = rag_mgr_inst
@@ -11,6 +11,10 @@ class MCPServer(Base):
enable = sqlalchemy.Column(sqlalchemy.Boolean, nullable=False, default=False) enable = sqlalchemy.Column(sqlalchemy.Boolean, nullable=False, default=False)
mode = sqlalchemy.Column(sqlalchemy.String(255), nullable=False) # stdio, sse, http mode = sqlalchemy.Column(sqlalchemy.String(255), nullable=False) # stdio, sse, http
extra_args = sqlalchemy.Column(sqlalchemy.JSON, nullable=False, default={}) extra_args = sqlalchemy.Column(sqlalchemy.JSON, nullable=False, default={})
# Markdown documentation captured from LangBot Space at install time so the
# detail page can show docs even when the server is offline / has no tools.
# Empty string for manually-created servers that have no marketplace README.
readme = sqlalchemy.Column(sqlalchemy.Text, nullable=False, server_default='', default='')
created_at = sqlalchemy.Column(sqlalchemy.DateTime, nullable=False, server_default=sqlalchemy.func.now()) created_at = sqlalchemy.Column(sqlalchemy.DateTime, nullable=False, server_default=sqlalchemy.func.now())
updated_at = sqlalchemy.Column( updated_at = sqlalchemy.Column(
sqlalchemy.DateTime, sqlalchemy.DateTime,
@@ -0,0 +1,34 @@
"""add readme column to mcp_servers
Revision ID: 0004_add_mcp_readme
Revises: 0003_add_rerank_models
Create Date: 2026-06-06
"""
import sqlalchemy as sa
from alembic import op
revision = '0004_add_mcp_readme'
down_revision = '0003_add_rerank_models'
branch_labels = None
depends_on = None
def upgrade() -> None:
# Add ``readme`` to mcp_servers if the table exists and the column is missing
# (the table may have been created by create_all() with the column already
# present on fresh installs, so guard against duplicate-add).
conn = op.get_bind()
inspector = sa.inspect(conn)
if 'mcp_servers' not in inspector.get_table_names():
return
columns = {col['name'] for col in inspector.get_columns('mcp_servers')}
if 'readme' not in columns:
op.add_column(
'mcp_servers',
sa.Column('readme', sa.Text(), nullable=False, server_default=''),
)
def downgrade() -> None:
op.drop_column('mcp_servers', 'readme')
+1 -1
View File
@@ -157,7 +157,7 @@ class RuntimePipeline:
bot_message=query.resp_messages[-1], bot_message=query.resp_messages[-1],
message=result.user_notice, message=result.user_notice,
quote_origin=query.pipeline_config['output']['misc']['quote-origin'], quote_origin=query.pipeline_config['output']['misc']['quote-origin'],
is_final=[msg.is_final for msg in query.resp_messages][-1], is_final=[msg.is_final for msg in query.resp_messages][0],
) )
else: else:
await query.adapter.reply_message( await query.adapter.reply_message(
+1 -5
View File
@@ -42,13 +42,9 @@ class QueryPool:
adapter: abstract_platform_adapter.AbstractMessagePlatformAdapter, adapter: abstract_platform_adapter.AbstractMessagePlatformAdapter,
pipeline_uuid: typing.Optional[str] = None, pipeline_uuid: typing.Optional[str] = None,
routed_by_rule: bool = False, routed_by_rule: bool = False,
variables: typing.Optional[dict[str, typing.Any]] = None,
) -> pipeline_query.Query: ) -> pipeline_query.Query:
async with self.condition: async with self.condition:
query_id = self.query_id_counter query_id = self.query_id_counter
initial_variables: dict[str, typing.Any] = {'_routed_by_rule': routed_by_rule}
if variables:
initial_variables.update(variables)
query = pipeline_query.Query( query = pipeline_query.Query(
bot_uuid=bot_uuid, bot_uuid=bot_uuid,
query_id=query_id, query_id=query_id,
@@ -57,7 +53,7 @@ class QueryPool:
sender_id=sender_id, sender_id=sender_id,
message_event=message_event, message_event=message_event,
message_chain=message_chain, message_chain=message_chain,
variables=initial_variables, variables={'_routed_by_rule': routed_by_rule},
resp_messages=[], resp_messages=[],
resp_message_chain=[], resp_message_chain=[],
adapter=adapter, adapter=adapter,
+76 -2
View File
@@ -32,6 +32,9 @@ class PreProcessor(stage.PipelineStage):
) -> entities.StageProcessResult: ) -> entities.StageProcessResult:
"""Process""" """Process"""
selected_runner = query.pipeline_config['ai']['runner']['runner'] selected_runner = query.pipeline_config['ai']['runner']['runner']
include_skill_authoring = (
selected_runner == 'local-agent' and getattr(self.ap, 'skill_service', None) is not None
)
session = await self.ap.sess_mgr.get_session(query) session = await self.ap.sess_mgr.get_session(query)
@@ -110,7 +113,11 @@ class PreProcessor(stage.PipelineStage):
# Get bound plugins and MCP servers for filtering tools # Get bound plugins and MCP servers for filtering tools
bound_plugins = query.variables.get('_pipeline_bound_plugins', None) bound_plugins = query.variables.get('_pipeline_bound_plugins', None)
bound_mcp_servers = query.variables.get('_pipeline_bound_mcp_servers', None) bound_mcp_servers = query.variables.get('_pipeline_bound_mcp_servers', None)
query.use_funcs = await self.ap.tool_mgr.get_all_tools(bound_plugins, bound_mcp_servers) query.use_funcs = await self.ap.tool_mgr.get_all_tools(
bound_plugins,
bound_mcp_servers,
include_skill_authoring=include_skill_authoring,
)
self.ap.logger.debug(f'Bound plugins: {bound_plugins}') self.ap.logger.debug(f'Bound plugins: {bound_plugins}')
self.ap.logger.debug(f'Bound MCP servers: {bound_mcp_servers}') self.ap.logger.debug(f'Bound MCP servers: {bound_mcp_servers}')
@@ -121,7 +128,11 @@ class PreProcessor(stage.PipelineStage):
if not query.use_funcs and query.variables.get('_fallback_model_uuids'): if not query.use_funcs and query.variables.get('_fallback_model_uuids'):
bound_plugins = query.variables.get('_pipeline_bound_plugins', None) bound_plugins = query.variables.get('_pipeline_bound_plugins', None)
bound_mcp_servers = query.variables.get('_pipeline_bound_mcp_servers', None) bound_mcp_servers = query.variables.get('_pipeline_bound_mcp_servers', None)
query.use_funcs = await self.ap.tool_mgr.get_all_tools(bound_plugins, bound_mcp_servers) query.use_funcs = await self.ap.tool_mgr.get_all_tools(
bound_plugins,
bound_mcp_servers,
include_skill_authoring=include_skill_authoring,
)
sender_name = '' sender_name = ''
@@ -237,4 +248,67 @@ class PreProcessor(stage.PipelineStage):
query.prompt.messages = event_ctx.event.default_prompt query.prompt.messages = event_ctx.event.default_prompt
query.messages = event_ctx.event.prompt query.messages = event_ctx.event.prompt
# =========== Skill awareness for the local-agent runner ===========
# The actual activation goes through the ``activate`` Tool Call so the
# LLM doesn't see full SKILL.md instructions until it commits to a
# skill (Claude Code's progressive disclosure). But the LLM still has
# to KNOW which skills exist to make that choice, so we:
# 1. resolve the pipeline's bound skills and stash them in
# ``query.variables['_pipeline_bound_skills']`` for downstream
# visibility checks (skill loader, native exec workdir);
# 2. inject a short ``Available Skills`` index (name + description
# only) into the system prompt. The contributor's original PR
# relied on this injection; without it the LLM never discovers
# the skills are there and just calls native tools instead.
if selected_runner == 'local-agent' and self.ap.skill_mgr:
pipeline_data = await self.ap.pipeline_service.get_pipeline(query.pipeline_uuid)
extensions_prefs = (pipeline_data or {}).get('extensions_preferences', {})
enable_all_skills = extensions_prefs.get('enable_all_skills', True)
if enable_all_skills:
bound_skills = None # None = all loaded skills are visible
else:
bound_skills = extensions_prefs.get('skills', [])
query.variables['_pipeline_bound_skills'] = bound_skills
skill_addition = self.ap.skill_mgr.build_skill_aware_prompt_addition(
bound_skills=bound_skills,
)
if skill_addition:
# Append to the first system message; create one if the
# prompt has none. Handles both plain-string and
# content-element (list) message bodies.
if query.prompt.messages and query.prompt.messages[0].role == 'system':
head = query.prompt.messages[0]
if isinstance(head.content, str):
head.content = head.content + skill_addition
elif isinstance(head.content, list):
appended = False
for ce in head.content:
if getattr(ce, 'type', None) == 'text':
ce.text = (ce.text or '') + skill_addition
appended = True
break
if not appended:
head.content.append(provider_message.ContentElement(type='text', text=skill_addition))
else:
query.prompt.messages.insert(
0,
provider_message.Message(role='system', content=skill_addition.strip()),
)
self.ap.logger.debug(
f'Skill index injected into system prompt: '
f'pipeline={query.pipeline_uuid} '
f'bound_skills={bound_skills or "all"} '
f'loaded_skills={len(self.ap.skill_mgr.skills)}'
)
else:
self.ap.logger.debug(
f'No skills available for prompt injection: '
f'pipeline={query.pipeline_uuid} '
f'loaded_skills={len(self.ap.skill_mgr.skills)} '
f'bound_skills={bound_skills}'
)
return entities.StageProcessResult(result_type=entities.ResultType.CONTINUE, new_query=query) return entities.StageProcessResult(result_type=entities.ResultType.CONTINUE, new_query=query)
@@ -5,6 +5,7 @@ import abc
from ...core import app from ...core import app
from .. import entities from .. import entities
import langbot_plugin.api.entities.builtin.pipeline.query as pipeline_query import langbot_plugin.api.entities.builtin.pipeline.query as pipeline_query
import langbot_plugin.api.entities.builtin.provider.message as provider_message
class MessageHandler(metaclass=abc.ABCMeta): class MessageHandler(metaclass=abc.ABCMeta):
@@ -31,3 +32,29 @@ class MessageHandler(metaclass=abc.ABCMeta):
if len(s0) > 20 or '\n' in s: if len(s0) > 20 or '\n' in s:
s0 = s0[:20] + '...' s0 = s0[:20] + '...'
return s0 return s0
def format_result_log(
self,
result: provider_message.Message | provider_message.MessageChunk,
) -> str | None:
if result.tool_calls:
tool_names = [tc.function.name for tc in result.tool_calls if tc.function and tc.function.name]
if tool_names:
return f'{result.role}: requested tools: {", ".join(tool_names)}'
return f'{result.role}: requested tool calls'
content = result.content
if isinstance(content, str):
if not content.strip():
return None
if result.role == 'tool':
if content.startswith('err:'):
return f'tool error: {self.cut_str(content)}'
return self.cut_str(result.readable_str())
if isinstance(content, list) and len(content) == 0:
return None
return self.cut_str(result.readable_str())
@@ -113,9 +113,11 @@ class ChatMessageHandler(handler.MessageHandler):
# This prevents memory overflow from thousands of log entries per conversation # This prevents memory overflow from thousands of log entries per conversation
# First chunk uses INFO level to confirm connection establishment # First chunk uses INFO level to confirm connection establishment
if chunk_count == 1: if chunk_count == 1:
self.ap.logger.info( summary = self.format_result_log(result)
f'Conversation({query.query_id}) Streaming started: {self.cut_str(result.readable_str())}' if summary is not None:
) self.ap.logger.info(f'Conversation({query.query_id}) Streaming started: {summary}')
else:
self.ap.logger.info(f'Conversation({query.query_id}) Streaming started')
elif chunk_count % 10 == 0: elif chunk_count % 10 == 0:
self.ap.logger.debug( self.ap.logger.debug(
f'Conversation({query.query_id}) Streaming chunk {chunk_count}: {self.cut_str(result.readable_str())}' f'Conversation({query.query_id}) Streaming chunk {chunk_count}: {self.cut_str(result.readable_str())}'
@@ -135,9 +137,9 @@ class ChatMessageHandler(handler.MessageHandler):
async for result in runner.run(query): async for result in runner.run(query):
query.resp_messages.append(result) query.resp_messages.append(result)
self.ap.logger.info( summary = self.format_result_log(result)
f'Conversation({query.query_id}) Response: {self.cut_str(result.readable_str())}' if summary is not None:
) self.ap.logger.info(f'Conversation({query.query_id}) Response: {summary}')
if result.content is not None: if result.content is not None:
text_length += len(result.content) text_length += len(result.content)
@@ -40,7 +40,7 @@ class SendResponseBackStage(stage.PipelineStage):
has_chunks = any(isinstance(msg, provider_message.MessageChunk) for msg in query.resp_messages) has_chunks = any(isinstance(msg, provider_message.MessageChunk) for msg in query.resp_messages)
# TODO 命令与流式的兼容性问题 # TODO 命令与流式的兼容性问题
if await query.adapter.is_stream_output_supported() and has_chunks: if await query.adapter.is_stream_output_supported() and has_chunks:
is_final = [msg.is_final for msg in query.resp_messages][-1] is_final = [msg.is_final for msg in query.resp_messages][0]
await query.adapter.reply_message_chunk( await query.adapter.reply_message_chunk(
message_source=query.message_event, message_source=query.message_event,
bot_message=query.resp_messages[-1], bot_message=query.resp_messages[-1],
-2
View File
@@ -501,8 +501,6 @@ class PlatformManager:
bot_entity.adapter_config, bot_entity.adapter_config,
logger, logger,
) )
if hasattr(adapter_inst, 'ap'):
adapter_inst.ap = self.ap
# 如果 adapter 支持 set_bot_uuid 方法,设置 bot_uuid(用于统一 webhook # 如果 adapter 支持 set_bot_uuid 方法,设置 bot_uuid(用于统一 webhook
if hasattr(adapter_inst, 'set_bot_uuid'): if hasattr(adapter_inst, 'set_bot_uuid'):
+17 -761
View File
@@ -1,20 +1,13 @@
import asyncio
import json
import pathlib
import traceback import traceback
import typing import typing
import uuid
from langbot.libs.dingtalk_api.dingtalkevent import DingTalkEvent from langbot.libs.dingtalk_api.dingtalkevent import DingTalkEvent
import langbot_plugin.api.entities.builtin.platform.message as platform_message import langbot_plugin.api.entities.builtin.platform.message as platform_message
import langbot_plugin.api.definition.abstract.platform.adapter as abstract_platform_adapter import langbot_plugin.api.definition.abstract.platform.adapter as abstract_platform_adapter
import langbot_plugin.api.entities.builtin.platform.events as platform_events import langbot_plugin.api.entities.builtin.platform.events as platform_events
import langbot_plugin.api.entities.builtin.platform.entities as platform_entities import langbot_plugin.api.entities.builtin.platform.entities as platform_entities
import langbot_plugin.api.entities.builtin.provider.session as provider_session
from langbot.libs.dingtalk_api.api import DingTalkClient from langbot.libs.dingtalk_api.api import DingTalkClient
import datetime import datetime
from langbot.pkg.platform.logger import EventLogger from langbot.pkg.platform.logger import EventLogger
from langbot.pkg.provider.runners.difysvapi import _format_human_input_text
class DingTalkMessageConverter(abstract_platform_adapter.AbstractMessageConverter): class DingTalkMessageConverter(abstract_platform_adapter.AbstractMessageConverter):
@@ -177,40 +170,6 @@ class DingTalkAdapter(abstract_platform_adapter.AbstractMessagePlatformAdapter):
card_instance_id_dict: ( card_instance_id_dict: (
dict # 回复卡片消息字典,key为消息id,value为回复卡片实例id,用于在流式消息时判断是否发送到指定卡片 dict # 回复卡片消息字典,key为消息id,value为回复卡片实例id,用于在流式消息时判断是否发送到指定卡片
) )
# outTrackId → form snapshot {session_key, launcher_type, launcher_id, form_token,
# workflow_run_id, actions, node_title, form_content, expires_at, open_space_id,
# user_id_hint, current_text}. Lookup keys for the data-source pull endpoint and
# the STREAM card-action callback.
card_state: dict
# session_key → out_track_id of the currently-active card for the
# conversation turn. Lets resumed-workflow chunks (which arrive on a
# synthetic event with a fresh resp_message_id) keep updating the same
# card the user clicked instead of getting a new one.
active_turn_card: dict
# session_key → accumulated streaming text for the active turn. Read
# by _paint_form_on_card so the post-pause form keeps the streamed
# context above the new prompt.
active_turn_text: dict
# event_type → callback. The abstract base class doesn't declare this,
# so we must do it here or pydantic silently drops `listeners={}` in
# super().__init__ and any access raises AttributeError.
listeners: typing.Dict[
typing.Type[platform_events.Event],
typing.Callable[[platform_events.Event, abstract_platform_adapter.AbstractMessagePlatformAdapter], None],
]
ap: typing.Any = None
bot_uuid: str = ''
# DingTalk media_id (`@xxx` format) for the bot avatar image, fetched
# on adapter startup by uploading the bundled LangBot logo via the
# legacy /media/upload endpoint. Empty string when the upload hasn't
# run yet or failed — the template's Avatar then falls back to its
# default (initials of `name`).
bot_avatar_media_id: str = ''
# Path to the LangBot logo bundled in the repo (`res/logo-blue.png`),
# resolved relative to this file. Updated to find the file even when
# LangBot is installed as a package or run from a different cwd.
_LOGO_PATH: typing.ClassVar[pathlib.Path] = pathlib.Path(__file__).resolve().parents[5] / 'res' / 'logo-blue.png'
def __init__(self, config: dict, logger: EventLogger): def __init__(self, config: dict, logger: EventLogger):
required_keys = [ required_keys = [
@@ -235,17 +194,10 @@ class DingTalkAdapter(abstract_platform_adapter.AbstractMessagePlatformAdapter):
config=config, config=config,
logger=logger, logger=logger,
card_instance_id_dict={}, card_instance_id_dict={},
card_state={},
active_turn_card={},
active_turn_text={},
bot_account_id=bot_account_id, bot_account_id=bot_account_id,
bot=bot, bot=bot,
listeners={}, listeners={},
) )
# Wire the card-action callback after super().__init__ so we can reference
# self.* — the client's handler stores this as a soft reference and reads
# it at fire time.
self.bot.card_action_callback = self._on_card_action
async def reply_message( async def reply_message(
self, self,
@@ -270,85 +222,28 @@ class DingTalkAdapter(abstract_platform_adapter.AbstractMessagePlatformAdapter):
quote_origin: bool = False, quote_origin: bool = False,
is_final: bool = False, is_final: bool = False,
): ):
# event = await DingTalkEventConverter.yiri2target(
# message_source,
# )
# incoming_message = event.incoming_message
# msg_id = incoming_message.message_id
message_id = bot_message.resp_message_id message_id = bot_message.resp_message_id
msg_seq = bot_message.msg_sequence msg_seq = bot_message.msg_sequence
form_template_id = (self.config.get('human_input_card_template_id') or '').strip()
form_data = getattr(bot_message, '_form_data', None)
if is_final and self.ap is not None:
self.ap.logger.info(
f'DingTalk reply_message_chunk final: form_data_present={form_data is not None}, '
f'form_template_configured={bool(form_template_id)}'
)
if form_data and is_final:
await self._handle_form_chunk(message_source, bot_message, message, form_data)
return
if (msg_seq - 1) % 8 == 0 or is_final: if (msg_seq - 1) % 8 == 0 or is_final:
markdown_enabled = self.config.get('markdown_card', False) markdown_enabled = self.config.get('markdown_card', False)
content, at = await DingTalkMessageConverter.yiri2target(message, markdown_enabled) content, at = await DingTalkMessageConverter.yiri2target(message, markdown_enabled)
card_instance, card_instance_id = self.card_instance_id_dict[message_id]
if not content and bot_message.content: if not content and bot_message.content:
content = bot_message.content # 兼容直接传入content的情况 content = bot_message.content # 兼容直接传入content的情况
# print(card_instance_id)
chat_card_entry = self.card_instance_id_dict.get(message_id)
if chat_card_entry is None:
# No streaming chat card was created for this query — common
# path for synthetic events (e.g. resumed workflow after a
# button click). Lazy-create one so the resumed output streams
# into a card just like a normal conversation, instead of
# being deferred and sent in one shot on is_final.
if not content:
return # nothing to stream yet
chat_card_entry = await self._lazy_create_resume_chat_card(message_source, message_id)
if chat_card_entry is None:
# Lazy-create failed (no template configured); fall back
# to a one-shot proactive message on the final chunk.
if is_final:
await self._send_proactive_to_event(message_source, content)
return
card_instance, card_instance_id = chat_card_entry
# btns is reserved exclusively for Dify form-action buttons.
# The template renders an Avatar header above the markdown
# content; no feedback buttons get injected here.
if content: if content:
if form_template_id: await self.bot.send_card_message(card_instance, card_instance_id, content, is_final)
# The card content has already been written via if is_final and bot_message.tool_calls is None:
# update_card_data (in _paint_form_on_card and the # self.seq = 1 # 消息回复结束之后重置seq
# initial card creation). The streaming endpoint self.card_instance_id_dict.pop(message_id) # 消息回复结束之后删除卡片实例id
# (PUT /v1.0/card/streaming) does not propagate
# updates on cards whose content was last set via
# update_card_data — they take different code paths
# on the DingTalk client. Stick with update_card_data
# for the whole turn for consistency.
try:
await self.bot.update_card_data(
out_track_id=card_instance_id,
card_param_map=self._card_params(
content=content,
btns='[]',
flowStatus='3' if is_final else '1',
),
)
except Exception:
if self.ap is not None:
self.ap.logger.exception('DingTalk: update card content failed')
else:
await self.bot.send_card_message(card_instance, card_instance_id, content, is_final)
if is_final:
if form_template_id and not content:
# Empty final chunk still needs to leave the card with
# flowStatus=3 so the spinner stops.
try:
await self.bot.update_card_data(
out_track_id=card_instance_id,
card_param_map=self._card_params(flowStatus='3'),
)
except Exception:
pass
if bot_message.tool_calls is None:
self.card_instance_id_dict.pop(message_id, None)
async def send_message(self, target_type: str, target_id: str, message: platform_message.MessageChain): async def send_message(self, target_type: str, target_id: str, message: platform_message.MessageChain):
markdown_enabled = self.config.get('markdown_card', False) markdown_enabled = self.config.get('markdown_card', False)
@@ -365,80 +260,16 @@ class DingTalkAdapter(abstract_platform_adapter.AbstractMessagePlatformAdapter):
return is_stream return is_stream
async def create_message_card(self, message_id, event): async def create_message_card(self, message_id, event):
form_template_id = (self.config.get('human_input_card_template_id') or '').strip() card_template_id = self.config['card_template_id']
legacy_template_id = self.config.get('card_template_id', '')
# Synthetic events (button clicks): look up the card already in
# active_turn_card so reply_message_chunk can stream to it.
if event is None or event.source_platform_object is None:
if form_template_id:
session_key = self._session_key_from_event(event) if event is not None else ''
carry = self.active_turn_card.get(session_key, '') if session_key else ''
if carry:
self.card_instance_id_dict[message_id] = (None, carry)
return True
return False
if form_template_id:
# Create one card with the form template, empty buttons,
# pending state. Streaming writes content to it; form pause
# paints buttons onto it. One card per turn, no duplication.
incoming_message = event.source_platform_object.incoming_message
out_track_id = uuid.uuid4().hex
is_group = str(incoming_message.conversation_type) == '2'
if is_group:
open_space_id = f'dtv1.card//IM_GROUP.{incoming_message.conversation_id}'
else:
open_space_id = f'dtv1.card//IM_ROBOT.{incoming_message.sender_staff_id}'
try:
await self.bot.create_and_deliver_card(
card_template_id=form_template_id,
out_track_id=out_track_id,
open_space_id=open_space_id,
is_group=is_group,
card_param_map=self._card_params(content='', btns='[]', flowStatus='1'),
callback_type='STREAM',
)
except Exception:
if self.ap is not None:
self.ap.logger.exception('DingTalk: create form-template card failed')
return False
self.card_instance_id_dict[message_id] = (None, out_track_id)
session_key = self._session_key_from_event(event)
if session_key:
self.active_turn_card[session_key] = out_track_id
self.active_turn_text[session_key] = ''
return True
# Legacy chat-card path (no form template).
incoming_message = event.source_platform_object.incoming_message incoming_message = event.source_platform_object.incoming_message
card_auto_layout = self.config.get('card_auto_layout', False) # message_id = incoming_message.message_id
card_auto_layout = self.config.get('card_ auto_layout', False)
card_instance, card_instance_id = await self.bot.create_and_card( card_instance, card_instance_id = await self.bot.create_and_card(
legacy_template_id, incoming_message, card_auto_layout=card_auto_layout card_template_id, incoming_message, card_auto_layout=card_auto_layout
) )
self.card_instance_id_dict[message_id] = (card_instance, card_instance_id) self.card_instance_id_dict[message_id] = (card_instance, card_instance_id)
return True return True
def _session_key_from_event(self, event) -> str:
"""Return launcher_type_launcher_id for an event, '' if unrecoverable."""
if event is None:
return ''
spo = event.source_platform_object
if spo is None:
try:
if isinstance(event, platform_events.GroupMessage):
return f'group_{event.group.id}'
return f'person_{event.sender.id}'
except Exception:
return ''
try:
inc = spo.incoming_message
if str(inc.conversation_type) == '2':
return f'group_{inc.conversation_id}'
return f'person_{inc.sender_staff_id}'
except Exception:
return ''
def register_listener( def register_listener(
self, self,
event_type: typing.Type[platform_events.Event], event_type: typing.Type[platform_events.Event],
@@ -461,36 +292,8 @@ class DingTalkAdapter(abstract_platform_adapter.AbstractMessagePlatformAdapter):
self.bot.on_message('GroupMessage')(on_message) self.bot.on_message('GroupMessage')(on_message)
async def run_async(self): async def run_async(self):
# Upload the bundled LangBot logo so the card Avatar can render
# via DingTalk's media CDN — external URLs (e.g. raw.githubusercontent)
# are blocked by DingTalk's Avatar.imageUrl resolver. Non-fatal if
# the upload fails: cards still render without an avatar image.
if self._LOGO_PATH.exists():
media_id = await self.bot.upload_image_media(str(self._LOGO_PATH))
if media_id:
self.bot_avatar_media_id = media_id
if self.ap is not None:
self.ap.logger.info(f'DingTalk bot avatar uploaded: media_id={media_id}')
else:
if self.ap is not None:
self.ap.logger.warning('DingTalk bot avatar upload failed; card will use default')
else:
if self.ap is not None:
self.ap.logger.warning(f'DingTalk bot avatar source not found: {self._LOGO_PATH}')
await self.bot.start() await self.bot.start()
def _card_params(self, **extra) -> dict:
"""Build a cardParamMap dict that always carries `bot_avatar`
(when uploaded) alongside whatever caller-specific params. The
bot_avatar key gets dropped on every update_card_data call
DingTalk wipes unspecified template variables, so re-sending it
on each update is mandatory."""
params = {}
if self.bot_avatar_media_id:
params['bot_avatar'] = self.bot_avatar_media_id
params.update(extra)
return params
async def kill(self) -> bool: async def kill(self) -> bool:
await self.bot.stop() await self.bot.stop()
return True return True
@@ -506,550 +309,3 @@ class DingTalkAdapter(abstract_platform_adapter.AbstractMessagePlatformAdapter):
], ],
): ):
return super().unregister_listener(event_type, callback) return super().unregister_listener(event_type, callback)
# ------------------------------------------------------------------
# Dify human-input form support
# ------------------------------------------------------------------
def set_bot_uuid(self, bot_uuid: str):
"""Receive the bot uuid from the platform manager.
Used to compose the public-facing unified-webhook URL for the card
dynamic-data-source pull endpoint.
"""
self.bot_uuid = bot_uuid
def _derive_open_space(self, message_source: platform_events.MessageEvent) -> tuple[str, bool]:
"""Return (openSpaceId, is_group) for the given inbound event."""
if isinstance(message_source, platform_events.GroupMessage):
return f'dtv1.card//IM_GROUP.{message_source.group.id}', True
return f'dtv1.card//IM_ROBOT.{message_source.sender.id}', False
def _derive_session_descriptor(
self, message_source: platform_events.MessageEvent
) -> tuple[provider_session.LauncherTypes, str, str]:
"""Return (launcher_type, launcher_id, sender_user_id) for routing."""
if isinstance(message_source, platform_events.GroupMessage):
return (
provider_session.LauncherTypes.GROUP,
str(message_source.group.id),
str(message_source.sender.id),
)
return (
provider_session.LauncherTypes.PERSON,
str(message_source.sender.id),
str(message_source.sender.id),
)
async def _handle_form_chunk(
self,
message_source: platform_events.MessageEvent,
bot_message,
message: platform_message.MessageChain,
form_data: dict,
) -> None:
"""Surface human-input prompt + buttons on the active card.
In single-card mode (form_template_id configured): update the
EXISTING card with form buttons so it transitions from streaming
output to prompt+buttons on the same card. In legacy mode:
finalize the chat card and deliver a separate form card.
"""
if self.ap is not None:
self.ap.logger.info(
f'DingTalk _handle_form_chunk: actions={len(form_data.get("actions") or [])}, '
f'node_title={form_data.get("node_title", "")!r}'
)
message_id = bot_message.resp_message_id
template_id = (self.config.get('human_input_card_template_id') or '').strip()
if template_id:
# Single-card mode: paint prompt + buttons onto the existing card.
session_key = self._session_key_from_event(message_source)
entry = self.card_instance_id_dict.get(message_id)
out_track_id = entry[1] if entry else None
if not out_track_id and session_key:
out_track_id = self.active_turn_card.get(session_key, '')
if out_track_id:
await self._paint_form_on_card(message_source, out_track_id, form_data, session_key)
self.card_instance_id_dict.pop(message_id, None)
return
# No existing card (e.g. Dify paused immediately with no LLM
# output before the pause). Create a form card directly.
await self._send_form_card(message_source, form_data, template_id)
self.card_instance_id_dict.pop(message_id, None)
return
# Legacy mode: finalize the streaming card with text fallback.
chat_card_entry = self.card_instance_id_dict.pop(message_id, None)
if chat_card_entry is not None:
_, chat_out_track_id = chat_card_entry
markdown_enabled = self.config.get('markdown_card', False)
text_content, _ = await DingTalkMessageConverter.yiri2target(message, markdown_enabled)
if not text_content and bot_message.content:
text_content = bot_message.content
try:
await self.bot.send_card_message(None, chat_out_track_id, text_content or '', True)
except Exception:
await self.logger.error(f'DingTalk: finalize chat card before form failed: {traceback.format_exc()}')
await self.send_message_text_form(message_source, form_data)
async def _paint_form_on_card(
self,
message_source: platform_events.MessageEvent,
out_track_id: str,
form_data: dict,
session_key: str,
) -> None:
"""Update an existing card's content + buttons for human-input."""
actions = list(form_data.get('actions') or [])
node_title = form_data.get('node_title', '') or 'Human Input Required'
form_content = form_data.get('form_content', '') or ''
# Record form state for the click-handler.
launcher_type, launcher_id, sender_user_id = self._derive_session_descriptor(message_source)
self.card_state[out_track_id] = {
'session_key': session_key,
'launcher_type': launcher_type.value,
'launcher_id': launcher_id,
'sender_user_id': sender_user_id,
'form_token': form_data.get('form_token', ''),
'workflow_run_id': form_data.get('workflow_run_id', ''),
'actions': actions,
'node_title': node_title,
'form_content': form_content,
}
btns = self._build_btns(actions, out_track_id)
parts: list[str] = []
prior = self.active_turn_text.get(session_key, '') if session_key else ''
if prior.strip():
parts.append(prior.rstrip())
parts.append('<hr>')
# DingTalk's card markdown widget strips `\n\n` paragraph breaks in
# template content slots, fusing inline siblings into a single line.
# Force visual line breaks with explicit HTML `<br>` tags so the
# title sits on its own line above form_content.
if node_title:
parts.append(f'**{node_title}**')
if form_content:
parts.append(form_content)
display_content = '<br><br>'.join(parts) or '请选择一个操作以继续。'
try:
await self.bot.update_card_data(
out_track_id=out_track_id,
card_param_map=self._card_params(
content=display_content,
btns=json.dumps(btns, ensure_ascii=False),
flowStatus='3',
),
)
except Exception:
if self.ap is not None:
self.ap.logger.exception('DingTalk: paint form on card failed')
await self.send_message_text_form(message_source, form_data)
return
if session_key:
self.active_turn_text[session_key] = display_content
@staticmethod
def _build_btns(actions: list, out_track_id: str) -> list:
btns = []
for idx, action in enumerate(actions):
action_id = str(action.get('id') or '')
title = str(action.get('title') or action_id or f'选项 {idx + 1}')
style = (action.get('button_style') or '').lower()
if style == 'primary' or (style == '' and idx == 0):
color = 'blue'
elif style == 'danger':
color = 'red'
else:
color = 'gray'
btns.append(
{
'text': title,
'color': color,
'status': 'normal',
'event': {
'type': 'sendCardRequest',
'params': {
'actionId': action_id,
'params': {'action_id': action_id, 'out_track_id': out_track_id},
},
},
}
)
return btns
async def _send_form_card(
self,
message_source: platform_events.MessageEvent,
form_data: dict,
template_id: str,
) -> None:
"""Deliver a new card pre-loaded with the human-input prompt + buttons."""
out_track_id = uuid.uuid4().hex
open_space_id, is_group = self._derive_open_space(message_source)
launcher_type, launcher_id, sender_user_id = self._derive_session_descriptor(message_source)
session_key = f'{launcher_type.value}_{launcher_id}'
actions = list(form_data.get('actions') or [])
node_title = form_data.get('node_title', '') or 'Human Input Required'
form_content = form_data.get('form_content', '') or ''
self.card_state[out_track_id] = {
'session_key': session_key,
'launcher_type': launcher_type.value,
'launcher_id': launcher_id,
'sender_user_id': sender_user_id,
'form_token': form_data.get('form_token', ''),
'workflow_run_id': form_data.get('workflow_run_id', ''),
'actions': actions,
'node_title': node_title,
'form_content': form_content,
'open_space_id': open_space_id,
'is_group': is_group,
}
parts = []
if node_title:
parts.append(f'**{node_title}**')
if form_content:
parts.append(form_content)
display_content = '<br><br>'.join(parts) or '请选择一个操作以继续。'
btns = []
for idx, action in enumerate(actions):
action_id = str(action.get('id') or '')
title = str(action.get('title') or action_id or f'选项 {idx + 1}')
style = (action.get('button_style') or '').lower()
if style == 'primary' or (style == '' and idx == 0):
color = 'blue'
elif style == 'danger':
color = 'red'
else:
color = 'gray'
btns.append(
{
'text': title,
'color': color,
'status': 'normal',
'event': {
'type': 'sendCardRequest',
'params': {
'actionId': action_id,
'params': {'action_id': action_id, 'out_track_id': out_track_id},
},
},
}
)
try:
if self.ap is not None:
self.ap.logger.info(
f'DingTalk _send_form_card: out_track_id={out_track_id} template_id={template_id} '
f'open_space_id={open_space_id} is_group={is_group} btns={len(btns)}'
)
await self.bot.create_and_deliver_card(
card_template_id=template_id,
out_track_id=out_track_id,
open_space_id=open_space_id,
is_group=is_group,
card_param_map=self._card_params(
content=display_content,
btns=json.dumps(btns, ensure_ascii=False),
flowStatus='3',
),
callback_type='STREAM',
)
except Exception:
await self.logger.error(f'DingTalk: deliver form card failed: {traceback.format_exc()}')
await self.send_message_text_form(message_source, form_data)
self.card_state.pop(out_track_id, None)
async def _lazy_create_resume_chat_card(
self,
message_source: platform_events.MessageEvent,
message_id: str,
) -> typing.Optional[tuple]:
"""Create a new card for resumed-workflow streaming output.
Used after a button click triggers a synthetic event the form
card stays put with the "已选择" notice, and a fresh card is
spawned here for the LLM reply to stream into.
"""
form_template_id = (self.config.get('human_input_card_template_id') or '').strip()
legacy_template_id = (self.config.get('card_template_id') or '').strip()
template_id = form_template_id or legacy_template_id
if not template_id:
return None
out_track_id = uuid.uuid4().hex
open_space_id, is_group = self._derive_open_space(message_source)
if form_template_id:
card_param_map = self._card_params(content='', btns='[]', flowStatus='1')
card_data_config = None
else:
# Legacy chat-card template doesn't carry a `bot_avatar`
# variable, so don't decorate the param map here.
card_param_map = {'content': '', 'query': '...'}
card_data_config = {'autoLayout': self.config.get('card_auto_layout', False)}
try:
success = await self.bot.create_and_deliver_card(
card_template_id=template_id,
out_track_id=out_track_id,
open_space_id=open_space_id,
is_group=is_group,
card_param_map=card_param_map,
card_data_config=card_data_config,
callback_type='STREAM',
)
except Exception:
if self.ap is not None:
self.ap.logger.exception('DingTalk: lazy create resume chat card failed')
return None
if not success:
return None
entry = (None, out_track_id)
self.card_instance_id_dict[message_id] = entry
# Register as the active card so any further chunks on this turn
# (and a subsequent re-pause) land on the same new card.
session_key = self._session_key_from_event(message_source)
if session_key:
self.active_turn_card[session_key] = out_track_id
self.active_turn_text[session_key] = ''
return entry
async def send_message_text_form(
self,
message_source: platform_events.MessageEvent,
form_data: dict,
) -> None:
"""Fallback: send the human-input prompt as plain text."""
display_text = _format_human_input_text(
form_data.get('node_title', ''),
form_data.get('form_content', ''),
form_data.get('actions', []) or [],
)
await self._send_proactive_to_event(message_source, display_text)
async def _send_proactive_to_event(
self,
message_source: platform_events.MessageEvent,
content: str,
) -> None:
"""Send `content` as a proactive message to the conversation behind
`message_source`. Used when no inbound chatbot message exists to
anchor a card on (e.g. resumed flows triggered by card actions).
"""
if not content:
return
if self.ap is not None:
target = (
str(message_source.group.id)
if isinstance(message_source, platform_events.GroupMessage)
else str(message_source.sender.id)
)
self.ap.logger.info(
f'DingTalk _send_proactive_to_event: target={target} '
f'is_group={isinstance(message_source, platform_events.GroupMessage)} content_len={len(content)}'
)
try:
if isinstance(message_source, platform_events.GroupMessage):
await self.bot.send_proactive_message_to_group(str(message_source.group.id), content)
else:
await self.bot.send_proactive_message_to_one(str(message_source.sender.id), content)
except Exception:
if self.ap is not None:
self.ap.logger.exception('DingTalk: send proactive message failed')
await self.logger.error(f'DingTalk: send proactive message failed: {traceback.format_exc()}')
async def _on_card_action(self, payload: dict) -> None:
"""Translate a card button click into a synthetic query.
Reads the clicked button's ``actionId`` (the real Dify action id —
the ButtonGroup template sends it back via `event.params.actionId`),
recovers the action title from ``card_state``, and enqueues a
synthetic `_dify_form_action` query the same way Lark / Telegram do.
"""
if self.ap is not None:
self.ap.logger.info(
f'DingTalk _on_card_action received: out_track_id={payload.get("out_track_id")} '
f'payload_action_id={payload.get("action_id")!r} params={payload.get("params")!r}'
)
out_track_id = payload.get('out_track_id') or ''
params = payload.get('params') or {}
# ButtonGroup `sendCardRequest` events surface the click id at the
# callback top level as `actionId`; fall back to `params.action_id`
# (alternate template wiring) and `params.actionId`.
raw_action_id = (
(payload.get('action_id') or '').strip()
or (params.get('action_id') or '').strip()
or (params.get('actionId') or '').strip()
or (params.get('id') or '').strip()
)
if not raw_action_id:
await self.logger.warning(f'DingTalk: card action with no action_id, payload={payload}')
return
state = self.card_state.get(out_track_id)
if state is None:
await self.logger.warning(f'DingTalk: card action received for unknown out_track_id={out_track_id}')
return
actions = state.get('actions', []) or []
action_id = raw_action_id
action_title = raw_action_id
for action in actions:
if str(action.get('id', '')) == raw_action_id:
action_title = action.get('title') or raw_action_id
break
launcher_type = (
provider_session.LauncherTypes.GROUP
if state.get('launcher_type') == provider_session.LauncherTypes.GROUP.value
else provider_session.LauncherTypes.PERSON
)
launcher_id = state.get('launcher_id', '')
sender_user_id = state.get('sender_user_id') or payload.get('user_id') or launcher_id
form_action_data = {
'form_token': state.get('form_token', ''),
'workflow_run_id': state.get('workflow_run_id', ''),
'action_id': action_id,
'action_title': action_title,
'node_title': state.get('node_title', ''),
'user': f'{launcher_type.value}_{launcher_id}',
'inputs': {},
}
message_chain = platform_message.MessageChain([platform_message.Plain(text=f'[Form Action: {action_title}]')])
if launcher_type == provider_session.LauncherTypes.GROUP:
synthetic_event = platform_events.GroupMessage(
sender=platform_entities.GroupMember(
id=sender_user_id,
member_name='',
permission=platform_entities.Permission.Member,
group=platform_entities.Group(
id=launcher_id,
name='',
permission=platform_entities.Permission.Member,
),
special_title='',
),
message_chain=message_chain,
time=int(datetime.datetime.now().timestamp()),
source_platform_object=None,
)
else:
synthetic_event = platform_events.FriendMessage(
sender=platform_entities.Friend(
id=sender_user_id,
nickname='',
remark='',
),
message_chain=message_chain,
time=int(datetime.datetime.now().timestamp()),
source_platform_object=None,
)
bot_uuid = ''
pipeline_uuid = None
if self.ap is not None:
for bot in self.ap.platform_mgr.bots:
if bot.adapter is self:
bot_uuid = bot.bot_entity.uuid
pipeline_uuid = bot.bot_entity.use_pipeline_uuid
break
try:
self.ap.logger.info(
f'DingTalk _on_card_action enqueuing form action: action_id={action_id!r} '
f'action_title={action_title!r} launcher_type={launcher_type.value} '
f'launcher_id={launcher_id} bot_uuid={bot_uuid} pipeline_uuid={pipeline_uuid}'
)
await self.ap.query_pool.add_query(
bot_uuid=bot_uuid,
launcher_type=launcher_type,
launcher_id=launcher_id,
sender_id=sender_user_id,
message_event=synthetic_event,
message_chain=message_chain,
adapter=self,
pipeline_uuid=pipeline_uuid,
variables={
'_dify_form_action': form_action_data,
'_routed_by_rule': True,
},
)
self.ap.logger.info('DingTalk _on_card_action: query enqueued OK')
except Exception:
self.ap.logger.exception('DingTalk: enqueue form action query failed')
return
# Visual feedback on the form card itself: keep the prompt visible,
# add a "已选择" line, remove the buttons. The resumed-workflow
# output lives on a separate new card (lazy-created in
# reply_message_chunk on the synthetic event), so the form card
# stays put as a record of the user's selection.
asyncio.create_task(
self._mark_card_resolved(
out_track_id,
action_title,
node_title=state.get('node_title', ''),
form_content=state.get('form_content', ''),
)
)
# Crucial: do NOT leave the form card's out_track_id in
# active_turn_card — otherwise create_message_card for the
# synthetic event would reuse it for the resume output, painting
# the LLM reply on top of the "已选择" notice. Clear it so the
# resume goes through the lazy-create path and spawns a fresh card.
session_key = state.get('session_key', '')
if session_key and self.active_turn_card.get(session_key) == out_track_id:
self.active_turn_card.pop(session_key, None)
self.active_turn_text.pop(session_key, None)
# Once consumed, drop the state — the runner clears _PENDING_FORMS too.
self.card_state.pop(out_track_id, None)
async def _mark_card_resolved(
self,
out_track_id: str,
action_title: str,
*,
node_title: str = '',
form_content: str = '',
) -> None:
"""Update the form card to acknowledge the user's selection.
Keeps the original prompt visible, adds a "已选择: X" notice, and
clears the buttons. The card stays as a permanent record of the
choice; the resumed workflow's output goes to a separate new card.
"""
parts: list[str] = []
if node_title:
parts.append(f'**{node_title}**')
if form_content:
parts.append(form_content)
parts.append(f'<hr>✅ 已选择:**{action_title}**')
content = '<br><br>'.join(parts)
if self.ap is not None:
self.ap.logger.info(f'DingTalk _mark_card_resolved: out_track_id={out_track_id} action={action_title!r}')
try:
await self.bot.update_card_data(
out_track_id=out_track_id,
card_param_map=self._card_params(
content=content,
btns='[]',
flowStatus='3',
),
)
except Exception:
if self.ap is not None:
self.ap.logger.exception('DingTalk: mark card resolved failed')
@@ -103,41 +103,6 @@ spec:
type: string type: string
required: true required: true
default: "填写你的卡片template_id" default: "填写你的卡片template_id"
- name: human_input_card_template_download
label:
en_US: Download Human Input Card Template
zh_Hans: 下载人工输入卡片模板
zh_Hant: 下載人工輸入卡片範本
description:
en_US: "Used as the only card template ID for the whole conversation turn. Download the built-in template, then import the JSON in DingTalk Open Platform > Card Platform / Card Template Management. After DingTalk creates the template, copy its template ID into the field below. The template already wires `content` (MarkdownBlock) and `btns` (ButtonGroup). Leave empty to fall back to the legacy two-card behavior."
zh_Hans: "用作整个对话回合唯一卡片的模板 ID。先下载内置模板,再到钉钉开放平台 > 卡片平台 / 卡片模板管理中导入该 JSON;钉钉生成模板后,将模板 ID 填到这里。模板已预先连好 `content` (MarkdownBlock) 与 `btns` (ButtonGroup)。留空则降级为旧的双卡行为。"
zh_Hant: "用作整個對話回合唯一卡片的範本 ID。先下載內建範本,再到釘釘開放平台 > 卡片平台 / 卡片範本管理中匯入該 JSON;釘釘產生範本後,將範本 ID 填到這裡。範本已預先連好 `content` (MarkdownBlock) 與 `btns` (ButtonGroup)。留空則降級為舊的雙卡行為。"
type: download-link
required: false
default: ""
url: /api/v1/platform/adapters/dingtalk/human-input-card-template
download_filename: dingtalk_human_input_card.json
help_links:
zh: https://open-dev.dingtalk.com/fe/card
en: https://open-dev.dingtalk.com/fe/card
ja: https://open-dev.dingtalk.com/fe/card
help_label:
en_US: Import Guide
zh_Hans: 导入指引
zh_Hant: 匯入指引
ja_JP: インポート手順
- name: human_input_card_template_id
label:
en_US: Human Input Card Template ID
zh_Hans: 人工输入卡片模板ID
zh_Hant: 人工輸入卡片範本ID
description:
en_US: "Paste the template ID generated after importing the human input card template."
zh_Hans: "填写导入人工输入卡片模板后生成的模板 ID。"
zh_Hant: "填寫匯入人工輸入卡片範本後產生的範本 ID。"
type: string
required: false
default: ""
execution: execution:
python: python:
path: ./dingtalk.py path: ./dingtalk.py
+5 -509
View File
@@ -1,7 +1,6 @@
from __future__ import annotations from __future__ import annotations
import discord import discord
from discord import ui as discord_ui
import typing import typing
import re import re
@@ -9,8 +8,6 @@ import base64
import uuid import uuid
import os import os
import datetime import datetime
import time
import traceback
# 使用BytesIO创建文件对象,避免路径问题 # 使用BytesIO创建文件对象,避免路径问题
import io import io
@@ -827,69 +824,6 @@ class DiscordEventConverter(abstract_platform_adapter.AbstractEventConverter):
) )
class DiscordFormView(discord_ui.View):
"""Discord ``ui.View`` that renders one button per Dify form action.
Each button's click triggers ``adapter._on_form_button_click`` which
acks the interaction, locks the buttons in place, and enqueues a
synthetic ``_dify_form_action`` query so the runner resumes the
workflow.
"""
# Discord button style mapping for Dify ``button_style`` values.
_STYLE_MAP: typing.ClassVar[dict] = {
'primary': discord.ButtonStyle.primary,
'danger': discord.ButtonStyle.danger,
'warning': discord.ButtonStyle.danger,
'success': discord.ButtonStyle.success,
'default': discord.ButtonStyle.secondary,
'': discord.ButtonStyle.secondary,
}
def __init__(
self,
adapter: 'DiscordAdapter',
session_key: str,
actions: list,
timeout: float = 1800,
):
super().__init__(timeout=timeout)
self._adapter = adapter
self._session_key = session_key
# Discord caps a view at 25 children (5 rows × 5 buttons). Trim
# silently — most Dify forms have ≤10 actions in practice.
for idx, action in enumerate(actions[:25]):
action_id = str(action.get('id') or '')
label = str(action.get('title') or action_id or f'Option {idx + 1}')
style = self._STYLE_MAP.get(
str(action.get('button_style') or '').lower(),
discord.ButtonStyle.secondary,
)
# custom_id must be unique within the view and ≤100 chars.
# Encode (session, idx) so we can recover the action even
# if Dify ids contain unsafe characters.
custom_id = f'lb_form:{idx}:{action_id[:80]}'[:100]
button = discord_ui.Button(
label=label[:80], # Discord label limit
style=style,
custom_id=custom_id,
)
button.callback = self._make_callback(action_id, label)
self.add_item(button)
def _make_callback(self, action_id: str, action_title: str):
async def _cb(interaction: discord.Interaction):
await self._adapter._on_form_button_click(
interaction=interaction,
session_key=self._session_key,
action_id=action_id,
action_title=action_title,
view=self,
)
return _cb
class DiscordAdapter(abstract_platform_adapter.AbstractMessagePlatformAdapter): class DiscordAdapter(abstract_platform_adapter.AbstractMessagePlatformAdapter):
bot: discord.Client = pydantic.Field(exclude=True) bot: discord.Client = pydantic.Field(exclude=True)
@@ -903,10 +837,6 @@ class DiscordAdapter(abstract_platform_adapter.AbstractMessagePlatformAdapter):
voice_manager: VoiceConnectionManager | None = pydantic.Field(exclude=True, default=None) voice_manager: VoiceConnectionManager | None = pydantic.Field(exclude=True, default=None)
# Injected by botmgr at construction so the form-button callback can
# enqueue a synthetic resume query (`_dify_form_action`) on the pool.
ap: typing.Any = pydantic.Field(exclude=True, default=None)
def __init__(self, config: dict, logger: abstract_platform_logger.AbstractEventLogger, **kwargs): def __init__(self, config: dict, logger: abstract_platform_logger.AbstractEventLogger, **kwargs):
bot_account_id = config['client_id'] bot_account_id = config['client_id']
@@ -930,18 +860,8 @@ class DiscordAdapter(abstract_platform_adapter.AbstractMessagePlatformAdapter):
args = {} args = {}
# Proxy: config > env var > auto-detect. if os.getenv('http_proxy'):
# discord.py uses aiohttp which does NOT respect http_proxy env args['proxy'] = os.getenv('http_proxy')
# vars by default — we must pass proxy= explicitly.
proxy = (
config.get('proxy')
or os.getenv('http_proxy')
or os.getenv('HTTP_PROXY')
or os.getenv('https_proxy')
or os.getenv('HTTPS_PROXY')
)
if proxy:
args['proxy'] = proxy
bot = MyClient(intents=intents, **args) bot = MyClient(intents=intents, **args)
@@ -955,19 +875,6 @@ class DiscordAdapter(abstract_platform_adapter.AbstractMessagePlatformAdapter):
**kwargs, **kwargs,
) )
# Per-resp-message-id buffer for the accumulated text yielded by
# the runner. Discord's edit-message ratelimit (5/5s) makes true
# progressive streaming impractical, so we collect chunks and
# render once on is_final. ``_form_data`` on the final chunk
# diverts to the button-view path.
self._stream_buffer: dict[str, str] = {}
# session_key -> {form_data, channel_id, thread_id, sender_id,
# posted_at, view_message_id}
# Populated when we send a form view; consumed when the user
# clicks a button so we know which workflow_run / form_token to
# resume.
self._pending_forms: dict[str, dict] = {}
# Voice functionality methods # Voice functionality methods
async def join_voice_channel(self, guild_id: int, channel_id: int, user_id: int = None) -> discord.VoiceClient: async def join_voice_channel(self, guild_id: int, channel_id: int, user_id: int = None) -> discord.VoiceClient:
""" """
@@ -1161,12 +1068,7 @@ class DiscordAdapter(abstract_platform_adapter.AbstractMessagePlatformAdapter):
): ):
msg_to_send, files = await self.message_converter.yiri2target(message) msg_to_send, files = await self.message_converter.yiri2target(message)
# Synthetic events (button-click resume) have no inbound discord assert isinstance(message_source.source_platform_object, discord.Message)
# Message. Route via the channel we cached when the user clicked.
source = message_source.source_platform_object
if not isinstance(source, discord.Message):
await self._reply_synthetic(message_source, msg_to_send, files)
return
args = { args = {
'content': msg_to_send, 'content': msg_to_send,
@@ -1176,7 +1078,7 @@ class DiscordAdapter(abstract_platform_adapter.AbstractMessagePlatformAdapter):
args['files'] = files args['files'] = files
if quote_origin: if quote_origin:
args['reference'] = source args['reference'] = message_source.source_platform_object
has_at = False has_at = False
@@ -1188,413 +1090,7 @@ class DiscordAdapter(abstract_platform_adapter.AbstractMessagePlatformAdapter):
if has_at: if has_at:
args['mention_author'] = True args['mention_author'] = True
await source.channel.send(**args) await message_source.source_platform_object.channel.send(**args)
async def _reply_synthetic(
self,
message_source: platform_events.MessageEvent,
msg_to_send: str,
files: list,
) -> None:
"""Deliver a reply for a button-click-resumed (synthetic) event.
We don't have an inbound discord.Message to anchor to; instead
look up the channel cached in ``_pending_forms[session_key +
'__last_channel']`` from the most recent button click.
"""
if isinstance(message_source, platform_events.GroupMessage):
# _handle_form_chunk uses channel_id alone as the session
# scope, and launcher_id was set to channel_id when
# synthesizing the event.
session_key = f'c:{message_source.group.id}'
else:
session_key = f'p:{message_source.sender.id}'
cached = self._pending_forms.get(session_key + '__last_channel') or {}
channel = cached.get('channel')
if channel is None:
if self.ap is not None:
self.ap.logger.warning(
f'Discord: synthetic reply has no cached channel for '
f'{session_key}; dropping content (len={len(msg_to_send)})'
)
return
args: dict[str, typing.Any] = {'content': msg_to_send}
if files:
args['files'] = files
try:
await channel.send(**args)
except Exception:
if self.ap is not None:
self.ap.logger.error(f'Discord: synthetic reply send failed: {traceback.format_exc()}')
# Discord allows 5 edits per 5 seconds per message. We throttle
# to one edit per 8 runner-chunks (runner already yields every 8
# text_chunks internally), which stays comfortably within limits.
_STREAM_EDIT_INTERVAL = 8
async def is_stream_output_supported(self) -> bool:
return True
async def create_message_card(self, message_id: str, event: platform_events.MessageEvent) -> bool:
"""Set up a stream context for progressive editing.
The first non-empty reply_message_chunk will send the initial
message; subsequent chunks edit it in place.
"""
source = event.source_platform_object
if not isinstance(source, discord.Message):
return False
self._stream_buffer[message_id] = {
'channel': source.channel,
'sent_message': None, # discord.Message set on first send
'last_content': '',
'chunk_count': 0,
}
return True
async def reply_message_chunk(
self,
message_source: platform_events.MessageEvent,
bot_message: typing.Any,
message: platform_message.MessageChain,
quote_origin: bool = False,
is_final: bool = False,
):
msg_id = (
bot_message.get('resp_message_id')
if isinstance(bot_message, dict)
else getattr(bot_message, 'resp_message_id', None)
)
text_parts = [m.text for m in message if isinstance(m, platform_message.Plain)]
chunk_text = '\n\n'.join(t for t in text_parts if t)
form_data = getattr(bot_message, '_form_data', None) if not isinstance(bot_message, dict) else None
ctx = self._stream_buffer.get(msg_id) if msg_id else None
# If the stream ctx was not set up (create_message_card wasn't
# called, e.g. synthetic event), or the final chunk carries a
# form, skip progressive editing entirely.
if ctx is None or form_data:
try:
if form_data and is_final:
await self._handle_form_chunk(message_source, form_data)
elif is_final and chunk_text:
await self.reply_message(
message_source,
platform_message.MessageChain([platform_message.Plain(text=chunk_text)]),
quote_origin,
)
finally:
self._stream_buffer.pop(msg_id, None)
return
# Progressive streaming path: send first chunk, edit subsequent.
ctx['chunk_count'] += 1
# Runner yields the full accumulated text on each chunk, so we
# always replace (not append).
if chunk_text:
ctx['last_content'] = chunk_text
sent = ctx['sent_message']
if sent is None:
# First non-empty chunk — send the initial message.
if not ctx['last_content']:
return # No content yet, wait for next chunk.
try:
sent = await ctx['channel'].send(ctx['last_content'])
ctx['sent_message'] = sent
except Exception:
if self.ap is not None:
self.ap.logger.error(f'Discord stream send failed: {traceback.format_exc()}')
self._stream_buffer.pop(msg_id, None)
return
if is_final:
# Final chunk — edit to the full content, then clean up.
if ctx['last_content'] and ctx['last_content'] != sent.content:
try:
await sent.edit(content=ctx['last_content'][:2000])
except Exception:
pass # Best-effort
self._stream_buffer.pop(msg_id, None)
elif (ctx['chunk_count'] % self._STREAM_EDIT_INTERVAL) == 0:
# Intermediate edit — throttle to avoid rate limits.
if ctx['last_content'] and ctx['last_content'] != sent.content:
try:
await sent.edit(content=ctx['last_content'][:2000])
except Exception:
pass # Rate-limited or deleted — ignore.
async def _handle_form_chunk(
self,
message_source: platform_events.MessageEvent,
form_data: dict,
) -> None:
"""Render a Dify form pause as a Discord embed + button View.
Mirrors the QQ / Telegram / Lark form path: the button's click
callback synthesizes a ``_dify_form_action`` query so the runner's
``_merge_pending_form_action`` resumes the workflow.
"""
source = message_source.source_platform_object
actions = form_data.get('actions') or []
if not actions:
# Nothing clickable — fall back to plain text.
if source is not None:
await self.reply_message(
message_source,
platform_message.MessageChain(
[platform_message.Plain(text=str(form_data.get('node_title') or ''))]
),
)
return
node_title = str(form_data.get('node_title') or 'Confirmation needed')
form_content = str(form_data.get('form_content') or '').strip()
# Two paths:
# (a) Real message — extract channel from source.
# (b) Synthetic event (button-click resume) — no
# source_platform_object; recover the channel we cached
# when the user clicked.
if isinstance(source, discord.Message):
channel = source.channel
guild_id = str(source.guild.id) if source.guild else ''
sender_id = str(source.author.id)
channel_id = str(source.channel.id)
session_key = f'c:{channel_id}' if guild_id else f'p:{sender_id}'
else:
# Synthetic event — resolve session_key from event shape,
# then look up the cached channel from the click.
if isinstance(message_source, platform_events.GroupMessage):
# launcher_id was set to channel_id when we synthesized.
channel_id = str(message_source.group.id)
session_key = f'c:{channel_id}'
else:
session_key = f'p:{message_source.sender.id}'
channel_id = ''
cached = self._pending_forms.get(session_key + '__last_channel')
channel = cached.get('channel') if cached else None
guild_id = (cached or {}).get('guild_id', '')
sender_id = str(message_source.sender.id) if message_source.sender else ''
if channel is None:
if self.ap is not None:
self.ap.logger.warning(
f'Discord: synthetic form chunk has no cached channel for '
f'{session_key}; cannot render form buttons'
)
return
body_parts: list[str] = []
if form_content:
body_parts.append(form_content)
body_parts.append('Please select an option below:')
embed_body = '\n\n'.join(body_parts)
# Discord embed.description has a 4096 char limit — defensive trim.
if len(embed_body) > 4000:
embed_body = embed_body[:3990] + '\n\n…(truncated)'
embed = discord.Embed(
title=node_title[:256],
description=embed_body,
color=discord.Color.blurple(),
)
view = DiscordFormView(
adapter=self,
session_key=session_key,
actions=actions,
timeout=1800, # 30 min — matches Dify form_token TTL
)
try:
sent_msg = await channel.send(embed=embed, view=view)
except Exception:
if self.ap is not None:
self.ap.logger.error(f'Discord: form view send failed: {traceback.format_exc()}')
return
self._pending_forms[session_key] = {
'form_data': form_data,
'channel_id': channel_id,
'guild_id': guild_id,
'sender_id': sender_id,
'view_message_id': str(sent_msg.id),
'posted_at': time.time(),
}
if self.ap is not None:
self.ap.logger.info(f'Discord: form view posted session={session_key} actions={len(actions)}')
async def _on_form_button_click(
self,
interaction: discord.Interaction,
session_key: str,
action_id: str,
action_title: str,
view: DiscordFormView,
) -> None:
"""Handle a click on a form button — ack, resume the workflow,
and disable the View buttons so the choice is visually locked in."""
import langbot_plugin.api.entities.builtin.provider.session as provider_session
# ACK first (3-second deadline before Discord shows "interaction failed").
try:
await interaction.response.defer()
except discord.HTTPException:
# Already responded somehow — proceed regardless.
pass
pending = self._pending_forms.pop(session_key, None)
if not pending:
if self.ap is not None:
self.ap.logger.warning(
f'Discord: button click on stale session {session_key}; ignoring (action_id={action_id!r})'
)
await self._lock_view_message(interaction, view, action_title, stale=True)
return
# Lock the buttons in place: disable everything, mark chosen one.
await self._lock_view_message(interaction, view, action_title)
form_data: dict = pending.get('form_data') or {}
guild_id = pending.get('guild_id', '')
channel_id = pending.get('channel_id', '')
sender_id = pending.get('sender_id', '')
# In group context the launcher is the CHANNEL (not the user who
# clicked) — matches how the original message was routed through
# the pipeline. Using the clicker's user id would mismatch the
# Dify session and produce "Workflow run not found".
if guild_id:
launcher_type = provider_session.LauncherTypes.GROUP
launcher_id = channel_id
else:
launcher_type = provider_session.LauncherTypes.PERSON
launcher_id = sender_id or str(interaction.user.id)
form_action_data = {
'form_token': form_data.get('form_token', ''),
'workflow_run_id': form_data.get('workflow_run_id', ''),
'action_id': action_id,
'action_title': action_title,
'node_title': form_data.get('node_title', ''),
'user': f'{launcher_type.value}_{launcher_id}',
'inputs': {},
}
message_chain = platform_message.MessageChain([platform_message.Plain(text=f'[Form Action: {action_title}]')])
# Synthesize a platform event so the pipeline can run the resume
# query. source_platform_object=None signals "no inbound discord
# message" — reply_message must tolerate this (it falls through
# to channel.send via the cached interaction.channel below).
if launcher_type == provider_session.LauncherTypes.GROUP:
synthetic_event: platform_events.MessageEvent = platform_events.GroupMessage(
sender=platform_entities.GroupMember(
id=sender_id,
member_name=interaction.user.display_name if interaction.user else '',
permission='MEMBER',
group=platform_entities.Group(
id=launcher_id,
name=channel_id,
permission=platform_entities.Permission.Member,
),
special_title='',
),
message_chain=message_chain,
time=int(time.time()),
source_platform_object=None,
)
else:
synthetic_event = platform_events.FriendMessage(
sender=platform_entities.Friend(
id=sender_id,
nickname=interaction.user.display_name if interaction.user else '',
remark='',
),
message_chain=message_chain,
time=int(time.time()),
source_platform_object=None,
)
if self.ap is None:
if self.logger:
await self.logger.error('Discord: ap not injected; cannot enqueue button-click query')
return
bot_uuid = ''
pipeline_uuid = None
for bot in self.ap.platform_mgr.bots:
if bot.adapter is self:
bot_uuid = bot.bot_entity.uuid
pipeline_uuid = bot.bot_entity.use_pipeline_uuid
break
# Remember the channel so _reply_synthetic and _handle_form_chunk
# (synthetic-event path) can find a target. guild_id is needed
# to reconstruct the launcher_type on subsequent form pauses.
self._pending_forms[session_key + '__last_channel'] = {
'channel': interaction.channel,
'guild_id': guild_id,
'posted_at': time.time(),
}
try:
await self.ap.query_pool.add_query(
bot_uuid=bot_uuid,
launcher_type=launcher_type,
launcher_id=launcher_id,
sender_id=sender_id,
message_event=synthetic_event,
message_chain=message_chain,
adapter=self,
pipeline_uuid=pipeline_uuid,
variables={
'_dify_form_action': form_action_data,
'_routed_by_rule': True,
},
)
if self.ap is not None:
self.ap.logger.info(
f'Discord: button-click query enqueued action_id={action_id!r} session={session_key}'
)
except Exception:
if self.ap is not None:
self.ap.logger.error(f'Discord: enqueue button-click query failed: {traceback.format_exc()}')
async def _lock_view_message(
self,
interaction: discord.Interaction,
view: DiscordFormView,
chosen_title: str,
stale: bool = False,
) -> None:
"""Disable all buttons on the form view and annotate the chosen
one mirrors DingTalk/Lark's in-card 已选择 feedback."""
try:
for child in view.children:
if not isinstance(child, discord_ui.Button):
continue
child.disabled = True
if not stale and child.label == chosen_title:
child.style = discord.ButtonStyle.success
if not (child.label or '').startswith(''):
child.label = f'{child.label}'
view.stop()
if interaction.message is not None:
await interaction.message.edit(view=view)
except Exception:
if self.ap is not None:
self.ap.logger.warning(f'Discord: lock-view-message failed (non-fatal): {traceback.format_exc()}')
async def is_muted(self, group_id: int) -> bool: async def is_muted(self, group_id: int) -> bool:
return False return False
File diff suppressed because it is too large Load Diff
+55 -4
View File
@@ -23,6 +23,57 @@ spec:
en: https://link.langbot.app/en/platforms/lark en: https://link.langbot.app/en/platforms/lark
ja: https://link.langbot.app/ja/platforms/lark ja: https://link.langbot.app/ja/platforms/lark
config: config:
- name: domain
label:
en_US: Platform Domain
zh_Hans: 平台域名
zh_Hant: 平台域名
ja_JP: プラットフォームドメイン
description:
en_US: Select the open platform domain. Use Feishu for Chinese mainland, Lark for international
zh_Hans: 选择开放平台域名,国内使用飞书,海外使用 Lark
zh_Hant: 選擇開放平台域名,國內使用飛書,海外使用 Lark
ja_JP: オープンプラットフォームのドメインを選択。中国国内は飛書、海外は Lark を使用
type: select
options:
- name: https://open.feishu.cn
label:
en_US: Feishu (open.feishu.cn)
zh_Hans: 飞书 (open.feishu.cn)
zh_Hant: 飛書 (open.feishu.cn)
ja_JP: 飛書 (open.feishu.cn)
- name: https://open.larksuite.com
label:
en_US: Lark (open.larksuite.com)
zh_Hans: Lark (open.larksuite.com)
zh_Hant: Lark (open.larksuite.com)
ja_JP: Lark (open.larksuite.com)
- name: custom
label:
en_US: Custom
zh_Hans: 自定义
zh_Hant: 自定義
ja_JP: カスタム
required: false
default: https://open.feishu.cn
- name: custom_domain
label:
en_US: Custom Domain
zh_Hans: 自定义域名
zh_Hant: 自定義域名
ja_JP: カスタムドメイン
description:
en_US: "Enter the full domain URL, e.g. https://open.example.com"
zh_Hans: "输入完整的域名 URL,例如 https://open.example.com"
zh_Hant: "輸入完整的域名 URL,例如 https://open.example.com"
ja_JP: "完全なドメイン URL を入力(例: https://open.example.com"
type: string
required: false
default: ""
show_if:
field: domain
operator: eq
value: custom
- name: one-click-create - name: one-click-create
label: label:
en_US: One-Click Create App en_US: One-Click Create App
@@ -140,10 +191,10 @@ spec:
zh_Hant: 應用類型 zh_Hant: 應用類型
ja_JP: アプリタイプ ja_JP: アプリタイプ
description: description:
en_US: Default to self-built application, refer to https://open.feishu.cn/document/platform-overveiw/overview en_US: "Default to self-built application, refer to https://open.feishu.cn/document/platform-overveiw/overview"
zh_Hans: 默认为企业自建应用,参考 https://open.feishu.cn/document/platform-overveiw/overview zh_Hans: "默认为企业自建应用,参考 https://open.feishu.cn/document/platform-overveiw/overview"
zh_Hant: 預設為企業自建應用,參考 https://open.feishu.cn/document/platform-overveiw/overview zh_Hant: "預設為企業自建應用,參考 https://open.feishu.cn/document/platform-overveiw/overview"
ja_JP: デフォルトはカスタムアプリです。詳細は https://open.feishu.cn/document/platform-overveiw/overview を参照してください ja_JP: "デフォルトはカスタムアプリです。詳細は https://open.feishu.cn/document/platform-overveiw/overview を参照してください"
type: select type: select
options: options:
- name: self - name: self
+1 -546
View File
@@ -11,7 +11,7 @@ import langbot_plugin.api.definition.abstract.platform.adapter as abstract_platf
import langbot_plugin.api.entities.builtin.platform.message as platform_message import langbot_plugin.api.entities.builtin.platform.message as platform_message
import langbot_plugin.api.entities.builtin.platform.events as platform_events import langbot_plugin.api.entities.builtin.platform.events as platform_events
import langbot_plugin.api.entities.builtin.platform.entities as platform_entities import langbot_plugin.api.entities.builtin.platform.entities as platform_entities
from langbot.libs.qq_official_api.api import QQOfficialClient, build_keyboard_from_form from langbot.libs.qq_official_api.api import QQOfficialClient
from langbot.libs.qq_official_api.qqofficialevent import QQOfficialEvent from langbot.libs.qq_official_api.qqofficialevent import QQOfficialEvent
from ...utils import image from ...utils import image
from ..logger import EventLogger from ..logger import EventLogger
@@ -191,7 +191,6 @@ class QQOfficialAdapter(abstract_platform_adapter.AbstractMessagePlatformAdapter
enable_webhook: bool = False enable_webhook: bool = False
message_converter: QQOfficialMessageConverter = QQOfficialMessageConverter() message_converter: QQOfficialMessageConverter = QQOfficialMessageConverter()
event_converter: QQOfficialEventConverter = QQOfficialEventConverter() event_converter: QQOfficialEventConverter = QQOfficialEventConverter()
ap: typing.Any = None
def __init__(self, config: dict, logger: EventLogger): def __init__(self, config: dict, logger: EventLogger):
enable_webhook = config.get('enable-webhook', False) enable_webhook = config.get('enable-webhook', False)
@@ -217,31 +216,6 @@ class QQOfficialAdapter(abstract_platform_adapter.AbstractMessagePlatformAdapter
self._stream_ctx_ts: dict[str, float] = {} self._stream_ctx_ts: dict[str, float] = {}
self._fallback_text: dict[str, str] = {} self._fallback_text: dict[str, str] = {}
self._fallback_text_ts: dict[str, float] = {} self._fallback_text_ts: dict[str, float] = {}
# Dify form-action bookkeeping for the human-input button flow.
# session_key = "<scene>_<id>" where scene is c2c/group/channel and
# id is user_openid / group_openid / channel_id.
# session_key -> {form_data, msg_id, event_id, scene, target_id,
# sender_id, posted_at}
# Set when we send a markdown+keyboard card and consulted when:
# (a) INTERACTION_CREATE fires — we look up the form by
# session_key (button's `data` carries the action_id),
# (b) the resumed-workflow query needs to find a passive-reply
# event_id (INTERACTION_CREATE id, 30-min validity).
self._pending_forms: dict[str, dict] = {}
# session_key -> most recent ``INTERACTION_CREATE`` event_id, used
# as the passive event_id for the resumed query's LLM output.
self._session_event_ids: dict[str, dict] = {}
# Per-anchor msg_seq counter. QQ accepts up to 5 passive replies
# per (msg_id|event_id) within 60 min, but each reuse needs a
# fresh ``msg_seq`` — re-sending with msg_seq=1 is silently dedup'd.
self._anchor_msg_seq: dict[str, int] = {}
# Wire button-click handler so webhook mode catches INTERACTION_CREATE.
# (ws mode is wired separately via on_event in _run_websocket so the
# raw payload bypasses get_message's message-only flattening.)
@self.bot.on_interaction()
async def _on_interaction(event_data: dict, interaction_id: typing.Optional[str]):
await self._handle_interaction_create(event_data, interaction_id)
async def reply_message( async def reply_message(
self, self,
@@ -253,13 +227,6 @@ class QQOfficialAdapter(abstract_platform_adapter.AbstractMessagePlatformAdapter
message_source, message_source,
) )
# Synthetic event (button-click resume): no inbound platform
# object → no msg_id. Route via the cached INTERACTION_CREATE
# event_id (valid 30 min, no quota cost).
if qq_official_event is None:
await self._reply_synthetic(message_source, message)
return
content_list = await QQOfficialMessageConverter.yiri2target(message) content_list = await QQOfficialMessageConverter.yiri2target(message)
# 确定 target_type 和 target_id # 确定 target_type 和 target_id
@@ -409,9 +376,6 @@ class QQOfficialAdapter(abstract_platform_adapter.AbstractMessagePlatformAdapter
await self.logger.info('QQ Official WebSocket connected and ready') await self.logger.info('QQ Official WebSocket connected and ready')
async def on_event(event_type: str, event_data: dict): async def on_event(event_type: str, event_data: dict):
# INTERACTION_CREATE is dispatched via bot.on_interaction()
# (registered in __init__) so we get the top-level ws_event_id
# — needed as the passive-reply event_id. It never reaches here.
# 只处理消息事件,忽略 READY/RESUMED 等系统事件 # 只处理消息事件,忽略 READY/RESUMED 等系统事件
message_event_types = { message_event_types = {
'C2C_MESSAGE_CREATE', 'C2C_MESSAGE_CREATE',
@@ -473,36 +437,12 @@ class QQOfficialAdapter(abstract_platform_adapter.AbstractMessagePlatformAdapter
async def is_stream_output_supported(self) -> bool: async def is_stream_output_supported(self) -> bool:
return self.config.get('enable-stream-reply', False) return self.config.get('enable-stream-reply', False)
@staticmethod
def _is_form_placeholder_chunk(text: str) -> bool:
"""Return True for invisible placeholder chunks used to carry forms."""
if not text:
return False
cleaned = text.replace('\u200b', '').replace('\u200c', '').replace('\u200d', '').replace('\ufeff', '').strip()
# Some Windows consoles/logs display the zero-width placeholder as
# mojibake. Treat those variants as the same non-user-facing marker.
return cleaned in {'', '鈥?', '​'}
async def create_message_card(self, message_id: str, event: platform_events.MessageEvent) -> bool: async def create_message_card(self, message_id: str, event: platform_events.MessageEvent) -> bool:
source = event.source_platform_object source = event.source_platform_object
# Synthetic events (button-click resume) have no source object —
# they ride a cached INTERACTION_CREATE event_id, not a streamable
# msg_id. Skip stream setup; reply_message handles the one-shot
# send at is_final.
if source is None:
return False
# Streaming API only supports C2C private chat # Streaming API only supports C2C private chat
if source.t != 'C2C_MESSAGE_CREATE': if source.t != 'C2C_MESSAGE_CREATE':
return False return False
# The stream endpoint still consumes msg_seq for this inbound msg_id.
# Keep the passive-reply counter in sync so a follow-up form card uses
# msg_seq=2 instead of being deduplicated by QQ as another seq=1 send.
if source.d_id:
self._anchor_msg_seq[source.d_id] = max(self._anchor_msg_seq.get(source.d_id, 0), 1)
ctx = { ctx = {
'user_openid': source.user_openid, 'user_openid': source.user_openid,
'msg_id': source.d_id, 'msg_id': source.d_id,
@@ -529,38 +469,12 @@ class QQOfficialAdapter(abstract_platform_adapter.AbstractMessagePlatformAdapter
): ):
# Periodically clean up stale stream contexts # Periodically clean up stale stream contexts
await self._cleanup_stale_streams() await self._cleanup_stale_streams()
# Dify human-input pause: when the runner attaches `_form_data` to
# the final chunk, finalize any in-flight stream session and send
# a markdown + keyboard message instead. Plain-text content from
# earlier chunks is already on the stream; we close it cleanly
# and the buttons land as a separate reply.
form_data = getattr(bot_message, '_form_data', None) if not isinstance(bot_message, dict) else None
if is_final:
_resume = getattr(bot_message, '_resume_from_form', None) if not isinstance(bot_message, dict) else None
_open_new = getattr(bot_message, '_open_new_card', None) if not isinstance(bot_message, dict) else None
if self.ap is not None:
self.ap.logger.info(
f'QQ Official reply_message_chunk final: '
f'type={type(bot_message).__name__} '
f'is_final={is_final} '
f'form_data_present={form_data is not None} '
f'resume_from_form={_resume} open_new_card={_open_new} '
f'content_len={len(getattr(bot_message, "content", "") or "")}'
)
if form_data and is_final:
await self._handle_form_chunk(message_source, message, form_data)
return
# 提取纯文本内容(当前 chunk 的文本) # 提取纯文本内容(当前 chunk 的文本)
text_parts = [] text_parts = []
for msg in message: for msg in message:
if type(msg) is platform_message.Plain: if type(msg) is platform_message.Plain:
text_parts.append(msg.text) text_parts.append(msg.text)
chunk_text = '\n\n'.join(text_parts) chunk_text = '\n\n'.join(text_parts)
if self._is_form_placeholder_chunk(chunk_text):
await self.logger.debug('QQ Official: skipped invisible form placeholder chunk')
return
message_id = ( message_id = (
bot_message.get('resp_message_id') bot_message.get('resp_message_id')
@@ -643,462 +557,3 @@ class QQOfficialAdapter(abstract_platform_adapter.AbstractMessagePlatformAdapter
], ],
): ):
return super().unregister_listener(event_type, callback) return super().unregister_listener(event_type, callback)
# ------------------------------------------------------------------
# Dify human-input button-interaction support
# ------------------------------------------------------------------
_PENDING_FORM_TTL = 1800 # 30 min — matches QQ passive-reply window.
_MAX_REPLIES_PER_ANCHOR = 5 # QQ hard limit per msg_id / event_id.
def _next_msg_seq(self, anchor: str) -> typing.Optional[int]:
"""Return the next msg_seq for an anchor, or ``None`` if the
anchor has already been used 5 times (further sends would be
silently dropped by QQ)."""
if not anchor:
return 1
used = self._anchor_msg_seq.get(anchor, 0)
if used >= self._MAX_REPLIES_PER_ANCHOR:
return None
self._anchor_msg_seq[anchor] = used + 1
return used + 1
async def _reply_synthetic(
self,
message_source: platform_events.MessageEvent,
message: platform_message.MessageChain,
) -> None:
"""Deliver a reply for a synthetic (button-click-resume) event.
Synthetic events have ``source_platform_object=None`` and no
fresh inbound msg_id. The previous INTERACTION_CREATE id we
cached in :attr:`_session_event_ids` is a valid passive-reply
anchor (``event_id``) for up to 30 minutes use it.
"""
if isinstance(message_source, platform_events.GroupMessage):
target_type = 'group'
group = getattr(message_source, 'group', None) or (
message_source.sender.group if hasattr(message_source.sender, 'group') else None
)
target_id = str(group.id) if group else None
else:
target_type = 'c2c'
target_id = str(message_source.sender.id) if message_source.sender else None
if not target_id:
await self.logger.warning('QQ Official: synthetic reply has no target_id; dropping')
return
session_key = f'{target_type}_{target_id}'
cached = self._session_event_ids.get(session_key)
event_id = cached.get('event_id') if cached else None
if cached and (time.time() - cached.get('posted_at', 0)) > self._PENDING_FORM_TTL:
event_id = None
if not event_id:
await self.logger.warning(
f'QQ Official: no cached event_id for {session_key}; '
f'cannot deliver synthetic reply within passive-reply window'
)
return
content_list = await QQOfficialMessageConverter.yiri2target(message)
text_parts = [c['content'] for c in content_list if c.get('type') == 'text' and c.get('content')]
if not text_parts:
await self.logger.info('QQ Official: synthetic reply has no text content; skipping')
return
text = '\n\n'.join(text_parts)
msg_seq = self._next_msg_seq(event_id)
if msg_seq is None:
await self.logger.warning(
f'QQ Official: anchor {event_id!r} exhausted (>5 passive replies); '
f'cannot deliver synthetic reply for {session_key}'
)
return
try:
if target_type == 'c2c':
await self.bot.send_private_text_msg(
user_openid=target_id,
content=text,
event_id=event_id,
msg_seq=msg_seq,
)
elif target_type == 'group':
await self.bot.send_group_text_msg(
group_openid=target_id,
content=text,
event_id=event_id,
msg_seq=msg_seq,
)
except Exception:
await self.logger.error(f'QQ Official: synthetic reply delivery failed: {traceback.format_exc()}')
def _resolve_target_from_source(self, source: QQOfficialEvent) -> typing.Optional[tuple[str, str]]:
"""Return ``(target_type, target_id)`` for sending a reply, or
``None`` if the scene cannot host a markdown+keyboard message."""
if source is None:
return None
if source.t == 'C2C_MESSAGE_CREATE':
return 'c2c', source.user_openid
if source.t == 'GROUP_AT_MESSAGE_CREATE':
return 'group', source.group_openid
if source.t == 'AT_MESSAGE_CREATE':
return 'channel', source.channel_id
# DIRECT_MESSAGE_CREATE uses the guild DM API which does not accept
# markdown+keyboard at the time of writing — caller falls back to text.
return None
def _resolve_target_from_event(
self, message_source: platform_events.MessageEvent
) -> typing.Optional[tuple[str, str]]:
"""Resolve ``(target_type, target_id)`` from the public event.
Prefers the platform-native source when present; falls back to
the synthesized event's sender/group fields so button-click
resume queries can still find a destination.
"""
source = message_source.source_platform_object
if source is not None:
return self._resolve_target_from_source(source)
if isinstance(message_source, platform_events.GroupMessage):
group = getattr(message_source, 'group', None) or (
message_source.sender.group
if message_source.sender and hasattr(message_source.sender, 'group')
else None
)
if group and getattr(group, 'id', None):
return 'group', str(group.id)
if isinstance(message_source, platform_events.FriendMessage):
if message_source.sender and getattr(message_source.sender, 'id', None):
return 'c2c', str(message_source.sender.id)
return None
def _prune_pending_forms(self) -> None:
now = time.time()
stale = [k for k, v in self._pending_forms.items() if now - v.get('posted_at', 0) > self._PENDING_FORM_TTL]
for k in stale:
self._pending_forms.pop(k, None)
stale_e = [
k for k, v in self._session_event_ids.items() if now - v.get('posted_at', 0) > self._PENDING_FORM_TTL
]
for k in stale_e:
self._session_event_ids.pop(k, None)
async def _handle_form_chunk(
self,
message_source: platform_events.MessageEvent,
message: platform_message.MessageChain,
form_data: dict,
) -> None:
"""Send the markdown + keyboard form prompt for a Dify pause.
Called from ``reply_message_chunk`` when the runner attaches
``_form_data`` to the final chunk. Replaces what would otherwise
be a plain-text numbered-list fallback.
"""
if self.ap is not None:
self.ap.logger.info(
f'QQ Official _handle_form_chunk entered; '
f'source_present={message_source.source_platform_object is not None} '
f'form_actions={len(form_data.get("actions") or [])}'
)
self._prune_pending_forms()
source = message_source.source_platform_object
scene_target = self._resolve_target_from_event(message_source)
if scene_target is None:
# No rich-UI fit — fall through to existing text path.
await self.logger.info('QQ Official: form chunk on unsupported scene; falling back to text')
text_parts = [m.text for m in message if type(m) is platform_message.Plain]
fallback_msg = platform_message.MessageChain([platform_message.Plain(text='\n\n'.join(text_parts))])
try:
await self.reply_message(message_source, fallback_msg)
except Exception:
await self.logger.error(f'QQ Official: form fallback text send failed: {traceback.format_exc()}')
return
target_type, target_id = scene_target
session_key = f'{target_type}_{target_id}'
# Cancel any in-flight stream / fallback ctx so plain-text prefix
# doesn't continue alongside the keyboard message.
msg_id = getattr(source, 'd_id', '') or '' if source is not None else ''
if msg_id:
self._stream_ctx.pop(msg_id, None)
self._stream_ctx_ts.pop(msg_id, None)
self._fallback_text.pop(msg_id, None)
self._fallback_text_ts.pop(msg_id, None)
node_title = form_data.get('node_title') or 'Confirmation needed'
form_content = form_data.get('form_content') or ''
parts = [f'### {node_title}']
if form_content.strip():
parts.append(form_content.strip())
parts.append('请点击下方按钮选择:')
markdown_content = '\n\n'.join(parts)
keyboard = build_keyboard_from_form(form_data, buttons_per_row=2)
if not keyboard.get('content', {}).get('rows'):
# No actions to render — fall back to plain text.
text_msg = platform_message.MessageChain([platform_message.Plain(text=markdown_content)])
try:
await self.reply_message(message_source, text_msg)
except Exception:
await self.logger.error(f'QQ Official: empty-keyboard fallback send failed: {traceback.format_exc()}')
return
# Prefer the inbound msg_id (no quota cost). If the source is a
# synthetic event from a prior click, the cached interaction id
# serves as event_id for up to 30 min.
event_id = None
if not msg_id:
cached = self._session_event_ids.get(session_key)
if cached and (time.time() - cached.get('posted_at', 0)) < self._PENDING_FORM_TTL:
event_id = cached.get('event_id')
anchor = msg_id or event_id or ''
msg_seq = self._next_msg_seq(anchor)
if msg_seq is None:
await self.logger.warning(
f'QQ Official: anchor {anchor!r} exhausted (>5 passive replies); '
f'cannot deliver form card for session={session_key}'
)
return
try:
await self.bot.send_markdown_keyboard(
target_type=target_type,
target_id=target_id,
markdown_content=markdown_content,
keyboard=keyboard,
msg_id=msg_id if (msg_id and not event_id) else None,
event_id=event_id,
msg_seq=msg_seq,
)
if self.ap is not None:
self.ap.logger.info(
f'QQ Official: form card sent '
f'target={target_type}/{target_id} '
f'msg_id={msg_id!r} event_id={event_id!r} msg_seq={msg_seq}'
)
except Exception:
if self.ap is not None:
self.ap.logger.error(
f'QQ Official: send_markdown_keyboard failed, falling back to text: {traceback.format_exc()}'
)
await self.logger.error(
f'QQ Official: send_markdown_keyboard failed, falling back to text: {traceback.format_exc()}'
)
text_msg = platform_message.MessageChain([platform_message.Plain(text=markdown_content)])
try:
await self.reply_message(message_source, text_msg)
except Exception:
pass
return
sender_id = ''
if source is not None:
sender_id = (
getattr(source, 'user_openid', None)
or getattr(source, 'member_openid', None)
or getattr(source, 'd_author_id', None)
or ''
)
if not sender_id and message_source.sender is not None:
sender_id = str(getattr(message_source.sender, 'id', '') or '')
self._pending_forms[session_key] = {
'form_data': form_data,
'msg_id': msg_id,
'sender_id': sender_id,
'target_type': target_type,
'target_id': target_id,
'source_event_t': source.t if source is not None else None,
'posted_at': time.time(),
}
await self.logger.info(
f'QQ Official: form posted session={session_key} actions={len(form_data.get("actions") or [])}'
)
async def _handle_interaction_create(
self,
event_data: dict,
ws_event_id: typing.Optional[str] = None,
) -> None:
"""Handle a button-click INTERACTION_CREATE event.
Two IDs at play (QQ keeps them separate):
ws_event_id top-level payload ``id`` (or webhook ``X-Bot-
Event-Id``). The ONLY value accepted as
``event_id`` for subsequent passive replies.
d['id'] the interaction id used for PUT
/interactions/{id} ack. Cannot be reused as
event_id (QQ returns 40034025 if you try).
Layout (https://bot.q.qq.com/.../msg-btn.html):
chat_type 0 channel / 1 group / 2 c2c
data.resolved.button_data what we set as ``action.data``
data.resolved.button_id ``id`` field on the button row
"""
import langbot_plugin.api.entities.builtin.provider.session as provider_session
if self.ap is not None:
self.ap.logger.info(
f'QQ Official _handle_interaction_create entered; '
f'ws_event_id={ws_event_id!r} '
f'interaction_id={(event_data.get("id") if isinstance(event_data, dict) else None)!r} '
f'chat_type={event_data.get("chat_type") if isinstance(event_data, dict) else None}'
)
if not isinstance(event_data, dict):
await self.logger.warning(f'QQ Official: INTERACTION_CREATE event_data is not dict: {type(event_data)}')
return
# ACK uses the interaction id, NOT the ws event id.
interaction_id = event_data.get('id') or ''
if interaction_id:
asyncio.create_task(self.bot.ack_interaction(interaction_id, code=0))
resolved = (event_data.get('data') or {}).get('resolved') or {}
action_id = str(resolved.get('button_data') or resolved.get('button_id') or '').strip()
if not action_id:
await self.logger.warning('QQ Official: INTERACTION_CREATE missing button_data/button_id; ignoring')
return
chat_type = event_data.get('chat_type')
scene_target: typing.Optional[tuple[str, str]] = None
if chat_type == 2 or event_data.get('user_openid'):
scene_target = ('c2c', event_data.get('user_openid') or '')
elif chat_type == 1 or event_data.get('group_openid'):
scene_target = ('group', event_data.get('group_openid') or '')
elif chat_type == 0 or event_data.get('channel_id'):
scene_target = ('channel', event_data.get('channel_id') or '')
if not scene_target or not scene_target[1]:
await self.logger.warning(f'QQ Official: INTERACTION_CREATE missing scene/target; raw={event_data}')
return
target_type, target_id = scene_target
session_key = f'{target_type}_{target_id}'
self._prune_pending_forms()
pending = self._pending_forms.pop(session_key, None)
if not pending:
await self.logger.warning(
f'QQ Official: no pending form for session {session_key}; click ignored (action_id={action_id!r})'
)
return
# Cache ws_event_id so a follow-up pause / text reply can use it
# as event_id for passive delivery (30-min window). Falls back to
# the interaction_id only if no ws_event_id was provided (e.g.
# tests / older payload shape) — QQ will reject that value but
# we log so the mismatch is debuggable.
cached_event_id = ws_event_id or interaction_id
if cached_event_id:
self._session_event_ids[session_key] = {
'event_id': cached_event_id,
'posted_at': time.time(),
}
# New anchor → fresh 5-reply budget.
self._anchor_msg_seq[cached_event_id] = 0
if self.ap is not None and not ws_event_id:
self.ap.logger.warning(
'QQ Official: INTERACTION_CREATE lacked ws_event_id; '
'falling back to interaction_id (passive reply may be rejected)'
)
form_data: dict = pending.get('form_data') or {}
actions = form_data.get('actions') or []
matched = next(
(a for a in actions if str(a.get('id', '')) == action_id),
None,
)
action_title = (matched or {}).get('title') or action_id
sender_id = pending.get('sender_id') or event_data.get('user_openid') or event_data.get('member_openid') or ''
# Build resume payload matching the shape every other adapter uses
# (DingTalk / Lark / Telegram / WeCom). The runner's
# _merge_pending_form_action consumes this verbatim.
if target_type == 'group' or target_type == 'channel':
launcher_type = provider_session.LauncherTypes.GROUP
launcher_id = target_id
else:
launcher_type = provider_session.LauncherTypes.PERSON
launcher_id = sender_id or target_id
form_action_data = {
'form_token': form_data.get('form_token', ''),
'workflow_run_id': form_data.get('workflow_run_id', ''),
'action_id': action_id,
'action_title': action_title,
'node_title': form_data.get('node_title', ''),
'user': f'{launcher_type.value}_{launcher_id}',
'inputs': {},
}
message_chain = platform_message.MessageChain([platform_message.Plain(text=f'[Form Action: {action_title}]')])
if launcher_type == provider_session.LauncherTypes.GROUP:
synthetic_event: platform_events.MessageEvent = platform_events.GroupMessage(
sender=platform_entities.GroupMember(
id=sender_id or launcher_id,
member_name='',
permission='MEMBER',
group=platform_entities.Group(
id=launcher_id,
name='',
permission=platform_entities.Permission.Member,
),
special_title='',
),
message_chain=message_chain,
time=int(time.time()),
source_platform_object=None,
)
else:
synthetic_event = platform_events.FriendMessage(
sender=platform_entities.Friend(
id=sender_id or launcher_id,
nickname='',
remark='',
),
message_chain=message_chain,
time=int(time.time()),
source_platform_object=None,
)
if self.ap is None:
await self.logger.error('QQ Official: ap not injected; cannot enqueue button-click query')
return
bot_uuid = ''
pipeline_uuid = None
for bot in self.ap.platform_mgr.bots:
if bot.adapter is self:
bot_uuid = bot.bot_entity.uuid
pipeline_uuid = bot.bot_entity.use_pipeline_uuid
break
try:
await self.ap.query_pool.add_query(
bot_uuid=bot_uuid,
launcher_type=launcher_type,
launcher_id=launcher_id,
sender_id=sender_id or launcher_id,
message_event=synthetic_event,
message_chain=message_chain,
adapter=self,
pipeline_uuid=pipeline_uuid,
variables={
'_dify_form_action': form_action_data,
'_routed_by_rule': True,
},
)
await self.logger.info(
f'QQ Official: button-click query enqueued action_id={action_id!r} session={session_key}'
)
except Exception:
await self.logger.error(f'QQ Official: enqueue button-click query failed: {traceback.format_exc()}')
@@ -19,18 +19,6 @@ spec:
en: https://link.langbot.app/en/platforms/qqofficial en: https://link.langbot.app/en/platforms/qqofficial
ja: https://link.langbot.app/ja/platforms/qqofficial ja: https://link.langbot.app/ja/platforms/qqofficial
config: config:
- name: one-click-bind
label:
en_US: One-Click QR Binding
zh_Hans: 一键扫码绑定
zh_Hant: 一鍵掃碼綁定
description:
en_US: Scan QR code with mobile QQ to auto-fill AppID and Secret (Token still needs to be filled manually)
zh_Hans: 使用手机 QQ 扫码绑定,自动填写 AppID 和密钥(Token 仍需手动填写)
zh_Hant: 使用手機 QQ 掃碼綁定,自動填寫 AppID 和密鑰(Token 仍需手動填寫)
type: qr-code-login
login_platform: qqofficial
required: false
- name: appid - name: appid
label: label:
en_US: App ID en_US: App ID
@@ -52,12 +40,8 @@ spec:
en_US: Token en_US: Token
zh_Hans: 令牌 zh_Hans: 令牌
zh_Hant: 令牌 zh_Hant: 令牌
description:
en_US: Optional. The QR binding cannot return this value; the current adapter implementation does not use it either, so it can be safely left blank.
zh_Hans: 可选。扫码绑定无法获取该字段,当前适配器实现也未使用该字段,留空即可。
zh_Hant: 可選。掃碼綁定無法取得此欄位,目前介面卡實作亦未使用,留空即可。
type: string type: string
required: false required: true
default: "" default: ""
- name: enable-webhook - name: enable-webhook
label: label:
+25 -296
View File
@@ -1,14 +1,14 @@
from __future__ import annotations from __future__ import annotations
import time
import telegram import telegram
import telegram.ext import telegram.ext
from telegram import Update, InlineKeyboardButton, InlineKeyboardMarkup from telegram import Update
from telegram.ext import ApplicationBuilder, ContextTypes, MessageHandler, CallbackQueryHandler, filters from telegram.ext import ApplicationBuilder, ContextTypes, MessageHandler, filters
import telegramify_markdown import telegramify_markdown
import typing import typing
import traceback import traceback
import json
import base64 import base64
import pydantic import pydantic
@@ -167,7 +167,7 @@ class TelegramEventConverter(abstract_platform_adapter.AbstractEventConverter):
time=event.message.date.timestamp(), time=event.message.date.timestamp(),
source_platform_object=event, source_platform_object=event,
) )
elif event.effective_chat.type in ('group', 'supergroup'): elif event.effective_chat.type == 'group' or 'supergroup':
return platform_events.GroupMessage( return platform_events.GroupMessage(
sender=platform_entities.GroupMember( sender=platform_entities.GroupMember(
id=event.effective_chat.id, id=event.effective_chat.id,
@@ -189,7 +189,6 @@ class TelegramEventConverter(abstract_platform_adapter.AbstractEventConverter):
class TelegramAdapter(abstract_platform_adapter.AbstractMessagePlatformAdapter): class TelegramAdapter(abstract_platform_adapter.AbstractMessagePlatformAdapter):
bot: telegram.Bot = pydantic.Field(exclude=True) bot: telegram.Bot = pydantic.Field(exclude=True)
application: telegram.ext.Application = pydantic.Field(exclude=True) application: telegram.ext.Application = pydantic.Field(exclude=True)
ap: typing.Any = pydantic.Field(exclude=True, default=None)
message_converter: TelegramMessageConverter = TelegramMessageConverter() message_converter: TelegramMessageConverter = TelegramMessageConverter()
event_converter: TelegramEventConverter = TelegramEventConverter() event_converter: TelegramEventConverter = TelegramEventConverter()
@@ -205,8 +204,6 @@ class TelegramAdapter(abstract_platform_adapter.AbstractMessagePlatformAdapter):
typing.Callable[[platform_events.Event, abstract_platform_adapter.AbstractMessagePlatformAdapter], None], typing.Callable[[platform_events.Event, abstract_platform_adapter.AbstractMessagePlatformAdapter], None],
] = {} ] = {}
_form_action_titles: typing.Dict[str, str] = {} # action_id -> action_title mapping
def __init__(self, config: dict, logger: abstract_platform_logger.AbstractEventLogger): def __init__(self, config: dict, logger: abstract_platform_logger.AbstractEventLogger):
async def telegram_callback(update: Update, context: ContextTypes.DEFAULT_TYPE): async def telegram_callback(update: Update, context: ContextTypes.DEFAULT_TYPE):
if update.message.from_user.is_bot: if update.message.from_user.is_bot:
@@ -227,118 +224,6 @@ class TelegramAdapter(abstract_platform_adapter.AbstractMessagePlatformAdapter):
telegram_callback, telegram_callback,
) )
) )
async def callback_query_handler(update: Update, context: ContextTypes.DEFAULT_TYPE):
query = update.callback_query
await query.answer()
try:
data = json.loads(query.data)
if data.get('form_action') or data.get('f'):
import langbot_plugin.api.entities.builtin.provider.session as provider_session
# workflow_run_id is not in the callback payload (too large
# for Telegram's 64-byte limit). Only w_suffix is sent;
# the runner resolves the full run id from _PENDING_FORMS.
w_suffix = data.get('w', '')
action_id = data.get('action_id') or data.get('a', '')
session_key = data.get('session_key') or data.get('s', '')
# Show selected action feedback by editing the original message
action_title = self._form_action_titles.get(action_id, action_id)
try:
original_text = query.message.text or ''
selected_text = f'{original_text}\n\n✅ Selected: {action_title}'
await query.edit_message_text(text=selected_text, reply_markup=None)
except Exception:
# If edit fails (e.g. message too long), just pass
pass
finally:
# Clean up the stored title
self._form_action_titles.pop(action_id, None)
if session_key.startswith('group_') or session_key.startswith('g:'):
launcher_type = provider_session.LauncherTypes.GROUP
launcher_id = (
session_key.split(':', 1)[1]
if session_key.startswith('g:')
else session_key[len('group_') :]
)
else:
launcher_type = provider_session.LauncherTypes.PERSON
launcher_id = (
session_key.split(':', 1)[1]
if session_key.startswith('p:')
else session_key[len('person_') :]
)
user_id = str(query.from_user.id)
# Find bot_uuid and pipeline_uuid
bot_uuid = ''
pipeline_uuid = None
for b in self.ap.platform_mgr.bots:
if b.adapter is self:
bot_uuid = b.bot_entity.uuid
pipeline_uuid = b.bot_entity.use_pipeline_uuid
break
form_action_data = {
# workflow_run_id is intentionally omitted; the runner
# resolves it from w_suffix via _PENDING_FORMS.
'w_suffix': w_suffix,
'action_id': action_id,
'user': f'{launcher_type.value}_{launcher_id}',
'inputs': {},
}
message_chain = platform_message.MessageChain(
[platform_message.Plain(text=f'[Form Action: {action_id}]')]
)
if launcher_type == provider_session.LauncherTypes.GROUP:
synthetic_event = platform_events.GroupMessage(
sender=platform_entities.GroupMember(
id=user_id,
member_name='',
permission=platform_entities.Permission.Member,
group=platform_entities.Group(
id=launcher_id,
name='',
permission=platform_entities.Permission.Member,
),
),
message_chain=message_chain,
source_platform_object=update,
)
else:
synthetic_event = platform_events.FriendMessage(
sender=platform_entities.Friend(
id=user_id,
nickname='',
remark='',
),
message_chain=message_chain,
source_platform_object=update,
)
await self.ap.query_pool.add_query(
bot_uuid=bot_uuid,
launcher_type=launcher_type,
launcher_id=launcher_id,
sender_id=user_id,
message_event=synthetic_event,
message_chain=message_chain,
adapter=self,
pipeline_uuid=pipeline_uuid,
variables={
'_dify_form_action': form_action_data,
'_routed_by_rule': True,
},
)
except Exception:
await self.logger.error(f'Error in telegram callback query: {traceback.format_exc()}')
application.add_handler(CallbackQueryHandler(callback_query_handler))
super().__init__( super().__init__(
config=config, config=config,
logger=logger, logger=logger,
@@ -429,34 +314,23 @@ class TelegramAdapter(abstract_platform_adapter.AbstractMessagePlatformAdapter):
args['parse_mode'] = 'MarkdownV2' args['parse_mode'] = 'MarkdownV2'
return args return args
async def _delete_group_stream_message(self, chat_mode: str, chat_id: int, stream_id: int | None):
if chat_mode != 'group' or stream_id is None:
return
try:
await self.bot.delete_message(chat_id=chat_id, message_id=stream_id)
except telegram.error.TelegramError:
pass
@staticmethod
def _is_form_placeholder_chunk(text: str) -> bool:
"""Return True for invisible placeholder chunks used to carry forms."""
if not text:
return True
cleaned = text.replace('\u200b', '').replace('\u200c', '').replace('\u200d', '').replace('\ufeff', '').strip()
return cleaned == ''
async def create_message_card(self, message_id, event): async def create_message_card(self, message_id, event):
assert isinstance(event.source_platform_object, Update) assert isinstance(event.source_platform_object, Update)
update = event.source_platform_object update = event.source_platform_object
chat_id = update.effective_chat.id chat_id = update.effective_chat.id
effective_message = update.effective_message chat_type = update.effective_chat.type
message_thread_id = getattr(effective_message, 'message_thread_id', None) if effective_message else None message_thread_id = update.message.message_thread_id
args = self._build_message_args(chat_id, 'Thinking...', message_thread_id) if chat_type == 'private':
send_msg = await self.bot.send_message(**args) draft_id = int(time.time() * 1000)
self.msg_stream_id[message_id] = ('message', send_msg.message_id, False) self.msg_stream_id[message_id] = ('private', draft_id)
args = self._build_message_args(chat_id, 'Thinking...', message_thread_id, draft_id=draft_id)
await self.bot.send_message_draft(**args)
else:
args = self._build_message_args(chat_id, 'Thinking...', message_thread_id)
send_msg = await self.bot.send_message(**args)
self.msg_stream_id[message_id] = ('group', send_msg.message_id)
return True return True
@@ -473,15 +347,12 @@ class TelegramAdapter(abstract_platform_adapter.AbstractMessagePlatformAdapter):
assert isinstance(message_source.source_platform_object, Update) assert isinstance(message_source.source_platform_object, Update)
update = message_source.source_platform_object update = message_source.source_platform_object
chat_id = update.effective_chat.id chat_id = update.effective_chat.id
effective_message = update.effective_message message_thread_id = update.message.message_thread_id
message_thread_id = getattr(effective_message, 'message_thread_id', None) if effective_message else None
if message_id not in self.msg_stream_id: if message_id not in self.msg_stream_id:
return return
stream_state = self.msg_stream_id[message_id] chat_mode, draft_id = self.msg_stream_id[message_id]
chat_mode, stream_id = stream_state[:2]
has_visible_content = len(stream_state) > 2 and stream_state[2]
components = await TelegramMessageConverter.yiri2target(message, self.bot) components = await TelegramMessageConverter.yiri2target(message, self.bot)
if not components or components[0]['type'] != 'text': if not components or components[0]['type'] != 'text':
@@ -490,68 +361,17 @@ class TelegramAdapter(abstract_platform_adapter.AbstractMessagePlatformAdapter):
return return
content = components[0]['text'] content = components[0]['text']
form_data = getattr(bot_message, '_form_data', None)
if form_data and is_final:
if not has_visible_content:
await self._send_form_action_buttons(message_source, form_data, edit_message_id=stream_id)
else:
await self._send_form_action_buttons(message_source, form_data)
self.msg_stream_id.pop(message_id, None)
return
if self._is_form_placeholder_chunk(content):
if is_final and bot_message.tool_calls is None and not has_visible_content:
await self._delete_group_stream_message(chat_mode, chat_id, stream_id)
self.msg_stream_id.pop(message_id, None)
return
if chat_mode == 'private': if chat_mode == 'private':
# Streaming via draft (ephemeral preview in the chat input area) args = self._build_message_args(chat_id, content, message_thread_id, draft_id=draft_id)
if (msg_seq - 1) % 8 == 0 or is_final: await self.bot.send_message_draft(**args)
args = self._build_message_args(chat_id, content, message_thread_id, draft_id=stream_id)
try:
await self.bot.send_message_draft(**args)
except telegram.error.BadRequest as exc:
if 'Message_too_long' in str(exc):
args['text'] = content[:4000] + '\n\n… (truncated)'
try:
await self.bot.send_message_draft(**args)
except telegram.error.RetryAfter:
pass
else:
pass # Ignore other draft errors (cosmetic)
self.msg_stream_id[message_id] = (chat_mode, stream_id, True)
if is_final and bot_message.tool_calls is None: if is_final and bot_message.tool_calls is None:
# Finalise: send the real message, discard the draft del args['draft_id']
args = self._build_message_args(chat_id, content, message_thread_id) await self.bot.send_message(**args)
try:
await self.bot.send_message(**args)
except telegram.error.BadRequest as exc:
if 'Message_too_long' in str(exc):
args['text'] = content[:4000] + '\n\n… (truncated)'
await self.bot.send_message(**args)
else:
raise
self.msg_stream_id.pop(message_id) self.msg_stream_id.pop(message_id)
else: else:
# Streaming via edit_message_text (persistent message) stream_id = draft_id
if stream_id is None: if (msg_seq - 1) % 8 == 0 or is_final:
args = self._build_message_args(chat_id, content, message_thread_id)
try:
send_msg = await self.bot.send_message(**args)
except telegram.error.BadRequest as exc:
if 'Message_too_long' in str(exc):
args['text'] = self._process_markdown(content[:4000] + '\n\n鈥?(truncated)')
send_msg = await self.bot.send_message(**args)
else:
raise
self.msg_stream_id[message_id] = (chat_mode, send_msg.message_id, True)
if is_final and bot_message.tool_calls is None:
self.msg_stream_id.pop(message_id, None)
return
if not has_visible_content or (msg_seq - 1) % 8 == 0 or is_final:
args = { args = {
'message_id': stream_id, 'message_id': stream_id,
'chat_id': chat_id, 'chat_id': chat_id,
@@ -559,102 +379,11 @@ class TelegramAdapter(abstract_platform_adapter.AbstractMessagePlatformAdapter):
} }
if self.config.get('markdown_card', False): if self.config.get('markdown_card', False):
args['parse_mode'] = 'MarkdownV2' args['parse_mode'] = 'MarkdownV2'
try: await self.bot.edit_message_text(**args)
await self.bot.edit_message_text(**args)
except telegram.error.BadRequest as exc:
if 'Message_too_long' in str(exc):
args['text'] = self._process_markdown(content[:4000] + '\n\n… (truncated)')
await self.bot.edit_message_text(**args)
else:
raise
self.msg_stream_id[message_id] = (chat_mode, stream_id, True)
if is_final and bot_message.tool_calls is None: if is_final and bot_message.tool_calls is None:
self.msg_stream_id.pop(message_id) self.msg_stream_id.pop(message_id)
async def _send_form_action_buttons(
self,
message_source: platform_events.MessageEvent,
form_data: dict,
edit_message_id: int | None = None,
):
"""Send inline keyboard buttons for Dify human_input_required form actions."""
actions = form_data.get('actions', [])
node_title = form_data.get('node_title', '')
form_content = form_data.get('form_content', '')
workflow_run_id = form_data.get('workflow_run_id', '')
# Telegram callback_data is capped at 64 bytes, so we identify the
# paused workflow by the last 8 chars of workflow_run_id (unique
# within a session with overwhelming probability).
w_suffix = workflow_run_id[-8:] if workflow_run_id else ''
if isinstance(message_source, platform_events.GroupMessage):
session_key = f'g:{message_source.group.id}'
else:
session_key = f'p:{message_source.sender.id}'
keyboard = []
oversized = False
for action in actions:
action_id = action.get('id', '')
action_title = action.get('title', action_id)
# Store action_id -> title mapping for displaying selection feedback
self._form_action_titles[action_id] = action_title
callback_payload = {'f': 1, 'a': action_id, 's': session_key}
if w_suffix:
callback_payload['w'] = w_suffix
callback_data = json.dumps(callback_payload, separators=(',', ':'))
if len(callback_data.encode('utf-8')) > 64:
oversized = True
break
keyboard.append([InlineKeyboardButton(action_title, callback_data=callback_data)])
update = message_source.source_platform_object
chat_id = update.effective_chat.id
effective_message = update.effective_message
message_thread_id = getattr(effective_message, 'message_thread_id', None) if effective_message else None
text_lines = [f'[{node_title}] Please select an action:']
if form_content:
text_lines.insert(0, form_content)
if oversized:
# callback_data exceeds Telegram's 64-byte limit — fall back to
# a plain-text numbered list so the user can reply by number.
for idx, action in enumerate(actions, start=1):
title = action.get('title') or action.get('id') or ''
text_lines.append(f' {idx}. {title}')
args = {
'chat_id': chat_id,
'text': '\n\n'.join(text_lines),
}
else:
reply_markup = InlineKeyboardMarkup(keyboard)
args = {
'chat_id': chat_id,
'text': '\n\n'.join(text_lines),
'reply_markup': reply_markup,
}
if message_thread_id:
args['message_thread_id'] = message_thread_id
if edit_message_id is not None:
edit_args = {
'chat_id': chat_id,
'message_id': edit_message_id,
'text': args['text'],
}
if 'reply_markup' in args:
edit_args['reply_markup'] = args['reply_markup']
try:
await self.bot.edit_message_text(**edit_args)
return
except telegram.error.TelegramError:
await self._delete_group_stream_message('group', chat_id, edit_message_id)
await self.bot.send_message(**args)
def get_launcher_id(self, event: platform_events.MessageEvent) -> str | None: def get_launcher_id(self, event: platform_events.MessageEvent) -> str | None:
if not isinstance(event.source_platform_object, Update): if not isinstance(event.source_platform_object, Update):
return None return None
+4 -328
View File
@@ -296,7 +296,6 @@ class WecomBotAdapter(abstract_platform_adapter.AbstractMessagePlatformAdapter):
listeners: dict = {} listeners: dict = {}
_stream_to_monitoring_msg: dict = {} # Maps stream_id to (monitoring_message_id, timestamp) _stream_to_monitoring_msg: dict = {} # Maps stream_id to (monitoring_message_id, timestamp)
_STREAM_MAPPING_TTL = 600 # 10 minutes _STREAM_MAPPING_TTL = 600 # 10 minutes
ap: typing.Any = None
def __init__(self, config: dict, logger: EventLogger): def __init__(self, config: dict, logger: EventLogger):
enable_webhook = config.get('enable-webhook', False) enable_webhook = config.get('enable-webhook', False)
@@ -337,25 +336,6 @@ class WecomBotAdapter(abstract_platform_adapter.AbstractMessagePlatformAdapter):
_stream_to_monitoring_msg={}, _stream_to_monitoring_msg={},
) )
# Both WecomBotClient (webhook) and WecomBotWsClient (ws long-conn)
# expose ``set_card_action_callback``. Wire the click handler so
# Dify human-input button taps resume the workflow on either mode.
if hasattr(self.bot, 'set_card_action_callback'):
self.bot.set_card_action_callback(self._on_card_action)
# Hand the client a `source` block so every interactive
# template_card it emits carries the LangBot logo + name at the
# top — the WeCom analogue of DingTalk's Avatar header.
# Always on; icon_url accepts plain HTTPS URLs (no upload needed).
if hasattr(self.bot, 'set_card_source'):
self.bot.set_card_source(
{
'icon_url': 'https://raw.githubusercontent.com/RockChinQ/LangBot/master/res/logo-blue.png',
'desc': 'LangBot',
'desc_color': 0,
}
)
async def reply_message( async def reply_message(
self, self,
message_source: platform_events.MessageEvent, message_source: platform_events.MessageEvent,
@@ -365,37 +345,15 @@ class WecomBotAdapter(abstract_platform_adapter.AbstractMessagePlatformAdapter):
content = await self.message_converter.yiri2target(message) content = await self.message_converter.yiri2target(message)
_ws_mode = not self.config.get('enable-webhook', False) _ws_mode = not self.config.get('enable-webhook', False)
event = message_source.source_platform_object
# Synthetic events (button-click resume queries) have no inbound
# platform object. Fall back to a proactive send so error
# messages and one-shot replies still reach the user.
if event is None:
if _ws_mode:
if isinstance(message_source, platform_events.GroupMessage):
chat_id = str(message_source.group.id)
else:
chat_id = str(message_source.sender.id)
try:
await self.bot.send_message(chat_id, content)
except Exception:
await self.logger.error(
f'WeComBot: proactive reply for synthetic event failed: {traceback.format_exc()}'
)
else:
await self.logger.warning(
'WeComBot webhook mode cannot reply to a synthetic event '
'(no req_id and no proactive-send credentials); dropping.'
)
return
if _ws_mode: if _ws_mode:
req_id = event.get('req_id', '') if isinstance(event, dict) else getattr(event, 'req_id', '') event = message_source.source_platform_object
req_id = event.get('req_id', '')
if req_id: if req_id:
await self.bot.reply_text(req_id, content) await self.bot.reply_text(req_id, content)
else: else:
await self.bot.set_message(event.message_id, content) await self.bot.set_message(event.message_id, content)
else: else:
await self.bot.set_message(event.message_id, content) await self.bot.set_message(message_source.source_platform_object.message_id, content)
async def reply_message_chunk( async def reply_message_chunk(
self, self,
@@ -406,56 +364,9 @@ class WecomBotAdapter(abstract_platform_adapter.AbstractMessagePlatformAdapter):
is_final: bool = False, is_final: bool = False,
): ):
content = await self.message_converter.yiri2target(message) content = await self.message_converter.yiri2target(message)
msg_id = message_source.source_platform_object.message_id
_ws_mode = not self.config.get('enable-webhook', False) _ws_mode = not self.config.get('enable-webhook', False)
# Synthetic events (e.g. button-click triggered form resume) have
# no inbound platform message — no msg_id, no req_id, no stream
# session. The output must go via the proactive-send path instead
# of the stream/reply path.
spo = message_source.source_platform_object
if spo is None:
return await self._handle_synthetic_chunk(message_source, bot_message, content, is_final, _ws_mode)
msg_id = spo.message_id
# Dify human-input pause: when the runner attaches `_form_data` to
# the final chunk, hand the button_interaction card off to the
# underlying client. In webhook mode the card is queued for the
# next followup poll; in ws mode it's sent as a reply frame
# immediately. Falls back to plain text when the bot has no active
# stream session for this msg_id (rare).
form_data = getattr(bot_message, '_form_data', None)
if form_data and is_final:
if hasattr(self.bot, 'push_form_pause'):
ok, stream_id, task_id = await self.bot.push_form_pause(msg_id, form_data)
if ok:
await self.logger.info(
f'WeComBot: pending button_interaction registered '
f'stream_id={stream_id} task_id={task_id} ws_mode={_ws_mode}'
)
return {'stream': True, 'form': True, 'task_id': task_id}
await self.logger.warning(
'WeComBot: cannot register form pause (no active stream session); falling back to plain text'
)
try:
from langbot.pkg.provider.runners.difysvapi import _format_human_input_text
fallback = _format_human_input_text(
form_data.get('node_title', ''),
form_data.get('form_content', ''),
form_data.get('actions', []) or [],
)
except Exception:
fallback = content or '(人工输入)'
if _ws_mode:
event = message_source.source_platform_object
req_id = event.get('req_id', '') if isinstance(event, dict) else getattr(event, 'req_id', '')
if req_id:
await self.bot.reply_text(req_id, fallback)
else:
await self.bot.set_message(msg_id, fallback)
return {'stream': False, 'form': True, 'fallback': True}
if _ws_mode: if _ws_mode:
success = await self.bot.push_stream_chunk(msg_id, content, is_final=is_final) success = await self.bot.push_stream_chunk(msg_id, content, is_final=is_final)
if not success and is_final: if not success and is_final:
@@ -474,130 +385,6 @@ class WecomBotAdapter(abstract_platform_adapter.AbstractMessagePlatformAdapter):
"""Whether streaming output is enabled for this bot instance.""" """Whether streaming output is enabled for this bot instance."""
return self.config.get('enable-stream-reply', True) return self.config.get('enable-stream-reply', True)
async def _handle_synthetic_chunk(
self,
message_source: platform_events.MessageEvent,
bot_message,
content: str,
is_final: bool,
ws_mode: bool,
) -> dict:
"""Handle reply_message_chunk for synthetic events (button clicks).
Synthetic events have no inbound message no msg_id, no req_id,
no stream session. We can't do incremental streaming, so we
buffer chunks per-conversation and flush on ``is_final`` via the
proactive send path.
Buffer keyed by ``(launcher_type, launcher_id)`` from the
synthetic event itself. Only ws mode has a usable proactive-send
path right now (``ws_client.send_message`` /
``ws_client.send_template_card``); webhook mode requires a
corpid/secret we don't have, so it logs and drops.
"""
if isinstance(message_source, platform_events.GroupMessage):
chat_id = str(message_source.group.id)
else:
chat_id = str(message_source.sender.id)
form_data = getattr(bot_message, '_form_data', None)
# Buffer streaming content until is_final.
buf_key = chat_id
if not hasattr(self, '_synthetic_buffers'):
# Attribute-not-declared trick: pydantic forbids dynamic attrs
# on the model, but plain instance dicts via object.__setattr__
# do work. Lazy-create on first call.
object.__setattr__(self, '_synthetic_buffers', {})
buffers: dict[str, str] = self._synthetic_buffers
if content and not form_data:
buffers[buf_key] = buffers.get(buf_key, '') + content
if not is_final:
return {'stream': True, 'synthetic': True, 'buffered': True}
final_content = buffers.pop(buf_key, '')
if content and final_content.startswith(content):
# is_final chunk re-emitted the full accumulated text — keep
# whichever is longer.
final_content = final_content if len(final_content) >= len(content) else content
elif content and not final_content:
final_content = content
if not ws_mode:
await self.logger.warning(
'WeComBot webhook mode cannot proactively push synthetic-event '
'output (no corpid/secret); the resume reply is dropped. '
f'content_len={len(final_content)} form_data_present={form_data is not None}'
)
return {'stream': False, 'synthetic': True, 'dropped': True}
# ws mode: proactive send.
try:
if form_data:
# Determine user_id / chat_id for the routing context of any
# subsequent click on this card.
if isinstance(message_source, platform_events.GroupMessage):
routing_chat_id = str(message_source.group.id)
routing_user_id = str(message_source.sender.id)
else:
routing_chat_id = ''
routing_user_id = str(message_source.sender.id)
payload = self._build_button_interaction_payload_from_form(
form_data,
user_id=routing_user_id,
chat_id=routing_chat_id,
)
await self.bot.send_template_card(chat_id, payload)
await self.logger.info(
f'WeComBot ws: proactively sent template_card for synthetic event '
f'chat_id={chat_id} form_token={form_data.get("form_token")!r} '
f'workflow_run_id={form_data.get("workflow_run_id")!r}'
)
elif final_content:
await self.bot.send_message(chat_id, final_content)
await self.logger.info(
f'WeComBot ws: proactively sent text for synthetic event chat_id={chat_id} len={len(final_content)}'
)
except Exception:
await self.logger.error(f'WeComBot: synthetic event proactive send failed: {traceback.format_exc()}')
return {'stream': False, 'synthetic': True, 'error': True}
return {'stream': True, 'synthetic': True}
def _build_button_interaction_payload_from_form(
self, form_data: dict, *, user_id: str = '', chat_id: str = ''
) -> dict:
"""Build a button_interaction payload + track task_id for click resolution.
Unlike the inbound-event path (where push_form_pause registers the
task_id with the active stream session), proactive sends still
need the task_id registered so button clicks find pending_form.
For ws mode we stash it directly on the ws_client's pending dict.
"""
from langbot.libs.wecom_ai_bot_api.api import build_button_interaction_payload
import secrets as _secrets
task_id = f'dify-{_secrets.token_hex(12)}'
source = getattr(self.bot, 'card_source', None)
payload = build_button_interaction_payload(form_data, task_id, source=source)
# Register task_id → form_data so the click callback can find it.
# user_id / chat_id are required so _on_card_action can route the
# resulting synthetic query back to the right user. msg_id / req_id
# / stream_id are intentionally empty — synthetic cards have no
# inbound message to anchor on.
if hasattr(self.bot, '_pending_forms_by_task'):
self.bot._pending_forms_by_task[task_id] = {
'form_data': form_data,
'msg_id': '',
'user_id': user_id,
'chat_id': chat_id,
'stream_id': '',
'req_id': '',
}
return payload
async def send_message(self, target_type, target_id, message): async def send_message(self, target_type, target_id, message):
_ws_mode = not self.config.get('enable-webhook', False) _ws_mode = not self.config.get('enable-webhook', False)
if _ws_mode: if _ws_mode:
@@ -744,114 +531,3 @@ class WecomBotAdapter(abstract_platform_adapter.AbstractMessagePlatformAdapter):
async def is_muted(self, group_id: int) -> bool: async def is_muted(self, group_id: int) -> bool:
pass pass
# ------------------------------------------------------------------
# Dify human-input button-interaction click handling
# ------------------------------------------------------------------
async def _on_card_action(self, session, action_id: str, task_id: str, raw_event: dict) -> None:
"""Translate a button click on a button_interaction card into a
synthetic ``_dify_form_action`` query enqueued on the pool.
Pattern mirrors DingTalk / Lark / Telegram so the runner's
``_merge_pending_form_action`` path resumes the workflow.
"""
import langbot_plugin.api.entities.builtin.provider.session as provider_session
form = session.pending_form or {}
await self.logger.info(
f'WeComBot _on_card_action: task_id={task_id} action_id={action_id!r} '
f'form_token={form.get("form_token")!r} workflow_run_id={form.get("workflow_run_id")!r} '
f'session.user_id={session.user_id!r} session.chat_id={session.chat_id!r}'
)
actions = form.get('actions') or []
clean_action_id = (action_id or '').strip()
action_title = clean_action_id
for a in actions:
if str(a.get('id', '')) == clean_action_id:
action_title = a.get('title') or clean_action_id
break
launcher_id = session.user_id or session.chat_id or ''
sender_user_id = session.user_id or launcher_id
# WeCom AI bot has both single-chat and group-chat; chat_id present
# indicates group context.
if session.chat_id:
launcher_type = provider_session.LauncherTypes.GROUP
launcher_id = session.chat_id
else:
launcher_type = provider_session.LauncherTypes.PERSON
launcher_id = session.user_id or ''
form_action_data = {
'form_token': form.get('form_token', ''),
'workflow_run_id': form.get('workflow_run_id', ''),
'action_id': clean_action_id,
'action_title': action_title,
'node_title': form.get('node_title', ''),
'user': f'{launcher_type.value}_{launcher_id}',
'inputs': {},
}
message_chain = platform_message.MessageChain([platform_message.Plain(text=f'[Form Action: {action_title}]')])
if launcher_type == provider_session.LauncherTypes.GROUP:
synthetic_event = platform_events.GroupMessage(
sender=platform_entities.GroupMember(
id=sender_user_id,
member_name='',
permission=platform_entities.Permission.Member,
group=platform_entities.Group(
id=launcher_id,
name='',
permission=platform_entities.Permission.Member,
),
special_title='',
),
message_chain=message_chain,
time=int(time.time()),
source_platform_object=None,
)
else:
synthetic_event = platform_events.FriendMessage(
sender=platform_entities.Friend(
id=sender_user_id,
nickname='',
remark='',
),
message_chain=message_chain,
time=int(time.time()),
source_platform_object=None,
)
if self.ap is None:
await self.logger.error('WeComBot: ap not injected; cannot enqueue button-click query')
return
bot_uuid = ''
pipeline_uuid = None
for bot in self.ap.platform_mgr.bots:
if bot.adapter is self:
bot_uuid = bot.bot_entity.uuid
pipeline_uuid = bot.bot_entity.use_pipeline_uuid
break
try:
await self.ap.query_pool.add_query(
bot_uuid=bot_uuid,
launcher_type=launcher_type,
launcher_id=launcher_id,
sender_id=sender_user_id,
message_event=synthetic_event,
message_chain=message_chain,
adapter=self,
pipeline_uuid=pipeline_uuid,
variables={
'_dify_form_action': form_action_data,
'_routed_by_rule': True,
},
)
await self.logger.info(f'WeComBot: button-click query enqueued action_id={clean_action_id!r}')
except Exception:
await self.logger.error(f'WeComBot: enqueue button-click query failed: {traceback.format_exc()}')
+213 -25
View File
@@ -18,6 +18,7 @@ from langbot_plugin.api.entities.builtin.pipeline.query import provider_session
from ..core import app from ..core import app
from . import handler from . import handler
from ..utils import platform from ..utils import platform
from ..utils.managed_runtime import ManagedRuntimeConnector
from langbot_plugin.runtime.io.controllers.stdio import ( from langbot_plugin.runtime.io.controllers.stdio import (
client as stdio_client_controller, client as stdio_client_controller,
) )
@@ -39,11 +40,9 @@ class PluginRuntimeNotConnectedError(RuntimeError):
"""Raised when plugin runtime operations are requested before connection.""" """Raised when plugin runtime operations are requested before connection."""
class PluginRuntimeConnector: class PluginRuntimeConnector(ManagedRuntimeConnector):
"""Plugin runtime connector""" """Plugin runtime connector"""
ap: app.Application
handler: handler.RuntimeConnectionHandler handler: handler.RuntimeConnectionHandler
handler_task: asyncio.Task handler_task: asyncio.Task
@@ -54,10 +53,6 @@ class PluginRuntimeConnector:
ctrl: stdio_client_controller.StdioClientController | ws_client_controller.WebSocketClientController ctrl: stdio_client_controller.StdioClientController | ws_client_controller.WebSocketClientController
runtime_subprocess_on_windows: asyncio.subprocess.Process | None = None
runtime_subprocess_on_windows_task: asyncio.Task | None = None
runtime_disconnect_callback: typing.Callable[ runtime_disconnect_callback: typing.Callable[
[PluginRuntimeConnector], typing.Coroutine[typing.Any, typing.Any, None] [PluginRuntimeConnector], typing.Coroutine[typing.Any, typing.Any, None]
] ]
@@ -72,7 +67,7 @@ class PluginRuntimeConnector:
[PluginRuntimeConnector], typing.Coroutine[typing.Any, typing.Any, None] [PluginRuntimeConnector], typing.Coroutine[typing.Any, typing.Any, None]
], ],
): ):
self.ap = ap super().__init__(ap)
self.runtime_disconnect_callback = runtime_disconnect_callback self.runtime_disconnect_callback = runtime_disconnect_callback
self.is_enable_plugin = self.ap.instance_config.data.get('plugin', {}).get('enable', True) self.is_enable_plugin = self.ap.instance_config.data.get('plugin', {}).get('enable', True)
@@ -108,6 +103,16 @@ class PluginRuntimeConnector:
self.handler_task = asyncio.create_task(self.handler.run()) self.handler_task = asyncio.create_task(self.handler.run())
_ = await self.handler.ping() _ = await self.handler.ping()
# Push the configured marketplace (Space) URL to the runtime so it
# downloads plugins from the same Space LangBot is bound to, rather
# than relying on the runtime's own env/default.
space_url = self.ap.instance_config.data.get('space', {}).get('url', '').rstrip('/')
if space_url:
try:
await self.handler.set_runtime_config(cloud_service_url=space_url)
self.ap.logger.info(f'Pushed marketplace URL to plugin runtime: {space_url}')
except Exception as e:
self.ap.logger.warning(f'Failed to push runtime config: {e}')
self.ap.logger.info('Connected to plugin runtime.') self.ap.logger.info('Connected to plugin runtime.')
await self.handler_task await self.handler_task
@@ -140,19 +145,7 @@ class PluginRuntimeConnector:
# We have to launch runtime via cmd but communicate via ws. # We have to launch runtime via cmd but communicate via ws.
self.ap.logger.info('(windows) use cmd to launch plugin runtime and communicate via ws') self.ap.logger.info('(windows) use cmd to launch plugin runtime and communicate via ws')
if self.runtime_subprocess_on_windows is None: # only launch once await self._start_runtime_subprocess('-m', 'langbot_plugin.cli.__init__', 'rt')
python_path = sys.executable
env = os.environ.copy()
self.runtime_subprocess_on_windows = await asyncio.create_subprocess_exec(
python_path,
'-m',
'langbot_plugin.cli.__init__',
'rt',
env=env,
)
# hold the process
self.runtime_subprocess_on_windows_task = asyncio.create_task(self.runtime_subprocess_on_windows.wait())
ws_url = 'ws://localhost:5400/control/ws' ws_url = 'ws://localhost:5400/control/ws'
@@ -236,6 +229,85 @@ class PluginRuntimeConnector:
return plugin_author, plugin_name return plugin_author, plugin_name
async def _install_mcp_from_marketplace(
self,
mcp_data: dict[str, Any],
task_context: taskmgr.TaskContext | None = None,
):
"""Install an MCP server from marketplace data.
Marketplace MCP records carry the runtime-ready ``mode`` and
``extra_args`` directly (the same shape LangBot stores in
``mcp_servers``), so they are used as-is rather than reconstructed.
For ``stdio`` this preserves ``command``/``args``/``env``/``box``;
for ``http``/``sse`` it preserves ``url``/``headers``/``timeout``/
``ssereadtimeout``.
"""
from ..entity.persistence import mcp as persistence_mcp
import uuid
mode = mcp_data.get('mode') or 'stdio'
extra_args = mcp_data.get('extra_args') or {}
# Marketplace records carry the rendered README markdown; persist it so
# the detail page Docs tab works offline and without a marketplace round-trip.
readme = mcp_data.get('readme') or ''
# Use __ instead of / to avoid URL routing issues with slashes
name = f'{mcp_data.get("author", "")}__{mcp_data.get("name", "")}'
# Check if MCP server already exists
existing = await self.ap.persistence_mgr.execute_async(
sqlalchemy.select(persistence_mcp.MCPServer).where(persistence_mcp.MCPServer.name == name)
)
if existing.scalar_one_or_none():
self.ap.logger.info(f'MCP server {name} already exists, skipping installation')
return
# Create MCP server record
server_uuid = str(uuid.uuid4())
server_data = {
'uuid': server_uuid,
'name': name,
'enable': True,
'mode': mode,
'extra_args': extra_args,
'readme': readme,
}
await self.ap.persistence_mgr.execute_async(sqlalchemy.insert(persistence_mcp.MCPServer).values(server_data))
# Start the MCP server
result = await self.ap.persistence_mgr.execute_async(
sqlalchemy.select(persistence_mcp.MCPServer).where(persistence_mcp.MCPServer.uuid == server_uuid)
)
server_entity = result.first()
if server_entity:
server_config = self.ap.persistence_mgr.serialize_model(persistence_mcp.MCPServer, server_entity)
if self.ap.tool_mgr.mcp_tool_loader:
mcp_task = asyncio.create_task(self.ap.tool_mgr.mcp_tool_loader.host_mcp_server(server_config))
self.ap.tool_mgr.mcp_tool_loader._hosted_mcp_tasks.append(mcp_task)
self.ap.logger.info(f'Installed MCP server {name} from marketplace')
async def _install_skill_from_zip(
self,
file_bytes: bytes,
filename: str,
task_context: taskmgr.TaskContext | None = None,
):
"""Install a skill from marketplace ZIP data."""
from ..api.http.service.skill import SkillService
skill_service = SkillService(self.ap)
self.ap.logger.info(f'Installing skill from marketplace ZIP ({len(file_bytes)} bytes)')
# Install from ZIP using skill service
result = await skill_service.install_from_zip_upload(
file_bytes=file_bytes,
filename=filename + '.zip',
)
self.ap.logger.info(f'Skill installed successfully: {result}')
def _build_plugin_startup_failure_message( def _build_plugin_startup_failure_message(
self, self,
plugin_author: str, plugin_author: str,
@@ -298,6 +370,117 @@ class PluginRuntimeConnector:
plugin_author = install_info.get('plugin_author') plugin_author = install_info.get('plugin_author')
plugin_name = install_info.get('plugin_name') plugin_name = install_info.get('plugin_name')
if install_source == PluginInstallSource.MARKETPLACE:
# Handle marketplace plugin/mcp/skill installation
plugin_author = install_info.get('plugin_author', '')
plugin_name = install_info.get('plugin_name', '')
space_url = (
self.ap.instance_config.data.get('space', {}).get('url', 'https://space.langbot.app').rstrip('/')
)
# Try MCP endpoint first
async with httpx.AsyncClient(trust_env=True, timeout=15) as client:
mcp_resp = await client.get(f'{space_url}/api/v1/marketplace/mcps/{plugin_author}/{plugin_name}')
if mcp_resp.status_code == 200:
mcp_data = mcp_resp.json().get('data', {}).get('mcp', {})
if mcp_data.get('mode'):
# It's an MCP - create server locally
self.ap.logger.info(f'Installing MCP from marketplace: {plugin_author}/{plugin_name}')
if task_context:
task_context.set_current_action('installing mcp server')
await self._install_mcp_from_marketplace(mcp_data, task_context)
# Best-effort install report (bumps marketplace install_count).
try:
await client.post(
f'{space_url}/api/v1/marketplace/mcps/{plugin_author}/{plugin_name}/install'
)
except Exception as report_err:
self.ap.logger.debug(f'Failed to report MCP install: {report_err}')
return
else:
raise Exception(f'MCP {plugin_author}/{plugin_name} has no mode')
elif mcp_resp.status_code == 404:
# Try skill endpoint - download ZIP and install
self.ap.logger.info(f'Trying skill endpoint for: {plugin_author}/{plugin_name}')
if task_context:
task_context.set_current_action('checking skill marketplace')
# Get skill detail to find version
skill_resp = await client.get(
f'{space_url}/api/v1/marketplace/skills/{plugin_author}/{plugin_name}'
)
if skill_resp.status_code == 200:
self.ap.logger.info(f'Installing skill from marketplace: {plugin_author}/{plugin_name}')
if task_context:
task_context.set_current_action('installing skill from marketplace')
# Download the skill ZIP (no version needed - uses latest)
if task_context:
task_context.set_current_action('downloading skill package')
download_resp = await client.get(
f'{space_url}/api/v1/marketplace/skills/download/{plugin_author}/{plugin_name}'
)
if download_resp.status_code != 200:
raise Exception(
f'Failed to download skill {plugin_author}/{plugin_name}: {download_resp.status_code}'
)
file_bytes = download_resp.content
file_size = len(file_bytes)
self.ap.logger.info(f'Downloaded skill ZIP ({file_size} bytes)')
# Install skill from ZIP using skill service
await self._install_skill_from_zip(file_bytes, f'{plugin_author}-{plugin_name}', task_context)
return
elif skill_resp.status_code == 404:
# Try plugin endpoint - get versions and download
self.ap.logger.info(f'Trying plugin endpoint for: {plugin_author}/{plugin_name}')
if task_context:
task_context.set_current_action('checking plugin marketplace')
# Get plugin versions to find latest
versions_resp = await client.get(
f'{space_url}/api/v1/marketplace/plugins/{plugin_author}/{plugin_name}/versions'
)
if versions_resp.status_code == 200:
versions_data = versions_resp.json().get('data', {}).get('versions', [])
if versions_data:
latest_version = versions_data[0].get('version', '')
if latest_version:
self.ap.logger.info(
f'Installing plugin from marketplace: {plugin_author}/{plugin_name} v{latest_version}'
)
if task_context:
task_context.set_current_action('downloading plugin package')
download_resp = await client.get(
f'{space_url}/api/v1/marketplace/plugins/download/{plugin_author}/{plugin_name}/{latest_version}'
)
if download_resp.status_code != 200:
raise Exception(
f'Failed to download plugin {plugin_author}/{plugin_name}: {download_resp.status_code}'
)
file_bytes = download_resp.content
self._inspect_plugin_package(file_bytes, task_context)
file_key = await self.handler.send_file(file_bytes, 'lbpkg')
install_info['plugin_file_key'] = file_key
self.ap.logger.info(f'Transfered file {file_key} to plugin runtime')
# Continue to install via runtime
else:
raise Exception(f'No version found for plugin {plugin_author}/{plugin_name}')
else:
raise Exception(f'Plugin {plugin_author}/{plugin_name} has no versions')
else:
raise Exception(f'Plugin {plugin_author}/{plugin_name} not found in marketplace')
else:
skill_resp.raise_for_status()
raise Exception(f'Failed to get skill {plugin_author}/{plugin_name}')
else:
mcp_resp.raise_for_status()
raise Exception(f'Failed to get MCP {plugin_author}/{plugin_name}')
if install_source == PluginInstallSource.LOCAL: if install_source == PluginInstallSource.LOCAL:
# transfer file before install # transfer file before install
file_bytes = install_info['plugin_file'] file_bytes = install_info['plugin_file']
@@ -613,13 +796,18 @@ class PluginRuntimeConnector:
return await self.handler.retrieve_knowledge(plugin_author, plugin_name, retriever_name, retrieval_context) return await self.handler.retrieve_knowledge(plugin_author, plugin_name, retriever_name, retrieval_context)
def dispose(self): def dispose(self):
# No need to consider the shutdown on Windows # On non-Windows stdio mode, terminate via the controller's process handle.
# for Windows can kill processes and subprocesses chainly # On Windows, the managed subprocess is cleaned up by the base class.
if (
if self.is_enable_plugin and isinstance(self.ctrl, stdio_client_controller.StdioClientController): self.is_enable_plugin
and hasattr(self, 'ctrl')
and isinstance(self.ctrl, stdio_client_controller.StdioClientController)
):
self.ap.logger.info('Terminating plugin runtime process...') self.ap.logger.info('Terminating plugin runtime process...')
self.ctrl.process.terminate() self.ctrl.process.terminate()
self._dispose_subprocess()
if self.heartbeat_task is not None: if self.heartbeat_task is not None:
self.heartbeat_task.cancel() self.heartbeat_task.cancel()
self.heartbeat_task = None self.heartbeat_task = None
+10
View File
@@ -779,6 +779,16 @@ class RuntimeConnectionHandler(handler.Handler):
timeout=10, timeout=10,
) )
async def set_runtime_config(self, cloud_service_url: str) -> dict[str, Any]:
"""Push runtime configuration (e.g. marketplace URL) to the runtime."""
return await self.call_action(
LangBotToRuntimeAction.SET_RUNTIME_CONFIG,
{
'cloud_service_url': cloud_service_url,
},
timeout=10,
)
async def install_plugin( async def install_plugin(
self, install_source: str, install_info: dict[str, Any] self, install_source: str, install_info: dict[str, Any]
) -> typing.AsyncGenerator[dict[str, Any], None]: ) -> typing.AsyncGenerator[dict[str, Any], None]:
+65 -31
View File
@@ -143,49 +143,83 @@ class ModelManager:
# get the latest models from space # get the latest models from space
space_models = await self.ap.space_service.get_models() space_models = await self.ap.space_service.get_models()
exists_llm_models_uuids = [m['uuid'] for m in await self.ap.llm_model_service.get_llm_models()] # Index existing models by uuid. Space reuses a model's uuid across
exists_embedding_models_uuids = [ # renames / re-specs (e.g. the uuid that used to be ``claude-opus-4-6``
m['uuid'] for m in await self.ap.embedding_models_service.get_embedding_models() # may later become ``claude-opus-4-7``). So for Space-managed models we
] # upsert: create when the uuid is new, otherwise update name/abilities/
# ranking to track Space. Models owned by other providers are never
# touched, even on an (unexpected) uuid collision.
existing_llm_models = {m['uuid']: m for m in await self.ap.llm_model_service.get_llm_models()}
existing_embedding_models = {
m['uuid']: m for m in await self.ap.embedding_models_service.get_embedding_models()
}
created = 0
updated = 0
for space_model in space_models: for space_model in space_models:
if space_model.category == 'chat': if space_model.category == 'chat':
uuid = space_model.uuid existing = existing_llm_models.get(space_model.uuid)
if existing is None:
if uuid in exists_llm_models_uuids: # model will be automatically loaded
continue await self.ap.llm_model_service.create_llm_model(
{
# model will be automatically loaded 'uuid': space_model.uuid,
await self.ap.llm_model_service.create_llm_model( 'name': space_model.model_id,
{ 'provider_uuid': space_model_provider.uuid,
'uuid': space_model.uuid, 'abilities': space_model.llm_abilities or [],
'extra_args': {},
'prefered_ranking': space_model.featured_order,
},
preserve_uuid=True,
auto_set_to_default_pipeline=False,
)
created += 1
elif existing.get('provider_uuid') == space_model_provider.uuid:
desired = {
'name': space_model.model_id, 'name': space_model.model_id,
'provider_uuid': space_model_provider.uuid, 'provider_uuid': space_model_provider.uuid,
'abilities': space_model.llm_abilities or [], 'abilities': space_model.llm_abilities or [],
'extra_args': {},
'prefered_ranking': space_model.featured_order, 'prefered_ranking': space_model.featured_order,
}, }
preserve_uuid=True, if (
auto_set_to_default_pipeline=False, existing.get('name') != desired['name']
) or list(existing.get('abilities') or []) != list(desired['abilities'])
or existing.get('prefered_ranking') != desired['prefered_ranking']
):
await self.ap.llm_model_service.update_llm_model(space_model.uuid, dict(desired))
updated += 1
elif space_model.category == 'embedding': elif space_model.category == 'embedding':
uuid = space_model.uuid existing = existing_embedding_models.get(space_model.uuid)
if existing is None:
if uuid in exists_embedding_models_uuids: # model will be automatically loaded
continue await self.ap.embedding_models_service.create_embedding_model(
{
# model will be automatically loaded 'uuid': space_model.uuid,
await self.ap.embedding_models_service.create_embedding_model( 'name': space_model.model_id,
{ 'provider_uuid': space_model_provider.uuid,
'uuid': space_model.uuid, 'extra_args': {},
'prefered_ranking': space_model.featured_order,
},
preserve_uuid=True,
)
created += 1
elif existing.get('provider_uuid') == space_model_provider.uuid:
desired = {
'name': space_model.model_id, 'name': space_model.model_id,
'provider_uuid': space_model_provider.uuid, 'provider_uuid': space_model_provider.uuid,
'extra_args': {},
'prefered_ranking': space_model.featured_order, 'prefered_ranking': space_model.featured_order,
}, }
preserve_uuid=True, if (
) existing.get('name') != desired['name']
or existing.get('prefered_ranking') != desired['prefered_ranking']
):
await self.ap.embedding_models_service.update_embedding_model(space_model.uuid, dict(desired))
updated += 1
if created or updated:
self.ap.logger.info(f'Synced models from LangBot Space: {created} added, {updated} updated.')
async def init_temporary_runtime_llm_model( async def init_temporary_runtime_llm_model(
self, self,
+7 -3
View File
@@ -2,8 +2,12 @@ from __future__ import annotations
import abc import abc
import typing import typing
from typing import TYPE_CHECKING
from ..core import app if TYPE_CHECKING:
from ..core import app
import langbot_plugin.api.entities.builtin.pipeline.query as pipeline_query
import langbot_plugin.api.entities.builtin.provider.message as provider_message
preregistered_runners: list[typing.Type[RequestRunner]] = [] preregistered_runners: list[typing.Type[RequestRunner]] = []
@@ -35,7 +39,7 @@ class RequestRunner(abc.ABC):
@abc.abstractmethod @abc.abstractmethod
async def run( async def run(
self, query: core_entities.Query self, query: pipeline_query.Query
) -> typing.AsyncGenerator[llm_entities.Message | llm_entities.MessageChunk, None]: ) -> typing.AsyncGenerator[provider_message.Message | provider_message.MessageChunk, None]:
"""运行请求""" """运行请求"""
pass pass
@@ -0,0 +1,511 @@
"""DeerFlow LangGraph API Runner
参考 astrbot deerflow_agent_runner 实现适配 LangBot Runner 接口
特点
- 使用 LangGraph HTTP API 接入 deer-flow 后端
- 自动管理 thread_id session 隔离
- 支持 SSE 流式响应解析
- 支持 streaming/非流式两种输出
- 处理 values / messages-tuple / custom 三种事件
"""
from __future__ import annotations
import asyncio
import hashlib
import json
import typing
from collections import deque
from dataclasses import dataclass, field
from langbot.pkg.provider import runner
from langbot.pkg.core import app
import langbot_plugin.api.entities.builtin.provider.message as provider_message
import langbot_plugin.api.entities.builtin.pipeline.query as pipeline_query
from langbot.libs.deerflow_api import client, errors, stream_utils
_MAX_VALUES_HISTORY = 200
@dataclass
class _StreamState:
"""流式状态跟踪"""
latest_text: str = ''
prev_text_for_streaming: str = ''
clarification_text: str = ''
task_failures: list[str] = field(default_factory=list)
seen_message_ids: set[str] = field(default_factory=set)
seen_message_order: deque[str] = field(default_factory=deque)
no_id_message_fingerprints: dict[int, str] = field(default_factory=dict)
baseline_initialized: bool = False
has_values_text: bool = False
run_values_messages: list[dict[str, typing.Any]] = field(default_factory=list)
timed_out: bool = False
@runner.runner_class('deerflow-api')
class DeerFlowAPIRunner(runner.RequestRunner):
"""DeerFlow LangGraph API 对话请求器"""
deerflow_client: client.AsyncDeerFlowClient
def __init__(self, ap: app.Application, pipeline_config: dict):
super().__init__(ap, pipeline_config)
cfg = self.pipeline_config['ai']['deerflow-api']
api_base = cfg.get('api-base', '').strip()
if not api_base or not api_base.startswith(('http://', 'https://')):
raise errors.DeerFlowAPIError(
message='DeerFlow API Base URL 格式错误,必须以 http:// 或 https:// 开头',
)
self.api_base = api_base
self.api_key = cfg.get('api-key', '')
self.auth_header = cfg.get('auth-header', '')
self.assistant_id = cfg.get('assistant-id', 'lead_agent')
self.model_name = cfg.get('model-name', '')
self.thinking_enabled = bool(cfg.get('thinking-enabled', False))
self.plan_mode = bool(cfg.get('plan-mode', False))
self.subagent_enabled = bool(cfg.get('subagent-enabled', False))
self.max_concurrent_subagents = int(cfg.get('max-concurrent-subagents', 3))
self.timeout = int(cfg.get('timeout', 300))
self.recursion_limit = int(cfg.get('recursion-limit', 1000))
self.deerflow_client = client.AsyncDeerFlowClient(
api_base=self.api_base,
api_key=self.api_key,
auth_header=self.auth_header,
)
# ------------------------------------------------------------------
# 辅助方法
# ------------------------------------------------------------------
def _fingerprint_message(self, message: dict[str, typing.Any]) -> str:
try:
raw = json.dumps(message, sort_keys=True, ensure_ascii=False, default=str)
except (TypeError, ValueError):
raw = repr(message)
return hashlib.sha1(raw.encode('utf-8', errors='ignore')).hexdigest()
def _remember_seen_message_id(self, state: _StreamState, msg_id: str) -> None:
if not msg_id or msg_id in state.seen_message_ids:
return
state.seen_message_ids.add(msg_id)
state.seen_message_order.append(msg_id)
while len(state.seen_message_order) > _MAX_VALUES_HISTORY:
dropped = state.seen_message_order.popleft()
state.seen_message_ids.discard(dropped)
def _extract_new_messages_from_values(
self,
values_messages: list[typing.Any],
state: _StreamState,
) -> list[dict[str, typing.Any]]:
new_messages: list[dict[str, typing.Any]] = []
no_id_indexes_seen: set[int] = set()
for idx, msg in enumerate(values_messages):
if not isinstance(msg, dict):
continue
msg_id = stream_utils.get_message_id(msg)
if msg_id:
if msg_id in state.seen_message_ids:
continue
self._remember_seen_message_id(state, msg_id)
new_messages.append(msg)
continue
no_id_indexes_seen.add(idx)
fp = self._fingerprint_message(msg)
if state.no_id_message_fingerprints.get(idx) == fp:
continue
state.no_id_message_fingerprints[idx] = fp
new_messages.append(msg)
for idx in list(state.no_id_message_fingerprints.keys()):
if idx not in no_id_indexes_seen:
state.no_id_message_fingerprints.pop(idx, None)
return new_messages
# ------------------------------------------------------------------
# 用户输入处理
# ------------------------------------------------------------------
def _build_user_content(
self,
prompt: str,
image_urls: list[str],
) -> typing.Any:
"""构建 LangGraph 兼容的 user content(支持多模态)"""
if not image_urls:
return prompt
content: list[dict[str, typing.Any]] = []
if prompt:
content.append({'type': 'text', 'text': prompt})
for url in image_urls:
if not isinstance(url, str):
continue
url = url.strip()
if not url:
continue
if url.startswith(('http://', 'https://', 'data:')):
content.append({'type': 'image_url', 'image_url': {'url': url}})
return content if content else prompt
def _preprocess_user_message(
self,
query: pipeline_query.Query,
) -> tuple[str, list[str]]:
"""提取用户消息的纯文本与图片 URL 列表"""
plain_text = ''
image_urls: list[str] = []
if isinstance(query.user_message.content, str):
plain_text = query.user_message.content
elif isinstance(query.user_message.content, list):
for ce in query.user_message.content:
if ce.type == 'text':
plain_text += ce.text
elif ce.type == 'image_base64':
# 转换为 data URI 形式
b64 = getattr(ce, 'image_base64', '')
if b64:
if not b64.startswith('data:'):
b64 = f'data:image/png;base64,{b64}'
image_urls.append(b64)
elif ce.type == 'image_url':
url = getattr(ce, 'image_url', '')
if url:
image_urls.append(url)
return plain_text, image_urls
# ------------------------------------------------------------------
# 请求构造
# ------------------------------------------------------------------
def _build_messages(
self,
prompt: str,
image_urls: list[str],
system_prompt: str = '',
) -> list[dict[str, typing.Any]]:
messages: list[dict[str, typing.Any]] = []
if system_prompt:
messages.append({'role': 'system', 'content': system_prompt})
messages.append(
{
'role': 'user',
'content': self._build_user_content(prompt, image_urls),
}
)
return messages
def _build_runtime_configurable(self, thread_id: str) -> dict[str, typing.Any]:
cfg: dict[str, typing.Any] = {
'thread_id': thread_id,
'thinking_enabled': self.thinking_enabled,
'is_plan_mode': self.plan_mode,
'subagent_enabled': self.subagent_enabled,
}
if self.subagent_enabled:
cfg['max_concurrent_subagents'] = self.max_concurrent_subagents
if self.model_name:
cfg['model_name'] = self.model_name
return cfg
def _build_payload(
self,
thread_id: str,
prompt: str,
image_urls: list[str],
system_prompt: str = '',
) -> dict[str, typing.Any]:
runtime_configurable = self._build_runtime_configurable(thread_id)
return {
'assistant_id': self.assistant_id,
'input': {
'messages': self._build_messages(prompt, image_urls, system_prompt),
},
'stream_mode': ['values', 'messages-tuple', 'custom'],
# DeerFlow 2.0 从 config.configurable 读取运行时覆盖
# 同时保留 context 字段做向后兼容
'context': dict(runtime_configurable),
'config': {
'recursion_limit': self.recursion_limit,
'configurable': runtime_configurable,
},
}
# ------------------------------------------------------------------
# Session/Thread 管理
# ------------------------------------------------------------------
async def _ensure_thread_id(self, query: pipeline_query.Query) -> str:
"""从 query.session 取/创建 deerflow thread_id
LangBot 使用 `query.session.using_conversation.uuid` 持久化 conversation id
我们复用这个字段存储 deerflow thread_id Dify Runner 同样做法
"""
thread_id = query.session.using_conversation.uuid or ''
if thread_id:
return thread_id
thread = await self.deerflow_client.create_thread(timeout=min(30, self.timeout))
thread_id = thread.get('thread_id', '')
if not thread_id:
raise errors.DeerFlowAPIError(message=f'DeerFlow create thread 返回数据缺少 thread_id: {thread}')
query.session.using_conversation.uuid = thread_id
return thread_id
# ------------------------------------------------------------------
# 流式事件处理
# ------------------------------------------------------------------
def _handle_values_event(
self,
data: typing.Any,
state: _StreamState,
) -> str | None:
"""处理 values 事件,返回新的完整文本(增量基础上的全量)"""
values_messages = stream_utils.extract_messages_from_values_data(data)
if not values_messages:
return None
new_messages: list[dict[str, typing.Any]] = []
if not state.baseline_initialized:
state.baseline_initialized = True
for idx, msg in enumerate(values_messages):
if not isinstance(msg, dict):
continue
new_messages.append(msg)
msg_id = stream_utils.get_message_id(msg)
if msg_id:
self._remember_seen_message_id(state, msg_id)
continue
state.no_id_message_fingerprints[idx] = self._fingerprint_message(msg)
else:
new_messages = self._extract_new_messages_from_values(values_messages, state)
latest_text = ''
if new_messages:
state.run_values_messages.extend(new_messages)
if len(state.run_values_messages) > _MAX_VALUES_HISTORY:
state.run_values_messages = state.run_values_messages[-_MAX_VALUES_HISTORY:]
latest_text = stream_utils.extract_latest_ai_text(state.run_values_messages)
if latest_text:
state.has_values_text = True
latest_clarification = stream_utils.extract_latest_clarification_text(
state.run_values_messages,
)
if latest_clarification:
state.clarification_text = latest_clarification
return latest_text or None
def _handle_message_event(
self,
data: typing.Any,
state: _StreamState,
) -> str | None:
"""处理 messages-tuple 事件,返回增量文本
values 事件已经提供完整文本时跳过 messages-tuple 的增量
"""
delta = stream_utils.extract_ai_delta_from_event_data(data)
if delta and not state.has_values_text:
state.latest_text += delta
return delta
maybe_clar = stream_utils.extract_clarification_from_event_data(data)
if maybe_clar:
state.clarification_text = maybe_clar
return None
def _build_final_text(self, state: _StreamState) -> str:
"""构建最终输出文本"""
if state.clarification_text:
return state.clarification_text
# 优先使用最后一条 AI message 的文本
latest_ai = stream_utils.extract_latest_ai_message(state.run_values_messages)
if latest_ai:
text = stream_utils.extract_text(latest_ai.get('content'))
if text:
if state.timed_out:
text += f'\n\nDeerFlow stream 在 {self.timeout}s 后超时,返回部分结果。'
return text
if state.latest_text:
text = state.latest_text
if state.timed_out:
text += f'\n\nDeerFlow stream 在 {self.timeout}s 后超时,返回部分结果。'
return text
# 提取任务失败信息作兜底
failure_text = stream_utils.build_task_failure_summary(state.task_failures)
if failure_text:
return failure_text
return 'DeerFlow 返回空响应'
# ------------------------------------------------------------------
# 主流程
# ------------------------------------------------------------------
async def _stream_messages_chunk(
self,
query: pipeline_query.Query,
) -> typing.AsyncGenerator[provider_message.MessageChunk, None]:
"""流式输出生成器"""
plain_text, image_urls = self._preprocess_user_message(query)
system_prompt = ''
# LangBot 的 pipeline 通常通过 prompt-preprocess 已注入 system prompt
# 这里保持空,让 prompt-preprocess 的内容作为 user message 一并送给 deerflow
thread_id = await self._ensure_thread_id(query)
payload = self._build_payload(
thread_id=thread_id,
prompt=plain_text or 'continue',
image_urls=image_urls,
system_prompt=system_prompt,
)
state = _StreamState()
prev_text = ''
message_idx = 0
try:
async for event in self.deerflow_client.stream_run(
thread_id=thread_id,
payload=payload,
timeout=self.timeout,
):
event_type = event.get('event')
data = event.get('data')
if event_type == 'values':
new_full = self._handle_values_event(data, state)
if new_full and new_full != prev_text:
delta = new_full[len(prev_text) :] if new_full.startswith(prev_text) else new_full
prev_text = new_full
if delta:
message_idx += 1
yield provider_message.MessageChunk(
role='assistant',
content=new_full,
is_final=False,
)
continue
if event_type in {'messages-tuple', 'messages', 'message'}:
delta = self._handle_message_event(data, state)
if delta:
prev_text = state.latest_text
message_idx += 1
yield provider_message.MessageChunk(
role='assistant',
content=prev_text,
is_final=False,
)
continue
if event_type == 'custom':
state.task_failures.extend(
stream_utils.extract_task_failures_from_custom_event(data),
)
continue
if event_type == 'error':
raise errors.DeerFlowAPIError(message=f'DeerFlow stream error event: {data}')
if event_type == 'end':
break
except (asyncio.TimeoutError, TimeoutError):
self.ap.logger.warning(f'DeerFlow stream timed out after {self.timeout}s for thread_id={thread_id}')
state.timed_out = True
# 最终消息
final_text = self._build_final_text(state)
yield provider_message.MessageChunk(
role='assistant',
content=final_text,
is_final=True,
)
async def _messages(
self,
query: pipeline_query.Query,
) -> typing.AsyncGenerator[provider_message.Message, None]:
"""非流式聚合输出"""
plain_text, image_urls = self._preprocess_user_message(query)
thread_id = await self._ensure_thread_id(query)
payload = self._build_payload(
thread_id=thread_id,
prompt=plain_text or 'continue',
image_urls=image_urls,
)
state = _StreamState()
try:
async for event in self.deerflow_client.stream_run(
thread_id=thread_id,
payload=payload,
timeout=self.timeout,
):
event_type = event.get('event')
data = event.get('data')
if event_type == 'values':
self._handle_values_event(data, state)
continue
if event_type in {'messages-tuple', 'messages', 'message'}:
self._handle_message_event(data, state)
continue
if event_type == 'custom':
state.task_failures.extend(
stream_utils.extract_task_failures_from_custom_event(data),
)
continue
if event_type == 'error':
raise errors.DeerFlowAPIError(message=f'DeerFlow stream error event: {data}')
if event_type == 'end':
break
except (asyncio.TimeoutError, TimeoutError):
self.ap.logger.warning(f'DeerFlow stream timed out after {self.timeout}s for thread_id={thread_id}')
state.timed_out = True
final_text = self._build_final_text(state)
yield provider_message.Message(
role='assistant',
content=final_text,
)
async def run(
self,
query: pipeline_query.Query,
) -> typing.AsyncGenerator[provider_message.Message, None]:
"""主入口:根据 adapter 是否支持流式输出,选择流式或非流式"""
if await query.adapter.is_stream_output_supported():
msg_idx = 0
async for msg in self._stream_messages_chunk(query):
msg_idx += 1
msg.msg_sequence = msg_idx
yield msg
else:
async for msg in self._messages(query):
yield msg
+13 -744
View File
@@ -2,11 +2,9 @@ from __future__ import annotations
import typing import typing
import json import json
import time
import uuid import uuid
import base64 import base64
import mimetypes import mimetypes
from collections import OrderedDict
from langbot.pkg.provider import runner from langbot.pkg.provider import runner
@@ -18,125 +16,6 @@ from langbot.libs.dify_service_api.v1 import client, errors
import httpx import httpx
# Module-level store for paused-workflow form state, keyed by session key
# (launcher_type_value + "_" + launcher_id). Each session holds an
# insertion-ordered dict of form_token -> form_data, allowing multiple
# Dify workflows to be paused simultaneously for the same session.
_PENDING_FORMS: dict[str, 'OrderedDict[str, dict[str, typing.Any]]'] = {}
_PENDING_FORM_DEFAULT_TTL = 30 * 60 # 30 minutes safety cap
def _session_key_from_query(query: pipeline_query.Query) -> str:
return f'{query.session.launcher_type.value}_{query.session.launcher_id}'
def _prune_pending_forms(now: float | None = None) -> None:
if now is None:
now = time.time()
for session_key in list(_PENDING_FORMS.keys()):
forms = _PENDING_FORMS[session_key]
expired_tokens = [token for token, data in forms.items() if data.get('_expires_at', 0) <= now]
for token in expired_tokens:
forms.pop(token, None)
if not forms:
_PENDING_FORMS.pop(session_key, None)
def _set_pending_form(session_key: str, form_data: dict[str, typing.Any]) -> None:
_prune_pending_forms()
stored = dict(form_data)
expiration_time = stored.get('expiration_time')
try:
expiration_ts = float(expiration_time) if expiration_time is not None else 0.0
except (TypeError, ValueError):
expiration_ts = 0.0
stored['_expires_at'] = expiration_ts or (time.time() + _PENDING_FORM_DEFAULT_TTL)
form_token = str(stored.get('form_token') or '')
forms = _PENDING_FORMS.setdefault(session_key, OrderedDict())
# Re-insert at the end so this becomes the "latest" entry
forms.pop(form_token, None)
forms[form_token] = stored
def _get_pending_form_by_token(session_key: str, form_token: str) -> dict[str, typing.Any] | None:
_prune_pending_forms()
forms = _PENDING_FORMS.get(session_key)
if not forms or not form_token:
return None
return forms.get(form_token)
def _get_pending_form_by_w_suffix(session_key: str, w_suffix: str) -> dict[str, typing.Any] | None:
"""Look up a pending form whose workflow_run_id ends with the given suffix.
Used by adapters (e.g. Telegram) whose callback payload is too small to
carry the full form_token / workflow_run_id.
"""
_prune_pending_forms()
forms = _PENDING_FORMS.get(session_key)
if not forms or not w_suffix:
return None
for token in reversed(forms):
form = forms[token]
if str(form.get('workflow_run_id', '')).endswith(w_suffix):
return form
return None
def _get_latest_pending_form(session_key: str) -> dict[str, typing.Any] | None:
_prune_pending_forms()
forms = _PENDING_FORMS.get(session_key)
if not forms:
return None
return forms[next(reversed(forms))]
def _iter_pending_forms(session_key: str) -> typing.Iterator[dict[str, typing.Any]]:
"""Iterate pending forms for a session, newest-first."""
_prune_pending_forms()
forms = _PENDING_FORMS.get(session_key)
if not forms:
return
for token in reversed(list(forms.keys())):
yield forms[token]
def _clear_pending_form(session_key: str, form_token: str | None = None) -> None:
"""Clear one specific pending form (by token) or all forms for the session."""
forms = _PENDING_FORMS.get(session_key)
if not forms:
return
if form_token is None:
_PENDING_FORMS.pop(session_key, None)
return
forms.pop(form_token, None)
if not forms:
_PENDING_FORMS.pop(session_key, None)
def _format_human_input_text(
node_title: str,
form_content: str,
actions: list[dict[str, typing.Any]],
) -> str:
"""Render a paused-workflow human-input prompt as plain text.
Used by adapters without rich UI (no buttons/cards) so users can reply
with the option number or the option title to resume the workflow.
"""
lines: list[str] = [f'[Human Input Required] {node_title or ""}'.rstrip()]
if form_content:
lines.append('')
lines.append(form_content)
if actions:
lines.append('')
lines.append('Reply with the number or title to continue:')
for idx, action in enumerate(actions, start=1):
title = action.get('title') or action.get('id') or ''
lines.append(f' {idx}. {title}')
return '\n'.join(lines)
@runner.runner_class('dify-service-api') @runner.runner_class('dify-service-api')
class DifyServiceAPIRunner(runner.RequestRunner): class DifyServiceAPIRunner(runner.RequestRunner):
"""Dify Service API 对话请求器""" """Dify Service API 对话请求器"""
@@ -147,7 +26,7 @@ class DifyServiceAPIRunner(runner.RequestRunner):
self.ap = ap self.ap = ap
self.pipeline_config = pipeline_config self.pipeline_config = pipeline_config
valid_app_types = ['chat', 'agent', 'workflow', 'chatflow'] valid_app_types = ['chat', 'agent', 'workflow']
if self.pipeline_config['ai']['dify-service-api']['app-type'] not in valid_app_types: if self.pipeline_config['ai']['dify-service-api']['app-type'] not in valid_app_types:
raise errors.DifyAPIError( raise errors.DifyAPIError(
f'不支持的 Dify 应用类型: {self.pipeline_config["ai"]["dify-service-api"]["app-type"]}' f'不支持的 Dify 应用类型: {self.pipeline_config["ai"]["dify-service-api"]["app-type"]}'
@@ -171,7 +50,7 @@ class DifyServiceAPIRunner(runner.RequestRunner):
Returns: Returns:
(处理后的内容, 提取的思维链内容) (处理后的内容, 提取的思维链内容)
""" """
remove_think = self.pipeline_config['output'].get('misc', {}).get('remove-think') remove_think = self.pipeline_config['output'].get('misc', '').get('remove-think')
thinking_content = '' thinking_content = ''
# 从 content 中提取 <think> 标签内容 # 从 content 中提取 <think> 标签内容
if content and '<think>' in content and '</think>' in content: if content and '<think>' in content and '</think>' in content:
@@ -295,21 +174,6 @@ class DifyServiceAPIRunner(runner.RequestRunner):
self, query: pipeline_query.Query self, query: pipeline_query.Query
) -> typing.AsyncGenerator[provider_message.Message, None]: ) -> typing.AsyncGenerator[provider_message.Message, None]:
"""调用聊天助手""" """调用聊天助手"""
# Check if this is a form action resume (button click or text match)
form_action_raw = query.variables.get('_dify_form_action')
session_key = _session_key_from_query(query)
if form_action_raw:
form_action = self._merge_pending_form_action(session_key, form_action_raw)
else:
form_action = self._match_pending_form_action(session_key, str(query.message_chain))
if form_action:
_clear_pending_form(session_key, form_action.get('form_token') or None)
async for msg in self._submit_workflow_form_blocking(form_action):
yield msg
return
cov_id = query.session.using_conversation.uuid or None cov_id = query.session.using_conversation.uuid or None
query.variables['conversation_id'] = cov_id query.variables['conversation_id'] = cov_id
@@ -348,45 +212,6 @@ class DifyServiceAPIRunner(runner.RequestRunner):
mode = 'workflow' mode = 'workflow'
if mode == 'workflow': if mode == 'workflow':
if chunk['event'] == 'workflow_paused':
reasons = chunk['data'].get('reasons', [])
workflow_run_id = chunk['data'].get('workflow_run_id', '')
for reason in reasons:
if reason.get('TYPE') != 'human_input_required':
continue
form_content = reason.get('form_content', '')
actions = reason.get('actions', [])
node_title = reason.get('node_title', '')
_set_pending_form(
_session_key_from_query(query),
{
'workflow_run_id': workflow_run_id,
'form_id': reason.get('form_id'),
'form_token': reason.get('form_token'),
'node_id': reason.get('node_id'),
'node_title': node_title,
'form_content': form_content,
'inputs': reason.get('inputs', {}),
'actions': actions,
'expiration_time': reason.get('expiration_time'),
'user': f'{query.session.launcher_type.value}_{query.session.launcher_id}',
},
)
query.variables['_dify_form_render'] = {
'form_content': form_content,
'actions': actions,
'node_title': node_title,
}
display_text = _format_human_input_text(node_title, form_content, actions)
yield provider_message.Message(
role='assistant',
content=display_text,
)
return
if chunk['event'] == 'node_finished': if chunk['event'] == 'node_finished':
if chunk['data']['node_type'] == 'answer': if chunk['data']['node_type'] == 'answer':
answer = self._extract_dify_text_output(chunk['data']['outputs'].get('answer')) answer = self._extract_dify_text_output(chunk['data']['outputs'].get('answer'))
@@ -510,190 +335,11 @@ class DifyServiceAPIRunner(runner.RequestRunner):
query.session.using_conversation.uuid = chunk['conversation_id'] query.session.using_conversation.uuid = chunk['conversation_id']
async def _submit_workflow_form_blocking(
self, form_action: dict
) -> typing.AsyncGenerator[provider_message.Message, None]:
"""Submit human input to resume a paused Dify workflow (non-streaming)."""
form_token = form_action['form_token']
workflow_run_id = form_action['workflow_run_id']
user = form_action['user']
action_id = form_action.get('action_id', '')
inputs = form_action.get('inputs', {})
async for chunk in self.dify_client.workflow_submit(
form_token=form_token,
workflow_run_id=workflow_run_id,
inputs=inputs,
user=user,
action=action_id,
timeout=120,
):
self.ap.logger.debug('dify-workflow-submit-chunk: ' + str(chunk))
if chunk['event'] == 'workflow_finished':
if chunk['data'].get('error'):
raise errors.DifyAPIError(chunk['data']['error'])
content, _ = self._process_thinking_content(chunk['data']['outputs']['summary'])
yield provider_message.Message(
role='assistant',
content=content,
)
return
if chunk['event'] == 'workflow_paused':
reasons = chunk['data'].get('reasons', [])
new_run_id = chunk['data'].get('workflow_run_id', workflow_run_id)
for reason in reasons:
if reason.get('TYPE') != 'human_input_required':
continue
form_content = reason.get('form_content', '')
actions = reason.get('actions', [])
paused_node_title = reason.get('node_title', '')
raw_inputs = reason.get('inputs', {})
_set_pending_form(
user,
{
'workflow_run_id': new_run_id,
'form_id': reason.get('form_id'),
'form_token': reason.get('form_token'),
'node_id': reason.get('node_id'),
'node_title': paused_node_title,
'form_content': form_content,
'inputs': raw_inputs if isinstance(raw_inputs, dict) else {},
'actions': actions,
'expiration_time': reason.get('expiration_time'),
'user': user,
},
)
display_text = _format_human_input_text(paused_node_title, form_content, actions)
yield provider_message.Message(
role='assistant',
content=display_text,
)
return
def _resolve_pending_form(self, session_key: str, form_action: dict) -> dict | None:
"""Locate the pending form this action targets.
Tries identifiers in order of specificity: form_token, full
workflow_run_id, workflow_run_id suffix (Telegram-style compact id),
then falls back to the newest pending form for the session.
"""
form_token = form_action.get('form_token')
if form_token:
form = _get_pending_form_by_token(session_key, form_token)
if form:
return form
workflow_run_id = form_action.get('workflow_run_id')
if workflow_run_id:
for form in _iter_pending_forms(session_key):
if form.get('workflow_run_id') == workflow_run_id:
return form
w_suffix = form_action.get('w_suffix')
if w_suffix:
form = _get_pending_form_by_w_suffix(session_key, w_suffix)
if form:
return form
return _get_latest_pending_form(session_key)
def _merge_pending_form_action(self, session_key: str, form_action: dict | None) -> dict | None:
"""Backfill resume fields from the matching pending form."""
if not form_action:
return None
merged_action = dict(form_action)
merged_action.pop('w_suffix', None)
pending_form = self._resolve_pending_form(session_key, form_action)
if pending_form:
merged_action['form_token'] = merged_action.get('form_token') or pending_form.get('form_token', '')
merged_action['workflow_run_id'] = merged_action.get('workflow_run_id') or pending_form.get(
'workflow_run_id', ''
)
merged_action.setdefault('inputs', pending_form.get('inputs', {}))
merged_action.setdefault('user', pending_form.get('user', ''))
merged_action.setdefault('node_title', pending_form.get('node_title', ''))
# Resolve clicked action's display title from the stored actions list
if 'action_title' not in merged_action:
clicked_id = merged_action.get('action_id', '')
for action in pending_form.get('actions', []):
if str(action.get('id', '')) == str(clicked_id):
merged_action['action_title'] = action.get('title', clicked_id)
break
return merged_action
def _match_pending_form_action(self, session_key: str, user_text: str) -> dict | None:
"""Match plain text replies against pending Dify form actions.
Resolution order:
1. A pure digit reply (e.g. "1", "2") maps to the 1-indexed action of
the most recent pending form. Lets users on plain-text platforms
pick options without retyping titles.
2. Otherwise, iterate pending forms newest-first and match each
action's title/id case-insensitively. The first hit wins, so when
two forms share a button label the newer one resolves.
"""
normalized_text = user_text.strip().lower()
if not normalized_text:
return None
def _build(pending_form: dict, action: dict) -> dict:
return {
'form_token': pending_form.get('form_token', ''),
'workflow_run_id': pending_form.get('workflow_run_id', ''),
'action_id': action.get('id', ''),
'action_title': action.get('title', action.get('id', '')),
'node_title': pending_form.get('node_title', ''),
'inputs': pending_form.get('inputs', {}),
'user': pending_form.get('user', ''),
}
if normalized_text.isdigit():
position = int(normalized_text)
latest_form = _get_latest_pending_form(session_key)
if latest_form is not None:
actions = latest_form.get('actions', [])
if 1 <= position <= len(actions):
return _build(latest_form, actions[position - 1])
for pending_form in _iter_pending_forms(session_key):
for action in pending_form.get('actions', []):
titles = {
str(action.get('title', '')).strip().lower(),
str(action.get('id', '')).strip().lower(),
}
if normalized_text in titles:
return _build(pending_form, action)
return None
async def _workflow_messages( async def _workflow_messages(
self, query: pipeline_query.Query self, query: pipeline_query.Query
) -> typing.AsyncGenerator[provider_message.Message, None]: ) -> typing.AsyncGenerator[provider_message.Message, None]:
"""调用工作流""" """调用工作流"""
# Check if this is a form action resume (button click or text match)
form_action_raw = query.variables.get('_dify_form_action')
session_key = _session_key_from_query(query)
if form_action_raw:
form_action = self._merge_pending_form_action(session_key, form_action_raw)
else:
form_action = self._match_pending_form_action(session_key, str(query.message_chain))
if form_action:
_clear_pending_form(session_key, form_action.get('form_token') or None)
async for msg in self._submit_workflow_form_blocking(form_action):
yield msg
return
if not query.session.using_conversation.uuid: if not query.session.using_conversation.uuid:
query.session.using_conversation.uuid = str(uuid.uuid4()) query.session.using_conversation.uuid = str(uuid.uuid4())
@@ -720,7 +366,6 @@ class DifyServiceAPIRunner(runner.RequestRunner):
} }
inputs.update(query.variables) inputs.update(query.variables)
human_input_yielded = False
async for chunk in self.dify_client.workflow_run( async for chunk in self.dify_client.workflow_run(
inputs=inputs, inputs=inputs,
@@ -732,45 +377,6 @@ class DifyServiceAPIRunner(runner.RequestRunner):
if chunk['event'] in ignored_events: if chunk['event'] in ignored_events:
continue continue
if chunk['event'] == 'workflow_paused':
reasons = chunk['data'].get('reasons', [])
workflow_run_id = chunk['data'].get('workflow_run_id', '')
for reason in reasons:
if reason.get('TYPE') == 'human_input_required':
form_content = reason.get('form_content', '')
actions = reason.get('actions', [])
node_title = reason.get('node_title', '')
_set_pending_form(
_session_key_from_query(query),
{
'workflow_run_id': workflow_run_id,
'form_id': reason.get('form_id'),
'form_token': reason.get('form_token'),
'node_id': reason.get('node_id'),
'node_title': node_title,
'form_content': form_content,
'inputs': reason.get('inputs', {}),
'actions': actions,
'expiration_time': reason.get('expiration_time'),
'user': f'{query.session.launcher_type.value}_{query.session.launcher_id}',
},
)
query.variables['_dify_form_render'] = {
'form_content': form_content,
'actions': actions,
'node_title': node_title,
}
display_text = _format_human_input_text(node_title, form_content, actions)
human_input_yielded = True
yield provider_message.Message(
role='assistant',
content=display_text,
)
if chunk['event'] == 'node_started': if chunk['event'] == 'node_started':
if chunk['data']['node_type'] == 'start' or chunk['data']['node_type'] == 'end': if chunk['data']['node_type'] == 'start' or chunk['data']['node_type'] == 'end':
continue continue
@@ -793,8 +399,6 @@ class DifyServiceAPIRunner(runner.RequestRunner):
yield msg yield msg
elif chunk['event'] == 'workflow_finished': elif chunk['event'] == 'workflow_finished':
if human_input_yielded:
break
if chunk['data']['error']: if chunk['data']['error']:
raise errors.DifyAPIError(chunk['data']['error']) raise errors.DifyAPIError(chunk['data']['error'])
content, _ = self._process_thinking_content(chunk['data']['outputs']['summary']) content, _ = self._process_thinking_content(chunk['data']['outputs']['summary'])
@@ -810,21 +414,6 @@ class DifyServiceAPIRunner(runner.RequestRunner):
self, query: pipeline_query.Query self, query: pipeline_query.Query
) -> typing.AsyncGenerator[provider_message.MessageChunk, None]: ) -> typing.AsyncGenerator[provider_message.MessageChunk, None]:
"""调用聊天助手""" """调用聊天助手"""
# Check if this is a form action resume (button click or text match)
form_action_raw = query.variables.get('_dify_form_action')
session_key = _session_key_from_query(query)
if form_action_raw:
form_action = self._merge_pending_form_action(session_key, form_action_raw)
else:
form_action = self._match_pending_form_action(session_key, str(query.message_chain))
if form_action:
_clear_pending_form(session_key, form_action.get('form_token') or None)
async for msg in self._submit_workflow_form(form_action):
yield msg
return
cov_id = query.session.using_conversation.uuid or None cov_id = query.session.using_conversation.uuid or None
query.variables['conversation_id'] = cov_id query.variables['conversation_id'] = cov_id
@@ -853,11 +442,8 @@ class DifyServiceAPIRunner(runner.RequestRunner):
think_start = False think_start = False
think_end = False think_end = False
yielded_final = False yielded_final = False
human_input_yielded = False
pending_form_data = None
display_text = ''
remove_think = self.pipeline_config['output'].get('misc', {}).get('remove-think') remove_think = self.pipeline_config['output'].get('misc', '').get('remove-think')
async for chunk in self.dify_client.chat_messages( async for chunk in self.dify_client.chat_messages(
inputs=inputs, inputs=inputs,
@@ -871,7 +457,7 @@ class DifyServiceAPIRunner(runner.RequestRunner):
if chunk['event'] == 'workflow_started': if chunk['event'] == 'workflow_started':
mode = 'workflow' mode = 'workflow'
elif chunk['event'] in ('node_started', 'node_finished', 'workflow_finished', 'workflow_paused'): elif chunk['event'] in ('node_started', 'node_finished', 'workflow_finished'):
# Some Dify deployments may omit workflow_started in streamed chunks. # Some Dify deployments may omit workflow_started in streamed chunks.
mode = 'workflow' mode = 'workflow'
@@ -899,61 +485,9 @@ class DifyServiceAPIRunner(runner.RequestRunner):
is_final = True is_final = True
elif chunk['event'] == 'workflow_finished': elif chunk['event'] == 'workflow_finished':
is_final = True is_final = True
if human_input_yielded:
break
if chunk['data'].get('error'): if chunk['data'].get('error'):
raise errors.DifyAPIError(chunk['data']['error']) raise errors.DifyAPIError(chunk['data']['error'])
if mode == 'workflow' and chunk['event'] == 'workflow_paused':
reasons = chunk['data'].get('reasons', [])
workflow_run_id = chunk['data'].get('workflow_run_id', '')
for reason in reasons:
if reason.get('TYPE') != 'human_input_required':
continue
form_content = reason.get('form_content', '')
actions = reason.get('actions', [])
node_title = reason.get('node_title', '')
raw_inputs = reason.get('inputs', {})
_set_pending_form(
_session_key_from_query(query),
{
'workflow_run_id': workflow_run_id,
'form_id': reason.get('form_id'),
'form_token': reason.get('form_token'),
'node_id': reason.get('node_id'),
'node_title': node_title,
'form_content': form_content,
'inputs': raw_inputs if isinstance(raw_inputs, dict) else {},
'actions': actions,
'expiration_time': reason.get('expiration_time'),
'user': f'{query.session.launcher_type.value}_{query.session.launcher_id}',
},
)
query.variables['_dify_form_render'] = {
'form_content': form_content,
'actions': actions,
'node_title': node_title,
}
display_text = _format_human_input_text(node_title, form_content, actions)
# Use a zero-width space so ResponseWrapper lets the chunk
# propagate to SendResponseBackStage, but the adapter
# detects _form_data and renders buttons instead of the
# plain-text prompt (mirrors _workflow_messages_chunk).
if not basic_mode_pending_chunk:
basic_mode_pending_chunk = ''
pending_form_data = {
'form_content': form_content,
'actions': actions,
'node_title': node_title,
'workflow_run_id': workflow_run_id,
'form_token': reason.get('form_token', ''),
}
human_input_yielded = True
if mode == 'workflow' and chunk['event'] == 'node_finished': if mode == 'workflow' and chunk['event'] == 'node_finished':
if chunk['data'].get('node_type') == 'answer': if chunk['data'].get('node_type') == 'answer':
answer = self._extract_dify_text_output(chunk['data'].get('outputs', {}).get('answer')) answer = self._extract_dify_text_output(chunk['data'].get('outputs', {}).get('answer'))
@@ -965,31 +499,15 @@ class DifyServiceAPIRunner(runner.RequestRunner):
and (is_final or message_idx % 8 == 0) and (is_final or message_idx % 8 == 0)
and (basic_mode_pending_chunk != '' or is_final) and (basic_mode_pending_chunk != '' or is_final)
): ):
final_content = basic_mode_pending_chunk if basic_mode_pending_chunk.strip() else '' # content, _ = self._process_thinking_content(basic_mode_pending_chunk)
msg = provider_message.MessageChunk( yield provider_message.MessageChunk(
role='assistant', role='assistant',
content=final_content, content=basic_mode_pending_chunk,
is_final=is_final, is_final=is_final,
) )
if is_final and pending_form_data:
msg._form_data = pending_form_data
pending_form_data = None
yield msg
if is_final: if is_final:
yielded_final = True yielded_final = True
# If the stream ended after workflow_paused without a
# workflow_finished event, yield a final chunk so the adapter
# can update the card and add buttons.
if human_input_yielded and not yielded_final:
msg = provider_message.MessageChunk(
role='assistant',
content=basic_mode_pending_chunk or display_text,
is_final=True,
)
msg._form_data = pending_form_data
yield msg
if chunk is None: if chunk is None:
raise errors.DifyAPIError('Dify API 没有返回任何响应,请检查网络连接和API配置') raise errors.DifyAPIError('Dify API 没有返回任何响应,请检查网络连接和API配置')
@@ -1027,7 +545,7 @@ class DifyServiceAPIRunner(runner.RequestRunner):
think_start = False think_start = False
think_end = False think_end = False
remove_think = self.pipeline_config['output'].get('misc', {}).get('remove-think') remove_think = self.pipeline_config['output'].get('misc', '').get('remove-think')
async for chunk in self.dify_client.chat_messages( async for chunk in self.dify_client.chat_messages(
inputs=inputs, inputs=inputs,
@@ -1118,181 +636,11 @@ class DifyServiceAPIRunner(runner.RequestRunner):
query.session.using_conversation.uuid = chunk['conversation_id'] query.session.using_conversation.uuid = chunk['conversation_id']
async def _submit_workflow_form(
self, form_action: dict
) -> typing.AsyncGenerator[provider_message.MessageChunk, None]:
"""Submit human input to resume a paused Dify workflow."""
form_token = form_action['form_token']
workflow_run_id = form_action['workflow_run_id']
user = form_action['user']
action_id = form_action.get('action_id', '')
action_title = form_action.get('action_title', '') or action_id
node_title = form_action.get('node_title', '')
inputs = form_action.get('inputs', {})
messsage_idx = 0
is_final = False
think_start = False
think_end = False
workflow_contents = ''
repause_form_data: dict | None = None
remove_think = self.pipeline_config['output'].get('misc', {}).get('remove-think')
async for chunk in self.dify_client.workflow_submit(
form_token=form_token,
workflow_run_id=workflow_run_id,
inputs=inputs,
user=user,
action=action_id,
timeout=120,
):
self.ap.logger.debug('dify-workflow-submit-chunk: ' + str(chunk))
yield_this_iteration = False
if chunk['event'] == 'workflow_finished':
is_final = True
yield_this_iteration = True
if chunk['data'].get('error'):
raise errors.DifyAPIError(chunk['data']['error'])
if chunk['event'] == 'workflow_paused':
reasons = chunk['data'].get('reasons', [])
new_run_id = chunk['data'].get('workflow_run_id', workflow_run_id)
for reason in reasons:
if reason.get('TYPE') != 'human_input_required':
continue
form_content = reason.get('form_content', '')
actions = reason.get('actions', [])
# Use a distinct name — `node_title` (the just-resolved step)
# must keep its value so the resume notice on the previous
# card still shows which step the user acted on.
paused_node_title = reason.get('node_title', '')
raw_inputs = reason.get('inputs', {})
_set_pending_form(
# Use the same session-key format as
# _session_key_from_query (launcher_type_launcher_id).
# The 'user' field is set by adapters in this format.
user,
{
'workflow_run_id': new_run_id,
'form_id': reason.get('form_id'),
'form_token': reason.get('form_token'),
'node_id': reason.get('node_id'),
'node_title': paused_node_title,
'form_content': form_content,
'inputs': raw_inputs if isinstance(raw_inputs, dict) else {},
'actions': actions,
'expiration_time': reason.get('expiration_time'),
'user': user,
},
)
repause_form_data = {
'form_content': form_content,
'actions': actions,
'node_title': paused_node_title,
'workflow_run_id': new_run_id,
'form_token': reason.get('form_token', ''),
}
# Ensure the final chunk has non-empty content so
# ResponseWrapper (which skips empty-content chunks) lets it
# propagate to SendResponseBackStage. Use a zero-width space
# so neither Lark nor Telegram renders visible noise — the
# adapter substitutes its own card text from _form_data.
if not workflow_contents:
workflow_contents = ''
is_final = True
yield_this_iteration = True
break
if chunk['event'] == 'text_chunk':
messsage_idx += 1
if remove_think:
if '<think>' in chunk['data']['text'] and not think_start:
think_start = True
continue
if '</think>' in chunk['data']['text'] and not think_end:
import re
content = re.sub(r'^\n</think>', '', chunk['data']['text'])
workflow_contents += content
think_end = True
elif think_end:
workflow_contents += chunk['data']['text']
if think_start:
continue
else:
workflow_contents += chunk['data']['text']
if messsage_idx % 8 == 0:
yield_this_iteration = True
# Chatflow apps return answers via 'message' events (answer field),
# not 'text_chunk' events (data.text field).
if chunk['event'] == 'message':
answer = chunk.get('answer', '')
if answer:
messsage_idx += 1
if remove_think:
if '<think>' in answer and not think_start:
think_start = True
continue
if '</think>' in answer and not think_end:
import re
content = re.sub(r'^\n</think>', '', answer)
workflow_contents += content
think_end = True
elif think_end:
workflow_contents += answer
if think_start:
continue
else:
workflow_contents += answer
if messsage_idx % 8 == 0:
yield_this_iteration = True
if yield_this_iteration:
msg = provider_message.MessageChunk(
role='assistant',
content=workflow_contents,
is_final=is_final,
)
msg._resume_from_form = True
if action_title:
msg._resume_action_title = action_title
if node_title:
msg._resume_node_title = node_title
if is_final and repause_form_data:
msg._form_data = repause_form_data
msg._open_new_card = True
yield msg
if is_final:
return
async def _workflow_messages_chunk( async def _workflow_messages_chunk(
self, query: pipeline_query.Query self, query: pipeline_query.Query
) -> typing.AsyncGenerator[provider_message.MessageChunk, None]: ) -> typing.AsyncGenerator[provider_message.MessageChunk, None]:
"""调用工作流""" """调用工作流"""
# Check if this is a form action resume (button click or text match)
form_action_raw = query.variables.get('_dify_form_action')
session_key = _session_key_from_query(query)
if form_action_raw:
form_action = self._merge_pending_form_action(session_key, form_action_raw)
else:
form_action = self._match_pending_form_action(session_key, str(query.message_chain))
if form_action:
_clear_pending_form(session_key, form_action.get('form_token') or None)
# Resume paused workflow via submit endpoint
async for msg in self._submit_workflow_form(form_action):
yield msg
return
if not query.session.using_conversation.uuid: if not query.session.using_conversation.uuid:
query.session.using_conversation.uuid = str(uuid.uuid4()) query.session.using_conversation.uuid = str(uuid.uuid4())
@@ -1324,15 +672,8 @@ class DifyServiceAPIRunner(runner.RequestRunner):
think_start = False think_start = False
think_end = False think_end = False
workflow_contents = '' workflow_contents = ''
workflow_run_id = ''
human_input_yielded = False
# Saved form data to attach to the final MessageChunk so the adapter remove_think = self.pipeline_config['output'].get('misc', '').get('remove-think')
# can detect it when is_final=True and render buttons.
pending_form_data = None
display_text = ''
remove_think = self.pipeline_config['output'].get('misc', {}).get('remove-think')
async for chunk in self.dify_client.workflow_run( async for chunk in self.dify_client.workflow_run(
inputs=inputs, inputs=inputs,
user=f'{query.session.launcher_type.value}_{query.session.launcher_id}', user=f'{query.session.launcher_type.value}_{query.session.launcher_id}',
@@ -1341,61 +682,7 @@ class DifyServiceAPIRunner(runner.RequestRunner):
): ):
self.ap.logger.debug('dify-workflow-chunk: ' + str(chunk)) self.ap.logger.debug('dify-workflow-chunk: ' + str(chunk))
if chunk['event'] in ignored_events: if chunk['event'] in ignored_events:
if chunk['event'] == 'workflow_started':
workflow_run_id = chunk['data'].get('workflow_run_id', '')
continue continue
if chunk['event'] == 'workflow_paused':
reasons = chunk['data'].get('reasons', [])
workflow_run_id = chunk['data'].get('workflow_run_id', workflow_run_id)
for reason in reasons:
if reason.get('TYPE') == 'human_input_required':
form_content = reason.get('form_content', '')
actions = reason.get('actions', [])
node_title = reason.get('node_title', '')
# Persist form state in module-level store keyed by session
raw_inputs = reason.get('inputs', {})
_set_pending_form(
_session_key_from_query(query),
{
'workflow_run_id': workflow_run_id,
'form_id': reason.get('form_id'),
'form_token': reason.get('form_token'),
'node_id': reason.get('node_id'),
'node_title': node_title,
'form_content': form_content,
'inputs': raw_inputs if isinstance(raw_inputs, dict) else {},
'actions': actions,
'expiration_time': reason.get('expiration_time'),
'user': f'{query.session.launcher_type.value}_{query.session.launcher_id}',
},
)
# Pass form render metadata to downstream stages
query.variables['_dify_form_render'] = {
'form_content': form_content,
'actions': actions,
'node_title': node_title,
}
display_text = _format_human_input_text(node_title, form_content, actions)
workflow_contents += display_text + '\n'
# Save form data to attach to the final chunk later.
# We do NOT yield here — the form content will be sent
# as the final MessageChunk (with is_final=True and
# _form_data) so the adapter can update the card and
# add buttons in one pass.
pending_form_data = {
'form_content': form_content,
'actions': actions,
'node_title': node_title,
'workflow_run_id': workflow_run_id,
'form_token': reason.get('form_token', ''),
}
human_input_yielded = True
if chunk['event'] == 'workflow_finished': if chunk['event'] == 'workflow_finished':
is_final = True is_final = True
if chunk['data']['error']: if chunk['data']['error']:
@@ -1443,35 +730,17 @@ class DifyServiceAPIRunner(runner.RequestRunner):
yield msg yield msg
if messsage_idx % 8 == 0 or is_final: if messsage_idx % 8 == 0 or is_final:
final_content = workflow_contents if workflow_contents.strip() else '' yield provider_message.MessageChunk(
msg = provider_message.MessageChunk(
role='assistant', role='assistant',
content=final_content, content=workflow_contents,
is_final=is_final, is_final=is_final,
) )
# Attach form data to the final chunk for the adapter
if is_final and pending_form_data:
msg._form_data = pending_form_data
pending_form_data = None
yield msg
# If the stream ended after workflow_paused without a
# workflow_finished event, yield a final chunk so the adapter
# can update the card and add buttons.
if human_input_yielded and not is_final:
msg = provider_message.MessageChunk(
role='assistant',
content=workflow_contents or display_text,
is_final=True,
)
msg._form_data = pending_form_data
yield msg
async def run(self, query: pipeline_query.Query) -> typing.AsyncGenerator[provider_message.Message, None]: async def run(self, query: pipeline_query.Query) -> typing.AsyncGenerator[provider_message.Message, None]:
"""运行请求""" """运行请求"""
if await query.adapter.is_stream_output_supported(): if await query.adapter.is_stream_output_supported():
msg_idx = 0 msg_idx = 0
if self.pipeline_config['ai']['dify-service-api']['app-type'] in ('chat', 'chatflow'): if self.pipeline_config['ai']['dify-service-api']['app-type'] == 'chat':
async for msg in self._chat_messages_chunk(query): async for msg in self._chat_messages_chunk(query):
msg_idx += 1 msg_idx += 1
msg.msg_sequence = msg_idx msg.msg_sequence = msg_idx
@@ -1491,7 +760,7 @@ class DifyServiceAPIRunner(runner.RequestRunner):
f'不支持的 Dify 应用类型: {self.pipeline_config["ai"]["dify-service-api"]["app-type"]}' f'不支持的 Dify 应用类型: {self.pipeline_config["ai"]["dify-service-api"]["app-type"]}'
) )
else: else:
if self.pipeline_config['ai']['dify-service-api']['app-type'] in ('chat', 'chatflow'): if self.pipeline_config['ai']['dify-service-api']['app-type'] == 'chat':
async for msg in self._chat_messages(query): async for msg in self._chat_messages(query):
yield msg yield msg
elif self.pipeline_config['ai']['dify-service-api']['app-type'] == 'agent': elif self.pipeline_config['ai']['dify-service-api']['app-type'] == 'agent':
+60 -3
View File
@@ -5,6 +5,7 @@ import copy
import typing import typing
from .. import runner from .. import runner
from ..modelmgr import requester as modelmgr_requester from ..modelmgr import requester as modelmgr_requester
from ..tools.loaders.native import EXEC_TOOL_NAME
import langbot_plugin.api.entities.builtin.pipeline.query as pipeline_query import langbot_plugin.api.entities.builtin.pipeline.query as pipeline_query
import langbot_plugin.api.entities.builtin.provider.message as provider_message import langbot_plugin.api.entities.builtin.provider.message as provider_message
import langbot_plugin.api.entities.builtin.rag.context as rag_context import langbot_plugin.api.entities.builtin.rag.context as rag_context
@@ -24,11 +25,44 @@ Respond in the same language as the user's input.
</user_message> </user_message>
""" """
SANDBOX_EXEC_TOOL_NAME = 'sandbox_exec'
SANDBOX_EXEC_SYSTEM_GUIDANCE = (
'When sandbox_exec is available, use it for exact calculations, statistics, structured data parsing, '
'and code execution instead of estimating mentally. If the user provides numbers, tables, CSV-like text, '
'JSON, or other data and asks for a computed answer, prefer running a short Python script in sandbox_exec '
'and then answer from the tool result.'
)
# Hard cap on tool-call rounds within a single agent turn. A looping or
# adversarial model can otherwise emit tool calls indefinitely (each potentially
# a sandbox exec), yielding a non-terminating request and runaway cost. Set
# generously so it never interrupts legitimate multi-step agentic workflows.
MAX_TOOL_CALL_ROUNDS = 128
@runner.runner_class('local-agent') @runner.runner_class('local-agent')
class LocalAgentRunner(runner.RequestRunner): class LocalAgentRunner(runner.RequestRunner):
"""Local agent request runner""" """Local agent request runner"""
def _build_request_messages(
self,
query: pipeline_query.Query,
user_message: provider_message.Message,
) -> list[provider_message.Message]:
req_messages = query.prompt.messages.copy() + query.messages.copy()
if any(getattr(tool, 'name', None) == EXEC_TOOL_NAME for tool in query.use_funcs or []):
req_messages.append(
provider_message.Message(
role='system',
content=self.ap.box_service.get_system_guidance(),
)
)
req_messages.append(user_message)
return req_messages
async def _get_model_candidates( async def _get_model_candidates(
self, self,
query: pipeline_query.Query, query: pipeline_query.Query,
@@ -131,6 +165,7 @@ class LocalAgentRunner(runner.RequestRunner):
) -> typing.AsyncGenerator[provider_message.Message | provider_message.MessageChunk, None]: ) -> typing.AsyncGenerator[provider_message.Message | provider_message.MessageChunk, None]:
"""Run request""" """Run request"""
pending_tool_calls = [] pending_tool_calls = []
initial_response_emitted = False
# Get knowledge bases list from query variables (set by PreProcessor, # Get knowledge bases list from query variables (set by PreProcessor,
# may have been modified by plugins during PromptPreProcessing) # may have been modified by plugins during PromptPreProcessing)
@@ -236,7 +271,7 @@ class LocalAgentRunner(runner.RequestRunner):
ce.text = final_user_message_text ce.text = final_user_message_text
break break
req_messages = query.prompt.messages.copy() + query.messages.copy() + [user_message] req_messages = self._build_request_messages(query, user_message)
try: try:
is_stream = await query.adapter.is_stream_output_supported() is_stream = await query.adapter.is_stream_output_supported()
@@ -264,7 +299,6 @@ class LocalAgentRunner(runner.RequestRunner):
query.use_funcs, query.use_funcs,
remove_think, remove_think,
) )
yield msg
final_msg = msg final_msg = msg
else: else:
# Streaming: invoke with fallback # Streaming: invoke with fallback
@@ -312,6 +346,7 @@ class LocalAgentRunner(runner.RequestRunner):
is_final=msg.is_final, is_final=msg.is_final,
msg_sequence=msg_sequence, msg_sequence=msg_sequence,
) )
initial_response_emitted = True
final_msg = provider_message.MessageChunk( final_msg = provider_message.MessageChunk(
role=last_role, role=last_role,
@@ -325,11 +360,25 @@ class LocalAgentRunner(runner.RequestRunner):
if isinstance(final_msg, provider_message.MessageChunk): if isinstance(final_msg, provider_message.MessageChunk):
first_end_sequence = final_msg.msg_sequence first_end_sequence = final_msg.msg_sequence
if not is_stream:
yield final_msg
elif not initial_response_emitted:
yield final_msg
initial_response_emitted = True
req_messages.append(final_msg) req_messages.append(final_msg)
# Once a model succeeds, commit to it for the tool call loop # Once a model succeeds, commit to it for the tool call loop
# (no fallback mid-conversation — different models may interpret tool results differently) # (no fallback mid-conversation — different models may interpret tool results differently)
tool_call_round = 0
while pending_tool_calls: while pending_tool_calls:
tool_call_round += 1
if tool_call_round > MAX_TOOL_CALL_ROUNDS:
self.ap.logger.warning(
f'Tool-call loop reached the {MAX_TOOL_CALL_ROUNDS}-round cap '
f'(query_id={query.query_id}); stopping to avoid a non-terminating request.'
)
break
for tool_call in pending_tool_calls: for tool_call in pending_tool_calls:
try: try:
func = tool_call.function func = tool_call.function
@@ -369,7 +418,15 @@ class LocalAgentRunner(runner.RequestRunner):
req_messages.append(msg) req_messages.append(msg)
except Exception as e: except Exception as e:
err_msg = provider_message.Message(role='tool', content=f'err: {e}', tool_call_id=tool_call.id) if is_stream:
err_msg = provider_message.MessageChunk(
role='tool',
content=f'err: {e}',
tool_call_id=tool_call.id,
is_final=True,
)
else:
err_msg = provider_message.Message(role='tool', content=f'err: {e}', tool_call_id=tool_call.id)
yield err_msg yield err_msg
@@ -0,0 +1,351 @@
from __future__ import annotations
import typing
import json
from langbot.pkg.provider import runner
from langbot.pkg.core import app
import langbot_plugin.api.entities.builtin.provider.message as provider_message
import langbot_plugin.api.entities.builtin.pipeline.query as pipeline_query
from langbot.libs.weknora_api import client, errors
@runner.runner_class('weknora-api')
class WeKnoraAPIRunner(runner.RequestRunner):
"""WeKnora API 对话请求器"""
weknora_client: client.AsyncWeKnoraClient
def __init__(self, ap: app.Application, pipeline_config: dict):
super().__init__(ap, pipeline_config)
valid_app_types = ['chat', 'agent']
if self.pipeline_config['ai']['weknora-api']['app-type'] not in valid_app_types:
raise errors.WeKnoraAPIError(
f'不支持的 WeKnora 应用类型: {self.pipeline_config["ai"]["weknora-api"]["app-type"]}'
)
api_key = self.pipeline_config['ai']['weknora-api'].get('api-key', '').strip()
if not api_key:
raise errors.WeKnoraAPIError(
'WeKnora API Key 未配置,请在流水线的 WeKnora API 配置中填入 API Key '
'(从 WeKnora 前端 设置 → API Keys 生成)'
)
base_url = self.pipeline_config['ai']['weknora-api'].get('base-url', '').strip()
if not base_url:
raise errors.WeKnoraAPIError('WeKnora Base URL 未配置,请填入服务器地址,例如 http://localhost:8080/api/v1')
self.weknora_client = client.AsyncWeKnoraClient(
api_key=api_key,
base_url=base_url,
)
async def _extract_plain_text(self, query: pipeline_query.Query) -> str:
"""从用户消息中提取纯文本内容"""
plain_text = ''
if isinstance(query.user_message.content, str):
plain_text = query.user_message.content
elif isinstance(query.user_message.content, list):
for ce in query.user_message.content:
if ce.type == 'text':
plain_text += ce.text
if not plain_text:
plain_text = self.pipeline_config['ai']['weknora-api'].get('base-prompt', '')
return plain_text
async def _ensure_session(self, query: pipeline_query.Query) -> str:
"""确保会话存在,如果不存在则创建"""
session_id = query.session.using_conversation.uuid or ''
if not session_id:
user_tag = f'{query.session.launcher_type.value}_{query.session.launcher_id}'
session_id = await self.weknora_client.create_session(title=f'IM Chat - {user_tag}')
query.session.using_conversation.uuid = session_id
return session_id
async def _agent_chat_messages(
self, query: pipeline_query.Query
) -> typing.AsyncGenerator[provider_message.Message, None]:
"""调用 Agent 智能对话(非流式聚合输出)"""
session_id = await self._ensure_session(query)
plain_text = await self._extract_plain_text(query)
user_tag = f'{query.session.launcher_type.value}_{query.session.launcher_id}'
config = self.pipeline_config['ai']['weknora-api']
agent_id = config.get('agent-id', 'builtin-smart-reasoning')
knowledge_base_ids = config.get('knowledge-base-ids', [])
web_search_enabled = config.get('web-search-enabled', False)
timeout = config.get('timeout', 120)
full_answer = ''
chunk = None
async for chunk in self.weknora_client.agent_chat(
session_id=session_id,
query=plain_text,
user=user_tag,
agent_id=agent_id,
knowledge_base_ids=knowledge_base_ids,
web_search_enabled=web_search_enabled,
timeout=timeout,
):
self.ap.logger.debug('weknora-agent-chunk: ' + str(chunk))
response_type = chunk.get('response_type', '')
content = chunk.get('content', '')
if response_type == 'tool_call':
# 工具调用
tool_data = chunk.get('data', {})
tool_name = tool_data.get('tool_name', '')
if tool_name:
yield provider_message.Message(
role='assistant',
tool_calls=[
provider_message.ToolCall(
id=chunk.get('id', ''),
type='function',
function=provider_message.FunctionCall(
name=tool_name,
arguments=json.dumps(tool_data.get('arguments', {})),
),
)
],
)
elif response_type == 'answer':
if content:
full_answer += content
elif response_type == 'error':
raise errors.WeKnoraAPIError(f'WeKnora 服务错误: {content}')
if chunk is None:
raise errors.WeKnoraAPIError('WeKnora API 没有返回任何响应,请检查网络连接和API配置')
if full_answer:
yield provider_message.Message(
role='assistant',
content=full_answer,
)
async def _chat_messages(
self, query: pipeline_query.Query
) -> typing.AsyncGenerator[provider_message.Message, None]:
"""调用知识库 RAG 问答(非流式聚合输出)"""
session_id = await self._ensure_session(query)
plain_text = await self._extract_plain_text(query)
user_tag = f'{query.session.launcher_type.value}_{query.session.launcher_id}'
config = self.pipeline_config['ai']['weknora-api']
agent_id = config.get('agent-id', 'builtin-quick-answer')
knowledge_base_ids = config.get('knowledge-base-ids', [])
timeout = config.get('timeout', 120)
full_answer = ''
chunk = None
async for chunk in self.weknora_client.knowledge_chat(
session_id=session_id,
query=plain_text,
user=user_tag,
agent_id=agent_id,
knowledge_base_ids=knowledge_base_ids,
timeout=timeout,
):
self.ap.logger.debug('weknora-chat-chunk: ' + str(chunk))
response_type = chunk.get('response_type', '')
content = chunk.get('content', '')
if response_type == 'answer':
if content:
full_answer += content
elif response_type == 'error':
raise errors.WeKnoraAPIError(f'WeKnora 服务错误: {content}')
if chunk is None:
raise errors.WeKnoraAPIError('WeKnora API 没有返回任何响应,请检查网络连接和API配置')
if full_answer:
yield provider_message.Message(
role='assistant',
content=full_answer,
)
async def _agent_chat_messages_chunk(
self, query: pipeline_query.Query
) -> typing.AsyncGenerator[provider_message.MessageChunk, None]:
"""调用 Agent 智能对话(流式输出)"""
session_id = await self._ensure_session(query)
plain_text = await self._extract_plain_text(query)
user_tag = f'{query.session.launcher_type.value}_{query.session.launcher_id}'
config = self.pipeline_config['ai']['weknora-api']
agent_id = config.get('agent-id', 'builtin-smart-reasoning')
knowledge_base_ids = config.get('knowledge-base-ids', [])
web_search_enabled = config.get('web-search-enabled', False)
timeout = config.get('timeout', 120)
pending_answer = ''
message_idx = 0
is_final = False
chunk = None
async for chunk in self.weknora_client.agent_chat(
session_id=session_id,
query=plain_text,
user=user_tag,
agent_id=agent_id,
knowledge_base_ids=knowledge_base_ids,
web_search_enabled=web_search_enabled,
timeout=timeout,
):
self.ap.logger.debug('weknora-agent-chunk: ' + str(chunk))
response_type = chunk.get('response_type', '')
content = chunk.get('content', '')
done = chunk.get('done', False)
if response_type == 'tool_call':
tool_data = chunk.get('data', {})
tool_name = tool_data.get('tool_name', '')
if tool_name:
message_idx += 1
yield provider_message.MessageChunk(
role='assistant',
tool_calls=[
provider_message.ToolCall(
id=chunk.get('id', ''),
type='function',
function=provider_message.FunctionCall(
name=tool_name,
arguments=json.dumps(tool_data.get('arguments', {})),
),
)
],
)
elif response_type == 'answer':
message_idx += 1
if content:
pending_answer += content
if done:
is_final = True
# 每 8 个 chunk 输出一次,或最终输出
if message_idx % 8 == 0 or is_final:
yield provider_message.MessageChunk(
role='assistant',
content=pending_answer,
is_final=is_final,
)
elif response_type == 'error':
raise errors.WeKnoraAPIError(f'WeKnora 服务错误: {content}')
if chunk is None:
raise errors.WeKnoraAPIError('WeKnora API 没有返回任何响应,请检查网络连接和API配置')
# 确保最终消息已发出
if not is_final and pending_answer:
yield provider_message.MessageChunk(
role='assistant',
content=pending_answer,
is_final=True,
)
async def _chat_messages_chunk(
self, query: pipeline_query.Query
) -> typing.AsyncGenerator[provider_message.MessageChunk, None]:
"""调用知识库 RAG 问答(流式输出)"""
session_id = await self._ensure_session(query)
plain_text = await self._extract_plain_text(query)
user_tag = f'{query.session.launcher_type.value}_{query.session.launcher_id}'
config = self.pipeline_config['ai']['weknora-api']
agent_id = config.get('agent-id', 'builtin-quick-answer')
knowledge_base_ids = config.get('knowledge-base-ids', [])
timeout = config.get('timeout', 120)
pending_answer = ''
message_idx = 0
is_final = False
chunk = None
async for chunk in self.weknora_client.knowledge_chat(
session_id=session_id,
query=plain_text,
user=user_tag,
agent_id=agent_id,
knowledge_base_ids=knowledge_base_ids,
timeout=timeout,
):
self.ap.logger.debug('weknora-chat-chunk: ' + str(chunk))
response_type = chunk.get('response_type', '')
content = chunk.get('content', '')
done = chunk.get('done', False)
if response_type == 'answer':
message_idx += 1
if content:
pending_answer += content
if done:
is_final = True
if message_idx % 8 == 0 or is_final:
yield provider_message.MessageChunk(
role='assistant',
content=pending_answer,
is_final=is_final,
)
elif response_type == 'error':
raise errors.WeKnoraAPIError(f'WeKnora 服务错误: {content}')
if chunk is None:
raise errors.WeKnoraAPIError('WeKnora API 没有返回任何响应,请检查网络连接和API配置')
if not is_final and pending_answer:
yield provider_message.MessageChunk(
role='assistant',
content=pending_answer,
is_final=True,
)
async def run(self, query: pipeline_query.Query) -> typing.AsyncGenerator[provider_message.Message, None]:
"""运行请求"""
app_type = self.pipeline_config['ai']['weknora-api']['app-type']
if await query.adapter.is_stream_output_supported():
msg_idx = 0
if app_type == 'agent':
async for msg in self._agent_chat_messages_chunk(query):
msg_idx += 1
msg.msg_sequence = msg_idx
yield msg
elif app_type == 'chat':
async for msg in self._chat_messages_chunk(query):
msg_idx += 1
msg.msg_sequence = msg_idx
yield msg
else:
raise errors.WeKnoraAPIError(f'不支持的 WeKnora 应用类型: {app_type}')
else:
if app_type == 'agent':
async for msg in self._agent_chat_messages(query):
yield msg
elif app_type == 'chat':
async for msg in self._chat_messages(query):
yield msg
else:
raise errors.WeKnoraAPIError(f'不支持的 WeKnora 应用类型: {app_type}')
+4 -2
View File
@@ -2,12 +2,14 @@ from __future__ import annotations
import abc import abc
import typing import typing
from typing import TYPE_CHECKING
from langbot_plugin.api.entities.events import pipeline_query from langbot_plugin.api.entities.events import pipeline_query
from ...core import app
import langbot_plugin.api.entities.builtin.resource.tool as resource_tool import langbot_plugin.api.entities.builtin.resource.tool as resource_tool
if TYPE_CHECKING:
from ...core import app
preregistered_loaders: list[typing.Type[ToolLoader]] = [] preregistered_loaders: list[typing.Type[ToolLoader]] = []
+202 -18
View File
@@ -20,6 +20,7 @@ from ....core import app
import langbot_plugin.api.entities.builtin.resource.tool as resource_tool import langbot_plugin.api.entities.builtin.resource.tool as resource_tool
import langbot_plugin.api.entities.builtin.provider.message as provider_message import langbot_plugin.api.entities.builtin.provider.message as provider_message
from ....entity.persistence import mcp as persistence_mcp from ....entity.persistence import mcp as persistence_mcp
from .mcp_stdio import BoxStdioSessionRuntime, MCPServerBoxConfig, MCPSessionErrorPhase # noqa: F401
class MCPSessionStatus(enum.Enum): class MCPSessionStatus(enum.Enum):
@@ -58,6 +59,12 @@ class RuntimeMCPSession:
error_message: str | None = None error_message: str | None = None
error_phase: MCPSessionErrorPhase | None = None
retry_count: int = 0
_box_stdio_runtime: BoxStdioSessionRuntime
def __init__(self, server_name: str, server_config: dict, enable: bool, ap: app.Application): def __init__(self, server_name: str, server_config: dict, enable: bool, ap: app.Application):
self.server_name = server_name self.server_name = server_name
self.server_uuid = server_config.get('uuid', '') self.server_uuid = server_config.get('uuid', '')
@@ -66,6 +73,13 @@ class RuntimeMCPSession:
self.enable = enable self.enable = enable
self.session = None self.session = None
# Transient test sessions (created from the config page "test" button,
# which carry no persisted server UUID) must NOT share the live
# "mcp-shared" Box session. Otherwise a failing test churns the shared
# session and tears down healthy, already-connected servers. Callers
# flag these via server_config['_transient'] = True.
self.is_transient = bool(server_config.get('_transient', False))
self.exit_stack = AsyncExitStack() self.exit_stack = AsyncExitStack()
self.functions = [] self.functions = []
@@ -75,7 +89,33 @@ class RuntimeMCPSession:
self._shutdown_event = asyncio.Event() self._shutdown_event = asyncio.Event()
self._ready_event = asyncio.Event() self._ready_event = asyncio.Event()
self._box_stdio_runtime = BoxStdioSessionRuntime(self)
self.box_config = self._box_stdio_runtime.config
async def _init_stdio_python_server(self): async def _init_stdio_python_server(self):
if self._uses_box_stdio():
await self._box_stdio_runtime.initialize()
return
# Box is configured (ap.box_service exists) but currently unavailable
# (disabled by config or connection failed). Refuse stdio MCP rather
# than silently falling through to host-stdio — the operator asked
# for the sandbox and the failure mode should be visible.
#
# Set ``error_phase = BOX_UNAVAILABLE`` BEFORE raising so the retry
# wrapper can short-circuit (retrying is pointless when Box is
# deliberately off) and the frontend can render a localized,
# actionable message instead of this raw RuntimeError. Keep the
# message itself short — the frontend ignores it for this phase.
box_service = getattr(self.ap, 'box_service', None)
if box_service is not None and not getattr(box_service, 'available', False):
self.error_phase = MCPSessionErrorPhase.BOX_UNAVAILABLE
if not getattr(box_service, 'enabled', True):
raise RuntimeError('box_disabled_in_config')
raise RuntimeError('box_unavailable')
# Legacy: no box_service installed at all (pre-Box dev mode). Fall
# through to host-stdio for backward compatibility.
server_params = StdioServerParameters( server_params = StdioServerParameters(
command=self.server_config['command'], command=self.server_config['command'],
args=self.server_config['args'], args=self.server_config['args'],
@@ -90,6 +130,9 @@ class RuntimeMCPSession:
await self.session.initialize() await self.session.initialize()
async def _init_box_stdio_server(self):
await self._box_stdio_runtime.initialize()
async def _init_sse_server(self): async def _init_sse_server(self):
sse_transport = await self.exit_stack.enter_async_context( sse_transport = await self.exit_stack.enter_async_context(
sse_client( sse_client(
@@ -124,8 +167,11 @@ class RuntimeMCPSession:
await self.session.initialize() await self.session.initialize()
_MAX_RETRIES = 3
_RETRY_DELAYS = [2, 4, 8]
async def _lifecycle_loop(self): async def _lifecycle_loop(self):
"""在后台任务中管理整个MCP会话的生命周期""" """Manage the full MCP session lifecycle in a background task."""
try: try:
if self.server_config['mode'] == 'stdio': if self.server_config['mode'] == 'stdio':
await self._init_stdio_python_server() await self._init_stdio_python_server()
@@ -134,49 +180,134 @@ class RuntimeMCPSession:
elif self.server_config['mode'] == 'http': elif self.server_config['mode'] == 'http':
await self._init_streamable_http_server() await self._init_streamable_http_server()
else: else:
raise ValueError(f'无法识别 MCP 服务器类型: {self.server_name}: {self.server_config}') raise ValueError(f'Unknown MCP server mode: {self.server_name}: {self.server_config}')
await self.refresh() await self.refresh()
self.status = MCPSessionStatus.CONNECTED self.status = MCPSessionStatus.CONNECTED
# 通知start()方法连接已建立 # Notify start() that connection is established
self._ready_event.set() self._ready_event.set()
# 等待shutdown信号 # Wait for shutdown signal, with optional health monitoring for Box stdio
await self._shutdown_event.wait() if self._uses_box_stdio():
monitor_task = asyncio.create_task(self._box_stdio_runtime.monitor_process_health())
shutdown_task = asyncio.create_task(self._shutdown_event.wait())
done, pending = await asyncio.wait(
[shutdown_task, monitor_task],
return_when=asyncio.FIRST_COMPLETED,
)
for task in pending:
task.cancel()
for task in done:
if task is monitor_task and not self._shutdown_event.is_set():
self.error_phase = MCPSessionErrorPhase.RUNTIME
raise Exception('Box managed process exited unexpectedly')
else:
await self._shutdown_event.wait()
except Exception as e: except Exception as e:
self.status = MCPSessionStatus.ERROR self.status = MCPSessionStatus.ERROR
self.error_message = str(e) self.error_message = str(e)
self.ap.logger.error(f'Error in MCP session lifecycle {self.server_name}: {e}\n{traceback.format_exc()}') self.ap.logger.error(f'Error in MCP session lifecycle {self.server_name}: {e}\n{traceback.format_exc()}')
# 即使出错也要设置ready事件,让start()方法知道初始化已完成 # Do NOT set _ready_event here — let _lifecycle_loop_with_retry
self._ready_event.set() # handle retries first. It will set the event when all retries
# are exhausted or on success.
raise # Re-raise so _lifecycle_loop_with_retry can catch it
finally: finally:
# 在同一个任务中清理所有资源 # Clean up all resources in the same task
try: try:
if self.exit_stack: if self.exit_stack:
await self.exit_stack.aclose() await self.exit_stack.aclose()
self.exit_stack = AsyncExitStack()
self.functions.clear() self.functions.clear()
self.session = None self.session = None
except Exception as e: except Exception as e:
self.ap.logger.error(f'Error cleaning up MCP session {self.server_name}: {e}\n{traceback.format_exc()}') self.ap.logger.error(f'Error cleaning up MCP session {self.server_name}: {e}\n{traceback.format_exc()}')
finally:
await self._cleanup_box_stdio_session()
async def _lifecycle_loop_with_retry(self):
"""Wrap _lifecycle_loop with retry and exponential backoff."""
for attempt in range(self._MAX_RETRIES + 1):
try:
await self._lifecycle_loop()
return # Normal shutdown, don't retry
except Exception as e:
self.retry_count = attempt + 1
if self._shutdown_event.is_set():
return # Shutdown requested, don't retry
# BOX_UNAVAILABLE is a deliberate refusal, not a transient
# failure — retrying produces log spam and a misleading
# "Failed after N attempts" message. Surface it immediately.
if self.error_phase == MCPSessionErrorPhase.BOX_UNAVAILABLE:
self.status = MCPSessionStatus.ERROR
self.error_message = str(e)
self._ready_event.set()
return
if attempt >= self._MAX_RETRIES:
self.status = MCPSessionStatus.ERROR
self.error_message = f'Failed after {self._MAX_RETRIES + 1} attempts: {self._describe_exception(e)}'
self._ready_event.set()
return
delay = self._RETRY_DELAYS[attempt]
self.ap.logger.warning(
f'MCP session {self.server_name} failed (attempt {attempt + 1}), '
f'retrying in {delay}s: {self._describe_exception(e)}'
)
await self._cleanup_box_stdio_session()
# Reset status for retry
self.status = MCPSessionStatus.CONNECTING
self.error_message = None
self.error_phase = None
await asyncio.sleep(delay)
@staticmethod
def _describe_exception(exc: BaseException) -> str:
"""Flatten an exception into its underlying leaf messages.
anyio / the MCP client wrap real failures in a TaskGroup, whose own
message is the unhelpful "unhandled errors in a TaskGroup (N
sub-exception)". Recurse into ExceptionGroups so the actual cause
(e.g. ``httpx.HTTPStatusError: Client error '410 Gone'``) is surfaced.
"""
leaves: list[str] = []
def visit(e: BaseException) -> None:
sub = getattr(e, 'exceptions', None)
if sub: # ExceptionGroup / BaseExceptionGroup
for child in sub:
visit(child)
else:
leaves.append(f'{type(e).__name__}: {e}')
visit(exc)
seen: set[str] = set()
unique = [m for m in leaves if not (m in seen or seen.add(m))]
return '; '.join(unique) if unique else f'{type(exc).__name__}: {exc}'
_MONITOR_POLL_INTERVAL = 5
_MONITOR_MAX_CONSECUTIVE_ERRORS = 3
async def _monitor_box_process_health(self):
await self._box_stdio_runtime.monitor_process_health()
async def start(self): async def start(self):
if not self.enable: if not self.enable:
return return
# 创建后台任务来管理生命周期 # Create background task for lifecycle management with retry
self._lifecycle_task = asyncio.create_task(self._lifecycle_loop()) self._lifecycle_task = asyncio.create_task(self._lifecycle_loop_with_retry())
# 等待连接建立或失败(带超时) # Wait for connection or failure (with timeout)
startup_timeout = (self.box_config.startup_timeout_sec + 30) if self._uses_box_stdio() else 30.0
try: try:
await asyncio.wait_for(self._ready_event.wait(), timeout=30.0) await asyncio.wait_for(self._ready_event.wait(), timeout=startup_timeout)
except asyncio.TimeoutError: except asyncio.TimeoutError:
self.status = MCPSessionStatus.ERROR self.status = MCPSessionStatus.ERROR
raise Exception('Connection timeout after 30 seconds') raise Exception(f'Connection timeout after {startup_timeout} seconds')
# 检查是否有错误 # Check for errors
if self.status == MCPSessionStatus.ERROR: if self.status == MCPSessionStatus.ERROR:
raise Exception('Connection failed, please check URL') raise Exception('Connection failed, please check URL')
@@ -232,18 +363,25 @@ class RuntimeMCPSession:
return self.functions return self.functions
def get_runtime_info_dict(self) -> dict: def get_runtime_info_dict(self) -> dict:
return { info = {
'status': self.status.value, 'status': self.status.value,
'error_message': self.error_message, 'error_message': self.error_message,
'error_phase': self.error_phase.value if self.error_phase else None,
'retry_count': self.retry_count,
'tool_count': len(self.get_tools()), 'tool_count': len(self.get_tools()),
'tools': [ 'tools': [
{ {
'name': tool.name, 'name': tool.name,
'description': tool.description, 'description': tool.description,
'parameters': tool.parameters,
} }
for tool in self.get_tools() for tool in self.get_tools()
], ],
} }
if self._uses_box_stdio():
info['box_session_id'] = self._build_box_session_id()
info['box_enabled'] = True
return info
async def shutdown(self): async def shutdown(self):
"""关闭会话并清理资源""" """关闭会话并清理资源"""
@@ -267,6 +405,46 @@ class RuntimeMCPSession:
except Exception as e: except Exception as e:
self.ap.logger.error(f'Error shutting down MCP session {self.server_name}: {e}\n{traceback.format_exc()}') self.ap.logger.error(f'Error shutting down MCP session {self.server_name}: {e}\n{traceback.format_exc()}')
def _uses_box_stdio(self) -> bool:
return self._box_stdio_runtime.uses_box_stdio()
def _build_box_session_id(self) -> str:
# Transient test sessions get their own isolated Box session so a
# failing/short-lived test can never disturb the shared session that
# hosts live, already-connected MCP servers.
if self.is_transient:
return f'mcp-test-{self.server_uuid}'
return 'mcp-shared'
def _rewrite_path(self, path: str, host_path: str | None) -> str:
return self._box_stdio_runtime.rewrite_path(path, host_path)
def _infer_host_path(self) -> str | None:
return self._box_stdio_runtime.infer_host_path()
@staticmethod
def _unwrap_venv_path(directory: str) -> str:
return BoxStdioSessionRuntime.unwrap_venv_path(directory)
def _resolve_host_path(self) -> str | None:
return self._box_stdio_runtime.resolve_host_path()
@staticmethod
def _detect_install_command(host_path: str) -> str | None:
return BoxStdioSessionRuntime.detect_install_command(host_path)
def _build_box_session_payload(self, session_id: str, host_path: str | None = None) -> dict:
return self._box_stdio_runtime.build_box_session_payload(session_id, host_path)
def _build_box_process_payload(self, host_path: str | None = None) -> dict:
return self._box_stdio_runtime.build_box_process_payload(host_path)
def _rewrite_venv_command(self, command: str, host_path: str) -> str:
return self._box_stdio_runtime.rewrite_venv_command(command, host_path)
async def _cleanup_box_stdio_session(self) -> None:
await self._box_stdio_runtime.cleanup_session()
# @loader.loader_class('mcp') # @loader.loader_class('mcp')
class MCPLoader(loader.ToolLoader): class MCPLoader(loader.ToolLoader):
@@ -332,15 +510,19 @@ class MCPLoader(loader.ToolLoader):
Args: Args:
server_config: 服务器配置字典必须包含: server_config: 服务器配置字典必须包含:
- name: 服务器名称 - name: 服务器名称
- mode: 连接模式 (stdio/sse) - mode: 连接模式 (stdio/sse/http)
- enable: 是否启用 - enable: 是否启用
- extra_args: 额外的配置参数 (可选) - extra_args: 额外的配置参数 (可选)
""" """
uuid_ = server_config.get('uuid') uuid_ = server_config.get('uuid')
is_transient = False
if not uuid_: if not uuid_:
self.ap.logger.warning('Server UUID is None for MCP server, maybe testing in the config page.') self.ap.logger.warning('Server UUID is None for MCP server, maybe testing in the config page.')
uuid_ = str(uuid_module.uuid4()) uuid_ = str(uuid_module.uuid4())
server_config['uuid'] = uuid_ server_config['uuid'] = uuid_
# No persisted UUID => this is a throwaway "test" session from the
# config page. Isolate it from the shared live Box session.
is_transient = True
name = server_config['name'] name = server_config['name']
uuid = server_config['uuid'] uuid = server_config['uuid']
@@ -353,6 +535,7 @@ class MCPLoader(loader.ToolLoader):
'uuid': uuid, 'uuid': uuid,
'mode': mode, 'mode': mode,
'enable': enable, 'enable': enable,
'_transient': is_transient,
**extra_args, **extra_args,
} }
@@ -431,12 +614,13 @@ class MCPLoader(loader.ToolLoader):
"""获取所有服务器的信息""" """获取所有服务器的信息"""
info = {} info = {}
for server_name, session in self.sessions.items(): for server_name, session in self.sessions.items():
tools = session.get_tools()
info[server_name] = { info[server_name] = {
'name': server_name, 'name': server_name,
'mode': session.server_config.get('mode'), 'mode': session.server_config.get('mode'),
'enable': session.enable, 'enable': session.enable,
'tools_count': len(session.get_tools()), 'tools_count': len(tools),
'tool_names': [f.name for f in session.get_tools()], 'tool_names': [f.name for f in tools],
} }
return info return info
@@ -0,0 +1,381 @@
from __future__ import annotations
import enum
import asyncio
import os
import shutil
import shlex
from typing import TYPE_CHECKING, Any
import pydantic
from mcp import ClientSession
from mcp.client.websocket import websocket_client
from ....box.workspace import (
BoxWorkspaceSession,
classify_python_workspace,
infer_workspace_host_path,
normalize_host_path,
rewrite_mounted_path,
rewrite_venv_command,
unwrap_venv_path,
)
if TYPE_CHECKING:
from .mcp import RuntimeMCPSession
class MCPSessionErrorPhase(enum.Enum):
"""Which phase of the MCP lifecycle failed."""
SESSION_CREATE = 'session_create'
DEP_INSTALL = 'dep_install'
PROCESS_START = 'process_start'
RELAY_CONNECT = 'relay_connect'
MCP_INIT = 'mcp_init'
RUNTIME = 'runtime'
TOOL_CALL = 'tool_call'
# Stdio MCP refused because Box is disabled in config or currently
# unavailable. Not transient — retries would be pointless. The frontend
# uses this phase to render a localized actionable message instead of
# the raw RuntimeError text.
BOX_UNAVAILABLE = 'box_unavailable'
class MCPServerBoxConfig(pydantic.BaseModel):
"""Structured configuration for running an MCP server inside a Box container."""
image: str | None = None
network: str = 'on' # MCP servers need network for dependency installation
host_path: str | None = None
host_path_mode: str = 'ro' # MCP servers default to read-write mount only when explicitly requested
env: dict[str, str] = pydantic.Field(default_factory=dict)
startup_timeout_sec: int = 120 # Longer default to allow dependency bootstrap
cpus: float | None = None
memory_mb: int | None = None
pids_limit: int | None = None
read_only_rootfs: bool | None = None
model_config = pydantic.ConfigDict(extra='ignore')
class BoxStdioSessionRuntime:
"""Encapsulate Box-backed stdio MCP session orchestration."""
def __init__(self, owner: RuntimeMCPSession):
self.owner = owner
self.config = MCPServerBoxConfig.model_validate(owner.server_config.get('box', {}))
@property
def ap(self):
return self.owner.ap
@property
def server_name(self) -> str:
return self.owner.server_name
@property
def server_config(self) -> dict:
return self.owner.server_config
def _build_workspace(
self,
*,
host_path: str | None | object = ...,
workdir: str = '/workspace',
mount_path: str = '/workspace',
) -> BoxWorkspaceSession:
resolved_host_path = self.resolve_host_path() if host_path is ... else host_path
return BoxWorkspaceSession(
self.ap.box_service,
self.owner._build_box_session_id(),
host_path=resolved_host_path,
host_path_mode=self.config.host_path_mode,
workdir=workdir,
env=self.config.env,
mount_path=mount_path,
network=self.config.network,
read_only_rootfs=self.config.read_only_rootfs if self.config.read_only_rootfs is not None else False,
image=self.config.image,
cpus=self.config.cpus,
memory_mb=self.config.memory_mb,
pids_limit=self.config.pids_limit,
persistent=True,
)
@property
def process_id(self) -> str:
"""Each MCP server gets a unique process_id within the shared session."""
return self.owner.server_uuid
def uses_box_stdio(self) -> bool:
if self.server_config.get('mode') != 'stdio':
return False
box_service = getattr(self.ap, 'box_service', None)
if box_service is None:
return False
# When Box is configured but currently unavailable (disabled or
# connection failed), do NOT silently fall through to host-stdio —
# that would bypass the sandbox the operator asked for. The caller
# is expected to refuse the stdio MCP server with a clear error.
return bool(getattr(box_service, 'available', False))
async def initialize(self) -> None:
await self._wait_for_box_runtime()
# All stdio MCP servers share one Box session. Per-server host paths
# are staged into the shared workspace instead of becoming session
# mounts, because an existing Docker container cannot add bind mounts.
workspace = self._build_workspace(host_path=None)
host_path = self.resolve_host_path()
process_cwd = '/workspace'
try:
await workspace.create_session()
except Exception:
self.owner.error_phase = MCPSessionErrorPhase.SESSION_CREATE
raise
if host_path:
process_cwd = await self._stage_host_path_to_shared_workspace(host_path)
install_cmd = self.detect_install_command(host_path, process_cwd)
if install_cmd:
self.ap.logger.info(
f'MCP server {self.server_name}: installing dependencies in Box with: {install_cmd}'
)
try:
result = await workspace.execute_raw(
install_cmd,
workdir=process_cwd,
timeout_sec=self.config.startup_timeout_sec or 120,
)
except Exception:
self.owner.error_phase = MCPSessionErrorPhase.DEP_INSTALL
raise
if not result.ok:
self.owner.error_phase = MCPSessionErrorPhase.DEP_INSTALL
stderr_preview = (result.stderr or '')[:500]
raise Exception(f'Dependency install failed (exit code {result.exit_code}): {stderr_preview}')
try:
process_workspace = (
self._build_workspace(host_path=host_path, workdir=process_cwd, mount_path=process_cwd)
if host_path
else workspace
)
payload = process_workspace.build_process_payload(
self.server_config['command'],
self.server_config.get('args', []),
env=self.server_config.get('env', {}),
cwd=process_cwd,
)
payload['process_id'] = self.process_id
await workspace.box_service.start_managed_process(workspace.session_id, payload)
except Exception:
self.owner.error_phase = MCPSessionErrorPhase.PROCESS_START
raise
try:
websocket_url = workspace.get_managed_process_websocket_url(self.process_id)
transport = await self.owner.exit_stack.enter_async_context(websocket_client(websocket_url))
read_stream, write_stream = transport
self.owner.session = await self.owner.exit_stack.enter_async_context(
ClientSession(read_stream, write_stream)
)
except Exception:
self.owner.error_phase = MCPSessionErrorPhase.RELAY_CONNECT
raise
try:
await self.owner.session.initialize()
except Exception:
self.owner.error_phase = MCPSessionErrorPhase.MCP_INIT
raise
async def monitor_process_health(self) -> None:
from langbot_plugin.box.models import BoxManagedProcessStatus
workspace = self._build_workspace()
consecutive_errors = 0
while not self.owner._shutdown_event.is_set():
try:
info = await workspace.get_managed_process(self.process_id)
if isinstance(info, dict):
status = info.get('status', '')
else:
status = getattr(info, 'status', '')
if status == BoxManagedProcessStatus.EXITED.value or status == BoxManagedProcessStatus.EXITED:
return
consecutive_errors = 0
except Exception as exc:
consecutive_errors += 1
self.ap.logger.warning(
f'MCP monitor for {self.server_name}: get_managed_process failed '
f'({consecutive_errors}/{self.owner._MONITOR_MAX_CONSECUTIVE_ERRORS}): '
f'{type(exc).__name__}: {exc}'
)
if consecutive_errors >= self.owner._MONITOR_MAX_CONSECUTIVE_ERRORS:
return
await asyncio.sleep(self.owner._MONITOR_POLL_INTERVAL)
async def _stage_host_path_to_shared_workspace(self, host_path: str) -> str:
source_path = normalize_host_path(host_path)
if not source_path:
return '/workspace'
if not os.path.isdir(source_path):
raise FileNotFoundError(f'MCP host_path does not exist or is not a directory: {host_path}')
self._validate_host_path(source_path)
shared_host_path = self._shared_workspace_host_path()
process_host_root = os.path.join(shared_host_path, '.mcp', self.process_id)
process_host_workspace = os.path.join(process_host_root, 'workspace')
await asyncio.to_thread(self._copy_workspace_tree, source_path, process_host_root, process_host_workspace)
return f'/workspace/.mcp/{self.process_id}/workspace'
def _validate_host_path(self, host_path: str) -> None:
self.ap.box_service.build_spec(
{
'session_id': f'mcp-validate-{self.process_id}',
'host_path': host_path,
'host_path_mode': self.config.host_path_mode,
'network': self.config.network,
'read_only_rootfs': self.config.read_only_rootfs if self.config.read_only_rootfs is not None else False,
}
)
def _shared_workspace_host_path(self) -> str:
default_workspace = getattr(self.ap.box_service, 'default_workspace', None)
if not default_workspace:
raise RuntimeError('Box default workspace is required for shared MCP host_path staging')
shared_host_path = normalize_host_path(default_workspace)
os.makedirs(shared_host_path, exist_ok=True)
return shared_host_path
@staticmethod
def _copy_workspace_tree(source_path: str, process_host_root: str, process_host_workspace: str) -> None:
shutil.rmtree(process_host_root, ignore_errors=True)
os.makedirs(process_host_root, exist_ok=True)
shutil.copytree(
source_path,
process_host_workspace,
symlinks=True,
ignore=shutil.ignore_patterns('.git', '__pycache__', '.pytest_cache', '.mypy_cache', '.ruff_cache'),
)
async def _cleanup_staged_workspace(self) -> None:
if not self.resolve_host_path():
return
try:
process_host_root = os.path.join(self._shared_workspace_host_path(), '.mcp', self.process_id)
await asyncio.to_thread(shutil.rmtree, process_host_root, True)
except Exception as exc:
self.ap.logger.warning(
f'MCP server {self.server_name}: failed to clean staged workspace '
f'process_id={self.process_id}: {type(exc).__name__}: {exc}'
)
async def _wait_for_box_runtime(self) -> None:
timeout_sec = max(float(self.config.startup_timeout_sec or 120), 1.0)
deadline = asyncio.get_running_loop().time() + timeout_sec
warned = False
while not getattr(self.ap.box_service, 'available', False):
if not warned:
self.ap.logger.warning(
f'MCP server {self.server_name}: waiting for Box runtime before starting stdio process'
)
warned = True
if asyncio.get_running_loop().time() >= deadline:
self.owner.error_phase = MCPSessionErrorPhase.SESSION_CREATE
raise Exception(f'Box runtime is not available after {int(timeout_sec)} seconds')
await asyncio.sleep(1)
async def cleanup_session(self) -> None:
if not self.uses_box_stdio():
return
workspace = self._build_workspace(host_path=None)
# Transient test sessions own their isolated Box session, so tear the
# whole session down rather than leaking it. This cannot affect live
# servers because they live in the separate shared session.
if getattr(self.owner, 'is_transient', False):
try:
await workspace.cleanup()
except Exception as exc:
self.ap.logger.warning(
f'MCP server {self.server_name}: failed to delete transient test session '
f'{self.owner._build_box_session_id()}: {type(exc).__name__}: {exc}'
)
await self._cleanup_staged_workspace()
return
# In the shared-session model, we do not delete the session itself.
# Stop only this MCP server's managed process; deleting the session
# would kill other MCP servers sharing the same container.
try:
await workspace.stop_managed_process(self.process_id)
except Exception as exc:
self.ap.logger.warning(
f'MCP server {self.server_name}: failed to stop managed process '
f'process_id={self.process_id}: {type(exc).__name__}: {exc}'
)
await self._cleanup_staged_workspace()
return
await self._cleanup_staged_workspace()
self.ap.logger.info(
f'MCP server {self.server_name}: stopped process_id={self.process_id} '
f'(shared session {self.owner._build_box_session_id()} kept alive)'
)
def rewrite_path(self, path: str, host_path: str | None) -> str:
return rewrite_mounted_path(path, host_path)
def infer_host_path(self) -> str | None:
return infer_workspace_host_path(self.server_config.get('command', ''), self.server_config.get('args', []))
@staticmethod
def unwrap_venv_path(directory: str) -> str:
return unwrap_venv_path(directory)
def resolve_host_path(self) -> str | None:
return self.config.host_path or self.infer_host_path()
@staticmethod
def detect_install_command(host_path: str, workspace_path: str = '/workspace') -> str | None:
workspace_kind = classify_python_workspace(host_path)
quoted_workspace_path = shlex.quote(workspace_path)
if workspace_kind == 'package':
return (
'mkdir -p /opt/_lb_src'
f' && tar -C {quoted_workspace_path}'
' --exclude=.venv --exclude=.git --exclude=__pycache__'
' --exclude=node_modules --exclude=.tox --exclude=.nox'
' --exclude="*.egg-info" --exclude=.uv-cache'
' -cf - .'
' | tar -C /opt/_lb_src -xf -'
' && pip install --no-cache-dir /opt/_lb_src'
' && rm -rf /opt/_lb_src'
)
if workspace_kind == 'requirements':
return f'pip install --no-cache-dir -r {quoted_workspace_path}/requirements.txt'
return None
def build_box_session_payload(self, session_id: str, host_path: str | None = None) -> dict[str, Any]:
workspace = self._build_workspace()
workspace.session_id = session_id
if host_path is not None:
workspace.host_path = host_path
return workspace.build_session_payload()
def build_box_process_payload(self, host_path: str | None = None) -> dict[str, Any]:
workspace = self._build_workspace()
if host_path is not None:
workspace.host_path = host_path
return workspace.build_process_payload(
self.server_config['command'],
self.server_config.get('args', []),
env=self.server_config.get('env', {}),
)
def rewrite_venv_command(self, command: str, host_path: str) -> str:
return rewrite_venv_command(command, host_path)
@@ -0,0 +1,846 @@
from __future__ import annotations
import json
import os
import langbot_plugin.api.entities.builtin.resource.tool as resource_tool
from langbot_plugin.api.entities.events import pipeline_query
from .. import loader
from . import skill as skill_loader
EXEC_TOOL_NAME = 'exec'
READ_TOOL_NAME = 'read'
WRITE_TOOL_NAME = 'write'
EDIT_TOOL_NAME = 'edit'
GLOB_TOOL_NAME = 'glob'
GREP_TOOL_NAME = 'grep'
_ALL_TOOL_NAMES = {EXEC_TOOL_NAME, READ_TOOL_NAME, WRITE_TOOL_NAME, EDIT_TOOL_NAME, GLOB_TOOL_NAME, GREP_TOOL_NAME}
# Skip these dirs during grep walk to avoid noise
_SKIP_DIRS = {'.git', 'node_modules', '__pycache__', '.venv', 'venv', '.tox', 'dist', 'build'}
class NativeToolLoader(loader.ToolLoader):
def __init__(self, ap):
super().__init__(ap)
self._tools: list[resource_tool.LLMTool] | None = None
self._backend_available: bool | None = None
async def initialize(self):
"""Check if backend is truly available at startup."""
self._backend_available = await self._check_backend_available()
if self._backend_available:
self.ap.logger.info('Native sandbox tools (exec/read/write/edit/glob/grep) are available.')
else:
self.ap.logger.warning(
'Native sandbox tools (exec/read/write/edit/glob/grep) are NOT available. '
'No sandbox backend (Docker/nsjail/E2B) is ready. '
'The LLM will not have access to code execution or file operation tools.'
)
async def _check_backend_available(self) -> bool:
"""Check if the box backend is truly available (not just the runtime)."""
box_service = getattr(self.ap, 'box_service', None)
if box_service is None:
return False
if not getattr(box_service, 'available', False):
return False
# Check if backend is truly available via get_status
try:
status = await box_service.get_status()
backend_info = status.get('backend', {})
return backend_info.get('available', False)
except Exception:
return False
async def get_tools(self, bound_plugins: list[str] | None = None) -> list[resource_tool.LLMTool]:
if not self._is_sandbox_available():
return []
if self._tools is None:
self._tools = [
self._build_exec_tool(),
self._build_read_tool(),
self._build_write_tool(),
self._build_edit_tool(),
self._build_glob_tool(),
self._build_grep_tool(),
]
return list(self._tools)
async def has_tool(self, name: str) -> bool:
return name in _ALL_TOOL_NAMES and self._is_sandbox_available()
async def invoke_tool(self, name: str, parameters: dict, query: pipeline_query.Query):
if name == EXEC_TOOL_NAME:
self.ap.logger.info(
'exec tool invoked: '
f'query_id={query.query_id} '
f'parameters={json.dumps(self._summarize_parameters(parameters), ensure_ascii=False)}'
)
return await self._invoke_exec(parameters, query)
if name == READ_TOOL_NAME:
return await self._invoke_read(parameters, query)
if name == WRITE_TOOL_NAME:
return await self._invoke_write(parameters, query)
if name == EDIT_TOOL_NAME:
return await self._invoke_edit(parameters, query)
if name == GLOB_TOOL_NAME:
return await self._invoke_glob(parameters, query)
if name == GREP_TOOL_NAME:
return await self._invoke_grep(parameters, query)
raise ValueError(f'未找到工具: {name}')
async def shutdown(self):
pass
async def _invoke_exec(self, parameters: dict, query: pipeline_query.Query) -> dict:
command = str(parameters['command'])
workdir = str(parameters.get('workdir', '/workspace') or '/workspace')
# Validate that skill references target activated skills.
selected_skill, _ = skill_loader.resolve_virtual_skill_path(
self.ap,
query,
workdir,
include_visible=False,
include_activated=True,
)
referenced_skill_names = skill_loader.find_referenced_skill_names(command)
if selected_skill is None and referenced_skill_names:
if len(referenced_skill_names) > 1:
raise ValueError('exec can target at most one activated skill package per call.')
selected_skill = skill_loader.get_activated_skill(query, referenced_skill_names[0])
if selected_skill is None:
raise ValueError(
f'Skill "{referenced_skill_names[0]}" must be activated before exec can run in its package.'
)
if selected_skill is not None:
selected_skill_name = str(selected_skill.get('name', '') or '')
if referenced_skill_names and any(name != selected_skill_name for name in referenced_skill_names):
raise ValueError('exec can reference files from only one activated skill package per call.')
package_root = str(selected_skill.get('package_root', '') or '').strip()
if not package_root:
raise ValueError(f'Activated skill "{selected_skill_name}" has no package_root.')
# Wrap command with Python venv bootstrap if the skill has a Python project.
# The venv is created inside the skill's mount path.
skill_mount = f'/workspace/.skills/{selected_skill_name}'
if skill_loader.should_prepare_skill_python_env(package_root):
parameters = dict(parameters)
parameters['command'] = skill_loader.wrap_skill_command_with_python_env(command, mount_path=skill_mount)
# All exec calls (with or without skills) go through the same container
# via execute_tool. Skills are mounted at /workspace/.skills/{name}/
# via extra_mounts built by BoxService.
result = await self.ap.box_service.execute_tool(parameters, query)
if selected_skill is not None:
self._refresh_skill_from_disk(selected_skill)
return result
def _resolve_host_path(
self,
query: pipeline_query.Query,
sandbox_path: str,
*,
include_visible: bool,
include_activated: bool,
) -> tuple[str, dict | None]:
selected_skill, rewritten_path = skill_loader.resolve_virtual_skill_path(
self.ap,
query,
sandbox_path,
include_visible=include_visible,
include_activated=include_activated,
)
box_service = self.ap.box_service
host_root = selected_skill.get('package_root') if selected_skill is not None else box_service.default_workspace
if not host_root:
raise ValueError('No host workspace configured for file operations.')
mount_path = '/workspace'
if not rewritten_path.startswith(mount_path):
raise ValueError(f'Path must be under {mount_path}.')
relative = rewritten_path[len(mount_path) :].lstrip('/')
host_path = os.path.realpath(os.path.join(host_root, relative))
host_root = os.path.realpath(host_root)
if not (host_path == host_root or host_path.startswith(host_root + os.sep)):
raise ValueError('Path escapes the workspace boundary.')
return host_path, selected_skill
def _resolve_skill_relative_path(
self,
query: pipeline_query.Query,
sandbox_path: str,
*,
include_visible: bool,
include_activated: bool,
) -> tuple[dict, str] | None:
selected_skill, rewritten_path = skill_loader.resolve_virtual_skill_path(
self.ap,
query,
sandbox_path,
include_visible=include_visible,
include_activated=include_activated,
)
if selected_skill is None:
return None
mount_path = '/workspace'
if not rewritten_path.startswith(mount_path):
raise ValueError(f'Path must be under {mount_path}.')
relative = rewritten_path[len(mount_path) :].lstrip('/') or '.'
return selected_skill, relative
def _should_use_box_workspace_files(self, selected_skill: dict | None) -> bool:
if selected_skill is not None:
return False
box_service = getattr(self.ap, 'box_service', None)
if box_service is None or not hasattr(box_service, 'execute_tool'):
return False
default_workspace = getattr(box_service, 'default_workspace', None)
return bool(default_workspace and not os.path.isdir(os.path.realpath(default_workspace)))
async def _run_workspace_file_script(self, script: str, query: pipeline_query.Query) -> dict:
result = await self.ap.box_service.execute_tool(
{
'command': f"python - <<'PY'\n{script}\nPY",
'timeout_sec': 30,
},
query,
)
if not result.get('ok'):
return {'ok': False, 'error': result.get('stderr') or result.get('stdout') or 'Box execution failed'}
stdout = str(result.get('stdout') or '').strip()
try:
return json.loads(stdout.splitlines()[-1])
except Exception:
return {'ok': False, 'error': stdout or 'Box file operation returned no result'}
async def _read_workspace_via_box(self, path: str, query: pipeline_query.Query) -> dict:
script = f"""
import json, os
path = {json.dumps(path)}
if not path.startswith('/workspace'):
print(json.dumps({{'ok': False, 'error': 'Path must be under /workspace.'}}))
elif not os.path.exists(path):
print(json.dumps({{'ok': False, 'error': f'File not found: {{path}}'}}))
elif os.path.isdir(path):
print(json.dumps({{'ok': True, 'content': '\\n'.join(sorted(os.listdir(path))), 'is_directory': True}}))
else:
with open(path, 'r', encoding='utf-8', errors='replace') as f:
print(json.dumps({{'ok': True, 'content': f.read()}}))
""".strip()
return await self._run_workspace_file_script(script, query)
async def _write_workspace_via_box(self, path: str, content: str, query: pipeline_query.Query) -> dict:
script = f"""
import json, os
path = {json.dumps(path)}
content = {json.dumps(content)}
if not path.startswith('/workspace'):
print(json.dumps({{'ok': False, 'error': 'Path must be under /workspace.'}}))
else:
os.makedirs(os.path.dirname(path) or '/workspace', exist_ok=True)
with open(path, 'w', encoding='utf-8') as f:
f.write(content)
print(json.dumps({{'ok': True, 'path': path}}))
""".strip()
return await self._run_workspace_file_script(script, query)
async def _edit_workspace_via_box(
self,
path: str,
old_string: str,
new_string: str,
query: pipeline_query.Query,
) -> dict:
script = f"""
import json, os
path = {json.dumps(path)}
old_string = {json.dumps(old_string)}
new_string = {json.dumps(new_string)}
if not path.startswith('/workspace'):
print(json.dumps({{'ok': False, 'error': 'Path must be under /workspace.'}}))
elif not os.path.isfile(path):
print(json.dumps({{'ok': False, 'error': f'File not found: {{path}}'}}))
else:
with open(path, 'r', encoding='utf-8', errors='replace') as f:
content = f.read()
count = content.count(old_string)
if count == 0:
print(json.dumps({{'ok': False, 'error': 'old_string not found in file.'}}))
elif count > 1:
print(json.dumps({{'ok': False, 'error': f'old_string matches {{count}} locations; provide a more unique string.'}}))
else:
with open(path, 'w', encoding='utf-8') as f:
f.write(content.replace(old_string, new_string, 1))
print(json.dumps({{'ok': True, 'path': path}}))
""".strip()
return await self._run_workspace_file_script(script, query)
async def _glob_workspace_via_box(self, path: str, pattern: str, query: pipeline_query.Query) -> dict:
script = f"""
import json, os
from pathlib import Path
path = {json.dumps(path)}
pattern = {json.dumps(pattern)}
skip_dirs = {json.dumps(sorted(_SKIP_DIRS))}
if not path.startswith('/workspace'):
print(json.dumps({{'ok': False, 'error': 'Path must be under /workspace.'}}))
elif not os.path.isdir(path):
print(json.dumps({{'ok': False, 'error': f'Path is not a directory: {{path}}'}}))
else:
base = Path(path)
hits = [
item for item in base.rglob(pattern)
if not any(part in skip_dirs for part in item.parts)
]
hits.sort(key=lambda item: item.stat().st_mtime if item.exists() else 0, reverse=True)
shown = hits[:100]
matches = []
for item in shown:
rel = os.path.relpath(str(item), path)
matches.append(os.path.join(path, rel).replace(os.sep, '/'))
print(json.dumps({{'ok': True, 'matches': matches, 'total': len(hits), 'truncated': len(hits) > 100}}))
""".strip()
return await self._run_workspace_file_script(script, query)
async def _grep_workspace_via_box(
self,
path: str,
pattern: str,
include: str | None,
query: pipeline_query.Query,
) -> dict:
script = f"""
import json, os, re
from pathlib import Path
path = {json.dumps(path)}
pattern = {json.dumps(pattern)}
include = {json.dumps(include)}
skip_dirs = {json.dumps(sorted(_SKIP_DIRS))}
try:
regex = re.compile(pattern)
except re.error as exc:
print(json.dumps({{'ok': False, 'error': f'Invalid regex: {{exc}}'}}))
else:
if not path.startswith('/workspace'):
print(json.dumps({{'ok': False, 'error': 'Path must be under /workspace.'}}))
elif not os.path.exists(path):
print(json.dumps({{'ok': False, 'error': f'Path not found: {{path}}'}}))
else:
base = Path(path)
if base.is_file():
files = [base]
else:
files = []
for item in base.rglob(include or '*'):
if any(part in skip_dirs for part in item.parts):
continue
if item.is_file():
files.append(item)
if len(files) >= 5000:
break
matches = []
for fp in files:
try:
text = fp.read_text(errors='ignore')
except OSError:
continue
for lineno, line in enumerate(text.splitlines(), 1):
if regex.search(line):
if base.is_file():
file_path = path
else:
rel = os.path.relpath(str(fp), path)
file_path = os.path.join(path, rel).replace(os.sep, '/')
matches.append({{'file': file_path, 'line': lineno, 'content': line.rstrip()}})
if len(matches) >= 200:
break
if len(matches) >= 200:
break
print(json.dumps({{'ok': True, 'matches': matches, 'total': len(matches), 'truncated': len(matches) >= 200}}))
""".strip()
return await self._run_workspace_file_script(script, query)
async def _invoke_read(self, parameters: dict, query: pipeline_query.Query) -> dict:
path = parameters['path']
self.ap.logger.info(f'read tool invoked: query_id={query.query_id} path={path}')
skill_request = self._resolve_skill_relative_path(
query,
path,
include_visible=True,
include_activated=True,
)
if skill_request is not None and hasattr(self.ap.box_service, 'read_skill_file'):
selected_skill, relative = skill_request
try:
result = await self.ap.box_service.read_skill_file(selected_skill['name'], relative)
return {'ok': True, 'content': result.get('content', '')}
except Exception:
try:
result = await self.ap.box_service.list_skill_files(selected_skill['name'], relative)
entries = [entry['name'] for entry in result.get('entries', [])]
return {'ok': True, 'content': '\n'.join(sorted(entries)), 'is_directory': True}
except Exception as exc:
return {'ok': False, 'error': str(exc)}
host_path, selected_skill = self._resolve_host_path(
query,
path,
include_visible=True,
include_activated=True,
)
if self._should_use_box_workspace_files(selected_skill):
return await self._read_workspace_via_box(path, query)
if not os.path.exists(host_path):
return {'ok': False, 'error': f'File not found: {path}'}
if os.path.isdir(host_path):
entries = os.listdir(host_path)
return {'ok': True, 'content': '\n'.join(sorted(entries)), 'is_directory': True}
with open(host_path, 'r', errors='replace') as f:
content = f.read()
return {'ok': True, 'content': content}
async def _invoke_write(self, parameters: dict, query: pipeline_query.Query) -> dict:
path = parameters['path']
content = parameters['content']
self.ap.logger.info(f'write tool invoked: query_id={query.query_id} path={path} length={len(content)}')
skill_request = self._resolve_skill_relative_path(
query,
path,
include_visible=False,
include_activated=True,
)
if skill_request is not None and hasattr(self.ap.box_service, 'write_skill_file'):
selected_skill, relative = skill_request
await self.ap.box_service.write_skill_file(selected_skill['name'], relative, content)
await self.ap.skill_mgr.reload_skills()
return {'ok': True, 'path': path}
host_path, selected_skill = self._resolve_host_path(
query,
path,
include_visible=False,
include_activated=True,
)
if self._should_use_box_workspace_files(selected_skill):
return await self._write_workspace_via_box(path, content, query)
os.makedirs(os.path.dirname(host_path), exist_ok=True)
with open(host_path, 'w', encoding='utf-8') as f:
f.write(content)
self._refresh_skill_from_disk(selected_skill)
return {'ok': True, 'path': path}
async def _invoke_edit(self, parameters: dict, query: pipeline_query.Query) -> dict:
path = parameters['path']
old_string = parameters['old_string']
new_string = parameters['new_string']
self.ap.logger.info(
f'edit tool invoked: query_id={query.query_id} path={path} '
f'old_len={len(old_string)} new_len={len(new_string)}'
)
skill_request = self._resolve_skill_relative_path(
query,
path,
include_visible=False,
include_activated=True,
)
if (
skill_request is not None
and hasattr(self.ap.box_service, 'read_skill_file')
and hasattr(self.ap.box_service, 'write_skill_file')
):
selected_skill, relative = skill_request
try:
result = await self.ap.box_service.read_skill_file(selected_skill['name'], relative)
except Exception:
return {'ok': False, 'error': f'File not found: {path}'}
content = result.get('content', '')
count = content.count(old_string)
if count == 0:
return {'ok': False, 'error': 'old_string not found in file.'}
if count > 1:
return {'ok': False, 'error': f'old_string matches {count} locations; provide a more unique string.'}
new_content = content.replace(old_string, new_string, 1)
await self.ap.box_service.write_skill_file(selected_skill['name'], relative, new_content)
await self.ap.skill_mgr.reload_skills()
return {'ok': True, 'path': path}
host_path, selected_skill = self._resolve_host_path(
query,
path,
include_visible=False,
include_activated=True,
)
if self._should_use_box_workspace_files(selected_skill):
return await self._edit_workspace_via_box(path, old_string, new_string, query)
if not os.path.isfile(host_path):
return {'ok': False, 'error': f'File not found: {path}'}
with open(host_path, 'r', encoding='utf-8', errors='replace') as f:
content = f.read()
count = content.count(old_string)
if count == 0:
return {'ok': False, 'error': 'old_string not found in file.'}
if count > 1:
return {'ok': False, 'error': f'old_string matches {count} locations; provide a more unique string.'}
new_content = content.replace(old_string, new_string, 1)
with open(host_path, 'w', encoding='utf-8') as f:
f.write(new_content)
self._refresh_skill_from_disk(selected_skill)
return {'ok': True, 'path': path}
def _refresh_skill_from_disk(self, selected_skill: dict | None) -> None:
if selected_skill is None:
return
skill_mgr = getattr(self.ap, 'skill_mgr', None)
if skill_mgr is None:
return
refresh_skill = getattr(skill_mgr, 'refresh_skill_from_disk', None)
if callable(refresh_skill):
refresh_skill(selected_skill.get('name', ''))
def _is_sandbox_available(self) -> bool:
"""Check if sandbox backend is available.
This checks the cached backend availability from initialization,
not just whether the box_service process is running.
"""
return bool(self._backend_available)
def _build_exec_tool(self) -> resource_tool.LLMTool:
return resource_tool.LLMTool(
name=EXEC_TOOL_NAME,
human_desc='Execute a command in an isolated environment',
description=(
'Run shell commands in an isolated execution environment. '
'Use this tool for bash commands, Python execution, and exact calculations over '
'user-provided data. Activated skill packages are addressable under '
'/workspace/.skills/<skill-name>; when running inside one, set workdir to that path. '
'To create a new skill package, prepare it under /workspace first, then use register_skill.'
),
parameters={
'type': 'object',
'properties': {
'command': {
'type': 'string',
'description': 'Shell command to execute.',
},
'workdir': {
'type': 'string',
'description': 'Working directory for the command. Defaults to /workspace.',
'default': '/workspace',
},
'timeout_sec': {
'type': 'integer',
'description': 'Execution timeout in seconds. Defaults to 30.',
'default': 30,
'minimum': 1,
},
'env': {
'type': 'object',
'description': 'Optional environment variables for the execution.',
'additionalProperties': {'type': 'string'},
'default': {},
},
'description': {
'type': 'string',
'description': 'Brief description of what this command does, for logging and audit.',
},
},
'required': ['command'],
'additionalProperties': False,
},
func=lambda parameters: parameters,
)
def _build_read_tool(self) -> resource_tool.LLMTool:
return resource_tool.LLMTool(
name=READ_TOOL_NAME,
human_desc='Read a file from the workspace',
description=(
'Read the contents of a file at the given path under /workspace. '
'Visible skill packages can be inspected through /workspace/.skills/<skill-name>/... .'
),
parameters={
'type': 'object',
'properties': {
'path': {
'type': 'string',
'description': 'Absolute path to the file (must be under /workspace).',
},
},
'required': ['path'],
'additionalProperties': False,
},
func=lambda parameters: parameters,
)
def _build_write_tool(self) -> resource_tool.LLMTool:
return resource_tool.LLMTool(
name=WRITE_TOOL_NAME,
human_desc='Write a file to the workspace',
description=(
'Create or overwrite a file at the given path under /workspace with the provided content. '
'Activated skill packages can be modified through /workspace/.skills/<skill-name>/... . '
'For new skills, write files under /workspace and then call register_skill.'
),
parameters={
'type': 'object',
'properties': {
'path': {
'type': 'string',
'description': 'Absolute path to the file (must be under /workspace).',
},
'content': {
'type': 'string',
'description': 'Content to write to the file.',
},
},
'required': ['path', 'content'],
'additionalProperties': False,
},
func=lambda parameters: parameters,
)
def _build_edit_tool(self) -> resource_tool.LLMTool:
return resource_tool.LLMTool(
name=EDIT_TOOL_NAME,
human_desc='Edit a file in the workspace',
description=(
'Perform an exact string replacement in a file under /workspace. '
'The old_string must appear exactly once in the file. Activated skill packages '
'can be edited through /workspace/.skills/<skill-name>/... . '
'For new skills, edit files under /workspace and then call register_skill.'
),
parameters={
'type': 'object',
'properties': {
'path': {
'type': 'string',
'description': 'Absolute path to the file (must be under /workspace).',
},
'old_string': {
'type': 'string',
'description': 'The exact string to find and replace.',
},
'new_string': {
'type': 'string',
'description': 'The replacement string.',
},
},
'required': ['path', 'old_string', 'new_string'],
'additionalProperties': False,
},
func=lambda parameters: parameters,
)
def _build_glob_tool(self) -> resource_tool.LLMTool:
return resource_tool.LLMTool(
name=GLOB_TOOL_NAME,
human_desc='Find files matching a glob pattern',
description=(
'Find files matching a glob pattern under /workspace. '
'Supports ** for recursive matching (e.g. **/*.py). '
'Results are sorted by modification time (newest first). '
'Visible and activated skill packages can be searched through /workspace/.skills/<skill-name>/...'
),
parameters={
'type': 'object',
'properties': {
'pattern': {
'type': 'string',
'description': 'Glob pattern, e.g. **/*.py or src/**/*.ts',
},
'path': {
'type': 'string',
'description': 'Directory to search in (must be under /workspace, default: /workspace)',
'default': '/workspace',
},
},
'required': ['pattern'],
'additionalProperties': False,
},
func=lambda parameters: parameters,
)
def _build_grep_tool(self) -> resource_tool.LLMTool:
return resource_tool.LLMTool(
name=GREP_TOOL_NAME,
human_desc='Search file contents with regex',
description=(
'Search file contents with regex pattern under /workspace. '
'Returns matching lines with file path and line number. '
'Visible and activated skill packages can be searched through /workspace/.skills/<skill-name>/...'
),
parameters={
'type': 'object',
'properties': {
'pattern': {
'type': 'string',
'description': 'Regex pattern to search for',
},
'path': {
'type': 'string',
'description': 'File or directory to search (must be under /workspace, default: /workspace)',
'default': '/workspace',
},
'include': {
'type': 'string',
'description': 'Only search files matching this glob (e.g. *.py)',
},
},
'required': ['pattern'],
'additionalProperties': False,
},
func=lambda parameters: parameters,
)
async def _invoke_glob(self, parameters: dict, query: pipeline_query.Query) -> dict:
pattern = parameters['pattern']
path = str(parameters.get('path', '/workspace') or '/workspace')
self.ap.logger.info(f'glob tool invoked: query_id={query.query_id} pattern={pattern} path={path}')
host_path, selected_skill = self._resolve_host_path(
query,
path,
include_visible=True,
include_activated=True,
)
if self._should_use_box_workspace_files(selected_skill):
return await self._glob_workspace_via_box(path, pattern, query)
if not os.path.isdir(host_path):
return {'ok': False, 'error': f'Path is not a directory: {path}'}
from pathlib import Path
base = Path(host_path)
hits = list(base.rglob(pattern))
# Filter out skipped directories
hits = [h for h in hits if not any(skip in h.parts for skip in _SKIP_DIRS)]
# Sort by mtime, newest first
hits.sort(key=lambda p: p.stat().st_mtime if p.exists() else 0, reverse=True)
total = len(hits)
shown = hits[:100]
# Convert back to sandbox paths
sandbox_paths = []
for h in shown:
rel = os.path.relpath(str(h), host_path)
sandbox_path = os.path.join(path, rel)
sandbox_paths.append(sandbox_path)
result_lines = sandbox_paths
result = '\n'.join(result_lines)
if total > 100:
result += f'\n... ({total} matches, showing first 100)'
return {'ok': True, 'matches': result_lines, 'total': total, 'truncated': total > 100}
async def _invoke_grep(self, parameters: dict, query: pipeline_query.Query) -> dict:
pattern = parameters['pattern']
path = str(parameters.get('path', '/workspace') or '/workspace')
include = parameters.get('include')
self.ap.logger.info(f'grep tool invoked: query_id={query.query_id} pattern={pattern} path={path}')
import re
from pathlib import Path
try:
regex = re.compile(pattern)
except re.error as e:
return {'ok': False, 'error': f'Invalid regex: {e}'}
host_path, selected_skill = self._resolve_host_path(
query,
path,
include_visible=True,
include_activated=True,
)
if self._should_use_box_workspace_files(selected_skill):
return await self._grep_workspace_via_box(path, pattern, include, query)
if not os.path.exists(host_path):
return {'ok': False, 'error': f'Path not found: {path}'}
base = Path(host_path)
if base.is_file():
files = [base]
else:
files = self._grep_walk(base, include)
matches = []
for fp in files:
try:
text = fp.read_text(errors='ignore')
except OSError:
continue
for lineno, line in enumerate(text.splitlines(), 1):
if regex.search(line):
rel = os.path.relpath(str(fp), host_path)
sandbox_path = os.path.join(path, rel)
matches.append(
{
'file': sandbox_path,
'line': lineno,
'content': line.rstrip(),
}
)
if len(matches) >= 200:
break
if len(matches) >= 200:
break
return {
'ok': True,
'matches': matches,
'total': len(matches),
'truncated': len(matches) >= 200,
}
@staticmethod
def _grep_walk(root, include: str | None) -> list:
"""Walk dir tree for grep, skipping junk dirs."""
results = []
for item in root.rglob(include or '*'):
if any(skip in item.parts for skip in _SKIP_DIRS):
continue
if item.is_file():
results.append(item)
if len(results) >= 5000:
break
return results
def _summarize_parameters(self, parameters: dict) -> dict:
summary = dict(parameters)
cmd = str(summary.get('command', '')).strip()
if len(cmd) > 400:
cmd = f'{cmd[:397]}...'
summary['command'] = cmd
env = summary.get('env')
if isinstance(env, dict):
summary['env_keys'] = sorted(str(key) for key in env.keys())
del summary['env']
return summary
@@ -0,0 +1,157 @@
from __future__ import annotations
import re
import typing
from ....box import workspace as box_workspace
if typing.TYPE_CHECKING:
from ....core import app
from langbot_plugin.api.entities.events import pipeline_query
ACTIVATED_SKILLS_KEY = '_activated_skills'
PIPELINE_BOUND_SKILLS_KEY = '_pipeline_bound_skills'
SKILL_MOUNT_PREFIX = '/workspace/.skills'
_SKILL_MOUNT_PATTERN = re.compile(r'/workspace/\.skills/([A-Za-z0-9_-]+)')
def get_virtual_skill_mount_path(skill_name: str) -> str:
return f'{SKILL_MOUNT_PREFIX}/{skill_name}'
def get_bound_skill_names(query: pipeline_query.Query) -> list[str] | None:
if query.variables is None:
return None
bound_skills = query.variables.get(PIPELINE_BOUND_SKILLS_KEY)
if bound_skills is None:
return None
if isinstance(bound_skills, list):
return [str(item) for item in bound_skills]
return None
def get_visible_skills(ap: app.Application, query: pipeline_query.Query) -> dict[str, dict]:
skill_mgr = getattr(ap, 'skill_mgr', None)
if skill_mgr is None:
return {}
visible_skills = getattr(skill_mgr, 'skills', {})
bound_skills = get_bound_skill_names(query)
if bound_skills is None:
return visible_skills
return {skill_name: skill_data for skill_name, skill_data in visible_skills.items() if skill_name in bound_skills}
def get_visible_skill(ap: app.Application, query: pipeline_query.Query, skill_name: str) -> dict | None:
return get_visible_skills(ap, query).get(skill_name)
def get_activated_skills(query: pipeline_query.Query) -> dict[str, dict]:
if query.variables is None:
return {}
activated = query.variables.get(ACTIVATED_SKILLS_KEY, {})
if not isinstance(activated, dict):
return {}
return activated
def get_activated_skill(query: pipeline_query.Query, skill_name: str) -> dict | None:
return get_activated_skills(query).get(skill_name)
def register_activated_skill(query: pipeline_query.Query, skill_data: dict) -> None:
if query.variables is None:
query.variables = {}
activated = query.variables.setdefault(ACTIVATED_SKILLS_KEY, {})
skill_name = str(skill_data.get('name', '') or '').strip()
if skill_name and skill_name not in activated:
activated[skill_name] = skill_data
def parse_skill_mount_path(sandbox_path: str) -> tuple[str | None, str]:
normalized_path = str(sandbox_path or '/workspace').strip() or '/workspace'
if normalized_path == SKILL_MOUNT_PREFIX:
raise ValueError(f'Path must include a skill name under {SKILL_MOUNT_PREFIX}/<skill-name>.')
prefix = f'{SKILL_MOUNT_PREFIX}/'
if not normalized_path.startswith(prefix):
return None, normalized_path
remainder = normalized_path[len(prefix) :]
skill_name, separator, tail = remainder.partition('/')
if not skill_name:
raise ValueError(f'Path must include a skill name under {SKILL_MOUNT_PREFIX}/<skill-name>.')
rewritten_path = '/workspace'
if separator:
rewritten_path = f'/workspace/{tail}'
return skill_name, rewritten_path
def resolve_virtual_skill_path(
ap: app.Application,
query: pipeline_query.Query,
sandbox_path: str,
*,
include_visible: bool,
include_activated: bool,
) -> tuple[dict | None, str]:
skill_name, rewritten_path = parse_skill_mount_path(sandbox_path)
if skill_name is None:
return None, rewritten_path
if include_activated:
activated_skill = get_activated_skill(query, skill_name)
if activated_skill is not None:
return activated_skill, rewritten_path
if include_visible:
visible_skill = get_visible_skill(ap, query, skill_name)
if visible_skill is not None:
return visible_skill, rewritten_path
activated_names = ', '.join(sorted(get_activated_skills(query).keys())) or 'none'
visible_names = ', '.join(sorted(get_visible_skills(ap, query).keys())) or 'none'
raise ValueError(
f'Skill "{skill_name}" is not available at this path. '
f'Activated skills: {activated_names}. Visible skills: {visible_names}.'
)
def find_referenced_skill_names(text: str) -> list[str]:
if not text:
return []
seen: list[str] = []
for match in _SKILL_MOUNT_PATTERN.findall(text):
if match not in seen:
seen.append(match)
return seen
def rewrite_command_for_skill_mount(command: str, skill_name: str) -> str:
virtual_root = get_virtual_skill_mount_path(skill_name)
rewritten = command.replace(f'{virtual_root}/', '/workspace/')
return rewritten.replace(virtual_root, '/workspace')
def build_skill_session_id(skill_data: dict, query: pipeline_query.Query) -> str:
skill_identifier = str(skill_data.get('name', 'unknown') or 'unknown')
launcher_type = getattr(query, 'launcher_type', None)
launcher_id = getattr(query, 'launcher_id', None)
query_id = getattr(query, 'query_id', 'unknown')
if launcher_type is not None and launcher_id is not None:
return f'skill-{launcher_type}_{launcher_id}-{skill_identifier}'
return f'skill-{query_id}-{skill_identifier}'
def should_prepare_skill_python_env(package_root: str | None) -> bool:
return box_workspace.should_prepare_python_env(package_root)
def wrap_skill_command_with_python_env(command: str, *, mount_path: str = '/workspace') -> str:
return box_workspace.wrap_python_command_with_env(command, mount_path=mount_path).rstrip()
@@ -0,0 +1,304 @@
from __future__ import annotations
import os
import typing
import langbot_plugin.api.entities.builtin.resource.tool as resource_tool
from .. import loader
# Align with Claude Code's Skill tool design:
# - activate: Activate a skill via Tool Call, returns SKILL.md content
# - register_skill: Register a skill from sandbox directory to data/skills/
# - This protects KV Cache and follows industry standard
ACTIVATE_SKILL_TOOL_NAME = 'activate'
REGISTER_SKILL_TOOL_NAME = 'register_skill'
SKILL_TOOL_NAMES = {
ACTIVATE_SKILL_TOOL_NAME,
REGISTER_SKILL_TOOL_NAME,
}
class SkillToolLoader(loader.ToolLoader):
"""Skill tools aligned with Claude Code's design."""
def __init__(self, ap):
super().__init__(ap)
self._tools: list[resource_tool.LLMTool] = []
self._sandbox_available: bool = False
async def initialize(self):
# Check if sandbox backend is available (same check as native tools)
self._sandbox_available = await self._check_sandbox_available()
if self._sandbox_available:
self._tools = [
self._build_activate_skill_tool(),
self._build_register_skill_tool(),
]
else:
self.ap.logger.info(
'Skill tools (activate/register_skill) are NOT available. '
'No sandbox backend (Docker/nsjail/E2B) is ready.'
)
async def _check_sandbox_available(self) -> bool:
"""Check if the box backend is truly available (not just the runtime)."""
box_service = getattr(self.ap, 'box_service', None)
if box_service is None:
return False
if not getattr(box_service, 'available', False):
return False
# Check if backend is truly available via get_status
try:
status = await box_service.get_status()
backend_info = status.get('backend', {})
return backend_info.get('available', False)
except Exception:
return False
async def get_tools(self, bound_plugins: list[str] | None = None) -> list[resource_tool.LLMTool]:
if not self._is_available():
return []
return list(self._tools)
async def has_tool(self, name: str) -> bool:
return self._is_available() and name in SKILL_TOOL_NAMES
def _is_available(self) -> bool:
"""Check if skill tools should be available.
Skill tools require both a skill manager and a sandbox backend.
"""
return self._has_skill_manager() and self._sandbox_available
async def invoke_tool(self, name: str, parameters: dict, query) -> typing.Any:
if name == ACTIVATE_SKILL_TOOL_NAME:
return await self._invoke_activate_skill(parameters, query)
if name == REGISTER_SKILL_TOOL_NAME:
return await self._invoke_register_skill(parameters)
raise ValueError(f'Unknown skill tool: {name}')
async def shutdown(self):
pass
def _has_skill_manager(self) -> bool:
return getattr(self.ap, 'skill_mgr', None) is not None
async def _invoke_activate_skill(self, parameters: dict, query) -> typing.Any:
"""Activate a skill and return SKILL.md content via Tool Result."""
skill_name = str(parameters.get('skill_name', '') or '').strip()
if not skill_name:
raise ValueError('skill_name is required')
skill_mgr = self.ap.skill_mgr
skill_data = skill_mgr.get_skill_by_name(skill_name)
if skill_data is None:
visible_skills = getattr(skill_mgr, 'skills', {})
available_names = ', '.join(sorted(visible_skills.keys())) or 'none'
raise ValueError(f'Skill "{skill_name}" not found. Available skills: {available_names}')
# Register activated skill for sandbox mount path resolution
from . import skill as skill_loader
skill_loader.register_activated_skill(query, skill_data)
# Return SKILL.md content as Tool Result (injects into context)
instructions = skill_data.get('instructions', '')
package_root = skill_data.get('package_root', '')
mount_path = skill_loader.get_virtual_skill_mount_path(skill_name)
# Build Tool Result content
result_content = f'<command-message>The "{skill_name}" skill is activated</command-message>\n'
result_content += '<skill-activation>\n'
result_content += f'<skill-name>{skill_name}</skill-name>\n'
result_content += f'<mount-path>{mount_path}</mount-path>\n'
result_content += f'<package-root>{package_root}</package-root>\n'
result_content += f'\n## Instructions\n{instructions}\n'
result_content += '\n## Runtime Context\n'
result_content += f'The skill package is mounted at {mount_path}. Use the standard tools to interact with it:\n'
result_content += f'- Use `read` to inspect files under {mount_path}\n'
result_content += f'- Use `exec` with workdir set to {mount_path} to run commands in that package\n'
result_content += '- Use `write` and `edit` on that path when the instructions require updating files\n'
result_content += '</skill-activation>\n'
return {
'activated': True,
'skill_name': skill_name,
'mount_path': mount_path,
'content': result_content,
}
async def _invoke_register_skill(self, parameters: dict) -> typing.Any:
"""Register a skill from sandbox directory to data/skills/."""
sandbox_path = str(parameters.get('path', '') or '').strip()
if not sandbox_path:
raise ValueError('path is required')
# Resolve sandbox path to host path
host_path = self._resolve_workspace_directory(sandbox_path)
# Get or create skill service
skill_service = getattr(self.ap, 'skill_service', None)
if skill_service is None:
raise ValueError('Skill service not available')
# Scan and register the skill
scanned = await skill_service.scan_directory_async(host_path)
# Override name if provided
skill_name = str(parameters.get('name') or scanned['name']).strip()
if not skill_name:
raise ValueError('skill name is required')
# Create the skill
created = await skill_service.create_skill(
{
'name': skill_name,
'display_name': str(parameters.get('display_name') or scanned.get('display_name', '')).strip(),
'description': str(parameters.get('description') or scanned.get('description', '')).strip(),
'instructions': str(parameters.get('instructions') or scanned.get('instructions', '')),
'package_root': host_path,
}
)
return {
'registered': True,
'skill_name': skill_name,
'source_path': sandbox_path,
'skill': created,
}
def _resolve_workspace_directory(self, sandbox_path: str) -> str:
"""Resolve sandbox path to host filesystem path."""
box_service = getattr(self.ap, 'box_service', None)
workspace_root = getattr(box_service, 'default_workspace', None)
if not workspace_root:
raise ValueError('No default workspace configured')
normalized_path = str(sandbox_path).strip() or '/workspace'
if not normalized_path.startswith('/workspace'):
raise ValueError('path must be under /workspace')
relative = normalized_path[len('/workspace') :].lstrip('/')
host_root = os.path.realpath(workspace_root)
host_path = os.path.realpath(os.path.join(host_root, relative))
# Security check: ensure path doesn't escape workspace
if not (host_path == host_root or host_path.startswith(host_root + os.sep)):
raise ValueError('path escapes the workspace boundary')
if getattr(box_service, 'available', False):
return host_path
if not os.path.isdir(host_path):
raise ValueError(f'Directory does not exist: {sandbox_path}')
return host_path
def _build_activate_skill_tool(self) -> resource_tool.LLMTool:
return resource_tool.LLMTool(
name=ACTIVATE_SKILL_TOOL_NAME,
human_desc='Activate a skill',
description=self._build_activate_tool_description(),
parameters={
'type': 'object',
'properties': {
'skill_name': {
'type': 'string',
'description': 'The skill name to activate (no arguments). E.g., "pdf" or "data-analysis"',
},
},
'required': ['skill_name'],
'additionalProperties': False,
},
func=lambda parameters: parameters,
)
def _build_register_skill_tool(self) -> resource_tool.LLMTool:
return resource_tool.LLMTool(
name=REGISTER_SKILL_TOOL_NAME,
human_desc='Register a skill from sandbox',
description=(
"Register a skill package from a directory under /workspace into LangBot's skill store. "
'Use this after creating or preparing a skill in the sandbox with exec/read/write/edit. '
'The directory must contain a SKILL.md file. '
'After registration, the skill can be activated with the activate tool.'
),
parameters={
'type': 'object',
'properties': {
'path': {
'type': 'string',
'description': 'Directory path under /workspace containing the skill package (must have SKILL.md)',
},
'name': {
'type': 'string',
'description': 'Optional skill name override. Defaults to the name in SKILL.md or directory name.',
},
'display_name': {
'type': 'string',
'description': 'Optional display name override.',
},
'description': {
'type': 'string',
'description': 'Optional description override.',
},
'instructions': {
'type': 'string',
'description': 'Optional instructions override.',
},
},
'required': ['path'],
'additionalProperties': False,
},
func=lambda parameters: parameters,
)
def _build_activate_tool_description(self) -> str:
"""Build tool description with embedded available_skills list."""
skill_mgr = getattr(self.ap, 'skill_mgr', None)
if skill_mgr is None:
return 'Activate a skill. No skills are currently available.'
skills = getattr(skill_mgr, 'skills', {})
if not skills:
return 'Activate a skill. No skills are currently available.'
# Build <available_skills> section
available_skills_lines = ['<available_skills>']
for skill_name, skill_data in sorted(skills.items()):
description = skill_data.get('description', '')
available_skills_lines.append('<skill>')
available_skills_lines.append(f'<name>{skill_name}</name>')
available_skills_lines.append(f'<description>{description}</description>')
available_skills_lines.append('</skill>')
available_skills_lines.append('</available_skills>')
available_skills_block = '\n'.join(available_skills_lines)
return f"""Activate a skill within the main conversation.
<skills_instructions>
When users ask you to perform tasks, check if any of the available skills
below can help complete the task more effectively. Skills provide specialized
capabilities and domain knowledge.
How to use skills:
- Invoke skills using this tool with the skill name only (no arguments)
- When you invoke a skill, you will see <command-message>
The skill is activated
</command-message>
- The skill's instructions will be provided in the tool result
- Examples:
- skill_name: "pdf" - invoke the pdf skill
- skill_name: "data-analysis" - invoke the data-analysis skill
Important:
- Only use skills listed in <available_skills> below
- Do not invoke a skill that is already running
- To create a new skill: prepare it in /workspace, then use register_skill tool
</skills_instructions>
{available_skills_block}"""
+42 -36
View File
@@ -1,15 +1,19 @@
from __future__ import annotations from __future__ import annotations
import typing import typing
from typing import TYPE_CHECKING
from ...core import app
from langbot.pkg.utils import importutil
from langbot.pkg.provider.tools import loaders
from langbot.pkg.provider.tools.loaders import mcp as mcp_loader, plugin as plugin_loader
import langbot_plugin.api.entities.builtin.resource.tool as resource_tool import langbot_plugin.api.entities.builtin.resource.tool as resource_tool
from langbot_plugin.api.entities.events import pipeline_query from langbot_plugin.api.entities.events import pipeline_query
importutil.import_modules_in_pkg(loaders) if TYPE_CHECKING:
from ...core import app
from langbot.pkg.provider.tools.loaders import (
mcp as mcp_loader,
native as native_loader,
plugin as plugin_loader,
skill_authoring as skill_authoring_loader,
)
class ToolManager: class ToolManager:
@@ -17,31 +21,53 @@ class ToolManager:
ap: app.Application ap: app.Application
native_tool_loader: native_loader.NativeToolLoader
plugin_tool_loader: plugin_loader.PluginToolLoader plugin_tool_loader: plugin_loader.PluginToolLoader
mcp_tool_loader: mcp_loader.MCPLoader mcp_tool_loader: mcp_loader.MCPLoader
skill_tool_loader: skill_authoring_loader.SkillToolLoader
def __init__(self, ap: app.Application): def __init__(self, ap: app.Application):
self.ap = ap self.ap = ap
async def initialize(self): async def initialize(self):
from langbot.pkg.utils import importutil
from langbot.pkg.provider.tools import loaders
from langbot.pkg.provider.tools.loaders import (
mcp as mcp_loader,
native as native_loader,
plugin as plugin_loader,
skill_authoring as skill_authoring_loader,
)
importutil.import_modules_in_pkg(loaders)
self.native_tool_loader = native_loader.NativeToolLoader(self.ap)
await self.native_tool_loader.initialize()
self.plugin_tool_loader = plugin_loader.PluginToolLoader(self.ap) self.plugin_tool_loader = plugin_loader.PluginToolLoader(self.ap)
await self.plugin_tool_loader.initialize() await self.plugin_tool_loader.initialize()
self.mcp_tool_loader = mcp_loader.MCPLoader(self.ap) self.mcp_tool_loader = mcp_loader.MCPLoader(self.ap)
await self.mcp_tool_loader.initialize() await self.mcp_tool_loader.initialize()
self.skill_tool_loader = skill_authoring_loader.SkillToolLoader(self.ap)
await self.skill_tool_loader.initialize()
async def get_all_tools( async def get_all_tools(
self, bound_plugins: list[str] | None = None, bound_mcp_servers: list[str] | None = None self,
bound_plugins: list[str] | None = None,
bound_mcp_servers: list[str] | None = None,
include_skill_authoring: bool = False,
) -> list[resource_tool.LLMTool]: ) -> list[resource_tool.LLMTool]:
"""获取所有函数"""
all_functions: list[resource_tool.LLMTool] = [] all_functions: list[resource_tool.LLMTool] = []
all_functions.extend(await self.native_tool_loader.get_tools())
if include_skill_authoring:
all_functions.extend(await self.skill_tool_loader.get_tools())
all_functions.extend(await self.plugin_tool_loader.get_tools(bound_plugins)) all_functions.extend(await self.plugin_tool_loader.get_tools(bound_plugins))
all_functions.extend(await self.mcp_tool_loader.get_tools(bound_mcp_servers)) all_functions.extend(await self.mcp_tool_loader.get_tools(bound_mcp_servers))
return all_functions return all_functions
async def generate_tools_for_openai(self, use_funcs: list[resource_tool.LLMTool]) -> list: async def generate_tools_for_openai(self, use_funcs: list[resource_tool.LLMTool]) -> list:
"""生成函数列表"""
tools = [] tools = []
for function in use_funcs: for function in use_funcs:
@@ -58,28 +84,6 @@ class ToolManager:
return tools return tools
async def generate_tools_for_anthropic(self, use_funcs: list[resource_tool.LLMTool]) -> list: async def generate_tools_for_anthropic(self, use_funcs: list[resource_tool.LLMTool]) -> list:
"""为anthropic生成函数列表
e.g.
[
{
"name": "get_stock_price",
"description": "Get the current stock price for a given ticker symbol.",
"input_schema": {
"type": "object",
"properties": {
"ticker": {
"type": "string",
"description": "The stock ticker symbol, e.g. AAPL for Apple Inc."
}
},
"required": ["ticker"]
}
}
]
"""
tools = [] tools = []
for function in use_funcs: for function in use_funcs:
@@ -93,16 +97,18 @@ class ToolManager:
return tools return tools
async def execute_func_call(self, name: str, parameters: dict, query: pipeline_query.Query) -> typing.Any: async def execute_func_call(self, name: str, parameters: dict, query: pipeline_query.Query) -> typing.Any:
"""执行函数调用""" if await self.native_tool_loader.has_tool(name):
return await self.native_tool_loader.invoke_tool(name, parameters, query)
if await self.plugin_tool_loader.has_tool(name): if await self.plugin_tool_loader.has_tool(name):
return await self.plugin_tool_loader.invoke_tool(name, parameters, query) return await self.plugin_tool_loader.invoke_tool(name, parameters, query)
elif await self.mcp_tool_loader.has_tool(name): if await self.mcp_tool_loader.has_tool(name):
return await self.mcp_tool_loader.invoke_tool(name, parameters, query) return await self.mcp_tool_loader.invoke_tool(name, parameters, query)
else: if await self.skill_tool_loader.has_tool(name):
raise ValueError(f'未找到工具: {name}') return await self.skill_tool_loader.invoke_tool(name, parameters, query)
raise ValueError(f'未找到工具: {name}')
async def shutdown(self): async def shutdown(self):
"""关闭所有工具""" await self.native_tool_loader.shutdown()
await self.plugin_tool_loader.shutdown() await self.plugin_tool_loader.shutdown()
await self.mcp_tool_loader.shutdown() await self.mcp_tool_loader.shutdown()
await self.skill_tool_loader.shutdown()
+3
View File
@@ -0,0 +1,3 @@
from .manager import SkillManager
__all__ = ['SkillManager']
+35
View File
@@ -0,0 +1,35 @@
from __future__ import annotations
import typing
from ..provider.tools.loaders import skill as skill_loader
if typing.TYPE_CHECKING:
from ..core import app
import langbot_plugin.api.entities.builtin.pipeline.query as pipeline_query
# Skill activation is now handled through Tool Call mechanism (activate tool).
# This file is kept for potential future extensions but the text marker
# detection mechanism has been removed.
def register_activated_skill(
ap: app.Application,
query: pipeline_query.Query,
skill_name: str,
) -> bool:
"""Register an activated skill for sandbox mount path resolution.
This is called by the activate tool when a skill is activated via Tool Call.
"""
skill_mgr = getattr(ap, 'skill_mgr', None)
if skill_mgr is None:
return False
skill_data = skill_mgr.get_skill_by_name(skill_name)
if skill_data is None:
return False
skill_loader.register_activated_skill(query, skill_data)
return True
+142
View File
@@ -0,0 +1,142 @@
from __future__ import annotations
import os
import typing
from ..core import app
if typing.TYPE_CHECKING:
pass
class SkillManager:
"""Skill manager backed by Box-managed or local filesystem packages.
In sandbox deployments, skills are loaded from the Box runtime. Local
data/skills remains as the fallback for non-Box development.
Skills are activated through the `activate` tool (Tool Call mechanism),
aligned with Claude Code's design. This protects KV Cache and follows
industry standard.
"""
ap: app.Application
skills: dict[str, dict]
def __init__(self, ap: app.Application):
self.ap = ap
self.skills = {}
async def initialize(self):
await self.reload_skills()
async def reload_skills(self):
"""Reload all skills from the Box runtime.
Box is the only source of truth for skills. When Box is unavailable
(disabled in config or unreachable) the cache is emptied there is
no local filesystem fallback. Skills whose ``package_root`` is no
longer visible on the LangBot-side filesystem are dropped so they
don't surface as stale ``extra_mounts``.
"""
self.skills = {}
box_service = getattr(self.ap, 'box_service', None)
if box_service is None or not getattr(box_service, 'available', False):
self.ap.logger.info('Box runtime unavailable; skill cache is empty.')
return
# LangBot may only validate Box-reported paths against its own
# filesystem when the two share one (local stdio mode). In separated
# deployments (Docker Compose, k8s sidecar, --standalone-box, remote
# endpoint) the package_root lives on the Box runtime's filesystem and
# is not resolvable here, so we trust what Box reports.
validate_locally = bool(getattr(box_service, 'shares_filesystem_with_box', False))
try:
dropped = 0
for skill_data in await box_service.list_skills():
skill_name = skill_data.get('name')
if not skill_name:
continue
package_root = str(skill_data.get('package_root', '') or '').strip()
if validate_locally and package_root and not os.path.isdir(package_root):
self.ap.logger.warning(
f'Skill "{skill_name}" reported by Box runtime but '
f'package_root missing on LangBot filesystem '
f'({package_root}); dropping from in-memory cache.'
)
dropped += 1
continue
self.skills[skill_name] = skill_data
if dropped:
self.ap.logger.warning(
f'Loaded {len(self.skills)} skills from Box runtime '
f'({dropped} dropped due to missing package_root).'
)
else:
self.ap.logger.info(f'Loaded {len(self.skills)} skills from Box runtime')
except Exception as exc:
self.ap.logger.warning(f'Failed to load skills from Box runtime: {exc}')
def refresh_skill_from_disk(self, skill_name: str) -> bool:
"""Confirm a single skill is present in the cache.
With Box as the only source of truth, the actual reload is driven by
SkillService callers awaiting ``reload_skills``; this method only
reports whether the cache still has the skill.
"""
if not skill_name:
return False
return skill_name in self.skills
def get_skill_by_name(self, name: str) -> dict | None:
"""Get skill data by name."""
return self.skills.get(name)
def get_skill_index(self, bound_skills: list[str] | None = None) -> str:
"""Render the pipeline-visible skills as a short ``name: description``
index suitable for the system prompt.
``bound_skills`` follows the same convention as
``query.variables['_pipeline_bound_skills']``: ``None`` means every
loaded skill is exposed; an explicit list filters to that subset.
Returns an empty string when no skills are visible.
"""
lines: list[str] = []
for skill in self.skills.values():
name = skill.get('name')
if not name:
continue
if bound_skills is not None and name not in bound_skills:
continue
display = skill.get('display_name') or name
description = (skill.get('description') or '').strip().replace('\n', ' ')
lines.append(f'- {name} ({display}): {description}')
if not lines:
return ''
return 'Available Skills:\n' + '\n'.join(lines)
def build_skill_aware_prompt_addition(self, bound_skills: list[str] | None = None) -> str:
"""Build the system-prompt addendum that makes the LLM aware of the
pipeline-visible skills.
Only metadata (name + description) is injected the full SKILL.md is
loaded later via the ``activate`` Tool Call, protecting KV cache and
matching Claude Code's progressive disclosure pattern. Returns an
empty string when no skills are visible (no prompt change at all).
"""
skill_index = self.get_skill_index(bound_skills)
if not skill_index:
return ''
return (
'\n\n'
f'{skill_index}\n\n'
"When the user's request clearly matches one or more skills "
'based on their descriptions above, call the `activate` tool with '
'the skill name to load its full instructions. Only the name and '
'description are visible here; the actual instructions arrive as '
'the tool result. If no skill is a clear match, respond normally '
'without activating any skill.'
)
+37
View File
@@ -0,0 +1,37 @@
"""Shared utilities for skill file parsing."""
import yaml
def parse_frontmatter(content: str) -> tuple[dict, str]:
"""Parse YAML frontmatter from markdown content.
Expects format:
---
name: my-skill
description: Does something
---
# Actual instructions...
Returns:
Tuple of (metadata dict, remaining content)
"""
if not content.startswith('---'):
return {}, content
parts = content.split('---', 2)
if len(parts) < 3:
return {}, content
frontmatter_str = parts[1].strip()
instructions = parts[2].strip()
try:
metadata = yaml.safe_load(frontmatter_str) or {}
except yaml.YAMLError:
metadata = {}
if not isinstance(metadata, dict):
metadata = {}
return metadata, instructions
+88
View File
@@ -0,0 +1,88 @@
"""Base class for connectors that may manage a local runtime subprocess."""
from __future__ import annotations
import asyncio
import os
import sys
from typing import TYPE_CHECKING, Awaitable, Callable
if TYPE_CHECKING:
from ..core import app as core_app
class ManagedRuntimeConnector:
"""Base class for connectors that may manage a local runtime subprocess.
Provides shared lifecycle helpers: subprocess launch, health-check retry,
and graceful termination. Concrete connectors (plugin, box, ) inherit
this and add their own protocol-specific logic.
"""
ap: 'core_app.Application'
runtime_subprocess: asyncio.subprocess.Process | None
runtime_subprocess_task: asyncio.Task | None
def __init__(self, ap: 'core_app.Application'):
self.ap = ap
self.runtime_subprocess = None
self.runtime_subprocess_task = None
async def _start_runtime_subprocess(self, *args: str) -> None:
"""Launch a local runtime as a subprocess of the current Python interpreter.
If a subprocess is already running (no *returncode* yet), this is a no-op.
"""
if self.runtime_subprocess is not None and self.runtime_subprocess.returncode is None:
return
python_path = sys.executable
env = os.environ.copy()
self.runtime_subprocess = await asyncio.create_subprocess_exec(
python_path,
*args,
env=env,
)
self.runtime_subprocess_task = asyncio.create_task(self.runtime_subprocess.wait())
async def _wait_until_ready(
self,
check: Callable[[], Awaitable[None]],
retries: int = 40,
interval: float = 0.25,
runtime_name: str = 'runtime',
) -> None:
"""Repeatedly call *check* until it succeeds or retries are exhausted.
Between attempts the method sleeps for *interval* seconds. If the
managed subprocess exits before readiness is confirmed, a
``RuntimeError`` is raised immediately.
"""
last_exc: Exception | None = None
for _ in range(retries):
# Fast-fail if the process already died.
if self.runtime_subprocess is not None and self.runtime_subprocess.returncode is not None:
raise RuntimeError(
f'local {runtime_name} exited before becoming ready (code {self.runtime_subprocess.returncode})'
)
try:
await check()
return
except Exception as exc:
last_exc = exc
await asyncio.sleep(interval)
if last_exc is not None:
raise last_exc
raise RuntimeError(f'local {runtime_name} did not become ready')
def _dispose_subprocess(self) -> None:
"""Terminate the managed subprocess and cancel its wait task."""
if self.runtime_subprocess is not None and self.runtime_subprocess.returncode is None:
self.ap.logger.info('Terminating managed runtime process...')
self.runtime_subprocess.terminate()
if self.runtime_subprocess_task is not None:
self.runtime_subprocess_task.cancel()
self.runtime_subprocess_task = None
+55 -19
View File
@@ -1,37 +1,70 @@
"""Utility functions for finding package resources""" """Utility functions for finding package resources and runtime data roots."""
import os import os
from pathlib import Path from pathlib import Path
_is_source_install = None _is_source_install = None
_source_root = None
def _find_source_root() -> Path | None:
"""Locate the LangBot repository root when running from source."""
global _source_root
if _source_root is not None:
return _source_root
current = Path(__file__).resolve()
for parent in current.parents:
if (parent / 'pyproject.toml').exists() and (parent / 'main.py').exists():
_source_root = parent
return parent
_source_root = None
return None
def _check_if_source_install() -> bool: def _check_if_source_install() -> bool:
""" """
Check if we're running from source directory or an installed package. Check if we're running from the LangBot source tree.
Cached to avoid repeated file I/O. Cached to avoid repeated filesystem scans.
""" """
global _is_source_install global _is_source_install
if _is_source_install is not None: if _is_source_install is not None:
return _is_source_install return _is_source_install
# Check if main.py exists in current directory with LangBot marker _is_source_install = _find_source_root() is not None
if os.path.exists('main.py'): return _is_source_install
try:
with open('main.py', 'r', encoding='utf-8') as f:
# Only read first 500 chars to check for marker
content = f.read(500)
if 'LangBot/main.py' in content:
_is_source_install = True
return True
except (IOError, OSError, UnicodeDecodeError):
# If we can't read the file, assume not a source install
pass
_is_source_install = False
return False def get_data_root() -> str:
"""
Get the runtime data root.
Priority:
1. LANGBOT_DATA_ROOT environment override
2. Source checkout root /data when running from source
3. Current working directory /data for installed-package usage
"""
env_root = os.environ.get('LANGBOT_DATA_ROOT', '').strip()
if env_root:
return str(Path(env_root).expanduser().resolve())
source_root = _find_source_root()
if source_root is not None:
return str((source_root / 'data').resolve())
return str((Path.cwd() / 'data').resolve())
def get_data_path(*parts: str) -> str:
"""Join path segments under the resolved data root."""
data_root = Path(get_data_root())
if not parts:
return str(data_root)
return str((data_root.joinpath(*parts)).resolve())
def get_frontend_path() -> str: def get_frontend_path() -> str:
@@ -76,8 +109,11 @@ def get_resource_path(resource: str) -> str:
Absolute path to the resource Absolute path to the resource
""" """
# First, check if resource exists in current directory (source install) # First, check if resource exists in current directory (source install)
if _check_if_source_install() and os.path.exists(resource): source_root = _find_source_root()
return resource if source_root is not None:
source_resource = source_root / resource
if source_resource.exists():
return str(source_resource)
# Second, check current directory anyway # Second, check current directory anyway
if os.path.exists(resource): if os.path.exists(resource):

Some files were not shown because too many files have changed in this diff Show More