Compare commits

..

129 Commits

Author SHA1 Message Date
Junyan Qin
cb79a6df23 chore: bump beta version 2026-05-21 14:01:45 +08:00
Junyan Qin
7cf4e58ed8 fix(ci): resolve langbot-plugin from PyPI and clear lint failures
CI on feat/sandbox failed across Unit Tests, Lint and Build Dev Image.
Root causes and fixes:

- pyproject.toml had a [tool.uv.sources] editable override pinning
  langbot-plugin to ../langbot-plugin-sdk. That path only exists in a
  paired local checkout, so `uv sync` failed on every CI runner
  ("Distribution not found"). Remove the override and regenerate uv.lock
  so langbot-plugin==0.4.0b1 resolves from PyPI, matching master.

- tests/integration/api/test_pipelines.py: the pipeline extensions
  endpoint now calls ap.skill_service.list_skills(); add the missing
  skill_service mock to the fake_pipeline_app fixture (the test came
  from master, the endpoint change from feat/sandbox).

- Apply ruff format to three src files and prettier to three web files
  that had committed formatting drift, failing `ruff format --check`
  and `pnpm lint`.
2026-05-21 13:38:27 +08:00
Junyan Qin
a39c4d5665 chore: bump langbot-plugin beta 1 2026-05-21 13:25:40 +08:00
Junyan Qin
34302213ae refactor(box): launch box runtime via the lbp CLI subcommand
Mirror the plugin runtime: box is now started through the same CLI entry
point (langbot_plugin.cli) instead of the box module directly.

- docker-compose.yaml: langbot_box command runs `langbot_plugin.cli ... box`
  (WebSocket is the default transport, no flag needed — matches `rt`).
- box/connector.py: both subprocess launch sites (_start_local_stdio and
  the Windows _start_subprocess_then_ws path) invoke
  `langbot_plugin.cli.__init__ box`, using `-s` for the stdio transport.
- docs/review: update stale `-m langbot_plugin.box[.server]` references.

Pairs with the SDK change that removes box's direct-launch entry points
(python -m langbot_plugin.box / .box.server) and the legacy --mode flag.
2026-05-21 13:21:03 +08:00
Junyan Qin
d1ddff9cdb test: reconcile master's unit tests with feat/sandbox refactors
The merge from master brought in new unit tests that target pre-refactor
APIs on feat/sandbox. Reconcile each:

- factories/app.py: FakeApp now exposes a Mock skill_mgr (with empty .skills
  dict + inert prompt-addition builder) and a Mock pipeline_service so the
  PreProcessor skill-index injection branch can run end-to-end in tests.

- pipeline/conftest.py: eagerly import langbot.pkg.pipeline.pipelinemgr so
  pipeline.stage is fully initialised before any individual stage test
  (preproc, longtext, ...) tries to lazy-load it. Without this preload,
  running test_preproc.py in isolation hit a circular-import error via the
  stage -> app -> pipelinemgr -> stage chain.

- provider/test_tool_manager.py: ToolManager now probes four loaders
  (native -> plugin -> mcp -> skill). Inject inert native + skill mocks in
  the execute_func_call fixture and assert all four shutdowns fire.

- utils/test_paths.py: drop the three cwd-dependent _check_if_source_install
  cases. The refactor walks Path(__file__).resolve().parents looking for
  pyproject.toml + main.py, so cwd no longer factors in and there's no
  file read to mock-fail. The positive case and caching test still apply.

- utils/test_version.py: delete entirely. is_newer and compare_version_str
  were removed when VersionManager was refactored to use the Space API for
  release checks (1b4107a9); the tests targeted a surface that no longer
  exists.
2026-05-21 00:04:34 +08:00
Junyan Qin
e65f851b2a Merge remote-tracking branch 'langbot-app/master' into feat/sandbox
Resolve conflicts in:
- .github/workflows/run-tests.yml: keep master's src/langbot/** paths plus feat/** push branch
- src/langbot/pkg/plugin/connector.py: keep both branches' marketplace MCP/skill
  install logic (HEAD) and runtime/wait helpers (master); add missing return in
  _inspect_plugin_package so LOCAL/GITHUB install paths get author/name back
- tests/unit_tests/pipeline/test_n8nsvapi.py: keep HEAD's try/finally sys.modules
  save/restore pattern
- web/src/app/home/components/dynamic-form/DynamicFormComponent.tsx: union
  imports + keep HEAD's disable_if/tooltip support and master's QrCodeLoginDialog
- web/src/i18n/locales/*: union of disjoint top-level keys from both branches
- web/src/app/home/market/page.tsx: accept our deletion (unified extensions page)
- uv.lock: regenerate via uv sync --dev
2026-05-20 23:58:21 +08:00
Junyan Qin
2cddc7efad feat(web): surface the specific Box failure reason in unavailable banner
When Box is configured but the runtime reports its backend is dead
(e.g. ``box.backend = nsjail`` but the binary is missing, or Docker
daemon crashed), the backend now returns a structured
``connector_error`` like ``Configured sandbox backend "nsjail" is
unavailable``. The previous notice only said "Box sandbox is
unavailable" + a generic "enable Box" hint, hiding the actionable
detail.

- ``useBoxStatus``: derive ``reason`` from ``status.connector_error``.
  Only exposed for the failed-state (``hint === 'boxUnavailable'``),
  since the disabled-by-config message already carries its reason
- ``BoxUnavailableNotice``: insert the reason as a small monospaced
  line between the state message and the action hint. The disabled
  variant is unchanged (operator chose the state)
- Wire ``reason`` through every existing call site (Skills page +
  detail, PipelineExtension, both MCP forms). Old unused ``context``
  prop dropped

Net layout (3 lines, still compact):

  ⚠ Box sandbox is unavailable — sandbox tools, skill add/edit, ...
    Configured sandbox backend "nsjail" is unavailable
    This feature requires the Box runtime. Enable it in config ...

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-20 23:43:39 +08:00
Junyan Qin
a2a9f426fa fix(box): downgrade get_status.available when backend probed unavailable
Until now ``BoxService.get_status`` returned ``available: true`` whenever
the runtime connector was healthy, even if the runtime itself reported
``backend: { available: false }`` (operator selected nsjail without the
binary, Docker daemon crashed mid-session, E2B credentials wrong, ...).
The dashboard / ``useBoxStatus`` hook / skill_service gate consumed the
top-level flag and showed "connected" while every actual call to native
exec or skill management would fail.

The native-tool loader already polled ``status.backend.available``
independently and hid its tools correctly, but every other consumer
(dashboard banner, the disabled-state hint, the LLM-facing message)
disagreed with it.

Combine the two in the payload: ``available = self._available AND
status.backend.available``. When ``backend.available`` is false we now
also surface a ``connector_error`` that names the backend
("Configured sandbox backend \"nsjail\" is unavailable") so the dialog
shows the actionable reason instead of an empty error pane. The
detailed ``backend`` object is preserved unchanged for the dialog.

Internal ``box_service.available`` (used by ``skill_service`` writes,
``mcp_stdio.uses_box_stdio``, the reconnect callback) is intentionally
NOT changed — it still tracks connector health only, so a backend blip
does not trigger spurious reconnect loops.

Tests:
- ``test_get_status_downgrades_available_when_backend_dead`` — exercise
  the new branch (connector OK, backend.available=false → top-level
  available=false, connector_error mentions the backend name)
- ``test_get_status_keeps_available_true_when_backend_ok`` — guard
  against regressing the happy path

Live-verified with ``box.backend: nsjail`` on macOS (no nsjail binary):
``GET /api/v1/box/status`` now returns ``available: false`` with the
named connector_error, instead of the previous misleading
``available: true``.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-20 23:38:02 +08:00
Junyan Qin
68bd786f39 fix(skill): re-inject skill index into local-agent system prompt
The contributor's original PR (#1917) appended an ``Available Skills``
index to the system prompt before the LLM saw the user message, so the
LLM could decide whether to activate a skill. ``7145447b`` removed the
text-marker activation flow and, together with it, the entire system
prompt injection — but the Tool Call replacement only put the available
skills inside the ``activate`` tool's description. In practice the LLM
ignores tool descriptions for selection and goes straight to native
tools, so user-visible skill activation silently broke.

Restore the injection, adapted for the Tool Call era:

- SkillManager regains ``get_skill_index(bound_skills)`` and
  ``build_skill_aware_prompt_addition(bound_skills)``. The addendum
  carries only ``name (display_name): description`` for each
  pipeline-visible skill plus one instruction line pointing at the
  ``activate`` tool. No SKILL.md contents — KV cache stays clean
- PreProcessor appends the addendum to the first system message (or
  inserts a new one) of ``query.prompt.messages`` for the local-agent
  runner. Handles plain-string and ContentElement[] bodies. Skips
  cleanly when no skills are visible
- 3 new test_preproc cases: injection happens, bound-skills subset
  honoured, empty addendum touches nothing. 280 passed

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-20 22:37:20 +08:00
Junyan Qin
42855cf4cc chore(skill): prune dead local-filesystem helpers left over from Box migration
Follow-up to the Box-only refactor. The previous commit removed the
local-fallback BRANCHES from every public method; this one removes the
HELPERS those branches called, which are now unreachable.

SkillService (service/skill.py): 787 → 449 lines
  Removed: scan_directory (sync), _read_skill_package, _write_skill_md,
  _resolve_create_field, _managed_skill_path,
  _managed_install_root_for_package, _normalize_package_root,
  _resolve_skill_path, _find_skill_entry, _discover_skill_directories,
  _safe_extract_zip, _extract_uploaded_skill_to_temp,
  _download_github_skill_to_temp, _resolve_github_source_root,
  _build_preview_target_dir, _preview_skill_candidates,
  _select_preview_candidates, _install_preview_candidates,
  _preview_source_root, _resolve_installed_skills, plus the
  module-level _FRONTMATTER_FIELDS and _build_skill_md.
  Kept (still needed by the surviving GitHub-import path):
  _download_github_asset, _download_github_skill_directory_as_zip,
  _find_github_skill_archive_entry, _copy_github_skill_directory_to_zip,
  _is_github_skill_md_url, _parse_github_skill_md_url,
  _resolve_github_skill_md_package_name, _validate_github_asset_url,
  _uploaded_skill_target_stem, _validate_skill_name.
  Imports dropped: shutil, tempfile, yaml, ....utils.paths.

SkillManager (skill/manager.py): 187 → 88 lines
  Removed: get_managed_skills_root, _discover_skill_directories,
  _find_skill_entry, _load_skill_file, _normalize_package_root.
  Imports dropped: datetime, parse_frontmatter, paths.

Tests:
  - test_skill_service.py: drop the 3 sync scan_directory tests +
    skill_service fixture + _create_skill_file helper
  - test_skill_tools.py: drop test_load_skill_file_success; rename
    TestSkillManagerPackageLoading → TestSkillManagerCache

Full unit suite: 277 passed, 1 skipped. ``ruff check`` clean.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-20 22:24:08 +08:00
Junyan Qin
cc072be7f7 refactor(skill): remove all local-filesystem fallbacks; Box is the sole source
Skills now flow exclusively through the Box runtime. Every read and write
method funnels through ``_box_service()``; when Box is unavailable
(disabled in config, connection failed, or simply not installed) the
operation either returns an empty surface (``list_skills`` → []) or
raises with a clear ``Box runtime ... not initialised / disabled /
unavailable: ...`` message via the new ``_require_box(action)`` helper.

Why: the legacy local-fallback path scanned ``data/skills/``, but Box
manages its own ``box.local.skills_root`` (default ``data/box/skills/``).
The two diverging directories caused stale / phantom skill lists when
Box flapped, and the local-fallback writes silently bypassed all the
sandboxing the operator had configured.

SkillService (``api/http/service/skill.py``):
- New ``_require_box(action)`` returns the box service or raises a
  structured ValueError. ``_require_box_for_write`` kept as alias
- ``list_skills`` → returns [] when Box is down so the UI can render
  the disabled banner cleanly
- ``get_skill`` / ``get_skill_by_name`` → return None
- All read-file / write-file / scan-dir / create / update / delete /
  install / preview methods → ``_require_box`` then box delegate.
  Local fallback bodies (shutil.copytree, tempfile.mkdtemp, preview
  pipelines) removed entirely

SkillManager (``pkg/skill/manager.py``):
- ``reload_skills`` returns early with empty cache when Box is down.
  data/skills/ discovery loop removed
- ``refresh_skill_from_disk`` now just reports cache presence; the
  on-disk re-parse is gone since Box is the only writer

Tests:
- Drop 11 obsolete test_skill_service.py tests that exercised the
  removed local-fallback paths (create/install/file/delete/update)
- Add list-empty + read-refused tests; flip the legacy-allow test to
  legacy-refuses-too
- Rewrite refresh_skill_from_disk test to match the new behaviour

Several helper methods (_managed_skill_path, _resolve_skill_path,
_preview_skill_candidates, _install_preview_candidates, etc.) are now
unreachable; a follow-up commit will prune them so this diff stays
reviewable.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-20 22:07:23 +08:00
Junyan Qin
49064ffc2d fix(web): prevent plugin config form overflow 2026-05-20 19:55:21 +08:00
Junyan Qin
aa8d53dde6 feat(mcp-web): block stdio MCP creation at the form when Box is unavailable
When Box is disabled in config (``box.enabled = false``) or unreachable,
saving a new MCP server in stdio mode produced one that could never
start — the user would only learn that from the runtime error on the
detail page. Stop the user before they save instead.

Both MCP forms (the page-level ``MCPForm.tsx`` and the older dialog
``MCPFormDialog.tsx``) now:

- Disable the ``stdio`` option in the mode select when Box is
  unavailable, with a small "(requires Box)" suffix so the reason is
  obvious. Existing stdio configs still display their current value
- Show ``BoxUnavailableNotice`` inline under the mode select when the
  currently-selected mode is stdio and Box is unavailable, so editing
  a stale stdio config makes the cause visible
- Disable the Save / Submit button while stdio is selected under that
  condition. ``MCPForm`` exposes a new ``onSaveBlockedChange`` prop
  so the parent ``MCPDetailContent`` can disable both its Submit and
  Save buttons. ``MCPFormDialog`` disables its Save button locally
- Refuse the submit handler too (Enter-key path) with a toast carrying
  the same i18n message

i18n: ``mcp.boxRequired`` (short tag in the disabled option) and
``mcp.stdioBlockedByBoxToast`` added to all 8 locales.

Backend runtime gate (``_init_stdio_python_server`` refusal +
``BOX_UNAVAILABLE`` error_phase + retry short-circuit) stays in place
as the last line of defence for API bypass.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-20 18:03:47 +08:00
Junyan Qin
216b1b9f03 feat(mcp): friendly UI message when stdio MCP refused by Box state
Previously the MCP detail dialog dumped the raw RuntimeError text from
``_init_stdio_python_server`` — English-only, prefixed with "Failed
after 4 attempts", and exposing internal config names. The retry
wrapper also kept retrying a refusal that is deterministically going
to fail again, polluting logs.

Replace the raw text with a structured signal:

- New ``MCPSessionErrorPhase.BOX_UNAVAILABLE`` enum value. The stdio
  refusal path sets it before raising and uses a short opaque
  discriminator (``box_disabled_in_config`` / ``box_unavailable``) as
  the message body — never user-facing
- ``_lifecycle_loop_with_retry`` short-circuits on
  ``BOX_UNAVAILABLE``: surfaces the error immediately, no retries,
  no "Failed after N attempts" prefix. Silences the warning storm
  seen during smoke-testing
- ``MCPServerRuntimeInfo`` (TS type) now declares ``error_phase``,
  ``retry_count``, ``box_session_id``, ``box_enabled`` to match what
  the backend already returns in get_runtime_info_dict()
- Both MCP detail forms (``mcp/components/mcp-form/MCPForm.tsx`` and
  ``plugins/mcp-server/mcp-form/MCPFormDialog.tsx``) detect
  ``error_phase === 'box_unavailable'`` and render a two-line
  localized notice: state line ("Box disabled / unreachable") plus
  remediation line ("enable Box or switch to http/sse")
- 8 locale files (en/zh-Hans/zh-Hant/ja/ru/vi/th/es) get
  ``mcp.boxDisabledStdioRefused``, ``mcp.boxUnavailableStdioRefused``,
  ``mcp.boxStdioRefusedSuggestion``

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-20 17:51:32 +08:00
Junyan Qin
9f9b112526 refactor(pipeline-form): swap Box banner for field-level disable_if + tooltip
The previous commit hard-coded a BoxUnavailableNotice banner above the
``local-agent`` stage card. That works, but it shouts at the user about
every field in that stage when in reality only one field —
``box-session-id-template`` — depends on the sandbox.

Use the dynamic-form schema's existing variable-injection mechanism
(``__system.*`` references via ``systemContext``) and add a sibling to
``show_if``: ``disable_if`` + ``disabled_tooltip``. The field stays
visible, becomes inert, and an info icon next to its label exposes the
reason on hover. The rest of the AI tab is left untouched.

- entities/form/dynamic.ts: extend IDynamicFormItemSchema with
  ``disable_if: IShowIfCondition`` and ``disabled_tooltip: I18nObject``
- DynamicFormComponent: evaluate disable_if with the same resolver as
  show_if; OR the result into isFieldDisabled; render an Info tooltip
  trigger next to the label when the condition matches
- ai.yaml metadata: attach disable_if (__system.box_available eq false)
  and a localized disabled_tooltip to box-session-id-template
- PipelineFormComponent: drop the BoxUnavailableNotice import and the
  per-stage banner; pass ``systemContext={ box_available: boxAvailable }``
  only for the local-agent stage so other stages aren't paying the
  re-render cost

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-20 17:42:17 +08:00
Junyan Qin
f7ee2c0961 docs(box): document the box.enabled toggle and gate behavior matrix
- docker-compose: move ``langbot_box`` under compose profiles
  (``box`` and ``all``) so ``docker compose up`` no longer requires
  the sandbox container. Inline comment explains how to pair the
  profile choice with ``box.enabled`` so the langbot service does not
  thrash trying to reach a runtime that was never started
- docs/review/box-architecture.md:
  - Annotate ``box.enabled`` in the config.yaml example, listing the
    exact side effects (no remote/stdio connect; tools/skills/MCP
    stdio off; reads still work)
  - Replace the bare compose snippet with the actual profile-driven
    invocation and the BOX__ENABLED pairing
  - New "关闭/连接失败时的行为矩阵" section: a single table mapping
    every consumer (native tools, activate/register_skill, stdio MCP,
    skill list/CRUD, pipeline AI config, extensions page, dashboard)
    to its disabled-state behavior, plus the legacy ``ap.box_service``
    distinguisher note

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-20 17:20:54 +08:00
Junyan Qin
446099ecda feat(web): surface Box disabled/unavailable state across consumers
When Box is disabled in config (``box.enabled = false``) or fails to
connect, every dependent UI surface now degrades visibly:

- ``useBoxStatus`` hook: shared, polled 30s, exposes ``available``,
  ``disabled`` (config-off) and a single ``hint`` key so callers don't
  have to re-derive the three states
- ``BoxUnavailableNotice`` reusable Alert banner driven by that hint
- Dashboard SystemStatusCards: three-state dot + label
  (connected / disabled-gray / disconnected-red); disabled state shows
  the ``boxDisabled`` hint, failed state continues to show the connector
  error. Plugin block kept untouched
- Skills page (create view) and SkillDetailContent (edit view):
  Save button disabled and banner inserted above the form when Box is
  unavailable — matches the backend gate added in the previous commit
- PipelineExtension skill section: ``enable_all_skills`` switch, Add
  Skill button and Remove buttons all gate on Box availability;
  banner inline under the section header
- PipelineFormComponent: banner above the ``local-agent`` stage card
  when Box is unavailable, since that stage carries the sandbox-bound
  ``box-session-id-template`` field
- Box status payload type (``ApiRespBoxStatus.enabled``) and 8 locale
  files updated with ``boxDisabled`` / ``boxUnavailable`` /
  ``boxRequiredHint`` strings

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-20 17:18:44 +08:00
Junyan Qin
ec2d21fe63 feat(box): add box.enabled toggle and gate consumers on availability
Make the Box sandbox runtime optional. When ``box.enabled`` is false in
config (or when an enabled Box fails to connect), every dependent feature
degrades to the same disabled-state UX rather than crashing or silently
falling back to less safe code paths.

Backend:

- config.yaml: new top-level ``box.enabled: true`` flag (default true)
- BoxService:
  - Read box.enabled on construction
  - initialize() short-circuits when disabled — no remote WS connect, no
    stdio subprocess fork
  - _on_runtime_disconnect is a no-op when disabled (no reconnect loop
    on a deliberately-off service)
  - get_status() now exposes ``enabled`` so the frontend can tell
    "disabled in config" from "configured but failed"
- MCP stdio loader (mcp_stdio.uses_box_stdio): requires box_service to
  be available, not just installed
- MCP _init_stdio_python_server: when ap.box_service exists but is
  unavailable, refuse the stdio server with an actionable error instead
  of silently falling through to host-stdio (which bypasses the sandbox
  the operator asked for). Setups without ap.box_service installed at
  all keep the legacy host-stdio fallback for pre-Box dev mode
- SkillService._require_box_for_write: refuses create/update/install/
  write_skill_file when ap.box_service is installed but unavailable.
  Distinguishes disabled vs failed in the error message so the UI can
  surface the right hint. Legacy setups (no ap.box_service) keep the
  local fallback path — that distinction is what keeps the existing
  local-skills tests valid

Tests:
- Box disabled-state behavior (4 cases)
- Skill write refusal in disabled & failed states (7 cases)
- MCP stdio runtime info policy updated to match new refuse-when-down
  behavior

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-20 17:07:53 +08:00
Junyan Qin
99328cf4c0 fix(skill): harden mount/reload paths and HTTP errors against stale skill cache
The Box backends behave inconsistently when extra_mounts reference a
missing host directory (nsjail aborts the entire sandbox start, Docker
silently creates a root-owned empty dir on the host, E2B silently skips
the upload). The cache in skill_mgr.skills is only refreshed on
in-process mutations, so out-of-band changes — container rebuilds,
manual rm in the box volume, anything the LangBot API didn't drive —
leave a stale skill that later produces one of those bad mount paths.

- box/service.py: build_skill_extra_mounts now filters skills whose
  package_root is not isdir on the LangBot-visible filesystem and logs
  a warning, instead of passing the bad mount through to the backend
- skill/manager.py: reload_skills (Box path) drops skills whose
  package_root is missing on the LangBot-side filesystem before they
  reach the in-memory cache, with a summary warning
- api/http/controller/groups/skills.py: file/CRUD handlers now also
  catch BoxError (RuntimeError subclass, previously slipping past
  ``except ValueError`` and surfacing as 500); list/get handlers gain
  a try/except so a transient Box RPC failure becomes a clean 400
  instead of a stack trace

Tests added for build_skill_extra_mounts (skip missing, skip empty,
no skill manager) and SkillManager.reload_skills (drop missing on Box
path). Full unit suite: 279 passed.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-20 16:50:46 +08:00
Junyan Qin
28c00cb8d1 ci(tests): run unit tests on every push to feat/** branches
- Add feat/** to push branches so long-lived feature branches are
  tested on every push (they accumulate large changes before a PR)
- Drop the push path filter entirely: every push to master/develop/
  feat/** now runs the full unit suite (the old 'pkg/**' filter never
  matched the real source path 'src/langbot/pkg/**', so backend-only
  pushes silently skipped tests)
- Fix the same broken path glob on the pull_request trigger
  ('pkg/**' -> 'src/langbot/pkg/**')

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-19 23:56:17 +08:00
Junyan Qin
18ad51e21e test: repair stale skill/sandbox tests for feat/sandbox
The skill subsystem moved to Tool-Call activation and a Box-managed
skill store; several tests still asserted removed APIs and a sys.modules
stub leaked across the suite. Full unit suite now green (was 23 failing).

- test_skill_tools: drop TestSkillManagerActivation (text-marker API
  removed); rewrite TestSkillActivationHelper around the current
  skill.activation.register_activated_skill; replace the CRUD
  TestSkillAuthoringToolLoader with TestSkillToolLoader covering the
  current activate/register_skill tools and sandbox-availability gating
- test_tool_manager_native: ToolManager attr is skill_tool_loader (not
  skill_authoring_tool_loader); native loader now exposes 6 tools
  (exec/read/write/edit/glob/grep) and requires initialize() with a
  backend-available get_status()
- test_localagent_sandbox_exec: remove obsolete activation-marker
  leakage tests and their helper providers
- test_model_service / pipeline conftest: give the mocks skill_mgr=None
  so PreProcessor's local-agent skill-binding guard short-circuits
- test_n8nsvapi: stop permanently overwriting sys.modules
  ('langbot.pkg.provider.runner' etc.); save and restore around the
  import so other modules get the real LocalAgentRunner base class

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-19 22:56:25 +08:00
Junyan Qin
5773e8aa27 refactor(box): use unified env-override mechanism for box.local config
The box module hand-rolled its own LANGBOT_BOX_LOCAL_* env parsing in two
places (connector._get_box_config and service._local_config), duplicating
logic that LoadConfigStage._apply_env_overrides_to_config already provides
generically via the SECTION__SUBSECTION__KEY convention.

- Drop the bespoke LANGBOT_BOX_LOCAL_* parsing; read box.local straight
  from instance_config (the unified BOX__LOCAL__* overrides are already
  applied before BoxService initializes)
- Harden _load_allowed_mount_roots to accept a comma-separated string,
  since the generic mechanism stores a freshly-created key as a raw
  string when config.yaml has no box.local.allowed_mount_roots entry
- docker-compose: rename the langbot container env vars to
  BOX__LOCAL__* (the canonical convention); remove them entirely from
  the langbot_box container — the Box runtime never reads box.local from
  env/config.yaml, it is configured via the INIT RPC action

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-19 22:38:17 +08:00
Junyan Qin
6351730891 docs(review): refresh box architecture review for feat/sandbox
Sync the docs/review/ suite to the current state of the feat/sandbox branch
(both LangBot and langbot-plugin-sdk), ~30 commits ahead of the prior review.

- box-architecture.md: rewrite for the new box.{backend,runtime,local,e2b}
  config schema, add E2B backend, 6 native tools (incl. glob/grep), Skill
  Tool Call activation, shared multi-process MCP container, SkillManager,
  BoxSkillStore (SDK), 25 actions, 9 error types, heartbeat/reconnect
- box-issues.md: move resolved items (reconnect, heartbeat, Windows, nsjail
  image conflict, frontend monitoring card) into a Resolved section; add
  new P0 (INIT/backend ordering), P1 (extra_mounts immutability after
  container creation), P2 (skill_store test gap, integration tests not in CI)
- box-session-scope.md: add §0 Implementation Status — Phase 1 shipped,
  MCP unification landed earlier than originally scoped
- box-test-coverage.md: realign file inventory (4,400 -> 6,500 LOC),
  add 7 new test files including SDK backend_selection/e2b/skill_store
- box-tob-analysis.md: connection recovery now满足基本要求; add E2B and
  backend self-heal to capabilities; tick off Phase 1 reconnect/heartbeat
- box-vs-plugin-runtime.md: heartbeat/reconnect/Windows support now aligned
  with Plugin Runtime; revise remaining gaps (WS auth, shared base class)
2026-05-19 13:31:26 +08:00
Junyan Qin
d80972417e fix(web): improve backend retry and sidebar scrolling 2026-05-19 11:40:20 +08:00
Junyan Qin
257d9d3a65 fix(mcp): stabilize shared box managed processes 2026-05-19 00:45:35 +08:00
Junyan Qin
747ea069aa feat: polish extension import flow 2026-05-18 23:32:56 +08:00
Junyan Qin
9e62227104 feat(web): improve skill import flow 2026-05-18 18:33:39 +08:00
Junyan Qin
971cc3f675 feat: install market extensions from card click 2026-05-18 17:41:43 +08:00
Junyan Qin
651904a5d4 fix: import github skill directories 2026-05-18 17:26:35 +08:00
Junyan Qin
bf8b51569f feat: support github skill installation 2026-05-17 23:09:10 +08:00
Junyan Qin
e814f359cb feat: manage skills through box runtime 2026-05-16 17:14:58 +08:00
Junyan Qin
c1f5ba1927 fix: align add extension marketplace ui 2026-05-15 18:55:25 +08:00
Junyan Qin
e8c7147d34 fix: refine extension ui and backend errors 2026-05-15 15:16:26 +08:00
Junyan Qin
98a106d3b5 feat: persist sidebar list expansion 2026-05-15 14:51:08 +08:00
Junyan Qin
ae11bce8b6 feat: polish extension detail pages 2026-05-15 14:41:23 +08:00
huanghuoguoguo
d5ce3b302e refactor: remove unused imports and clean up code in various files 2026-05-14 09:15:18 +08:00
huanghuoguoguo
656dafb07a feat(toolmgr): enhance tool initialization with backend availability checks 2026-05-14 09:01:20 +08:00
huanghuoguoguo
fd03b202a8 fix(native): update tool descriptions to use register_skill
Replace references to removed import_skill_from_directory with
register_skill in exec/write/edit tool descriptions.
2026-05-13 22:50:21 +08:00
huanghuoguoguo
d786b3475f fix(toolmgr): correct skill_tool_loader attribute name
Rename skill_authoring_tool_loader to skill_tool_loader in execute_func_call
and shutdown methods to match the attribute defined in initialize().

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-13 22:15:06 +08:00
huanghuoguoguo
17ae6950aa fix(skill): improve file browsing and fix path handling
- Fix nested directory display in skill file tree (preserve root entries)
- Fix file content display when clicking files in skill browser
- Add skill manager and tool manager as proper package modules
- Separate fileContent state to allow editing non-SKILL.md files

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-13 22:08:58 +08:00
huanghuoguoguo
b9e8827c7f fix(skill): copy builtin skills to data/skills on startup
- Builtin skills (templates/skills/) are now copied to data/skills/
- Users can view and manage builtin skills in the UI
- Rename SkillAuthoringToolLoader to SkillToolLoader

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-13 21:45:37 +08:00
huanghuoguoguo
77a85c5c23 feat(skill): add skill file browsing capability
- Add API endpoints for listing/reading/writing skill files
- Add FileTree component in SkillForm for directory browsing
- Users can now view scripts/, references/, assets/ directories
- Files can be selected and edited in the instructions textarea
- Add translations for new file browsing features

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-13 21:26:03 +08:00
huanghuoguoguo
892556da2a feat(tools): add glob and grep native sandbox tools
Add file discovery and content search capabilities to the sandbox:
- glob: Find files by pattern (supports ** recursive matching)
- grep: Search file contents with regex patterns

Both tools respect skill package paths and include safety limits
(max 100 files for glob, max 200 matches for grep).

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2026-05-13 21:18:12 +08:00
huanghuoguoguo
7145447bcb feat(skill): align skill system with Claude Code's Tool Call design
- Replace text marker activation with `activate` tool (Tool Call mechanism)
- Replace 7 authoring tools with 2: `activate` + `register_skill`
- Add builtin skills loading from templates/skills/
- Add create-skill as first builtin skill
- Remove SKILL_ACTIVATION_MARKER and text detection methods
- Tool Result returns SKILL.md content (protects KV Cache)

This aligns with Claude Code's progressive disclosure pattern:
- Metadata (name+description) always visible in tool description
- SKILL.md body loaded on activate via Tool Call
- Bundled resources accessible through virtual path mapping

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-13 21:15:39 +08:00
Junyan Qin
4db0f20dc4 fix(skill): remove auto activation setting 2026-05-13 00:51:16 +08:00
Junyan Qin
a565f3e022 fix(box): harden sandbox session isolation 2026-05-13 00:20:07 +08:00
Junyan Qin
e4c674a9f0 fix(box): restore sandbox config and shared mcp runtime 2026-05-12 23:25:43 +08:00
Junyan Qin
afc37958c1 fix: preserve monitoring card borders under sticky filters 2026-05-12 18:30:19 +08:00
Junyan Qin
b73900718a fix: constrain home page content width 2026-05-12 18:23:51 +08:00
WangCham
3f7031b6f0 feat: delete version for mcp and skills 2026-05-12 11:28:43 +08:00
WangCham
3db2ddd2c7 feat: change ui 2026-05-11 22:38:39 +08:00
Junyan Qin
dd809d36f8 feat(extensions): mobile-friendly layout for extensions and add-extension pages
- Stack the extensions page header vertically on small screens, let the
  filter Tabs scroll horizontally if they overflow, hide the debug
  button label below sm and let the install/debug controls wrap.
- Constrain the debug popover and its inputs to the viewport width so
  they no longer overflow on phone-sized screens.
- Drop the card grid from a fixed 30rem column to a min(100%, 22rem)
  column at base / 28rem at sm, and reduce the gap, so cards render
  cleanly at 360px+ widths in both flat and grouped views.
- Make the add-extension header actions wrap on lg- viewports and the
  install dialog responsive instead of a hard 500px box.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-10 00:15:55 +08:00
Junyan Qin
6f97877a5a feat(sidebar): unify installed-extensions list with plugins, MCP and skills
- Render plugins, MCP servers and skills together under the "Installed
  Extensions" sidebar entry, alphabetically sorted to match the list page.
- Resolve per-item routes by extension type (plugin -> /home/extensions,
  mcp -> /home/mcp, skill -> /home/skills) and gate the plugin-only hover
  context menu on extensionType === 'plugin'.
- Lift the "group by type" toggle into SidebarDataContext (still persisted
  in localStorage) so the sidebar groups items with section headers
  whenever the list page has the toggle enabled.
- Show lucide fallback icons (Server / Sparkles / Puzzle) tinted in the
  LangBot blue for MCP, skill, and missing-icon plugin items, overriding
  the SidebarMenuSubButton svg color rule.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-10 00:10:04 +08:00
Junyan Qin
14c2da4d29 feat(extensions): fallback lucide icon when extension icon is missing
Render a tinted lucide icon (Puzzle / Server / Sparkles) on the extension
card when the icon URL is empty or the image fails to load. Picked icons
distinct from EventListener (AudioWaveform) and KnowledgeEngine (Book) to
avoid visual collision with plugin component badges.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-09 23:57:24 +08:00
Junyan Qin
8ff60c5b98 feat(extensions): unify extensions endpoint and refresh extensions page UX
- Rename /home/plugins route to /home/extensions and update all sidebar links.
- Add unified GET /api/v1/extensions returning plugins, MCP servers and skills,
  sorted by name; replace the three separate frontend fetches with this single call.
- Migrate the extensions page to shadcn primitives (Tabs/Card/Alert/Badge/Skeleton/
  Switch/Label) and clean up hardcoded color tokens on the extension card.
- Add a localStorage-persisted "Group by type" switch that, when enabled in the
  All Types tab, renders extensions grouped by type with a compact section header.
- Show a spinner while loading and rename the empty-state copy from
  "No plugins installed" to "No extensions installed".
- Rename the "格式 / Formats" filter label to "类型 / Types" across all 8 locales.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-09 23:50:17 +08:00
Junyan Qin
46a9ed3da6 chore: rename extension zh translation 2026-05-09 23:04:02 +08:00
WangCham
f3d45eeeab feat: youhua qianduan 2026-05-09 16:47:23 +08:00
WangCham
fffc862fe6 feat: refactor market 2026-05-09 11:49:44 +08:00
WangCham
f306c762c8 feat: translate 2026-05-08 19:23:31 +08:00
Junyan Qin
ad9aa39281 fix: align box runtime launch args 2026-05-08 18:07:55 +08:00
WangCham
e412ed5527 feat: youhua frontend 2026-05-07 18:19:48 +08:00
WangCham
188511a911 feat: delete old filter 2026-05-07 13:34:26 +08:00
WangCham
58f9ff94d3 feat: successfully install 2026-05-07 13:19:02 +08:00
Junyan Qin
80911a3d91 Merge remote-tracking branch 'origin/master' into feat/sandbox
# Conflicts:
#	src/langbot/pkg/api/http/controller/groups/plugins.py
#	src/langbot/pkg/core/app.py
#	src/langbot/pkg/core/stages/build_app.py
#	src/langbot/templates/config.yaml
#	uv.lock
#	web/src/app/home/components/home-sidebar/HomeSidebar.tsx
#	web/src/app/home/components/home-sidebar/SidebarDataContext.tsx
#	web/src/app/home/layout.tsx
#	web/src/app/home/plugins/components/plugin-market/PluginMarketComponent.tsx
#	web/src/i18n/locales/en-US.ts
#	web/src/i18n/locales/es-ES.ts
#	web/src/i18n/locales/ja-JP.ts
#	web/src/i18n/locales/th-TH.ts
#	web/src/i18n/locales/vi-VN.ts
#	web/src/i18n/locales/zh-Hans.ts
#	web/src/i18n/locales/zh-Hant.ts
#	web/src/router.tsx
2026-05-05 14:05:53 +08:00
WangCham
f9347811b1 feat: add download button 2026-05-04 22:20:34 +08:00
Junyan Qin
db135f217f feat(web): add tooltips for truncated fields in system status dialog
Wrap session_id, image, and mount path fields with Tooltip components
so hovering over truncated text shows the full value.
2026-05-04 21:33:45 +08:00
Junyan Qin
fe9aed4ec9 fix(web): widen system status dialog and fix scroll border issue
Use max-w-2xl (matching other dialogs) instead of max-w-lg. Move
overflow-y-auto to an inner container with overflow-hidden on
DialogContent to prevent padding bleed at scroll edges.
2026-05-04 21:33:45 +08:00
Junyan Qin
f19cd4032d refactor(web): replace popover with dialog for system status details
Replace the dropdown popover with a proper Dialog for runtime status
details. Add a small info button on the System Status card that opens
the dialog. Session details now show in a spacious 2-column grid layout
with full image name, backend, CPU/memory, network, mount path, and
created/last-used timestamps.
2026-05-04 21:33:45 +08:00
Junyan Qin
e955b3d6e8 feat(box): add global sandbox scope option
Add a 'Global (shared by all)' option to the sandbox scope selector.
Uses a constant '{global}' template variable that always resolves to
'global', so all users and chats share one sandbox container.
2026-05-04 21:33:45 +08:00
Junyan Qin
f196cbc79d feat(web): show active sandbox details in dashboard Box status popover
Fetch box sessions alongside status and display each active sandbox
in the popover with session ID, image, resources (CPU/memory), and
last used time.
2026-05-04 21:33:37 +08:00
Junyan Qin
dfd4ab791e fix(web): fix system status card stuck in loading state
fetchStatus(showLoading=false) never called setLoading(false), so the
initial loading=true was never cleared. Simplify to always setLoading
in the finally block — the spinner only shows on the very first load
since subsequent fetches complete near-instantly.
2026-05-04 21:33:31 +08:00
Junyan Qin
e0510bca6b fix(web): refresh system status card when clicking Refresh Data button
Pass a refreshKey prop through OverviewCards to SystemStatusCard that
increments on each Refresh Data click, triggering a re-fetch of Plugin
and Box runtime status alongside the monitoring data refresh.
2026-05-04 21:33:31 +08:00
Junyan Qin
2dfd9d5dce fix(box): detect disconnect when handler.run() returns normally
The generic Handler.run() catches ConnectionClosedError and breaks out
of its loop (normal return) instead of raising, because it has no
disconnect_callback. The old code only triggered reconnection in the
except branch, so a clean WebSocket close was never detected.

Now treat handler.run() returning normally (after successful handshake)
as a disconnect event, triggering the reconnection callback.
2026-05-04 21:33:18 +08:00
Junyan Qin
3e2190a153 fix(box): add persistent reconnection loop with exponential backoff
The previous disconnect handler only retried once and then gave up.
Now spawns a background task that retries with exponential backoff
(3s, 6s, 12s, ... up to 60s) until the Box runtime is reachable again.
Uses a _reconnecting guard to prevent duplicate loops. Calls
connector.dispose() before each retry to clean up stale tasks.
2026-05-04 21:33:18 +08:00
Junyan Qin
7e0a1974b6 fix(box): handle RPC failure in get_status/get_sessions gracefully
When the Box runtime disconnects, there is a race between the heartbeat
flipping _available=false and the frontend polling get_status(). If the
poll arrives first, client.get_status() throws a ConnectionClosedError
which propagated as a 500, causing the frontend to show a grey dot
(null status) instead of a red dot with error details.

Now get_status() catches RPC errors and returns available=false with
the exception message as connector_error. get_sessions() returns an
empty list when unavailable or on RPC failure.
2026-05-04 21:33:18 +08:00
Junyan Qin
d47803db2c fix(web): auto-refresh system status and show disconnect errors in real time
Poll Plugin Runtime and Box Runtime status every 30 seconds so the
dashboard reflects disconnections without a manual page refresh.
Also re-fetch when the popover is opened for immediate feedback.
2026-05-04 21:33:18 +08:00
Junyan Qin
7858d17008 feat: show connector error details for Plugin and Box runtime status
Record Box connector error in BoxService and expose it as
'connector_error' in GET /api/v1/box/status when unavailable.
Display error messages in the dashboard System Status popover
for both Plugin Runtime (plugin_connector_error) and Box Runtime
(connector_error) when they are disconnected.
2026-05-04 21:33:18 +08:00
Junyan Qin
eaffde0f89 refactor(web): compact system status into a single card alongside metrics
Replace the separate two-card row with a single compact 'System Status'
card placed as the 5th column in the metrics grid. Shows green/red dots
for Plugin Runtime and Box Runtime. Click to expand a popover with
connection details (backend, profile, sandbox count).
2026-05-04 21:33:18 +08:00
Junyan Qin
b71f690886 feat(web): move runtime status to dashboard, clean up plugin debug popover
Add SystemStatusCards component to the monitoring dashboard showing
Plugin Runtime and Box Runtime connection status with details (backend,
profile, sandbox count). Remove all Box/session status from the plugin
page debug popover — it now only shows debug URL and key.

Includes i18n for all 8 supported languages.
2026-05-04 21:33:18 +08:00
Junyan Qin
29eadcb5ab feat(box): add heartbeat and reconnection for Box runtime connector
Add 20-second heartbeat ping loop to detect silent Box runtime
disconnections. On disconnect, set available=false and attempt
reconnection after 3 seconds via the disconnect callback chain.

- BoxRuntimeConnector: heartbeat loop, disconnect callback parameter,
  disconnect detection in connection callback and WS failure handler
- BoxService: wire disconnect callback to toggle available state and
  re-initialize the connector on reconnection
2026-05-04 21:33:18 +08:00
Junyan Qin
5a4ec62b14 feat(box): support custom sandbox container image via config.yaml
Add 'image' field to box config section. When set, it overrides the
profile default image (python:3.11-slim) for all sandbox containers.
Priority: caller-specified > config.yaml image > profile default.
2026-05-04 21:33:18 +08:00
Junyan Qin
cbb36139f4 feat(box): add startup and availability logging for sandbox tools
Log Box runtime initialization result (success with profile info, or
failure warning). Log native tool availability status at ToolManager
startup so it's immediately clear whether exec/read/write/edit tools
are registered for the LLM.
2026-05-04 21:33:18 +08:00
Junyan Qin
cee5e9e0e2 feat(web): show active sandbox details in Box status popover
Display sandbox count and a detailed list of active sessions including
session ID, image, backend, resources (CPU/memory), network mode, and
last used time. Fetched from GET /api/v1/box/sessions in parallel.
Includes i18n for all 8 supported languages.
2026-05-04 21:33:18 +08:00
Junyan Qin
7e50063731 feat(box): configurable sandbox scope and unified skill containers
Replace the per-message session_id with a template-based system
configurable per pipeline via 'Sandbox Scope' in the local-agent panel.
Default scope is per-chat ({launcher_type}_{launcher_id}).

Unify skill exec into the same container as default exec — skills are
mounted at /workspace/.skills/{name}/ via extra_mounts instead of
getting separate containers. All pipeline-bound skills are injected
at container creation time.

- Add box-session-id-template to pipeline metadata (select, 4 options, 8 languages)
- Add resolve_box_session_id() and build_skill_extra_mounts() to BoxService
- Rewrite native.py skill exec path to use execute_tool with shared session
- Update tests for new session_id format
- Add design doc: docs/review/box-session-scope.md
2026-05-04 21:33:18 +08:00
Junyan Qin
ec00e49ef1 fix(web): remove ephemeral sandbox count from Box status display
The active_sessions count reflects transient sandbox containers that
expire after 5 minutes of inactivity, making it misleading in the UI.
Keep only connection status, backend, profile, and error count.
2026-05-04 21:33:03 +08:00
Junyan Qin
e2d555a945 feat(web): show Box runtime status in plugin debug info popover
Add Box status section to the debug info popover on the plugin list page,
displaying connection status, backend info, profile, active sessions,
and recent error count. Fetched from GET /api/v1/box/status in parallel
with plugin debug info. Includes i18n for all 8 supported languages.
2026-05-04 21:33:03 +08:00
Junyan Qin
aa40151964 refactor(box): use single port with path-based routing for Box WS
Update connector to use ws://host:5410/rpc/ws instead of ws://host:5411.
Update review docs to reflect the single-port architecture.
2026-05-04 21:33:03 +08:00
Junyan Qin
f4406cd972 feat(box): add --standalone-box flag and 3-way transport decision for Box runtime
Align Box runtime connection logic with Plugin runtime's pattern:
- Docker: WebSocket to langbot_box container (ws://langbot_box:5411)
- --standalone-box: WebSocket to external Box process (ws://localhost:5411)
- Windows: subprocess + WebSocket (workaround for async stdio limitation)
- Unix/macOS: subprocess + stdio pipe (unchanged)

BoxRuntimeConnector now inherits ManagedRuntimeConnector for subprocess
lifecycle reuse. Add langbot_box service to docker-compose.yaml.
2026-05-04 21:33:03 +08:00
Junyan Qin
1b4107a90a refactor: use Space API for release checks and stop idle polling
- version.py: switch release list API from GitHub to space.langbot.app,
  remove unused in-place update logic (update_all, compare_version_str),
  translate all comments/logs to English
- PluginInstallTaskContext: only poll when active install tasks exist
2026-05-04 21:33:03 +08:00
Junyan Qin
c7e8f19f0d fix(deps): update langbot-plugin version and add new dependencies 2026-05-04 21:33:03 +08:00
Junyan Qin
94da5bf05d fix(web): stop polling plugin tasks when no active installs
The PluginInstallTaskProvider was unconditionally polling
getAsyncTasks every 3s on all /home/* routes. Now it only
syncs once on mount and starts periodic polling only when
there are active (non-terminal) install tasks.
2026-05-04 21:33:03 +08:00
Junyan Qin
f6e7983890 refactor(web): replace all hardcoded SVG icons with lucide-react
Unify icon usage across the entire frontend by replacing 67 hardcoded
SVG icons with lucide-react components across ~25 files. This improves
consistency, maintainability, and reduces bundle duplication.

Key replacements:
- Sidebar nav: Zap, LayoutDashboard, Bot, Workflow, BookMarked, etc.
- MCP forms: Loader2, XCircle, Trash2
- Monitoring: Sparkles, MessageSquare, CheckCircle2, RefreshCw, etc.
- Cards: Clock, Star, Workflow, Hexagon, Puzzle, Github, etc.
- Misc: Paperclip, AudioLines, CloudUpload, Layers, Heart, Smile

Zero hardcoded <svg> tags remain in .tsx files.
2026-05-04 21:33:03 +08:00
Junyan Qin
3340e984ed feat(web): improve login error layout and add Terms of Service link
- Improve backend connection error display with bordered container,
  inline icon, and better visual hierarchy
- Extract actual error message from axios response object
- Add Terms of Service link (https://langbot.app/terms) to login footer
- Add termsOfService i18n key for all 7 locales
2026-05-04 21:23:23 +08:00
Junyan Qin
b2ae4a6a82 docs(review): update Box architecture review documents
Replace old review docs with 5 focused documents:
- box-architecture.md: deep architecture analysis (LangBot + SDK)
- box-issues.md: 22 issues rated P0/P1/P2
- box-test-coverage.md: test coverage analysis
- box-tob-analysis.md: toB commercialization analysis
- box-vs-plugin-runtime.md: Box vs Plugin runtime comparison
2026-05-04 21:23:23 +08:00
Junyan Qin
bae6535005 style(web): align plugin list header button heights 2026-05-04 21:23:23 +08:00
Junyan Qin
fad69c70b6 fix(web): prevent first-emission snapshot from swallowing unsaved changes in pipeline editor
When switching runner (e.g. local-agent → n8n), the newly mounted stage's
first emit would re-capture the saved snapshot, erasing the dirty state
caused by the runner change. The save button would incorrectly go dim.

- Skip snapshot re-capture in handleDynamicFormEmit when form is already dirty
- Add mount-time emit to N8nAuthFormComponent (matching DynamicFormComponent)
- Use stable onSubmitRef to prevent useEffect subscription churn
- Add previousInitialValues guard to prevent initialValues echo loops
2026-05-04 21:23:23 +08:00
youhuanghe
2697d82286 refactor(box): run Box Runtime as subprocess inside LangBot container
Remove the separate langbot_box_runtime Docker service. Box Runtime
  now always launches as a local stdio subprocess, regardless of whether
  LangBot runs in Docker or not. The WebSocket transport path is kept
  only for explicit runtime_url configuration (remote deployment).

  This simplifies deployment by eliminating cross-container path mapping
  and network hops. Box Runtime is a pure scheduling process (talks to
  Docker socket / nsjail), it does not execute user code or touch the
  filesystem, so container isolation is unnecessary — unlike Plugin
  Runtime.
2026-05-04 21:23:23 +08:00
youhuanghe
a8eb6e6984 refactor(box): introduce reusable workspace session helper 2026-05-04 21:23:23 +08:00
youhuanghe
51fcf26571 refactor(mcp): extract box stdio runtime helper 2026-05-04 21:23:23 +08:00
huanghuoguoguo
fd68c16056 feat(sandbox): add MCP box integration on top of sandbox base (#2083) 2026-05-04 21:23:23 +08:00
fdc310
4b8a8c5e31 feat(skills): add Agent Skills management system (#1917)
* feat(skills): add Agent Skills management system

Implement comprehensive skills management feature inspired by agentskills spec:

Backend:
- Add Skill and SkillPipelineBinding database entities
- Add database migration (dbm018) for skills tables
- Implement SkillManager for skill loading, matching, and resolution
- Implement SkillService for CRUD operations
- Add skills API endpoints for skill and pipeline binding management
- Integrate skill index injection into pipeline preprocessor
- Add skill activation detection in LocalAgentRunner

Frontend:
- Add Skills page with listing, search, and type filter
- Add SkillDetailDialog for create/edit with preview
- Add SkillCard and SkillForm components
- Add skills API methods to BackendClient
- Add skills entry to sidebar navigation
- Add i18n translations (en-US, zh-Hans)

Features:
- Support skill and workflow types
- Sub-skill composition via {{INVOKE_SKILL: name}} syntax
- Progressive disclosure (index in prompt, full instructions on activation)
- Pipeline-specific skill bindings with priority

* fix: resolve cherry-pick conflicts for agentskills onto sandbox

- Remove non-existent external_kb service import
- Add skill_mgr mock to localagent sandbox_exec tests
- Keep database version at 24 (sandbox branch's latest)

* feat(skills): upgrade to package-backed skills with sandbox execution

  Evolve the skills system from pure prompt-based to package-backed with
  sandbox tool execution support:

  - Add source_type/package_root/entry_file/skill_tools fields to Skill entity
  - SkillManager loads SKILL.md from local package directories
  - SkillToolLoader as 4th dispatch layer in ToolManager (query-scoped)
  - LocalAgent injects skill tools into use_funcs on skill activation
  - BoxService.execute_skill_tool() runs scripts in sandbox (ro mount, env params)
  - Skill tool names auto-namespaced as skill__{skill}__{tool}
  - API validation for package_root allowlist and entry path traversal
  - Frontend source_type toggle, package_root input, skill_tools editor
  - Migration renumbered to 025 with ALTER TABLE fallback for existing DBs
  - Fix unclosed limitation section in i18n files
  - Fix skills API methods misplaced outside BackendClient class

* fix: test info

* feat(skills): switch skills to package-backed storage and add import tooling
  - skills 从 inline/package 双轨收敛成 package-first
  - instructions 改为写入并读取 SKILL.md
  - 新增本地目录扫描和 GitHub 安装 skill
  - 前端把 skills 整合进 plugins 页,新增 SkillsComponent 和 GitHub 导入弹窗
  - skill form 去掉 source_type / type 筛选,改成目录扫描驱动
  - Box skill tool 挂载模式从 ro 改成 rw
  - 测试和中英文文案同步更新

* feat: simplify langbot skill create and import

* refactor(skills): clean up legacy skill API and harden activation flow

* refactor(skills): remove skill dependency expansion and add skill_get

* fix: lint

* fix: delete

* fix(skills): align tool manager loader initialization

* refactor: remove sandbox execute skill

* fix(skills): hide activation markers and isolate skill activation flow

* refactor(skills): switch skill model to filesystem-backed packages

* refactor(skills): switch skill model to filesystem-backed packages

* refactor(skills): unify runtime skill access around filesystem paths

* refactor(skills): unify runtime skill access around filesystem paths

* feat(skills): align rw package design and fix skill activation, visibility, and lint issues

* refactor(skills): replace rich authoring API with import/reload flow and update
  Box design doc

* feat(box): add sandbox_exec tool loop for local-agent calculations

* feat(box): add host workspace mounting and sandbox_exec guidance

* feat(box): add BoxProfile with resource limits and improved output truncation

  - Implement head+tail output truncation (60/40 split) so LLM sees both
    beginning and final results; add streaming byte-limited reads in backend
    to prevent unbounded memory usage (_MAX_RAW_OUTPUT_BYTES = 1MB)
  - Define BoxProfile model with locked fields and max_timeout_sec clamping
  - Add four built-in profiles: default, offline_readonly, network_basic,
    network_extended with differentiated resource and security constraints
  - Add resource limit fields to BoxSpec (cpus, memory_mb, pids_limit,
    read_only_rootfs) and pass corresponding container CLI flags
    (--cpus, --memory, --pids-limit, --read-only, --tmpfs)
  - Profile loaded from config (box.profile), applied in service layer
    before BoxSpec validation; locked fields cannot be overridden by
    tool-call parameters

* feat(box): add obs

* refactor(box): unify box service lifecycle and local runtime
  management

* refactor(box): remove legacy in-process runtime code and clean up smells

After the architecture settled on always using an independent Box Runtime
service, several pieces of compatibility code and design shortcuts were
left behind. This commit cleans them up:

- Remove `LocalBoxRuntimeClient` and `create_box_runtime_client` from
  production code (moved to test-only helper).
- Remove unused `_clip_bytes` method from backend.
- Remove `__langbot_session_placeholder__` hack by making `BoxSpec.cmd`
  default to empty and validating non-empty only in `runtime.execute()`.
- Extract `get_box_config()` helper to eliminate 5× duplicated config
  access boilerplate.
- Remove `session_id`/`host_path`/`host_path_mode` from the LLM-facing
  tool schema to enforce request-scoped session isolation.
- Fix dual shutdown path: `NativeToolLoader.shutdown()` no longer calls
  `box_service.shutdown()` (handled by `Application.dispose()`).
- Simplify `_assert_session_compatible` with a loop.
- Inline client creation in `BoxRuntimeConnector`.
- Remove redundant `BOX__RUNTIME_URL` env var from docker-compose
  (auto-detected by code).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(box/mcp): integrate MCP stdio with Box sandbox — auto-isolation, dep install, security

  ## Summary

  When Podman/Docker is available, all stdio-mode MCP servers now automatically
  run inside Box containers with dependency installation, path rewriting, and
  lifecycle management. When no container runtime exists, LangBot starts normally
  and stdio MCP falls back to host-direct execution.

  ## What changed

  ### MCP stdio → Box integration (mcp.py)
  - Add `MCPServerBoxConfig` pydantic model for structured box configuration
    with validation and defaults (network, host_path_mode, timeouts, resources)
  - Auto-infer `host_path` from command/args with venv detection: recognizes
    `.venv/bin/python` patterns and walks up to the project root
  - Rewrite host paths to container `/workspace` paths transparently
  - Replace venv python commands with container-native `python`
  - Auto-detect `pyproject.toml`/`setup.py`/`requirements.txt` and run
    `pip install` inside the container before starting the MCP server
  - Copy project to `/tmp` before install to handle read-only mounts
  - Add retry with exponential backoff (3 retries, 2s/4s/8s delays)
  - Add Box managed process health monitoring (poll every 5s)
  - Fix session leak: `_cleanup_box_stdio_session()` now runs in `finally`
    block of `_lifecycle_loop`, covering all exit paths
  - Fix retry logic: `_ready_event` is only set after all retries exhaust
    or on success, not on first failure
  - Enhance `get_runtime_info_dict()` with `box_session_id` and `box_enabled`

  ### Box security (security.py — new)
  - `validate_sandbox_security()` blocks dangerous host paths:
    `/etc`, `/proc`, `/sys`, `/dev`, `/root`, `/boot`, `/run`,
    docker.sock, podman socket
  - Called at the start of `CLISandboxBackend.start_session()`

  ### Box models (models.py)
  - Add `BoxHostMountMode.NONE` — skips volume mount entirely
  - Adjust `validate_host_mount_consistency` to allow arbitrary workdir
    when `host_path_mode=NONE`

  ### Box backend (backend.py)
  - Add `validate_sandbox_security()` call in `start_session()`
  - Add `langbot.box.config_hash` label on containers for drift detection
  - Handle `BoxHostMountMode.NONE` — skip `-v` mount arg
  - Add `cleanup_orphaned_containers()` to base class (no-op default) and
    CLI implementation (single batched `rm -f` command)

  ### Box runtime (runtime.py)
  - Call `cleanup_orphaned_containers()` during `initialize()` to remove
    lingering containers from previous runs

  ### Box service (service.py)
  - Graceful degradation: `initialize()` catches runtime errors and sets
    `available=False` instead of crashing LangBot startup
  - Add `available` property and guard on `execute_sandbox_tool()`
  - Add `skip_host_mount_validation` parameter to `build_spec()` and
    `create_session()` — MCP paths are admin-configured and trusted,
    bypassing `allowed_host_mount_roots` restrictions meant for
    LLM-generated sandbox_exec commands

  ### Default behavior
  - stdio MCP servers automatically use Box when `box_service.available`
    is True (Podman/Docker detected); no explicit `box` config needed
  - When no container runtime exists, falls back to host-direct stdio
  - MCP Box defaults: `network=on` (for pip install), `read_only_rootfs=false`
    (for site-packages), `host_path_mode=ro`, `startup_timeout=120s`

  ### Tests
  - `test_box_security.py`: blocked paths, safe paths, subpath rejection
  - `test_mcp_box_integration.py`: config model, path rewriting, venv
    unwrap, host_path inference, payload building, runtime info, box
    availability check
  - `test_box_service.py`: `BoxHostMountMode.NONE` validation tests

* feat(box/mcp): instance-based orphan cleanup, error classification, session API, and integration tests

  ## Changes

  ### Precise orphan container cleanup
  - Runtime generates a unique instance_id on startup
  - Every container gets a `langbot.box.instance_id` label
  - `cleanup_orphaned_containers()` only removes containers from
    previous instances, preserving containers owned by the current one
  - Containers from older versions (no label) are also cleaned up
  - `cleanup_orphaned_containers` added to `BaseSandboxBackend` as
    a no-op default method, removing hasattr duck-typing

  ### Fine-grained MCP error classification
  - New `MCPSessionErrorPhase` enum with 7 phases: session_create,
    dep_install, process_start, relay_connect, mcp_init, runtime,
    tool_call
  - Each phase in `_init_box_stdio_server()` sets the error phase
    before re-raising, enabling precise failure diagnosis
  - `retry_count` tracked across retry attempts
  - `get_runtime_info_dict()` exposes `error_phase` and `retry_count`

  ### GET /v1/sessions/{id} API
  - `BoxRuntime.get_session()` returns session details including
    managed process info when present
  - `handle_get_session` HTTP handler + route in server.py
  - `BoxRuntimeClient.get_session()` abstract method + remote impl

  ### stdio defaults to Box when runtime is available
  - `_uses_box_stdio()` checks `box_service.available` instead of
    requiring explicit `box` key in server_config
  - `BoxService.initialize()` catches runtime errors gracefully,
    sets `available=False` instead of crashing LangBot startup
  - When no container runtime exists, stdio MCP falls back to
    host-direct execution

  ### Code quality (from /simplify review)
  - Extracted `_VENV_DIRS` / `_VENV_BIN_DIRS` module-level constants
  - Removed dead `_box_network_mode()` method and unused `bc` variable
  - Fixed broken import `from ....box.models` → `from ...box.models`
  - Cached `_resolve_host_path()` result — computed once, passed through
  - Config hash now includes `host_path` field
  - Batched orphan cleanup into single `rm -f` command

  ### Session leak fix
  - `_cleanup_box_stdio_session()` now runs in `_lifecycle_loop`'s
    finally block, covering all exit paths (normal shutdown, error,
    retry, final failure)

  ### Integration tests
  - 6 end-to-end tests covering managed process lifecycle, WebSocket
    stdio bidirectional IO, session cleanup verification, single
    session query, process exit detection, and orphan cleanup safety

* refactor: use rpc

* fix: import

* refactor(box): clean up sandbox subsystem code quality and efficiency

  - Fix O(n²) stderr trimming in runtime.py with running length tracker
  - Remove dead code: RESERVED_CONTAINER_PATHS, _subprocess_wait_task,
    unused config_hash computation, unused imports
  - Deduplicate connection callback in BoxRuntimeConnector, parse URL once
  - Use enum comparison instead of stringly-typed spec.network.value check
  - Replace manual _result_to_dict/_session_to_dict with model_dump()
  - Cache NativeToolLoader tool definition and sandbox system guidance
  - Extract _is_path_under() helper to eliminate duplicated path checks
  - Import SANDBOX_EXEC_TOOL_NAME from native.py instead of redefining
  - Add JSON startswith guard in logging_utils to skip futile json.loads
  - Fix ruff lint errors (F401 unused imports, F841 unused variables)

* fix: ruff

* refactor(sandbox): keep box logic out of pipeline and localagent

  - Move sandbox system-prompt guidance from LocalAgentRunner into
    BoxService.get_system_guidance() so all box domain knowledge stays
    in the box module.
  - Remove standalone logging_utils.py; merge format_result_log() into
    MessageHandler base class alongside cut_str().
  - Strip sandbox-specific JSON parsing from log formatting; tool
    results now use generic truncation.
  - Revert TYPE_CHECKING changes in stage.py and runner.py that were
    unrelated to this feature.
  - Skip two test files affected by a pre-existing circular import
    (runner ↔ app) until the import cycle is resolved in a separate PR.

* refactor(box): move box runtime to langbot-plugin-sdk

  Extract self-contained box runtime modules (actions, backend, client,
  errors, models, runtime, security, server) to langbot-plugin-sdk and
  update all imports to use `langbot_plugin.box.*`. Keep only service
  and
  connector in LangBot core as they depend on the Application context.

  - Update docker-compose to use `langbot_plugin.box.server` entry
  point
  - Update pyproject.toml to use local SDK via `tool.uv.sources`
  - Remove migrated source files and their unit/integration tests
  - Update remaining test imports to match new module paths

* fix: ruff

* fix(box): tighten sandbox exposure and restore box integration coverage

* refactor(types): remove quoted annotations under postponed evaluation

* chore(sandbox): move MCP loader changes to follow-up branch

* refactor(plugins): simplify GitHub install flow to default master archive

* revert(api): restore plugin GitHub import flow in plugins controller

* Improve data-root handling and skill install previews

* Add managed skill authoring tools for local agents

* Refactor the skills UI around sidebar detail pages

* Document why managed skill authoring tools bypass box

* fix: lint

* feat(web): refactor plugin/skill install flows and fix skills page

- Fix sidebar skill icon
- Add skills route and error page component
- Refactor plugin GitHub install from dialog modal to inline card
- Add skill install dropdown menu (create/upload/github) in sidebar
- Wire sidebar → skills page communication via pendingSkillInstallAction context
- Add i18n keys for error page and skill install actions

* fix(web): persist sidebar collapsible section open state on navigation

Sections opened via sub-item navigation now retain their expanded state
when the user switches to a different section, instead of collapsing
because the isActive fallback becomes false.

---------

Co-authored-by: youhuanghe <1051233107@qq.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: Junyan Qin <rockchinq@gmail.com>
2026-05-04 21:23:23 +08:00
youhuanghe
fcf74c3b6c feat(box): add session workspace quota enforcement and SDK quota metadata 2026-05-04 21:23:23 +08:00
youhuanghe
0f00269a08 chore(sandbox): move MCP loader changes to follow-up branch 2026-05-04 21:23:23 +08:00
youhuanghe
93104a947a feat(box): unify native agent tools around exec/read/write/edit 2026-05-04 21:23:23 +08:00
youhuanghe
3f368c5764 refactor(types): remove quoted annotations under postponed evaluation 2026-05-04 21:23:23 +08:00
youhuanghe
2911220054 fix(box): tighten sandbox exposure and restore box integration coverage 2026-05-04 21:23:23 +08:00
youhuanghe
63d22b1f8e refactor(box): derive paths from shared host root 2026-05-04 21:23:23 +08:00
youhuanghe
bfeb8315aa feat: enhance sandbox api 2026-05-04 21:23:23 +08:00
youhuanghe
9e0fa375e9 fix: ruff 2026-05-04 21:23:23 +08:00
youhuanghe
b64a23f9ac refactor(box): move box runtime to langbot-plugin-sdk
Extract self-contained box runtime modules (actions, backend, client,
  errors, models, runtime, security, server) to langbot-plugin-sdk and
  update all imports to use `langbot_plugin.box.*`. Keep only service
  and
  connector in LangBot core as they depend on the Application context.

  - Update docker-compose to use `langbot_plugin.box.server` entry
  point
  - Update pyproject.toml to use local SDK via `tool.uv.sources`
  - Remove migrated source files and their unit/integration tests
  - Update remaining test imports to match new module paths
2026-05-04 21:23:23 +08:00
youhuanghe
c095e830c7 fix: ruff 2026-05-04 21:23:23 +08:00
youhuanghe
42fa75331b refactor(sandbox): keep box logic out of pipeline and localagent
- Move sandbox system-prompt guidance from LocalAgentRunner into
    BoxService.get_system_guidance() so all box domain knowledge stays
    in the box module.
  - Remove standalone logging_utils.py; merge format_result_log() into
    MessageHandler base class alongside cut_str().
  - Strip sandbox-specific JSON parsing from log formatting; tool
    results now use generic truncation.
  - Revert TYPE_CHECKING changes in stage.py and runner.py that were
    unrelated to this feature.
  - Skip two test files affected by a pre-existing circular import
    (runner ↔ app) until the import cycle is resolved in a separate PR.
2026-05-04 21:23:23 +08:00
youhuanghe
a7664d1665 fix: ruff 2026-05-04 21:23:23 +08:00
youhuanghe
76fbd08680 refactor(box): clean up sandbox subsystem code quality and efficiency
- Fix O(n²) stderr trimming in runtime.py with running length tracker
  - Remove dead code: RESERVED_CONTAINER_PATHS, _subprocess_wait_task,
    unused config_hash computation, unused imports
  - Deduplicate connection callback in BoxRuntimeConnector, parse URL once
  - Use enum comparison instead of stringly-typed spec.network.value check
  - Replace manual _result_to_dict/_session_to_dict with model_dump()
  - Cache NativeToolLoader tool definition and sandbox system guidance
  - Extract _is_path_under() helper to eliminate duplicated path checks
  - Import SANDBOX_EXEC_TOOL_NAME from native.py instead of redefining
  - Add JSON startswith guard in logging_utils to skip futile json.loads
  - Fix ruff lint errors (F401 unused imports, F841 unused variables)
2026-05-04 21:23:23 +08:00
youhuanghe
fbe6e145ec fix: import 2026-05-04 21:23:23 +08:00
youhuanghe
14057d1722 refactor: use rpc 2026-05-04 21:23:23 +08:00
youhuanghe
791d052687 feat(box/mcp): instance-based orphan cleanup, error classification, session API, and integration tests
## Changes

  ### Precise orphan container cleanup
  - Runtime generates a unique instance_id on startup
  - Every container gets a `langbot.box.instance_id` label
  - `cleanup_orphaned_containers()` only removes containers from
    previous instances, preserving containers owned by the current one
  - Containers from older versions (no label) are also cleaned up
  - `cleanup_orphaned_containers` added to `BaseSandboxBackend` as
    a no-op default method, removing hasattr duck-typing

  ### Fine-grained MCP error classification
  - New `MCPSessionErrorPhase` enum with 7 phases: session_create,
    dep_install, process_start, relay_connect, mcp_init, runtime,
    tool_call
  - Each phase in `_init_box_stdio_server()` sets the error phase
    before re-raising, enabling precise failure diagnosis
  - `retry_count` tracked across retry attempts
  - `get_runtime_info_dict()` exposes `error_phase` and `retry_count`

  ### GET /v1/sessions/{id} API
  - `BoxRuntime.get_session()` returns session details including
    managed process info when present
  - `handle_get_session` HTTP handler + route in server.py
  - `BoxRuntimeClient.get_session()` abstract method + remote impl

  ### stdio defaults to Box when runtime is available
  - `_uses_box_stdio()` checks `box_service.available` instead of
    requiring explicit `box` key in server_config
  - `BoxService.initialize()` catches runtime errors gracefully,
    sets `available=False` instead of crashing LangBot startup
  - When no container runtime exists, stdio MCP falls back to
    host-direct execution

  ### Code quality (from /simplify review)
  - Extracted `_VENV_DIRS` / `_VENV_BIN_DIRS` module-level constants
  - Removed dead `_box_network_mode()` method and unused `bc` variable
  - Fixed broken import `from ....box.models` → `from ...box.models`
  - Cached `_resolve_host_path()` result — computed once, passed through
  - Config hash now includes `host_path` field
  - Batched orphan cleanup into single `rm -f` command

  ### Session leak fix
  - `_cleanup_box_stdio_session()` now runs in `_lifecycle_loop`'s
    finally block, covering all exit paths (normal shutdown, error,
    retry, final failure)

  ### Integration tests
  - 6 end-to-end tests covering managed process lifecycle, WebSocket
    stdio bidirectional IO, session cleanup verification, single
    session query, process exit detection, and orphan cleanup safety
2026-05-04 21:23:23 +08:00
youhuanghe
e8aa7b2e6d feat(box/mcp): integrate MCP stdio with Box sandbox — auto-isolation, dep install, security
## Summary

  When Podman/Docker is available, all stdio-mode MCP servers now automatically
  run inside Box containers with dependency installation, path rewriting, and
  lifecycle management. When no container runtime exists, LangBot starts normally
  and stdio MCP falls back to host-direct execution.

  ## What changed

  ### MCP stdio → Box integration (mcp.py)
  - Add `MCPServerBoxConfig` pydantic model for structured box configuration
    with validation and defaults (network, host_path_mode, timeouts, resources)
  - Auto-infer `host_path` from command/args with venv detection: recognizes
    `.venv/bin/python` patterns and walks up to the project root
  - Rewrite host paths to container `/workspace` paths transparently
  - Replace venv python commands with container-native `python`
  - Auto-detect `pyproject.toml`/`setup.py`/`requirements.txt` and run
    `pip install` inside the container before starting the MCP server
  - Copy project to `/tmp` before install to handle read-only mounts
  - Add retry with exponential backoff (3 retries, 2s/4s/8s delays)
  - Add Box managed process health monitoring (poll every 5s)
  - Fix session leak: `_cleanup_box_stdio_session()` now runs in `finally`
    block of `_lifecycle_loop`, covering all exit paths
  - Fix retry logic: `_ready_event` is only set after all retries exhaust
    or on success, not on first failure
  - Enhance `get_runtime_info_dict()` with `box_session_id` and `box_enabled`

  ### Box security (security.py — new)
  - `validate_sandbox_security()` blocks dangerous host paths:
    `/etc`, `/proc`, `/sys`, `/dev`, `/root`, `/boot`, `/run`,
    docker.sock, podman socket
  - Called at the start of `CLISandboxBackend.start_session()`

  ### Box models (models.py)
  - Add `BoxHostMountMode.NONE` — skips volume mount entirely
  - Adjust `validate_host_mount_consistency` to allow arbitrary workdir
    when `host_path_mode=NONE`

  ### Box backend (backend.py)
  - Add `validate_sandbox_security()` call in `start_session()`
  - Add `langbot.box.config_hash` label on containers for drift detection
  - Handle `BoxHostMountMode.NONE` — skip `-v` mount arg
  - Add `cleanup_orphaned_containers()` to base class (no-op default) and
    CLI implementation (single batched `rm -f` command)

  ### Box runtime (runtime.py)
  - Call `cleanup_orphaned_containers()` during `initialize()` to remove
    lingering containers from previous runs

  ### Box service (service.py)
  - Graceful degradation: `initialize()` catches runtime errors and sets
    `available=False` instead of crashing LangBot startup
  - Add `available` property and guard on `execute_sandbox_tool()`
  - Add `skip_host_mount_validation` parameter to `build_spec()` and
    `create_session()` — MCP paths are admin-configured and trusted,
    bypassing `allowed_host_mount_roots` restrictions meant for
    LLM-generated sandbox_exec commands

  ### Default behavior
  - stdio MCP servers automatically use Box when `box_service.available`
    is True (Podman/Docker detected); no explicit `box` config needed
  - When no container runtime exists, falls back to host-direct stdio
  - MCP Box defaults: `network=on` (for pip install), `read_only_rootfs=false`
    (for site-packages), `host_path_mode=ro`, `startup_timeout=120s`

  ### Tests
  - `test_box_security.py`: blocked paths, safe paths, subpath rejection
  - `test_mcp_box_integration.py`: config model, path rewriting, venv
    unwrap, host_path inference, payload building, runtime info, box
    availability check
  - `test_box_service.py`: `BoxHostMountMode.NONE` validation tests
2026-05-04 21:23:23 +08:00
youhuanghe
c802dc8029 fix: fix box intergration test 2026-05-04 21:23:23 +08:00
youhuanghe
55fc0caf2b feat: add test 2026-05-04 21:23:23 +08:00
youhuanghe
6391678fdb refactor(box): remove legacy in-process runtime code and clean up smells
After the architecture settled on always using an independent Box Runtime
service, several pieces of compatibility code and design shortcuts were
left behind. This commit cleans them up:

- Remove `LocalBoxRuntimeClient` and `create_box_runtime_client` from
  production code (moved to test-only helper).
- Remove unused `_clip_bytes` method from backend.
- Remove `__langbot_session_placeholder__` hack by making `BoxSpec.cmd`
  default to empty and validating non-empty only in `runtime.execute()`.
- Extract `get_box_config()` helper to eliminate 5× duplicated config
  access boilerplate.
- Remove `session_id`/`host_path`/`host_path_mode` from the LLM-facing
  tool schema to enforce request-scoped session isolation.
- Fix dual shutdown path: `NativeToolLoader.shutdown()` no longer calls
  `box_service.shutdown()` (handled by `Application.dispose()`).
- Simplify `_assert_session_compatible` with a loop.
- Inline client creation in `BoxRuntimeConnector`.
- Remove redundant `BOX__RUNTIME_URL` env var from docker-compose
  (auto-detected by code).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-05-04 21:23:23 +08:00
youhuanghe
eaae31edd0 refactor(box): unify box service lifecycle and local runtime
management
2026-05-04 21:23:23 +08:00
youhuanghe
15c03fe96b feat(box): add obs 2026-05-04 21:23:23 +08:00
youhuanghe
86b2d517f2 feat(box): add BoxProfile with resource limits and improved output truncation
- Implement head+tail output truncation (60/40 split) so LLM sees both
    beginning and final results; add streaming byte-limited reads in backend
    to prevent unbounded memory usage (_MAX_RAW_OUTPUT_BYTES = 1MB)
  - Define BoxProfile model with locked fields and max_timeout_sec clamping
  - Add four built-in profiles: default, offline_readonly, network_basic,
    network_extended with differentiated resource and security constraints
  - Add resource limit fields to BoxSpec (cpus, memory_mb, pids_limit,
    read_only_rootfs) and pass corresponding container CLI flags
    (--cpus, --memory, --pids-limit, --read-only, --tmpfs)
  - Profile loaded from config (box.profile), applied in service layer
    before BoxSpec validation; locked fields cannot be overridden by
    tool-call parameters
2026-05-04 21:23:23 +08:00
youhuanghe
70c56af4ee feat(box): add host workspace mounting and sandbox_exec guidance 2026-05-04 21:23:23 +08:00
youhuanghe
ba7a45713d feat(box): add sandbox_exec tool loop for local-agent calculations 2026-05-04 21:23:23 +08:00
WangCham
3b3deec080 feat: modify frontend 2026-05-04 17:50:19 +08:00
WangCham
58ec377413 feat: add filter 2026-05-02 23:02:56 +08:00
WangCham
7c50aabe65 feat: add mcp and skills 2026-05-02 17:38:18 +08:00
302 changed files with 22851 additions and 41400 deletions

View File

@@ -15,14 +15,10 @@ on:
branches:
- master
- develop
paths:
- 'src/langbot/**'
- 'tests/**'
- '.github/workflows/run-tests.yml'
- 'pyproject.toml'
- 'uv.lock'
- 'run_tests.sh'
- 'scripts/test-*.sh'
- 'feat/**'
# No path filter on push: every push to the branches above runs the
# full unit-test suite. feat/** branches in particular must be tested
# on every push (they accumulate large changes before a PR exists).
jobs:
test:

View File

@@ -25,7 +25,7 @@
<a href="https://link.langbot.app/zh/docs/guide">文档</a>
<a href="https://link.langbot.app/zh/docs/api">API</a>
<a href="https://space.langbot.app/cloud">Cloud</a>
<a href="https://space.langbot.app">插件市场</a>
<a href="https://space.langbot.app">扩展市场</a>
<a href="https://langbot.featurebase.app/roadmap">路线图</a>
</div>

View File

@@ -1,163 +0,0 @@
#!/usr/bin/env python3
"""Compare YAML node definitions with frontend node-configs."""
import yaml
import os
import re
import json
# 1. Parse YAML files
yaml_dir = 'src/langbot/templates/metadata/nodes'
yaml_nodes = {}
for filename in sorted(os.listdir(yaml_dir)):
if filename.endswith('.yaml'):
filepath = os.path.join(yaml_dir, filename)
with open(filepath, 'r') as f:
data = yaml.safe_load(f)
node_name = data.get('name', filename.replace('.yaml', ''))
yaml_nodes[node_name] = {
'category': data.get('category', ''),
'inputs': [i['name'] for i in data.get('inputs', [])],
'outputs': [o['name'] for o in data.get('outputs', [])],
'config': [c['name'] for c in data.get('config', [])]
}
# 2. Parse frontend node-configs TypeScript files
node_configs_dir = 'web/src/app/home/workflows/components/workflow-editor/node-configs'
frontend_nodes = {}
def parse_ts_file(filepath):
"""Parse a TypeScript file to extract node configurations."""
with open(filepath, 'r') as f:
content = f.read()
# Find all node type definitions
# Pattern: nodeType: 'xxx'
node_type_pattern = r"nodeType:\s*'([^']+)'"
node_types = re.findall(node_type_pattern, content)
# For each node type, extract inputs, outputs, and config
for node_type in node_types:
# Find the config object for this node type
# Look for the section between this nodeType and the next one or end of object
pattern = rf"nodeType:\s*'({re.escape(node_type)})'.*?(?=nodeType:|export\s+(const|function)|$)"
match = re.search(pattern, content, re.DOTALL)
if match:
section = match.group(0)
# Extract inputs
inputs = re.findall(r"createInput\('([^']+)'", section)
# Extract outputs
outputs = re.findall(r"createOutput\('([^']+)'", section)
# Extract config names
config_names = re.findall(r"name:\s*'([^']+)'", section)
# Remove duplicates while preserving order
seen = set()
unique_config = []
for c in config_names:
if c not in seen:
seen.add(c)
unique_config.append(c)
frontend_nodes[node_type] = {
'inputs': inputs,
'outputs': outputs,
'config': unique_config
}
# Parse all config files
for filename in os.listdir(node_configs_dir):
if filename.endswith('.ts') and filename != 'types.ts' and filename != 'index.ts':
filepath = os.path.join(node_configs_dir, filename)
parse_ts_file(filepath)
# 3. Compare and report differences
print("=" * 80)
print("WORKFLOW NODE COMPARISON REPORT: YAML vs Frontend")
print("=" * 80)
all_node_types = sorted(set(list(yaml_nodes.keys()) + list(frontend_nodes.keys())))
discrepancies = []
for node_type in all_node_types:
yaml_def = yaml_nodes.get(node_type)
frontend_def = frontend_nodes.get(node_type)
node_discrepancies = []
if not yaml_def:
print(f"\n⚠️ {node_type}: ONLY in frontend (not in YAML)")
continue
if not frontend_def:
print(f"\n⚠️ {node_type}: ONLY in YAML (not in frontend)")
continue
# Compare inputs
yaml_inputs = set(yaml_def['inputs'])
frontend_inputs = set(frontend_def['inputs'])
if yaml_inputs != frontend_inputs:
only_yaml = yaml_inputs - frontend_inputs
only_frontend = frontend_inputs - yaml_inputs
node_discrepancies.append({
'type': 'inputs',
'only_yaml': list(only_yaml),
'only_frontend': list(only_frontend)
})
# Compare outputs
yaml_outputs = set(yaml_def['outputs'])
frontend_outputs = set(frontend_def['outputs'])
if yaml_outputs != frontend_outputs:
only_yaml = yaml_outputs - frontend_outputs
only_frontend = frontend_outputs - yaml_outputs
node_discrepancies.append({
'type': 'outputs',
'only_yaml': list(only_yaml),
'only_frontend': list(only_frontend)
})
# Compare config
yaml_config = set(yaml_def['config'])
frontend_config = set(frontend_def['config'])
if yaml_config != frontend_config:
only_yaml = yaml_config - frontend_config
only_frontend = frontend_config - yaml_config
node_discrepancies.append({
'type': 'config',
'only_yaml': list(only_yaml),
'only_frontend': list(only_frontend)
})
if node_discrepancies:
print(f"\n{node_type} ({yaml_def['category']}): HAS DISCREPANCIES")
for d in node_discrepancies:
print(f" {d['type']}:")
if d['only_yaml']:
print(f" Only in YAML: {d['only_yaml']}")
if d['only_frontend']:
print(f" Only in Frontend: {d['only_frontend']}")
discrepancies.append((node_type, node_discrepancies))
else:
print(f"\n{node_type} ({yaml_def['category']}): OK")
print(f"\n{'=' * 80}")
print(f"SUMMARY: {len(discrepancies)} nodes with discrepancies out of {len(all_node_types)} total")
print(f"{'=' * 80}")
# Output as JSON for further processing
output = {
'yaml_nodes': {k: v for k, v in yaml_nodes.items()},
'frontend_nodes': {k: v for k, v in frontend_nodes.items()},
'discrepancies': {k: v for k, v in discrepancies}
}
with open('node_comparison.json', 'w') as f:
json.dump(output, f, indent=2)
print(f"\nDetailed comparison saved to node_comparison.json")

View File

@@ -18,6 +18,40 @@ services:
networks:
- langbot_network
# The Box sandbox runtime is optional. It is only started when you run
# ``docker compose --profile box up`` (or ``docker compose --profile all
# up``). With Box off, LangBot keeps the dashboard / skills list visible
# (read-only) but disables sandbox tools, skill add/edit and stdio MCP —
# set ``box.enabled: false`` in ``data/config.yaml`` (or
# ``BOX__ENABLED=false`` in the langbot service env below) to match.
langbot_box:
image: rockchin/langbot:latest
container_name: langbot_box
profiles: ["box", "all"]
volumes:
# Keep the source and target path identical because langbot_box uses the
# host Docker socket to create sandbox containers. Override
# LANGBOT_BOX_ROOT with an absolute path if you do not want the default.
- ${LANGBOT_BOX_ROOT:-${PWD}/data/box}:${LANGBOT_BOX_ROOT:-${PWD}/data/box}
# Mount container runtime socket for Box sandbox backend.
# Uncomment the one that matches your container runtime:
# - /var/run/podman/podman.sock:/var/run/podman/podman.sock # Podman
- /var/run/docker.sock:/var/run/docker.sock # Docker
restart: on-failure
environment:
- TZ=Asia/Shanghai
# The Box runtime does NOT read box.local.* from config.yaml or env; it
# receives its configuration from LangBot via the INIT RPC action.
# Do not add LANGBOT_BOX_* / BOX__* here — they would be silently ignored.
# Launched through the same CLI entry point as the plugin runtime
# (`langbot_plugin.cli.__init__ <subcommand>`). WebSocket is the default
# control transport — mirrors `rt`, which also runs with no flag. Pass
# `-s` / `--stdio-control` only for the stdio mode LangBot uses outside
# containers.
command: ["uv", "run", "--no-sync", "-m", "langbot_plugin.cli.__init__", "box"]
networks:
- langbot_network
langbot:
image: rockchin/langbot:latest
container_name: langbot
@@ -26,6 +60,13 @@ services:
restart: on-failure
environment:
- TZ=Asia/Shanghai
# Unified env-override convention: SECTION__SUBSECTION__KEY overrides the
# matching config.yaml field (see LoadConfigStage). These map onto
# box.local.* and are forwarded to the Box runtime via INIT RPC.
- BOX__LOCAL__HOST_ROOT=${LANGBOT_BOX_ROOT:-${PWD}/data/box}
- BOX__LOCAL__DEFAULT_WORKSPACE=default
- BOX__LOCAL__SKILLS_ROOT=skills
- BOX__LOCAL__ALLOWED_MOUNT_ROOTS=${LANGBOT_BOX_ROOT:-${PWD}/data/box}
ports:
- 5300:5300 # For web ui and webhook callback
- 2280-2285:2280-2285 # For platform reverse connection
@@ -34,4 +75,4 @@ services:
networks:
langbot_network:
driver: bridge
driver: bridge

View File

@@ -1,713 +0,0 @@
# Workflow 系统开发者文档
本文档面向 LangBot 开发者,详细介绍 Workflow 系统的技术架构、核心组件和扩展方法。
## 目录
- [系统架构概述](#系统架构概述)
- [目录结构](#目录结构)
- [核心组件](#核心组件)
- [后端模块](#后端模块)
- [前端组件](#前端组件)
- [数据库表结构](#数据库表结构)
- [API 接口文档](#api-接口文档)
- [如何添加新节点类型](#如何添加新节点类型)
- [调试功能实现](#调试功能实现)
---
## 系统架构概述
Workflow 系统采用前后端分离架构,主要包含以下层次:
```
┌─────────────────────────────────────────────────────────────┐
│ 前端层 (React) │
│ ┌─────────────┬──────────────┬──────────────┬───────────┐ │
│ │ 可视化编辑器 │ 节点面板 │ 属性面板 │ 调试器 │ │
│ │ ReactFlow │ NodePalette │ PropertyPanel│ Debugger │ │
│ └─────────────┴──────────────┴──────────────┴───────────┘ │
├─────────────────────────────────────────────────────────────┤
│ API 层 (Quart) │
│ ┌─────────────┬──────────────┬──────────────────────────┐ │
│ │ Workflow API│ Debug API │ Node Types API │ │
│ └─────────────┴──────────────┴──────────────────────────┘ │
├─────────────────────────────────────────────────────────────┤
│ 核心引擎层 (Python) │
│ ┌─────────────┬──────────────┬──────────────┬───────────┐ │
│ │ Executor │ Registry │ Node │ Entities │ │
│ │ 执行引擎 │ 节点注册表 │ 节点基类 │ 数据结构 │ │
│ └─────────────┴──────────────┴──────────────┴───────────┘ │
├─────────────────────────────────────────────────────────────┤
│ 存储层 (SQLAlchemy) │
│ ┌─────────────┬──────────────┬──────────────────────────┐ │
│ │ Workflow │ Executions │ Triggers │ │
│ └─────────────┴──────────────┴──────────────────────────┘ │
└─────────────────────────────────────────────────────────────┘
```
---
## 目录结构
### 后端代码结构
```
LangBot/src/langbot/pkg/
├── workflow/ # Workflow 核心模块
│ ├── __init__.py # 模块初始化,导出公共接口
│ ├── entities.py # 数据实体定义
│ ├── executor.py # 执行引擎
│ ├── node.py # 节点基类和装饰器
│ ├── registry.py # 节点类型注册表
│ └── nodes/ # 内置节点实现
│ ├── __init__.py # 注册所有内置节点
│ ├── trigger.py # 触发节点
│ ├── process.py # 处理节点
│ ├── control.py # 控制节点
│ └── action.py # 动作节点
├── entity/persistence/
│ └── workflow.py # 数据库模型
├── api/http/
│ ├── controller/groups/workflows/
│ │ └── workflows.py # API 路由控制器
│ └── service/
│ └── workflow.py # 业务逻辑服务
└── persistence/migrations/
└── dbm026_workflow_tables.py # 数据库迁移
```
### 前端代码结构
```
LangBot/web/src/app/home/workflows/
├── page.tsx # Workflow 列表页
├── WorkflowDetailContent.tsx # 详情页内容
├── store/
│ └── useWorkflowStore.ts # Zustand 状态管理
└── components/
├── workflow-editor/ # 可视化编辑器
│ ├── index.ts # 导出
│ ├── WorkflowEditorComponent.tsx # 主编辑器组件
│ ├── WorkflowNodeComponent.tsx # 自定义节点组件
│ ├── NodePalette.tsx # 节点面板
│ ├── PropertyPanel.tsx # 属性面板
│ └── node-configs/ # 节点配置元数据
│ ├── types.ts # 配置类型定义
│ ├── trigger-configs.ts
│ ├── ai-configs.ts
│ ├── process-configs.ts
│ ├── control-configs.ts
│ ├── action-configs.ts
│ ├── integration-configs.ts
│ └── index.ts # 配置汇总
├── workflow-debugger/ # 调试器组件
│ ├── index.ts
│ └── WorkflowDebugger.tsx
├── workflow-form/ # 表单组件
│ └── WorkflowFormComponent.tsx
└── workflow-executions/ # 执行历史组件
└── WorkflowExecutionsTab.tsx
```
---
## 核心组件
### 后端模块
#### 1. 执行引擎 (WorkflowExecutor)
位置:[`executor.py`](../../src/langbot/pkg/workflow/executor.py)
执行引擎负责工作流的实际执行,包括:
- **拓扑排序**:确定节点执行顺序
- **节点执行**:调用各节点的 execute 方法
- **控制流处理**:处理条件分支、循环、并行执行
- **错误处理**:支持重试机制
```python
class WorkflowExecutor:
async def execute(
self,
workflow: WorkflowDefinition,
context: ExecutionContext,
start_node_id: Optional[str] = None
) -> ExecutionContext:
"""执行工作流"""
# 1. 构建执行图
# 2. 初始化节点状态
# 3. 找到起始节点
# 4. 按拓扑顺序执行
```
**调试执行器 (DebugWorkflowExecutor)**
继承自 WorkflowExecutor增加了调试支持
- 断点支持
- 单步执行
- 暂停/继续
- 实时日志
```python
class DebugWorkflowExecutor(WorkflowExecutor):
async def execute_debug(
self,
workflow: WorkflowDefinition,
context: ExecutionContext,
debug_state: DebugExecutionState,
) -> ExecutionContext:
"""调试模式执行"""
```
#### 2. 节点注册表 (NodeTypeRegistry)
位置:[`registry.py`](../../src/langbot/pkg/workflow/registry.py)
单例模式管理所有节点类型:
```python
class NodeTypeRegistry:
_instance: Optional['NodeTypeRegistry'] = None
def register(self, node_type: str, node_class: type[WorkflowNode]):
"""注册节点类型"""
def create_instance(self, node_type: str, node_id: str, config: dict) -> WorkflowNode:
"""创建节点实例"""
def list_all(self) -> list[dict]:
"""获取所有节点类型的 Schema"""
```
#### 3. 节点基类 (WorkflowNode)
位置:[`node.py`](../../src/langbot/pkg/workflow/node.py)
所有节点必须继承此基类:
```python
class WorkflowNode(abc.ABC):
# 节点元数据
type_name: str = ""
name: str = ""
description: str = ""
category: str = "misc"
icon: str = ""
# 端口定义
inputs: list[NodePort] = []
outputs: list[NodePort] = []
# 配置 Schema
config_schema: list[NodeConfig] = []
@abc.abstractmethod
async def execute(
self,
inputs: dict[str, Any],
context: ExecutionContext
) -> dict[str, Any]:
"""执行节点逻辑"""
pass
```
#### 4. 数据实体 (entities.py)
主要数据结构:
```python
class WorkflowDefinition:
"""工作流定义"""
uuid: str
name: str
nodes: list[NodeDefinition]
edges: list[EdgeDefinition]
settings: WorkflowSettings
class ExecutionContext:
"""执行上下文"""
execution_id: str
workflow_id: str
status: ExecutionStatus
variables: dict
node_states: dict[str, NodeState]
history: list[ExecutionStep]
```
### 前端组件
#### 1. WorkflowEditorComponent
主编辑器组件,基于 React Flow 实现:
- **画布交互**:拖拽、缩放、平移
- **节点连接**:自动验证端口类型
- **撤销/重做**:基于历史记录栈
- **复制/粘贴**:支持多选复制
关键功能:
```tsx
function WorkflowEditorInner() {
const { nodes, edges, onNodesChange, onEdgesChange, onConnect } = useWorkflowStore();
// 拖放添加节点
const onDrop = useCallback((event: React.DragEvent) => {
const type = event.dataTransfer.getData('application/reactflow');
const position = screenToFlowPosition({ x: event.clientX, y: event.clientY });
addNode(type, position);
}, []);
// 复制粘贴
const handleCopy = useCallback(() => { ... }, []);
const handlePaste = useCallback(() => { ... }, []);
}
```
#### 2. NodePalette
节点面板组件,展示可用节点类型:
```tsx
function NodePalette() {
// 按类别组织节点
const categories = [
{ id: 'trigger', name: '触发节点', icon: Zap },
{ id: 'ai', name: 'AI 节点', icon: Brain },
{ id: 'process', name: '处理节点', icon: Cpu },
{ id: 'control', name: '控制节点', icon: GitBranch },
{ id: 'action', name: '动作节点', icon: Send },
{ id: 'integration', name: '集成节点', icon: Plug },
];
// 拖拽开始
const onDragStart = (event: React.DragEvent, nodeType: string) => {
event.dataTransfer.setData('application/reactflow', nodeType);
};
}
```
#### 3. PropertyPanel
属性面板组件,动态渲染节点配置表单:
```tsx
function PropertyPanel() {
const { selectedNodeId, nodes, updateNodeData } = useWorkflowStore();
// 根据节点类型获取配置元数据
const selectedNode = nodes.find(n => n.id === selectedNodeId);
const nodeConfig = getNodeConfig(selectedNode?.data?.nodeType);
// 动态渲染配置字段
return (
<div>
{nodeConfig?.fields.map(field => (
<ConfigField key={field.name} field={field} />
))}
</div>
);
}
```
#### 4. WorkflowDebugger
调试器组件,支持实时调试:
```tsx
function WorkflowDebugger({ workflowUuid, workflow }) {
const [debugState, setDebugState] = useState<DebugState>('idle');
const [executionId, setExecutionId] = useState<string>('');
const [logs, setLogs] = useState<ExecutionLog[]>([]);
// 启动调试
const startDebug = async () => {
const result = await backendClient.post(
`/api/v1/workflows/${workflowUuid}/debug/start`,
{ context, variables, breakpoints }
);
setExecutionId(result.execution_id);
};
// 轮询状态
useEffect(() => {
if (debugState === 'running') {
const interval = setInterval(fetchState, 500);
return () => clearInterval(interval);
}
}, [debugState]);
}
```
#### 5. useWorkflowStore
Zustand 状态管理:
```typescript
interface WorkflowState {
nodes: WorkflowNode[];
edges: WorkflowEdge[];
selectedNodeId: string | null;
history: HistoryEntry[];
historyIndex: number;
isDirty: boolean;
// Actions
addNode: (type: string, position: XYPosition) => void;
updateNodeData: (nodeId: string, data: Partial<NodeData>) => void;
deleteNode: (nodeId: string) => void;
undo: () => void;
redo: () => void;
}
export const useWorkflowStore = create<WorkflowState>((set, get) => ({
// ... state and actions
}));
```
---
## 数据库表结构
### workflows 表
```sql
CREATE TABLE workflows (
uuid VARCHAR(255) PRIMARY KEY,
name VARCHAR(255) NOT NULL,
description TEXT,
emoji VARCHAR(10) DEFAULT '🔄',
version INTEGER DEFAULT 1,
is_enabled BOOLEAN DEFAULT TRUE,
definition JSON NOT NULL, -- 节点和边定义
global_config JSON DEFAULT '{}', -- 全局配置
extensions_preferences JSON, -- 插件和 MCP 配置
created_at TIMESTAMP,
updated_at TIMESTAMP
);
```
### workflow_versions 表
```sql
CREATE TABLE workflow_versions (
id INTEGER PRIMARY KEY AUTOINCREMENT,
workflow_uuid VARCHAR(255) NOT NULL,
version INTEGER NOT NULL,
definition JSON NOT NULL,
global_config JSON DEFAULT '{}',
created_at TIMESTAMP,
created_by VARCHAR(255),
UNIQUE(workflow_uuid, version)
);
```
### workflow_executions 表
```sql
CREATE TABLE workflow_executions (
uuid VARCHAR(255) PRIMARY KEY,
workflow_uuid VARCHAR(255) NOT NULL,
workflow_version INTEGER NOT NULL,
status VARCHAR(20) NOT NULL, -- pending/running/completed/failed/cancelled
trigger_type VARCHAR(50),
trigger_data JSON,
variables JSON,
start_time TIMESTAMP,
end_time TIMESTAMP,
error TEXT,
created_at TIMESTAMP
);
```
### workflow_node_executions 表
```sql
CREATE TABLE workflow_node_executions (
id INTEGER PRIMARY KEY AUTOINCREMENT,
execution_uuid VARCHAR(255) NOT NULL,
node_id VARCHAR(100) NOT NULL,
node_type VARCHAR(50) NOT NULL,
status VARCHAR(20) NOT NULL,
inputs JSON,
outputs JSON,
start_time TIMESTAMP,
end_time TIMESTAMP,
error TEXT,
retry_count INTEGER DEFAULT 0
);
```
### workflow_triggers 表
```sql
CREATE TABLE workflow_triggers (
uuid VARCHAR(255) PRIMARY KEY,
workflow_uuid VARCHAR(255) NOT NULL,
type VARCHAR(50) NOT NULL, -- message/cron/event/webhook
config JSON NOT NULL,
is_enabled BOOLEAN DEFAULT TRUE,
priority INTEGER DEFAULT 0,
created_at TIMESTAMP,
updated_at TIMESTAMP
);
```
---
## API 接口文档
### Workflow CRUD
| 方法 | 路径 | 描述 |
|-----|------|------|
| GET | `/api/v1/workflows` | 获取工作流列表 |
| POST | `/api/v1/workflows` | 创建工作流 |
| GET | `/api/v1/workflows/:uuid` | 获取单个工作流 |
| PUT | `/api/v1/workflows/:uuid` | 更新工作流 |
| DELETE | `/api/v1/workflows/:uuid` | 删除工作流 |
| POST | `/api/v1/workflows/:uuid/copy` | 复制工作流 |
### 执行相关
| 方法 | 路径 | 描述 |
|-----|------|------|
| POST | `/api/v1/workflows/:uuid/execute` | 手动执行工作流 |
| GET | `/api/v1/workflows/:uuid/executions` | 获取执行记录 |
### 版本管理
| 方法 | 路径 | 描述 |
|-----|------|------|
| GET | `/api/v1/workflows/:uuid/versions` | 获取版本列表 |
| POST | `/api/v1/workflows/:uuid/rollback/:version` | 回滚到指定版本 |
### 调试 API
| 方法 | 路径 | 描述 |
|-----|------|------|
| POST | `/api/v1/workflows/:uuid/debug/start` | 启动调试 |
| POST | `/api/v1/workflows/:uuid/debug/:exec_id/pause` | 暂停执行 |
| POST | `/api/v1/workflows/:uuid/debug/:exec_id/resume` | 继续执行 |
| POST | `/api/v1/workflows/:uuid/debug/:exec_id/stop` | 停止执行 |
| POST | `/api/v1/workflows/:uuid/debug/:exec_id/step` | 单步执行 |
| GET | `/api/v1/workflows/:uuid/debug/:exec_id/state` | 获取调试状态 |
### 节点类型
| 方法 | 路径 | 描述 |
|-----|------|------|
| GET | `/api/v1/workflows/_/node-types` | 获取所有节点类型 |
| GET | `/api/v1/workflows/_/node-types/categories` | 按类别获取节点类型 |
---
## 如何添加新节点类型
### 步骤 1创建节点类
`LangBot/src/langbot/pkg/workflow/nodes/` 下创建或修改文件:
```python
from ..node import WorkflowNode, NodePort, NodeConfig, workflow_node
from ..entities import ExecutionContext
@workflow_node('my_custom_node')
class MyCustomNode(WorkflowNode):
"""自定义节点"""
# 元数据
type_name = 'my_custom_node'
name = '我的自定义节点'
description = '这是一个自定义节点'
category = 'process' # trigger/process/control/action/integration
icon = '🔧'
# 输入端口
inputs = [
NodePort(name='input', type='string', description='输入数据', required=True),
]
# 输出端口
outputs = [
NodePort(name='output', type='string', description='输出数据'),
]
# 配置字段
config_schema = [
NodeConfig(
name='option',
type='select',
required=True,
options=['选项A', '选项B'],
description='选择一个选项'
),
NodeConfig(
name='value',
type='string',
required=False,
default='默认值',
description='配置值'
),
]
async def execute(
self,
inputs: dict[str, Any],
context: ExecutionContext
) -> dict[str, Any]:
"""执行节点逻辑"""
input_data = inputs.get('input', '')
option = self.get_config('option')
value = self.get_config('value', '')
# 处理逻辑
result = f"处理: {input_data} with {option} and {value}"
return {'output': result}
```
### 步骤 2注册节点
`LangBot/src/langbot/pkg/workflow/nodes/__init__.py` 中导入:
```python
from .process import (
CodeExecutorNode,
HttpRequestNode,
DataTransformNode,
MyCustomNode, # 添加新节点
)
```
### 步骤 3添加前端配置
`LangBot/web/src/app/home/workflows/components/workflow-editor/node-configs/` 目录下添加配置:
```typescript
// process-configs.ts
export const processNodeConfigs: NodeConfigMap = {
// ... 其他配置
my_custom_node: {
type: 'my_custom_node',
label: 'workflows.nodes.myCustomNode',
description: 'workflows.nodes.myCustomNodeDesc',
icon: 'Wrench',
category: 'process',
fields: [
{
name: 'option',
type: 'select',
label: 'workflows.fields.option',
required: true,
options: [
{ value: '选项A', label: '选项 A' },
{ value: '选项B', label: '选项 B' },
],
},
{
name: 'value',
type: 'string',
label: 'workflows.fields.value',
required: false,
defaultValue: '默认值',
},
],
},
};
```
### 步骤 4添加国际化
`LangBot/web/src/i18n/locales/` 中添加翻译:
```typescript
// zh-Hans.ts
workflows: {
nodes: {
myCustomNode: '我的自定义节点',
myCustomNodeDesc: '这是一个自定义节点',
},
fields: {
option: '选项',
value: '值',
},
}
```
---
## 调试功能实现
### 后端调试状态管理
```python
class DebugExecutionState:
"""调试执行状态"""
def __init__(self, execution_id: str, breakpoints: list[str] = None):
self.execution_id = execution_id
self.status: str = 'running'
self.is_paused: bool = False
self.is_stopped: bool = False
self.breakpoints: set[str] = set(breakpoints or [])
self.logs: list[ExecutionLog] = []
self._pause_event = asyncio.Event()
def pause(self):
"""暂停执行"""
self.is_paused = True
self._pause_event.clear()
def resume(self):
"""继续执行"""
self.is_paused = False
self._pause_event.set()
async def wait_if_paused(self):
"""如果暂停则等待"""
if self.is_paused:
await self._pause_event.wait()
```
### 前端调试流程
1. **设置断点**:点击节点设置断点
2. **启动调试**:调用 `/debug/start` 启动调试执行
3. **轮询状态**:定期调用 `/debug/:id/state` 获取状态
4. **控制执行**:调用 pause/resume/step/stop 控制执行
5. **查看日志**:实时显示执行日志和节点状态
```typescript
// 调试状态轮询
const fetchDebugState = async () => {
const state = await backendClient.get(
`/api/v1/workflows/${workflowUuid}/debug/${executionId}/state`
);
// 更新节点状态
setNodeStates(state.node_states);
// 追加新日志
if (state.new_logs.length > 0) {
setLogs(prev => [...prev, ...state.new_logs]);
}
// 检查完成状态
if (state.status === 'completed' || state.status === 'error') {
setDebugState('idle');
}
};
```
---
## 扩展阅读
- [Workflow 功能设计文档](../../../plans/langbot-workflow-design.md)
- [用户使用指南](../user-guide/workflow-guide.md)
- [API 认证文档](../API_KEY_AUTH.md)

View File

@@ -0,0 +1,594 @@
# Box 系统架构深度分析
> 更新日期: 2026-05-19
> 分支: `feat/sandbox` (LangBot + langbot-plugin-sdk)
> 相关文档: [问题清单](./box-issues.md) | [Session 作用域](./box-session-scope.md) | [Runtime 对比](./box-vs-plugin-runtime.md) | [测试覆盖](./box-test-coverage.md) | [toB 分析](./box-tob-analysis.md)
---
## 1. 全局架构
```
┌──────────────────────────────────────────────────────────────────┐
│ LangBot 主进程 │
│ │
│ LocalAgentRunner ──> ToolManager ──> NativeToolLoader │
│ │ │ │ │
│ │ │ exec / read / write / edit │
│ │ │ glob / grep │
│ │ │ │
│ │ ├──> MCPLoader ──> BoxStdioSession │
│ │ │ (shared 容器, 多 process) │
│ │ │ │
│ │ ├──> SkillToolLoader (activate 工具) │
│ │ │ │
│ │ ├──> SkillAuthoringToolLoader │
│ │ │ │
│ │ └──> PluginToolLoader │
│ │ │
│ BoxService (门面) │
│ ├─ Profile 管理 (locked 字段) │
│ ├─ Host mount 校验 (allowed_mount_roots) │
│ ├─ Workspace quota 检查 │
│ ├─ 输出截断 (head+tail) │
│ ├─ Session ID 模板解析 (resolve_box_session_id) │
│ ├─ 技能挂载组装 (build_skill_extra_mounts) │
│ ├─ 重连循环 (_reconnect_loop, 指数退避) │
│ └─ BoxRuntimeConnector │
│ ├─ 心跳 loop (20s ping) │
│ └─ ActionRPCBoxClient │
│ │ Action RPC (stdio 或 WebSocket) │
│ │
│ SkillManager (skill_mgr) │
│ └─ 从 Box runtime 拉取 skills, 不可用时回落 data/skills │
└──────────────────────────────────────────────────────────────────┘
┌──────────────────────────────────────────────────────────────────┐
│ Box Runtime 进程 (SDK 侧) │
│ │
│ BoxServerHandler (Action RPC 处理, INIT 配置注入) │
│ │ │
│ BoxRuntime (session 管理 / 进程生命周期 / TTL reaper) │
│ │ └─ session.managed_processes: dict[pid, _ManagedProcess]
│ │ │
│ Backend (启动时根据 box.backend 配置选择): │
│ DockerBackend ──┐ │
│ PodmanBackend ──┤── CLISandboxBackend │
│ NsjailBackend ──┘ (本地 CLI 或 fallback 到容器内 CLI) │
│ E2BBackend (云沙箱, 需要 E2B_API_KEY) │
│ │
│ BoxSkillStore │
│ ├─ list / get / create / update / delete │
│ ├─ scan_skill_directory / read_skill_file / write_skill_file │
│ └─ preview_skill_zip / install_skill_zip (zip 或 GitHub) │
│ │
│ aiohttp 单端口服务 (默认 :5410): │
│ /rpc/ws — Action RPC │
│ /v1/sessions/{id}/managed-process/ws — 默认 process │
│ /v1/sessions/{id}/managed-process/{pid}/ws — 指定 process │
└──────────────────────────────────────────────────────────────────┘
┌──────────────────────────────────────────────────────────────────┐
│ 容器 / 沙箱 (Docker/Podman 容器, nsjail sandbox, 或 E2B 远程沙箱) │
│ - 隔离文件系统 / 网络 / PID 命名空间 │
│ - 资源限制 (CPU, 内存, PID 数, 可选 workspace 配额) │
│ - 主挂载 (host_path → mount_path) + 任意条 extra_mounts │
│ └─ Skills 通过 extra_mounts 挂在 /workspace/.skills/<name> │
│ - exec: 用户命令在此执行 │
│ - managed process: 多个长驻进程并存 (MCP Server / 自定义服务) │
└──────────────────────────────────────────────────────────────────┘
```
**核心设计原则**:
- Box Runtime 作为独立进程运行,通过 Action RPC 与 LangBot 主进程通信,两者复用 SDK 的 IO 层Handler → Connection → Controller
- 一个 session_id 对应一个容器/沙箱实例。同一 session 内可并存多条 mount 与多个 managed process
- Skill / 默认 exec / MCP Server 共享同一个 session 容器(详见 [box-session-scope.md](./box-session-scope.md)
---
## 2. LangBot 侧模块
### 2.1 BoxService (`pkg/box/service.py`, 722 行)
应用层门面,协调 Profile、安全校验、配额、连接、Skill 挂载与 Session 模板:
主要公开方法(按定义顺序):
```
BoxService
├─ initialize() 连接 Box Runtime + 默认 workspace 准备
├─ _on_runtime_disconnect(connector) 触发重连
├─ _reconnect_loop(connector) 指数退避重连
├─ available (property) 连接状态
├─ resolve_box_session_id(query) 从 pipeline 模板解析 session_id
├─ build_skill_extra_mounts(query) 组装 pipeline-bound skill 的挂载列表
├─ execute_tool(parameters, query) Agent 调用 exec 时的入口
│ ├─ _apply_profile / build_spec
│ ├─ _validate_host_mount
│ ├─ _enforce_workspace_quota (phase=pre)
│ ├─ client.execute(spec)
│ ├─ _enforce_workspace_quota (phase=post)
│ └─ _truncate (stdout/stderr)
├─ execute_spec_payload(spec_payload, ...) 内部入口(其他 loader 调用)
├─ create_session(spec_payload, ...) 显式创建 session
├─ start_managed_process(session_id, ...) 启动 managed process
├─ get_managed_process(session_id, pid) 查询进程状态pid 默认 'default'
├─ stop_managed_process(session_id, pid) 单独停止某个 managed process
├─ get_managed_process_websocket_url(...) 返回 WS attach URL
├─ list_skills() / get_skill(name) Skill 元数据
├─ create_skill / update_skill / delete_skill Skill CRUD
├─ scan_skill_directory(path) 扫描目录
├─ list_skill_files / read_skill_file / write_skill_file
├─ preview_skill_zip / install_skill_zip zip / GitHub 安装
├─ shutdown() / dispose() 清理RPC SHUTDOWN + 进程终止
├─ get_status() / get_sessions() / get_recent_errors()
└─ get_system_guidance() LLM 系统提示
```
**Profile 系统**: 4 个内置 Profile`default` / `offline_readonly` / `network_basic` / `network_extended``locked` frozenset 字段不可被 LLM 覆盖。参数合并顺序Profile defaults → LLM 请求参数 → locked 强制值。
**输出截断**: 默认 4000 字符上限,保留前 60% + 后 40%,中间插入 `[...truncated...]`
**Skill 挂载合并**: `execute_tool()` 调用时,`build_skill_extra_mounts(query)` 会把当前 pipeline-bound 的所有 skill 的 `package_root` 作为 `extra_mounts` 加入 BoxSpec挂在 `/workspace/.skills/<name>`。LLM 通过 `activate` 工具显式激活某个 skill 后,工具调用才允许引用这个 skill 的虚拟路径。
### 2.2 BoxRuntimeConnector (`pkg/box/connector.py`, 357 行)
管理与 Box Runtime 的通信连接:
- **本地 stdio**: Unix/macOS 默认路径fork `python -m langbot_plugin.cli.__init__ box -s --ws-control-port {port}` 子进程(与 plugin runtime 统一走 `lbp` CLI 入口)
- **本地 subprocess + WS**: Windows 本地asyncio ProactorEventLoop 不支持 stdio pipe
- **远程 WebSocket**: Docker 部署 / `box.runtime.endpoint` 显式配置时,连接 `ws://{host}:{port}/rpc/ws`
- **同步等待**: `asyncio.Event` + `wait_for(timeout=30s)` 模式确认连接
- **心跳**: `_heartbeat_loop()` 每 20s 调用 `ping()`,失败仅 DEBUG 日志(断开检测靠 connection close
- **重连**: `runtime_disconnect_callback` 由 BoxService 提供,触发 `_reconnect_loop`
- **INIT 注入**: 连接建立后立即下发当前 `box.*` 配置子树(剔除 `runtime` 私有字段Runtime 据此初始化 backend
> **历史改进**: 2026-04-16 版本本文档曾列 P0 「Box 无心跳 / 无重连」已修复commit `2dfd9d5d`、`c6882cf`、`5029d9c` 等)。
### 2.3 BoxWorkspaceSession 工具 (`pkg/box/workspace.py`, 413 行)
此文件目前提供两类能力:
1. **路径与命令重写工具函数**`normalize_host_path` / `rewrite_mounted_path` / `unwrap_venv_path` / `rewrite_venv_command` / `infer_workspace_host_path`,被 MCP loader 与 Skill 路径解析共用。
2. **`BoxWorkspaceSession`** — 围绕 BoxService 的轻量包装,专供 MCP-in-Box 场景使用(管理一个共享 session 的 session_id、构建挂载 payload、stage host 文件到共享 workspace
**变化点**: 早期 Skill exec 会为每个 skill 创建独立 BoxWorkspaceSession独占 session当前实现已转为 `extra_mounts` 模式Skill 不再独占容器,只追加挂载。这部分 wrapping 逻辑已从 native loader 移除。
### 2.4 policy.py (`pkg/box/policy.py`, 98 行) — 仍是死代码
三层安全策略设计(`SandboxPolicy` / `ToolPolicy` / `ElevatedPolicy`),全项目无任何导入或调用。详见 [问题清单 #1](./box-issues.md)。
### 2.5 SkillManager (`pkg/skill/manager.py`, 186 行)
```
SkillManager
├─ initialize() 调用 reload_skills()
├─ reload_skills() 先从 Box runtime list_skills()
│ 不可用则回落 data/skills/ 扫描
├─ refresh_skill_from_disk() 单 skill 重新加载
├─ get_skill_by_name(name)
└─ get_managed_skills_root() 返回 Box 视角的 skills_root 路径
```
skill 元数据通过 `parse_frontmatter` 解析 `SKILL.md` 头部(`name` / `description` / `instructions`),不再做整体扫描的代价(典型 < 50 个)。
### 2.6 Skill activation (`pkg/skill/activation.py`, 33 行) + Skill loader 辅助
历史上 skill 通过 LLM 在文本中输出 `[ACTIVATE_SKILL:name]` 标记激活;当前已改为 **Tool Call 机制**
- `SkillToolLoader` (`pkg/provider/tools/loaders/skill.py`, 157 行) 暴露 `activate` 工具,参数为 skill 名
- 工具实现调用 `register_activated_skill(query, skill_data)`,将激活态写入 `query.variables['_activated_skills']`
- 这种 KV-cache-friendly 模式对齐 Claude Code 设计;详见 [box-session-scope.md §4.3](./box-session-scope.md) 的 Tool Call 描述
`activation.py` 现仅保留对外辅助函数pipeline 层调用 loader 的 `register_activated_skill`)。
---
## 3. SDK 侧模块
### 3.1 BoxRuntime (`box/runtime.py`, 599 行)
核心编排器,管理 session 生命周期与 backend 调度:
```
Session 生命周期:
Client EXEC / CREATE_SESSION
_get_or_create_session(spec)
├─ _reap_expired_sessions_locked() 清理 TTL 过期 session
├─ 已存在? → _assert_session_compatible() → 复用
├─ Backend session 失踪? → 重建 (commit c6882cf)
└─ 新建? → backend.start_session(spec) → 创建容器
│ └─ 应用 spec.extra_mounts (多挂载)
execute(spec)
├─ 获取 session lock (每 session 独立)
├─ backend.exec(session, spec) 在容器中执行命令
├─ 更新 last_used_at
└─ 超时? → 销毁 session
Session 保持存活直到:
├─ TTL 过期 (默认 300s下次操作时清理)
├─ 执行超时 (自动销毁)
├─ 客户端 DELETE_SESSION
└─ SHUTDOWN
```
**关键设计**:
- 每 session 有独立 `asyncio.Lock`,同一 session 内的命令串行执行
- 每 session 维护 `managed_processes: dict[process_id, _ManagedProcess]`支持多个长驻进程并存MCP / 自定义)
- 全局 `_lock` 保护 `_sessions` dict 的读写
- 兼容性检查:比较核心 spec 字段,`image` 字段对不支持自定义镜像的 backendnsjail/E2B会跳过
**Backend 选择 (`_select_backend`)**: 优先级
1. 显式 `box.backend` 配置(`docker` / `nsjail` / `e2b`
2. `local` (默认) → Docker / Podman / nsjail CLI 顺序探测
3. `get_status` 调用时若当前 backend 不可用,会尝试重新选择 (commit `e5617c7`)
### 3.2 Backend 系统
#### CLISandboxBackend (`box/backend.py`, 411 行)
Docker / Podman 公共基类:
```
start_session(spec):
1. validate_sandbox_security(spec)
2. docker/podman run -d --rm --name <name>
--network none (可选)
--cpus/--memory/--pids-limit
--read-only + --tmpfs /tmp
-v <host>:<mount>:<mode> 主挂载
-v <extra.host>:<extra.mount>:.. 额外挂载 (extra_mounts)
<image> sh -lc 'while true; do sleep 3600; done'
3. 返回 BoxSessionInfo
exec(session, spec):
docker/podman exec -e KEY=VAL <container>
sh -lc 'mkdir -p <workdir> && cd <workdir> && <cmd>'
start_managed_process(session, spec):
docker/podman exec -i <container>
sh -lc 'mkdir -p <cwd> && cd <cwd> && exec <command> <args>'
返回 asyncio.subprocess.Process (stdin/stdout PIPE)
```
容器以 idle 进程启动,实际命令通过 `docker exec` 执行。`--rm` 确保容器退出时自动清理。
**Windows 支持**: backend 内对 Windows 路径处理与 subprocess 调用做了适配commit `120817a`)。
**孤儿清理**: 启动时枚举 `langbot.box=true` 标签的容器instance_id 不匹配的强制删除。
#### NsjailBackend (`box/nsjail_backend.py`, 552 行)
轻量级 Linux 沙箱(无容器引擎依赖):
- 使用 namespace 隔离user/mount/pid/ipc/uts/cgroup/net
- 挂载宿主 `/usr`/`/lib`/`/bin`/`/sbin` 只读 + 选定 `/etc` 条目
- 每 session 创建独立目录workspace/tmp/home
- 资源限制: cgroup v2 优先fallback 到 rlimit
- **CLI 兼容**: 通过 `shutil.which(self._nsjail_bin)` 检测系统安装版 nsjail不存在时再尝试容器内 nsjailcommit `686fcc0``feed530`
- **无自定义镜像**: 使用宿主 OS`image` 字段固定为 `'host'`,兼容性检查跳过 image
#### E2BBackend (`box/e2b_backend.py`, 429 行)
云沙箱后端commit `75b547f` 引入):
- 通过 `e2b` SDK 与 E2B 平台通信
- 配置:`box.e2b.api_key` / `api_url` / `template`
- 支持 `extra_mounts`commit `0fea9b1` 同步上传文件)
- 无本地容器引擎依赖,适合无 Docker 的部署或 SaaS 多租户场景
- 不支持自定义 image 字段,由 template 控制
### 3.3 Server (`box/server.py`, 508 行)
单端口 aiohttp 服务(默认 5410通过路径区分commit `8c71ec5` 合并端口):
1. **Action RPC** (`/rpc/ws`): `BoxServerHandler` 处理所有 action包括 `INIT` 配置注入、skill store 操作等
2. **WS Relay** (`/v1/sessions/{id}/managed-process/ws``/v1/sessions/{id}/managed-process/{pid}/ws`): 双向桥接 WebSocket ↔ 指定 managed process stdin/stdout
stdio 模式同样会在 5410 启动 aiohttp专门承担 managed process attachAction RPC 走 stdin/stdout。
### 3.4 Client (`box/client.py`, 377 行)
`ActionRPCBoxClient` 封装 `Handler.call_action()` 调用:
- 25+ 方法对应 25+ 个 RPC actionexec / session / managed-process / skill / status / shutdown
- 错误还原: `_translate_action_error()` 通过字符串前缀匹配还原 SDK 侧异常类型
- `execute()` timeout = 300s其他默认 15s
- `BoxRuntimeClient` 是 ABC供后续可能的非 RPC 实现复用
包级别 `__init__.py` 显式导出:`BoxRuntimeClient``ActionRPCBoxClient`commit `df9c722`)。
### 3.5 Actions (`box/actions.py`, 34 行)
`LangBotToBoxAction` 枚举共定义 **25 个** action
| 类别 | Actions |
|------|---------|
| 控制 | `INIT``HEALTH``STATUS``GET_BACKEND_INFO``SHUTDOWN` |
| 执行 | `EXEC` |
| Session | `CREATE_SESSION` / `GET_SESSION` / `GET_SESSIONS` / `DELETE_SESSION` |
| Managed Process | `START_MANAGED_PROCESS` / `GET_MANAGED_PROCESS` / `STOP_MANAGED_PROCESS` |
| Skill | `LIST_SKILLS` / `GET_SKILL` / `CREATE_SKILL` / `UPDATE_SKILL` / `DELETE_SKILL` / `SCAN_SKILL_DIRECTORY` / `LIST_SKILL_FILES` / `READ_SKILL_FILE` / `WRITE_SKILL_FILE` / `PREVIEW_SKILL_ZIP` / `INSTALL_SKILL_ZIP` |
### 3.6 Models (`box/models.py`, 331 行)
核心数据模型:
| 模型 | 用途 |
|------|------|
| `BoxNetworkMode` | `OFF` / `ON` |
| `BoxExecutionStatus` | `COMPLETED` / `TIMED_OUT` |
| `BoxHostMountMode` | `NONE` / `READ_ONLY` / `READ_WRITE` |
| `BoxManagedProcessStatus` | `RUNNING` / `EXITED` |
| `BoxMountSpec` | 单条挂载host_path/mount_path/mode**新增** |
| `BoxSpec` | 执行请求;新增 `extra_mounts: list[BoxMountSpec]``persistent``workspace_quota_mb` |
| `BoxProfile` | 4 个内置 Profile + `locked` frozenset |
| `BoxSessionInfo` | Session 状态(含 backend_name/created_at/last_used_at |
| `BoxManagedProcessSpec` | 长驻进程参数process_id/command/args/env/cwd |
| `BoxManagedProcessInfo` | 进程状态status/exit_code/stderr_preview/attached |
| `BoxExecutionResult` | 执行结果status/exit_code/stdout/stderr/duration_ms |
`BoxSpec` 校验器: `workdir` 默认继承 `mount_path``host_path` 支持 POSIX 和 Windows 路径;设置 `host_path``workdir` 必须在 `mount_path` 下。
### 3.7 BoxSkillStore (`box/skill_store.py`, 647 行)
新增模块commit `4ab3502`),把 skill 持久化收归 Box runtime
```
BoxSkillStore
├─ list_skills() / get_skill(name)
├─ create_skill(data) / update_skill(name, data) / delete_skill(name)
├─ scan_skill_directory(path) 扫描目录返回候选 skill 包列表
├─ list_skill_files(name, path) 浏览 skill 内文件树
├─ read_skill_file(name, path) / write_skill_file(name, path, content)
├─ preview_skill_zip(zip_bytes, ...) 不落盘预览 zip 内容
└─ install_skill_zip(zip_bytes, ...) 解压、校验、复制到 skills_root
└─ 支持 source_subdir / target_suffixcommit 1aa043f
```
GitHub 安装路径HTTP 层(`api/http/service/skill.py`)先 `git clone` 拉取,再走 `install_skill_zip` 或 directory 路径。Skill 文件存放于 `box.local.skills_root`(默认 `skills`,相对 `host_root`),容器内对应 `/workspace/.skills/`
### 3.8 Security (`box/security.py`, 52 行)
`validate_sandbox_security()`: 黑名单校验 host_path阻止挂载 `/etc`/`/proc`/`/sys`/`/dev`/`/root`/`/boot` 及 Docker/Podman socket。
**已知缺陷**: 根路径 `/` 未拦截,用户 home 目录未拦截,是 denylist 而非 allowlist 策略。详见 [问题清单 #5](./box-issues.md)。
### 3.9 Errors (`box/errors.py`, 33 行)
| 异常类型 | 含义 |
|----------|------|
| `BoxError` | 基类 |
| `BoxValidationError` | spec/参数校验失败 |
| `BoxBackendUnavailableError` | 无可用 backend |
| `BoxRuntimeUnavailableError` | Runtime 服务不可用 |
| `BoxSessionConflictError` | session 已存在但 spec 不兼容 |
| `BoxSessionNotFoundError` | session 不存在 |
| `BoxManagedProcessConflictError` | session 已有同名 process |
| `BoxManagedProcessNotFoundError` | process 不存在 |
---
## 4. 工具系统集成
### 4.1 ToolManager 编排 (`toolmgr.py`)
```
ToolManager.initialize()
├─ NativeToolLoader (exec / read / write / edit / glob / grep)
├─ PluginToolLoader (插件工具)
├─ MCPLoader (MCP Server 工具)
├─ SkillToolLoader (activate 工具 — Tool Call 激活)
└─ SkillAuthoringToolLoader (Skill CRUD)
工具调用优先级: native → plugin → mcp → skill → skill_authoring
```
### 4.2 Native Tools (`native.py`, 846 行)
| 工具 | 是否在 Box 中执行 | 是否访问宿主文件系统 |
|------|:---:|:---:|
| `exec` | 是 | 否 |
| `read` | **否** | **是** — 直接 `open()` 宿主文件 |
| `write` | **否** | **是** — 直接 `open()` 宿主文件 |
| `edit` | **否** | **是** — 直接 `open()` 宿主文件 |
| `glob` | **否** | **是** — 直接遍历宿主目录 |
| `grep` | **否** | **是** — 直接读宿主文件 |
**沙箱边界不对称**: 这是刻意的设计权衡 — `read`/`write`/`edit`/`glob`/`grep` 绕过沙箱以获得性能(避免容器 I/O 开销与跨进程拷贝),但意味着 LLM 可以直接读写 `allowed_mount_roots` 下任何文件。Skill 路径经 `_resolve_host_path()` 重写,禁止穿越 `package_root`
**exec 的 Skill 分支**: 命令中引用 `/workspace/.skills/<name>` 的 skill 时:
1. 验证 skill 已激活
2. 单次 exec 只能引用一个 skill 包
3. 若 skill 是 Python 项目(有 `requirements.txt``pyproject.toml`),命令会被 venv bootstrap 包裹(在 skill 挂载点内创建 `.venv`
4. 调用 `box_service.execute_tool()` → 走默认 session_id 与已组装好的 `extra_mounts`**不再为每 skill 起独立 session**
### 4.3 MCP-in-Box (`mcp_stdio.py`, 354 行)
`BoxStdioSessionRuntime` 让 MCP stdio 服务器在 Box 容器中运行,**共享 session、多 process**模式commit `529088e`
```
initialize()
1. 复用/创建共享 session (session_id = _build_box_session_id())
- persistent=True长期保持
2. workspace.execute_raw(install_cmd) 安装依赖 (可选)
3. 将每个 MCP server 文件 stage 到 /workspace/.mcp/<process_id>/
4. workspace.start_managed_process(process_id=<server>)
5. websocket_client(ws_url) 通过 WS relay 连接
6. ClientSession.initialize() MCP 协议握手
```
配置 (`MCPServerBoxConfig`): `network='on'` (MCP 服务器通常需要网络)`host_path_mode='ro'` (默认只读)`startup_timeout_sec=120` (留时间给 pip install)。
每条 MCP server 是同一 session 中的一个 managed process独立的 `process_id`、独立 attach URL互不阻塞。
---
## 5. 启动与生命周期
### 5.1 启动顺序 (`build_app.py`)
```
BuildAppStage.run(ap)
├─ ... (persistence, models, sessions) ...
├─ BoxService(ap)
├─ box_service.initialize()
│ └─ connector.initialize()
│ ├─ [stdio] fork box subprocess
│ ├─ [subprocess+WS] Windows 本地
│ └─ [remote WS] connect URL
│ └─ 启动心跳 _heartbeat_task
├─ ap.box_service = box_service
├─ ToolManager(ap)
├─ tool_mgr.initialize()
│ ├─ NativeToolLoader (检查 box_service.available)
│ ├─ PluginToolLoader
│ ├─ MCPLoader (Box 可用时stdio MCP 走沙箱)
│ └─ SkillAuthoringToolLoader
├─ ap.tool_mgr = tool_mgr
├─ ... (platform, pipeline) ...
├─ SkillManager.initialize() (从 Box runtime 加载 skill 列表)
└─ ... (RAG, HTTP, plugins) ...
```
BoxService 在 ToolManager **之前**初始化。ToolManager 创建 loader 时检查 `box_service.available`
### 5.2 初始化失败处理
```python
try:
await self._runtime_connector.initialize()
self._available = True
except Exception as e:
self._available = False
logger.warning(f"Box runtime unavailable: {e}")
```
**静默降级**: Box 初始化失败不会阻止应用启动,仅导致 6 个 native tool、所有 Skill 工具和 MCP-in-Box 工具不暴露给 LLM。与 Plugin 的行为不同Plugin 失败会抛异常)。
### 5.3 销毁流程
```
app.dispose()
└─ box_service.dispose()
├─ connector.dispose()
│ ├─ cancel _heartbeat_task
│ ├─ cancel _handler_task / _ctrl_task
│ └─ terminate subprocess (SIGTERM)
└─ loop.create_task(client.shutdown())
└─ RPC SHUTDOWN → Box Runtime 清理所有容器
```
Box 额外做了 RPC SHUTDOWN 通知 Runtime 主动清理容器,比 Plugin 的直接杀进程更安全。
---
## 6. 配置
### config.yaml (重构后)
```yaml
box:
enabled: true # 整个 Box 子系统的总开关。设为 false 时:
# - 不连接远程 Box runtime不 fork 本地 stdio 子进程
# - sandbox 工具 (exec/read/write/edit/glob/grep) 不暴露给 LLM
# - skill 添加/编辑 / GitHub 安装 / 文件写入全部拒绝
# - stdio 模式的 MCP server 启动时报错http/sse 模式不受影响)
# - skill 列表/读取保持只读可用
# BOX__ENABLED 环境变量可覆盖(统一约定)
backend: 'local' # 'local' (探测) / 'docker' / 'nsjail' / 'e2b'
# BOX_BACKEND 环境变量优先级更高
runtime:
endpoint: '' # 外部 Runtime 的 WS 基地址 'ws://host:5410'
# 留空 = 本地自管 Runtime
local:
profile: 'default'
image: '' # 覆盖 profile 默认 image
host_root: './data/box' # 工作区挂载根Docker 部署需绝对路径
default_workspace: '' # 默认 '<host_root>/default'
skills_root: 'skills' # Box 管理的 skill 包目录(相对 host_root
allowed_mount_roots: # 默认 ['<host_root>']
- './data/box'
- '/tmp'
workspace_quota_mb: null # 配额覆盖null = 走 profile
e2b:
api_key: '' # 也可走 E2B_API_KEY 环境变量
api_url: '' # 自托管 E2B 时填写
template: '' # 默认 template ID
```
> **重大变更**: 较 2026-04-16 文档配置结构完全重组commit `eefdea4`)。原字段 `box.profile` / `box.runtime_url` / `box.shared_host_root` / `box.allowed_host_mount_roots` 全部迁入 `box.local.*` 子表,新增 `box.backend` 与 `box.e2b.*` 配置组。
### docker-compose.yaml
`langbot_box` 服务受 compose profile 控制,默认 `docker compose up` **不会**启动它。需要 sandbox 时:
```bash
docker compose --profile box up # 启动 langbot + langbot_box + plugin runtime
docker compose --profile all up # 同上
docker compose up # 只起 langbot + plugin runtime (box 关闭)
```
若不起 `langbot_box`,需要同步在 `data/config.yaml` 中设 `box.enabled: false`(或 langbot 容器 env 加 `BOX__ENABLED=false`),否则 LangBot 会一直尝试连接不存在的 Box runtime 并报错。
```yaml
# langbot_box 的关键 volume
volumes:
- ${LANGBOT_BOX_ROOT}:${LANGBOT_BOX_ROOT} # 工作区挂载(源/目标同路径)
- /var/run/docker.sock:/var/run/docker.sock # Docker backend 复用宿主 docker
```
### 关闭/连接失败时的行为矩阵
`box.enabled = false` 与"启用但连接失败"在用户可观察行为上**完全一致**——都通过 `BoxService.available = False` 表达,只是 `get_status` 多返回 `enabled` 字段供前端区分文案。
| 消费方 | Box 可用 | Box 不可用(disabled 或 failed) |
|---|---|---|
| native exec/read/write/edit/glob/grep 工具 | 暴露给 LLM | **不暴露** |
| `activate` / `register_skill` 工具 | 暴露给 LLM | **不暴露** |
| stdio MCP server | 在 Box 内启动 | **`_init_stdio_python_server` 抛 RuntimeError** 拒绝;不退化到宿主 stdio |
| http/sse MCP server | 正常 | 正常(不依赖 Box) |
| Skill 列表/读取 (`list_skills`/`get_skill`/`read_skill_file`) | 走 Box runtime | 走 LangBot 本地 `data/skills/` 只读 fallback |
| Skill 创建/编辑/安装/写文件 | 走 Box runtime | **HTTP 400** + 明确错误信息(`_require_box_for_write`) |
| Pipeline AI 配置中 `box-session-id-template` | 正常生效 | **前端 banner** 提示字段无效 |
| Pipeline 扩展页 `enable_all_skills` / 绑定 skill | 可编辑 | **前端禁用** + banner |
| 仪表盘 Box 状态卡片 | 绿点 / "已连接" | 灰点 / "已禁用"(disabled) 或 红点 / "已断开"(failed) |
> 后端拒写的边界条件:如果 `ap.box_service` **完全没装**(老式 dev mode,没经过 BuildAppStage),`_require_box_for_write` 视作 no-op,保留 `data/skills/` 本地路径——以兼容历史测试与最小化设置。生产环境总会装 `ap.box_service`,因此该 fallback 不会被触发。
### Pipeline 配置 (templates/metadata/pipeline/ai.yaml)
`local-agent.config.box-session-id-template` 控制 session 作用域,预设:
- `{launcher_type}_{launcher_id}` — 每个会话 (推荐,默认)
- `{launcher_type}_{launcher_id}_{sender_id}` — 群聊每个用户
- `{launcher_type}_{launcher_id}_{conversation_id}` — 每个对话上下文
- `{query_id}` — 每条消息(完全隔离)
详见 [box-session-scope.md](./box-session-scope.md)。
### REST API
| 端点 | 方法 | 说明 | 前端 |
|------|------|------|:---:|
| `/api/v1/box/status` | GET | 可用性、Profile、后端信息 | ✅ 监控页 |
| `/api/v1/box/sessions` | GET | 活跃 session 列表 | ❌ |
| `/api/v1/box/errors` | GET | 最近 50 条错误 | ❌ |
| `/api/v1/skills` 等 | GET/POST/PUT/DELETE | Skill CRUD、文件浏览、zip/GitHub 安装、preview | ✅ Skill 管理页 |
前端 `web/src/app/home/monitoring/components/overview-cards/SystemStatusCards.tsx` 已接入 `/api/v1/box/status`,展示 backend 名称、profile 与活跃 session 数。Sessions 与 errors API 仍未接入。

157
docs/review/box-issues.md Normal file
View File

@@ -0,0 +1,157 @@
# Box 系统架构问题清单
> 更新日期: 2026-05-19
> 分支: `feat/sandbox` (LangBot + langbot-plugin-sdk)
---
## 已解决(自上一轮 review
下列原 P0/P1 项在最新分支已被修复,仅作记录:
| 原编号 | 问题 | 处理 commit / 说明 |
|--------|------|---------------------|
| #3 | Box 无重连机制 | `_make_connection_callback` 已接入 `runtime_disconnect_callback``BoxService._reconnect_loop()` 实现指数退避重连 (`2dfd9d5d``c6882cf`) |
| #4 | Box 无心跳 | `BoxRuntimeConnector._heartbeat_loop()`,间隔 20s沿用 Plugin 模式) |
| #10 | Windows 兼容 | connector 增加 Windows 分支 (subprocess + WS)backend 适配 Windows Docker (`120817a``fafb7a4`) |
| #12 | nsjail image 字段冲突 | `_assert_session_compatible()` 在不支持自定义镜像的 backend 跳过 image 字段 |
| #22 | 前端无 Box UI | 监控页 `SystemStatusCards.tsx` 已接入 `/api/v1/box/status`Skill 管理页接入了全部 skill APIsessions/errors API 仍未接入) |
---
## P0 — 合并前建议修复
### 1. policy.py 是死代码
- **位置**: `pkg/box/policy.py` (98 行)
- **现状**: `SandboxPolicy``ToolPolicy``ElevatedPolicy` 三个类已定义,但全项目无任何导入或调用
- **影响**: 三层安全策略(沙箱模式 / 工具白名单 / 权限提升)完全未生效。当前实际策略仍是"Box 可用就暴露全部 6 个 native tool不可用就全部隐藏"
- **建议**: 要么删除死代码,要么接入 NativeToolLoader 的工具暴露 / exec 调用链。如果短期不会接入,至少在 `pkg/box/__init__.py` 显式标注其状态
### 2. WebSocket relay 无认证
- **位置**: SDK `box/server.py` — Action RPC 路径 `/rpc/ws` 与 managed-process relay `/v1/sessions/{id}/managed-process/{pid}/ws`
- **现状**: 任何能访问 5410 端口的客户端都可以连接attach 任意 session 的 managed process stdin/stdout或直接发起 EXEC
- **影响**: 容器化 / Docker compose 部署中,若 Box runtime 端口外暴露,网络内的攻击者可直接控制沙箱
- **建议**: 至少加 token 认证INIT 时下发WS 连接 query string 或 header 校验);多 process 后 attach 面更大,更不能裸奔
### 3. security.py 根路径未拦截
- **位置**: SDK `box/security.py` `BLOCKED_HOST_PATHS_POSIX`
- **现状**: 黑名单中没有 `/``host_path="/"` 可通过校验并挂载整个主机文件系统;用户 home 目录、`/var` 等也未拦截
- **建议**: 将 `/` 加入黑名单,或改用白名单策略与 LangBot 侧 `allowed_mount_roots` 二次拦截
### 4. INIT 与 backend 初始化的竞态
- **位置**: SDK `box/runtime.py` `init()` 在握手后才下发实际配置;`backend` 在 INIT 之前可能已经按默认值实例化
- **现状**: commit `5029d9c` 修复了 "init config before backend reuse" 的部分场景,但 backend 重新实例化时若有正在执行的 session可能命中旧 backend
- **建议**: 整理 init/handshake 顺序——要么 INIT 完成前不接受任何业务 action要么允许 backend 配置变更时显式清理现有 session
---
## P1 — 合并后优先跟进
### 5. Session 数量无上限
- **位置**: SDK `box/runtime.py` `_get_or_create_session()`
- **现状**: `_sessions` dict 无容量限制,恶意或异常调用可创建无限 session
- **建议**: 加 `max_sessions` 配置项,达到上限时拒绝新建或按 LRU 清理
### 6. Quota 检查存在 TOCTOU
- **位置**: `pkg/box/service.py` `_enforce_workspace_quota()`
- **现状**: 应用层先读磁盘大小再执行命令,两步之间有竞态窗口
- **建议**: 短期用 Docker `--storage-opt size=` 做内核级限制;长期用 Redis 原子计数器做预留式配额
### 7. 全局锁持有期间执行慢操作
- **位置**: SDK `box/runtime.py` `_get_or_create_session()``self._lock` 下调用 `backend.start_session()` (即 `docker run` / `nsjail` 进程启动 / E2B `Sandbox.create`)
- **影响**: `docker run` 可能耗时数秒含镜像拉取、E2B 冷启动通常 > 1s期间阻塞所有并发请求
- **建议**: 在 `_lock` 下仅做状态检查和 session 注册,容器创建在锁外执行
### 8. Session 清理是机会性的
- **位置**: SDK `box/runtime.py` `_reap_expired_sessions_locked()` — 仅在 `_get_or_create_session()` 时调用
- **影响**: 如果长时间无新 session 请求,过期 session含容器不会被清理
- **建议**: 加一个独立的 `asyncio.create_task` 定时清理(如每 60s 一次)
### 9. server.py 直接访问 runtime 私有字段
- **位置**: SDK `box/server.py` — managed-process WS handler 直接读 `runtime._sessions`
- **影响**: 绕过锁和封装,在并发场景下可能读到不一致状态
- **建议**: 在 BoxRuntime 上增加公共方法(如 `get_session_managed_process(session_id, process_id)`
### 10. workspace quota 检查阻塞事件循环
- **位置**: `pkg/box/service.py` `_get_workspace_size_bytes()` — 使用同步 `os.scandir` 递归遍历
- **影响**: 大工作区可能阻塞 asyncio event loop
- **建议**: 用 `asyncio.to_thread()` 包装,或用 `aiofiles` 异步扫描
### 11. extra_mounts 一旦容器创建即固定
- **位置**: SDK `box/runtime.py` 的兼容性检查;`pkg/box/service.py:build_skill_extra_mounts()`
- **现状**: Skill 挂载在容器创建时一次性写入;同一 session 后续 pipeline 切换 skill 列表时,新挂载不会生效(除非销毁重建)
- **影响**: 用户长时间共享 session 的场景下,新激活的 skill 可能挂不上
- **建议**: 要么在创建时把 pipeline 绑定的所有 skill 都挂上(实际现状)+ 写入文档;要么变更挂载时强制销毁 session 重建(已被 commit `5029d9c` 部分覆盖,需校验)
---
## P2 — 后续迭代
### 12. 重复的 `_is_path_under` 函数
- **位置**: `pkg/box/service.py` 行 30 附近 — 同名函数定义两次
- **建议**: 删除重复定义
### 13. localagent.py 工具循环无迭代上限
- **位置**: `pkg/provider/runners/localagent.py` `while pending_tool_calls` 循环
- **影响**: 恶意或混乱的 LLM 可无限产生 tool call消耗资源
- **建议**: 加 `max_tool_iterations` 配置项(如默认 50 次)
### 14. localagent.py 中的死代码
- **位置**: `pkg/provider/runners/localagent.py:29-35` 附近 — 旧命名 `SANDBOX_EXEC_TOOL_NAME``SANDBOX_EXEC_SYSTEM_GUIDANCE`
- **现状**: 旧命名方案的遗留常量,从未被引用(实际使用 `EXEC_TOOL_NAME` from native.py
- **建议**: 删除
### 15. @loader_class 装饰器未使用
- **位置**: `pkg/provider/tools/loader.py``preregistered_loaders` 列表和 `@loader_class` 装饰器
- **现状**: 各 loader 的 `@loader_class` 多数被注释掉ToolManager 手动实例化所有 loader
- **建议**: 要么启用装饰器自动注册,要么删除未用的机制
### 16. 工具名冲突风险
- **位置**: `pkg/provider/tools/toolmgr.py` `execute_func_call()` — 按优先级 native → plugin → mcp → skill → skill_authoring 分发
- **影响**: 如果 plugin 或 MCP 有名为 `exec`/`read`/`write`/`edit`/`glob`/`grep`/`activate` 的工具,会被前序 loader 静默遮蔽
- **建议**: 加命名空间前缀或冲突检测告警
### 17. client.py 反序列化不一致
- **位置**: SDK `box/client.py``execute()` 与其他方法对返回值的反序列化方式不统一(部分手动构造 model部分用 `model_validate`
- **建议**: 统一使用 `model_validate`
### 18. 错误类型还原基于字符串前缀匹配
- **位置**: SDK `box/client.py` `_translate_action_error()`
- **影响**: 如果 server 端错误消息格式变化client 会回退到通用 `BoxError`,丢失类型信息
- **建议**: 在 ActionResponse 中增加结构化的错误类型字段(如 `error_code` 枚举)
### 19. 前端只用到了 status
- **位置**: `web/src/app/home/monitoring/...` 已接入 `/api/v1/box/status`
- **现状**: `/api/v1/box/sessions``/api/v1/box/errors` 后端可用、前端未消费
- **建议**: 在监控页或独立 Box 详情页展示活跃 session 列表与最近错误,提升运维体感
### 20. skill_store 测试覆盖偏薄
- **位置**: SDK `tests/box/test_skill_store.py` 仅 88 行
- **现状**: 相对 `skill_store.py` 的 647 行实现单测覆盖度不够GitHub 安装路径、`source_subdir` / `target_suffix` 组合、损坏 zip 的错误处理等场景未覆盖
- **建议**: 至少补到核心 path 覆盖preview/install/list/file CRUD 各 2~3 个 case
### 21. 集成测试未进 CI
- **位置**: LangBot `tests/integration_tests/box/test_box_integration.py``test_box_mcp_integration.py`SDK 端的 E2B 真机测试
- **现状**: 容器实际执行、E2B 真实 sandbox、Managed process WS attach 均仅本地能跑
- **建议**: 加一个可选的 Docker-in-Docker CI stage或在合并前手动跑 checklist

View File

@@ -0,0 +1,401 @@
# Box Session Scope Design
> Date: 2026-04-18 (last reviewed 2026-05-19)
> Branch: `feat/sandbox` (LangBot + langbot-plugin-sdk)
> Related: [Box Architecture](./box-architecture.md) | [Box vs Plugin Runtime](./box-vs-plugin-runtime.md)
---
## 0. Implementation Status (2026-05-19)
This document was authored as a design proposal. The current `feat/sandbox` branch
has shipped the design largely as written:
| Item | Status | Notes |
|------|--------|-------|
| `BoxMountSpec` + `BoxSpec.extra_mounts` | ✅ Shipped | SDK `box/models.py` |
| Docker / nsjail / E2B backends apply extra mounts | ✅ Shipped | Last gap closed by SDK commit `0fea9b1` (E2B) |
| `box-session-id-template` in `local-agent` pipeline config | ✅ Shipped | `templates/metadata/pipeline/ai.yaml`, default `{launcher_type}_{launcher_id}` |
| `BoxService.resolve_box_session_id(query)` | ✅ Shipped | `pkg/box/service.py:166` |
| `BoxService.build_skill_extra_mounts(query)` | ✅ Shipped | `pkg/box/service.py:189` |
| Skill exec uses unified container + extra mounts | ✅ Shipped | `pkg/provider/tools/loaders/native.py` skill branch |
| MCP-in-Box uses shared persistent session, multi-process | ✅ Shipped (earlier than originally scoped) | SDK commit `529088e`, LangBot `mcp_stdio.py:_build_box_session_id` |
| `BoxManagedProcessSpec.process_id` + multi-process per session | ✅ Shipped | `BoxRuntime` keeps `managed_processes: dict[pid, _ManagedProcess]` |
| Per-tenant / quota integration with templates | ❌ Not started | See [box-tob-analysis.md](./box-tob-analysis.md) |
The "Phase 2 deferred" note in §10 is **out of date** — MCP unification went in on
the same line. Pipeline-scoped (not user-scoped) MCP container is the realized
behavior: each pipeline's MCP servers share one `mcp-<pipeline>` session, and
user exec sessions use the template-derived id.
The remaining open work is multi-tenant overlays (tenant_id in session_id,
quota counters keyed by tenant), tracked in the toB analysis doc rather than here.
---
## 1. Problems
### 1.1 Default exec: per-message containers
Currently, `BoxService.execute_tool()` sets `session_id = str(query.query_id)` — an
auto-incrementing integer per incoming message. Every user message creates a new sandbox
container. Dependencies installed and in-container state are lost between messages.
### 1.2 Three isolated container pools
Default exec, skills, and MCP servers each manage their own containers with
independent session IDs:
| Path | Session ID | Container |
|--------------|-----------------------------------------------|-------------|
| Default exec | `str(query_id)` (per message) | Ephemeral |
| Skill exec | `skill-{launcher}_{id}-{skill_name}` | Per skill |
| MCP stdio | `mcp-{server_uuid}` | Per server |
This means a single logical user interaction can spawn 3+ containers that cannot
share state, see each other's files, or reuse installed dependencies.
### 1.3 Single bind mount limitation
`BoxSpec` currently supports only **one** `host_path``mount_path` bind mount.
This prevents mounting both a default workspace and skill directories into the
same container.
---
## 2. Concept Model
```
Platform Message
→ Query (query_id: int, auto-increment, per message)
→ Session (launcher_type + launcher_id, per chat window)
→ Conversation (uuid, per dialogue context within a Session)
```
| Concept | Key | Example | Scope |
|---------------|-------------------------------------|----------------------------|------------------------------|
| Query | `query_id` | `42` | Single message |
| Session | `launcher_type` + `launcher_id` | `group_123456` | Chat window (group or PM) |
| Conversation | `conversation_id` (UUID) | `a1b2c3d4-...` | Dialogue context within a Session |
| Sender | `sender_id` | `789` | Individual user |
Note: in a **group chat**, all users share the same Session (keyed by `group_id`). The
individual sender is tracked as `sender_id` but does not affect Session/Conversation routing.
---
## 3. Target Scenarios
| # | Scenario | Box Granularity | Desired `session_id` |
|----|--------------------------------|------------------------------------------|---------------------------------------------------------|
| 1 | Personal assistant | 1 Box per user, long-lived | `{launcher_type}_{launcher_id}` |
| 2 | Customer service | 1 Box per customer, cross-pipeline | `{launcher_type}_{launcher_id}` |
| 3 | Internal employee tool | 1 Box per employee | `{launcher_type}_{launcher_id}` |
| 4 | Group chat shared assistant | 1 Box per group | `{launcher_type}_{launcher_id}` |
| 5 | Group chat isolated per user | 1 Box per user within a group | `{launcher_type}_{launcher_id}_{sender_id}` |
| 6 | Teaching (cross-channel) | 1 Box per student across groups/PMs | `{sender_id}` |
| 7 | One-off execution | 1 Box per message (current behavior) | `{query_id}` |
| 8 | Multi-project development | 1 Box per conversation context | `{launcher_type}_{launcher_id}_{conversation_id}` |
No single fixed granularity covers all scenarios. A template-based approach is needed.
---
## 4. Design Overview
Two key changes:
1. **Unified container**: exec, skills, and MCP all share the same container per
session scope. No more separate container pools.
2. **Configurable session scope**: `session_id` is generated from a template with
pipeline variables, configurable per pipeline.
### 4.1 Unified Container with Multiple Mounts
A single container per session scope is created on first use. It has:
- **Primary mount**: default workspace at `/workspace` (from `default_host_workspace`)
- **Skill mounts**: each pipeline-bound skill's `package_root` mounted at
`/workspace/.skills/{skill_name}/`
- **MCP servers**: run as managed processes inside the same container
```
Container (session_id = "group_123456")
/workspace/ ← default workspace (bind mount, rw)
/workspace/.skills/web-search/ ← skill package (bind mount, rw)
/workspace/.skills/data-analysis/ ← skill package (bind mount, rw)
[managed process: mcp-server-a] ← MCP server running inside
[managed process: mcp-server-b] ← MCP server running inside
```
This requires extending `BoxSpec` to support multiple mounts (see §5).
### 4.2 Session ID Template
A new field `box-session-id-template` in the `local-agent` pipeline runner config
controls the session scope:
```yaml
# templates/metadata/pipeline/ai.yaml (under local-agent.config)
- name: box-session-id-template
label:
en_US: Sandbox Scope
zh_Hans: 沙箱作用域
description:
en_US: >-
Determines how sandbox environments are shared. Use variables to
control isolation granularity.
zh_Hans: >-
决定沙箱环境的共享方式。使用变量控制隔离粒度。
type: select
required: false
default: "{launcher_type}_{launcher_id}"
options:
- value: "{launcher_type}_{launcher_id}"
label:
en_US: Per chat (Recommended)
zh_Hans: 每个会话(推荐)
- value: "{launcher_type}_{launcher_id}_{sender_id}"
label:
en_US: Per user in chat
zh_Hans: 会话中每个用户
- value: "{launcher_type}_{launcher_id}_{conversation_id}"
label:
en_US: Per conversation context
zh_Hans: 每个对话上下文
- value: "{query_id}"
label:
en_US: Per message (isolated)
zh_Hans: 每条消息(完全隔离)
```
Available template variables (populated by PreProcessor in `query.variables`):
| Variable | Source | Example |
|---------------------|---------------------------------|----------------------|
| `{launcher_type}` | `query.session.launcher_type` | `person` / `group` |
| `{launcher_id}` | `query.session.launcher_id` | `123456` |
| `{sender_id}` | `query.sender_id` | `789` |
| `{conversation_id}` | `conversation.uuid` | `a1b2c3d4-...` |
| `{query_id}` | `query.query_id` | `42` |
Default `{launcher_type}_{launcher_id}` covers scenarios 14 out of the box.
---
## 5. SDK Changes: Multi-Mount BoxSpec
### 5.1 Model Extension
```python
# box/models.py
class BoxMountSpec(pydantic.BaseModel):
"""A single bind mount specification."""
host_path: str
mount_path: str
mode: BoxHostMountMode = BoxHostMountMode.READ_WRITE
class BoxSpec(pydantic.BaseModel):
# ... existing fields ...
host_path: str | None = None # Primary mount (backward compat)
host_path_mode: BoxHostMountMode = BoxHostMountMode.READ_WRITE
mount_path: str = DEFAULT_BOX_MOUNT_PATH
extra_mounts: list[BoxMountSpec] = [] # NEW: additional mounts
```
`extra_mounts` is additive — the existing `host_path` / `mount_path` pair remains
the primary mount for backward compatibility.
### 5.2 Backend: Apply Extra Mounts
```python
# box/backend.py — CLISandboxBackend.start_session()
# Primary mount (unchanged)
if spec.host_path is not None and spec.host_path_mode != BoxHostMountMode.NONE:
args.extend(['-v', f'{spec.host_path}:{spec.mount_path}:{spec.host_path_mode.value}'])
# Extra mounts (NEW)
for mount in spec.extra_mounts:
if mount.mode != BoxHostMountMode.NONE:
args.extend(['-v', f'{mount.host_path}:{mount.mount_path}:{mount.mode.value}'])
```
Same pattern for nsjail backend.
---
## 6. LangBot Changes
### 6.1 Session ID Resolution
In `BoxService.execute_tool()`:
```python
# Before:
spec_payload.setdefault('session_id', str(query.query_id))
# After:
template = (query.pipeline_config or {}).get('ai', {}) \
.get('local-agent', {}).get('box-session-id-template',
'{launcher_type}_{launcher_id}')
variables = query.variables or {}
session_id = template.format_map(collections.defaultdict(
lambda: 'unknown', variables
))
spec_payload.setdefault('session_id', session_id)
```
### 6.2 Skill Exec: Use Same Container
Currently `native.py:_invoke_exec` creates a separate `BoxWorkspaceSession` per
skill with `host_path=package_root`. Instead:
1. Use the **same session_id** as default exec (from the template).
2. Pass the skill's `package_root` as an **extra mount** at
`/workspace/.skills/{skill_name}/` instead of replacing `/workspace`.
3. The container already has the default workspace at `/workspace`.
```python
# native.py — _invoke_exec, skill branch (REVISED)
# Same session_id as default exec
session_id = resolve_box_session_id(query)
spec_payload = {
'cmd': rewritten_command,
'workdir': rewritten_workdir,
'session_id': session_id,
'extra_mounts': [{
'host_path': package_root,
'mount_path': f'/workspace/.skills/{selected_skill_name}',
'mode': 'rw',
}],
}
result = await self.ap.box_service.execute_spec_payload(spec_payload, query)
```
The virtual path `/workspace/.skills/{name}` no longer needs rewriting at the
command level — it maps directly to the bind mount path inside the container.
### 6.3 MCP: Use Same Container
MCP servers should run inside the same container as exec and skills. Changes:
1. `BoxStdioSessionRuntime` uses the pipeline's session_id template instead of
`mcp-{server_uuid}`.
2. MCP server's working directory is a subdirectory (e.g. `/workspace/.mcp/{name}/`).
3. MCP server's dependencies are mounted or installed into that subdirectory.
4. The MCP server runs as a managed process inside the shared container.
Since MCP servers start at LangBot boot (not per-query), the session must be
created eagerly. The container will be kept alive by the managed process
exemption in TTL reaping (`runtime.py:259`).
**Note**: MCP sessions are pipeline-scoped (not per-launcher), so their session_id
should be a **fixed identifier per pipeline** rather than the user-facing template.
This means one shared MCP container per pipeline, with user exec sessions separate.
Alternatively, in a future iteration, MCP managed processes could be launched
lazily into the user's container on first MCP tool call. This is more complex
but maximizes sharing. For V1, keeping MCP containers at pipeline scope is
simpler and more predictable.
---
## 7. Mount Layout Summary
### Default exec (no skills activated)
```
Container (session_id from template)
/workspace/ ← default_host_workspace (rw)
```
### Exec with activated skills
```
Container (same session_id)
/workspace/ ← default_host_workspace (rw)
/workspace/.skills/web-search/ ← skill package_root (rw)
/workspace/.skills/data-analysis/ ← skill package_root (rw)
```
Extra mounts are **additive** — they are added when the container is first
created (or on the first exec that references a skill). Since Docker bind
mounts are specified at container creation time, skills must be known at
creation time.
**Resolution**: When creating a container, inject `extra_mounts` for **all
pipeline-bound skills** (from `extensions_preferences`), not just the
currently activated one. This way any skill can be activated later without
recreating the container.
### MCP servers (V1: pipeline-scoped)
```
Container (session_id = "mcp-pipeline-{pipeline_uuid}")
/workspace/ ← MCP shared workspace
/workspace/.mcp/server-a/ ← MCP server A files
/workspace/.mcp/server-b/ ← MCP server B files
[managed process: server-a]
[managed process: server-b]
```
---
## 8. Data Migration
Existing pipelines do not have `box-session-id-template`. The backend uses
`.get(..., default)` so missing keys fall back to `{launcher_type}_{launcher_id}`.
This changes behavior from per-message to per-launcher for existing pipelines.
Recommendation: **accept the behavior change** — per-launcher is the more
intuitive default, and the old per-message behavior was rarely desired.
---
## 9. Cloud Quota Implications
| Scope | Typical concurrent containers |
|-----------------------------------------------|-------------------------------|
| `{query_id}` (per message) | Many, short-lived |
| `{launcher_type}_{launcher_id}` (per chat) | = active chat count |
| `{sender_id}` (per user) | = active user count |
| `{conversation_id}` (per conversation) | Between per-chat and per-msg |
With the unified container model, each scope value maps to exactly **one**
container (instead of potentially 3+ per-message). This significantly reduces
resource usage.
Quota enforcement point: `BoxRuntime._get_or_create_session()` in the SDK.
---
## 10. Implementation Phases
### Phase 1: Session scope + skill unification (this PR)
1. **SDK**: Extend `BoxSpec` with `extra_mounts: list[BoxMountSpec]`.
2. **SDK**: Update Docker/nsjail backends to apply extra mounts.
3. **LangBot**: Add `box-session-id-template` to `local-agent` YAML metadata
and default pipeline config JSON.
4. **LangBot**: Update `BoxService.execute_tool()` to use template interpolation.
5. **LangBot**: Update `native.py:_invoke_exec` skill branch to use same
session_id + extra mounts instead of separate `BoxWorkspaceSession`.
6. **LangBot**: On container creation, inject extra mounts for all
pipeline-bound skills.
7. **Frontend**: No code change — `DynamicFormComponent` renders `select` fields.
8. **Tests**: Unit tests for template interpolation and multi-mount specs.
### Phase 2: MCP unification (future)
1. Refactor `BoxStdioSessionRuntime` to use pipeline-scoped shared container.
2. MCP servers become managed processes in the shared container.
3. Support multiple concurrent managed processes per container.
MCP unification is deferred because it requires changes to the managed process
model (currently 1 managed process per session) and has startup ordering
concerns (MCP servers start at boot, before any user query determines
a session_id).

View File

@@ -0,0 +1,121 @@
# Box 系统测试覆盖分析
> 更新日期: 2026-05-19
> 分支: `feat/sandbox` (LangBot + langbot-plugin-sdk)
---
## 1. 测试文件清单
### LangBot 仓库
| 文件 | 行数 | CI 运行 | 覆盖范围 |
|------|------|---------|---------|
| `tests/unit_tests/box/test_box_connector.py` | 106 | 是 | Connector 传输决策、WS relay URL、dispose、心跳/重连 |
| `tests/unit_tests/box/test_box_service.py` | 1224 | 是 | Service 核心逻辑(最全面) |
| `tests/unit_tests/box/test_workspace.py` | 147 | 是 | WorkspaceSession 路径重写、payload 构建 |
| `tests/unit_tests/provider/test_mcp_box_integration.py` | 707 | 是 | MCP Box 配置、路径重写、payload、shared-session/multi-process、runtime info |
| `tests/unit_tests/provider/test_localagent_sandbox_exec.py` | 444 | 是 | LocalAgent exec 流程、流式、Skill 激活 (Tool Call) |
| `tests/unit_tests/provider/test_tool_manager_native.py` | 249 | 是 | ToolManager 路由、native tool CRUD、路径穿越、6 工具暴露 |
| `tests/unit_tests/provider/test_skill_tools.py` | 582 | 是 | Skill 管理、Tool Call 激活、路径、authoring CRUD |
| `tests/unit_tests/test_skill_service.py` | 396 | 是 | HTTP serviceskill CRUD、zip/GitHub install、文件浏览 |
| `tests/unit_tests/test_paths.py` | 23 | 是 | paths 工具 |
| `tests/unit_tests/test_preproc.py` | 134 | 是 | PreProcessor 注入 session 变量、bound skill 解析 |
| `tests/unit_tests/pipeline/test_chat_handler_logging.py` | 78 | 是 | Chat handler 日志相关回归 |
| `tests/integration_tests/box/test_box_integration.py` | 329 | **否** | 真实容器执行、超时、网络隔离 |
| `tests/integration_tests/box/test_box_mcp_integration.py` | 368 | **否** | Managed process、WS attach、shared-session 清理 |
### SDK 仓库
| 文件 | 行数 | CI 运行 | 覆盖范围 |
|------|------|---------|---------|
| `tests/box/test_backend_selection.py` | 255 | 是 | 显式 backend / local 模式探测顺序 / 配置变更触发 reselect |
| `tests/box/test_nsjail_backend.py` | 452 | 是 | nsjail 可用性、安装版 CLI vs 容器内 CLI、session、arg 构建、资源限制 |
| `tests/box/test_e2b_backend.py` | 482 | 是 | E2B SDK mock、session 生命周期、extra_mounts 同步 |
| `tests/box/test_skill_store.py` | 88 | 是 | zip preview/install、基础 file CRUD |
**总计**: 17 个测试文件, ~6,500 行测试代码; 其中 2 个集成测试(约 700 行)在 CI 中不运行。
> 较 2026-04-16 版增加:`test_skill_service.py`、`test_paths.py`、`test_preproc.py`、`test_chat_handler_logging.py` (LangBot)`test_backend_selection.py`、`test_e2b_backend.py`、`test_skill_store.py` (SDK)。`test_nsjail_backend.py` 增加 CLI 兼容性 case (commit `feed530`)。
---
## 2. 覆盖良好的区域
| 区域 | 质量 | 说明 |
|------|------|------|
| BoxRuntime session 管理 | 优秀 | session 复用、冲突检测、TTL 配置、消失 session 重建 |
| BoxService Profile 系统 | 优秀 | 4 个内置 Profile、locked/unlocked 字段、timeout clamp |
| BoxService host mount 安全 | 优秀 | allowed_mount_roots、disallowed_roots、shared host root |
| BoxService workspace quota | 优秀 | 前置/后置配额检查、超额清理 |
| BoxService 输出截断 | 优秀 | 短/精确边界/长输出、独立 stderr |
| BoxService 可观测性 | 优秀 | 状态报告、error ring buffer、buffer 上限 |
| BoxService session 模板 | 良好 | `resolve_box_session_id` + `build_skill_extra_mounts` 在 service / native / mcp 三处都有覆盖 |
| RPC client/server 协议 | 优秀 | execute/get_sessions/delete/create/conflict error |
| BoxRuntimeConnector | 良好 | local/remote 模式、Docker 平台、relay URL、心跳与重连回调 |
| BoxWorkspaceSession | 良好 | payload 构建、managed process 路径重写、stage host file |
| BoxHostMountMode.NONE | 良好 | 枚举校验、workdir 约束 |
| NsjailBackend | 良好 | 可用性、安装版 vs 容器内、session 生命周期、arg 构建、资源限制 |
| E2BBackend | 良好 | mock SDK、session/extra_mounts 同步 |
| Backend selection | 良好 | 显式 backend 优先级、local 探测顺序、配置变更触发 reselect |
| MCP Box 集成 | 良好 | config model、路径重写、payload、shared-session 多 process |
| Native tool loader | 良好 | 6 工具exec/read/write/edit/glob/grep、路径穿越拦截 |
| LocalAgent exec 流程 | 良好 | 完整 tool call 循环、流式、system prompt 注入、Tool Call 激活 |
| Skill 系统 | 良好 | 加载、Tool Call 激活、marker、路径解析、authoring CRUD、HTTP service |
---
## 3. 覆盖缺失的区域
### 3.1 零测试 / 严重不足
| 区域 | 源文件 | 影响 |
|------|--------|------|
| **`security.py`** | SDK `box/security.py` (52 行) | `validate_sandbox_security()` 无任何测试。阻止 `/etc`/`/proc`/Docker socket 等危险挂载的安全函数从未被验证 |
| **`policy.py`** | `pkg/box/policy.py` (98 行) | 三层安全策略无测试(也是死代码) |
| **`skill_store.py` 边缘场景** | SDK `box/skill_store.py` (647 行) vs 测试 88 行 | GitHub 安装路径、`source_subdir` / `target_suffix` 组合、损坏 zip、文件冲突等场景未覆盖 |
### 3.2 未测试的关键路径
| 区域 | 说明 |
|------|------|
| **Session TTL 过期** | 测试配置了 `session_ttl_sec` 但从未推进时间验证过期清理 |
| **并发 session 访问** | 无并发 exec / 并发创建 / race condition 测试 |
| **Container backend (Docker)** | 仅通过集成测试覆盖CI 不运行),单元测试全用 FakeBackend |
| **E2B 真实 sandbox** | 单测全是 mock未对接真实 E2B API |
| **BoxRuntime shutdown()** | 在 test cleanup 中调用但未验证行为 |
| **BoxServerHandler 错误路径** | 畸形请求、未知 action 类型 |
| **WS relay** | 仅在集成测试中覆盖CI 不运行) |
| **NsjailBackend managed process** | 完全未测试 |
| **MCP stdio 完整生命周期** | 依赖安装 → 进程启动 → 健康检查 → 多 process 并发 → 重试 |
| **BoxService start/stop_managed_process** | 单 process 流转有单测,多 process 互不阻塞主要靠集成测试 |
| **重连指数退避** | connector 单测覆盖回调接线,未实际跑完整重连周期 |
### 3.3 边缘情况缺失
| 区域 | 说明 |
|------|------|
| BoxSpec 校验 | 无效 session_id 格式、超长命令、env 特殊字符 |
| BoxSpec.extra_mounts | 重复 mount_path、与 host_path 冲突、绝对 vs 相对路径 |
| BoxExecutionResult | 仅 COMPLETED 和 TIMED_OUT无 ERROR 状态测试 |
| 多后端 fallback | local 模式探测顺序仅靠 mock无真实 Docker 不可用 → nsjail 真机 fallback 测试 |
| Profile YAML 加载 | 测试用硬编码字符串,未从真实 config.yaml 加载 |
| INIT 配置变更触发 backend 重建 | 单测仅在初始化场景验证 |
---
## 4. 集成测试 vs CI 的差距
CI 仅运行 `tests/unit_tests/`,以下场景**从未在自动化中验证**:
- 真实容器的创建/执行/销毁
- 容器网络隔离(`--network none`
- 容器资源限制生效cpus/memory/pids_limit
- Managed process 的 WS 双向 I/O
- 多 process 同 session 并发 I/O
- 孤儿容器清理
- Session 删除清理容器
- 进程退出检测
- E2B 真实 sandbox 行为
**建议**: 在 CI 中加一个可选的 Docker-in-Docker 集成测试 stage至少覆盖核心执行路径exec / MCP attach / session 销毁)。

View File

@@ -0,0 +1,166 @@
# Box 系统 toB 商业化分析
> 更新日期: 2026-05-19
> 分支: `feat/sandbox` (LangBot + langbot-plugin-sdk)
---
## 1. 现有优势
| 能力 | toB 价值 | 代码位置 |
|------|---------|---------|
| **沙箱隔离执行** | 企业安全运行不受信代码的基础能力 | SDK `box/backend.py` |
| **多后端支持** | 适配不同企业容器基础设施 (Podman/Docker/nsjail/E2B) | SDK `box/runtime.py` `_select_backend()` |
| **E2B 云沙箱** | SaaS / 无 Docker 部署的兜底执行环境 | SDK `box/e2b_backend.py` |
| **连接自愈** | 心跳 + 自动重连,单点 Box runtime 故障可恢复 | `pkg/box/connector.py` `_heartbeat_loop`, `pkg/box/service.py` `_reconnect_loop` |
| **Profile + locked 字段** | 运维锁定安全边界LLM/用户无法绕过 | `pkg/box/service.py`, SDK `box/models.py` |
| **资源限制** | CPU/内存/PID 数限制防止资源滥用 | SDK `backend.py` `--cpus/--memory/--pids-limit` |
| **Workspace quota** | 磁盘用量控制 | `pkg/box/service.py` `_enforce_workspace_quota` |
| **静默降级** | Box 不可用不影响其他功能,降低部署门槛 | `pkg/box/service.py:78` `_available=False` |
| **孤儿容器清理** | 防止泄漏的容器持续占用资源 | SDK `backend.py` `cleanup_orphaned_containers` |
| **网络隔离** | `--network none` 防止数据外泄 | SDK `backend.py` start_session |
| **只读根文件系统** | `--read-only` 防止容器被持久篡改 | SDK `backend.py` start_session |
| **Host path 白名单** | `allowed_host_mount_roots` 限制可挂载目录 | `pkg/box/service.py` `_validate_host_mount` |
---
## 2. toB 差距分析
### 2.1 安全与合规
| 维度 | 现状 | toB 要求 | 优先级 |
|------|------|---------|--------|
| **WS relay 认证** | 无认证,任何人可 attach | 至少 token 认证 | **P0** |
| **安全策略** | policy.py 是死代码,实际无细粒度控制 | 工具级 allow/deny、沙箱模式控制 | **P0** |
| **审计日志** | 仅内存中 50 条 `_recent_errors` | 持久化审计:谁何时执行了什么、结果如何 | **P0** |
| **Host path 校验** | 黑名单策略,`/` 未拦截 | 白名单策略,默认拒绝 | **P1** |
| **数据驻留** | 无控制 | GDPR / 等保要求的数据隔离 | **P2** |
### 2.2 多租户
| 维度 | 现状 | toB 要求 | 优先级 |
|------|------|---------|--------|
| **租户隔离** | 无租户概念 | BoxSpec/Profile 绑定 tenant_id | **P0** |
| **RBAC** | 仅 token 认证 | admin/operator/viewer 角色权限 | **P0** |
| **资源配额** | 单一 workspace quota | 每租户 CPU 时间/内存/并发/执行次数配额 | **P1** |
| **Session 隔离** | 所有 session 共享 dict | 按租户分区,互不可见 | **P1** |
### 2.3 可靠性
| 维度 | 现状 | toB 要求 | 优先级 |
|------|------|---------|--------|
| **连接恢复** | 已实现20s 心跳 + `_reconnect_loop` 指数退避 | 已满足基本要求 | 已有 |
| **Session 清理** | 机会性(仅新建时触发) | 定时清理 + 独立 reaper | **P1** |
| **水平扩展** | 单 Box Runtime 实例 | 多实例负载均衡(按 tenant 路由) | **P1** |
| **优雅降级** | 已有_available=False | 已满足基本要求 | 已有 |
| **Backend 自愈** | 已实现:`get_status` 时若 backend 不可用会重新选择 | 已满足基本要求 | 已有 |
### 2.4 可观测性
| 维度 | 现状 | toB 要求 | 优先级 |
|------|------|---------|--------|
| **监控指标** | 无 Prometheus metrics | session 数/执行延迟/资源用量/错误率 | **P1** |
| **结构化日志** | Python logging, 无结构化 | JSON 格式日志,含 trace_id/tenant_id | **P1** |
| **前端面板** | 监控页接入 `/api/v1/box/status`backend 名 + 活跃 session 数);`sessions` / `errors` 仍未接入 | 完整状态面板 + 历史错误/审计列表 | **P2** |
---
## 3. SaaS 部署架构建议
### 3.1 方案 A: 共享 Box Runtime Pool (快速上线)
```
LangBot Instance ──> Box Runtime (共享)
├─ tenant_id 标签隔离
├─ Redis 配额计数器
└─ Container labels: langbot.tenant_id=xxx
```
- **优点**: 改动最小,加 tenant_id 到 BoxSpec/labels 即可
- **缺点**: 容器引擎共享,安全隔离弱
### 3.2 方案 B: 每租户 K8s Namespace + gVisor (推荐中期)
```
LangBot ──> K8s API
├─ namespace: tenant-xxx
│ ├─ RuntimeClass: gVisor (runsc)
│ ├─ ResourceQuota
│ └─ NetworkPolicy
└─ namespace: tenant-yyy
└─ ...
```
- **优点**: 强隔离namespace + gVisor原生 K8s 配额
- **缺点**: 需要重写 backend 为 K8s Job部署复杂度高
### 3.3 方案 C: K8s Job 直接编排 (长期)
```
LangBot ──> K8s Job per execution
├─ 每次执行创建 Job
├─ Pod Security Standards
├─ 自动调度和资源分配
└─ Job TTL Controller 自动清理
```
- **优点**: 最强隔离,天然水平扩展
- **缺点**: 冷启动延迟,架构重写
**推荐演进路径**: A → B → C
---
## 4. 配额体系建议
### 三层配额
| 层 | 实现 | 作用 |
|----|------|------|
| **内核层** | Docker `--cpus`/`--memory`/`--storage-opt` | 硬性资源上限,不可绕过 |
| **应用层** | Redis 原子计数器 | 并发 session 数/执行次数/CPU 时间预算 |
| **计费层** | 月度聚合 | 按租户计费session-hours/execution-count |
### Profile 与套餐映射
| 套餐 | Profile | locked 字段 | 配额 |
|------|---------|------------|------|
| Free | `offline_readonly` | network, host_path_mode, rootfs | 10 exec/天, 0.5 CPU, 256MB |
| Pro | `default` | (无) | 100 exec/天, 1 CPU, 512MB |
| Enterprise | `network_extended` | (按需) | 无限, 2 CPU, 1GB, 自定义镜像 |
### TOCTOU 配额修复
当前 `_enforce_workspace_quota` 的 TOCTOU 问题可通过两种方式解决:
1. **预留式配额** (应用层): Redis `INCRBY` 预扣额度 → 执行 → 成功则扣减,失败则回滚
2. **内核级限制** (Docker): `--storage-opt size=500m` 直接限制容器可写层大小
---
## 5. 优先实施路线
### Phase 1 (2-4 周): 安全基线
- [ ] WS relay 加 token 认证
- [ ] 接入或删除 policy.py
- [x] ~~Box 加重连和心跳~~(已完成,见 [box-issues.md 已解决](./box-issues.md)
- [ ] 审计日志持久化(至少写文件/数据库)
- [ ] `security.py``/` 拦截,考虑白名单
- [ ] INIT 与 backend 初始化顺序整理(避免 backend 在配置到达前实例化)
### Phase 2 (4-8 周): 多租户基础
- [ ] BoxSpec 加 `tenant_id` 字段
- [ ] 容器 labels 加 tenant 标识
- [ ] Redis 配额计数器(并发/执行次数/时间)
- [ ] RBAC 基础框架
- [ ] 定时 session reaper
### Phase 3 (8-16 周): 生产就绪
- [ ] Prometheus metrics exporter
- [ ] 前端 Box 状态面板
- [ ] K8s backend 支持 (方案 B)
- [ ] 结构化日志 (JSON, trace_id)
- [ ] 水平扩展支持

View File

@@ -0,0 +1,221 @@
# Box Runtime vs Plugin Runtime: 连接架构对比
> 更新日期: 2026-05-19
> 分支: `feat/sandbox` (LangBot + langbot-plugin-sdk)
---
## 1. 总体差异
| 维度 | Plugin Runtime | Box Runtime |
|------|---------------|-------------|
| **继承关系** | `PluginRuntimeConnector(ManagedRuntimeConnector)` | `BoxRuntimeConnector`(独立类) |
| **传输分支** | 3 条 (Docker/WS, Win32/subprocess+WS, Unix/stdio) | 3 条 (本地 stdio, Win32/subprocess+WS, 远程 WS) |
| **心跳** | 20s ping loop | 20s ping loop`_heartbeat_loop` |
| **重连** | WS 模式: sleep 3s → re-initialize | 由 BoxService `_reconnect_loop` 处理,指数退避 |
| **Handler 类型** | `RuntimeConnectionHandler` (1132 行, 25+ action) | 基础 `Handler` + `BoxServerHandler`SDK 端 25 action |
| **Client 抽象** | Handler 即 API | 独立 `ActionRPCBoxClient` 封装 Handler |
| **启用/禁用** | `is_enable_plugin` 开关 | 无开关(可用/不可用由初始化结果决定) |
| **初始化失败** | 异常上抛 | 静默降级 `_available=False` |
| **Shutdown** | 直接杀进程 | RPC SHUTDOWN → 清理容器 → 再杀进程 |
---
## 2. 传输决策
### Plugin: 3-路决策
```python
# pkg/plugin/connector.py:106-165
if get_platform() == 'docker' or use_websocket_to_connect_plugin_runtime():
# Docker/WS → ws://langbot_plugin_runtime:5400/control/ws
elif get_platform() == 'win32':
# Windows → 起子进程(无 pipe) + ws://localhost:5400/control/ws
else:
# Unix/Mac → StdioClientController(python -m langbot_plugin.cli rt -s)
```
### Box: 3-路决策
```python
# pkg/box/connector.py
if self._uses_websocket():
if platform.get_platform() == 'win32' and not self.configured_runtime_url:
await self._start_subprocess_then_ws() # subprocess + ws://localhost:5410/rpc/ws
else:
await self._connect_remote_ws() # ws://{host}:5410/rpc/ws
else:
await self._start_local_stdio() # StdioClientController
```
> 历史2026-04-16 版本本文档曾把 Box 描述为 2 路决策(缺 Windows 分支)。现已对齐 Plugin 的 3 路设计。
### 决策矩阵
| 环境 | Plugin | Box |
|------|--------|-----|
| Docker | WS → `:5400` | WS → `:5410/rpc/ws` |
| `--standalone-box` | N/A | WS → `localhost:5410/rpc/ws` |
| Windows 非 Docker | subprocess + WS (`:5400`) | subprocess + WS (`localhost:5410/rpc/ws`) |
| Unix/Mac 非 Docker | stdio | stdio |
| 手动配置 URL | 通过配置项 | WS → 用户配置的 URL |
---
## 3. 连接建立
### 同步模式差异
**Plugin**: `new_connection_callback` 内直接 ping + await handler_task`initialize()` 通过 `create_task()` 异步启动,不阻塞等待连接。
**Box**: 使用 `asyncio.Event` + `wait_for(timeout=30s)` 模式,`initialize()` 同步等待连接成功或超时。
### Box stdio 路径
```
connector._start_local_stdio()
├─ connected = asyncio.Event()
├─ ctrl = StdioClientController(python, ['-m', 'langbot_plugin.cli.__init__', 'box', '-s', '--ws-control-port', N])
├─ _ctrl_task = create_task(ctrl.run(callback))
│ callback:
│ handler = Handler(connection) ← 基础 Handler, 无 disconnect_callback
│ client.set_handler(handler)
│ _handler_task = create_task(handler.run())
│ call_action(PING, {}) ← 握手, timeout=15s
│ connected.set() ← 通知外层
│ await _handler_task ← 阻塞直到断开
└─ await wait_for(connected.wait(), 30s) ← 同步等待
```
### Plugin stdio 路径
```
connector.initialize()
├─ ctrl = StdioClientController(python, ['-m', 'langbot_plugin.cli', 'rt', '-s'])
├─ task = ctrl.run(callback)
│ callback:
│ disconnect_callback:
│ [WS] → runtime_disconnect_callback → 重连
│ [stdio] → 仅日志, 不重连
│ handler = RuntimeConnectionHandler(conn, disconnect_cb, ap)
│ create_task(handler.run())
│ handler.ping() ← 握手, timeout=10s
│ await handler_task ← 阻塞直到断开
├─ create_task(heartbeat_loop()) ← 20s ping loop
└─ create_task(task) ← 不等待连接
```
---
## 4. 心跳与重连
### 心跳
| 维度 | Plugin | Box |
|------|--------|-----|
| 有心跳? | 是 | 是(`connector.py` `_heartbeat_loop` |
| 间隔 | 20s | 20s |
| 失败处理 | 仅 DEBUG 日志,不触发重连 | 仅 DEBUG 日志,依赖 connection close 触发重连 |
| 生命周期 | 整个应用生命周期 | 连接建立后启动;`dispose()` 时 cancel |
### 重连
| 维度 | Plugin | Box |
|------|--------|-----|
| Docker/WS 断开 | `runtime_disconnect_callback` → sleep 3s → re-initialize | `runtime_disconnect_callback``BoxService._reconnect_loop()`(指数退避) |
| WS 连接失败 | 同上 | 同上;初次失败时 `_available=False`,重连成功后恢复 |
| stdio 断开 | 仅日志,不重连 | 接同样回调stdio 重连需重新 fork 子进程 |
| 重连退避 | 固定 3s无 backoff | 指数退避 |
> 历史2026-04-16 版本本文档曾把心跳与重连标记为 Box 缺失。这两项已在 commit `2dfd9d5d` / `c6882cf` / `5029d9c` 等修复(详见 [box-issues.md 已解决](./box-issues.md))。
---
## 5. 共享 IO 层
两者复用同一套 SDK IO 基础设施:
```
Handler ← ABC (runtime/io/handler.py)
├── RuntimeConnectionHandler (Plugin 用, LangBot 侧)
├── ControlConnectionHandler (Plugin 用, SDK 侧)
├── BoxServerHandler (Box 用, SDK 侧)
└── 匿名 Handler 实例 (Box 用, LangBot 侧)
Connection ← ABC
├── StdioConnection (stdio: 16KB chunks, 应用层分帧协议)
└── WebSocketConnection (WS: 64KB chunks, 原生 WS 分帧)
Controller ← ABC
├── StdioClientController (fork 子进程, pipe stdin/stdout)
├── StdioServerController (接管当前进程 stdin/stdout)
├── WebSocketClientController (连接 WS 服务端)
└── WebSocketServerController (监听 WS 端口)
```
共享的核心机制:
- `call_action()` / `call_action_generator()` — RPC 调用/流式调用
- `ActionRequest` / `ActionResponse` — 请求/响应协议
- `seq_id` 关联 — 并发请求复用单连接
- `CommonAction.PING` — 两者都用于初始握手
- 文件传输 (`send_file`) — Plugin 用Box 不用
---
## 6. 端口方案
| 服务 | Plugin | Box |
|------|--------|-----|
| Action RPC (stdio) | stdin/stdout | stdin/stdout |
| Action RPC (WS) | `:5400` | `:5410/rpc/ws` |
| 辅助服务 | debug WS `:5401` | managed process WS relay `:5410/v1/sessions/{id}/managed-process/ws` |
**Box 特点**: 单端口 aiohttp 服务(默认 5410通过路径区分 Action RPC 和 managed process relay。即使在 stdio 模式,也在 `:5410` 启动 aiohttp 用于 managed process attach。Plugin 在 stdio 模式不开额外端口。
---
## 7. 销毁对比
### Plugin
```python
dispose():
if stdio: ctrl.process.terminate()
_dispose_subprocess() # Windows 子进程
heartbeat_task.cancel()
```
### Box
```python
connector.dispose():
_handler_task.cancel()
_ctrl_task.cancel()
_subprocess.terminate()
service.dispose():
connector.dispose()
loop.create_task(client.shutdown()) # RPC SHUTDOWN → 清理所有容器
```
Box 的 RPC SHUTDOWN 确保容器被正确停止不会成为孤儿。Plugin 直接杀进程。
---
## 8. 改进建议
### P0
1. **两者都加 WS 认证**: 至少 token 认证INIT 时下发,连接时校验)
### P1
2. **考虑 Box 继承 ManagedRuntimeConnector**: 复用 `_start_runtime_subprocess` / `_wait_until_ready` / `_dispose_subprocess`,减少重复代码
3. **Plugin 重连加退避**: 固定 3s 无 backoff 可能造成日志洪水,建议向 Box 的指数退避看齐
4. **统一连接管理模式**: Event-based (Box) vs direct-await (Plugin),考虑收敛为一种
### 已完成(自上一轮)
- ~~Box 加重连~~commit `2dfd9d5d`
- ~~Box 加心跳~~20s loop 与 Plugin 一致)
- ~~Box 加 Windows 支持~~commit `120817a` / `fafb7a4`

View File

@@ -1,425 +0,0 @@
# Workflow 用户指南
本文档帮助您了解和使用 LangBot 的 Workflow工作流功能通过可视化方式构建自动化的对话处理流程。
## 目录
- [功能介绍](#功能介绍)
- [快速入门](#快速入门)
- [节点类型说明](#节点类型说明)
- [编辑器使用指南](#编辑器使用指南)
- [调试功能](#调试功能)
- [常见问题解答](#常见问题解答)
---
## 功能介绍
### 什么是 Workflow
Workflow工作流是 LangBot 提供的可视化自动化编排系统。通过拖拽节点、连接边的方式,您可以:
- 📝 **构建复杂的对话流程**:使用条件分支、循环等控制节点
- 🤖 **调用 AI 能力**:集成 LLM、知识库检索、参数提取
- 🔗 **连接外部服务**:集成 Dify、n8n、Coze 等平台
-**自动化任务执行**消息触发、定时触发、Webhook 触发
### Workflow vs Pipeline
| 对比项 | Pipeline | Workflow |
|-------|----------|----------|
| 配置方式 | 表单配置 | 可视化拖拽 |
| 流程控制 | 线性执行 | 支持分支、循环、并行 |
| 适用场景 | 简单对话 | 复杂流程 |
| 学习曲线 | 低 | 中等 |
---
## 快速入门
### 第一步:创建 Workflow
1. 在侧边栏点击 **Workflow** 进入工作流列表
2. 点击右上角 **创建工作流** 按钮
3. 填写基本信息:
- **名称**:给工作流起一个描述性的名字
- **描述**:可选,说明工作流的用途
- **图标**:选择一个 emoji 作为标识
### 第二步:添加节点
进入编辑器后,左侧是节点面板,中间是画布区域,右侧是属性面板。
1. **添加触发节点**:从左侧面板拖拽一个"消息触发"节点到画布
2. **添加 AI 节点**:拖拽一个"LLM 调用"节点
3. **添加回复节点**:拖拽一个"回复消息"节点
### 第三步:连接节点
1. 将鼠标悬停在触发节点的输出端口(右侧小圆点)
2. 按住鼠标拖拽到 LLM 节点的输入端口(左侧小圆点)
3. 同样方式连接 LLM 节点和回复节点
```
[消息触发] ──▶ [LLM 调用] ──▶ [回复消息]
```
### 第四步:配置节点
点击 LLM 调用节点,在右侧属性面板配置:
- **运行方式**:选择"本地 Agent"
- **系统提示词**:描述 AI 的角色和行为
- **模型**:选择要使用的 LLM 模型
点击回复消息节点配置:
- **消息内容**:设置为 `{{nodes.llm_call.outputs.response}}`(引用 LLM 输出)
### 第五步:保存并绑定
1. 点击工具栏的 **保存** 按钮
2. 返回 Bot 配置页面
3. 在 Bot 的绑定设置中选择 **Workflow**,然后选择刚创建的工作流
恭喜!您已经创建了第一个 Workflow。
---
## 节点类型说明
### 触发节点 (Trigger)
触发节点是工作流的入口,定义何时启动执行。
| 节点 | 说明 | 输出 |
|-----|------|------|
| 消息触发 | 收到消息时触发 | message, sender_id, platform |
| 定时触发 | 按 Cron 表达式定时触发 | timestamp |
| Webhook 触发 | 收到 HTTP 请求时触发 | request_body, headers |
| 事件触发 | 系统事件触发 | event_type, event_data |
**消息触发配置示例**
```yaml
触发条件:
- 关键词匹配: ["帮助", "help"]
- 平台: ["wechat", "qq"]
```
### AI 节点
AI 节点用于调用各种 AI 能力。
| 节点 | 说明 | 典型用途 |
|-----|------|---------|
| LLM 调用 | 调用大语言模型 | 生成回复、理解意图 |
| 问题分类器 | 对用户问题分类 | 路由到不同处理分支 |
| 参数提取器 | 从文本提取结构化数据 | 提取订单号、日期等 |
| 知识库检索 | 查询知识库 | RAG 增强回复 |
**LLM 调用配置示例**
```yaml
运行方式: 本地 Agent
模型: gpt-4
系统提示词: |
你是一个友好的客服助手。
请根据用户的问题提供帮助。
温度: 0.7
最大 Token 数: 2000
```
### 处理节点 (Process)
处理节点用于数据处理和外部调用。
| 节点 | 说明 | 典型用途 |
|-----|------|---------|
| 代码执行 | 执行 Python/JavaScript 代码 | 数据处理、格式转换 |
| HTTP 请求 | 发送 HTTP 请求 | 调用外部 API |
| 数据转换 | JSON/模板转换 | 数据格式化 |
**HTTP 请求配置示例**
```yaml
URL: https://api.example.com/data
方法: POST
请求头:
Content-Type: application/json
Authorization: Bearer {{variables.api_key}}
请求体: |
{"query": "{{message.content}}"}
```
### 控制节点 (Control)
控制节点用于流程控制。
| 节点 | 说明 | 用途 |
|-----|------|------|
| 条件分支 | 二选一分支 | if-else 逻辑 |
| 多路分支 | 多选一分支 | switch-case 逻辑 |
| 循环 | 遍历数组 | 批量处理 |
| 并行 | 同时执行多分支 | 并发处理 |
| 等待 | 暂停执行 | 延时处理 |
| 合并 | 合并多个分支 | 汇总结果 |
**条件分支配置示例**
```yaml
条件表达式: "{{nodes.classifier.outputs.category}}" == "complaint"
真分支: 投诉处理
假分支: 普通咨询
```
### 动作节点 (Action)
动作节点执行具体操作。
| 节点 | 说明 | 用途 |
|-----|------|------|
| 发送消息 | 主动发送消息 | 通知、推送 |
| 回复消息 | 回复当前消息 | 对话回复 |
| 存储数据 | 保存数据到存储 | 持久化 |
| 调用 Pipeline | 调用现有 Pipeline | 复用现有流程 |
**回复消息配置示例**
```yaml
消息内容: |
感谢您的咨询!
{{nodes.llm_call.outputs.response}}
如有其他问题,随时联系我。
```
### 集成节点 (Integration)
集成节点连接外部平台。
| 节点 | 说明 | 平台 |
|-----|------|------|
| Dify 工作流 | 调用 Dify 应用 | Dify |
| Dify 知识库 | 查询 Dify 知识库 | Dify |
| n8n 工作流 | 调用 n8n 流程 | n8n |
| Langflow | 调用 Langflow 流程 | Langflow |
| Coze Bot | 调用扣子 Bot | Coze |
**Dify 工作流配置示例**
```yaml
API 地址: https://api.dify.ai/v1
API Key: sk-xxxxx
应用类型: workflow
同步对话历史: true
```
---
## 编辑器使用指南
### 画布操作
| 操作 | 方式 |
|-----|------|
| 平移画布 | 按住鼠标中键/空格+左键 拖拽 |
| 缩放画布 | 鼠标滚轮 / 工具栏按钮 |
| 框选多个节点 | 按住 Shift + 拖拽框选 |
| 适应视图 | 点击工具栏"适应"按钮 |
### 节点操作
| 操作 | 方式 |
|-----|------|
| 添加节点 | 从左侧面板拖拽到画布 |
| 移动节点 | 点击节点拖拽 |
| 删除节点 | 选中后按 Delete / 点击工具栏删除 |
| 复制节点 | 选中后 Ctrl+C / 工具栏复制 |
| 粘贴节点 | Ctrl+V / 工具栏粘贴 |
### 连接操作
| 操作 | 方式 |
|-----|------|
| 创建连接 | 从输出端口拖拽到输入端口 |
| 删除连接 | 点击连接线后按 Delete |
| 选中连接 | 点击连接线 |
### 快捷键
| 快捷键 | 功能 |
|-------|------|
| Ctrl + Z | 撤销 |
| Ctrl + Shift + Z | 重做 |
| Ctrl + C | 复制 |
| Ctrl + V | 粘贴 |
| Delete | 删除选中 |
| Ctrl + S | 保存 |
### 工具栏功能
```
[撤销] [重做] | [放大] [缩小] [适应] | [复制] [粘贴] [删除] | [保存] [调试]
```
---
## 调试功能
### 启动调试
1. 点击工具栏的 **调试** 按钮
2. 在调试面板中配置初始数据:
- **输入消息**:模拟用户发送的消息
- **会话 ID**:可选,用于测试会话变量
- **变量**:设置初始变量值
3. 点击 **开始调试** 按钮
### 调试控制
| 按钮 | 功能 |
|-----|------|
| ▶️ 开始/继续 | 开始或继续执行 |
| ⏸️ 暂停 | 暂停执行 |
| ⏹️ 停止 | 停止执行 |
| ⏭️ 单步 | 执行下一个节点 |
### 断点
- **设置断点**:点击节点上的断点图标
- **断点触发**:执行到断点时自动暂停
- **查看状态**:在暂停时查看节点的输入输出
### 执行日志
调试面板下方显示实时日志:
```
[INFO] 2024-01-15 10:30:00 - Starting debug execution
[INFO] 2024-01-15 10:30:00 - Executing node: message_trigger
[DEBUG] 2024-01-15 10:30:00 - Node inputs: {"message": "你好"}
[INFO] 2024-01-15 10:30:01 - Node completed in 50ms
[INFO] 2024-01-15 10:30:01 - Executing node: llm_call
...
```
### 节点状态颜色
| 颜色 | 状态 |
|-----|------|
| 灰色 | 待执行 |
| 蓝色 | 执行中 |
| 绿色 | 已完成 |
| 红色 | 失败 |
| 黄色 | 已跳过 |
---
## 常见问题解答
### Q1如何在节点间传递数据
使用表达式语法引用其他节点的输出:
```
{{nodes.节点ID.outputs.输出名称}}
```
例如:
- `{{nodes.llm_call.outputs.response}}` - 引用 LLM 节点的响应
- `{{nodes.http_request.outputs.body}}` - 引用 HTTP 请求的响应体
### Q2如何使用变量
Workflow 支持三种变量类型:
1. **工作流变量**`{{variables.变量名}}`
2. **会话变量**`{{conversation_variables.变量名}}`
3. **消息上下文**`{{message.content}}``{{message.sender_id}}`
### Q3条件分支如何写条件表达式
支持以下运算符:
- 比较:`==`, `!=`, `>`, `<`, `>=`, `<=`
- 逻辑:`and`, `or`, `not`
- 包含:`in`
示例:
```python
# 字符串比较
"{{nodes.classifier.outputs.intent}}" == "purchase"
# 数值比较
{{nodes.extractor.outputs.amount}} > 1000
# 包含检查
"退款" in "{{message.content}}"
```
### Q4如何处理错误
1. **节点级重试**:在节点配置中设置重试次数
2. **全局错误处理**:在 Workflow 设置中配置错误处理策略
3. **条件分支**:使用条件节点检查上一节点的状态
### Q5如何查看执行历史
1. 进入 Workflow 详情页
2. 点击 **执行历史** 标签
3. 查看每次执行的状态、耗时、输入输出
### Q6Workflow 可以被多个 Bot 使用吗?
是的。一个 Workflow 可以被多个 Bot 绑定使用,但每个 Bot 只能绑定一个处理单元Pipeline 或 Workflow
### Q7如何复制现有的 Workflow
在 Workflow 列表页,点击工作流卡片右上角的菜单,选择"复制"即可创建副本。
### Q8支持版本回滚吗
支持。每次保存都会创建新版本。在 Workflow 详情页可以查看版本历史并回滚到指定版本。
---
## 最佳实践
### 1. 合理命名
- 为节点和 Workflow 使用描述性名称
- 使用统一的命名规范
### 2. 模块化设计
- 将复杂流程拆分为多个小 Workflow
- 使用"调用 Pipeline"节点复用现有流程
### 3. 错误处理
- 为关键节点设置重试机制
- 使用条件分支处理异常情况
- 添加日志记录便于排查问题
### 4. 测试先行
- 使用调试功能充分测试
- 准备多种测试场景
- 检查边界情况
### 5. 性能优化
- 避免不必要的节点
- 使用并行节点提高效率
- 合理设置超时时间
---
## 更多资源
- [开发者文档](../development/workflow-system.md)
- [设计文档](../../../plans/langbot-workflow-design.md)
- [API 文档](../service-api-openapi.json)

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

View File

@@ -1,6 +1,6 @@
[project]
name = "langbot"
version = "4.9.7"
version = "4.10.0-beta.1"
description = "Production-grade platform for building agentic IM bots"
readme = "README.md"
license-files = ["LICENSE"]
@@ -70,7 +70,7 @@ dependencies = [
"chromadb>=1.0.0,<2.0.0",
"qdrant-client (>=1.15.1,<2.0.0)",
"pyseekdb==1.1.0.post3",
"langbot-plugin @ file:///home/typer/Desktop/langbot-plugin-sdk",
"langbot-plugin==0.4.0b1",
"asyncpg>=0.30.0",
"line-bot-sdk>=3.19.0",
"matrix-nio>=0.25.2",
@@ -223,4 +223,3 @@ skip-magic-trailing-comma = false
# Like Black, automatically detect the appropriate line ending.
line-ending = "auto"

View File

@@ -1,3 +1,3 @@
"""LangBot - Production-grade platform for building agentic IM bots"""
__version__ = '4.9.7'
__version__ = '4.10.0-beta.1'

View File

@@ -5,6 +5,8 @@ import argparse
import sys
import os
from langbot.pkg.utils import paths
# ASCII art banner
asciiart = r"""
_ ___ _
@@ -27,6 +29,12 @@ async def main_entry(loop: asyncio.AbstractEventLoop):
help='Use standalone plugin runtime / 使用独立插件运行时',
default=False,
)
parser.add_argument(
'--standalone-box',
action='store_true',
help='Use standalone box runtime / 使用独立 Box 运行时',
default=False,
)
parser.add_argument('--debug', action='store_true', help='Debug mode / 调试模式', default=False)
args = parser.parse_args()
@@ -35,6 +43,11 @@ async def main_entry(loop: asyncio.AbstractEventLoop):
platform.standalone_runtime = True
if args.standalone_box:
from langbot.pkg.utils import platform
platform.standalone_box = True
if args.debug:
from langbot.pkg.utils import constants
@@ -87,7 +100,7 @@ def main():
# Set up the working directory
# When installed as a package, we need to handle the working directory differently
# We'll create data directory in current working directory if not exists
os.makedirs('data', exist_ok=True)
os.makedirs(paths.get_data_root(), exist_ok=True)
loop = asyncio.new_event_loop()

View File

@@ -0,0 +1,22 @@
from __future__ import annotations
from .. import group
@group.group_class('box', '/api/v1/box')
class BoxRouterGroup(group.RouterGroup):
async def initialize(self) -> None:
@self.route('/status', methods=['GET'], auth_type=group.AuthType.USER_TOKEN)
async def _() -> str:
status = await self.ap.box_service.get_status()
return self.success(data=status)
@self.route('/sessions', methods=['GET'], auth_type=group.AuthType.USER_TOKEN)
async def _() -> str:
sessions = await self.ap.box_service.get_sessions()
return self.success(data=sessions)
@self.route('/errors', methods=['GET'], auth_type=group.AuthType.USER_TOKEN)
async def _() -> str:
errors = self.ap.box_service.get_recent_errors()
return self.success(data=errors)

View File

@@ -0,0 +1,52 @@
from __future__ import annotations
import asyncio
import quart
from .. import group
@group.group_class('extensions', '/api/v1/extensions')
class ExtensionsRouterGroup(group.RouterGroup):
"""Unified API for installed extensions (plugins, MCP servers, skills)."""
async def initialize(self) -> None:
@self.route('', methods=['GET'], auth_type=group.AuthType.USER_TOKEN_OR_API_KEY)
async def _() -> quart.Response:
plugins, mcp_servers, skills = await asyncio.gather(
self.ap.plugin_connector.list_plugins(),
self.ap.mcp_service.get_mcp_servers(contain_runtime_info=True),
self.ap.skill_service.list_skills(),
return_exceptions=True,
)
def _sort_key(item: dict) -> str:
if item['type'] == 'plugin':
return (
item['plugin']
.get('manifest', {})
.get('manifest', {})
.get('metadata', {})
.get('name', '')
.lower()
)
if item['type'] == 'mcp':
return (item['server'].get('name') or '').lower()
if item['type'] == 'skill':
return (item['skill'].get('display_name') or item['skill'].get('name') or '').lower()
return ''
extensions: list[dict] = []
if isinstance(plugins, list):
for plugin in plugins:
extensions.append({'type': 'plugin', 'plugin': plugin})
if isinstance(mcp_servers, list):
for server in mcp_servers:
extensions.append({'type': 'mcp', 'server': server})
if isinstance(skills, list):
for skill in skills:
extensions.append({'type': 'skill', 'skill': skill})
extensions.sort(key=_sort_key)
return self.success(data={'extensions': extensions})

View File

@@ -73,15 +73,21 @@ class PipelinesRouterGroup(group.RouterGroup):
plugins = await self.ap.plugin_connector.list_plugins(component_kinds=pipeline_component_kinds)
mcp_servers = await self.ap.mcp_service.get_mcp_servers(contain_runtime_info=True)
# Get available skills
available_skills = await self.ap.skill_service.list_skills()
extensions_prefs = pipeline.get('extensions_preferences', {})
return self.success(
data={
'enable_all_plugins': extensions_prefs.get('enable_all_plugins', True),
'enable_all_mcp_servers': extensions_prefs.get('enable_all_mcp_servers', True),
'enable_all_skills': extensions_prefs.get('enable_all_skills', True),
'bound_plugins': extensions_prefs.get('plugins', []),
'available_plugins': plugins,
'bound_mcp_servers': extensions_prefs.get('mcp_servers', []),
'available_mcp_servers': mcp_servers,
'bound_skills': extensions_prefs.get('skills', []),
'available_skills': available_skills,
}
)
elif quart.request.method == 'PUT':
@@ -89,11 +95,19 @@ class PipelinesRouterGroup(group.RouterGroup):
json_data = await quart.request.json
enable_all_plugins = json_data.get('enable_all_plugins', True)
enable_all_mcp_servers = json_data.get('enable_all_mcp_servers', True)
enable_all_skills = json_data.get('enable_all_skills', True)
bound_plugins = json_data.get('bound_plugins', [])
bound_mcp_servers = json_data.get('bound_mcp_servers', [])
bound_skills = json_data.get('bound_skills', [])
await self.ap.pipeline_service.update_pipeline_extensions(
pipeline_uuid, bound_plugins, bound_mcp_servers, enable_all_plugins, enable_all_mcp_servers
pipeline_uuid,
bound_plugins,
bound_mcp_servers,
enable_all_plugins,
enable_all_mcp_servers,
bound_skills=bound_skills,
enable_all_skills=enable_all_skills,
)
return self.success()

View File

@@ -1,11 +1,15 @@
from __future__ import annotations
import base64
import io
import quart
import re
import httpx
import uuid
import os
import zipfile
import yaml
from urllib.parse import urlparse
import posixpath
import sqlalchemy
@@ -53,6 +57,97 @@ def _get_request_origin() -> str:
@group.group_class('plugins', '/api/v1/plugins')
class PluginsRouterGroup(group.RouterGroup):
@staticmethod
def _normalize_archive_path(path: str) -> str:
normalized = str(path or '').replace('\\', '/').strip('/')
return posixpath.normpath(normalized) if normalized else ''
@classmethod
def _component_source_path(cls, entry) -> str:
if isinstance(entry, dict):
return cls._normalize_archive_path(entry.get('path') or '')
return cls._normalize_archive_path(str(entry or ''))
@classmethod
def _count_component_configs(cls, component_config, archive_names: list[str]) -> int:
normalized_names = [cls._normalize_archive_path(name) for name in archive_names]
component_files: set[str] = set()
if isinstance(component_config, list):
return len(component_config)
if not isinstance(component_config, dict):
return 1 if component_config else 0
for entry in component_config.get('fromFiles') or []:
source_path = cls._component_source_path(entry)
if source_path and source_path in normalized_names:
component_files.add(source_path)
for entry in component_config.get('fromDirs') or []:
source_dir = cls._component_source_path(entry).rstrip('/')
if not source_dir:
continue
prefix = f'{source_dir}/'
for archive_name in normalized_names:
if not archive_name.startswith(prefix):
continue
if archive_name.lower().endswith(('.yaml', '.yml')):
component_files.add(archive_name)
if component_files:
return len(component_files)
return 1 if any(key in component_config for key in ('path', 'name', 'kind')) else 0
@classmethod
def _count_plugin_components(cls, components, archive_names: list[str]) -> dict[str, int]:
if not isinstance(components, dict):
return {}
component_counts: dict[str, int] = {}
for kind, component_config in components.items():
count = cls._count_component_configs(component_config, archive_names)
if count > 0:
component_counts[str(kind)] = count
return component_counts
@staticmethod
def _parse_github_repo_url(repo_url: str) -> dict | None:
raw_url = str(repo_url or '').strip()
if not raw_url:
return None
if not re.match(r'^[a-zA-Z][a-zA-Z0-9+.-]*://', raw_url):
raw_url = f'https://{raw_url}'
parsed = urlparse(raw_url)
if parsed.netloc.lower() not in ('github.com', 'www.github.com'):
return None
parts = [part for part in parsed.path.strip('/').split('/') if part]
if len(parts) < 2:
return None
owner = parts[0]
repo = parts[1]
if repo.endswith('.git'):
repo = repo[:-4]
if not owner or not repo:
return None
ref = ''
subdir = ''
if len(parts) >= 4 and parts[2] in ('tree', 'blob'):
ref = parts[3]
subdir = '/'.join(parts[4:]).strip('/')
return {
'owner': owner,
'repo': repo,
'ref': ref,
'subdir': subdir,
}
async def _check_extensions_limit(self) -> str | None:
"""Check if extensions limit is reached. Returns error response if limit exceeded, None otherwise."""
limitation = self.ap.instance_config.data.get('system', {}).get('limitation', {})
@@ -254,17 +349,37 @@ class PluginsRouterGroup(group.RouterGroup):
data = await quart.request.json
repo_url = data.get('repo_url', '')
# Parse GitHub repository URL to extract owner and repo
# Supports: https://github.com/owner/repo or github.com/owner/repo
pattern = r'github\.com/([^/]+)/([^/]+?)(?:\.git)?(?:/.*)?$'
match = re.search(pattern, repo_url)
if not match:
parsed_repo = self._parse_github_repo_url(repo_url)
if not parsed_repo:
return self.http_status(400, -1, 'Invalid GitHub repository URL')
owner, repo = match.groups()
owner = parsed_repo['owner']
repo = parsed_repo['repo']
requested_ref = parsed_repo['ref']
requested_subdir = parsed_repo['subdir']
try:
if requested_ref:
return self.success(
data={
'releases': [
{
'id': 0,
'tag_name': requested_ref,
'name': requested_ref,
'published_at': '',
'prerelease': False,
'draft': False,
'source_type': 'branch',
'archive_url': f'https://api.github.com/repos/{owner}/{repo}/zipball/{requested_ref}',
}
],
'owner': owner,
'repo': repo,
'source_subdir': requested_subdir,
}
)
# Fetch releases from GitHub API
url = f'https://api.github.com/repos/{owner}/{repo}/releases'
async with httpx.AsyncClient(
@@ -290,7 +405,14 @@ class PluginsRouterGroup(group.RouterGroup):
}
)
return self.success(data={'releases': formatted_releases, 'owner': owner, 'repo': repo})
return self.success(
data={
'releases': formatted_releases,
'owner': owner,
'repo': repo,
'source_subdir': requested_subdir,
}
)
except httpx.RequestError as e:
return self.http_status(500, -1, f'Failed to fetch releases: {str(e)}')
@@ -445,6 +567,62 @@ class PluginsRouterGroup(group.RouterGroup):
return self.success(data={'task_id': wrapper.id})
@self.route('/install/local/preview', methods=['POST'], auth_type=group.AuthType.USER_TOKEN_OR_API_KEY)
async def _() -> str:
file = (await quart.request.files).get('file')
if file is None:
return self.http_status(400, -1, 'file is required')
file_bytes = file.read()
try:
with zipfile.ZipFile(io.BytesIO(file_bytes)) as zf:
names = [name for name in zf.namelist() if not name.endswith('/')]
manifest_name = next(
(
name
for name in names
if name.replace('\\', '/').strip('/').lower() in ('manifest.yaml', 'manifest.yml')
),
None,
)
if manifest_name is None:
return self.http_status(400, -1, 'manifest.yaml is required')
manifest = yaml.safe_load(zf.read(manifest_name).decode('utf-8')) or {}
requirements: list[str] = []
requirements_name = next(
(name for name in names if name.replace('\\', '/').strip('/').lower() == 'requirements.txt'),
None,
)
if requirements_name is not None:
requirements = [
line.strip()
for line in zf.read(requirements_name).decode('utf-8', errors='ignore').splitlines()
if line.strip() and not line.strip().startswith('#')
]
spec = manifest.get('spec') or {}
components = spec.get('components') or {}
component_counts = self._count_plugin_components(components, names)
component_types = list(component_counts.keys())
return self.success(
data={
'filename': file.filename or 'local plugin',
'size': len(file_bytes),
'manifest': manifest,
'metadata': manifest.get('metadata') or {},
'component_types': component_types,
'component_counts': component_counts,
'requirements': requirements,
'file_count': len(names),
}
)
except zipfile.BadZipFile:
return self.http_status(400, -1, 'invalid .lbpkg file')
except Exception as exc:
return self.http_status(500, -1, f'Failed to preview plugin package: {exc}')
@self.route('/config-files', methods=['POST'], auth_type=group.AuthType.USER_TOKEN)
async def _() -> str:
"""Upload a file for plugin configuration"""

View File

@@ -31,6 +31,9 @@ class MCPRouterGroup(group.RouterGroup):
@self.route('/servers/<server_name>', methods=['GET', 'PUT', 'DELETE'], auth_type=group.AuthType.USER_TOKEN)
async def _(server_name: str) -> str:
"""获取、更新或删除MCP服务器配置"""
from urllib.parse import unquote
server_name = unquote(server_name)
server_data = await self.ap.mcp_service.get_mcp_server_by_name(server_name)
if server_data is None:
@@ -57,6 +60,9 @@ class MCPRouterGroup(group.RouterGroup):
@self.route('/servers/<server_name>/test', methods=['POST'], auth_type=group.AuthType.USER_TOKEN)
async def _(server_name: str) -> str:
"""测试MCP服务器连接"""
from urllib.parse import unquote
server_name = unquote(server_name)
server_data = await quart.request.json
task_id = await self.ap.mcp_service.test_mcp_server(server_name=server_name, server_data=server_data)
return self.success(data={'task_id': task_id})

View File

@@ -0,0 +1,190 @@
from __future__ import annotations
import quart
from langbot_plugin.box.errors import BoxError
from .. import group
@group.group_class('skills', '/api/v1/skills')
class SkillsRouterGroup(group.RouterGroup):
"""Skills management API endpoints."""
async def initialize(self) -> None:
@self.route('', methods=['GET', 'POST'], auth_type=group.AuthType.USER_TOKEN_OR_API_KEY)
async def list_or_create_skills() -> quart.Response:
if quart.request.method == 'GET':
try:
skills = await self.ap.skill_service.list_skills()
except (ValueError, BoxError) as exc:
return self.http_status(400, -1, str(exc))
return self.success(data={'skills': skills})
data = await quart.request.json
if 'name' not in data or not data['name']:
return self.http_status(400, -1, 'Missing required field: name')
try:
skill = await self.ap.skill_service.create_skill(data)
return self.success(data={'skill': skill})
except (ValueError, BoxError) as exc:
return self.http_status(400, -1, str(exc))
@self.route('/<skill_name>', methods=['GET', 'PUT', 'DELETE'], auth_type=group.AuthType.USER_TOKEN_OR_API_KEY)
async def get_update_delete_skill(skill_name: str) -> quart.Response:
if quart.request.method == 'GET':
try:
skill = await self.ap.skill_service.get_skill(skill_name)
except (ValueError, BoxError) as exc:
return self.http_status(400, -1, str(exc))
if not skill:
return self.http_status(404, -1, 'Skill not found')
return self.success(data={'skill': skill})
if quart.request.method == 'PUT':
data = await quart.request.json
try:
skill = await self.ap.skill_service.update_skill(skill_name, data)
return self.success(data={'skill': skill})
except (ValueError, BoxError) as exc:
return self.http_status(400, -1, str(exc))
try:
await self.ap.skill_service.delete_skill(skill_name)
return self.success()
except (ValueError, BoxError) as exc:
return self.http_status(400, -1, str(exc))
@self.route('/<skill_name>/files', methods=['GET'], auth_type=group.AuthType.USER_TOKEN_OR_API_KEY)
async def list_skill_files(skill_name: str) -> quart.Response:
"""List files in skill package directory."""
path = quart.request.args.get('path', '.').strip()
include_hidden = quart.request.args.get('include_hidden', 'false').lower() == 'true'
try:
result = await self.ap.skill_service.list_skill_files(
skill_name,
path=path,
include_hidden=include_hidden,
)
return self.success(data=result)
except (ValueError, BoxError) as exc:
return self.http_status(400, -1, str(exc))
@self.route(
'/<skill_name>/files/<path:path>', methods=['GET', 'PUT'], auth_type=group.AuthType.USER_TOKEN_OR_API_KEY
)
async def read_or_write_skill_file(skill_name: str, path: str) -> quart.Response:
"""Read or write a file in skill package."""
if quart.request.method == 'GET':
try:
result = await self.ap.skill_service.read_skill_file(skill_name, path)
return self.success(data=result)
except (ValueError, BoxError) as exc:
return self.http_status(400, -1, str(exc))
# PUT - write file
data = await quart.request.json
content = data.get('content', '')
if content is None:
return self.http_status(400, -1, 'Missing required field: content')
try:
result = await self.ap.skill_service.write_skill_file(skill_name, path, content)
return self.success(data=result)
except (ValueError, BoxError) as exc:
return self.http_status(400, -1, str(exc))
@self.route('/<skill_name>/preview', methods=['GET'], auth_type=group.AuthType.USER_TOKEN_OR_API_KEY)
async def preview_skill(skill_name: str) -> quart.Response:
skill = self.ap.skill_mgr.get_skill_by_name(skill_name)
if not skill:
return self.http_status(404, -1, 'Skill not found')
return self.success(data={'instructions': skill.get('instructions', '')})
@self.route('/install/github', methods=['POST'], auth_type=group.AuthType.USER_TOKEN_OR_API_KEY)
async def install_skill_from_github() -> quart.Response:
data = await quart.request.json
required_fields = ['asset_url', 'owner', 'repo']
for field in required_fields:
if field not in data or not data[field]:
return self.http_status(400, -1, f'Missing required field: {field}')
asset_url = str(data['asset_url']).strip().lower().split('?', 1)[0].split('#', 1)[0]
if not asset_url.endswith('skill.md') and not data.get('release_tag'):
return self.http_status(400, -1, 'Missing required field: release_tag')
try:
skill = await self.ap.skill_service.install_from_github(data)
return self.success(data={'skills': skill})
except (ValueError, BoxError) as exc:
return self.http_status(400, -1, str(exc))
except Exception as exc:
return self.http_status(500, -1, f'Failed to install skill: {exc}')
@self.route('/install/github/preview', methods=['POST'], auth_type=group.AuthType.USER_TOKEN_OR_API_KEY)
async def preview_skill_from_github() -> quart.Response:
data = await quart.request.json
required_fields = ['asset_url', 'owner', 'repo']
for field in required_fields:
if field not in data or not data[field]:
return self.http_status(400, -1, f'Missing required field: {field}')
asset_url = str(data['asset_url']).strip().lower().split('?', 1)[0].split('#', 1)[0]
if not asset_url.endswith('skill.md') and not data.get('release_tag'):
return self.http_status(400, -1, 'Missing required field: release_tag')
try:
preview = await self.ap.skill_service.preview_install_from_github(data)
return self.success(data={'skills': preview})
except (ValueError, BoxError) as exc:
return self.http_status(400, -1, str(exc))
except Exception as exc:
return self.http_status(500, -1, f'Failed to preview skill: {exc}')
@self.route('/install/upload', methods=['POST'], auth_type=group.AuthType.USER_TOKEN_OR_API_KEY)
async def install_skill_from_upload() -> quart.Response:
file = (await quart.request.files).get('file')
if file is None:
return self.http_status(400, -1, 'file is required')
form = await quart.request.form
try:
skill = await self.ap.skill_service.install_from_zip_upload(
file_bytes=file.read(),
filename=file.filename or '',
source_paths=form.getlist('source_paths'),
)
return self.success(data={'skills': skill})
except (ValueError, BoxError) as exc:
return self.http_status(400, -1, str(exc))
except Exception as exc:
return self.http_status(500, -1, f'Failed to install skill: {exc}')
@self.route('/install/upload/preview', methods=['POST'], auth_type=group.AuthType.USER_TOKEN_OR_API_KEY)
async def preview_skill_from_upload() -> quart.Response:
file = (await quart.request.files).get('file')
if file is None:
return self.http_status(400, -1, 'file is required')
try:
preview = await self.ap.skill_service.preview_install_from_zip_upload(
file_bytes=file.read(),
filename=file.filename or '',
)
return self.success(data={'skills': preview})
except (ValueError, BoxError) as exc:
return self.http_status(400, -1, str(exc))
except Exception as exc:
return self.http_status(500, -1, f'Failed to preview skill: {exc}')
@self.route('/scan', methods=['GET'], auth_type=group.AuthType.USER_TOKEN_OR_API_KEY)
async def scan_skill_directory() -> quart.Response:
path = quart.request.args.get('path', '').strip()
if not path:
return self.http_status(400, -1, 'Missing required parameter: path')
try:
result = await self.ap.skill_service.scan_directory_async(path)
return self.success(data=result)
except (ValueError, BoxError) as exc:
return self.http_status(400, -1, str(exc))

View File

@@ -1,5 +0,0 @@
# Workflow router group
from .workflows import WorkflowsRouterGroup, ExecutionsRouterGroup
from .websocket_chat import WorkflowWebSocketChatRouterGroup
__all__ = ['WorkflowsRouterGroup', 'ExecutionsRouterGroup', 'WorkflowWebSocketChatRouterGroup']

View File

@@ -1,260 +0,0 @@
"""Workflow WebSocket聊天路由 - 支持工作流调试的双向实时通信"""
import asyncio
import datetime
import json
import logging
import quart
from ... import group
from ......platform.sources.websocket_manager import ws_connection_manager
logger = logging.getLogger(__name__)
@group.group_class('workflow_websocket_chat', '/api/v1/workflows/<workflow_uuid>/ws')
class WorkflowWebSocketChatRouterGroup(group.RouterGroup):
async def initialize(self) -> None:
@self.quart_app.websocket(self.path + '/connect')
async def workflow_websocket_connect(workflow_uuid: str):
"""
建立工作流WebSocket连接
URL参数:
- workflow_uuid: 工作流UUID
- session_type: 会话类型 (person/group)
"""
try:
session_type = quart.websocket.args.get('session_type', 'person')
logger.info(
'Workflow WebSocket connect request received',
extra={
'workflow_uuid': workflow_uuid,
'session_type': session_type,
'path': quart.websocket.path,
'query_string': quart.websocket.query_string.decode('utf-8', errors='ignore'),
'remote_addr': getattr(quart.websocket, 'remote_addr', None),
'user_agent': quart.websocket.headers.get('User-Agent', ''),
'host': quart.websocket.headers.get('Host', ''),
'origin': quart.websocket.headers.get('Origin', ''),
},
)
if session_type not in ['person', 'group']:
await quart.websocket.send(
json.dumps({'type': 'error', 'message': 'session_type must be person or group'})
)
return
websocket_adapter = self.ap.platform_mgr.websocket_proxy_bot.adapter
if not websocket_adapter:
logger.warning(
'Workflow WebSocket adapter missing',
extra={
'workflow_uuid': workflow_uuid,
'session_type': session_type,
},
)
await quart.websocket.send(json.dumps({'type': 'error', 'message': 'WebSocket adapter not found'}))
return
connection = await ws_connection_manager.add_connection(
websocket=quart.websocket._get_current_object(),
pipeline_uuid=workflow_uuid,
session_type=session_type,
metadata={'user_agent': quart.websocket.headers.get('User-Agent', ''), 'is_workflow': True},
)
await quart.websocket.send(
json.dumps(
{
'type': 'connected',
'connection_id': connection.connection_id,
'workflow_uuid': workflow_uuid,
'session_type': session_type,
'timestamp': connection.created_at.isoformat(),
}
)
)
logger.debug(
f'Workflow WebSocket connection established: {connection.connection_id} '
f'(workflow={workflow_uuid}, session_type={session_type})'
)
receive_task = asyncio.create_task(self._handle_receive(connection, websocket_adapter))
send_task = asyncio.create_task(self._handle_send(connection))
try:
await asyncio.gather(receive_task, send_task)
except Exception as e:
logger.error(f'Workflow WebSocket task execution error: {e}')
finally:
await ws_connection_manager.remove_connection(connection.connection_id)
logger.debug(f'Workflow WebSocket connection cleaned: {connection.connection_id}')
except Exception as e:
logger.error(
'Workflow WebSocket connection error',
exc_info=True,
extra={
'workflow_uuid': workflow_uuid,
'session_type': quart.websocket.args.get('session_type', 'person'),
'path': quart.websocket.path,
'query_string': quart.websocket.query_string.decode('utf-8', errors='ignore'),
'remote_addr': getattr(quart.websocket, 'remote_addr', None),
},
)
try:
await quart.websocket.send(json.dumps({'type': 'error', 'message': str(e)}))
except Exception as send_error:
logger.debug(
'Failed to send error message to workflow websocket client',
exc_info=True,
extra={
'workflow_uuid': workflow_uuid,
'send_error': str(send_error),
},
)
@self.route('/messages/<session_type>', methods=['GET'])
async def get_messages(workflow_uuid: str, session_type: str) -> str:
"""获取工作流消息历史"""
try:
if session_type not in ['person', 'group']:
return self.http_status(400, -1, 'session_type must be person or group')
websocket_adapter = self.ap.platform_mgr.websocket_proxy_bot.adapter
if not websocket_adapter:
return self.http_status(404, -1, 'WebSocket adapter not found')
messages = websocket_adapter.get_websocket_messages(workflow_uuid, session_type)
return self.success(data={'messages': messages})
except Exception as e:
return self.http_status(500, -1, f'Internal server error: {str(e)}')
@self.route('/reset/<session_type>', methods=['POST'])
async def reset_session(workflow_uuid: str, session_type: str) -> str:
"""重置工作流会话"""
try:
if session_type not in ['person', 'group']:
return self.http_status(400, -1, 'session_type must be person or group')
websocket_adapter = self.ap.platform_mgr.websocket_proxy_bot.adapter
if not websocket_adapter:
return self.http_status(404, -1, 'WebSocket adapter not found')
websocket_adapter.reset_session(workflow_uuid, session_type)
return self.success(data={'message': 'Session reset successfully'})
except Exception as e:
return self.http_status(500, -1, f'Internal server error: {str(e)}')
@self.route('/connections', methods=['GET'])
async def get_connections(workflow_uuid: str) -> str:
"""获取当前工作流连接统计"""
try:
stats = ws_connection_manager.get_stats()
connections = await ws_connection_manager.get_connections_by_pipeline(workflow_uuid)
return self.success(
data={
'stats': stats,
'connections': [
{
'connection_id': conn.connection_id,
'session_type': conn.session_type,
'created_at': conn.created_at.isoformat(),
'last_active': conn.last_active.isoformat(),
'is_active': conn.is_active,
}
for conn in connections
],
}
)
except Exception as e:
return self.http_status(500, -1, f'Internal server error: {str(e)}')
@self.route('/broadcast', methods=['POST'])
async def broadcast_message(workflow_uuid: str) -> str:
"""向所有工作流连接广播消息"""
try:
data = await quart.request.get_json()
message = data.get('message')
if not message:
return self.http_status(400, -1, 'message is required')
broadcast_data = {
'type': 'broadcast',
'message': message,
'timestamp': datetime.datetime.now().isoformat(),
}
await ws_connection_manager.broadcast_to_pipeline(workflow_uuid, broadcast_data)
return self.success(data={'message': 'Broadcast sent successfully'})
except Exception as e:
return self.http_status(500, -1, f'Internal server error: {str(e)}')
async def _handle_receive(self, connection, websocket_adapter):
"""处理接收消息的任务"""
try:
while connection.is_active:
message = await quart.websocket.receive()
await ws_connection_manager.update_activity(connection.connection_id)
try:
data = json.loads(message)
message_type = data.get('type', 'message')
if message_type == 'ping':
await connection.send_queue.put(
{'type': 'pong', 'timestamp': datetime.datetime.now().isoformat()}
)
elif message_type == 'message':
logger.debug(f'收到工作流消息: {data} from {connection.connection_id}')
await websocket_adapter.handle_websocket_message(connection, data)
elif message_type == 'disconnect':
logger.debug(f'Client disconnected: {connection.connection_id}')
break
else:
logger.warning(f'Unknown message type: {message_type}')
except json.JSONDecodeError:
logger.error(f'Invalid JSON message: {message}')
await connection.send_queue.put({'type': 'error', 'message': 'Invalid JSON format'})
except Exception as e:
logger.error(f'Receive message error: {e}', exc_info=True)
finally:
connection.is_active = False
async def _handle_send(self, connection):
"""处理发送消息的任务"""
try:
while connection.is_active:
try:
message = await asyncio.wait_for(connection.send_queue.get(), timeout=1.0)
await quart.websocket.send(json.dumps(message))
except asyncio.TimeoutError:
continue
except Exception as e:
logger.error(f'Send message error: {e}', exc_info=True)
finally:
connection.is_active = False

View File

@@ -1,482 +0,0 @@
from __future__ import annotations
import quart
from ... import group
from ....service.workflow import WorkflowExecutionFailedError
@group.group_class('workflows', '/api/v1/workflows')
class WorkflowsRouterGroup(group.RouterGroup):
"""Workflow API router group"""
async def initialize(self) -> None:
# Workflow CRUD
@self.route('', methods=['GET', 'POST'], auth_type=group.AuthType.USER_TOKEN_OR_API_KEY)
async def _() -> str:
if quart.request.method == 'GET':
sort_by = quart.request.args.get('sort_by', 'created_at')
sort_order = quart.request.args.get('sort_order', 'DESC')
enabled_only = quart.request.args.get('enabled_only', 'false').lower() == 'true'
return self.success(
data={'workflows': await self.ap.workflow_service.get_workflows(sort_by, sort_order, enabled_only)}
)
elif quart.request.method == 'POST':
json_data = await quart.request.json
workflow_uuid = await self.ap.workflow_service.create_workflow(json_data)
return self.success(data={'uuid': workflow_uuid})
# Get node types (available nodes for the editor)
@self.route('/_/node-types', methods=['GET'], auth_type=group.AuthType.USER_TOKEN_OR_API_KEY)
async def _() -> str:
return self.success(
data={
'node_types': await self.ap.workflow_service.get_node_types(),
'categories': await self.ap.workflow_service.get_node_types_by_category_meta(),
}
)
# Get node types by category
@self.route('/_/node-types/categories', methods=['GET'], auth_type=group.AuthType.USER_TOKEN_OR_API_KEY)
async def _() -> str:
return self.success(data={'categories': await self.ap.workflow_service.get_node_types_by_category()})
# Single workflow operations
@self.route(
'/<workflow_uuid>', methods=['GET', 'PUT', 'DELETE'], auth_type=group.AuthType.USER_TOKEN_OR_API_KEY
)
async def _(workflow_uuid: str) -> str:
if quart.request.method == 'GET':
workflow = await self.ap.workflow_service.get_workflow(workflow_uuid)
if workflow is None:
return self.http_status(404, -1, 'workflow not found')
return self.success(data={'workflow': workflow})
elif quart.request.method == 'PUT':
json_data = await quart.request.json
try:
await self.ap.workflow_service.update_workflow(workflow_uuid, json_data)
return self.success()
except ValueError as e:
return self.http_status(404, -1, str(e))
elif quart.request.method == 'DELETE':
await self.ap.workflow_service.delete_workflow(workflow_uuid)
return self.success()
# Publish workflow (enable)
@self.route('/<workflow_uuid>/publish', methods=['POST'], auth_type=group.AuthType.USER_TOKEN_OR_API_KEY)
async def _(workflow_uuid: str) -> str:
try:
await self.ap.workflow_service.publish_workflow(workflow_uuid)
return self.success()
except ValueError as e:
return self.http_status(404, -1, str(e))
# Unpublish workflow (disable)
@self.route('/<workflow_uuid>/unpublish', methods=['POST'], auth_type=group.AuthType.USER_TOKEN_OR_API_KEY)
async def _(workflow_uuid: str) -> str:
try:
await self.ap.workflow_service.unpublish_workflow(workflow_uuid)
return self.success()
except ValueError as e:
return self.http_status(404, -1, str(e))
# Copy workflow
@self.route('/<workflow_uuid>/copy', methods=['POST'], auth_type=group.AuthType.USER_TOKEN_OR_API_KEY)
async def _(workflow_uuid: str) -> str:
try:
new_uuid = await self.ap.workflow_service.copy_workflow(workflow_uuid)
return self.success(data={'uuid': new_uuid})
except ValueError as e:
return self.http_status(404, -1, str(e))
# Execute workflow manually
@self.route('/<workflow_uuid>/execute', methods=['POST'], auth_type=group.AuthType.USER_TOKEN_OR_API_KEY)
async def _(workflow_uuid: str) -> str:
json_data = await quart.request.json or {}
trigger_data = json_data.get('trigger_data', {})
session_id = json_data.get('session_id')
user_id = json_data.get('user_id')
bot_id = json_data.get('bot_id')
try:
execution_id = await self.ap.workflow_service.execute_workflow(
workflow_uuid,
trigger_type='manual',
trigger_data=trigger_data,
session_id=session_id,
user_id=user_id,
bot_id=bot_id,
)
return self.success(data={'execution_id': execution_id})
except ValueError as e:
return self.http_status(404, -1, str(e))
except WorkflowExecutionFailedError as e:
return self.http_status(500, -1, e.message)
# Get workflow executions
@self.route('/<workflow_uuid>/executions', methods=['GET'], auth_type=group.AuthType.USER_TOKEN_OR_API_KEY)
async def _(workflow_uuid: str) -> str:
limit = int(quart.request.args.get('limit', 50))
offset = int(quart.request.args.get('offset', 0))
executions = await self.ap.workflow_service.get_executions(
workflow_uuid=workflow_uuid, limit=limit, offset=offset
)
return self.success(data=executions)
@self.route(
'/<workflow_uuid>/executions/<execution_uuid>',
methods=['GET'],
auth_type=group.AuthType.USER_TOKEN_OR_API_KEY,
)
async def _(workflow_uuid: str, execution_uuid: str) -> str:
execution = await self.ap.workflow_service.get_execution(execution_uuid)
if execution is None:
return self.http_status(404, -1, 'execution not found')
if execution.get('workflow_uuid') != workflow_uuid:
return self.http_status(404, -1, 'execution not found in workflow')
return self.success(data={'execution': execution})
# Get workflow versions
@self.route('/<workflow_uuid>/versions', methods=['GET'], auth_type=group.AuthType.USER_TOKEN_OR_API_KEY)
async def _(workflow_uuid: str) -> str:
versions = await self.ap.workflow_service.get_versions(workflow_uuid)
return self.success(data={'versions': versions})
# Rollback to a specific version
@self.route(
'/<workflow_uuid>/rollback/<int:version>', methods=['POST'], auth_type=group.AuthType.USER_TOKEN_OR_API_KEY
)
async def _(workflow_uuid: str, version: int) -> str:
try:
await self.ap.workflow_service.rollback_to_version(workflow_uuid, version)
return self.success()
except ValueError as e:
return self.http_status(404, -1, str(e))
# Workflow extensions (plugins and MCP servers)
@self.route(
'/<workflow_uuid>/extensions', methods=['GET', 'PUT'], auth_type=group.AuthType.USER_TOKEN_OR_API_KEY
)
async def _(workflow_uuid: str) -> str:
if quart.request.method == 'GET':
workflow = await self.ap.workflow_service.get_workflow(workflow_uuid)
if workflow is None:
return self.http_status(404, -1, 'workflow not found')
# Get available plugins and MCP servers
pipeline_component_kinds = ['Command', 'EventListener', 'Tool']
plugins = await self.ap.plugin_connector.list_plugins(component_kinds=pipeline_component_kinds)
mcp_servers = await self.ap.mcp_service.get_mcp_servers(contain_runtime_info=True)
extensions_prefs = workflow.get('extensions_preferences', {})
return self.success(
data={
'enable_all_plugins': extensions_prefs.get('enable_all_plugins', True),
'enable_all_mcp_servers': extensions_prefs.get('enable_all_mcp_servers', True),
'bound_plugins': extensions_prefs.get('plugins', []),
'available_plugins': plugins,
'bound_mcp_servers': extensions_prefs.get('mcp_servers', []),
'available_mcp_servers': mcp_servers,
}
)
elif quart.request.method == 'PUT':
json_data = await quart.request.json
enable_all_plugins = json_data.get('enable_all_plugins', True)
enable_all_mcp_servers = json_data.get('enable_all_mcp_servers', True)
bound_plugins = json_data.get('bound_plugins', [])
bound_mcp_servers = json_data.get('bound_mcp_servers', [])
try:
await self.ap.workflow_service.update_workflow_extensions(
workflow_uuid, bound_plugins, bound_mcp_servers, enable_all_plugins, enable_all_mcp_servers
)
return self.success()
except ValueError as e:
return self.http_status(404, -1, str(e))
# Debug API - Start debug execution
@self.route('/<workflow_uuid>/debug/start', methods=['POST'], auth_type=group.AuthType.USER_TOKEN_OR_API_KEY)
async def _(workflow_uuid: str) -> str:
json_data = await quart.request.json or {}
context = json_data.get('context', {})
variables = json_data.get('variables', {})
breakpoints = json_data.get('breakpoints', [])
try:
execution_id = await self.ap.workflow_service.start_debug_execution(
workflow_uuid, context=context, variables=variables, breakpoints=breakpoints
)
return self.success(data={'execution_id': execution_id})
except ValueError as e:
return self.http_status(404, -1, str(e))
# Debug API - Pause execution
@self.route(
'/<workflow_uuid>/debug/<execution_uuid>/pause',
methods=['POST'],
auth_type=group.AuthType.USER_TOKEN_OR_API_KEY,
)
async def _(workflow_uuid: str, execution_uuid: str) -> str:
try:
await self.ap.workflow_service.pause_debug_execution(workflow_uuid, execution_uuid)
return self.success()
except ValueError as e:
return self.http_status(404, -1, str(e))
# Debug API - Resume execution
@self.route(
'/<workflow_uuid>/debug/<execution_uuid>/resume',
methods=['POST'],
auth_type=group.AuthType.USER_TOKEN_OR_API_KEY,
)
async def _(workflow_uuid: str, execution_uuid: str) -> str:
try:
await self.ap.workflow_service.resume_debug_execution(workflow_uuid, execution_uuid)
return self.success()
except ValueError as e:
return self.http_status(404, -1, str(e))
# Debug API - Step execution
@self.route(
'/<workflow_uuid>/debug/<execution_uuid>/step',
methods=['POST'],
auth_type=group.AuthType.USER_TOKEN_OR_API_KEY,
)
async def _(workflow_uuid: str, execution_uuid: str) -> str:
try:
result = await self.ap.workflow_service.step_debug_execution(workflow_uuid, execution_uuid)
return self.success(data=result)
except ValueError as e:
return self.http_status(404, -1, str(e))
# Debug API - Stop execution
@self.route(
'/<workflow_uuid>/debug/<execution_uuid>/stop',
methods=['POST'],
auth_type=group.AuthType.USER_TOKEN_OR_API_KEY,
)
async def _(workflow_uuid: str, execution_uuid: str) -> str:
try:
await self.ap.workflow_service.stop_debug_execution(workflow_uuid, execution_uuid)
return self.success()
except ValueError as e:
return self.http_status(404, -1, str(e))
# Debug API - Get debug state
@self.route(
'/<workflow_uuid>/debug/<execution_uuid>/state',
methods=['GET'],
auth_type=group.AuthType.USER_TOKEN_OR_API_KEY,
)
async def _(workflow_uuid: str, execution_uuid: str) -> str:
try:
state = await self.ap.workflow_service.get_debug_state(workflow_uuid, execution_uuid)
return self.success(data=state)
except ValueError as e:
return self.http_status(404, -1, str(e))
# Get execution logs
@self.route(
'/<workflow_uuid>/executions/<execution_uuid>/logs',
methods=['GET'],
auth_type=group.AuthType.USER_TOKEN_OR_API_KEY,
)
async def _(workflow_uuid: str, execution_uuid: str) -> str:
limit = int(quart.request.args.get('limit', 100))
offset = int(quart.request.args.get('offset', 0))
try:
result = await self.ap.workflow_service.get_execution_logs(workflow_uuid, execution_uuid, limit, offset)
return self.success(data=result)
except ValueError as e:
return self.http_status(404, -1, str(e))
# Rerun execution
@self.route(
'/<workflow_uuid>/executions/<execution_uuid>/rerun',
methods=['POST'],
auth_type=group.AuthType.USER_TOKEN_OR_API_KEY,
)
async def _(workflow_uuid: str, execution_uuid: str) -> str:
try:
new_execution_id = await self.ap.workflow_service.rerun_execution(workflow_uuid, execution_uuid)
return self.success(data={'execution_uuid': new_execution_id})
except ValueError as e:
return self.http_status(404, -1, str(e))
# Get workflow statistics
@self.route('/<workflow_uuid>/stats', methods=['GET'], auth_type=group.AuthType.USER_TOKEN_OR_API_KEY)
async def _(workflow_uuid: str) -> str:
try:
stats = await self.ap.workflow_service.get_workflow_stats(workflow_uuid)
return self.success(data=stats)
except ValueError as e:
return self.http_status(404, -1, str(e))
# LLM Node Performance Test Endpoint
# Tests each step of LLM node execution with detailed timing
@self.route('/_/test/llm-node', methods=['POST'], auth_type=group.AuthType.USER_TOKEN_OR_API_KEY)
async def _() -> str:
"""Test LLM node performance with detailed step-by-step timing.
Request body:
{
"model_uuid": "uuid-of-model",
"system_prompt": "optional system prompt",
"user_prompt": "test message",
"temperature": 0.7,
"max_tokens": 100
}
Response includes timing for each step:
- model_fetch: Time to get model from model_mgr
- prompt_build: Time to build messages
- llm_call: Time for actual LLM invocation
- total: Total time
- usage: Token usage information
"""
import time
json_data = await quart.request.json
if not json_data:
return self.http_status(400, -1, 'Request body is required')
model_uuid = json_data.get('model_uuid', '')
if not model_uuid:
return self.http_status(400, -1, 'model_uuid is required')
user_prompt = json_data.get('user_prompt', 'test')
system_prompt = json_data.get('system_prompt', '')
temperature = json_data.get('temperature')
max_tokens = json_data.get('max_tokens', 0)
timings = {}
errors = []
# Step 1: Model fetch
t_start = time.perf_counter()
try:
runtime_model = await self.ap.model_mgr.get_model_by_uuid(model_uuid)
timings['model_fetch_ms'] = round((time.perf_counter() - t_start) * 1000, 2)
timings['model_found'] = True
timings['model_name'] = runtime_model.model_entity.name if runtime_model else None
except Exception as e:
timings['model_fetch_ms'] = round((time.perf_counter() - t_start) * 1000, 2)
timings['model_found'] = False
errors.append(f'Model fetch failed: {str(e)}')
return self.http_status(400, -1, {
'error': errors[0],
'timings': timings,
})
# Step 2: Build messages
t_start = time.perf_counter()
import langbot_plugin.api.entities.builtin.provider.message as provider_message
messages = []
if system_prompt:
messages.append(provider_message.Message(role='system', content=system_prompt))
messages.append(provider_message.Message(role='user', content=user_prompt))
timings['prompt_build_ms'] = round((time.perf_counter() - t_start) * 1000, 2)
# Step 3: Build extra args
extra_args = {}
if temperature is not None:
extra_args['temperature'] = float(temperature)
if max_tokens and int(max_tokens) > 0:
extra_args['max_tokens'] = int(max_tokens)
# Step 4: LLM call
t_start = time.perf_counter()
try:
result_message = await runtime_model.provider.invoke_llm(
query=None,
model=runtime_model,
messages=messages,
funcs=None,
extra_args=extra_args,
)
timings['llm_call_ms'] = round((time.perf_counter() - t_start) * 1000, 2)
timings['llm_call_success'] = True
# Extract response text
response_text = ''
if isinstance(result_message.content, str):
response_text = result_message.content
elif isinstance(result_message.content, list):
for elem in result_message.content:
if hasattr(elem, 'text') and elem.text:
response_text += elem.text
elif isinstance(elem, str):
response_text += elem
timings['response_length'] = len(response_text)
timings['response_preview'] = response_text[:200]
# Extract usage
usage = {'prompt_tokens': 0, 'completion_tokens': 0, 'total_tokens': 0}
if hasattr(result_message, 'usage') and result_message.usage:
u = result_message.usage
usage = {
'prompt_tokens': getattr(u, 'prompt_tokens', 0) or 0,
'completion_tokens': getattr(u, 'completion_tokens', 0) or 0,
'total_tokens': getattr(u, 'total_tokens', 0) or 0,
}
timings['usage'] = usage
except Exception as e:
timings['llm_call_ms'] = round((time.perf_counter() - t_start) * 1000, 2)
timings['llm_call_success'] = False
errors.append(f'LLM call failed: {str(e)}')
# Calculate total
timings['total_ms'] = round(sum([
timings.get('model_fetch_ms', 0),
timings.get('prompt_build_ms', 0),
timings.get('llm_call_ms', 0),
]), 2)
# Add breakdown percentage
if timings['total_ms'] > 0:
timings['breakdown'] = {
'model_fetch_pct': round(timings.get('model_fetch_ms', 0) / timings['total_ms'] * 100, 1),
'prompt_build_pct': round(timings.get('prompt_build_ms', 0) / timings['total_ms'] * 100, 1),
'llm_call_pct': round(timings.get('llm_call_ms', 0) / timings['total_ms'] * 100, 1),
}
if errors:
timings['errors'] = errors
return self.success(data={'test_result': timings})
@group.group_class('executions', '/api/v1/executions')
class ExecutionsRouterGroup(group.RouterGroup):
"""Workflow execution API router group"""
async def initialize(self) -> None:
# Get all executions (across all workflows)
@self.route('', methods=['GET'], auth_type=group.AuthType.USER_TOKEN_OR_API_KEY)
async def _() -> str:
limit = int(quart.request.args.get('limit', 50))
offset = int(quart.request.args.get('offset', 0))
status = quart.request.args.get('status')
executions = await self.ap.workflow_service.get_executions(limit=limit, offset=offset, status=status)
return self.success(data=executions)
# Get single execution
@self.route('/<execution_uuid>', methods=['GET'], auth_type=group.AuthType.USER_TOKEN_OR_API_KEY)
async def _(execution_uuid: str) -> str:
execution = await self.ap.workflow_service.get_execution(execution_uuid)
if execution is None:
return self.http_status(404, -1, 'execution not found')
return self.success(data={'execution': execution})
# Cancel execution
@self.route('/<execution_uuid>/cancel', methods=['POST'], auth_type=group.AuthType.USER_TOKEN_OR_API_KEY)
async def _(execution_uuid: str) -> str:
try:
await self.ap.workflow_service.cancel_execution(execution_uuid)
return self.success()
except ValueError as e:
return self.http_status(404, -1, str(e))
except RuntimeError as e:
return self.http_status(400, -1, str(e))

View File

@@ -17,7 +17,6 @@ from .groups import platform as groups_platform
from .groups import pipelines as groups_pipelines
from .groups import knowledge as groups_knowledge
from .groups import resources as groups_resources
from .groups import workflows as groups_workflows
importutil.import_modules_in_pkg(groups)
importutil.import_modules_in_pkg(groups_provider)
@@ -25,7 +24,6 @@ importutil.import_modules_in_pkg(groups_platform)
importutil.import_modules_in_pkg(groups_pipelines)
importutil.import_modules_in_pkg(groups_knowledge)
importutil.import_modules_in_pkg(groups_resources)
importutil.import_modules_in_pkg(groups_workflows)
class HTTPController:

View File

@@ -99,23 +99,16 @@ class BotService:
# TODO: 检查配置信息格式
bot_data['uuid'] = str(uuid.uuid4())
# Set default binding_type if not provided
if 'binding_type' not in bot_data:
bot_data['binding_type'] = 'pipeline'
# checkout the default pipeline (for backward compatibility)
# bind the most recently updated pipeline if any exist
result = await self.ap.persistence_mgr.execute_async(
sqlalchemy.select(persistence_pipeline.LegacyPipeline).where(
persistence_pipeline.LegacyPipeline.is_default == True
)
sqlalchemy.select(persistence_pipeline.LegacyPipeline)
.order_by(persistence_pipeline.LegacyPipeline.updated_at.desc())
.limit(1)
)
pipeline = result.first()
if pipeline is not None:
bot_data['use_pipeline_uuid'] = pipeline.uuid
bot_data['use_pipeline_name'] = pipeline.name
# Also set binding_uuid for new unified binding model
if 'binding_uuid' not in bot_data:
bot_data['binding_uuid'] = pipeline.uuid
await self.ap.persistence_mgr.execute_async(sqlalchemy.insert(persistence_bot.Bot).values(bot_data))
@@ -127,38 +120,26 @@ class BotService:
async def update_bot(self, bot_uuid: str, bot_data: dict) -> None:
"""Update bot"""
if 'uuid' in bot_data:
del bot_data['uuid']
update_data = bot_data.copy()
# Handle binding_type and binding_uuid for the new unified binding model
# If binding_type is explicitly set to 'workflow', skip pipeline validation
binding_type = bot_data.get('binding_type')
if 'uuid' in update_data:
del update_data['uuid']
# set use_pipeline_name (for backward compatibility with 'pipeline' binding_type)
if 'use_pipeline_uuid' in bot_data:
# set use_pipeline_name
if 'use_pipeline_uuid' in update_data:
result = await self.ap.persistence_mgr.execute_async(
sqlalchemy.select(persistence_pipeline.LegacyPipeline).where(
persistence_pipeline.LegacyPipeline.uuid == bot_data['use_pipeline_uuid']
persistence_pipeline.LegacyPipeline.uuid == update_data['use_pipeline_uuid']
)
)
pipeline = result.first()
if pipeline is not None:
bot_data['use_pipeline_name'] = pipeline.name
# Also sync to binding_uuid if binding_type is 'pipeline' or not set
if binding_type is None or binding_type == 'pipeline':
bot_data['binding_uuid'] = bot_data['use_pipeline_uuid']
bot_data['binding_type'] = 'pipeline'
update_data['use_pipeline_name'] = pipeline.name
else:
raise Exception('Pipeline not found')
# If binding_uuid is set directly (for workflow), sync use_pipeline_uuid for backward compatibility
if 'binding_uuid' in bot_data and binding_type == 'workflow':
# For workflow binding, we don't sync to use_pipeline_uuid
# but we ensure binding_type is correctly set
bot_data['binding_type'] = 'workflow'
await self.ap.persistence_mgr.execute_async(
sqlalchemy.update(persistence_bot.Bot).values(bot_data).where(persistence_bot.Bot.uuid == bot_uuid)
sqlalchemy.update(persistence_bot.Bot).values(update_data).where(persistence_bot.Bot.uuid == bot_uuid)
)
await self.ap.platform_mgr.remove_bot(bot_uuid)

View File

@@ -73,20 +73,6 @@ class PipelineService:
return self.ap.persistence_mgr.serialize_model(persistence_pipeline.LegacyPipeline, pipeline)
async def get_pipeline_by_name(self, pipeline_name: str) -> dict | None:
result = await self.ap.persistence_mgr.execute_async(
sqlalchemy.select(persistence_pipeline.LegacyPipeline).where(
persistence_pipeline.LegacyPipeline.name == pipeline_name
)
)
pipeline = result.first()
if pipeline is None:
return None
return self.ap.persistence_mgr.serialize_model(persistence_pipeline.LegacyPipeline, pipeline)
async def create_pipeline(self, pipeline_data: dict, default: bool = False) -> str:
from ....utils import paths as path_utils
@@ -229,6 +215,8 @@ class PipelineService:
bound_mcp_servers: list[str] = None,
enable_all_plugins: bool = True,
enable_all_mcp_servers: bool = True,
bound_skills: list[str] = None,
enable_all_skills: bool = True,
) -> None:
"""Update the bound plugins and MCP servers for a pipeline"""
# Get current pipeline
@@ -246,9 +234,12 @@ class PipelineService:
extensions_preferences = pipeline.extensions_preferences or {}
extensions_preferences['enable_all_plugins'] = enable_all_plugins
extensions_preferences['enable_all_mcp_servers'] = enable_all_mcp_servers
extensions_preferences['enable_all_skills'] = enable_all_skills
extensions_preferences['plugins'] = bound_plugins
if bound_mcp_servers is not None:
extensions_preferences['mcp_servers'] = bound_mcp_servers
if bound_skills is not None:
extensions_preferences['skills'] = bound_skills
await self.ap.persistence_mgr.execute_async(
sqlalchemy.update(persistence_pipeline.LegacyPipeline)

View File

@@ -0,0 +1,428 @@
from __future__ import annotations
import io
import inspect
import os
import posixpath
import zipfile
from typing import Optional
from urllib.parse import quote, unquote, urlparse
import httpx
from ....core import app
from ....skill.utils import parse_frontmatter
_PUBLIC_SKILL_FIELDS = (
'name',
'display_name',
'description',
'instructions',
'package_root',
'created_at',
'updated_at',
)
_GITHUB_ASSET_HOSTS = {
'github.com',
'api.github.com',
'objects.githubusercontent.com',
'githubusercontent.com',
'raw.githubusercontent.com',
'codeload.github.com',
}
class SkillService:
"""Filesystem-backed skill management service."""
ap: app.Application
def __init__(self, ap: app.Application) -> None:
self.ap = ap
def _box_service(self):
box_service = getattr(self.ap, 'box_service', None)
if box_service is not None and getattr(box_service, 'available', False):
return box_service
return None
def _require_box(self, action: str):
"""Return the Box service or raise if it is not available.
Box is the only source of truth for skills. Every read and write
operation goes through it — there is no local-filesystem fallback.
"""
box_service = self._box_service()
if box_service is not None:
return box_service
ap_box = getattr(self.ap, 'box_service', None)
if ap_box is None:
reason = 'not initialised'
elif not getattr(ap_box, 'enabled', True):
reason = 'disabled in config (box.enabled = false)'
else:
connector_error = getattr(ap_box, '_connector_error', '') or 'currently unavailable'
reason = f'unavailable: {connector_error}'
raise ValueError(
f'{action} requires the Box runtime, which is {reason}. '
f'Enable Box in config.yaml (box.enabled = true) and ensure the '
f'runtime is reachable before retrying.'
)
def _require_box_for_write(self, action: str) -> None:
"""Backwards-compatible alias preserved for clarity at call sites."""
self._require_box(action)
@staticmethod
def _serialize_skill(skill: dict) -> dict:
return {field: skill.get(field) for field in _PUBLIC_SKILL_FIELDS if field in skill}
async def list_skills(self) -> list[dict]:
# When Box is unavailable, surface an empty list rather than raising —
# the skills page should render cleanly, and the UI separately renders
# a "Box disabled / unavailable" banner via useBoxStatus.
box_service = self._box_service()
if box_service is None:
return []
return [self._serialize_skill(skill) for skill in await box_service.list_skills()]
async def get_skill(self, skill_name: str) -> Optional[dict]:
box_service = self._box_service()
if box_service is None:
return None
skill = await box_service.get_skill(skill_name)
return self._serialize_skill(skill) if skill else None
async def get_skill_by_name(self, name: str) -> Optional[dict]:
return await self.get_skill(name)
async def create_skill(self, data: dict) -> dict:
box_service = self._require_box('Creating a skill')
created = await box_service.create_skill(data)
await self._reload_skills()
return self._serialize_skill(created)
async def update_skill(self, skill_name: str, data: dict) -> dict:
box_service = self._require_box('Editing a skill')
updated = await box_service.update_skill(skill_name, data)
await self._reload_skills()
return self._serialize_skill(updated)
async def delete_skill(self, skill_name: str) -> bool:
box_service = self._require_box('Deleting a skill')
await box_service.delete_skill(skill_name)
await self._reload_skills()
return True
async def list_skill_files(
self,
skill_name: str,
path: str = '.',
include_hidden: bool = False,
max_entries: int = 200,
) -> dict:
box_service = self._require_box('Browsing skill files')
return await box_service.list_skill_files(skill_name, path, include_hidden, max_entries)
async def read_skill_file(self, skill_name: str, path: str) -> dict:
box_service = self._require_box('Reading a skill file')
return await box_service.read_skill_file(skill_name, path)
async def write_skill_file(self, skill_name: str, path: str, content: str) -> dict:
box_service = self._require_box('Editing skill files')
result = await box_service.write_skill_file(skill_name, path, content)
await self._reload_skills()
return result
async def install_from_github(self, data: dict) -> list[dict]:
box_service = self._require_box('Installing a skill from GitHub')
owner = str(data['owner']).strip()
repo = str(data['repo']).strip()
release_tag = str(data.get('release_tag', '')).strip()
raw_asset_url = str(data['asset_url']).strip()
if self._is_github_skill_md_url(raw_asset_url):
return await self._install_github_skill_md(raw_asset_url, owner=owner, repo=repo, data=data)
asset_url = self._validate_github_asset_url(raw_asset_url, owner=owner, repo=repo, release_tag=release_tag)
source_subdir = str(data.get('source_subdir', '') or '').strip()
zip_bytes = await self._download_github_asset(asset_url)
filename = f'{repo}-{release_tag.lstrip("v").replace("/", "-") or "source"}.zip'
installed = await box_service.install_skill_zip(
zip_bytes,
filename,
source_paths=data.get('source_paths') or [],
source_path=str(data.get('source_path', '') or ''),
source_subdir=source_subdir,
)
await self._reload_skills()
return [self._serialize_skill(skill) for skill in installed]
async def preview_install_from_github(self, data: dict) -> list[dict]:
box_service = self._require_box('Previewing a skill from GitHub')
owner = str(data['owner']).strip()
repo = str(data['repo']).strip()
release_tag = str(data.get('release_tag', '')).strip()
raw_asset_url = str(data['asset_url']).strip()
if self._is_github_skill_md_url(raw_asset_url):
return await self._preview_github_skill_md(raw_asset_url, owner=owner, repo=repo)
asset_url = self._validate_github_asset_url(raw_asset_url, owner=owner, repo=repo, release_tag=release_tag)
source_subdir = str(data.get('source_subdir', '') or '').strip()
zip_bytes = await self._download_github_asset(asset_url)
return await box_service.preview_skill_zip(
zip_bytes,
f'{repo}-{release_tag.lstrip("v").replace("/", "-") or "source"}.zip',
source_subdir=source_subdir,
)
async def install_from_zip_upload(
self,
*,
file_bytes: bytes,
filename: str,
source_paths: list[str] | None = None,
source_path: str = '',
) -> list[dict]:
box_service = self._require_box('Installing a skill from upload')
installed = await box_service.install_skill_zip(
file_bytes,
filename,
source_paths=source_paths or [],
source_path=source_path,
)
await self._reload_skills()
return [self._serialize_skill(skill) for skill in installed]
async def preview_install_from_zip_upload(self, *, file_bytes: bytes, filename: str) -> list[dict]:
box_service = self._require_box('Previewing a skill upload')
return await box_service.preview_skill_zip(file_bytes, filename)
async def _install_github_skill_md(self, asset_url: str, *, owner: str, repo: str, data: dict) -> list[dict]:
box_service = self._require_box('Installing a skill from GitHub')
zip_bytes, filename, _package_name = await self._download_github_skill_directory_as_zip(
asset_url,
owner=owner,
repo=repo,
)
installed = await box_service.install_skill_zip(
zip_bytes,
filename,
source_paths=data.get('source_paths') or [],
source_path=str(data.get('source_path', '') or ''),
target_suffix='',
)
await self._reload_skills()
return [self._serialize_skill(skill) for skill in installed]
async def _preview_github_skill_md(self, asset_url: str, *, owner: str, repo: str) -> list[dict]:
box_service = self._require_box('Previewing a skill from GitHub')
zip_bytes, _filename, package_name = await self._download_github_skill_directory_as_zip(
asset_url,
owner=owner,
repo=repo,
)
return await box_service.preview_skill_zip(zip_bytes, f'{package_name}.zip', target_suffix='')
async def reload_skills(self) -> list[dict]:
await self._reload_skills()
return await self.list_skills()
async def scan_directory_async(self, path: str) -> dict:
box_service = self._require_box('Scanning a skill directory')
return await box_service.scan_skill_directory(path)
async def _reload_skills(self) -> None:
skill_mgr = getattr(self.ap, 'skill_mgr', None)
reload_skills = getattr(skill_mgr, 'reload_skills', None)
if not callable(reload_skills):
return
result = reload_skills()
if inspect.isawaitable(result):
await result
async def _download_github_asset(self, asset_url: str) -> bytes:
async with httpx.AsyncClient(follow_redirects=True, timeout=120) as client:
resp = await client.get(asset_url)
resp.raise_for_status()
return resp.content
async def _download_github_skill_directory_as_zip(
self, asset_url: str, *, owner: str, repo: str
) -> tuple[bytes, str, str]:
info = self._parse_github_skill_md_url(asset_url, owner=owner, repo=repo)
archive_url = f'https://codeload.github.com/{owner}/{repo}/zip/{quote(info["ref"], safe="/")}'
archive_bytes = await self._download_github_asset(archive_url)
try:
source_archive = zipfile.ZipFile(io.BytesIO(archive_bytes), 'r')
except zipfile.BadZipFile as exc:
raise ValueError('GitHub repository archive must be a valid .zip archive') from exc
with source_archive as source_zip:
skill_entry = self._find_github_skill_archive_entry(source_zip, info['file_path'])
try:
skill_md_content = source_zip.read(skill_entry).decode('utf-8')
except UnicodeDecodeError as exc:
raise ValueError('GitHub SKILL.md must be valid UTF-8 text') from exc
package_name = self._resolve_github_skill_md_package_name(skill_md_content, info['package_name'])
source_skill_dir = posixpath.dirname(posixpath.normpath(skill_entry.filename))
buffer = io.BytesIO()
with zipfile.ZipFile(buffer, 'w', zipfile.ZIP_DEFLATED) as target_zip:
self._copy_github_skill_directory_to_zip(source_zip, target_zip, source_skill_dir, package_name)
return buffer.getvalue(), f'{package_name}.zip', package_name
def _find_github_skill_archive_entry(self, archive: zipfile.ZipFile, file_path: str) -> zipfile.ZipInfo:
normalized_file_path = posixpath.normpath(file_path).lower()
for member in archive.infolist():
if member.is_dir():
continue
normalized_member = posixpath.normpath(member.filename)
path_parts = normalized_member.split('/', 1)
if len(path_parts) != 2:
continue
archive_relative_path = path_parts[1].lower()
if archive_relative_path == normalized_file_path:
return member
raise ValueError(f'GitHub archive does not contain requested SKILL.md: {file_path}')
def _copy_github_skill_directory_to_zip(
self,
source_zip: zipfile.ZipFile,
target_zip: zipfile.ZipFile,
source_skill_dir: str,
package_name: str,
) -> None:
normalized_source_dir = posixpath.normpath(source_skill_dir)
source_prefix = f'{normalized_source_dir}/'
copied_files = 0
for member in source_zip.infolist():
normalized_member = posixpath.normpath(member.filename)
if normalized_member != normalized_source_dir and not normalized_member.startswith(source_prefix):
continue
relative_path = posixpath.relpath(normalized_member, normalized_source_dir)
if relative_path in ('', '.'):
continue
if relative_path.startswith('../') or relative_path == '..' or posixpath.isabs(relative_path):
raise ValueError(f'GitHub archive contains an unsafe skill path: {member.filename}')
target_name = f'{package_name}/{relative_path}'
if member.is_dir() and not target_name.endswith('/'):
target_name = f'{target_name}/'
target_info = zipfile.ZipInfo(target_name, date_time=member.date_time)
target_info.external_attr = member.external_attr
target_info.compress_type = zipfile.ZIP_DEFLATED
if member.is_dir():
target_zip.writestr(target_info, b'')
continue
target_zip.writestr(target_info, source_zip.read(member))
copied_files += 1
if copied_files == 0:
raise ValueError('GitHub skill directory is empty')
def _uploaded_skill_target_stem(self, filename: str) -> str:
stem = os.path.splitext(os.path.basename(str(filename or '').strip()))[0]
safe_stem = ''.join(ch if ch.isalnum() or ch in ('-', '_') else '-' for ch in stem).strip('-_')
if not safe_stem:
safe_stem = 'uploaded-skill'
return safe_stem
@staticmethod
def _is_github_skill_md_url(asset_url: str) -> bool:
parsed = urlparse(str(asset_url or '').strip())
normalized_path = posixpath.normpath(parsed.path or '/')
return normalized_path.lower().endswith('/skill.md')
def _parse_github_skill_md_url(self, asset_url: str, *, owner: str, repo: str) -> dict:
parsed = urlparse(str(asset_url or '').strip())
if parsed.scheme != 'https' or not parsed.netloc:
raise ValueError('asset_url must be a valid HTTPS GitHub SKILL.md URL')
host = parsed.netloc.lower()
path_parts = [unquote(part) for part in (parsed.path or '').split('/') if part]
if host == 'github.com':
if (
len(path_parts) < 5
or path_parts[0] != owner
or path_parts[1] != repo
or path_parts[2]
not in (
'blob',
'raw',
)
):
raise ValueError('GitHub SKILL.md URL must point to the requested owner/repo blob path')
ref = path_parts[3]
file_path = '/'.join(path_parts[4:])
elif host == 'raw.githubusercontent.com':
if len(path_parts) < 4 or path_parts[0] != owner or path_parts[1] != repo:
raise ValueError('GitHub SKILL.md URL must point to the requested owner/repo raw path')
ref = path_parts[2]
file_path = '/'.join(path_parts[3:])
else:
raise ValueError('asset_url must point to a GitHub SKILL.md file')
normalized_file_path = posixpath.normpath(file_path)
normalized_file_path_lower = normalized_file_path.lower()
if normalized_file_path_lower != 'skill.md' and not normalized_file_path_lower.endswith('/skill.md'):
raise ValueError('GitHub skill import requires a URL ending with SKILL.md')
parent_dir = posixpath.basename(posixpath.dirname(normalized_file_path)) or repo
return {
'ref': ref,
'file_path': normalized_file_path,
'package_name': self._uploaded_skill_target_stem(parent_dir),
}
def _resolve_github_skill_md_package_name(self, content: str, fallback: str) -> str:
metadata, _instructions = parse_frontmatter(content)
candidate = str(metadata.get('name') or fallback or '').strip()
try:
return self._validate_skill_name(candidate)
except ValueError:
return self._validate_skill_name(fallback)
@staticmethod
def _validate_github_asset_url(asset_url: str, *, owner: str, repo: str, release_tag: str) -> str:
parsed = urlparse(str(asset_url).strip())
if parsed.scheme != 'https' or not parsed.netloc:
raise ValueError('asset_url must be a valid HTTPS GitHub asset URL')
host = parsed.netloc.lower()
if host not in _GITHUB_ASSET_HOSTS:
raise ValueError('asset_url must point to a GitHub-hosted release asset or archive')
normalized_path = posixpath.normpath(parsed.path or '/')
allowed_prefixes = [
f'/repos/{owner}/{repo}/',
f'/{owner}/{repo}/',
]
if not any(normalized_path.startswith(prefix) for prefix in allowed_prefixes):
raise ValueError('asset_url does not match the requested owner/repo')
if release_tag and release_tag not in parsed.path and release_tag not in parsed.query:
raise ValueError('asset_url does not match the requested release_tag')
return parsed.geturl()
@staticmethod
def _validate_skill_name(name: str) -> str:
name = str(name or '').strip()
if not name:
raise ValueError('Skill name is required')
if not name.replace('-', '').replace('_', '').isalnum():
raise ValueError('Skill name can only contain letters, numbers, hyphens and underscores')
if len(name) > 64:
raise ValueError('Skill name cannot exceed 64 characters')
return name

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,5 @@
"""LangBot Box runtime package."""
from .workspace import BoxWorkspaceSession
__all__ = ['BoxWorkspaceSession']

View File

@@ -0,0 +1,354 @@
from __future__ import annotations
import asyncio
import json
import os
import sys
import typing
from typing import TYPE_CHECKING
from urllib.parse import urlparse
from langbot_plugin.entities.io.actions.enums import CommonAction
from langbot_plugin.runtime.io.handler import Handler
from langbot_plugin.runtime.io.connection import Connection
from langbot_plugin.box.client import ActionRPCBoxClient
from langbot_plugin.box.errors import BoxRuntimeUnavailableError
from langbot_plugin.box.actions import LangBotToBoxAction
from ..utils import platform
from ..utils.managed_runtime import ManagedRuntimeConnector
if TYPE_CHECKING:
from ..core import app as core_app
# Default Docker Compose service name for the standalone Box container.
_DOCKER_BOX_HOST = 'langbot_box'
_DEFAULT_PORT = 5410
_HEARTBEAT_INTERVAL_SEC = 20
# Top-level keys under ``box`` that are LangBot-internal and should not be
# forwarded to the Box runtime.
_INTERNAL_BOX_CONFIG_KEYS = frozenset({'runtime'})
def _get_box_config(ap) -> dict:
"""Return the 'box' section from instance config.
Environment-variable overrides are handled uniformly by
``LoadConfigStage._apply_env_overrides_to_config`` using the
``SECTION__SUBSECTION__KEY`` convention (e.g. ``BOX__LOCAL__HOST_ROOT``,
``BOX__LOCAL__ALLOWED_MOUNT_ROOTS="/a,/b"``) before this is read, so no
box-specific env parsing is needed here.
"""
instance_config = getattr(ap, 'instance_config', None)
config_data = getattr(instance_config, 'data', {}) if instance_config is not None else {}
return dict(config_data.get('box', {}) or {})
def _get_runtime_endpoint(box_cfg: dict) -> str:
runtime_cfg = box_cfg.get('runtime') or {}
return str(runtime_cfg.get('endpoint', '')).strip()
def _filter_config_for_runtime(box_cfg: dict) -> dict:
return {k: v for k, v in box_cfg.items() if k not in _INTERNAL_BOX_CONFIG_KEYS}
def resolve_box_ws_relay_url(ap: core_app.Application) -> str:
"""Derive the WS relay base URL used for managed-process attach.
The WS relay serves the ``/v1/sessions/{id}/managed-process/ws`` endpoint
on the *relay* port (default 5410).
"""
box_cfg = _get_box_config(ap)
# Explicit runtime endpoint takes precedence. The config value is a base
# URL; endpoint-specific paths are appended by the SDK client.
endpoint = _get_runtime_endpoint(box_cfg)
if endpoint:
parsed = urlparse(endpoint)
scheme = parsed.scheme or 'ws'
if scheme == 'ws':
scheme = 'http'
elif scheme == 'wss':
scheme = 'https'
host = parsed.hostname or '127.0.0.1'
port = parsed.port or _DEFAULT_PORT
return f'{scheme}://{host}:{port}'
# In Docker, relay lives on the box runtime container.
if platform.get_platform() == 'docker':
return f'http://{_DOCKER_BOX_HOST}:{_DEFAULT_PORT}'
return f'http://127.0.0.1:{_DEFAULT_PORT}'
class BoxRuntimeConnector(ManagedRuntimeConnector):
"""Connect to the Box runtime via action RPC.
Transport decision (mirrors Plugin runtime logic):
1. Docker / --standalone-box / explicit runtime.endpoint -> WebSocket to external Box process
2. Windows (non-Docker) -> subprocess + WebSocket (Windows lacks async stdio pipe)
3. Unix / macOS -> subprocess + stdio pipe
"""
def __init__(
self,
ap: core_app.Application,
runtime_disconnect_callback: typing.Callable[
['BoxRuntimeConnector'], typing.Coroutine[typing.Any, typing.Any, None]
]
| None = None,
):
super().__init__(ap)
self.runtime_disconnect_callback = runtime_disconnect_callback
self.configured_runtime_endpoint = self._load_configured_runtime_endpoint()
self.ws_relay_base_url = resolve_box_ws_relay_url(ap)
self.client = ActionRPCBoxClient(logger=ap.logger)
self._handler: Handler | None = None
self._handler_task: asyncio.Task | None = None
self._ctrl_task: asyncio.Task | None = None
self._heartbeat_task: asyncio.Task | None = None
# Parse the relay URL once for reuse.
parsed = urlparse(self.ws_relay_base_url)
self._relay_host = parsed.hostname or '127.0.0.1'
self._relay_port = parsed.port or _DEFAULT_PORT
self._filtered_box_config = _filter_config_for_runtime(_get_box_config(ap))
def _uses_websocket(self) -> bool:
"""Whether the connector should use WebSocket to reach the Box runtime.
True when:
- Running inside Docker (Box runtime is a separate container)
- The ``--standalone-box`` CLI flag was passed
- An explicit ``runtime.endpoint`` was configured
"""
return bool(
self.configured_runtime_endpoint
or platform.get_platform() == 'docker'
or platform.use_websocket_to_connect_box_runtime()
)
async def initialize(self) -> None:
if self._uses_websocket():
if platform.get_platform() == 'win32' and not self.configured_runtime_endpoint:
await self._start_subprocess_then_ws()
else:
await self._connect_remote_ws()
else:
await self._start_local_stdio()
# Start heartbeat after successful connection
if self._heartbeat_task is None:
self._heartbeat_task = asyncio.create_task(self._heartbeat_loop())
# -- heartbeat -----------------------------------------------------------
async def _heartbeat_loop(self) -> None:
"""Periodically ping the Box runtime to detect silent disconnections."""
while True:
await asyncio.sleep(_HEARTBEAT_INTERVAL_SEC)
try:
await self.ping()
self.ap.logger.debug('Heartbeat to Box runtime success.')
except Exception as e:
self.ap.logger.debug(f'Failed to heartbeat to Box runtime: {e}')
async def ping(self) -> None:
if self._handler is None:
raise BoxRuntimeUnavailableError('Box runtime is not connected')
await self._handler.call_action(CommonAction.PING, {})
# -- transport paths -----------------------------------------------------
async def _start_local_stdio(self) -> None:
"""Launch box server as subprocess and connect via stdio (Unix/macOS)."""
from langbot_plugin.runtime.io.controllers.stdio.client import StdioClientController
self.ap.logger.info('Use stdio to connect to box runtime')
python_path = sys.executable
env = os.environ.copy()
if self._filtered_box_config:
env['LANGBOT_BOX_CONFIG'] = json.dumps(self._filtered_box_config)
connected = asyncio.Event()
connect_error: list[Exception] = []
ctrl = StdioClientController(
command=python_path,
# Launched through the same CLI entry point as the plugin runtime
# (cli.__init__ <subcommand>); `-s` selects the stdio transport,
# mirroring `rt -s`.
args=['-m', 'langbot_plugin.cli.__init__', 'box', '-s', '--ws-control-port', str(self._relay_port)],
env=env,
)
self._ctrl_task = asyncio.create_task(
ctrl.run(self._make_connection_callback('stdio', connected, connect_error))
)
try:
await asyncio.wait_for(connected.wait(), timeout=30.0)
except asyncio.TimeoutError:
raise BoxRuntimeUnavailableError('box runtime subprocess did not connect in time')
if connect_error:
raise BoxRuntimeUnavailableError(f'box runtime connection failed: {connect_error[0]}')
self._subprocess = ctrl.process
async def _start_subprocess_then_ws(self) -> None:
"""Launch box server as detached subprocess, then connect via WS (Windows)."""
self.ap.logger.info('(windows) Use cmd to launch box runtime and communicate via ws')
env = os.environ.copy()
if self._filtered_box_config:
env['LANGBOT_BOX_CONFIG'] = json.dumps(self._filtered_box_config)
python_path = sys.executable
# Launched through the same CLI entry point as the plugin runtime
# (cli.__init__ <subcommand>); no flag => WebSocket transport.
self.runtime_subprocess = await asyncio.create_subprocess_exec(
python_path,
'-m',
'langbot_plugin.cli.__init__',
'box',
'--ws-control-port',
str(self._relay_port),
env=env,
)
self.runtime_subprocess_task = asyncio.create_task(self.runtime_subprocess.wait())
ws_url = f'ws://localhost:{self._relay_port}/rpc/ws'
await self._connect_ws(ws_url, '(windows) WebSocket')
async def _connect_remote_ws(self) -> None:
"""Connect to a remote (or Docker) box server via WebSocket."""
ws_url = self._resolve_rpc_ws_url()
self.ap.logger.info(f'Use WebSocket to connect to box runtime ({ws_url})')
await self._connect_ws(ws_url, 'WebSocket')
# -- helpers -------------------------------------------------------------
def _resolve_rpc_ws_url(self) -> str:
"""Determine the action-RPC WebSocket URL.
All endpoints share a single port; action RPC is at ``/rpc/ws``.
"""
if self.configured_runtime_endpoint:
base = self.configured_runtime_endpoint.rstrip('/')
parsed = urlparse(base)
scheme = parsed.scheme or 'ws'
if scheme in ('http', 'https'):
scheme = 'wss' if scheme == 'https' else 'ws'
host = parsed.hostname or '127.0.0.1'
port = parsed.port or _DEFAULT_PORT
return f'{scheme}://{host}:{port}/rpc/ws'
if platform.get_platform() == 'docker':
return f'ws://{_DOCKER_BOX_HOST}:{_DEFAULT_PORT}/rpc/ws'
return f'ws://localhost:{self._relay_port}/rpc/ws'
async def _connect_ws(self, ws_url: str, transport_name: str) -> None:
"""Shared WebSocket connection procedure."""
from langbot_plugin.runtime.io.controllers.ws.client import WebSocketClientController
connected = asyncio.Event()
connect_error: list[Exception] = []
async def on_connect_failed(ctrl, exc):
if exc is not None:
self.ap.logger.error(f'Failed to connect to Box runtime ({ws_url}): {exc}')
else:
self.ap.logger.error(f'Failed to connect to Box runtime ({ws_url}), trying to reconnect...')
connect_error.append(exc or BoxRuntimeUnavailableError('ws connection failed'))
connected.set()
if self.runtime_disconnect_callback is not None:
await self.runtime_disconnect_callback(self)
ctrl = WebSocketClientController(ws_url=ws_url, make_connection_failed_callback=on_connect_failed)
self._ctrl_task = asyncio.create_task(
ctrl.run(self._make_connection_callback(transport_name, connected, connect_error))
)
try:
await asyncio.wait_for(connected.wait(), timeout=30.0)
except asyncio.TimeoutError:
raise BoxRuntimeUnavailableError(f'box runtime ws connection timed out ({ws_url})')
if connect_error:
raise BoxRuntimeUnavailableError(f'box runtime connection failed: {connect_error[0]}')
def _make_connection_callback(
self,
transport_name: str,
connected: asyncio.Event,
connect_error: list[Exception],
):
async def new_connection_callback(connection: Connection) -> None:
handler = Handler(connection)
self._handler = handler
self.client.set_handler(handler)
self._handler_task = asyncio.create_task(handler.run())
try:
await handler.call_action(CommonAction.PING, {})
if self._filtered_box_config:
await handler.call_action(LangBotToBoxAction.INIT, self._filtered_box_config)
self.ap.logger.debug('Sent box configuration to Box runtime via INIT.')
self.ap.logger.info(f'Connected to Box runtime via {transport_name}.')
connected.set()
await self._handler_task
except Exception as exc:
if not connected.is_set():
connect_error.append(exc)
connected.set()
return
# If we reach here, handler.run() returned normally (connection
# closed) or raised after the initial handshake succeeded.
# Either way, treat it as a disconnect.
if connected.is_set():
if self._uses_websocket():
self.ap.logger.error('Disconnected from Box runtime, trying to reconnect...')
if self.runtime_disconnect_callback is not None:
await self.runtime_disconnect_callback(self)
else:
self.ap.logger.error(
'Disconnected from Box runtime via stdio. '
'Cannot automatically reconnect — please restart LangBot.'
)
return new_connection_callback
# -- lifecycle -----------------------------------------------------------
def dispose(self) -> None:
if self._heartbeat_task is not None:
self._heartbeat_task.cancel()
self._heartbeat_task = None
if self._handler_task is not None:
self._handler_task.cancel()
self._handler_task = None
if self._ctrl_task is not None:
self._ctrl_task.cancel()
self._ctrl_task = None
# stdio-managed subprocess (stored as self._subprocess by _start_local_stdio)
if hasattr(self, '_subprocess') and self._subprocess is not None and self._subprocess.returncode is None:
self.ap.logger.info('Terminating managed box runtime process...')
self._subprocess.terminate()
# Subprocess launched by ManagedRuntimeConnector._start_runtime_subprocess (Windows path)
self._dispose_subprocess()
# -- config helpers ------------------------------------------------------
def _load_configured_runtime_endpoint(self) -> str:
return _get_runtime_endpoint(_get_box_config(self.ap))

View File

@@ -0,0 +1,98 @@
"""Three-layer security policy for LangBot Box.
The design separates concerns into three independent layers, aligned with
OpenCode / OpenClaw patterns:
1. **SandboxPolicy** *where* tools run (host vs sandbox).
2. **ToolPolicy** *which* tools are allowed (allow/deny lists).
3. **ElevatedPolicy** *whether* a single exec call may temporarily
escape the default sandbox boundary.
These three layers are orthogonal:
- ToolPolicy is a hard boundary; ``elevated`` cannot bypass a denied tool.
- SandboxPolicy decides the default execution location.
- ElevatedPolicy only affects ``exec`` and only when the framework allows it.
"""
from __future__ import annotations
import enum
from typing import Sequence
# ── Layer 1: Sandbox Policy ──────────────────────────────────────────
class SandboxMode(str, enum.Enum):
"""Determines when agent execution is routed through the sandbox."""
OFF = 'off'
"""Sandbox disabled; all exec runs on the host."""
NON_DEFAULT = 'non_default'
"""Only non-default sessions are sandboxed (e.g. sub-agents, MCP)."""
ALL = 'all'
"""Every agent exec call is routed through the sandbox."""
class SandboxPolicy:
"""Decides whether a given execution context should use the sandbox."""
def __init__(self, mode: SandboxMode = SandboxMode.ALL):
self.mode = mode
def should_sandbox(self, *, is_default_session: bool = True) -> bool:
if self.mode == SandboxMode.OFF:
return False
if self.mode == SandboxMode.ALL:
return True
# NON_DEFAULT: sandbox everything except the default session
return not is_default_session
# ── Layer 2: Tool Policy ─────────────────────────────────────────────
class ToolPolicy:
"""Controls which tools are available to the current agent/session.
Rules:
- ``deny`` always takes precedence over ``allow``.
- An empty ``allow`` list means "all tools allowed" (no allowlist filter).
- ``elevated`` cannot bypass a denied tool.
"""
def __init__(
self,
allow: Sequence[str] = (),
deny: Sequence[str] = (),
):
self._allow: frozenset[str] = frozenset(allow)
self._deny: frozenset[str] = frozenset(deny)
def is_tool_allowed(self, tool_name: str) -> bool:
if tool_name in self._deny:
return False
if self._allow and tool_name not in self._allow:
return False
return True
# ── Layer 3: Elevated Policy ─────────────────────────────────────────
class ElevatedPolicy:
"""Controls whether ``exec`` may request temporary privilege escalation.
``elevated`` only applies to the ``exec`` tool. It means "run this
command outside the default sandbox boundary" (e.g. with network, or
on the host). The framework decides whether to honor the request.
"""
def __init__(self, *, allow_elevated: bool = False, require_approval: bool = True):
self.allow_elevated = allow_elevated
self.require_approval = require_approval
def is_elevation_permitted(self) -> bool:
return self.allow_elevated

View File

@@ -0,0 +1,794 @@
from __future__ import annotations
import asyncio
import collections
import datetime as _dt
import enum
import json
import os
from typing import TYPE_CHECKING
import pydantic
from langbot_plugin.box.client import BoxRuntimeClient
from .connector import BoxRuntimeConnector, _get_box_config
from langbot_plugin.box.errors import BoxError, BoxValidationError
from langbot_plugin.box.models import (
BUILTIN_PROFILES,
BoxExecutionResult,
BoxManagedProcessInfo,
BoxManagedProcessSpec,
BoxProfile,
BoxSpec,
)
_INT_ADAPTER = pydantic.TypeAdapter(int)
_UTC = _dt.timezone.utc
_MAX_RECENT_ERRORS = 50
_MIB = 1024 * 1024
def _is_path_under(path: str, root: str) -> bool:
"""Check whether *path* equals *root* or is a child of *root*."""
return path == root or path.startswith(f'{root}{os.sep}')
if TYPE_CHECKING:
from ..core import app as core_app
import langbot_plugin.api.entities.builtin.pipeline.query as pipeline_query
class BoxService:
def __init__(
self,
ap: core_app.Application,
client: BoxRuntimeClient | None = None,
output_limit_chars: int = 4000,
):
self.ap = ap
self._enabled = self._load_enabled()
self._runtime_connector: BoxRuntimeConnector | None = None
if client is None:
# Always construct a connector — its __init__ is side-effect free
# (no I/O, no subprocess). When ``box.enabled = false`` we simply
# skip ``connector.initialize()`` so no connection is attempted.
self._runtime_connector = BoxRuntimeConnector(ap, runtime_disconnect_callback=self._on_runtime_disconnect)
client = self._runtime_connector.client
self.client = client
self.output_limit_chars = output_limit_chars
self.host_root = self._load_host_root()
self.allowed_mount_roots = self._load_allowed_mount_roots()
self.default_workspace = self._load_default_workspace()
self.profile = self._load_profile()
self.custom_image = self._load_custom_image()
self.workspace_quota_mb = self._load_workspace_quota_mb()
self._recent_errors: collections.deque[dict] = collections.deque(maxlen=_MAX_RECENT_ERRORS)
self._shutdown_task = None
self._available = False
self._connector_error: str = ''
self._reconnecting = False
@property
def enabled(self) -> bool:
"""Whether Box is enabled in config. False means the operator has
deliberately turned the sandbox off via ``box.enabled = false``.
Disabled and "enabled but unavailable" are reported as the same
``available = False`` to consumers, but distinguished in get_status."""
return self._enabled
async def initialize(self):
self._ensure_default_workspace()
if not self._enabled:
# Disabled by config: do NOT connect to a remote runtime, do NOT
# fork a stdio subprocess. Every consumer of box_service should
# gate on ``available`` and degrade gracefully.
self._available = False
self._connector_error = 'Box runtime is disabled in config (box.enabled = false)'
self.ap.logger.info(
'Box runtime disabled by config; sandbox features (exec/read/write/edit, '
'skill add/edit, stdio MCP) will be unavailable.'
)
return
try:
if self._runtime_connector is not None:
await self._runtime_connector.initialize()
else:
await self.client.initialize()
self._available = True
self._connector_error = ''
self.ap.logger.info(
f'LangBot Box runtime initialized: profile={self.profile.name} '
f'default_workspace={self.default_workspace or "(none)"}'
)
except Exception as exc:
self.ap.logger.warning(f'LangBot Box runtime unavailable, sandbox features disabled: {exc}')
self._available = False
self._connector_error = str(exc)
async def _on_runtime_disconnect(self, connector: BoxRuntimeConnector) -> None:
"""Called by the connector when the Box runtime connection drops.
Spawns a background reconnection loop so the caller is not blocked.
Skipped entirely when Box is disabled by config — that path should
never have connected in the first place.
"""
if not self._enabled:
return
if self._reconnecting:
return # Another reconnect loop is already running
self._reconnecting = True
self._available = False
self._connector_error = 'Disconnected from Box runtime'
self.ap.logger.warning('Box runtime disconnected, sandbox features temporarily disabled.')
asyncio.create_task(self._reconnect_loop(connector))
async def _reconnect_loop(self, connector: BoxRuntimeConnector) -> None:
"""Retry reconnection with exponential backoff (3s → 60s max)."""
delay = 3
max_delay = 60
try:
while True:
self.ap.logger.info(f'Attempting to reconnect to Box runtime in {delay}s...')
await asyncio.sleep(delay)
try:
connector.dispose()
await connector.initialize()
self._available = True
self._connector_error = ''
self.ap.logger.info('Box runtime reconnected, sandbox features restored.')
return
except Exception as exc:
self._connector_error = str(exc)
self.ap.logger.warning(f'Box runtime reconnection failed: {exc}')
delay = min(delay * 2, max_delay)
finally:
self._reconnecting = False
@property
def available(self) -> bool:
return self._available
async def execute_spec_payload(
self,
spec_payload: dict,
query: pipeline_query.Query,
*,
skip_host_mount_validation: bool = False,
) -> dict:
if not self._available:
raise BoxError('Box runtime is not available. Install and start Docker to use sandbox features.')
try:
spec = self.build_spec(spec_payload, skip_host_mount_validation=skip_host_mount_validation)
except BoxError as exc:
self._record_error(exc, query)
raise
self.ap.logger.info(
'LangBot Box request: '
f'query_id={query.query_id} '
f'spec={json.dumps(self._summarize_spec(spec), ensure_ascii=False)}'
)
try:
self._enforce_workspace_quota(spec, phase='before execution')
except BoxError as exc:
self._record_error(exc, query)
raise
try:
result = await self.client.execute(spec)
except BoxError as exc:
self._record_error(exc, query)
raise
try:
self._enforce_workspace_quota(spec, phase='after execution')
except BoxError as exc:
await self._cleanup_exceeded_session(spec)
self._record_error(exc, query)
raise
self.ap.logger.info(
'LangBot Box result: '
f'query_id={query.query_id} '
f'summary={json.dumps(self._summarize_result(result), ensure_ascii=False)}'
)
return self._serialize_result(result)
def resolve_box_session_id(self, query: pipeline_query.Query) -> str:
"""Resolve the Box session_id from the pipeline's template and query variables."""
template = (
(query.pipeline_config or {})
.get('ai', {})
.get('local-agent', {})
.get('box-session-id-template', '{launcher_type}_{launcher_id}')
)
variables = dict(query.variables or {})
launcher_type = getattr(query, 'launcher_type', None)
if hasattr(launcher_type, 'value'):
launcher_type = launcher_type.value
launcher_id = getattr(query, 'launcher_id', None)
sender_id = getattr(query, 'sender_id', None)
query_id = getattr(query, 'query_id', None)
variables.setdefault('query_id', str(query_id or 'unknown'))
variables.setdefault('launcher_type', str(launcher_type or 'query'))
variables.setdefault('launcher_id', str(launcher_id or query_id or 'unknown'))
variables.setdefault('sender_id', str(sender_id or launcher_id or query_id or 'unknown'))
variables.setdefault('global', 'global')
return template.format_map(collections.defaultdict(lambda: 'unknown', variables))
def build_skill_extra_mounts(self, query: pipeline_query.Query) -> list[dict]:
"""Build extra_mounts entries for all pipeline-bound skills.
This ensures that when a container is first created it already has
all skill packages mounted, regardless of which skill is currently
activated.
Skills whose ``package_root`` is missing or no longer a directory on
the LangBot-visible filesystem are skipped with a warning instead of
being passed through to the backend. Without this guard the three
backends behave inconsistently on a stale mount: nsjail refuses to
start the sandbox (failing every exec in the session), Docker
silently auto-creates a root-owned empty directory on the host, and
E2B silently skips the upload — none of which surfaces an
actionable error to the agent or operator.
"""
skill_mgr = getattr(self.ap, 'skill_mgr', None)
if skill_mgr is None:
return []
from ..provider.tools.loaders import skill as skill_loader
visible_skills = skill_loader.get_visible_skills(self.ap, query)
mounts: list[dict] = []
for skill_name, skill_data in visible_skills.items():
package_root = str(skill_data.get('package_root', '') or '').strip()
if not package_root:
continue
if not os.path.isdir(package_root):
self.ap.logger.warning(
f'Skill "{skill_name}" package_root missing on filesystem '
f'({package_root}); skipping mount to prevent sandbox failures. '
f'The skill cache may be stale — consider reloading skills.'
)
continue
mounts.append(
{
'host_path': package_root,
'mount_path': f'/workspace/.skills/{skill_name}',
'mode': 'rw',
}
)
return mounts
async def execute_tool(self, parameters: dict, query: pipeline_query.Query) -> dict:
"""Execute an agent-facing ``exec`` tool call.
Translates the agent-facing ``command`` field to the internal
``BoxSpec.cmd`` field and injects the session id from the query.
"""
spec_payload: dict = {'cmd': parameters['command']}
# Pass through allowed agent-facing fields
for key in ('workdir', 'timeout_sec', 'env'):
if key in parameters:
spec_payload[key] = parameters[key]
# Inject context the agent must not control
spec_payload.setdefault('session_id', self.resolve_box_session_id(query))
# Mount all pipeline-bound skills so they are available in the container
if 'extra_mounts' not in spec_payload:
spec_payload['extra_mounts'] = self.build_skill_extra_mounts(query)
return await self.execute_spec_payload(spec_payload, query)
async def shutdown(self):
await self.client.shutdown()
def dispose(self):
if self._runtime_connector is not None:
self._runtime_connector.dispose()
loop = getattr(self.ap, 'event_loop', None)
if loop is not None and not loop.is_closed() and (self._shutdown_task is None or self._shutdown_task.done()):
self._shutdown_task = loop.create_task(self.shutdown())
async def get_sessions(self) -> list[dict]:
if not self._available:
return []
try:
return await self.client.get_sessions()
except Exception:
return []
def build_spec(self, spec_payload: dict, skip_host_mount_validation: bool = False) -> BoxSpec:
spec_payload = dict(spec_payload)
spec_payload.setdefault('env', {})
if spec_payload.get('host_path') in (None, '') and self.default_workspace is not None:
spec_payload['host_path'] = self.default_workspace
if spec_payload.get('workspace_quota_mb') in (None, '') and self.workspace_quota_mb is not None:
spec_payload['workspace_quota_mb'] = self.workspace_quota_mb
# Global custom image overrides profile default (but not caller-specified image)
if self.custom_image and 'image' not in spec_payload:
spec_payload['image'] = self.custom_image
self._apply_profile(spec_payload)
try:
spec = BoxSpec.model_validate(spec_payload)
except pydantic.ValidationError as exc:
first_error = exc.errors()[0]
raise BoxValidationError(first_error.get('msg', 'invalid box arguments')) from exc
if not skip_host_mount_validation:
self._validate_host_mount(spec)
return spec
async def create_session(self, spec_payload: dict, *, skip_host_mount_validation: bool = False) -> dict:
spec = self.build_spec(spec_payload, skip_host_mount_validation=skip_host_mount_validation)
return await self.client.create_session(spec)
async def start_managed_process(self, session_id: str, process_payload: dict) -> BoxManagedProcessInfo:
process_spec = BoxManagedProcessSpec.model_validate(process_payload)
return await self.client.start_managed_process(session_id, process_spec)
async def get_managed_process(self, session_id: str, process_id: str = 'default') -> BoxManagedProcessInfo:
return await self.client.get_managed_process(session_id, process_id)
async def stop_managed_process(self, session_id: str, process_id: str = 'default') -> None:
return await self.client.stop_managed_process(session_id, process_id)
def get_managed_process_websocket_url(self, session_id: str, process_id: str = 'default') -> str:
getter = getattr(self.client, 'get_managed_process_websocket_url', None)
if getter is None:
raise BoxValidationError('box runtime client does not support managed process websocket attach')
ws_relay_base_url = (
self._runtime_connector.ws_relay_base_url
if self._runtime_connector is not None
else 'http://127.0.0.1:5410'
)
return getter(session_id, ws_relay_base_url, process_id)
async def list_skills(self) -> list[dict]:
return await self.client.list_skills()
async def get_skill(self, name: str) -> dict | None:
return await self.client.get_skill(name)
async def create_skill(self, skill: dict) -> dict:
return await self.client.create_skill(skill)
async def update_skill(self, name: str, skill: dict) -> dict:
return await self.client.update_skill(name, skill)
async def delete_skill(self, name: str) -> None:
await self.client.delete_skill(name)
async def scan_skill_directory(self, path: str) -> dict:
return await self.client.scan_skill_directory(path)
async def list_skill_files(
self,
name: str,
path: str = '.',
include_hidden: bool = False,
max_entries: int = 200,
) -> dict:
return await self.client.list_skill_files(name, path, include_hidden, max_entries)
async def read_skill_file(self, name: str, path: str) -> dict:
return await self.client.read_skill_file(name, path)
async def write_skill_file(self, name: str, path: str, content: str) -> dict:
return await self.client.write_skill_file(name, path, content)
async def preview_skill_zip(
self,
file_bytes: bytes,
filename: str,
source_subdir: str = '',
target_suffix: str = 'upload',
) -> list[dict]:
return await self.client.preview_skill_zip(file_bytes, filename, source_subdir, target_suffix)
async def install_skill_zip(
self,
file_bytes: bytes,
filename: str,
source_paths: list[str] | None = None,
source_path: str = '',
source_subdir: str = '',
target_suffix: str = 'upload',
) -> list[dict]:
return await self.client.install_skill_zip(
file_bytes,
filename,
source_paths,
source_path,
source_subdir,
target_suffix,
)
def _serialize_result(self, result: BoxExecutionResult) -> dict:
stdout, stdout_truncated = self._truncate(result.stdout)
stderr, stderr_truncated = self._truncate(result.stderr)
return {
'session_id': result.session_id,
'backend': result.backend_name,
'status': result.status.value,
'ok': result.ok,
'exit_code': result.exit_code,
'stdout': stdout,
'stderr': stderr,
'stdout_truncated': stdout_truncated,
'stderr_truncated': stderr_truncated,
'duration_ms': result.duration_ms,
}
def _truncate(self, text: str) -> tuple[str, bool]:
if len(text) <= self.output_limit_chars:
return text, False
if self.output_limit_chars <= 0:
return '', True
head_size = 0
tail_size = 0
notice = ''
# Recompute once the omitted count is known so the final payload
# stays within output_limit_chars even after adding the notice.
for _ in range(4):
omitted = max(len(text) - head_size - tail_size, 0)
notice = f'\n\n... [{omitted} characters truncated] ...\n\n'
available = self.output_limit_chars - len(notice)
if available <= 0:
return notice[: self.output_limit_chars], True
new_head_size = int(available * 0.6)
new_tail_size = available - new_head_size
if new_head_size == head_size and new_tail_size == tail_size:
break
head_size = new_head_size
tail_size = new_tail_size
head = text[:head_size]
tail = text[-tail_size:] if tail_size else ''
truncated = f'{head}{notice}{tail}'
return truncated[: self.output_limit_chars], True
def _summarize_spec(self, spec: BoxSpec) -> dict:
cmd = spec.cmd.strip()
if len(cmd) > 400:
cmd = f'{cmd[:397]}...'
return {
'session_id': spec.session_id,
'workdir': spec.workdir,
'mount_path': spec.mount_path,
'timeout_sec': spec.timeout_sec,
'network': spec.network.value,
'image': spec.image,
'host_path': spec.host_path,
'host_path_mode': spec.host_path_mode.value,
'cpus': spec.cpus,
'memory_mb': spec.memory_mb,
'pids_limit': spec.pids_limit,
'read_only_rootfs': spec.read_only_rootfs,
'workspace_quota_mb': spec.workspace_quota_mb,
'env_keys': sorted(spec.env.keys()),
'cmd': cmd,
}
def _summarize_result(self, result: BoxExecutionResult) -> dict:
stdout_preview = result.stdout[:200]
stderr_preview = result.stderr[:200]
if len(result.stdout) > 200:
stdout_preview = f'{stdout_preview}...'
if len(result.stderr) > 200:
stderr_preview = f'{stderr_preview}...'
return {
'session_id': result.session_id,
'backend': result.backend_name,
'status': result.status.value,
'exit_code': result.exit_code,
'duration_ms': result.duration_ms,
'stdout_preview': stdout_preview,
'stderr_preview': stderr_preview,
}
def _local_config(self) -> dict:
"""Return ``box.local`` from instance config.
Environment overrides are applied uniformly by
``LoadConfigStage._apply_env_overrides_to_config`` (e.g.
``BOX__LOCAL__HOST_ROOT``) before this is read, so no box-specific
env parsing happens here.
"""
return dict(_get_box_config(self.ap).get('local') or {})
def _load_allowed_mount_roots(self) -> list[str]:
configured_roots = self._local_config().get('allowed_mount_roots', [])
# The unified env-override mechanism stores a brand-new key as a raw
# string when the key is absent from config.yaml. Accept a
# comma-separated string as well as a list so that
# ``BOX__LOCAL__ALLOWED_MOUNT_ROOTS="/a,/b"`` keeps working even when
# the config file has no ``box.local.allowed_mount_roots`` entry.
if isinstance(configured_roots, str):
configured_roots = [item.strip() for item in configured_roots.split(',') if item.strip()]
normalized_roots: list[str] = []
for root in configured_roots:
root_value = str(root).strip()
if not root_value:
continue
normalized_roots.append(os.path.realpath(os.path.abspath(root_value)))
if not normalized_roots and self.host_root is not None:
normalized_roots.append(self.host_root)
return normalized_roots
def _load_host_root(self) -> str | None:
host_root = str(self._local_config().get('host_root', '')).strip()
if not host_root:
return None
return os.path.realpath(os.path.abspath(host_root))
def _load_default_workspace(self) -> str | None:
default_workspace = str(self._local_config().get('default_workspace', '')).strip()
if not default_workspace:
if self.host_root is None:
return None
default_workspace = os.path.join(self.host_root, 'default')
elif not os.path.isabs(default_workspace) and self.host_root is not None:
default_workspace = os.path.join(self.host_root, default_workspace)
return os.path.realpath(os.path.abspath(default_workspace))
def get_skills_root(self) -> str | None:
skills_root = str(self._local_config().get('skills_root', '') or 'skills').strip()
if not skills_root:
skills_root = 'skills'
if not os.path.isabs(skills_root) and self.host_root is not None:
skills_root = os.path.join(self.host_root, skills_root)
return os.path.realpath(os.path.abspath(skills_root))
def _load_enabled(self) -> bool:
"""Read ``box.enabled`` (top-level, not ``box.local.*``). Default True
— disabling is opt-in. Accepts bool, ``'true'``/``'false'`` strings,
and the standard env-overridden truthy values that
``LoadConfigStage._apply_env_overrides_to_config`` produces."""
raw = _get_box_config(self.ap).get('enabled', True)
if isinstance(raw, bool):
return raw
return str(raw).strip().lower() not in ('false', '0', 'no', 'off', '')
def _load_custom_image(self) -> str | None:
raw = str(self._local_config().get('image', '') or '').strip()
return raw or None
def _load_workspace_quota_mb(self) -> int | None:
raw_value = self._local_config().get('workspace_quota_mb')
if raw_value in (None, ''):
return None
try:
value = _INT_ADAPTER.validate_python(raw_value)
except pydantic.ValidationError as exc:
raise BoxValidationError('workspace_quota_mb must be an integer greater than or equal to 0') from exc
if value < 0:
raise BoxValidationError('workspace_quota_mb must be greater than or equal to 0')
return value
def _ensure_default_workspace(self):
if self.default_workspace is None:
return
if os.path.isdir(self.default_workspace):
return
if os.path.exists(self.default_workspace):
raise BoxValidationError('box.local.default_workspace must point to a directory on the host')
if not self.allowed_mount_roots:
raise BoxValidationError(
'box.local.default_workspace cannot be created because no allowed_mount_roots are configured'
)
for allowed_root in self.allowed_mount_roots:
if _is_path_under(self.default_workspace, allowed_root):
os.makedirs(self.default_workspace, exist_ok=True)
return
allowed_roots = ', '.join(self.allowed_mount_roots)
raise BoxValidationError(f'box.local.default_workspace is outside allowed_mount_roots: {allowed_roots}')
def _validate_host_mount(self, spec: BoxSpec):
if spec.host_path is None:
return
host_path = os.path.realpath(spec.host_path)
if not os.path.isdir(host_path):
raise BoxValidationError('host_path must point to an existing directory on the host')
if not self.allowed_mount_roots:
raise BoxValidationError('host_path mounting is disabled because no allowed_mount_roots are configured')
for allowed_root in self.allowed_mount_roots:
if _is_path_under(host_path, allowed_root):
return
allowed_roots = ', '.join(self.allowed_mount_roots)
raise BoxValidationError(f'host_path is outside allowed_mount_roots: {allowed_roots}')
def _load_profile(self) -> BoxProfile:
profile_name = str(self._local_config().get('profile', 'default')).strip() or 'default'
profile = BUILTIN_PROFILES.get(profile_name)
if profile is None:
available = ', '.join(sorted(BUILTIN_PROFILES))
raise BoxValidationError(f"unknown box profile '{profile_name}', available profiles: {available}")
return profile
def _apply_profile(self, params: dict):
"""Merge profile defaults into *params* in-place, enforce locked fields and clamp timeout."""
profile = self.profile
_PROFILE_FIELDS = (
'image',
'network',
'timeout_sec',
'host_path_mode',
'cpus',
'memory_mb',
'pids_limit',
'read_only_rootfs',
'workspace_quota_mb',
)
for field in _PROFILE_FIELDS:
profile_value = getattr(profile, field)
raw_value = profile_value.value if isinstance(profile_value, enum.Enum) else profile_value
if field in profile.locked:
params[field] = raw_value
elif field not in params:
params[field] = raw_value
timeout = params.get('timeout_sec')
try:
normalized_timeout = _INT_ADAPTER.validate_python(timeout)
except pydantic.ValidationError:
return
if normalized_timeout > profile.max_timeout_sec:
params['timeout_sec'] = profile.max_timeout_sec
def _get_workspace_size_bytes(self, root: str) -> int:
total = 0
def _walk(path: str):
nonlocal total
try:
with os.scandir(path) as entries:
for entry in entries:
try:
if entry.is_symlink():
total += entry.stat(follow_symlinks=False).st_size
continue
if entry.is_dir(follow_symlinks=False):
_walk(entry.path)
continue
total += entry.stat(follow_symlinks=False).st_size
except FileNotFoundError:
continue
except FileNotFoundError:
return
_walk(root)
return total
def _enforce_workspace_quota(self, spec: BoxSpec, *, phase: str) -> None:
if spec.host_path is None or spec.workspace_quota_mb <= 0:
return
host_path = os.path.realpath(spec.host_path)
if not os.path.isdir(host_path):
return
used_bytes = self._get_workspace_size_bytes(host_path)
limit_bytes = spec.workspace_quota_mb * _MIB
if used_bytes <= limit_bytes:
return
raise BoxValidationError(
f'workspace quota exceeded {phase}: '
f'used={used_bytes} bytes limit={limit_bytes} bytes '
f'host_path={host_path} session_id={spec.session_id}'
)
async def _cleanup_exceeded_session(self, spec: BoxSpec) -> None:
try:
await self.client.delete_session(spec.session_id)
except Exception as exc:
self.ap.logger.warning(
'Failed to clean up Box session after workspace quota was exceeded: '
f'session_id={spec.session_id} error={exc}'
)
# ── Observability ─────────────────────────────────────────────────
def _record_error(self, exc: Exception, query: pipeline_query.Query):
self._recent_errors.append(
{
'timestamp': _dt.datetime.now(_UTC).isoformat(),
'type': type(exc).__name__,
'message': str(exc),
'query_id': str(query.query_id),
}
)
def get_recent_errors(self) -> list[dict]:
return list(self._recent_errors)
def get_system_guidance(self) -> str:
"""Return LLM system-prompt guidance for the exec tool.
All execution-specific prompt text is kept here so that callers
(e.g. LocalAgentRunner) stay free of box domain knowledge.
"""
guidance = (
'When the exec tool is available, use it for exact calculations, statistics, structured data parsing, '
'and code execution instead of estimating mentally. If the user provides numbers, tables, CSV-like text, '
'JSON, or other data and asks for a computed answer, prefer running a short Python script via exec '
'and then answer from the tool result. Unless the user explicitly asks for the script, code, or implementation '
'details, do not include the generated script in the final answer; return the result and a brief explanation only.'
)
if self.default_workspace:
guidance += (
' A default workspace is mounted at /workspace for file tasks. When the user asks to read, create, or '
'modify local files in the working directory, use exec with /workspace paths directly; do not ask the '
'user for directory parameters unless they explicitly need a different directory.'
)
return guidance
async def get_status(self) -> dict:
if not self._available:
return {
'available': False,
'enabled': self._enabled,
'profile': self.profile.name,
'recent_error_count': len(self._recent_errors),
'connector_error': self._connector_error,
}
try:
runtime_status = await self.client.get_status()
except Exception as exc:
# RPC failed — the runtime likely just disconnected and the
# heartbeat hasn't flipped _available yet.
return {
'available': False,
'enabled': self._enabled,
'profile': self.profile.name,
'recent_error_count': len(self._recent_errors),
'connector_error': str(exc),
}
# Backend state can be unavailable even when the connector is healthy
# (operator selected nsjail but the binary is missing, Docker daemon
# went down after the runtime started, E2B credentials wrong, ...).
# Report the combined state in the top-level ``available`` so the
# frontend banner / ``useBoxStatus`` hook / native-tool gate all
# agree on "actually usable" rather than "connector alive". The
# detailed ``backend`` object stays in the payload so the dialog
# can still show which backend was tried.
backend_info = runtime_status.get('backend') if isinstance(runtime_status, dict) else None
backend_ok = bool(backend_info and backend_info.get('available', False))
payload = {
**runtime_status,
'available': backend_ok,
'enabled': self._enabled,
'profile': self.profile.name,
'recent_error_count': len(self._recent_errors),
}
if not backend_ok and 'connector_error' not in payload:
backend_name = backend_info.get('name') if backend_info else None
if backend_name:
payload['connector_error'] = f'Configured sandbox backend "{backend_name}" is unavailable'
else:
payload['connector_error'] = 'No supported sandbox backend (Docker / nsjail / E2B) is available'
return payload

View File

@@ -0,0 +1,413 @@
"""Reusable workspace/session helpers built on top of Box.
This module is the middle layer between the raw Box runtime primitives and
application-specific flows such as skills or MCP stdio.
It intentionally stays generic:
- path and virtualenv rewriting are workspace concerns
- Python project detection/bootstrap are workspace concerns
- session exec / managed-process helpers are workspace concerns
Higher layers add their own semantics on top, for example:
- skills choose a stable per-skill session id and use repeated exec
- MCP stdio chooses how to prepare dependencies and attaches to a managed process
"""
from __future__ import annotations
import os
import textwrap
from typing import Any
PYTHON_MANIFEST_FILES = (
'requirements.txt',
'pyproject.toml',
'setup.py',
'setup.cfg',
)
_VENV_DIRS = frozenset({'.venv', 'venv', 'env', '.env'})
_VENV_BIN_DIRS = frozenset({'bin', 'Scripts'})
def normalize_host_path(path: str | None) -> str:
if path is None:
return ''
stripped = str(path).strip()
if not stripped:
return ''
return os.path.realpath(os.path.abspath(stripped))
def rewrite_mounted_path(path: str, host_path: str | None, *, mount_path: str = '/workspace') -> str:
"""Translate a host path into the path visible inside the sandbox mount."""
if not host_path or not path:
return path
normalized_host = os.path.realpath(host_path)
normalized_path = os.path.realpath(path)
if normalized_path.startswith(normalized_host + '/'):
return mount_path + normalized_path[len(normalized_host) :]
if normalized_path == normalized_host:
return mount_path
return path
def unwrap_venv_path(directory: str) -> str:
"""Collapse ``.../.venv/bin`` style paths back to the project root."""
parts = directory.replace('\\', '/').split('/')
for i in range(len(parts) - 1, 0, -1):
if parts[i] in _VENV_BIN_DIRS and i >= 1:
venv_dir = parts[i - 1]
if venv_dir in _VENV_DIRS:
project_root = '/'.join(parts[: i - 1])
return project_root if project_root else '/'
return directory
def infer_workspace_host_path(command: str, args: list[str] | None = None) -> str | None:
"""Infer the project/workspace root from absolute command/arg paths."""
candidates: list[str] = []
for part in [command, *(args or [])]:
if not os.path.isabs(part):
continue
if os.path.exists(part):
directory = os.path.dirname(part)
candidates.append(os.path.realpath(unwrap_venv_path(directory)))
if not candidates:
return None
common = os.path.commonpath(candidates)
return common if common != '/' else None
def rewrite_venv_command(command: str, host_path: str | None, *, mount_path: str = '/workspace') -> str:
"""Rewrite host venv interpreters to plain ``python`` inside the sandbox.
Once a project is mounted into the sandbox, host virtualenv paths are no
longer valid. For those paths we intentionally drop down to ``python`` and
let the sandbox-side environment/bootstrap decide what interpreter to use.
"""
if not host_path or not command:
return command
normalized_host = os.path.realpath(host_path)
normalized_command = os.path.realpath(command)
if not normalized_command.startswith(normalized_host + '/'):
return command
rel = normalized_command[len(normalized_host) + 1 :]
parts = rel.replace('\\', '/').split('/')
if len(parts) >= 3 and parts[0] in _VENV_DIRS and parts[1] in _VENV_BIN_DIRS and parts[2].startswith('python'):
return 'python'
return rewrite_mounted_path(normalized_command, host_path, mount_path=mount_path)
def list_python_manifest_files(host_path: str | None) -> list[str]:
normalized_root = normalize_host_path(host_path)
if not normalized_root:
return []
return [filename for filename in PYTHON_MANIFEST_FILES if os.path.isfile(os.path.join(normalized_root, filename))]
def classify_python_workspace(host_path: str | None) -> str | None:
"""Return the generic Python workspace shape, without app-specific policy."""
manifest_files = set(list_python_manifest_files(host_path))
if not manifest_files:
return None
if {'pyproject.toml', 'setup.py', 'setup.cfg'} & manifest_files:
return 'package'
if 'requirements.txt' in manifest_files:
return 'requirements'
return None
def should_prepare_python_env(host_path: str | None) -> bool:
normalized_root = normalize_host_path(host_path)
if not normalized_root:
return False
if os.path.isdir(os.path.join(normalized_root, '.venv')):
return True
return bool(list_python_manifest_files(normalized_root))
def wrap_python_command_with_env(command: str, *, mount_path: str = '/workspace') -> str:
"""Wrap a command with a reusable sandbox-local Python env bootstrap.
This is the generic "workspace is a Python project" path used by mutable
workspaces such as skills. Read-only installation strategies stay in the
higher-level caller because they are application policy, not workspace
semantics.
"""
bootstrap = textwrap.dedent(
f"""
set -e
_LB_VENV_DIR="{mount_path}/.venv"
_LB_META_DIR="{mount_path}/.langbot"
_LB_META_FILE="$_LB_META_DIR/python-env.json"
_LB_LOCK_DIR="$_LB_META_DIR/python-env.lock"
_LB_TMP_DIR="{mount_path}/.tmp"
_LB_PIP_CACHE_DIR="{mount_path}/.cache/pip"
mkdir -p "$_LB_META_DIR" "$_LB_TMP_DIR" "$_LB_PIP_CACHE_DIR"
export TMPDIR="$_LB_TMP_DIR"
export TEMP="$_LB_TMP_DIR"
export TMP="$_LB_TMP_DIR"
export PIP_CACHE_DIR="$_LB_PIP_CACHE_DIR"
_lb_python_meta() {{
python - <<'PY'
import hashlib
import json
import os
import sys
root = "{mount_path}"
digest = hashlib.sha256()
manifest_files = []
for rel in ("requirements.txt", "pyproject.toml", "setup.py", "setup.cfg"):
path = os.path.join(root, rel)
if not os.path.isfile(path):
continue
manifest_files.append(rel)
with open(path, "rb") as handle:
digest.update(rel.encode("utf-8"))
digest.update(b"\\0")
digest.update(handle.read())
digest.update(b"\\0")
print(
json.dumps(
{{
"python_executable": sys.executable,
"python_version": list(sys.version_info[:3]),
"manifest_files": manifest_files,
"manifest_sha256": digest.hexdigest(),
}},
sort_keys=True,
)
)
PY
}}
_LB_CURRENT_META="$(_lb_python_meta)"
_LB_NEEDS_BOOTSTRAP=0
if [ ! -x "$_LB_VENV_DIR/bin/python" ]; then
_LB_NEEDS_BOOTSTRAP=1
elif [ ! -f "$_LB_META_FILE" ]; then
_LB_NEEDS_BOOTSTRAP=1
elif [ "$(cat "$_LB_META_FILE")" != "$_LB_CURRENT_META" ]; then
_LB_NEEDS_BOOTSTRAP=1
fi
if [ "$_LB_NEEDS_BOOTSTRAP" -eq 1 ]; then
_LB_LOCK_WAIT=0
while ! mkdir "$_LB_LOCK_DIR" 2>/dev/null; do
if [ "$_LB_LOCK_WAIT" -ge 120 ]; then
echo "Timed out waiting for Python environment lock: $_LB_LOCK_DIR" >&2
exit 1
fi
sleep 1
_LB_LOCK_WAIT=$((_LB_LOCK_WAIT + 1))
done
_lb_cleanup_lock() {{
rmdir "$_LB_LOCK_DIR" >/dev/null 2>&1 || true
}}
trap _lb_cleanup_lock EXIT INT TERM
_LB_CURRENT_META="$(_lb_python_meta)"
_LB_NEEDS_BOOTSTRAP=0
if [ ! -x "$_LB_VENV_DIR/bin/python" ]; then
_LB_NEEDS_BOOTSTRAP=1
elif [ ! -f "$_LB_META_FILE" ]; then
_LB_NEEDS_BOOTSTRAP=1
elif [ "$(cat "$_LB_META_FILE")" != "$_LB_CURRENT_META" ]; then
_LB_NEEDS_BOOTSTRAP=1
fi
if [ "$_LB_NEEDS_BOOTSTRAP" -eq 1 ]; then
rm -rf "$_LB_VENV_DIR"
python -m venv "$_LB_VENV_DIR"
. "$_LB_VENV_DIR/bin/activate"
python -m pip install --upgrade pip setuptools wheel
if [ -f "{mount_path}/requirements.txt" ]; then
python -m pip install -r "{mount_path}/requirements.txt"
elif [ -f "{mount_path}/pyproject.toml" ] || [ -f "{mount_path}/setup.py" ] || [ -f "{mount_path}/setup.cfg" ]; then
python -m pip install "{mount_path}"
fi
printf '%s' "$_LB_CURRENT_META" > "$_LB_META_FILE"
fi
fi
export VIRTUAL_ENV="$_LB_VENV_DIR"
export PATH="$_LB_VENV_DIR/bin:$PATH"
{command}
"""
).strip()
return bootstrap + '\n'
class BoxWorkspaceSession:
"""High-level handle for one reusable workspace-backed Box session.
The Box runtime already understands sessions and managed processes. This
wrapper adds LangBot's workspace-centric view on top: a mounted host path,
a stable ``session_id``, optional environment defaults, and convenience
helpers for exec or long-running processes inside that workspace.
"""
def __init__(
self,
box_service,
session_id: str,
*,
host_path: str | None = None,
host_path_mode: str = 'rw',
workdir: str = '/workspace',
env: dict[str, str] | None = None,
mount_path: str = '/workspace',
network: str | None = None,
read_only_rootfs: bool | None = None,
image: str | None = None,
cpus: float | None = None,
memory_mb: int | None = None,
pids_limit: int | None = None,
persistent: bool = False,
):
self.box_service = box_service
self.session_id = session_id
self.host_path = host_path
self.host_path_mode = host_path_mode
self.workdir = workdir
self.env = dict(env or {})
self.mount_path = mount_path
self.network = network
self.read_only_rootfs = read_only_rootfs
self.image = image
self.cpus = cpus
self.memory_mb = memory_mb
self.pids_limit = pids_limit
self.persistent = persistent
def rewrite_path(self, path: str) -> str:
return rewrite_mounted_path(path, self.host_path, mount_path=self.mount_path)
def rewrite_venv_command(self, command: str) -> str:
return rewrite_venv_command(command, self.host_path, mount_path=self.mount_path)
def build_session_payload(self) -> dict[str, Any]:
# Keep this payload generic so callers can reuse the same workspace
# handle for plain exec, file-producing tasks, or managed processes.
payload: dict[str, Any] = {
'session_id': self.session_id,
'workdir': self.workdir,
'env': self.env,
'persistent': self.persistent,
}
if self.network is not None:
payload['network'] = self.network
if self.read_only_rootfs is not None:
payload['read_only_rootfs'] = self.read_only_rootfs
if self.host_path:
payload['host_path'] = self.host_path
payload['host_path_mode'] = self.host_path_mode
for key in ('image', 'cpus', 'memory_mb', 'pids_limit'):
value = getattr(self, key)
if value is not None:
payload[key] = value
return payload
def build_exec_payload(
self,
cmd: str,
*,
workdir: str | None = None,
env: dict[str, str] | None = None,
timeout_sec: int | None = None,
) -> dict[str, Any]:
# Exec payloads inherit the session-level workspace config, then layer
# per-call command/workdir/env overrides on top.
payload = self.build_session_payload()
payload['cmd'] = cmd
payload['workdir'] = workdir or self.workdir
if timeout_sec is not None:
payload['timeout_sec'] = timeout_sec
resolved_env = self.env if env is None else env
if resolved_env:
payload['env'] = resolved_env
elif 'env' in payload and not payload['env']:
payload.pop('env')
return payload
async def execute_raw(
self,
cmd: str,
*,
workdir: str | None = None,
env: dict[str, str] | None = None,
timeout_sec: int | None = None,
):
payload = self.build_exec_payload(cmd, workdir=workdir, env=env, timeout_sec=timeout_sec)
return await self.box_service.client.execute(self.box_service.build_spec(payload))
async def execute_for_query(
self,
query,
cmd: str,
*,
workdir: str | None = None,
env: dict[str, str] | None = None,
timeout_sec: int | None = None,
) -> dict:
payload = self.build_exec_payload(cmd, workdir=workdir, env=env, timeout_sec=timeout_sec)
return await self.box_service.execute_spec_payload(payload, query)
async def create_session(self):
return await self.box_service.create_session(self.build_session_payload())
def build_process_payload(
self,
command: str,
args: list[str] | None = None,
*,
env: dict[str, str] | None = None,
cwd: str = '/workspace',
) -> dict[str, Any]:
# Managed processes run inside the same workspace model as one-shot
# execs, so path/venv rewriting is shared here.
normalized_command = command
normalized_args = list(args or [])
normalized_cwd = cwd
if self.host_path:
normalized_command = self.rewrite_venv_command(command)
normalized_args = [self.rewrite_path(arg) for arg in normalized_args]
normalized_cwd = self.rewrite_path(cwd)
return {
'command': normalized_command,
'args': normalized_args,
'env': dict(env or {}),
'cwd': normalized_cwd,
}
async def start_managed_process(
self,
command: str,
args: list[str] | None = None,
*,
process_id: str = 'default',
env: dict[str, str] | None = None,
cwd: str = '/workspace',
):
payload = self.build_process_payload(command, args, env=env, cwd=cwd)
payload['process_id'] = process_id
return await self.box_service.start_managed_process(self.session_id, payload)
async def get_managed_process(self, process_id: str = 'default'):
return await self.box_service.get_managed_process(self.session_id, process_id)
async def stop_managed_process(self, process_id: str = 'default') -> None:
await self.box_service.stop_managed_process(self.session_id, process_id)
def get_managed_process_websocket_url(self, process_id: str = 'default') -> str:
return self.box_service.get_managed_process_websocket_url(self.session_id, process_id)
async def cleanup(self) -> None:
await self.box_service.client.delete_session(self.session_id)

View File

@@ -9,6 +9,7 @@ from ..platform import botmgr as im_mgr
from ..platform.webhook_pusher import WebhookPusher
from ..provider.session import sessionmgr as llm_session_mgr
from ..provider.modelmgr import modelmgr as llm_model_mgr
from ..box import service as box_service_module
from langbot.pkg.provider.tools import toolmgr as llm_tool_mgr
from ..config import manager as config_mgr
@@ -31,9 +32,8 @@ from ..api.http.service import mcp as mcp_service
from ..api.http.service import apikey as apikey_service
from ..api.http.service import webhook as webhook_service
from ..api.http.service import monitoring as monitoring_service
from ..api.http.service import workflow as workflow_service
from ..api.http.service import skill as skill_service
from ..api.http.service import maintenance as maintenance_service
from ..discover import engine as discover_engine
from ..storage import mgr as storagemgr
from ..utils import logcache
@@ -44,6 +44,7 @@ from ..rag.service import RAGRuntimeService
from ..vector import mgr as vectordb_mgr
from ..telemetry import telemetry as telemetry_module
from ..survey import manager as survey_module
from ..skill import manager as skill_mgr
class Application:
@@ -71,6 +72,7 @@ class Application:
# TODO move to pipeline
tool_mgr: llm_tool_mgr.ToolManager = None
box_service: box_service_module.BoxService = None
# ======= Config manager =======
@@ -151,14 +153,16 @@ class Application:
webhook_service: webhook_service.WebhookService = None
workflow_service: workflow_service.WorkflowService = None
telemetry: telemetry_module.TelemetryManager = None
survey: survey_module.SurveyManager = None
monitoring_service: monitoring_service.MonitoringService = None
skill_service: skill_service.SkillService = None
skill_mgr: skill_mgr.SkillManager = None
maintenance_service: maintenance_service.MaintenanceService = None
def __init__(self):
@@ -240,22 +244,6 @@ class Application:
scopes=[core_entities.LifecycleControlScope.APPLICATION],
)
async def workflow_execution_cleanup_loop():
check_interval_seconds = 60
while True:
try:
cancelled = await self.workflow_service.cleanup_stale_executions()
if cancelled > 0:
self.logger.info(f'Workflow execution auto-cleanup: cancelled {cancelled} stale executions')
except Exception as e:
self.logger.warning(f'Workflow execution auto-cleanup error: {e}')
await asyncio.sleep(check_interval_seconds)
self.task_mgr.create_task(
workflow_execution_cleanup_loop(),
name='workflow-execution-cleanup',
scopes=[core_entities.LifecycleControlScope.APPLICATION],
)
# Start storage/log maintenance task if enabled
storage_cleanup_cfg = self.instance_config.data.get('storage', {}).get('cleanup', {})
if storage_cleanup_cfg.get('enabled', True) and self.maintenance_service is not None:
@@ -320,7 +308,10 @@ class Application:
return parsed
def dispose(self):
self.plugin_connector.dispose()
if self.plugin_connector is not None:
self.plugin_connector.dispose()
if self.box_service is not None:
self.box_service.dispose()
async def print_web_access_info(self):
"""Print access webui tips"""

View File

@@ -62,4 +62,6 @@ async def main(loop: asyncio.AbstractEventLoop):
app_inst = await make_app(loop)
await app_inst.run()
except Exception:
if app_inst is not None:
app_inst.dispose()
traceback.print_exc()

View File

@@ -6,6 +6,7 @@ from .. import stage, app
from ...utils import version, proxy
from ...pipeline import pool, controller, pipelinemgr
from ...pipeline import aggregator as message_aggregator
from ...box import service as box_service
from ...plugin import connector as plugin_connector
from ...command import cmdmgr
from ...provider.session import sessionmgr as llm_session_mgr
@@ -28,7 +29,8 @@ from ...api.http.service import mcp as mcp_service
from ...api.http.service import apikey as apikey_service
from ...api.http.service import webhook as webhook_service
from ...api.http.service import monitoring as monitoring_service
from ...api.http.service import workflow as workflow_service
from ...api.http.service import skill as skill_service
from ...skill import manager as skill_mgr
from ...api.http.service import maintenance as maintenance_service
from ...discover import engine as discover_engine
from ...storage import mgr as storagemgr
@@ -87,8 +89,8 @@ class BuildAppStage(stage.BootingStage):
webhook_service_inst = webhook_service.WebhookService(ap)
ap.webhook_service = webhook_service_inst
workflow_service_inst = workflow_service.WorkflowService(ap)
ap.workflow_service = workflow_service_inst
skill_service_inst = skill_service.SkillService(ap)
ap.skill_service = skill_service_inst
proxy_mgr = proxy.ProxyManager(ap)
await proxy_mgr.initialize()
@@ -133,6 +135,10 @@ class BuildAppStage(stage.BootingStage):
await llm_session_mgr_inst.initialize()
ap.sess_mgr = llm_session_mgr_inst
box_service_inst = box_service.BoxService(ap)
await box_service_inst.initialize()
ap.box_service = box_service_inst
llm_tool_mgr_inst = llm_tool_mgr.ToolManager(ap)
await llm_tool_mgr_inst.initialize()
ap.tool_mgr = llm_tool_mgr_inst
@@ -153,6 +159,11 @@ class BuildAppStage(stage.BootingStage):
msg_aggregator_inst = message_aggregator.MessageAggregator(ap)
ap.msg_aggregator = msg_aggregator_inst
# Initialize skill manager
skill_mgr_inst = skill_mgr.SkillManager(ap)
await skill_mgr_inst.initialize()
ap.skill_mgr = skill_mgr_inst
rag_mgr_inst = rag_mgr.RAGManager(ap)
await rag_mgr_inst.initialize()
ap.rag_mgr = rag_mgr_inst

View File

@@ -221,34 +221,3 @@ class LoadConfigStage(stage.BootingStage):
ap.pipeline_config_meta_safety = await load_resource_yaml_template_data('metadata/pipeline/safety.yaml')
ap.pipeline_config_meta_ai = await load_resource_yaml_template_data('metadata/pipeline/ai.yaml')
ap.pipeline_config_meta_output = await load_resource_yaml_template_data('metadata/pipeline/output.yaml')
# Load workflow node metadata from YAML files. YAML is the source of
# truth for workflow editor metadata; Python classes provide execution
# logic and are bound through the registry.
from langbot.pkg.workflow.metadata import NodeMetadataLoader
from langbot.pkg.workflow.registry import NodeTypeRegistry
workflow_metadata_loader = NodeMetadataLoader()
workflow_node_count = await workflow_metadata_loader.load_core_metadata()
ap.workflow_node_configs = workflow_metadata_loader.get_all_metadata()
ap.workflow_node_metadata_loader = workflow_metadata_loader
workflow_registry = NodeTypeRegistry.instance()
for node_config in ap.workflow_node_configs.values():
workflow_registry.register_metadata(node_config, source=node_config.get('_source', 'core'))
# Auto-discover and register workflow nodes using discovery engine
if hasattr(ap, 'discover') and ap.discover is not None:
workflow_registry.discover_nodes(ap.discover)
workflow_load_errors = workflow_metadata_loader.get_load_errors()
if workflow_load_errors:
print(f'Workflow node metadata load errors: {len(workflow_load_errors)}')
for error in workflow_load_errors:
print(f" - {error.get('file')}: {error.get('error')}")
print(
f'Loaded {workflow_node_count} workflow node metadata files; '
f'registered {workflow_registry.metadata_count()} metadata definitions, '
f'{workflow_registry.count()} node types'
)

View File

@@ -304,65 +304,3 @@ class ComponentDiscoveryEngine:
if component.kind == kind:
result.append(component)
return result
def discover_workflow_nodes(self, nodes_dir: str) -> typing.List[typing.Type]:
"""Discover workflow node classes from a directory of Python modules.
Scans all .py files in the given directory, imports them, and collects
classes that are subclasses of WorkflowNode.
Args:
nodes_dir: Directory path like 'pkg/workflow/nodes/'
Returns:
List of WorkflowNode subclasses found
"""
from langbot.pkg.workflow.node import WorkflowNode
node_classes: typing.List[typing.Type[WorkflowNode]] = []
# Normalize path
if nodes_dir.endswith('/'):
nodes_dir = nodes_dir[:-1]
# Import the nodes package to trigger all module imports
module_path = nodes_dir.replace('/', '.').replace('\\', '.')
package_path = module_path
try:
# Import the package __init__ to trigger submodule imports
importlib.import_module(f'langbot.{package_path}')
except ImportError:
self.ap.logger.warning(f'Failed to import workflow nodes package: langbot.{package_path}')
# Since workflow/__init__.py is empty, explicitly import all .py files in the nodes directory
import os
# engine.py is in langbot/pkg/discover/, nodes are in langbot/pkg/workflow/nodes/
nodes_abs_path = os.path.abspath(os.path.join(os.path.dirname(__file__), '..', 'workflow', 'nodes'))
if os.path.isdir(nodes_abs_path):
for filename in os.listdir(nodes_abs_path):
if filename.endswith('.py') and not filename.startswith('_'):
module_name = filename[:-3]
try:
importlib.import_module(f'langbot.{package_path}.{module_name}')
except ImportError as e:
self.ap.logger.warning(f'Failed to import workflow node module: {module_name}: {e}')
# Now collect all WorkflowNode subclasses from sys.modules
import sys
prefix = f'langbot.{package_path}.'
for mod_name, mod in sys.modules.items():
if mod_name.startswith(prefix) and mod is not None:
for attr_name in dir(mod):
attr = getattr(mod, attr_name)
if (
isinstance(attr, type)
and issubclass(attr, WorkflowNode)
and attr is not WorkflowNode
and hasattr(attr, 'type_name')
and attr.type_name
):
if attr not in node_classes:
node_classes.append(attr)
return node_classes

View File

@@ -17,13 +17,6 @@ class Bot(Base):
use_pipeline_name = sqlalchemy.Column(sqlalchemy.String(255), nullable=True)
use_pipeline_uuid = sqlalchemy.Column(sqlalchemy.String(255), nullable=True)
pipeline_routing_rules = sqlalchemy.Column(sqlalchemy.JSON, nullable=False, server_default='[]')
# New unified binding fields
# binding_type: 'pipeline' or 'workflow'
binding_type = sqlalchemy.Column(sqlalchemy.String(32), nullable=False, server_default='pipeline')
# binding_uuid: UUID of the bound Pipeline or Workflow
binding_uuid = sqlalchemy.Column(sqlalchemy.String(64), nullable=True)
created_at = sqlalchemy.Column(sqlalchemy.DateTime, nullable=False, server_default=sqlalchemy.func.now())
updated_at = sqlalchemy.Column(
sqlalchemy.DateTime,

View File

@@ -1,126 +0,0 @@
"""Workflow persistence entities"""
import sqlalchemy
from .base import Base
class Workflow(Base):
"""Workflow definition"""
__tablename__ = 'workflows'
uuid = sqlalchemy.Column(sqlalchemy.String(255), primary_key=True, unique=True)
name = sqlalchemy.Column(sqlalchemy.String(255), nullable=False)
description = sqlalchemy.Column(sqlalchemy.Text, nullable=True)
emoji = sqlalchemy.Column(sqlalchemy.String(10), nullable=True, default='🔄')
version = sqlalchemy.Column(sqlalchemy.Integer, nullable=False, default=1)
is_enabled = sqlalchemy.Column(sqlalchemy.Boolean, nullable=False, default=True)
# Workflow definition stored as JSON
# Contains: nodes, edges, variables, settings
definition = sqlalchemy.Column(sqlalchemy.JSON, nullable=False, default={})
# Global config (inherited from Pipeline capabilities)
# Contains: safety, output configs
global_config = sqlalchemy.Column(sqlalchemy.JSON, nullable=False, default={})
# Extensions preferences (same as Pipeline)
extensions_preferences = sqlalchemy.Column(
sqlalchemy.JSON,
nullable=False,
default={'enable_all_plugins': True, 'enable_all_mcp_servers': True, 'plugins': [], 'mcp_servers': []},
)
created_at = sqlalchemy.Column(sqlalchemy.DateTime, nullable=False, server_default=sqlalchemy.func.now())
updated_at = sqlalchemy.Column(
sqlalchemy.DateTime,
nullable=False,
server_default=sqlalchemy.func.now(),
onupdate=sqlalchemy.func.now(),
)
class WorkflowVersion(Base):
"""Workflow version history"""
__tablename__ = 'workflow_versions'
id = sqlalchemy.Column(sqlalchemy.Integer, primary_key=True, autoincrement=True)
workflow_uuid = sqlalchemy.Column(sqlalchemy.String(255), nullable=False, index=True)
version = sqlalchemy.Column(sqlalchemy.Integer, nullable=False)
definition = sqlalchemy.Column(sqlalchemy.JSON, nullable=False)
global_config = sqlalchemy.Column(sqlalchemy.JSON, nullable=False, default={})
created_at = sqlalchemy.Column(sqlalchemy.DateTime, nullable=False, server_default=sqlalchemy.func.now())
created_by = sqlalchemy.Column(sqlalchemy.String(255), nullable=True)
__table_args__ = (sqlalchemy.UniqueConstraint('workflow_uuid', 'version', name='uq_workflow_version'),)
class WorkflowTrigger(Base):
"""Workflow trigger configuration"""
__tablename__ = 'workflow_triggers'
uuid = sqlalchemy.Column(sqlalchemy.String(255), primary_key=True, unique=True)
workflow_uuid = sqlalchemy.Column(sqlalchemy.String(255), nullable=False, index=True)
type = sqlalchemy.Column(sqlalchemy.String(50), nullable=False) # message, cron, event, webhook
config = sqlalchemy.Column(sqlalchemy.JSON, nullable=False, default={})
is_enabled = sqlalchemy.Column(sqlalchemy.Boolean, nullable=False, default=True)
priority = sqlalchemy.Column(sqlalchemy.Integer, nullable=False, default=0)
created_at = sqlalchemy.Column(sqlalchemy.DateTime, nullable=False, server_default=sqlalchemy.func.now())
updated_at = sqlalchemy.Column(
sqlalchemy.DateTime,
nullable=False,
server_default=sqlalchemy.func.now(),
onupdate=sqlalchemy.func.now(),
)
class WorkflowExecution(Base):
"""Workflow execution record"""
__tablename__ = 'workflow_executions'
uuid = sqlalchemy.Column(sqlalchemy.String(255), primary_key=True, unique=True)
workflow_uuid = sqlalchemy.Column(sqlalchemy.String(255), nullable=False, index=True)
workflow_version = sqlalchemy.Column(sqlalchemy.Integer, nullable=False)
status = sqlalchemy.Column(sqlalchemy.String(20), nullable=False) # pending, running, completed, failed, cancelled
trigger_type = sqlalchemy.Column(sqlalchemy.String(50), nullable=True)
trigger_data = sqlalchemy.Column(sqlalchemy.JSON, nullable=True)
variables = sqlalchemy.Column(sqlalchemy.JSON, nullable=True)
start_time = sqlalchemy.Column(sqlalchemy.DateTime, nullable=True)
end_time = sqlalchemy.Column(sqlalchemy.DateTime, nullable=True)
error = sqlalchemy.Column(sqlalchemy.Text, nullable=True)
created_at = sqlalchemy.Column(sqlalchemy.DateTime, nullable=False, server_default=sqlalchemy.func.now())
class WorkflowNodeExecution(Base):
"""Workflow node execution record"""
__tablename__ = 'workflow_node_executions'
id = sqlalchemy.Column(sqlalchemy.Integer, primary_key=True, autoincrement=True)
execution_uuid = sqlalchemy.Column(sqlalchemy.String(255), nullable=False, index=True)
node_id = sqlalchemy.Column(sqlalchemy.String(100), nullable=False)
node_type = sqlalchemy.Column(sqlalchemy.String(50), nullable=False)
status = sqlalchemy.Column(sqlalchemy.String(20), nullable=False) # pending, running, completed, failed, skipped
inputs = sqlalchemy.Column(sqlalchemy.JSON, nullable=True)
outputs = sqlalchemy.Column(sqlalchemy.JSON, nullable=True)
start_time = sqlalchemy.Column(sqlalchemy.DateTime, nullable=True)
end_time = sqlalchemy.Column(sqlalchemy.DateTime, nullable=True)
error = sqlalchemy.Column(sqlalchemy.Text, nullable=True)
retry_count = sqlalchemy.Column(sqlalchemy.Integer, nullable=False, default=0)
class ScheduledJob(Base):
"""Scheduled job for cron triggers"""
__tablename__ = 'workflow_scheduled_jobs'
uuid = sqlalchemy.Column(sqlalchemy.String(255), primary_key=True, unique=True)
trigger_uuid = sqlalchemy.Column(sqlalchemy.String(255), nullable=False, index=True)
cron_expression = sqlalchemy.Column(sqlalchemy.String(100), nullable=True)
next_run_time = sqlalchemy.Column(sqlalchemy.DateTime, nullable=True)
last_run_time = sqlalchemy.Column(sqlalchemy.DateTime, nullable=True)
is_enabled = sqlalchemy.Column(sqlalchemy.Boolean, nullable=False, default=True)

View File

@@ -1,158 +0,0 @@
"""Add workflow tables and update bot binding fields"""
import sqlalchemy
from .. import migration
@migration.migration_class(26)
class DBMigrateWorkflowTables(migration.DBMigration):
"""Add workflow tables and update bot binding fields"""
async def upgrade(self):
# Create workflows table
await self.ap.persistence_mgr.execute_async(
sqlalchemy.text("""
CREATE TABLE IF NOT EXISTS workflows (
uuid VARCHAR(255) PRIMARY KEY,
name VARCHAR(255) NOT NULL,
description TEXT,
emoji VARCHAR(10) DEFAULT '🔄',
version INTEGER NOT NULL DEFAULT 1,
is_enabled BOOLEAN NOT NULL DEFAULT 1,
definition JSON NOT NULL DEFAULT '{}',
global_config JSON NOT NULL DEFAULT '{}',
extensions_preferences JSON NOT NULL DEFAULT '{"enable_all_plugins": true, "enable_all_mcp_servers": true, "plugins": [], "mcp_servers": []}',
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
)
""")
)
# Create workflow_versions table
await self.ap.persistence_mgr.execute_async(
sqlalchemy.text("""
CREATE TABLE IF NOT EXISTS workflow_versions (
id INTEGER PRIMARY KEY AUTOINCREMENT,
workflow_uuid VARCHAR(255) NOT NULL,
version INTEGER NOT NULL,
definition JSON NOT NULL,
global_config JSON NOT NULL DEFAULT '{}',
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
created_by VARCHAR(255),
UNIQUE(workflow_uuid, version)
)
""")
)
# Create workflow_triggers table
await self.ap.persistence_mgr.execute_async(
sqlalchemy.text("""
CREATE TABLE IF NOT EXISTS workflow_triggers (
uuid VARCHAR(255) PRIMARY KEY,
workflow_uuid VARCHAR(255) NOT NULL,
type VARCHAR(50) NOT NULL,
config JSON NOT NULL DEFAULT '{}',
is_enabled BOOLEAN NOT NULL DEFAULT 1,
priority INTEGER NOT NULL DEFAULT 0,
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
)
""")
)
# Create workflow_executions table
await self.ap.persistence_mgr.execute_async(
sqlalchemy.text("""
CREATE TABLE IF NOT EXISTS workflow_executions (
uuid VARCHAR(255) PRIMARY KEY,
workflow_uuid VARCHAR(255) NOT NULL,
workflow_version INTEGER NOT NULL,
status VARCHAR(20) NOT NULL,
trigger_type VARCHAR(50),
trigger_data JSON,
variables JSON,
start_time TIMESTAMP,
end_time TIMESTAMP,
error TEXT,
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
)
""")
)
# Create workflow_node_executions table
await self.ap.persistence_mgr.execute_async(
sqlalchemy.text("""
CREATE TABLE IF NOT EXISTS workflow_node_executions (
id INTEGER PRIMARY KEY AUTOINCREMENT,
execution_uuid VARCHAR(255) NOT NULL,
node_id VARCHAR(100) NOT NULL,
node_type VARCHAR(50) NOT NULL,
status VARCHAR(20) NOT NULL,
inputs JSON,
outputs JSON,
start_time TIMESTAMP,
end_time TIMESTAMP,
error TEXT,
retry_count INTEGER NOT NULL DEFAULT 0
)
""")
)
# Create workflow_scheduled_jobs table
await self.ap.persistence_mgr.execute_async(
sqlalchemy.text("""
CREATE TABLE IF NOT EXISTS workflow_scheduled_jobs (
uuid VARCHAR(255) PRIMARY KEY,
trigger_uuid VARCHAR(255) NOT NULL,
cron_expression VARCHAR(100),
next_run_time TIMESTAMP,
last_run_time TIMESTAMP,
is_enabled BOOLEAN NOT NULL DEFAULT 1
)
""")
)
# Create indexes
await self.ap.persistence_mgr.execute_async(
sqlalchemy.text('CREATE INDEX IF NOT EXISTS idx_workflow_versions_uuid ON workflow_versions(workflow_uuid)')
)
await self.ap.persistence_mgr.execute_async(
sqlalchemy.text('CREATE INDEX IF NOT EXISTS idx_workflow_triggers_uuid ON workflow_triggers(workflow_uuid)')
)
await self.ap.persistence_mgr.execute_async(
sqlalchemy.text(
'CREATE INDEX IF NOT EXISTS idx_workflow_executions_uuid ON workflow_executions(workflow_uuid)'
)
)
await self.ap.persistence_mgr.execute_async(
sqlalchemy.text(
'CREATE INDEX IF NOT EXISTS idx_workflow_node_executions_uuid ON workflow_node_executions(execution_uuid)'
)
)
await self.ap.persistence_mgr.execute_async(
sqlalchemy.text(
'CREATE INDEX IF NOT EXISTS idx_workflow_scheduled_jobs_trigger ON workflow_scheduled_jobs(trigger_uuid)'
)
)
# Update bots table: add binding_type column (default to 'pipeline' for backward compatibility)
# Check if column exists first (SQLite doesn't support IF NOT EXISTS for columns)
try:
await self.ap.persistence_mgr.execute_async(sqlalchemy.text('SELECT binding_type FROM bots LIMIT 1'))
except Exception:
# Column doesn't exist, add it
await self.ap.persistence_mgr.execute_async(
sqlalchemy.text("ALTER TABLE bots ADD COLUMN binding_type VARCHAR(20) NOT NULL DEFAULT 'pipeline'")
)
async def downgrade(self):
# Drop tables in reverse order
await self.ap.persistence_mgr.execute_async(sqlalchemy.text('DROP TABLE IF EXISTS workflow_scheduled_jobs'))
await self.ap.persistence_mgr.execute_async(sqlalchemy.text('DROP TABLE IF EXISTS workflow_node_executions'))
await self.ap.persistence_mgr.execute_async(sqlalchemy.text('DROP TABLE IF EXISTS workflow_executions'))
await self.ap.persistence_mgr.execute_async(sqlalchemy.text('DROP TABLE IF EXISTS workflow_triggers'))
await self.ap.persistence_mgr.execute_async(sqlalchemy.text('DROP TABLE IF EXISTS workflow_versions'))
await self.ap.persistence_mgr.execute_async(sqlalchemy.text('DROP TABLE IF EXISTS workflows'))
# Remove binding_type column from bots (SQLite doesn't support DROP COLUMN directly)
# This would need a table recreation in SQLite, so we'll skip it in downgrade

View File

@@ -1,49 +0,0 @@
"""Add binding_uuid field to bots table and migrate data"""
import sqlalchemy
from .. import migration
@migration.migration_class(27)
class DBMigrateBotBindingFields(migration.DBMigration):
"""Add binding_uuid field to bots table and migrate existing data"""
async def upgrade(self):
# Add binding_uuid column to bots table
# Check if column exists first (SQLite doesn't support IF NOT EXISTS for columns)
try:
await self.ap.persistence_mgr.execute_async(sqlalchemy.text('SELECT binding_uuid FROM bots LIMIT 1'))
except Exception:
# Column doesn't exist, add it
await self.ap.persistence_mgr.execute_async(
sqlalchemy.text('ALTER TABLE bots ADD COLUMN binding_uuid VARCHAR(64)')
)
# Migrate existing data: copy use_pipeline_uuid to binding_uuid for records
# that have a pipeline bound and binding_uuid is not set yet
await self.ap.persistence_mgr.execute_async(
sqlalchemy.text("""
UPDATE bots
SET binding_uuid = use_pipeline_uuid
WHERE use_pipeline_uuid IS NOT NULL
AND use_pipeline_uuid != ''
AND (binding_uuid IS NULL OR binding_uuid = '')
""")
)
# Ensure binding_type is 'pipeline' for records that were migrated
await self.ap.persistence_mgr.execute_async(
sqlalchemy.text("""
UPDATE bots
SET binding_type = 'pipeline'
WHERE binding_uuid IS NOT NULL
AND binding_uuid != ''
AND (binding_type IS NULL OR binding_type = '')
""")
)
async def downgrade(self):
# SQLite doesn't support DROP COLUMN directly
# This would need a table recreation in SQLite, so we'll skip it in downgrade
# The column will remain but won't be used
pass

View File

@@ -13,7 +13,7 @@ import langbot_plugin.api.entities.builtin.platform.message as platform_message
import langbot_plugin.api.entities.builtin.platform.events as platform_events
import langbot_plugin.api.entities.events as events
from ..utils import importutil
from .config import coerce_pipeline_config
from .config_coercion import coerce_pipeline_config
import langbot_plugin.api.entities.builtin.provider.session as provider_session
import langbot_plugin.api.entities.builtin.pipeline.query as pipeline_query
@@ -284,9 +284,9 @@ class RuntimePipeline:
# Record query start and store message_id
message_id = ''
try:
from . import monitor
from . import monitoring_helper
message_id = await monitor.MonitoringHelper.record_query_start(
message_id = await monitoring_helper.MonitoringHelper.record_query_start(
ap=self.ap,
query=query,
bot_id=query.bot_uuid or 'unknown',
@@ -338,7 +338,7 @@ class RuntimePipeline:
# Record query success only if no error occurred during processing
if not query.variables.get('_monitoring_has_error', False):
try:
await monitor.MonitoringHelper.record_query_success(
await monitoring_helper.MonitoringHelper.record_query_success(
ap=self.ap,
message_id=message_id,
query=query,
@@ -348,7 +348,7 @@ class RuntimePipeline:
# Record bot response message
try:
await monitor.MonitoringHelper.record_query_response(
await monitoring_helper.MonitoringHelper.record_query_response(
ap=self.ap,
query=query,
bot_id=query.bot_uuid or 'unknown',
@@ -367,9 +367,9 @@ class RuntimePipeline:
# Record query error
try:
from . import monitor
from . import monitoring_helper
await monitor.MonitoringHelper.record_query_error(
await monitoring_helper.MonitoringHelper.record_query_error(
ap=self.ap,
query=query,
bot_id=query.bot_uuid or 'unknown',
@@ -384,8 +384,7 @@ class RuntimePipeline:
finally:
self.ap.logger.debug(f'Query {query.query_id} processed')
# Use pop with default to avoid KeyError if query was never cached
self.ap.query_pool.cached_queries.pop(query.query_id, None)
del self.ap.query_pool.cached_queries[query.query_id]
class PipelineManager:

View File

@@ -32,6 +32,9 @@ class PreProcessor(stage.PipelineStage):
) -> entities.StageProcessResult:
"""Process"""
selected_runner = query.pipeline_config['ai']['runner']['runner']
include_skill_authoring = (
selected_runner == 'local-agent' and getattr(self.ap, 'skill_service', None) is not None
)
session = await self.ap.sess_mgr.get_session(query)
@@ -110,7 +113,11 @@ class PreProcessor(stage.PipelineStage):
# Get bound plugins and MCP servers for filtering tools
bound_plugins = query.variables.get('_pipeline_bound_plugins', None)
bound_mcp_servers = query.variables.get('_pipeline_bound_mcp_servers', None)
query.use_funcs = await self.ap.tool_mgr.get_all_tools(bound_plugins, bound_mcp_servers)
query.use_funcs = await self.ap.tool_mgr.get_all_tools(
bound_plugins,
bound_mcp_servers,
include_skill_authoring=include_skill_authoring,
)
self.ap.logger.debug(f'Bound plugins: {bound_plugins}')
self.ap.logger.debug(f'Bound MCP servers: {bound_mcp_servers}')
@@ -121,7 +128,11 @@ class PreProcessor(stage.PipelineStage):
if not query.use_funcs and query.variables.get('_fallback_model_uuids'):
bound_plugins = query.variables.get('_pipeline_bound_plugins', None)
bound_mcp_servers = query.variables.get('_pipeline_bound_mcp_servers', None)
query.use_funcs = await self.ap.tool_mgr.get_all_tools(bound_plugins, bound_mcp_servers)
query.use_funcs = await self.ap.tool_mgr.get_all_tools(
bound_plugins,
bound_mcp_servers,
include_skill_authoring=include_skill_authoring,
)
sender_name = ''
@@ -237,4 +248,67 @@ class PreProcessor(stage.PipelineStage):
query.prompt.messages = event_ctx.event.default_prompt
query.messages = event_ctx.event.prompt
# =========== Skill awareness for the local-agent runner ===========
# The actual activation goes through the ``activate`` Tool Call so the
# LLM doesn't see full SKILL.md instructions until it commits to a
# skill (Claude Code's progressive disclosure). But the LLM still has
# to KNOW which skills exist to make that choice, so we:
# 1. resolve the pipeline's bound skills and stash them in
# ``query.variables['_pipeline_bound_skills']`` for downstream
# visibility checks (skill loader, native exec workdir);
# 2. inject a short ``Available Skills`` index (name + description
# only) into the system prompt. The contributor's original PR
# relied on this injection; without it the LLM never discovers
# the skills are there and just calls native tools instead.
if selected_runner == 'local-agent' and self.ap.skill_mgr:
pipeline_data = await self.ap.pipeline_service.get_pipeline(query.pipeline_uuid)
extensions_prefs = (pipeline_data or {}).get('extensions_preferences', {})
enable_all_skills = extensions_prefs.get('enable_all_skills', True)
if enable_all_skills:
bound_skills = None # None = all loaded skills are visible
else:
bound_skills = extensions_prefs.get('skills', [])
query.variables['_pipeline_bound_skills'] = bound_skills
skill_addition = self.ap.skill_mgr.build_skill_aware_prompt_addition(
bound_skills=bound_skills,
)
if skill_addition:
# Append to the first system message; create one if the
# prompt has none. Handles both plain-string and
# content-element (list) message bodies.
if query.prompt.messages and query.prompt.messages[0].role == 'system':
head = query.prompt.messages[0]
if isinstance(head.content, str):
head.content = head.content + skill_addition
elif isinstance(head.content, list):
appended = False
for ce in head.content:
if getattr(ce, 'type', None) == 'text':
ce.text = (ce.text or '') + skill_addition
appended = True
break
if not appended:
head.content.append(provider_message.ContentElement(type='text', text=skill_addition))
else:
query.prompt.messages.insert(
0,
provider_message.Message(role='system', content=skill_addition.strip()),
)
self.ap.logger.debug(
f'Skill index injected into system prompt: '
f'pipeline={query.pipeline_uuid} '
f'bound_skills={bound_skills or "all"} '
f'loaded_skills={len(self.ap.skill_mgr.skills)}'
)
else:
self.ap.logger.debug(
f'No skills available for prompt injection: '
f'pipeline={query.pipeline_uuid} '
f'loaded_skills={len(self.ap.skill_mgr.skills)} '
f'bound_skills={bound_skills}'
)
return entities.StageProcessResult(result_type=entities.ResultType.CONTINUE, new_query=query)

View File

@@ -5,6 +5,7 @@ import abc
from ...core import app
from .. import entities
import langbot_plugin.api.entities.builtin.pipeline.query as pipeline_query
import langbot_plugin.api.entities.builtin.provider.message as provider_message
class MessageHandler(metaclass=abc.ABCMeta):
@@ -31,3 +32,29 @@ class MessageHandler(metaclass=abc.ABCMeta):
if len(s0) > 20 or '\n' in s:
s0 = s0[:20] + '...'
return s0
def format_result_log(
self,
result: provider_message.Message | provider_message.MessageChunk,
) -> str | None:
if result.tool_calls:
tool_names = [tc.function.name for tc in result.tool_calls if tc.function and tc.function.name]
if tool_names:
return f'{result.role}: requested tools: {", ".join(tool_names)}'
return f'{result.role}: requested tool calls'
content = result.content
if isinstance(content, str):
if not content.strip():
return None
if result.role == 'tool':
if content.startswith('err:'):
return f'tool error: {self.cut_str(content)}'
return self.cut_str(result.readable_str())
if isinstance(content, list) and len(content) == 0:
return None
return self.cut_str(result.readable_str())

View File

@@ -113,9 +113,11 @@ class ChatMessageHandler(handler.MessageHandler):
# This prevents memory overflow from thousands of log entries per conversation
# First chunk uses INFO level to confirm connection establishment
if chunk_count == 1:
self.ap.logger.info(
f'Conversation({query.query_id}) Streaming started: {self.cut_str(result.readable_str())}'
)
summary = self.format_result_log(result)
if summary is not None:
self.ap.logger.info(f'Conversation({query.query_id}) Streaming started: {summary}')
else:
self.ap.logger.info(f'Conversation({query.query_id}) Streaming started')
elif chunk_count % 10 == 0:
self.ap.logger.debug(
f'Conversation({query.query_id}) Streaming chunk {chunk_count}: {self.cut_str(result.readable_str())}'
@@ -135,9 +137,9 @@ class ChatMessageHandler(handler.MessageHandler):
async for result in runner.run(query):
query.resp_messages.append(result)
self.ap.logger.info(
f'Conversation({query.query_id}) Response: {self.cut_str(result.readable_str())}'
)
summary = self.format_result_log(result)
if summary is not None:
self.ap.logger.info(f'Conversation({query.query_id}) Response: {summary}')
if result.content is not None:
text_length += len(result.content)

View File

@@ -2,6 +2,7 @@ from __future__ import annotations
import asyncio
import json
import re
import traceback
import sqlalchemy
@@ -53,24 +54,29 @@ class RuntimeBot:
self.task_context = taskmgr.TaskContext()
self.logger = logger
@staticmethod
def _match_operator(actual: str, operator: str, expected: str) -> bool:
"""Evaluate a single operator condition."""
if operator == 'eq':
return actual == expected
elif operator == 'neq':
return actual != expected
elif operator == 'contains':
return expected in actual
elif operator == 'not_contains':
return expected not in actual
elif operator == 'starts_with':
return actual.startswith(expected)
elif operator == 'regex':
try:
return bool(re.search(expected, actual))
except re.error:
return False
return False
PIPELINE_DISCARD = '__discard__'
PIPELINE_DISCARD_DISPLAY_NAME = 'Discarded'
def get_binding_info(self) -> tuple[str, str | None]:
"""Get the binding type and UUID for this bot.
Returns:
tuple: (binding_type, binding_uuid) where binding_type is 'pipeline' or 'workflow'
"""
binding_type = getattr(self.bot_entity, 'binding_type', 'pipeline') or 'pipeline'
binding_uuid = getattr(self.bot_entity, 'binding_uuid', None)
# Fallback to use_pipeline_uuid for backward compatibility
if not binding_uuid and binding_type == 'pipeline':
binding_uuid = self.bot_entity.use_pipeline_uuid
return binding_type, binding_uuid
def resolve_pipeline_uuid(
self,
launcher_type: str,
@@ -78,26 +84,56 @@ class RuntimeBot:
message_text: str,
message_element_types: list[str] | None = None,
) -> tuple[str | None, bool]:
"""Resolve pipeline UUID for message processing.
"""Resolve pipeline UUID based on routing rules.
NOTE: Routing rules have been removed. Bot now directly binds to a
Pipeline or Workflow. This method is kept for backward compatibility
but only returns the direct binding.
Rules are evaluated in order; first match wins.
Falls back to use_pipeline_uuid if no rule matches.
Rule types:
- launcher_type: session type ("person" / "group")
- launcher_id: session / group id
- message_content: message text content
- message_has_element: message contains element of given type
(Image, Voice, File, Forward, Face, At, AtAll, Quote)
Operators: eq (has), neq (doesn't have)
Operators: eq, neq, contains, not_contains, starts_with, regex
When pipeline_uuid is ``__discard__``, the message should be
silently dropped by the caller.
Returns:
tuple: (pipeline_uuid, routed_by_rule) - routed_by_rule is always False
as routing rules are no longer used.
tuple: (pipeline_uuid, routed_by_rule) - routed_by_rule is True
when a routing rule matched, False when falling back to default.
"""
binding_type, binding_uuid = self.get_binding_info()
rules = self.bot_entity.pipeline_routing_rules or []
element_type_set = set(message_element_types or [])
# If bound to workflow, return None for pipeline_uuid
# The caller should check binding_type and handle accordingly
if binding_type == 'workflow':
# For workflow binding, we still need to return something
# The actual workflow handling should be done by the caller
return None, False
for rule in rules:
rule_type = rule.get('type')
operator = rule.get('operator', 'eq')
rule_value = rule.get('value', '')
target_uuid = rule.get('pipeline_uuid')
if not rule_type or not target_uuid:
continue
return binding_uuid, False
if rule_type == 'launcher_type':
if self._match_operator(launcher_type, operator, rule_value):
return target_uuid, True
elif rule_type == 'launcher_id':
if self._match_operator(str(launcher_id), operator, str(rule_value)):
return target_uuid, True
elif rule_type == 'message_content':
if self._match_operator(message_text, operator, rule_value):
return target_uuid, True
elif rule_type == 'message_has_element':
has_element = rule_value in element_type_set
if operator == 'eq' and has_element:
return target_uuid, True
elif operator == 'neq' and not has_element:
return target_uuid, True
return self.bot_entity.use_pipeline_uuid, False
async def _record_discarded_message(
self,

View File

@@ -373,7 +373,6 @@ class WebSocketAdapter(abstract_platform_adapter.AbstractMessagePlatformAdapter)
"""
pipeline_uuid = connection.pipeline_uuid
session_type = connection.session_type
is_workflow = bool(connection.metadata.get('is_workflow'))
# 获取stream参数默认为True
self.stream_enabled = message_data.get('stream', True)
@@ -415,60 +414,6 @@ class WebSocketAdapter(abstract_platform_adapter.AbstractMessagePlatformAdapter)
session_type=session_type,
)
if is_workflow:
# 设置 pipeline_uuid以便工作流节点发送消息时能正确广播
self.ap.platform_mgr.websocket_proxy_bot.bot_entity.use_pipeline_uuid = pipeline_uuid
message_content = str(message_chain)
message_context = {
'message_id': str(message_id),
'message_content': message_content,
'sender_id': f'websocket_{connection.connection_id}',
'sender_name': 'User',
'platform': 'websocket',
'conversation_id': connection.connection_id,
'is_group': session_type == 'group',
'group_id': 'websocketgroup' if session_type == 'group' else None,
'mentions': [],
'reply_to': None,
'raw_message': {
'message': message_chain_obj,
'connection_id': connection.connection_id,
'session_type': session_type,
},
}
trigger_data = {
'message': message_content,
'message_chain': message_chain_obj,
'session_type': session_type,
'connection_id': connection.connection_id,
'message_context': message_context,
}
try:
from ...api.http.service.workflow import WorkflowExecutionFailedError
# Log workflow execution start (matching pipeline logging)
session_id = f'{session_type}_{connection.connection_id}'
logger.info(f'Processing request from {session_id} (0): {message_content}')
execution_id = await self.ap.workflow_service.execute_workflow(
pipeline_uuid,
trigger_type='message',
trigger_data=trigger_data,
session_id=session_id,
user_id=message_context['sender_id'],
bot_id=self.ap.platform_mgr.websocket_proxy_bot.bot_entity.uuid,
)
# Removed success broadcast - only show error on failure
except WorkflowExecutionFailedError as e:
await connection.send_queue.put({'type': 'error', 'message': e.message})
except Exception as e:
logger.error(f'Workflow websocket execution error: {e}', exc_info=True)
await connection.send_queue.put({'type': 'error', 'message': str(e)})
return
# 添加消息源
message_chain.insert(0, platform_message.Source(id=message_id, time=datetime.now().timestamp()))

View File

@@ -18,6 +18,7 @@ from langbot_plugin.api.entities.builtin.pipeline.query import provider_session
from ..core import app
from . import handler
from ..utils import platform
from ..utils.managed_runtime import ManagedRuntimeConnector
from langbot_plugin.runtime.io.controllers.stdio import (
client as stdio_client_controller,
)
@@ -39,11 +40,9 @@ class PluginRuntimeNotConnectedError(RuntimeError):
"""Raised when plugin runtime operations are requested before connection."""
class PluginRuntimeConnector:
class PluginRuntimeConnector(ManagedRuntimeConnector):
"""Plugin runtime connector"""
ap: app.Application
handler: handler.RuntimeConnectionHandler
handler_task: asyncio.Task
@@ -54,10 +53,6 @@ class PluginRuntimeConnector:
ctrl: stdio_client_controller.StdioClientController | ws_client_controller.WebSocketClientController
runtime_subprocess_on_windows: asyncio.subprocess.Process | None = None
runtime_subprocess_on_windows_task: asyncio.Task | None = None
runtime_disconnect_callback: typing.Callable[
[PluginRuntimeConnector], typing.Coroutine[typing.Any, typing.Any, None]
]
@@ -72,7 +67,7 @@ class PluginRuntimeConnector:
[PluginRuntimeConnector], typing.Coroutine[typing.Any, typing.Any, None]
],
):
self.ap = ap
super().__init__(ap)
self.runtime_disconnect_callback = runtime_disconnect_callback
self.is_enable_plugin = self.ap.instance_config.data.get('plugin', {}).get('enable', True)
@@ -140,19 +135,7 @@ class PluginRuntimeConnector:
# We have to launch runtime via cmd but communicate via ws.
self.ap.logger.info('(windows) use cmd to launch plugin runtime and communicate via ws')
if self.runtime_subprocess_on_windows is None: # only launch once
python_path = sys.executable
env = os.environ.copy()
self.runtime_subprocess_on_windows = await asyncio.create_subprocess_exec(
python_path,
'-m',
'langbot_plugin.cli.__init__',
'rt',
env=env,
)
# hold the process
self.runtime_subprocess_on_windows_task = asyncio.create_task(self.runtime_subprocess_on_windows.wait())
await self._start_runtime_subprocess('-m', 'langbot_plugin.cli.__init__', 'rt')
ws_url = 'ws://localhost:5400/control/ws'
@@ -236,6 +219,88 @@ class PluginRuntimeConnector:
return plugin_author, plugin_name
async def _install_mcp_from_marketplace(
self,
mcp_data: dict[str, Any],
task_context: taskmgr.TaskContext | None = None,
):
"""Install an MCP server from marketplace data."""
from ..entity.persistence import mcp as persistence_mcp
import uuid
config = mcp_data.get('config', {})
url = config.get('url', '')
# Use __ instead of / to avoid URL routing issues with slashes
name = f'{mcp_data.get("author", "")}__{mcp_data.get("name", "")}'
# Determine mode from URL
if 'sse' in url.lower():
mode = 'sse'
elif url.startswith('http'):
mode = 'http'
else:
mode = 'stdio'
# Build extra_args from config
extra_args = {
'url': url,
'timeout': config.get('timeout', 30),
'sse_read_timeout': config.get('sse_read_timeout', 300),
}
# Check if MCP server already exists
existing = await self.ap.persistence_mgr.execute_async(
sqlalchemy.select(persistence_mcp.MCPServer).where(persistence_mcp.MCPServer.name == name)
)
if existing.scalar_one_or_none():
self.ap.logger.info(f'MCP server {name} already exists, skipping installation')
return
# Create MCP server record
server_uuid = str(uuid.uuid4())
server_data = {
'uuid': server_uuid,
'name': name,
'enable': True,
'mode': mode,
'extra_args': extra_args,
}
await self.ap.persistence_mgr.execute_async(sqlalchemy.insert(persistence_mcp.MCPServer).values(server_data))
# Start the MCP server
result = await self.ap.persistence_mgr.execute_async(
sqlalchemy.select(persistence_mcp.MCPServer).where(persistence_mcp.MCPServer.uuid == server_uuid)
)
server_entity = result.first()
if server_entity:
server_config = self.ap.persistence_mgr.serialize_model(persistence_mcp.MCPServer, server_entity)
if self.ap.tool_mgr.mcp_tool_loader:
mcp_task = asyncio.create_task(self.ap.tool_mgr.mcp_tool_loader.host_mcp_server(server_config))
self.ap.tool_mgr.mcp_tool_loader._hosted_mcp_tasks.append(mcp_task)
self.ap.logger.info(f'Installed MCP server {name} from marketplace')
async def _install_skill_from_zip(
self,
file_bytes: bytes,
filename: str,
task_context: taskmgr.TaskContext | None = None,
):
"""Install a skill from marketplace ZIP data."""
from ..api.http.service.skill import SkillService
skill_service = SkillService(self.ap)
self.ap.logger.info(f'Installing skill from marketplace ZIP ({len(file_bytes)} bytes)')
# Install from ZIP using skill service
result = await skill_service.install_from_zip_upload(
file_bytes=file_bytes,
filename=filename + '.zip',
)
self.ap.logger.info(f'Skill installed successfully: {result}')
def _build_plugin_startup_failure_message(
self,
plugin_author: str,
@@ -298,6 +363,110 @@ class PluginRuntimeConnector:
plugin_author = install_info.get('plugin_author')
plugin_name = install_info.get('plugin_name')
if install_source == PluginInstallSource.MARKETPLACE:
# Handle marketplace plugin/mcp/skill installation
plugin_author = install_info.get('plugin_author', '')
plugin_name = install_info.get('plugin_name', '')
space_url = (
self.ap.instance_config.data.get('space', {}).get('url', 'https://space.langbot.app').rstrip('/')
)
# Try MCP endpoint first
async with httpx.AsyncClient(trust_env=True, timeout=15) as client:
mcp_resp = await client.get(f'{space_url}/api/v1/marketplace/mcps/{plugin_author}/{plugin_name}')
if mcp_resp.status_code == 200:
mcp_data = mcp_resp.json().get('data', {}).get('mcp', {})
if mcp_data.get('config'):
# It's an MCP - create server locally
self.ap.logger.info(f'Installing MCP from marketplace: {plugin_author}/{plugin_name}')
if task_context:
task_context.set_current_action('installing mcp server')
await self._install_mcp_from_marketplace(mcp_data, task_context)
return
else:
raise Exception(f'MCP {plugin_author}/{plugin_name} has no config')
elif mcp_resp.status_code == 404:
# Try skill endpoint - download ZIP and install
self.ap.logger.info(f'Trying skill endpoint for: {plugin_author}/{plugin_name}')
if task_context:
task_context.set_current_action('checking skill marketplace')
# Get skill detail to find version
skill_resp = await client.get(
f'{space_url}/api/v1/marketplace/skills/{plugin_author}/{plugin_name}'
)
if skill_resp.status_code == 200:
self.ap.logger.info(f'Installing skill from marketplace: {plugin_author}/{plugin_name}')
if task_context:
task_context.set_current_action('installing skill from marketplace')
# Download the skill ZIP (no version needed - uses latest)
if task_context:
task_context.set_current_action('downloading skill package')
download_resp = await client.get(
f'{space_url}/api/v1/marketplace/skills/download/{plugin_author}/{plugin_name}'
)
if download_resp.status_code != 200:
raise Exception(
f'Failed to download skill {plugin_author}/{plugin_name}: {download_resp.status_code}'
)
file_bytes = download_resp.content
file_size = len(file_bytes)
self.ap.logger.info(f'Downloaded skill ZIP ({file_size} bytes)')
# Install skill from ZIP using skill service
await self._install_skill_from_zip(file_bytes, f'{plugin_author}-{plugin_name}', task_context)
return
elif skill_resp.status_code == 404:
# Try plugin endpoint - get versions and download
self.ap.logger.info(f'Trying plugin endpoint for: {plugin_author}/{plugin_name}')
if task_context:
task_context.set_current_action('checking plugin marketplace')
# Get plugin versions to find latest
versions_resp = await client.get(
f'{space_url}/api/v1/marketplace/plugins/{plugin_author}/{plugin_name}/versions'
)
if versions_resp.status_code == 200:
versions_data = versions_resp.json().get('data', {}).get('versions', [])
if versions_data:
latest_version = versions_data[0].get('version', '')
if latest_version:
self.ap.logger.info(
f'Installing plugin from marketplace: {plugin_author}/{plugin_name} v{latest_version}'
)
if task_context:
task_context.set_current_action('downloading plugin package')
download_resp = await client.get(
f'{space_url}/api/v1/marketplace/plugins/download/{plugin_author}/{plugin_name}/{latest_version}'
)
if download_resp.status_code != 200:
raise Exception(
f'Failed to download plugin {plugin_author}/{plugin_name}: {download_resp.status_code}'
)
file_bytes = download_resp.content
self._extract_deps_metadata(file_bytes, task_context)
file_key = await self.handler.send_file(file_bytes, 'lbpkg')
install_info['plugin_file_key'] = file_key
self.ap.logger.info(f'Transfered file {file_key} to plugin runtime')
# Continue to install via runtime
else:
raise Exception(f'No version found for plugin {plugin_author}/{plugin_name}')
else:
raise Exception(f'Plugin {plugin_author}/{plugin_name} has no versions')
else:
raise Exception(f'Plugin {plugin_author}/{plugin_name} not found in marketplace')
else:
skill_resp.raise_for_status()
raise Exception(f'Failed to get skill {plugin_author}/{plugin_name}')
else:
mcp_resp.raise_for_status()
raise Exception(f'Failed to get MCP {plugin_author}/{plugin_name}')
if install_source == PluginInstallSource.LOCAL:
# transfer file before install
file_bytes = install_info['plugin_file']
@@ -613,13 +782,18 @@ class PluginRuntimeConnector:
return await self.handler.retrieve_knowledge(plugin_author, plugin_name, retriever_name, retrieval_context)
def dispose(self):
# No need to consider the shutdown on Windows
# for Windows can kill processes and subprocesses chainly
if self.is_enable_plugin and isinstance(self.ctrl, stdio_client_controller.StdioClientController):
# On non-Windows stdio mode, terminate via the controller's process handle.
# On Windows, the managed subprocess is cleaned up by the base class.
if (
self.is_enable_plugin
and hasattr(self, 'ctrl')
and isinstance(self.ctrl, stdio_client_controller.StdioClientController)
):
self.ap.logger.info('Terminating plugin runtime process...')
self.ctrl.process.terminate()
self._dispose_subprocess()
if self.heartbeat_task is not None:
self.heartbeat_task.cancel()
self.heartbeat_task = None

View File

@@ -84,7 +84,7 @@ class RuntimeProvider:
# Import monitoring helper
try:
from ...pipeline import monitor
from ...pipeline import monitoring_helper
# Get monitoring metadata from query variables
if query.variables:
@@ -96,7 +96,7 @@ class RuntimeProvider:
pipeline_name = 'Unknown'
message_id = None
await monitor.MonitoringHelper.record_llm_call(
await monitoring_helper.MonitoringHelper.record_llm_call(
ap=self.requester.ap,
query=query,
bot_id=query.bot_uuid or 'unknown',
@@ -154,7 +154,7 @@ class RuntimeProvider:
# Import monitoring helper
try:
from ...pipeline import monitor
from ...pipeline import monitoring_helper
# Get monitoring metadata from query variables
if query.variables:
@@ -166,7 +166,7 @@ class RuntimeProvider:
pipeline_name = 'Unknown'
message_id = None
await monitor.MonitoringHelper.record_llm_call(
await monitoring_helper.MonitoringHelper.record_llm_call(
ap=self.requester.ap,
query=query,
bot_id=query.bot_uuid or 'unknown',

View File

@@ -2,8 +2,12 @@ from __future__ import annotations
import abc
import typing
from typing import TYPE_CHECKING
from ..core import app
if TYPE_CHECKING:
from ..core import app
import langbot_plugin.api.entities.builtin.pipeline.query as pipeline_query
import langbot_plugin.api.entities.builtin.provider.message as provider_message
preregistered_runners: list[typing.Type[RequestRunner]] = []
@@ -35,7 +39,7 @@ class RequestRunner(abc.ABC):
@abc.abstractmethod
async def run(
self, query: core_entities.Query
) -> typing.AsyncGenerator[llm_entities.Message | llm_entities.MessageChunk, None]:
self, query: pipeline_query.Query
) -> typing.AsyncGenerator[provider_message.Message | provider_message.MessageChunk, None]:
"""运行请求"""
pass

View File

@@ -5,6 +5,7 @@ import copy
import typing
from .. import runner
from ..modelmgr import requester as modelmgr_requester
from ..tools.loaders.native import EXEC_TOOL_NAME
import langbot_plugin.api.entities.builtin.pipeline.query as pipeline_query
import langbot_plugin.api.entities.builtin.provider.message as provider_message
import langbot_plugin.api.entities.builtin.rag.context as rag_context
@@ -24,11 +25,37 @@ Respond in the same language as the user's input.
</user_message>
"""
SANDBOX_EXEC_TOOL_NAME = 'sandbox_exec'
SANDBOX_EXEC_SYSTEM_GUIDANCE = (
'When sandbox_exec is available, use it for exact calculations, statistics, structured data parsing, '
'and code execution instead of estimating mentally. If the user provides numbers, tables, CSV-like text, '
'JSON, or other data and asks for a computed answer, prefer running a short Python script in sandbox_exec '
'and then answer from the tool result.'
)
@runner.runner_class('local-agent')
class LocalAgentRunner(runner.RequestRunner):
"""Local agent request runner"""
def _build_request_messages(
self,
query: pipeline_query.Query,
user_message: provider_message.Message,
) -> list[provider_message.Message]:
req_messages = query.prompt.messages.copy() + query.messages.copy()
if any(getattr(tool, 'name', None) == EXEC_TOOL_NAME for tool in query.use_funcs or []):
req_messages.append(
provider_message.Message(
role='system',
content=self.ap.box_service.get_system_guidance(),
)
)
req_messages.append(user_message)
return req_messages
async def _get_model_candidates(
self,
query: pipeline_query.Query,
@@ -131,6 +158,7 @@ class LocalAgentRunner(runner.RequestRunner):
) -> typing.AsyncGenerator[provider_message.Message | provider_message.MessageChunk, None]:
"""Run request"""
pending_tool_calls = []
initial_response_emitted = False
# Get knowledge bases list from query variables (set by PreProcessor,
# may have been modified by plugins during PromptPreProcessing)
@@ -236,7 +264,7 @@ class LocalAgentRunner(runner.RequestRunner):
ce.text = final_user_message_text
break
req_messages = query.prompt.messages.copy() + query.messages.copy() + [user_message]
req_messages = self._build_request_messages(query, user_message)
try:
is_stream = await query.adapter.is_stream_output_supported()
@@ -264,7 +292,6 @@ class LocalAgentRunner(runner.RequestRunner):
query.use_funcs,
remove_think,
)
yield msg
final_msg = msg
else:
# Streaming: invoke with fallback
@@ -312,6 +339,7 @@ class LocalAgentRunner(runner.RequestRunner):
is_final=msg.is_final,
msg_sequence=msg_sequence,
)
initial_response_emitted = True
final_msg = provider_message.MessageChunk(
role=last_role,
@@ -325,6 +353,12 @@ class LocalAgentRunner(runner.RequestRunner):
if isinstance(final_msg, provider_message.MessageChunk):
first_end_sequence = final_msg.msg_sequence
if not is_stream:
yield final_msg
elif not initial_response_emitted:
yield final_msg
initial_response_emitted = True
req_messages.append(final_msg)
# Once a model succeeds, commit to it for the tool call loop
@@ -369,7 +403,15 @@ class LocalAgentRunner(runner.RequestRunner):
req_messages.append(msg)
except Exception as e:
err_msg = provider_message.Message(role='tool', content=f'err: {e}', tool_call_id=tool_call.id)
if is_stream:
err_msg = provider_message.MessageChunk(
role='tool',
content=f'err: {e}',
tool_call_id=tool_call.id,
is_final=True,
)
else:
err_msg = provider_message.Message(role='tool', content=f'err: {e}', tool_call_id=tool_call.id)
yield err_msg

View File

@@ -2,12 +2,14 @@ from __future__ import annotations
import abc
import typing
from typing import TYPE_CHECKING
from langbot_plugin.api.entities.events import pipeline_query
from ...core import app
import langbot_plugin.api.entities.builtin.resource.tool as resource_tool
if TYPE_CHECKING:
from ...core import app
preregistered_loaders: list[typing.Type[ToolLoader]] = []

View File

@@ -20,6 +20,7 @@ from ....core import app
import langbot_plugin.api.entities.builtin.resource.tool as resource_tool
import langbot_plugin.api.entities.builtin.provider.message as provider_message
from ....entity.persistence import mcp as persistence_mcp
from .mcp_stdio import BoxStdioSessionRuntime, MCPServerBoxConfig, MCPSessionErrorPhase # noqa: F401
class MCPSessionStatus(enum.Enum):
@@ -58,6 +59,12 @@ class RuntimeMCPSession:
error_message: str | None = None
error_phase: MCPSessionErrorPhase | None = None
retry_count: int = 0
_box_stdio_runtime: BoxStdioSessionRuntime
def __init__(self, server_name: str, server_config: dict, enable: bool, ap: app.Application):
self.server_name = server_name
self.server_uuid = server_config.get('uuid', '')
@@ -75,7 +82,33 @@ class RuntimeMCPSession:
self._shutdown_event = asyncio.Event()
self._ready_event = asyncio.Event()
self._box_stdio_runtime = BoxStdioSessionRuntime(self)
self.box_config = self._box_stdio_runtime.config
async def _init_stdio_python_server(self):
if self._uses_box_stdio():
await self._box_stdio_runtime.initialize()
return
# Box is configured (ap.box_service exists) but currently unavailable
# (disabled by config or connection failed). Refuse stdio MCP rather
# than silently falling through to host-stdio — the operator asked
# for the sandbox and the failure mode should be visible.
#
# Set ``error_phase = BOX_UNAVAILABLE`` BEFORE raising so the retry
# wrapper can short-circuit (retrying is pointless when Box is
# deliberately off) and the frontend can render a localized,
# actionable message instead of this raw RuntimeError. Keep the
# message itself short — the frontend ignores it for this phase.
box_service = getattr(self.ap, 'box_service', None)
if box_service is not None and not getattr(box_service, 'available', False):
self.error_phase = MCPSessionErrorPhase.BOX_UNAVAILABLE
if not getattr(box_service, 'enabled', True):
raise RuntimeError('box_disabled_in_config')
raise RuntimeError('box_unavailable')
# Legacy: no box_service installed at all (pre-Box dev mode). Fall
# through to host-stdio for backward compatibility.
server_params = StdioServerParameters(
command=self.server_config['command'],
args=self.server_config['args'],
@@ -90,6 +123,9 @@ class RuntimeMCPSession:
await self.session.initialize()
async def _init_box_stdio_server(self):
await self._box_stdio_runtime.initialize()
async def _init_sse_server(self):
sse_transport = await self.exit_stack.enter_async_context(
sse_client(
@@ -124,8 +160,11 @@ class RuntimeMCPSession:
await self.session.initialize()
_MAX_RETRIES = 3
_RETRY_DELAYS = [2, 4, 8]
async def _lifecycle_loop(self):
"""在后台任务中管理整个MCP会话的生命周期"""
"""Manage the full MCP session lifecycle in a background task."""
try:
if self.server_config['mode'] == 'stdio':
await self._init_stdio_python_server()
@@ -134,49 +173,109 @@ class RuntimeMCPSession:
elif self.server_config['mode'] == 'http':
await self._init_streamable_http_server()
else:
raise ValueError(f'无法识别 MCP 服务器类型: {self.server_name}: {self.server_config}')
raise ValueError(f'Unknown MCP server mode: {self.server_name}: {self.server_config}')
await self.refresh()
self.status = MCPSessionStatus.CONNECTED
# 通知start()方法连接已建立
# Notify start() that connection is established
self._ready_event.set()
# 等待shutdown信号
await self._shutdown_event.wait()
# Wait for shutdown signal, with optional health monitoring for Box stdio
if self._uses_box_stdio():
monitor_task = asyncio.create_task(self._box_stdio_runtime.monitor_process_health())
shutdown_task = asyncio.create_task(self._shutdown_event.wait())
done, pending = await asyncio.wait(
[shutdown_task, monitor_task],
return_when=asyncio.FIRST_COMPLETED,
)
for task in pending:
task.cancel()
for task in done:
if task is monitor_task and not self._shutdown_event.is_set():
self.error_phase = MCPSessionErrorPhase.RUNTIME
raise Exception('Box managed process exited unexpectedly')
else:
await self._shutdown_event.wait()
except Exception as e:
self.status = MCPSessionStatus.ERROR
self.error_message = str(e)
self.ap.logger.error(f'Error in MCP session lifecycle {self.server_name}: {e}\n{traceback.format_exc()}')
# 即使出错也要设置ready事件让start()方法知道初始化已完成
self._ready_event.set()
# Do NOT set _ready_event here — let _lifecycle_loop_with_retry
# handle retries first. It will set the event when all retries
# are exhausted or on success.
raise # Re-raise so _lifecycle_loop_with_retry can catch it
finally:
# 在同一个任务中清理所有资源
# Clean up all resources in the same task
try:
if self.exit_stack:
await self.exit_stack.aclose()
self.exit_stack = AsyncExitStack()
self.functions.clear()
self.session = None
except Exception as e:
self.ap.logger.error(f'Error cleaning up MCP session {self.server_name}: {e}\n{traceback.format_exc()}')
finally:
await self._cleanup_box_stdio_session()
async def _lifecycle_loop_with_retry(self):
"""Wrap _lifecycle_loop with retry and exponential backoff."""
for attempt in range(self._MAX_RETRIES + 1):
try:
await self._lifecycle_loop()
return # Normal shutdown, don't retry
except Exception as e:
self.retry_count = attempt + 1
if self._shutdown_event.is_set():
return # Shutdown requested, don't retry
# BOX_UNAVAILABLE is a deliberate refusal, not a transient
# failure — retrying produces log spam and a misleading
# "Failed after N attempts" message. Surface it immediately.
if self.error_phase == MCPSessionErrorPhase.BOX_UNAVAILABLE:
self.status = MCPSessionStatus.ERROR
self.error_message = str(e)
self._ready_event.set()
return
if attempt >= self._MAX_RETRIES:
self.status = MCPSessionStatus.ERROR
self.error_message = f'Failed after {self._MAX_RETRIES + 1} attempts: {e}'
self._ready_event.set()
return
delay = self._RETRY_DELAYS[attempt]
self.ap.logger.warning(
f'MCP session {self.server_name} failed (attempt {attempt + 1}), retrying in {delay}s: {e}'
)
await self._cleanup_box_stdio_session()
# Reset status for retry
self.status = MCPSessionStatus.CONNECTING
self.error_message = None
self.error_phase = None
await asyncio.sleep(delay)
_MONITOR_POLL_INTERVAL = 5
_MONITOR_MAX_CONSECUTIVE_ERRORS = 3
async def _monitor_box_process_health(self):
await self._box_stdio_runtime.monitor_process_health()
async def start(self):
if not self.enable:
return
# 创建后台任务来管理生命周期
self._lifecycle_task = asyncio.create_task(self._lifecycle_loop())
# Create background task for lifecycle management with retry
self._lifecycle_task = asyncio.create_task(self._lifecycle_loop_with_retry())
# 等待连接建立或失败(带超时)
# Wait for connection or failure (with timeout)
startup_timeout = (self.box_config.startup_timeout_sec + 30) if self._uses_box_stdio() else 30.0
try:
await asyncio.wait_for(self._ready_event.wait(), timeout=30.0)
await asyncio.wait_for(self._ready_event.wait(), timeout=startup_timeout)
except asyncio.TimeoutError:
self.status = MCPSessionStatus.ERROR
raise Exception('Connection timeout after 30 seconds')
raise Exception(f'Connection timeout after {startup_timeout} seconds')
# 检查是否有错误
# Check for errors
if self.status == MCPSessionStatus.ERROR:
raise Exception('Connection failed, please check URL')
@@ -232,18 +331,25 @@ class RuntimeMCPSession:
return self.functions
def get_runtime_info_dict(self) -> dict:
return {
info = {
'status': self.status.value,
'error_message': self.error_message,
'error_phase': self.error_phase.value if self.error_phase else None,
'retry_count': self.retry_count,
'tool_count': len(self.get_tools()),
'tools': [
{
'name': tool.name,
'description': tool.description,
'parameters': tool.parameters,
}
for tool in self.get_tools()
],
}
if self._uses_box_stdio():
info['box_session_id'] = self._build_box_session_id()
info['box_enabled'] = True
return info
async def shutdown(self):
"""关闭会话并清理资源"""
@@ -267,6 +373,41 @@ class RuntimeMCPSession:
except Exception as e:
self.ap.logger.error(f'Error shutting down MCP session {self.server_name}: {e}\n{traceback.format_exc()}')
def _uses_box_stdio(self) -> bool:
return self._box_stdio_runtime.uses_box_stdio()
def _build_box_session_id(self) -> str:
return 'mcp-shared'
def _rewrite_path(self, path: str, host_path: str | None) -> str:
return self._box_stdio_runtime.rewrite_path(path, host_path)
def _infer_host_path(self) -> str | None:
return self._box_stdio_runtime.infer_host_path()
@staticmethod
def _unwrap_venv_path(directory: str) -> str:
return BoxStdioSessionRuntime.unwrap_venv_path(directory)
def _resolve_host_path(self) -> str | None:
return self._box_stdio_runtime.resolve_host_path()
@staticmethod
def _detect_install_command(host_path: str) -> str | None:
return BoxStdioSessionRuntime.detect_install_command(host_path)
def _build_box_session_payload(self, session_id: str, host_path: str | None = None) -> dict:
return self._box_stdio_runtime.build_box_session_payload(session_id, host_path)
def _build_box_process_payload(self, host_path: str | None = None) -> dict:
return self._box_stdio_runtime.build_box_process_payload(host_path)
def _rewrite_venv_command(self, command: str, host_path: str) -> str:
return self._box_stdio_runtime.rewrite_venv_command(command, host_path)
async def _cleanup_box_stdio_session(self) -> None:
await self._box_stdio_runtime.cleanup_session()
# @loader.loader_class('mcp')
class MCPLoader(loader.ToolLoader):
@@ -332,7 +473,7 @@ class MCPLoader(loader.ToolLoader):
Args:
server_config: 服务器配置字典,必须包含:
- name: 服务器名称
- mode: 连接模式 (stdio/sse)
- mode: 连接模式 (stdio/sse/http)
- enable: 是否启用
- extra_args: 额外的配置参数 (可选)
"""
@@ -431,12 +572,13 @@ class MCPLoader(loader.ToolLoader):
"""获取所有服务器的信息"""
info = {}
for server_name, session in self.sessions.items():
tools = session.get_tools()
info[server_name] = {
'name': server_name,
'mode': session.server_config.get('mode'),
'enable': session.enable,
'tools_count': len(session.get_tools()),
'tool_names': [f.name for f in session.get_tools()],
'tools_count': len(tools),
'tool_names': [f.name for f in tools],
}
return info

View File

@@ -0,0 +1,366 @@
from __future__ import annotations
import enum
import asyncio
import os
import shutil
import shlex
from typing import TYPE_CHECKING, Any
import pydantic
from mcp import ClientSession
from mcp.client.websocket import websocket_client
from ....box.workspace import (
BoxWorkspaceSession,
classify_python_workspace,
infer_workspace_host_path,
normalize_host_path,
rewrite_mounted_path,
rewrite_venv_command,
unwrap_venv_path,
)
if TYPE_CHECKING:
from .mcp import RuntimeMCPSession
class MCPSessionErrorPhase(enum.Enum):
"""Which phase of the MCP lifecycle failed."""
SESSION_CREATE = 'session_create'
DEP_INSTALL = 'dep_install'
PROCESS_START = 'process_start'
RELAY_CONNECT = 'relay_connect'
MCP_INIT = 'mcp_init'
RUNTIME = 'runtime'
TOOL_CALL = 'tool_call'
# Stdio MCP refused because Box is disabled in config or currently
# unavailable. Not transient — retries would be pointless. The frontend
# uses this phase to render a localized actionable message instead of
# the raw RuntimeError text.
BOX_UNAVAILABLE = 'box_unavailable'
class MCPServerBoxConfig(pydantic.BaseModel):
"""Structured configuration for running an MCP server inside a Box container."""
image: str | None = None
network: str = 'on' # MCP servers need network for dependency installation
host_path: str | None = None
host_path_mode: str = 'ro' # MCP servers default to read-write mount only when explicitly requested
env: dict[str, str] = pydantic.Field(default_factory=dict)
startup_timeout_sec: int = 120 # Longer default to allow dependency bootstrap
cpus: float | None = None
memory_mb: int | None = None
pids_limit: int | None = None
read_only_rootfs: bool | None = None
model_config = pydantic.ConfigDict(extra='ignore')
class BoxStdioSessionRuntime:
"""Encapsulate Box-backed stdio MCP session orchestration."""
def __init__(self, owner: RuntimeMCPSession):
self.owner = owner
self.config = MCPServerBoxConfig.model_validate(owner.server_config.get('box', {}))
@property
def ap(self):
return self.owner.ap
@property
def server_name(self) -> str:
return self.owner.server_name
@property
def server_config(self) -> dict:
return self.owner.server_config
def _build_workspace(
self,
*,
host_path: str | None | object = ...,
workdir: str = '/workspace',
mount_path: str = '/workspace',
) -> BoxWorkspaceSession:
resolved_host_path = self.resolve_host_path() if host_path is ... else host_path
return BoxWorkspaceSession(
self.ap.box_service,
self.owner._build_box_session_id(),
host_path=resolved_host_path,
host_path_mode=self.config.host_path_mode,
workdir=workdir,
env=self.config.env,
mount_path=mount_path,
network=self.config.network,
read_only_rootfs=self.config.read_only_rootfs if self.config.read_only_rootfs is not None else False,
image=self.config.image,
cpus=self.config.cpus,
memory_mb=self.config.memory_mb,
pids_limit=self.config.pids_limit,
persistent=True,
)
@property
def process_id(self) -> str:
"""Each MCP server gets a unique process_id within the shared session."""
return self.owner.server_uuid
def uses_box_stdio(self) -> bool:
if self.server_config.get('mode') != 'stdio':
return False
box_service = getattr(self.ap, 'box_service', None)
if box_service is None:
return False
# When Box is configured but currently unavailable (disabled or
# connection failed), do NOT silently fall through to host-stdio —
# that would bypass the sandbox the operator asked for. The caller
# is expected to refuse the stdio MCP server with a clear error.
return bool(getattr(box_service, 'available', False))
async def initialize(self) -> None:
await self._wait_for_box_runtime()
# All stdio MCP servers share one Box session. Per-server host paths
# are staged into the shared workspace instead of becoming session
# mounts, because an existing Docker container cannot add bind mounts.
workspace = self._build_workspace(host_path=None)
host_path = self.resolve_host_path()
process_cwd = '/workspace'
try:
await workspace.create_session()
except Exception:
self.owner.error_phase = MCPSessionErrorPhase.SESSION_CREATE
raise
if host_path:
process_cwd = await self._stage_host_path_to_shared_workspace(host_path)
install_cmd = self.detect_install_command(host_path, process_cwd)
if install_cmd:
self.ap.logger.info(
f'MCP server {self.server_name}: installing dependencies in Box with: {install_cmd}'
)
try:
result = await workspace.execute_raw(
install_cmd,
workdir=process_cwd,
timeout_sec=self.config.startup_timeout_sec or 120,
)
except Exception:
self.owner.error_phase = MCPSessionErrorPhase.DEP_INSTALL
raise
if not result.ok:
self.owner.error_phase = MCPSessionErrorPhase.DEP_INSTALL
stderr_preview = (result.stderr or '')[:500]
raise Exception(f'Dependency install failed (exit code {result.exit_code}): {stderr_preview}')
try:
process_workspace = (
self._build_workspace(host_path=host_path, workdir=process_cwd, mount_path=process_cwd)
if host_path
else workspace
)
payload = process_workspace.build_process_payload(
self.server_config['command'],
self.server_config.get('args', []),
env=self.server_config.get('env', {}),
cwd=process_cwd,
)
payload['process_id'] = self.process_id
await workspace.box_service.start_managed_process(workspace.session_id, payload)
except Exception:
self.owner.error_phase = MCPSessionErrorPhase.PROCESS_START
raise
try:
websocket_url = workspace.get_managed_process_websocket_url(self.process_id)
transport = await self.owner.exit_stack.enter_async_context(websocket_client(websocket_url))
read_stream, write_stream = transport
self.owner.session = await self.owner.exit_stack.enter_async_context(
ClientSession(read_stream, write_stream)
)
except Exception:
self.owner.error_phase = MCPSessionErrorPhase.RELAY_CONNECT
raise
try:
await self.owner.session.initialize()
except Exception:
self.owner.error_phase = MCPSessionErrorPhase.MCP_INIT
raise
async def monitor_process_health(self) -> None:
from langbot_plugin.box.models import BoxManagedProcessStatus
workspace = self._build_workspace()
consecutive_errors = 0
while not self.owner._shutdown_event.is_set():
try:
info = await workspace.get_managed_process(self.process_id)
if isinstance(info, dict):
status = info.get('status', '')
else:
status = getattr(info, 'status', '')
if status == BoxManagedProcessStatus.EXITED.value or status == BoxManagedProcessStatus.EXITED:
return
consecutive_errors = 0
except Exception as exc:
consecutive_errors += 1
self.ap.logger.warning(
f'MCP monitor for {self.server_name}: get_managed_process failed '
f'({consecutive_errors}/{self.owner._MONITOR_MAX_CONSECUTIVE_ERRORS}): '
f'{type(exc).__name__}: {exc}'
)
if consecutive_errors >= self.owner._MONITOR_MAX_CONSECUTIVE_ERRORS:
return
await asyncio.sleep(self.owner._MONITOR_POLL_INTERVAL)
async def _stage_host_path_to_shared_workspace(self, host_path: str) -> str:
source_path = normalize_host_path(host_path)
if not source_path:
return '/workspace'
if not os.path.isdir(source_path):
raise FileNotFoundError(f'MCP host_path does not exist or is not a directory: {host_path}')
self._validate_host_path(source_path)
shared_host_path = self._shared_workspace_host_path()
process_host_root = os.path.join(shared_host_path, '.mcp', self.process_id)
process_host_workspace = os.path.join(process_host_root, 'workspace')
await asyncio.to_thread(self._copy_workspace_tree, source_path, process_host_root, process_host_workspace)
return f'/workspace/.mcp/{self.process_id}/workspace'
def _validate_host_path(self, host_path: str) -> None:
self.ap.box_service.build_spec(
{
'session_id': f'mcp-validate-{self.process_id}',
'host_path': host_path,
'host_path_mode': self.config.host_path_mode,
'network': self.config.network,
'read_only_rootfs': self.config.read_only_rootfs if self.config.read_only_rootfs is not None else False,
}
)
def _shared_workspace_host_path(self) -> str:
default_workspace = getattr(self.ap.box_service, 'default_workspace', None)
if not default_workspace:
raise RuntimeError('Box default workspace is required for shared MCP host_path staging')
shared_host_path = normalize_host_path(default_workspace)
os.makedirs(shared_host_path, exist_ok=True)
return shared_host_path
@staticmethod
def _copy_workspace_tree(source_path: str, process_host_root: str, process_host_workspace: str) -> None:
shutil.rmtree(process_host_root, ignore_errors=True)
os.makedirs(process_host_root, exist_ok=True)
shutil.copytree(
source_path,
process_host_workspace,
symlinks=True,
ignore=shutil.ignore_patterns('.git', '__pycache__', '.pytest_cache', '.mypy_cache', '.ruff_cache'),
)
async def _cleanup_staged_workspace(self) -> None:
if not self.resolve_host_path():
return
try:
process_host_root = os.path.join(self._shared_workspace_host_path(), '.mcp', self.process_id)
await asyncio.to_thread(shutil.rmtree, process_host_root, True)
except Exception as exc:
self.ap.logger.warning(
f'MCP server {self.server_name}: failed to clean staged workspace '
f'process_id={self.process_id}: {type(exc).__name__}: {exc}'
)
async def _wait_for_box_runtime(self) -> None:
timeout_sec = max(float(self.config.startup_timeout_sec or 120), 1.0)
deadline = asyncio.get_running_loop().time() + timeout_sec
warned = False
while not getattr(self.ap.box_service, 'available', False):
if not warned:
self.ap.logger.warning(
f'MCP server {self.server_name}: waiting for Box runtime before starting stdio process'
)
warned = True
if asyncio.get_running_loop().time() >= deadline:
self.owner.error_phase = MCPSessionErrorPhase.SESSION_CREATE
raise Exception(f'Box runtime is not available after {int(timeout_sec)} seconds')
await asyncio.sleep(1)
async def cleanup_session(self) -> None:
if not self.uses_box_stdio():
return
# In the shared-session model, we do not delete the session itself.
# Stop only this MCP server's managed process; deleting the session
# would kill other MCP servers sharing the same container.
workspace = self._build_workspace(host_path=None)
try:
await workspace.stop_managed_process(self.process_id)
except Exception as exc:
self.ap.logger.warning(
f'MCP server {self.server_name}: failed to stop managed process '
f'process_id={self.process_id}: {type(exc).__name__}: {exc}'
)
await self._cleanup_staged_workspace()
return
await self._cleanup_staged_workspace()
self.ap.logger.info(
f'MCP server {self.server_name}: stopped process_id={self.process_id} '
f'(shared session {self.owner._build_box_session_id()} kept alive)'
)
def rewrite_path(self, path: str, host_path: str | None) -> str:
return rewrite_mounted_path(path, host_path)
def infer_host_path(self) -> str | None:
return infer_workspace_host_path(self.server_config.get('command', ''), self.server_config.get('args', []))
@staticmethod
def unwrap_venv_path(directory: str) -> str:
return unwrap_venv_path(directory)
def resolve_host_path(self) -> str | None:
return self.config.host_path or self.infer_host_path()
@staticmethod
def detect_install_command(host_path: str, workspace_path: str = '/workspace') -> str | None:
workspace_kind = classify_python_workspace(host_path)
quoted_workspace_path = shlex.quote(workspace_path)
if workspace_kind == 'package':
return (
'mkdir -p /opt/_lb_src'
f' && tar -C {quoted_workspace_path}'
' --exclude=.venv --exclude=.git --exclude=__pycache__'
' --exclude=node_modules --exclude=.tox --exclude=.nox'
' --exclude="*.egg-info" --exclude=.uv-cache'
' -cf - .'
' | tar -C /opt/_lb_src -xf -'
' && pip install --no-cache-dir /opt/_lb_src'
' && rm -rf /opt/_lb_src'
)
if workspace_kind == 'requirements':
return f'pip install --no-cache-dir -r {quoted_workspace_path}/requirements.txt'
return None
def build_box_session_payload(self, session_id: str, host_path: str | None = None) -> dict[str, Any]:
workspace = self._build_workspace()
workspace.session_id = session_id
if host_path is not None:
workspace.host_path = host_path
return workspace.build_session_payload()
def build_box_process_payload(self, host_path: str | None = None) -> dict[str, Any]:
workspace = self._build_workspace()
if host_path is not None:
workspace.host_path = host_path
return workspace.build_process_payload(
self.server_config['command'],
self.server_config.get('args', []),
env=self.server_config.get('env', {}),
)
def rewrite_venv_command(self, command: str, host_path: str) -> str:
return rewrite_venv_command(command, host_path)

View File

@@ -0,0 +1,846 @@
from __future__ import annotations
import json
import os
import langbot_plugin.api.entities.builtin.resource.tool as resource_tool
from langbot_plugin.api.entities.events import pipeline_query
from .. import loader
from . import skill as skill_loader
EXEC_TOOL_NAME = 'exec'
READ_TOOL_NAME = 'read'
WRITE_TOOL_NAME = 'write'
EDIT_TOOL_NAME = 'edit'
GLOB_TOOL_NAME = 'glob'
GREP_TOOL_NAME = 'grep'
_ALL_TOOL_NAMES = {EXEC_TOOL_NAME, READ_TOOL_NAME, WRITE_TOOL_NAME, EDIT_TOOL_NAME, GLOB_TOOL_NAME, GREP_TOOL_NAME}
# Skip these dirs during grep walk to avoid noise
_SKIP_DIRS = {'.git', 'node_modules', '__pycache__', '.venv', 'venv', '.tox', 'dist', 'build'}
class NativeToolLoader(loader.ToolLoader):
def __init__(self, ap):
super().__init__(ap)
self._tools: list[resource_tool.LLMTool] | None = None
self._backend_available: bool | None = None
async def initialize(self):
"""Check if backend is truly available at startup."""
self._backend_available = await self._check_backend_available()
if self._backend_available:
self.ap.logger.info('Native sandbox tools (exec/read/write/edit/glob/grep) are available.')
else:
self.ap.logger.warning(
'Native sandbox tools (exec/read/write/edit/glob/grep) are NOT available. '
'No sandbox backend (Docker/nsjail/E2B) is ready. '
'The LLM will not have access to code execution or file operation tools.'
)
async def _check_backend_available(self) -> bool:
"""Check if the box backend is truly available (not just the runtime)."""
box_service = getattr(self.ap, 'box_service', None)
if box_service is None:
return False
if not getattr(box_service, 'available', False):
return False
# Check if backend is truly available via get_status
try:
status = await box_service.get_status()
backend_info = status.get('backend', {})
return backend_info.get('available', False)
except Exception:
return False
async def get_tools(self, bound_plugins: list[str] | None = None) -> list[resource_tool.LLMTool]:
if not self._is_sandbox_available():
return []
if self._tools is None:
self._tools = [
self._build_exec_tool(),
self._build_read_tool(),
self._build_write_tool(),
self._build_edit_tool(),
self._build_glob_tool(),
self._build_grep_tool(),
]
return list(self._tools)
async def has_tool(self, name: str) -> bool:
return name in _ALL_TOOL_NAMES and self._is_sandbox_available()
async def invoke_tool(self, name: str, parameters: dict, query: pipeline_query.Query):
if name == EXEC_TOOL_NAME:
self.ap.logger.info(
'exec tool invoked: '
f'query_id={query.query_id} '
f'parameters={json.dumps(self._summarize_parameters(parameters), ensure_ascii=False)}'
)
return await self._invoke_exec(parameters, query)
if name == READ_TOOL_NAME:
return await self._invoke_read(parameters, query)
if name == WRITE_TOOL_NAME:
return await self._invoke_write(parameters, query)
if name == EDIT_TOOL_NAME:
return await self._invoke_edit(parameters, query)
if name == GLOB_TOOL_NAME:
return await self._invoke_glob(parameters, query)
if name == GREP_TOOL_NAME:
return await self._invoke_grep(parameters, query)
raise ValueError(f'未找到工具: {name}')
async def shutdown(self):
pass
async def _invoke_exec(self, parameters: dict, query: pipeline_query.Query) -> dict:
command = str(parameters['command'])
workdir = str(parameters.get('workdir', '/workspace') or '/workspace')
# Validate that skill references target activated skills.
selected_skill, _ = skill_loader.resolve_virtual_skill_path(
self.ap,
query,
workdir,
include_visible=False,
include_activated=True,
)
referenced_skill_names = skill_loader.find_referenced_skill_names(command)
if selected_skill is None and referenced_skill_names:
if len(referenced_skill_names) > 1:
raise ValueError('exec can target at most one activated skill package per call.')
selected_skill = skill_loader.get_activated_skill(query, referenced_skill_names[0])
if selected_skill is None:
raise ValueError(
f'Skill "{referenced_skill_names[0]}" must be activated before exec can run in its package.'
)
if selected_skill is not None:
selected_skill_name = str(selected_skill.get('name', '') or '')
if referenced_skill_names and any(name != selected_skill_name for name in referenced_skill_names):
raise ValueError('exec can reference files from only one activated skill package per call.')
package_root = str(selected_skill.get('package_root', '') or '').strip()
if not package_root:
raise ValueError(f'Activated skill "{selected_skill_name}" has no package_root.')
# Wrap command with Python venv bootstrap if the skill has a Python project.
# The venv is created inside the skill's mount path.
skill_mount = f'/workspace/.skills/{selected_skill_name}'
if skill_loader.should_prepare_skill_python_env(package_root):
parameters = dict(parameters)
parameters['command'] = skill_loader.wrap_skill_command_with_python_env(command, mount_path=skill_mount)
# All exec calls (with or without skills) go through the same container
# via execute_tool. Skills are mounted at /workspace/.skills/{name}/
# via extra_mounts built by BoxService.
result = await self.ap.box_service.execute_tool(parameters, query)
if selected_skill is not None:
self._refresh_skill_from_disk(selected_skill)
return result
def _resolve_host_path(
self,
query: pipeline_query.Query,
sandbox_path: str,
*,
include_visible: bool,
include_activated: bool,
) -> tuple[str, dict | None]:
selected_skill, rewritten_path = skill_loader.resolve_virtual_skill_path(
self.ap,
query,
sandbox_path,
include_visible=include_visible,
include_activated=include_activated,
)
box_service = self.ap.box_service
host_root = selected_skill.get('package_root') if selected_skill is not None else box_service.default_workspace
if not host_root:
raise ValueError('No host workspace configured for file operations.')
mount_path = '/workspace'
if not rewritten_path.startswith(mount_path):
raise ValueError(f'Path must be under {mount_path}.')
relative = rewritten_path[len(mount_path) :].lstrip('/')
host_path = os.path.realpath(os.path.join(host_root, relative))
host_root = os.path.realpath(host_root)
if not (host_path == host_root or host_path.startswith(host_root + os.sep)):
raise ValueError('Path escapes the workspace boundary.')
return host_path, selected_skill
def _resolve_skill_relative_path(
self,
query: pipeline_query.Query,
sandbox_path: str,
*,
include_visible: bool,
include_activated: bool,
) -> tuple[dict, str] | None:
selected_skill, rewritten_path = skill_loader.resolve_virtual_skill_path(
self.ap,
query,
sandbox_path,
include_visible=include_visible,
include_activated=include_activated,
)
if selected_skill is None:
return None
mount_path = '/workspace'
if not rewritten_path.startswith(mount_path):
raise ValueError(f'Path must be under {mount_path}.')
relative = rewritten_path[len(mount_path) :].lstrip('/') or '.'
return selected_skill, relative
def _should_use_box_workspace_files(self, selected_skill: dict | None) -> bool:
if selected_skill is not None:
return False
box_service = getattr(self.ap, 'box_service', None)
if box_service is None or not hasattr(box_service, 'execute_tool'):
return False
default_workspace = getattr(box_service, 'default_workspace', None)
return bool(default_workspace and not os.path.isdir(os.path.realpath(default_workspace)))
async def _run_workspace_file_script(self, script: str, query: pipeline_query.Query) -> dict:
result = await self.ap.box_service.execute_tool(
{
'command': f"python - <<'PY'\n{script}\nPY",
'timeout_sec': 30,
},
query,
)
if not result.get('ok'):
return {'ok': False, 'error': result.get('stderr') or result.get('stdout') or 'Box execution failed'}
stdout = str(result.get('stdout') or '').strip()
try:
return json.loads(stdout.splitlines()[-1])
except Exception:
return {'ok': False, 'error': stdout or 'Box file operation returned no result'}
async def _read_workspace_via_box(self, path: str, query: pipeline_query.Query) -> dict:
script = f"""
import json, os
path = {json.dumps(path)}
if not path.startswith('/workspace'):
print(json.dumps({{'ok': False, 'error': 'Path must be under /workspace.'}}))
elif not os.path.exists(path):
print(json.dumps({{'ok': False, 'error': f'File not found: {{path}}'}}))
elif os.path.isdir(path):
print(json.dumps({{'ok': True, 'content': '\\n'.join(sorted(os.listdir(path))), 'is_directory': True}}))
else:
with open(path, 'r', encoding='utf-8', errors='replace') as f:
print(json.dumps({{'ok': True, 'content': f.read()}}))
""".strip()
return await self._run_workspace_file_script(script, query)
async def _write_workspace_via_box(self, path: str, content: str, query: pipeline_query.Query) -> dict:
script = f"""
import json, os
path = {json.dumps(path)}
content = {json.dumps(content)}
if not path.startswith('/workspace'):
print(json.dumps({{'ok': False, 'error': 'Path must be under /workspace.'}}))
else:
os.makedirs(os.path.dirname(path) or '/workspace', exist_ok=True)
with open(path, 'w', encoding='utf-8') as f:
f.write(content)
print(json.dumps({{'ok': True, 'path': path}}))
""".strip()
return await self._run_workspace_file_script(script, query)
async def _edit_workspace_via_box(
self,
path: str,
old_string: str,
new_string: str,
query: pipeline_query.Query,
) -> dict:
script = f"""
import json, os
path = {json.dumps(path)}
old_string = {json.dumps(old_string)}
new_string = {json.dumps(new_string)}
if not path.startswith('/workspace'):
print(json.dumps({{'ok': False, 'error': 'Path must be under /workspace.'}}))
elif not os.path.isfile(path):
print(json.dumps({{'ok': False, 'error': f'File not found: {{path}}'}}))
else:
with open(path, 'r', encoding='utf-8', errors='replace') as f:
content = f.read()
count = content.count(old_string)
if count == 0:
print(json.dumps({{'ok': False, 'error': 'old_string not found in file.'}}))
elif count > 1:
print(json.dumps({{'ok': False, 'error': f'old_string matches {{count}} locations; provide a more unique string.'}}))
else:
with open(path, 'w', encoding='utf-8') as f:
f.write(content.replace(old_string, new_string, 1))
print(json.dumps({{'ok': True, 'path': path}}))
""".strip()
return await self._run_workspace_file_script(script, query)
async def _glob_workspace_via_box(self, path: str, pattern: str, query: pipeline_query.Query) -> dict:
script = f"""
import json, os
from pathlib import Path
path = {json.dumps(path)}
pattern = {json.dumps(pattern)}
skip_dirs = {json.dumps(sorted(_SKIP_DIRS))}
if not path.startswith('/workspace'):
print(json.dumps({{'ok': False, 'error': 'Path must be under /workspace.'}}))
elif not os.path.isdir(path):
print(json.dumps({{'ok': False, 'error': f'Path is not a directory: {{path}}'}}))
else:
base = Path(path)
hits = [
item for item in base.rglob(pattern)
if not any(part in skip_dirs for part in item.parts)
]
hits.sort(key=lambda item: item.stat().st_mtime if item.exists() else 0, reverse=True)
shown = hits[:100]
matches = []
for item in shown:
rel = os.path.relpath(str(item), path)
matches.append(os.path.join(path, rel).replace(os.sep, '/'))
print(json.dumps({{'ok': True, 'matches': matches, 'total': len(hits), 'truncated': len(hits) > 100}}))
""".strip()
return await self._run_workspace_file_script(script, query)
async def _grep_workspace_via_box(
self,
path: str,
pattern: str,
include: str | None,
query: pipeline_query.Query,
) -> dict:
script = f"""
import json, os, re
from pathlib import Path
path = {json.dumps(path)}
pattern = {json.dumps(pattern)}
include = {json.dumps(include)}
skip_dirs = {json.dumps(sorted(_SKIP_DIRS))}
try:
regex = re.compile(pattern)
except re.error as exc:
print(json.dumps({{'ok': False, 'error': f'Invalid regex: {{exc}}'}}))
else:
if not path.startswith('/workspace'):
print(json.dumps({{'ok': False, 'error': 'Path must be under /workspace.'}}))
elif not os.path.exists(path):
print(json.dumps({{'ok': False, 'error': f'Path not found: {{path}}'}}))
else:
base = Path(path)
if base.is_file():
files = [base]
else:
files = []
for item in base.rglob(include or '*'):
if any(part in skip_dirs for part in item.parts):
continue
if item.is_file():
files.append(item)
if len(files) >= 5000:
break
matches = []
for fp in files:
try:
text = fp.read_text(errors='ignore')
except OSError:
continue
for lineno, line in enumerate(text.splitlines(), 1):
if regex.search(line):
if base.is_file():
file_path = path
else:
rel = os.path.relpath(str(fp), path)
file_path = os.path.join(path, rel).replace(os.sep, '/')
matches.append({{'file': file_path, 'line': lineno, 'content': line.rstrip()}})
if len(matches) >= 200:
break
if len(matches) >= 200:
break
print(json.dumps({{'ok': True, 'matches': matches, 'total': len(matches), 'truncated': len(matches) >= 200}}))
""".strip()
return await self._run_workspace_file_script(script, query)
async def _invoke_read(self, parameters: dict, query: pipeline_query.Query) -> dict:
path = parameters['path']
self.ap.logger.info(f'read tool invoked: query_id={query.query_id} path={path}')
skill_request = self._resolve_skill_relative_path(
query,
path,
include_visible=True,
include_activated=True,
)
if skill_request is not None and hasattr(self.ap.box_service, 'read_skill_file'):
selected_skill, relative = skill_request
try:
result = await self.ap.box_service.read_skill_file(selected_skill['name'], relative)
return {'ok': True, 'content': result.get('content', '')}
except Exception:
try:
result = await self.ap.box_service.list_skill_files(selected_skill['name'], relative)
entries = [entry['name'] for entry in result.get('entries', [])]
return {'ok': True, 'content': '\n'.join(sorted(entries)), 'is_directory': True}
except Exception as exc:
return {'ok': False, 'error': str(exc)}
host_path, selected_skill = self._resolve_host_path(
query,
path,
include_visible=True,
include_activated=True,
)
if self._should_use_box_workspace_files(selected_skill):
return await self._read_workspace_via_box(path, query)
if not os.path.exists(host_path):
return {'ok': False, 'error': f'File not found: {path}'}
if os.path.isdir(host_path):
entries = os.listdir(host_path)
return {'ok': True, 'content': '\n'.join(sorted(entries)), 'is_directory': True}
with open(host_path, 'r', errors='replace') as f:
content = f.read()
return {'ok': True, 'content': content}
async def _invoke_write(self, parameters: dict, query: pipeline_query.Query) -> dict:
path = parameters['path']
content = parameters['content']
self.ap.logger.info(f'write tool invoked: query_id={query.query_id} path={path} length={len(content)}')
skill_request = self._resolve_skill_relative_path(
query,
path,
include_visible=False,
include_activated=True,
)
if skill_request is not None and hasattr(self.ap.box_service, 'write_skill_file'):
selected_skill, relative = skill_request
await self.ap.box_service.write_skill_file(selected_skill['name'], relative, content)
await self.ap.skill_mgr.reload_skills()
return {'ok': True, 'path': path}
host_path, selected_skill = self._resolve_host_path(
query,
path,
include_visible=False,
include_activated=True,
)
if self._should_use_box_workspace_files(selected_skill):
return await self._write_workspace_via_box(path, content, query)
os.makedirs(os.path.dirname(host_path), exist_ok=True)
with open(host_path, 'w', encoding='utf-8') as f:
f.write(content)
self._refresh_skill_from_disk(selected_skill)
return {'ok': True, 'path': path}
async def _invoke_edit(self, parameters: dict, query: pipeline_query.Query) -> dict:
path = parameters['path']
old_string = parameters['old_string']
new_string = parameters['new_string']
self.ap.logger.info(
f'edit tool invoked: query_id={query.query_id} path={path} '
f'old_len={len(old_string)} new_len={len(new_string)}'
)
skill_request = self._resolve_skill_relative_path(
query,
path,
include_visible=False,
include_activated=True,
)
if (
skill_request is not None
and hasattr(self.ap.box_service, 'read_skill_file')
and hasattr(self.ap.box_service, 'write_skill_file')
):
selected_skill, relative = skill_request
try:
result = await self.ap.box_service.read_skill_file(selected_skill['name'], relative)
except Exception:
return {'ok': False, 'error': f'File not found: {path}'}
content = result.get('content', '')
count = content.count(old_string)
if count == 0:
return {'ok': False, 'error': 'old_string not found in file.'}
if count > 1:
return {'ok': False, 'error': f'old_string matches {count} locations; provide a more unique string.'}
new_content = content.replace(old_string, new_string, 1)
await self.ap.box_service.write_skill_file(selected_skill['name'], relative, new_content)
await self.ap.skill_mgr.reload_skills()
return {'ok': True, 'path': path}
host_path, selected_skill = self._resolve_host_path(
query,
path,
include_visible=False,
include_activated=True,
)
if self._should_use_box_workspace_files(selected_skill):
return await self._edit_workspace_via_box(path, old_string, new_string, query)
if not os.path.isfile(host_path):
return {'ok': False, 'error': f'File not found: {path}'}
with open(host_path, 'r', encoding='utf-8', errors='replace') as f:
content = f.read()
count = content.count(old_string)
if count == 0:
return {'ok': False, 'error': 'old_string not found in file.'}
if count > 1:
return {'ok': False, 'error': f'old_string matches {count} locations; provide a more unique string.'}
new_content = content.replace(old_string, new_string, 1)
with open(host_path, 'w', encoding='utf-8') as f:
f.write(new_content)
self._refresh_skill_from_disk(selected_skill)
return {'ok': True, 'path': path}
def _refresh_skill_from_disk(self, selected_skill: dict | None) -> None:
if selected_skill is None:
return
skill_mgr = getattr(self.ap, 'skill_mgr', None)
if skill_mgr is None:
return
refresh_skill = getattr(skill_mgr, 'refresh_skill_from_disk', None)
if callable(refresh_skill):
refresh_skill(selected_skill.get('name', ''))
def _is_sandbox_available(self) -> bool:
"""Check if sandbox backend is available.
This checks the cached backend availability from initialization,
not just whether the box_service process is running.
"""
return bool(self._backend_available)
def _build_exec_tool(self) -> resource_tool.LLMTool:
return resource_tool.LLMTool(
name=EXEC_TOOL_NAME,
human_desc='Execute a command in an isolated environment',
description=(
'Run shell commands in an isolated execution environment. '
'Use this tool for bash commands, Python execution, and exact calculations over '
'user-provided data. Activated skill packages are addressable under '
'/workspace/.skills/<skill-name>; when running inside one, set workdir to that path. '
'To create a new skill package, prepare it under /workspace first, then use register_skill.'
),
parameters={
'type': 'object',
'properties': {
'command': {
'type': 'string',
'description': 'Shell command to execute.',
},
'workdir': {
'type': 'string',
'description': 'Working directory for the command. Defaults to /workspace.',
'default': '/workspace',
},
'timeout_sec': {
'type': 'integer',
'description': 'Execution timeout in seconds. Defaults to 30.',
'default': 30,
'minimum': 1,
},
'env': {
'type': 'object',
'description': 'Optional environment variables for the execution.',
'additionalProperties': {'type': 'string'},
'default': {},
},
'description': {
'type': 'string',
'description': 'Brief description of what this command does, for logging and audit.',
},
},
'required': ['command'],
'additionalProperties': False,
},
func=lambda parameters: parameters,
)
def _build_read_tool(self) -> resource_tool.LLMTool:
return resource_tool.LLMTool(
name=READ_TOOL_NAME,
human_desc='Read a file from the workspace',
description=(
'Read the contents of a file at the given path under /workspace. '
'Visible skill packages can be inspected through /workspace/.skills/<skill-name>/... .'
),
parameters={
'type': 'object',
'properties': {
'path': {
'type': 'string',
'description': 'Absolute path to the file (must be under /workspace).',
},
},
'required': ['path'],
'additionalProperties': False,
},
func=lambda parameters: parameters,
)
def _build_write_tool(self) -> resource_tool.LLMTool:
return resource_tool.LLMTool(
name=WRITE_TOOL_NAME,
human_desc='Write a file to the workspace',
description=(
'Create or overwrite a file at the given path under /workspace with the provided content. '
'Activated skill packages can be modified through /workspace/.skills/<skill-name>/... . '
'For new skills, write files under /workspace and then call register_skill.'
),
parameters={
'type': 'object',
'properties': {
'path': {
'type': 'string',
'description': 'Absolute path to the file (must be under /workspace).',
},
'content': {
'type': 'string',
'description': 'Content to write to the file.',
},
},
'required': ['path', 'content'],
'additionalProperties': False,
},
func=lambda parameters: parameters,
)
def _build_edit_tool(self) -> resource_tool.LLMTool:
return resource_tool.LLMTool(
name=EDIT_TOOL_NAME,
human_desc='Edit a file in the workspace',
description=(
'Perform an exact string replacement in a file under /workspace. '
'The old_string must appear exactly once in the file. Activated skill packages '
'can be edited through /workspace/.skills/<skill-name>/... . '
'For new skills, edit files under /workspace and then call register_skill.'
),
parameters={
'type': 'object',
'properties': {
'path': {
'type': 'string',
'description': 'Absolute path to the file (must be under /workspace).',
},
'old_string': {
'type': 'string',
'description': 'The exact string to find and replace.',
},
'new_string': {
'type': 'string',
'description': 'The replacement string.',
},
},
'required': ['path', 'old_string', 'new_string'],
'additionalProperties': False,
},
func=lambda parameters: parameters,
)
def _build_glob_tool(self) -> resource_tool.LLMTool:
return resource_tool.LLMTool(
name=GLOB_TOOL_NAME,
human_desc='Find files matching a glob pattern',
description=(
'Find files matching a glob pattern under /workspace. '
'Supports ** for recursive matching (e.g. **/*.py). '
'Results are sorted by modification time (newest first). '
'Visible and activated skill packages can be searched through /workspace/.skills/<skill-name>/...'
),
parameters={
'type': 'object',
'properties': {
'pattern': {
'type': 'string',
'description': 'Glob pattern, e.g. **/*.py or src/**/*.ts',
},
'path': {
'type': 'string',
'description': 'Directory to search in (must be under /workspace, default: /workspace)',
'default': '/workspace',
},
},
'required': ['pattern'],
'additionalProperties': False,
},
func=lambda parameters: parameters,
)
def _build_grep_tool(self) -> resource_tool.LLMTool:
return resource_tool.LLMTool(
name=GREP_TOOL_NAME,
human_desc='Search file contents with regex',
description=(
'Search file contents with regex pattern under /workspace. '
'Returns matching lines with file path and line number. '
'Visible and activated skill packages can be searched through /workspace/.skills/<skill-name>/...'
),
parameters={
'type': 'object',
'properties': {
'pattern': {
'type': 'string',
'description': 'Regex pattern to search for',
},
'path': {
'type': 'string',
'description': 'File or directory to search (must be under /workspace, default: /workspace)',
'default': '/workspace',
},
'include': {
'type': 'string',
'description': 'Only search files matching this glob (e.g. *.py)',
},
},
'required': ['pattern'],
'additionalProperties': False,
},
func=lambda parameters: parameters,
)
async def _invoke_glob(self, parameters: dict, query: pipeline_query.Query) -> dict:
pattern = parameters['pattern']
path = str(parameters.get('path', '/workspace') or '/workspace')
self.ap.logger.info(f'glob tool invoked: query_id={query.query_id} pattern={pattern} path={path}')
host_path, selected_skill = self._resolve_host_path(
query,
path,
include_visible=True,
include_activated=True,
)
if self._should_use_box_workspace_files(selected_skill):
return await self._glob_workspace_via_box(path, pattern, query)
if not os.path.isdir(host_path):
return {'ok': False, 'error': f'Path is not a directory: {path}'}
from pathlib import Path
base = Path(host_path)
hits = list(base.rglob(pattern))
# Filter out skipped directories
hits = [h for h in hits if not any(skip in h.parts for skip in _SKIP_DIRS)]
# Sort by mtime, newest first
hits.sort(key=lambda p: p.stat().st_mtime if p.exists() else 0, reverse=True)
total = len(hits)
shown = hits[:100]
# Convert back to sandbox paths
sandbox_paths = []
for h in shown:
rel = os.path.relpath(str(h), host_path)
sandbox_path = os.path.join(path, rel)
sandbox_paths.append(sandbox_path)
result_lines = sandbox_paths
result = '\n'.join(result_lines)
if total > 100:
result += f'\n... ({total} matches, showing first 100)'
return {'ok': True, 'matches': result_lines, 'total': total, 'truncated': total > 100}
async def _invoke_grep(self, parameters: dict, query: pipeline_query.Query) -> dict:
pattern = parameters['pattern']
path = str(parameters.get('path', '/workspace') or '/workspace')
include = parameters.get('include')
self.ap.logger.info(f'grep tool invoked: query_id={query.query_id} pattern={pattern} path={path}')
import re
from pathlib import Path
try:
regex = re.compile(pattern)
except re.error as e:
return {'ok': False, 'error': f'Invalid regex: {e}'}
host_path, selected_skill = self._resolve_host_path(
query,
path,
include_visible=True,
include_activated=True,
)
if self._should_use_box_workspace_files(selected_skill):
return await self._grep_workspace_via_box(path, pattern, include, query)
if not os.path.exists(host_path):
return {'ok': False, 'error': f'Path not found: {path}'}
base = Path(host_path)
if base.is_file():
files = [base]
else:
files = self._grep_walk(base, include)
matches = []
for fp in files:
try:
text = fp.read_text(errors='ignore')
except OSError:
continue
for lineno, line in enumerate(text.splitlines(), 1):
if regex.search(line):
rel = os.path.relpath(str(fp), host_path)
sandbox_path = os.path.join(path, rel)
matches.append(
{
'file': sandbox_path,
'line': lineno,
'content': line.rstrip(),
}
)
if len(matches) >= 200:
break
if len(matches) >= 200:
break
return {
'ok': True,
'matches': matches,
'total': len(matches),
'truncated': len(matches) >= 200,
}
@staticmethod
def _grep_walk(root, include: str | None) -> list:
"""Walk dir tree for grep, skipping junk dirs."""
results = []
for item in root.rglob(include or '*'):
if any(skip in item.parts for skip in _SKIP_DIRS):
continue
if item.is_file():
results.append(item)
if len(results) >= 5000:
break
return results
def _summarize_parameters(self, parameters: dict) -> dict:
summary = dict(parameters)
cmd = str(summary.get('command', '')).strip()
if len(cmd) > 400:
cmd = f'{cmd[:397]}...'
summary['command'] = cmd
env = summary.get('env')
if isinstance(env, dict):
summary['env_keys'] = sorted(str(key) for key in env.keys())
del summary['env']
return summary

View File

@@ -0,0 +1,157 @@
from __future__ import annotations
import re
import typing
from ....box import workspace as box_workspace
if typing.TYPE_CHECKING:
from ....core import app
from langbot_plugin.api.entities.events import pipeline_query
ACTIVATED_SKILLS_KEY = '_activated_skills'
PIPELINE_BOUND_SKILLS_KEY = '_pipeline_bound_skills'
SKILL_MOUNT_PREFIX = '/workspace/.skills'
_SKILL_MOUNT_PATTERN = re.compile(r'/workspace/\.skills/([A-Za-z0-9_-]+)')
def get_virtual_skill_mount_path(skill_name: str) -> str:
return f'{SKILL_MOUNT_PREFIX}/{skill_name}'
def get_bound_skill_names(query: pipeline_query.Query) -> list[str] | None:
if query.variables is None:
return None
bound_skills = query.variables.get(PIPELINE_BOUND_SKILLS_KEY)
if bound_skills is None:
return None
if isinstance(bound_skills, list):
return [str(item) for item in bound_skills]
return None
def get_visible_skills(ap: app.Application, query: pipeline_query.Query) -> dict[str, dict]:
skill_mgr = getattr(ap, 'skill_mgr', None)
if skill_mgr is None:
return {}
visible_skills = getattr(skill_mgr, 'skills', {})
bound_skills = get_bound_skill_names(query)
if bound_skills is None:
return visible_skills
return {skill_name: skill_data for skill_name, skill_data in visible_skills.items() if skill_name in bound_skills}
def get_visible_skill(ap: app.Application, query: pipeline_query.Query, skill_name: str) -> dict | None:
return get_visible_skills(ap, query).get(skill_name)
def get_activated_skills(query: pipeline_query.Query) -> dict[str, dict]:
if query.variables is None:
return {}
activated = query.variables.get(ACTIVATED_SKILLS_KEY, {})
if not isinstance(activated, dict):
return {}
return activated
def get_activated_skill(query: pipeline_query.Query, skill_name: str) -> dict | None:
return get_activated_skills(query).get(skill_name)
def register_activated_skill(query: pipeline_query.Query, skill_data: dict) -> None:
if query.variables is None:
query.variables = {}
activated = query.variables.setdefault(ACTIVATED_SKILLS_KEY, {})
skill_name = str(skill_data.get('name', '') or '').strip()
if skill_name and skill_name not in activated:
activated[skill_name] = skill_data
def parse_skill_mount_path(sandbox_path: str) -> tuple[str | None, str]:
normalized_path = str(sandbox_path or '/workspace').strip() or '/workspace'
if normalized_path == SKILL_MOUNT_PREFIX:
raise ValueError(f'Path must include a skill name under {SKILL_MOUNT_PREFIX}/<skill-name>.')
prefix = f'{SKILL_MOUNT_PREFIX}/'
if not normalized_path.startswith(prefix):
return None, normalized_path
remainder = normalized_path[len(prefix) :]
skill_name, separator, tail = remainder.partition('/')
if not skill_name:
raise ValueError(f'Path must include a skill name under {SKILL_MOUNT_PREFIX}/<skill-name>.')
rewritten_path = '/workspace'
if separator:
rewritten_path = f'/workspace/{tail}'
return skill_name, rewritten_path
def resolve_virtual_skill_path(
ap: app.Application,
query: pipeline_query.Query,
sandbox_path: str,
*,
include_visible: bool,
include_activated: bool,
) -> tuple[dict | None, str]:
skill_name, rewritten_path = parse_skill_mount_path(sandbox_path)
if skill_name is None:
return None, rewritten_path
if include_activated:
activated_skill = get_activated_skill(query, skill_name)
if activated_skill is not None:
return activated_skill, rewritten_path
if include_visible:
visible_skill = get_visible_skill(ap, query, skill_name)
if visible_skill is not None:
return visible_skill, rewritten_path
activated_names = ', '.join(sorted(get_activated_skills(query).keys())) or 'none'
visible_names = ', '.join(sorted(get_visible_skills(ap, query).keys())) or 'none'
raise ValueError(
f'Skill "{skill_name}" is not available at this path. '
f'Activated skills: {activated_names}. Visible skills: {visible_names}.'
)
def find_referenced_skill_names(text: str) -> list[str]:
if not text:
return []
seen: list[str] = []
for match in _SKILL_MOUNT_PATTERN.findall(text):
if match not in seen:
seen.append(match)
return seen
def rewrite_command_for_skill_mount(command: str, skill_name: str) -> str:
virtual_root = get_virtual_skill_mount_path(skill_name)
rewritten = command.replace(f'{virtual_root}/', '/workspace/')
return rewritten.replace(virtual_root, '/workspace')
def build_skill_session_id(skill_data: dict, query: pipeline_query.Query) -> str:
skill_identifier = str(skill_data.get('name', 'unknown') or 'unknown')
launcher_type = getattr(query, 'launcher_type', None)
launcher_id = getattr(query, 'launcher_id', None)
query_id = getattr(query, 'query_id', 'unknown')
if launcher_type is not None and launcher_id is not None:
return f'skill-{launcher_type}_{launcher_id}-{skill_identifier}'
return f'skill-{query_id}-{skill_identifier}'
def should_prepare_skill_python_env(package_root: str | None) -> bool:
return box_workspace.should_prepare_python_env(package_root)
def wrap_skill_command_with_python_env(command: str, *, mount_path: str = '/workspace') -> str:
return box_workspace.wrap_python_command_with_env(command, mount_path=mount_path).rstrip()

View File

@@ -0,0 +1,304 @@
from __future__ import annotations
import os
import typing
import langbot_plugin.api.entities.builtin.resource.tool as resource_tool
from .. import loader
# Align with Claude Code's Skill tool design:
# - activate: Activate a skill via Tool Call, returns SKILL.md content
# - register_skill: Register a skill from sandbox directory to data/skills/
# - This protects KV Cache and follows industry standard
ACTIVATE_SKILL_TOOL_NAME = 'activate'
REGISTER_SKILL_TOOL_NAME = 'register_skill'
SKILL_TOOL_NAMES = {
ACTIVATE_SKILL_TOOL_NAME,
REGISTER_SKILL_TOOL_NAME,
}
class SkillToolLoader(loader.ToolLoader):
"""Skill tools aligned with Claude Code's design."""
def __init__(self, ap):
super().__init__(ap)
self._tools: list[resource_tool.LLMTool] = []
self._sandbox_available: bool = False
async def initialize(self):
# Check if sandbox backend is available (same check as native tools)
self._sandbox_available = await self._check_sandbox_available()
if self._sandbox_available:
self._tools = [
self._build_activate_skill_tool(),
self._build_register_skill_tool(),
]
else:
self.ap.logger.info(
'Skill tools (activate/register_skill) are NOT available. '
'No sandbox backend (Docker/nsjail/E2B) is ready.'
)
async def _check_sandbox_available(self) -> bool:
"""Check if the box backend is truly available (not just the runtime)."""
box_service = getattr(self.ap, 'box_service', None)
if box_service is None:
return False
if not getattr(box_service, 'available', False):
return False
# Check if backend is truly available via get_status
try:
status = await box_service.get_status()
backend_info = status.get('backend', {})
return backend_info.get('available', False)
except Exception:
return False
async def get_tools(self, bound_plugins: list[str] | None = None) -> list[resource_tool.LLMTool]:
if not self._is_available():
return []
return list(self._tools)
async def has_tool(self, name: str) -> bool:
return self._is_available() and name in SKILL_TOOL_NAMES
def _is_available(self) -> bool:
"""Check if skill tools should be available.
Skill tools require both a skill manager and a sandbox backend.
"""
return self._has_skill_manager() and self._sandbox_available
async def invoke_tool(self, name: str, parameters: dict, query) -> typing.Any:
if name == ACTIVATE_SKILL_TOOL_NAME:
return await self._invoke_activate_skill(parameters, query)
if name == REGISTER_SKILL_TOOL_NAME:
return await self._invoke_register_skill(parameters)
raise ValueError(f'Unknown skill tool: {name}')
async def shutdown(self):
pass
def _has_skill_manager(self) -> bool:
return getattr(self.ap, 'skill_mgr', None) is not None
async def _invoke_activate_skill(self, parameters: dict, query) -> typing.Any:
"""Activate a skill and return SKILL.md content via Tool Result."""
skill_name = str(parameters.get('skill_name', '') or '').strip()
if not skill_name:
raise ValueError('skill_name is required')
skill_mgr = self.ap.skill_mgr
skill_data = skill_mgr.get_skill_by_name(skill_name)
if skill_data is None:
visible_skills = getattr(skill_mgr, 'skills', {})
available_names = ', '.join(sorted(visible_skills.keys())) or 'none'
raise ValueError(f'Skill "{skill_name}" not found. Available skills: {available_names}')
# Register activated skill for sandbox mount path resolution
from . import skill as skill_loader
skill_loader.register_activated_skill(query, skill_data)
# Return SKILL.md content as Tool Result (injects into context)
instructions = skill_data.get('instructions', '')
package_root = skill_data.get('package_root', '')
mount_path = skill_loader.get_virtual_skill_mount_path(skill_name)
# Build Tool Result content
result_content = f'<command-message>The "{skill_name}" skill is activated</command-message>\n'
result_content += '<skill-activation>\n'
result_content += f'<skill-name>{skill_name}</skill-name>\n'
result_content += f'<mount-path>{mount_path}</mount-path>\n'
result_content += f'<package-root>{package_root}</package-root>\n'
result_content += f'\n## Instructions\n{instructions}\n'
result_content += '\n## Runtime Context\n'
result_content += f'The skill package is mounted at {mount_path}. Use the standard tools to interact with it:\n'
result_content += f'- Use `read` to inspect files under {mount_path}\n'
result_content += f'- Use `exec` with workdir set to {mount_path} to run commands in that package\n'
result_content += '- Use `write` and `edit` on that path when the instructions require updating files\n'
result_content += '</skill-activation>\n'
return {
'activated': True,
'skill_name': skill_name,
'mount_path': mount_path,
'content': result_content,
}
async def _invoke_register_skill(self, parameters: dict) -> typing.Any:
"""Register a skill from sandbox directory to data/skills/."""
sandbox_path = str(parameters.get('path', '') or '').strip()
if not sandbox_path:
raise ValueError('path is required')
# Resolve sandbox path to host path
host_path = self._resolve_workspace_directory(sandbox_path)
# Get or create skill service
skill_service = getattr(self.ap, 'skill_service', None)
if skill_service is None:
raise ValueError('Skill service not available')
# Scan and register the skill
scanned = await skill_service.scan_directory_async(host_path)
# Override name if provided
skill_name = str(parameters.get('name') or scanned['name']).strip()
if not skill_name:
raise ValueError('skill name is required')
# Create the skill
created = await skill_service.create_skill(
{
'name': skill_name,
'display_name': str(parameters.get('display_name') or scanned.get('display_name', '')).strip(),
'description': str(parameters.get('description') or scanned.get('description', '')).strip(),
'instructions': str(parameters.get('instructions') or scanned.get('instructions', '')),
'package_root': host_path,
}
)
return {
'registered': True,
'skill_name': skill_name,
'source_path': sandbox_path,
'skill': created,
}
def _resolve_workspace_directory(self, sandbox_path: str) -> str:
"""Resolve sandbox path to host filesystem path."""
box_service = getattr(self.ap, 'box_service', None)
workspace_root = getattr(box_service, 'default_workspace', None)
if not workspace_root:
raise ValueError('No default workspace configured')
normalized_path = str(sandbox_path).strip() or '/workspace'
if not normalized_path.startswith('/workspace'):
raise ValueError('path must be under /workspace')
relative = normalized_path[len('/workspace') :].lstrip('/')
host_root = os.path.realpath(workspace_root)
host_path = os.path.realpath(os.path.join(host_root, relative))
# Security check: ensure path doesn't escape workspace
if not (host_path == host_root or host_path.startswith(host_root + os.sep)):
raise ValueError('path escapes the workspace boundary')
if getattr(box_service, 'available', False):
return host_path
if not os.path.isdir(host_path):
raise ValueError(f'Directory does not exist: {sandbox_path}')
return host_path
def _build_activate_skill_tool(self) -> resource_tool.LLMTool:
return resource_tool.LLMTool(
name=ACTIVATE_SKILL_TOOL_NAME,
human_desc='Activate a skill',
description=self._build_activate_tool_description(),
parameters={
'type': 'object',
'properties': {
'skill_name': {
'type': 'string',
'description': 'The skill name to activate (no arguments). E.g., "pdf" or "data-analysis"',
},
},
'required': ['skill_name'],
'additionalProperties': False,
},
func=lambda parameters: parameters,
)
def _build_register_skill_tool(self) -> resource_tool.LLMTool:
return resource_tool.LLMTool(
name=REGISTER_SKILL_TOOL_NAME,
human_desc='Register a skill from sandbox',
description=(
"Register a skill package from a directory under /workspace into LangBot's skill store. "
'Use this after creating or preparing a skill in the sandbox with exec/read/write/edit. '
'The directory must contain a SKILL.md file. '
'After registration, the skill can be activated with the activate tool.'
),
parameters={
'type': 'object',
'properties': {
'path': {
'type': 'string',
'description': 'Directory path under /workspace containing the skill package (must have SKILL.md)',
},
'name': {
'type': 'string',
'description': 'Optional skill name override. Defaults to the name in SKILL.md or directory name.',
},
'display_name': {
'type': 'string',
'description': 'Optional display name override.',
},
'description': {
'type': 'string',
'description': 'Optional description override.',
},
'instructions': {
'type': 'string',
'description': 'Optional instructions override.',
},
},
'required': ['path'],
'additionalProperties': False,
},
func=lambda parameters: parameters,
)
def _build_activate_tool_description(self) -> str:
"""Build tool description with embedded available_skills list."""
skill_mgr = getattr(self.ap, 'skill_mgr', None)
if skill_mgr is None:
return 'Activate a skill. No skills are currently available.'
skills = getattr(skill_mgr, 'skills', {})
if not skills:
return 'Activate a skill. No skills are currently available.'
# Build <available_skills> section
available_skills_lines = ['<available_skills>']
for skill_name, skill_data in sorted(skills.items()):
description = skill_data.get('description', '')
available_skills_lines.append('<skill>')
available_skills_lines.append(f'<name>{skill_name}</name>')
available_skills_lines.append(f'<description>{description}</description>')
available_skills_lines.append('</skill>')
available_skills_lines.append('</available_skills>')
available_skills_block = '\n'.join(available_skills_lines)
return f"""Activate a skill within the main conversation.
<skills_instructions>
When users ask you to perform tasks, check if any of the available skills
below can help complete the task more effectively. Skills provide specialized
capabilities and domain knowledge.
How to use skills:
- Invoke skills using this tool with the skill name only (no arguments)
- When you invoke a skill, you will see <command-message>
The skill is activated
</command-message>
- The skill's instructions will be provided in the tool result
- Examples:
- skill_name: "pdf" - invoke the pdf skill
- skill_name: "data-analysis" - invoke the data-analysis skill
Important:
- Only use skills listed in <available_skills> below
- Do not invoke a skill that is already running
- To create a new skill: prepare it in /workspace, then use register_skill tool
</skills_instructions>
{available_skills_block}"""

View File

@@ -1,15 +1,19 @@
from __future__ import annotations
import typing
from typing import TYPE_CHECKING
from ...core import app
from langbot.pkg.utils import importutil
from langbot.pkg.provider.tools import loaders
from langbot.pkg.provider.tools.loaders import mcp as mcp_loader, plugin as plugin_loader
import langbot_plugin.api.entities.builtin.resource.tool as resource_tool
from langbot_plugin.api.entities.events import pipeline_query
importutil.import_modules_in_pkg(loaders)
if TYPE_CHECKING:
from ...core import app
from langbot.pkg.provider.tools.loaders import (
mcp as mcp_loader,
native as native_loader,
plugin as plugin_loader,
skill_authoring as skill_authoring_loader,
)
class ToolManager:
@@ -17,31 +21,53 @@ class ToolManager:
ap: app.Application
native_tool_loader: native_loader.NativeToolLoader
plugin_tool_loader: plugin_loader.PluginToolLoader
mcp_tool_loader: mcp_loader.MCPLoader
skill_tool_loader: skill_authoring_loader.SkillToolLoader
def __init__(self, ap: app.Application):
self.ap = ap
async def initialize(self):
from langbot.pkg.utils import importutil
from langbot.pkg.provider.tools import loaders
from langbot.pkg.provider.tools.loaders import (
mcp as mcp_loader,
native as native_loader,
plugin as plugin_loader,
skill_authoring as skill_authoring_loader,
)
importutil.import_modules_in_pkg(loaders)
self.native_tool_loader = native_loader.NativeToolLoader(self.ap)
await self.native_tool_loader.initialize()
self.plugin_tool_loader = plugin_loader.PluginToolLoader(self.ap)
await self.plugin_tool_loader.initialize()
self.mcp_tool_loader = mcp_loader.MCPLoader(self.ap)
await self.mcp_tool_loader.initialize()
self.skill_tool_loader = skill_authoring_loader.SkillToolLoader(self.ap)
await self.skill_tool_loader.initialize()
async def get_all_tools(
self, bound_plugins: list[str] | None = None, bound_mcp_servers: list[str] | None = None
self,
bound_plugins: list[str] | None = None,
bound_mcp_servers: list[str] | None = None,
include_skill_authoring: bool = False,
) -> list[resource_tool.LLMTool]:
"""获取所有函数"""
all_functions: list[resource_tool.LLMTool] = []
all_functions.extend(await self.native_tool_loader.get_tools())
if include_skill_authoring:
all_functions.extend(await self.skill_tool_loader.get_tools())
all_functions.extend(await self.plugin_tool_loader.get_tools(bound_plugins))
all_functions.extend(await self.mcp_tool_loader.get_tools(bound_mcp_servers))
return all_functions
async def generate_tools_for_openai(self, use_funcs: list[resource_tool.LLMTool]) -> list:
"""生成函数列表"""
tools = []
for function in use_funcs:
@@ -58,28 +84,6 @@ class ToolManager:
return tools
async def generate_tools_for_anthropic(self, use_funcs: list[resource_tool.LLMTool]) -> list:
"""为anthropic生成函数列表
e.g.
[
{
"name": "get_stock_price",
"description": "Get the current stock price for a given ticker symbol.",
"input_schema": {
"type": "object",
"properties": {
"ticker": {
"type": "string",
"description": "The stock ticker symbol, e.g. AAPL for Apple Inc."
}
},
"required": ["ticker"]
}
}
]
"""
tools = []
for function in use_funcs:
@@ -93,16 +97,18 @@ class ToolManager:
return tools
async def execute_func_call(self, name: str, parameters: dict, query: pipeline_query.Query) -> typing.Any:
"""执行函数调用"""
if await self.native_tool_loader.has_tool(name):
return await self.native_tool_loader.invoke_tool(name, parameters, query)
if await self.plugin_tool_loader.has_tool(name):
return await self.plugin_tool_loader.invoke_tool(name, parameters, query)
elif await self.mcp_tool_loader.has_tool(name):
if await self.mcp_tool_loader.has_tool(name):
return await self.mcp_tool_loader.invoke_tool(name, parameters, query)
else:
raise ValueError(f'未找到工具: {name}')
if await self.skill_tool_loader.has_tool(name):
return await self.skill_tool_loader.invoke_tool(name, parameters, query)
raise ValueError(f'未找到工具: {name}')
async def shutdown(self):
"""关闭所有工具"""
await self.native_tool_loader.shutdown()
await self.plugin_tool_loader.shutdown()
await self.mcp_tool_loader.shutdown()
await self.skill_tool_loader.shutdown()

View File

@@ -0,0 +1,3 @@
from .manager import SkillManager
__all__ = ['SkillManager']

View File

@@ -0,0 +1,35 @@
from __future__ import annotations
import typing
from ..provider.tools.loaders import skill as skill_loader
if typing.TYPE_CHECKING:
from ..core import app
import langbot_plugin.api.entities.builtin.pipeline.query as pipeline_query
# Skill activation is now handled through Tool Call mechanism (activate tool).
# This file is kept for potential future extensions but the text marker
# detection mechanism has been removed.
def register_activated_skill(
ap: app.Application,
query: pipeline_query.Query,
skill_name: str,
) -> bool:
"""Register an activated skill for sandbox mount path resolution.
This is called by the activate tool when a skill is activated via Tool Call.
"""
skill_mgr = getattr(ap, 'skill_mgr', None)
if skill_mgr is None:
return False
skill_data = skill_mgr.get_skill_by_name(skill_name)
if skill_data is None:
return False
skill_loader.register_activated_skill(query, skill_data)
return True

View File

@@ -0,0 +1,135 @@
from __future__ import annotations
import os
import typing
from ..core import app
if typing.TYPE_CHECKING:
pass
class SkillManager:
"""Skill manager backed by Box-managed or local filesystem packages.
In sandbox deployments, skills are loaded from the Box runtime. Local
data/skills remains as the fallback for non-Box development.
Skills are activated through the `activate` tool (Tool Call mechanism),
aligned with Claude Code's design. This protects KV Cache and follows
industry standard.
"""
ap: app.Application
skills: dict[str, dict]
def __init__(self, ap: app.Application):
self.ap = ap
self.skills = {}
async def initialize(self):
await self.reload_skills()
async def reload_skills(self):
"""Reload all skills from the Box runtime.
Box is the only source of truth for skills. When Box is unavailable
(disabled in config or unreachable) the cache is emptied — there is
no local filesystem fallback. Skills whose ``package_root`` is no
longer visible on the LangBot-side filesystem are dropped so they
don't surface as stale ``extra_mounts``.
"""
self.skills = {}
box_service = getattr(self.ap, 'box_service', None)
if box_service is None or not getattr(box_service, 'available', False):
self.ap.logger.info('Box runtime unavailable; skill cache is empty.')
return
try:
dropped = 0
for skill_data in await box_service.list_skills():
skill_name = skill_data.get('name')
if not skill_name:
continue
package_root = str(skill_data.get('package_root', '') or '').strip()
if package_root and not os.path.isdir(package_root):
self.ap.logger.warning(
f'Skill "{skill_name}" reported by Box runtime but '
f'package_root missing on LangBot filesystem '
f'({package_root}); dropping from in-memory cache.'
)
dropped += 1
continue
self.skills[skill_name] = skill_data
if dropped:
self.ap.logger.warning(
f'Loaded {len(self.skills)} skills from Box runtime '
f'({dropped} dropped due to missing package_root).'
)
else:
self.ap.logger.info(f'Loaded {len(self.skills)} skills from Box runtime')
except Exception as exc:
self.ap.logger.warning(f'Failed to load skills from Box runtime: {exc}')
def refresh_skill_from_disk(self, skill_name: str) -> bool:
"""Confirm a single skill is present in the cache.
With Box as the only source of truth, the actual reload is driven by
SkillService callers awaiting ``reload_skills``; this method only
reports whether the cache still has the skill.
"""
if not skill_name:
return False
return skill_name in self.skills
def get_skill_by_name(self, name: str) -> dict | None:
"""Get skill data by name."""
return self.skills.get(name)
def get_skill_index(self, bound_skills: list[str] | None = None) -> str:
"""Render the pipeline-visible skills as a short ``name: description``
index suitable for the system prompt.
``bound_skills`` follows the same convention as
``query.variables['_pipeline_bound_skills']``: ``None`` means every
loaded skill is exposed; an explicit list filters to that subset.
Returns an empty string when no skills are visible.
"""
lines: list[str] = []
for skill in self.skills.values():
name = skill.get('name')
if not name:
continue
if bound_skills is not None and name not in bound_skills:
continue
display = skill.get('display_name') or name
description = (skill.get('description') or '').strip().replace('\n', ' ')
lines.append(f'- {name} ({display}): {description}')
if not lines:
return ''
return 'Available Skills:\n' + '\n'.join(lines)
def build_skill_aware_prompt_addition(self, bound_skills: list[str] | None = None) -> str:
"""Build the system-prompt addendum that makes the LLM aware of the
pipeline-visible skills.
Only metadata (name + description) is injected — the full SKILL.md is
loaded later via the ``activate`` Tool Call, protecting KV cache and
matching Claude Code's progressive disclosure pattern. Returns an
empty string when no skills are visible (no prompt change at all).
"""
skill_index = self.get_skill_index(bound_skills)
if not skill_index:
return ''
return (
'\n\n'
f'{skill_index}\n\n'
"When the user's request clearly matches one or more skills "
'based on their descriptions above, call the `activate` tool with '
'the skill name to load its full instructions. Only the name and '
'description are visible here; the actual instructions arrive as '
'the tool result. If no skill is a clear match, respond normally '
'without activating any skill.'
)

View File

@@ -0,0 +1,37 @@
"""Shared utilities for skill file parsing."""
import yaml
def parse_frontmatter(content: str) -> tuple[dict, str]:
"""Parse YAML frontmatter from markdown content.
Expects format:
---
name: my-skill
description: Does something
---
# Actual instructions...
Returns:
Tuple of (metadata dict, remaining content)
"""
if not content.startswith('---'):
return {}, content
parts = content.split('---', 2)
if len(parts) < 3:
return {}, content
frontmatter_str = parts[1].strip()
instructions = parts[2].strip()
try:
metadata = yaml.safe_load(frontmatter_str) or {}
except yaml.YAMLError:
metadata = {}
if not isinstance(metadata, dict):
metadata = {}
return metadata, instructions

View File

@@ -0,0 +1,88 @@
"""Base class for connectors that may manage a local runtime subprocess."""
from __future__ import annotations
import asyncio
import os
import sys
from typing import TYPE_CHECKING, Awaitable, Callable
if TYPE_CHECKING:
from ..core import app as core_app
class ManagedRuntimeConnector:
"""Base class for connectors that may manage a local runtime subprocess.
Provides shared lifecycle helpers: subprocess launch, health-check retry,
and graceful termination. Concrete connectors (plugin, box, …) inherit
this and add their own protocol-specific logic.
"""
ap: 'core_app.Application'
runtime_subprocess: asyncio.subprocess.Process | None
runtime_subprocess_task: asyncio.Task | None
def __init__(self, ap: 'core_app.Application'):
self.ap = ap
self.runtime_subprocess = None
self.runtime_subprocess_task = None
async def _start_runtime_subprocess(self, *args: str) -> None:
"""Launch a local runtime as a subprocess of the current Python interpreter.
If a subprocess is already running (no *returncode* yet), this is a no-op.
"""
if self.runtime_subprocess is not None and self.runtime_subprocess.returncode is None:
return
python_path = sys.executable
env = os.environ.copy()
self.runtime_subprocess = await asyncio.create_subprocess_exec(
python_path,
*args,
env=env,
)
self.runtime_subprocess_task = asyncio.create_task(self.runtime_subprocess.wait())
async def _wait_until_ready(
self,
check: Callable[[], Awaitable[None]],
retries: int = 40,
interval: float = 0.25,
runtime_name: str = 'runtime',
) -> None:
"""Repeatedly call *check* until it succeeds or retries are exhausted.
Between attempts the method sleeps for *interval* seconds. If the
managed subprocess exits before readiness is confirmed, a
``RuntimeError`` is raised immediately.
"""
last_exc: Exception | None = None
for _ in range(retries):
# Fast-fail if the process already died.
if self.runtime_subprocess is not None and self.runtime_subprocess.returncode is not None:
raise RuntimeError(
f'local {runtime_name} exited before becoming ready (code {self.runtime_subprocess.returncode})'
)
try:
await check()
return
except Exception as exc:
last_exc = exc
await asyncio.sleep(interval)
if last_exc is not None:
raise last_exc
raise RuntimeError(f'local {runtime_name} did not become ready')
def _dispose_subprocess(self) -> None:
"""Terminate the managed subprocess and cancel its wait task."""
if self.runtime_subprocess is not None and self.runtime_subprocess.returncode is None:
self.ap.logger.info('Terminating managed runtime process...')
self.runtime_subprocess.terminate()
if self.runtime_subprocess_task is not None:
self.runtime_subprocess_task.cancel()
self.runtime_subprocess_task = None

View File

@@ -1,37 +1,70 @@
"""Utility functions for finding package resources"""
"""Utility functions for finding package resources and runtime data roots."""
import os
from pathlib import Path
_is_source_install = None
_source_root = None
def _find_source_root() -> Path | None:
"""Locate the LangBot repository root when running from source."""
global _source_root
if _source_root is not None:
return _source_root
current = Path(__file__).resolve()
for parent in current.parents:
if (parent / 'pyproject.toml').exists() and (parent / 'main.py').exists():
_source_root = parent
return parent
_source_root = None
return None
def _check_if_source_install() -> bool:
"""
Check if we're running from source directory or an installed package.
Cached to avoid repeated file I/O.
Check if we're running from the LangBot source tree.
Cached to avoid repeated filesystem scans.
"""
global _is_source_install
if _is_source_install is not None:
return _is_source_install
# Check if main.py exists in current directory with LangBot marker
if os.path.exists('main.py'):
try:
with open('main.py', 'r', encoding='utf-8') as f:
# Only read first 500 chars to check for marker
content = f.read(500)
if 'LangBot/main.py' in content:
_is_source_install = True
return True
except (IOError, OSError, UnicodeDecodeError):
# If we can't read the file, assume not a source install
pass
_is_source_install = _find_source_root() is not None
return _is_source_install
_is_source_install = False
return False
def get_data_root() -> str:
"""
Get the runtime data root.
Priority:
1. LANGBOT_DATA_ROOT environment override
2. Source checkout root /data when running from source
3. Current working directory /data for installed-package usage
"""
env_root = os.environ.get('LANGBOT_DATA_ROOT', '').strip()
if env_root:
return str(Path(env_root).expanduser().resolve())
source_root = _find_source_root()
if source_root is not None:
return str((source_root / 'data').resolve())
return str((Path.cwd() / 'data').resolve())
def get_data_path(*parts: str) -> str:
"""Join path segments under the resolved data root."""
data_root = Path(get_data_root())
if not parts:
return str(data_root)
return str((data_root.joinpath(*parts)).resolve())
def get_frontend_path() -> str:
@@ -76,8 +109,11 @@ def get_resource_path(resource: str) -> str:
Absolute path to the resource
"""
# First, check if resource exists in current directory (source install)
if _check_if_source_install() and os.path.exists(resource):
return resource
source_root = _find_source_root()
if source_root is not None:
source_resource = source_root / resource
if source_resource.exists():
return str(source_resource)
# Second, check current directory anyway
if os.path.exists(resource):

View File

@@ -16,7 +16,14 @@ def get_platform() -> str:
standalone_runtime = False
standalone_box = False
def use_websocket_to_connect_plugin_runtime() -> bool:
"""是否使用 websocket 连接插件运行时"""
return standalone_runtime
def use_websocket_to_connect_box_runtime() -> bool:
"""Whether to use WebSocket to connect to an external box runtime."""
return standalone_box

View File

@@ -1,6 +1,5 @@
from __future__ import annotations
import os
import typing
import logging
@@ -11,7 +10,7 @@ from . import constants
class VersionManager:
"""版本管理器"""
"""Version manager"""
ap: app.Application
@@ -22,190 +21,68 @@ class VersionManager:
pass
def get_current_version(self) -> str:
current_tag = constants.semantic_version
return current_tag
return constants.semantic_version
async def get_release_list(self) -> list:
"""获取发行列表"""
"""Fetch release list from Space API (cached GitHub releases)."""
try:
rls_list_resp = requests.get(
url='https://api.github.com/repos/langbot-app/LangBot/releases',
url='https://space.langbot.app/api/v1/dist/info/releases',
proxies=self.ap.proxy_mgr.get_forward_proxies(),
timeout=5,
timeout=10,
)
rls_list_resp.raise_for_status() # 检查请求是否成功
rls_list = rls_list_resp.json()
return rls_list
rls_list_resp.raise_for_status()
resp_json = rls_list_resp.json()
if resp_json.get('code') == 0 and isinstance(resp_json.get('data'), list):
return resp_json['data']
self.ap.logger.warning(f'Failed to fetch release list: unexpected response: {resp_json.get("msg", "")}')
return []
except Exception as e:
self.ap.logger.warning(f'获取发行列表失败: {e}')
pass
self.ap.logger.warning(f'Failed to fetch release list: {e}')
return []
async def update_all(self):
"""检查更新并下载源码"""
current_tag = self.get_current_version()
rls_list = await self.get_release_list()
latest_rls = {}
rls_notes = []
latest_tag_name = ''
for rls in rls_list:
rls_notes.append(rls['name']) # 使用发行名称作为note
if latest_tag_name == '':
latest_tag_name = rls['tag_name']
if rls['tag_name'] == current_tag:
break
if latest_rls == {}:
latest_rls = rls
self.ap.logger.info('更新日志: {}'.format(rls_notes))
if latest_rls == {} and not self.is_newer(latest_tag_name, current_tag): # 没有新版本
return False
# 下载最新版本的zip到temp目录
self.ap.logger.info('开始下载最新版本: {}'.format(latest_rls['zipball_url']))
zip_url = latest_rls['zipball_url']
zip_resp = requests.get(url=zip_url, proxies=self.ap.proxy_mgr.get_forward_proxies())
zip_data = zip_resp.content
# 检查temp/updater目录
if not os.path.exists('temp'):
os.mkdir('temp')
if not os.path.exists('temp/updater'):
os.mkdir('temp/updater')
with open('temp/updater/{}.zip'.format(latest_rls['tag_name']), 'wb') as f:
f.write(zip_data)
self.ap.logger.info('下载最新版本完成: {}'.format('temp/updater/{}.zip'.format(latest_rls['tag_name'])))
# 解压zip到temp/updater/<tag_name>/
import zipfile
# 检查目标文件夹
if os.path.exists('temp/updater/{}'.format(latest_rls['tag_name'])):
import shutil
shutil.rmtree('temp/updater/{}'.format(latest_rls['tag_name']))
os.mkdir('temp/updater/{}'.format(latest_rls['tag_name']))
with zipfile.ZipFile('temp/updater/{}.zip'.format(latest_rls['tag_name']), 'r') as zip_ref:
zip_ref.extractall('temp/updater/{}'.format(latest_rls['tag_name']))
# 覆盖源码
source_root = ''
# 找到temp/updater/<tag_name>/中的第一个子目录路径
for root, dirs, files in os.walk('temp/updater/{}'.format(latest_rls['tag_name'])):
if root != 'temp/updater/{}'.format(latest_rls['tag_name']):
source_root = root
break
# 覆盖源码
import shutil
for root, dirs, files in os.walk(source_root):
# 覆盖所有子文件子目录
for file in files:
src = os.path.join(root, file)
dst = src.replace(source_root, '.')
if os.path.exists(dst):
os.remove(dst)
# 检查目标文件夹是否存在
if not os.path.exists(os.path.dirname(dst)):
os.makedirs(os.path.dirname(dst))
# 检查目标文件是否存在
if not os.path.exists(dst):
# 创建目标文件
open(dst, 'w').close()
shutil.copy(src, dst)
# 把current_tag写入文件
current_tag = latest_rls['tag_name']
with open('current_tag', 'w') as f:
f.write(current_tag)
# TODO statistics
async def is_new_version_available(self) -> bool:
"""检查是否有新版本"""
# 从github获取release列表
"""Check whether a newer version is available."""
rls_list = await self.get_release_list()
if rls_list is None:
if not rls_list:
return False
# 获取当前版本
current_tag = self.get_current_version()
# 检查是否有新版本
latest_tag_name = ''
for rls in rls_list:
if latest_tag_name == '':
latest_tag_name = rls['tag_name']
break
latest_tag_name = rls.get('tag_name', '')
break
return self.is_newer(latest_tag_name, current_tag)
return self._is_newer(latest_tag_name, current_tag)
def is_newer(self, new_tag: str, old_tag: str):
"""判断版本是否更新,忽略第四位版本和第一位版本"""
if new_tag == old_tag:
def _is_newer(self, new_tag: str, old_tag: str) -> bool:
"""Check if new_tag is a newer version than old_tag.
Compares the first three segments (major.minor.patch) only.
Returns False if the major version differs (breaking change boundary).
"""
if not new_tag or not old_tag or new_tag == old_tag:
return False
new_tag = new_tag.split('.')
old_tag = old_tag.split('.')
new_parts = new_tag.split('.')
old_parts = old_tag.split('.')
# 判断主版本是否相同
if new_tag[0] != old_tag[0]:
# Different major version — not considered an upgrade
if new_parts[0] != old_parts[0]:
return False
if len(new_tag) < 4:
if len(new_parts) < 4:
return True
# 合成前三段,判断是否相同
new_tag = '.'.join(new_tag[:3])
old_tag = '.'.join(old_tag[:3])
return new_tag != old_tag
def compare_version_str(v0: str, v1: str) -> int:
"""比较两个版本号"""
# 删除版本号前的v
if v0.startswith('v'):
v0 = v0[1:]
if v1.startswith('v'):
v1 = v1[1:]
v0: list = v0.split('.')
v1: list = v1.split('.')
# 如果两个版本号节数不同把短的后面用0补齐
if len(v0) < len(v1):
v0.extend(['0'] * (len(v1) - len(v0)))
elif len(v0) > len(v1):
v1.extend(['0'] * (len(v0) - len(v1)))
# 从高位向低位比较
for i in range(len(v0)):
if int(v0[i]) > int(v1[i]):
return 1
elif int(v0[i]) < int(v1[i]):
return -1
return 0
return '.'.join(new_parts[:3]) != '.'.join(old_parts[:3])
async def show_version_update(self) -> typing.Tuple[str, int]:
try:
if await self.ap.ver_mgr.is_new_version_available():
if await self.is_new_version_available():
return (
'New version available:\n有新版本可用,根据文档更新: \nhttps://link.langbot.app/zh/docs/update',
'New version available. Update guide: https://link.langbot.app/en/docs/update',
logging.INFO,
)
except Exception as e:
return f'Error checking version update: {e}', logging.WARNING

View File

@@ -1,204 +0,0 @@
"""Workflow-Pipeline通信适配器
这个模块提供了Workflow和Pipeline之间的通信适配使用SDK标准的MessageEnvelope格式。
"""
from __future__ import annotations
import logging
from typing import Any, Optional
logger = logging.getLogger(__name__)
class _WorkflowPipelineCaptureAdapter:
"""Workflow-Pipeline通信适配器
用于在Workflow节点和Pipeline之间进行标准化的消息传递。
支持MessageEnvelope格式的双向转换。
"""
def __init__(self, context: Any):
"""初始化适配器
Args:
context: ExecutionContext - Workflow执行上下文
"""
self.context = context
self.responses: list[dict[str, Any]] = []
self.bot_account_id: Optional[str] = None
self._logger = logging.getLogger(__name__)
async def call_pipeline_with_envelope(
self,
envelope: Any,
pipeline_executor: Any
) -> Any:
"""使用MessageEnvelope调用Pipeline
Args:
envelope: MessageEnvelope - 标准消息信封
pipeline_executor: Pipeline执行器实例
Returns:
MessageEnvelope - 执行结果信封
"""
try:
# 动态导入以避免循环依赖
from langbot_plugin_sdk.workflow import envelope_to_query, query_to_envelope
# 1. 转换为Query
query = envelope_to_query(envelope)
# 2. 调用Pipeline
result_query = await pipeline_executor.execute(query)
# 3. 转换回Envelope
result_envelope = query_to_envelope(result_query, envelope)
self._logger.debug(
f'Pipeline execution completed for workflow {envelope.workflow_id}',
extra={
'workflow_id': envelope.workflow_id,
'execution_id': envelope.execution_id,
'node_id': envelope.node_id,
}
)
return result_envelope
except Exception as e:
self._logger.error(
f'Pipeline execution failed: {e}',
exc_info=True,
extra={
'workflow_id': envelope.workflow_id,
'execution_id': envelope.execution_id,
'node_id': envelope.node_id,
}
)
raise
def validate_envelope(self, envelope: Any) -> bool:
"""验证MessageEnvelope的有效性
Args:
envelope: MessageEnvelope - 要验证的消息信封
Returns:
bool - 验证是否通过
"""
required_fields = [
'message_id',
'workflow_id',
'node_id',
'execution_id',
'payload',
'launcher_type',
]
for field in required_fields:
if not hasattr(envelope, field):
self._logger.warning(
f'MessageEnvelope missing required field: {field}'
)
return False
return True
def get_responses(self) -> list[dict[str, Any]]:
"""获取所有响应
Returns:
list - 响应列表
"""
return self.responses.copy()
def add_response(self, response: dict[str, Any]) -> None:
"""添加响应
Args:
response: dict - 响应数据
"""
self.responses.append(response)
def get_last_text_response(self) -> str:
"""获取最后一个文本响应
Returns:
str - 最后一个响应的文本内容
"""
if not self.responses:
return ''
last_response = self.responses[-1]
return str(last_response.get('content', '') or '')
def clear_responses(self) -> None:
"""清空所有响应"""
self.responses.clear()
class WorkflowPipelineCompatibilityLayer:
"""Workflow-Pipeline兼容性层
提供向后兼容性支持旧的Pipeline Query格式和新的MessageEnvelope格式。
"""
def __init__(self):
"""初始化兼容性层"""
self._logger = logging.getLogger(__name__)
def is_workflow_context(self, query: Any) -> bool:
"""检查Query是否包含Workflow上下文
Args:
query: Query - Pipeline Query对象
Returns:
bool - 是否来自Workflow
"""
if hasattr(query, 'is_from_workflow'):
return query.is_from_workflow()
if hasattr(query, 'get_workflow_context'):
context = query.get_workflow_context()
return bool(context and context.get('workflow_id'))
return False
def get_workflow_id(self, query: Any) -> Optional[str]:
"""从Query获取Workflow ID
Args:
query: Query - Pipeline Query对象
Returns:
str - Workflow ID如果不存在则返回None
"""
if hasattr(query, 'get_workflow_id'):
return query.get_workflow_id()
if hasattr(query, 'get_workflow_context'):
context = query.get_workflow_context()
return context.get('workflow_id') if context else None
return None
def get_execution_id(self, query: Any) -> Optional[str]:
"""从Query获取执行ID
Args:
query: Query - Pipeline Query对象
Returns:
str - 执行ID如果不存在则返回None
"""
if hasattr(query, 'get_execution_id'):
return query.get_execution_id()
if hasattr(query, 'get_workflow_context'):
context = query.get_workflow_context()
return context.get('execution_id') if context else None
return None

View File

@@ -1,509 +0,0 @@
"""Workflow debug execution support.
This module provides debugging capabilities for workflow execution, including:
- ExecutionLog: Structured log entries for execution tracking
- DebugExecutionState: State management for debug sessions (pause, resume, breakpoints)
- DebugWorkflowExecutor: Extended executor with step-by-step debugging support
"""
from __future__ import annotations
import asyncio
import logging
import traceback
import uuid
from datetime import datetime
from typing import Any, Optional, TYPE_CHECKING
from .entities import (
WorkflowDefinition,
NodeDefinition,
EdgeDefinition,
ExecutionContext,
ExecutionStatus,
NodeState,
NodeStatus,
)
from .executor import WorkflowExecutor
if TYPE_CHECKING:
from ..core import app
logger = logging.getLogger(__name__)
class ExecutionLog:
"""Execution log entry"""
def __init__(self, level: str, message: str, node_id: Optional[str] = None, data: Optional[dict] = None):
self.id = str(uuid.uuid4())
self.timestamp = datetime.now().isoformat()
self.level = level
self.message = message
self.node_id = node_id
self.data = data or {}
def to_dict(self) -> dict:
return {
'id': self.id,
'timestamp': self.timestamp,
'level': self.level,
'message': self.message,
'node_id': self.node_id,
'data': self.data,
}
class DebugExecutionState:
"""State for a debug execution"""
def __init__(self, execution_id: str, breakpoints: list[str] = None):
self.execution_id = execution_id
self.status: str = 'running'
self.is_paused: bool = False
self.is_stopped: bool = False
self.current_node_id: Optional[str] = None
self.breakpoints: set[str] = set(breakpoints or [])
self.logs: list[ExecutionLog] = []
self.pending_logs: list[ExecutionLog] = []
self._pause_event = asyncio.Event()
self._pause_event.set() # Initially not paused
self._stop_event = asyncio.Event()
def add_log(self, level: str, message: str, node_id: str = None, data: dict = None):
"""Add a log entry"""
log = ExecutionLog(level, message, node_id, data)
self.logs.append(log)
self.pending_logs.append(log)
logger.log(
getattr(logging, level.upper(), logging.INFO),
f'[Workflow Debug] {message}',
extra={'node_id': node_id, 'data': data},
)
def get_pending_logs(self) -> list[dict]:
"""Get and clear pending logs"""
logs = [log.to_dict() for log in self.pending_logs]
self.pending_logs = []
return logs
def pause(self):
"""Pause execution"""
self.is_paused = True
self._pause_event.clear()
self.add_log('info', 'Execution paused')
def resume(self):
"""Resume execution"""
self.is_paused = False
self._pause_event.set()
self.add_log('info', 'Execution resumed')
def stop(self):
"""Stop execution"""
self.is_stopped = True
self.status = 'cancelled'
self._stop_event.set()
self._pause_event.set() # Release any pause
self.add_log('info', 'Execution stopped')
async def wait_if_paused(self):
"""Wait if execution is paused"""
if self.is_paused:
self.add_log('info', 'Waiting for resume...')
await self._pause_event.wait()
def check_breakpoint(self, node_id: str) -> bool:
"""Check if there's a breakpoint at the given node"""
return node_id in self.breakpoints
class DebugWorkflowExecutor(WorkflowExecutor):
"""
Debug-enabled workflow executor with step-by-step execution support.
Extends WorkflowExecutor with debugging capabilities.
"""
# Class-level storage for active debug sessions
_debug_states: dict[str, DebugExecutionState] = {}
def __init__(self, ap: Optional['app.Application'] = None):
super().__init__(ap)
@classmethod
def get_debug_state(cls, execution_id: str) -> Optional[DebugExecutionState]:
"""Get debug state for an execution"""
return cls._debug_states.get(execution_id)
@classmethod
def create_debug_state(cls, execution_id: str, breakpoints: list[str] = None) -> DebugExecutionState:
"""Create a new debug state"""
state = DebugExecutionState(execution_id, breakpoints)
cls._debug_states[execution_id] = state
return state
@classmethod
def remove_debug_state(cls, execution_id: str):
"""Remove debug state for an execution"""
cls._debug_states.pop(execution_id, None)
async def execute_debug(
self,
workflow: WorkflowDefinition,
context: ExecutionContext,
debug_state: DebugExecutionState,
) -> ExecutionContext:
"""
Execute a workflow in debug mode.
Args:
workflow: Workflow definition
context: Execution context
debug_state: Debug execution state
Returns:
Updated execution context
"""
context.status = ExecutionStatus.RUNNING
context.start_time = datetime.now()
debug_state.add_log('info', f'Starting debug execution for workflow: {workflow.name}')
try:
# Build execution graph
node_map = {node.id: node for node in workflow.nodes}
edge_map = self._build_edge_map(workflow.edges)
self._edges = workflow.edges
# Initialize node states
for node in workflow.nodes:
if node.id not in context.node_states:
context.node_states[node.id] = NodeState(node_id=node.id)
# Find start node(s)
start_nodes = self._find_start_nodes(workflow.nodes, workflow.edges)
if not start_nodes:
raise ValueError('No start nodes found in workflow')
debug_state.add_log('info', f'Found {len(start_nodes)} start node(s)')
# Execute from start nodes
for start_node in start_nodes:
if debug_state.is_stopped:
break
await self._execute_debug_from_node(
start_node, node_map, edge_map, context, debug_state, workflow.settings.max_retries
)
# Set final status
if debug_state.is_stopped:
context.status = ExecutionStatus.CANCELLED
debug_state.status = 'cancelled'
else:
all_completed = all(
state.status in (NodeStatus.COMPLETED, NodeStatus.SKIPPED) for state in context.node_states.values()
)
if all_completed:
context.status = ExecutionStatus.COMPLETED
debug_state.status = 'completed'
debug_state.add_log('info', 'Workflow execution completed successfully')
else:
has_failed = any(state.status == NodeStatus.FAILED for state in context.node_states.values())
if has_failed:
context.status = ExecutionStatus.FAILED
debug_state.status = 'error'
except Exception as e:
context.status = ExecutionStatus.FAILED
context.error = str(e)
debug_state.status = 'error'
debug_state.add_log('error', f'Workflow execution failed: {e}', data={'traceback': traceback.format_exc()})
logger.error(f'Debug workflow execution failed: {e}\n{traceback.format_exc()}')
finally:
context.end_time = datetime.now()
return context
async def _execute_debug_from_node(
self,
node: NodeDefinition,
node_map: dict[str, NodeDefinition],
edge_map: dict[str, list[EdgeDefinition]],
context: ExecutionContext,
debug_state: DebugExecutionState,
max_retries: int = 3,
):
"""Execute workflow from a node with debug support"""
# Check if stopped
if debug_state.is_stopped:
return
# Wait if paused
await debug_state.wait_if_paused()
# Check if should skip
if await self._should_skip_node(node, context):
if context.node_states[node.id].status == NodeStatus.SKIPPED:
debug_state.add_log('info', f'Skipping node: {node.id}', node_id=node.id)
return
# Check breakpoint
if debug_state.check_breakpoint(node.id):
debug_state.add_log('info', f'Hit breakpoint at node: {node.id}', node_id=node.id)
debug_state.pause()
await debug_state.wait_if_paused()
# Update current node
debug_state.current_node_id = node.id
debug_state.add_log('info', f'Executing node: {node.id} ({node.type})', node_id=node.id)
# Execute node
await self._execute_debug_node(node, context, debug_state, max_retries)
# Check if stopped or failed
if debug_state.is_stopped:
return
if context.node_states[node.id].status == NodeStatus.FAILED:
return
# Get outgoing edges
outgoing_edges = edge_map.get(node.id, [])
# Execute next nodes
for edge in outgoing_edges:
if debug_state.is_stopped:
break
target_node = node_map.get(edge.target_node)
if not target_node:
continue
# Check edge condition
if edge.condition:
condition_met = await self._evaluate_condition(edge.condition, context)
if not condition_met:
debug_state.add_log('debug', f'Edge condition not met: {edge.condition}', node_id=node.id)
continue
# Check if all inputs are ready
if await self._inputs_ready(target_node, edge_map, context):
await self._execute_debug_from_node(target_node, node_map, edge_map, context, debug_state, max_retries)
async def _execute_debug_node(
self, node: NodeDefinition, context: ExecutionContext, debug_state: DebugExecutionState, max_retries: int = 3
):
"""Execute a single node with debug logging"""
node_state = context.node_states[node.id]
node_state.status = NodeStatus.RUNNING
node_state.start_time = datetime.now()
# Get node instance (pass ap for access to services)
node_instance = self.registry.create_instance(node.type, node.id, node.config, ap=self.ap)
if not node_instance:
node_state.status = NodeStatus.FAILED
node_state.error = f'Unknown node type: {node.type}'
node_state.end_time = datetime.now()
debug_state.add_log('error', f'Unknown node type: {node.type}', node_id=node.id)
self._record_execution_step(node, node_state, context)
await self._persist_node_execution(node, node_state, context)
return
# Resolve inputs
inputs = await self._resolve_inputs(node, context)
node_state.inputs = inputs
debug_state.add_log(
'debug', 'Node inputs resolved', node_id=node.id, data={'inputs': self._safe_serialize(inputs)}
)
# Validate inputs
validation_errors = await node_instance.validate_inputs(inputs)
if validation_errors:
node_state.status = NodeStatus.FAILED
node_state.error = '; '.join(validation_errors)
node_state.end_time = datetime.now()
debug_state.add_log('error', f'Input validation failed: {node_state.error}', node_id=node.id)
self._record_execution_step(node, node_state, context)
await self._persist_node_execution(node, node_state, context)
return
# Execute with retries
for attempt in range(max_retries + 1):
if debug_state.is_stopped:
node_state.status = NodeStatus.FAILED
node_state.error = 'Execution stopped'
node_state.end_time = datetime.now()
break
try:
outputs = await node_instance.execute(inputs, context)
node_state.outputs = outputs
node_state.status = NodeStatus.COMPLETED
node_state.end_time = datetime.now()
duration_ms = int((node_state.end_time - node_state.start_time).total_seconds() * 1000)
debug_state.add_log(
'info',
f'Node completed in {duration_ms}ms',
node_id=node.id,
data={'outputs': self._safe_serialize(outputs), 'duration_ms': duration_ms},
)
break
except Exception as e:
node_state.retry_count = attempt + 1
debug_state.add_log(
'warning', f'Node execution failed (attempt {attempt + 1}/{max_retries + 1}): {e}', node_id=node.id
)
if attempt < max_retries:
await asyncio.sleep(1)
else:
node_state.status = NodeStatus.FAILED
node_state.error = str(e)
node_state.end_time = datetime.now()
debug_state.add_log(
'error',
f'Node failed after {max_retries + 1} attempts: {e}',
node_id=node.id,
data={'error': str(e), 'traceback': traceback.format_exc()},
)
self._record_execution_step(node, node_state, context)
await self._persist_node_execution(node, node_state, context)
async def step_execute(
self,
workflow: WorkflowDefinition,
context: ExecutionContext,
debug_state: DebugExecutionState,
) -> dict:
"""
Execute one step (one node) in debug mode.
Returns:
Dict with node_id, node_state, and completed status
"""
# Find next node to execute
next_node = self._find_next_executable_node(workflow, context)
if not next_node:
debug_state.status = 'completed'
return {'completed': True}
# Execute single node
debug_state.current_node_id = next_node.id
await self._execute_debug_node(next_node, context, debug_state, workflow.settings.max_retries)
node_state = context.node_states.get(next_node.id)
# Check if workflow is complete
all_done = all(
state.status in (NodeStatus.COMPLETED, NodeStatus.SKIPPED, NodeStatus.FAILED)
for state in context.node_states.values()
)
if all_done:
debug_state.status = 'completed'
context.status = ExecutionStatus.COMPLETED
return {
'node_id': next_node.id,
'node_state': {
'status': node_state.status.value if node_state else 'unknown',
'inputs': self._safe_serialize(node_state.inputs) if node_state else {},
'outputs': self._safe_serialize(node_state.outputs) if node_state else {},
'error': node_state.error if node_state else None,
},
'completed': all_done,
}
def _find_next_executable_node(
self, workflow: WorkflowDefinition, context: ExecutionContext
) -> Optional[NodeDefinition]:
"""Find the next node that can be executed"""
edge_map = self._build_edge_map(workflow.edges)
for node in workflow.nodes:
state = context.node_states.get(node.id)
# Skip completed, running, or failed nodes
if state and state.status in (
NodeStatus.COMPLETED,
NodeStatus.RUNNING,
NodeStatus.FAILED,
NodeStatus.SKIPPED,
):
continue
# Check if this node's inputs are ready
incoming_nodes = set()
for source_id, edges in edge_map.items():
for edge in edges:
if edge.target_node == node.id:
incoming_nodes.add(source_id)
# If no incoming nodes, it's a start node
if not incoming_nodes:
return node
# Check if all incoming nodes are done
all_incoming_done = True
for source_id in incoming_nodes:
source_state = context.node_states.get(source_id)
if not source_state or source_state.status not in (NodeStatus.COMPLETED, NodeStatus.SKIPPED):
all_incoming_done = False
break
if all_incoming_done:
return node
return None
def _safe_serialize(self, data: Any) -> Any:
"""Safely serialize data for logging"""
if data is None:
return None
if isinstance(data, (str, int, float, bool)):
return data
if isinstance(data, (list, tuple)):
return [self._safe_serialize(item) for item in data[:100]] # Limit list size
if isinstance(data, dict):
result = {}
for key, value in list(data.items())[:50]: # Limit dict size
result[str(key)] = self._safe_serialize(value)
return result
# For complex objects, try to convert to string
try:
return str(data)[:1000] # Limit string length
except Exception:
return '<non-serializable>'
def get_execution_state(self, context: ExecutionContext, debug_state: DebugExecutionState) -> dict:
"""Get current execution state for API response"""
node_states = {}
for node_id, state in context.node_states.items():
node_states[node_id] = {
'status': state.status.value,
'inputs': self._safe_serialize(state.inputs),
'outputs': self._safe_serialize(state.outputs),
'error': state.error,
'startTime': state.start_time.isoformat() if state.start_time else None,
'endTime': state.end_time.isoformat() if state.end_time else None,
'duration': int((state.end_time - state.start_time).total_seconds() * 1000)
if state.start_time and state.end_time
else None,
}
return {
'status': debug_state.status,
'current_node_id': debug_state.current_node_id,
'node_states': node_states,
'new_logs': debug_state.get_pending_logs(),
'error': context.error,
}

View File

@@ -1,155 +0,0 @@
"""Workflow entities and data models
This module defines workflow entities using SDK standard entities where available,
and local-specific entities for LangBot_copy-specific functionality.
"""
from __future__ import annotations
from datetime import datetime
from typing import Any, Optional
import pydantic
# Import SDK entities for standard workflow protocol types
from langbot_plugin.api.entities.builtin.workflow.entities import (
ExecutionContext,
ExecutionStep,
MessageContext,
NodeDefinition,
NodeState,
PortDefinition,
)
from langbot_plugin.api.entities.builtin.workflow.enums import (
ExecutionStatus,
NodeStatus,
)
class Position(pydantic.BaseModel):
"""Node position on canvas"""
x: float = 0
y: float = 0
class EdgeDefinition(pydantic.BaseModel):
"""Workflow edge definition (connection between nodes)"""
id: str
source_node: str
source_port: str = 'output'
target_node: str
target_port: str = 'input'
condition: Optional[str] = None # Optional condition expression
class TriggerDefinition(pydantic.BaseModel):
"""Workflow trigger definition"""
id: str
type: str # message, cron, event, webhook
config: dict[str, Any] = {}
enabled: bool = True
class WorkflowSettings(pydantic.BaseModel):
"""Workflow settings"""
# Execution settings
max_execution_time: int = 300 # seconds
max_retries: int = 3
retry_delay: int = 5 # seconds
# Error handling
error_handling: str = 'stop' # stop, continue, retry
# Logging
log_level: str = 'info'
save_execution_history: bool = True
# Concurrency
max_concurrent_executions: int = 10
class SafetyConfig(pydantic.BaseModel):
"""Safety configuration (inherited from Pipeline)"""
content_filter: dict[str, Any] = {'enable': False, 'sensitive_words': [], 'replace_with': '***'}
rate_limit: dict[str, Any] = {'enable': False, 'requests_per_minute': 60, 'burst_limit': 10}
class OutputConfig(pydantic.BaseModel):
"""Output configuration (inherited from Pipeline)"""
long_text_processing: dict[str, Any] = {
'strategy': 'split', # split, truncate, file
'max_length': 4000,
'split_separator': '\n\n',
}
force_delay: dict[str, Any] = {'enable': False, 'min_delay_ms': 0, 'max_delay_ms': 0}
misc: dict[str, Any] = {}
class WorkflowGlobalConfig(pydantic.BaseModel):
"""Workflow global configuration (inherited from Pipeline capabilities)"""
safety: SafetyConfig = SafetyConfig()
output: OutputConfig = OutputConfig()
class ExtensionsPreferences(pydantic.BaseModel):
"""Extensions preferences (same as Pipeline)"""
enable_all_plugins: bool = True
enable_all_mcp_servers: bool = True
plugins: list[str] = []
mcp_servers: list[str] = []
class ConversationVariable(pydantic.BaseModel):
"""Conversation-level variable definition"""
name: str
type: str = 'string' # string, number, boolean, object, array
description: str = ''
default_value: Any = None
max_length: Optional[int] = None # For strings
class WorkflowDefinition(pydantic.BaseModel):
"""Complete workflow definition"""
uuid: str
name: str
description: str = ''
emoji: str = '💼'
version: int = 1
# Workflow graph
nodes: list[NodeDefinition] = []
edges: list[EdgeDefinition] = []
# Variables
variables: dict[str, Any] = {} # Global variables
conversation_variables: list[ConversationVariable] = [] # Session-level variables
# Settings
settings: WorkflowSettings = WorkflowSettings()
# Triggers (for automation)
triggers: list[TriggerDefinition] = []
# Global configuration (inherited from Pipeline)
global_config: WorkflowGlobalConfig = WorkflowGlobalConfig()
# Extensions
extensions_preferences: ExtensionsPreferences = ExtensionsPreferences()
# Metadata
is_enabled: bool = True
created_at: Optional[datetime] = None
updated_at: Optional[datetime] = None
# Source tracking (for imported workflows)
source: Optional[str] = None # dify, n8n, langflow, etc.
source_id: Optional[str] = None

View File

@@ -1,837 +0,0 @@
"""Workflow execution engine.
This module contains the core workflow execution logic:
- WorkflowExecutor: Main execution engine with control flow handling
- ParallelExecutor: Parallel branch execution
- LoopExecutor: Loop/iterator execution
Debug execution support has been moved to the ``debug`` module.
"""
from __future__ import annotations
import ast
import asyncio
import logging
import operator
import uuid
from datetime import datetime
from typing import Any, Optional, TYPE_CHECKING
import sqlalchemy
from .entities import (
WorkflowDefinition,
NodeDefinition,
EdgeDefinition,
ExecutionContext,
ExecutionStatus,
NodeState,
NodeStatus,
ExecutionStep,
)
from ..entity.persistence import workflow as persistence_workflow
from .registry import NodeTypeRegistry
from . import monitor
if TYPE_CHECKING:
from ..core import app
logger = logging.getLogger(__name__)
# ─── Safe expression evaluator (replaces eval()) ─────────────────────
# Uses Python's ast module to whitelist only comparison / boolean / arithmetic
# operations. No function calls, attribute access, or subscript injection.
_SAFE_OPS = {
ast.Add: operator.add,
ast.Sub: operator.sub,
ast.Mult: operator.mul,
ast.Div: operator.truediv,
ast.FloorDiv: operator.floordiv,
ast.Mod: operator.mod,
ast.Pow: operator.pow,
ast.USub: operator.neg,
ast.UAdd: operator.pos,
ast.Not: operator.not_,
ast.Eq: operator.eq,
ast.NotEq: operator.ne,
ast.Lt: operator.lt,
ast.LtE: operator.le,
ast.Gt: operator.gt,
ast.GtE: operator.ge,
ast.Is: operator.is_,
ast.IsNot: operator.is_not,
ast.In: lambda a, b: a in b,
ast.NotIn: lambda a, b: a not in b,
}
def _safe_eval(expr: str) -> Any:
"""Evaluate a simple expression safely via AST whitelist.
Supports: literals, comparisons (==, !=, <, >, <=, >=, in, not in, is, is not),
boolean logic (and, or, not), arithmetic (+, -, *, /, //, %, **), and
string operations (contains via ``in``).
Raises ``ValueError`` on any disallowed construct (function calls,
attribute access, imports, etc.).
"""
tree = ast.parse(expr.strip(), mode='eval')
return _eval_node(tree.body)
def _eval_node(node: ast.AST) -> Any:
# Literals: numbers, strings, True/False/None
if isinstance(node, ast.Constant):
return node.value
# Unary operators: -x, +x, not x
if isinstance(node, ast.UnaryOp):
op_fn = _SAFE_OPS.get(type(node.op))
if op_fn is None:
raise ValueError(f'Unsupported unary op: {type(node.op).__name__}')
return op_fn(_eval_node(node.operand))
# Binary operators: x + y, x * y, etc.
if isinstance(node, ast.BinOp):
op_fn = _SAFE_OPS.get(type(node.op))
if op_fn is None:
raise ValueError(f'Unsupported binary op: {type(node.op).__name__}')
return op_fn(_eval_node(node.left), _eval_node(node.right))
# Comparisons: x == y, x > y, x in y, etc. (chained)
if isinstance(node, ast.Compare):
left = _eval_node(node.left)
for op, comparator in zip(node.ops, node.comparators):
op_fn = _SAFE_OPS.get(type(op))
if op_fn is None:
raise ValueError(f'Unsupported comparison: {type(op).__name__}')
right = _eval_node(comparator)
if not op_fn(left, right):
return False
left = right
return True
# Boolean operators: x and y, x or y
if isinstance(node, ast.BoolOp):
if isinstance(node.op, ast.And):
return all(_eval_node(v) for v in node.values)
if isinstance(node.op, ast.Or):
return any(_eval_node(v) for v in node.values)
# Ternary: x if cond else y
if isinstance(node, ast.IfExp):
return _eval_node(node.body) if _eval_node(node.test) else _eval_node(node.orelse)
# Tuples / Lists (used in "x in [1,2,3]")
if isinstance(node, (ast.Tuple, ast.List)):
return [_eval_node(e) for e in node.elts]
# Name lookup only allow None, True, False
if isinstance(node, ast.Name):
if node.id == 'None':
return None
if node.id == 'True':
return True
if node.id == 'False':
return False
raise ValueError(f'Unsupported variable reference: {node.id}')
raise ValueError(f'Unsupported expression node: {type(node).__name__}')
class WorkflowExecutor:
"""
Workflow execution engine.
Handles the execution of workflow definitions with proper control flow.
"""
def __init__(self, ap: Optional['app.Application'] = None):
self.ap = ap
self.registry = NodeTypeRegistry.instance()
self._edges: list[EdgeDefinition] = []
async def execute(
self, workflow: WorkflowDefinition, context: ExecutionContext, start_node_id: Optional[str] = None
) -> ExecutionContext:
"""
Execute a workflow.
Args:
workflow: Workflow definition
context: Execution context
start_node_id: Optional starting node (for resumption)
Returns:
Updated execution context
"""
context.status = ExecutionStatus.RUNNING
context.start_time = datetime.now()
# Note: Frontend panel logging has been removed.
# A new solution will be implemented separately.
monitoring_message_id = ''
try:
# Build execution graph
node_map = {node.id: node for node in workflow.nodes}
edge_map = self._build_edge_map(workflow.edges)
self._edges = workflow.edges
# Initialize node states
for node in workflow.nodes:
if node.id not in context.node_states:
context.node_states[node.id] = NodeState(node_id=node.id, node_type=node.type, status=NodeStatus.PENDING)
# Find start node(s)
if start_node_id:
start_nodes = [node_map[start_node_id]]
else:
start_nodes = self._find_start_nodes(workflow.nodes, workflow.edges)
if not start_nodes:
raise ValueError('No start nodes found in workflow')
# Execute from start nodes
for start_node in start_nodes:
await self._execute_from_node(
start_node, node_map, edge_map, context, workflow.settings.max_retries, path=set()
)
# Check final status
all_completed = all(
state.status in (NodeStatus.COMPLETED, NodeStatus.SKIPPED) for state in context.node_states.values()
)
if all_completed:
context.status = ExecutionStatus.COMPLETED
else:
# Some nodes might still be waiting
has_failed = any(state.status == NodeStatus.FAILED for state in context.node_states.values())
if has_failed:
context.status = ExecutionStatus.FAILED
except Exception as e:
context.status = ExecutionStatus.FAILED
context.error = str(e)
logger.error(
'Workflow execution failed',
exc_info=True,
extra={
'workflow_id': workflow.uuid,
'execution_id': context.execution_id,
'node_states': {
node_id: {
'status': state.status.value if state.status else None,
'error': state.error,
}
for node_id, state in context.node_states.items()
},
},
)
# Note: Frontend panel logging has been removed.
# A new solution will be implemented separately.
finally:
context.end_time = datetime.now()
# Note: Frontend panel logging has been removed.
# A new solution will be implemented separately.
return context
async def _execute_from_node(
self,
node: NodeDefinition,
node_map: dict[str, NodeDefinition],
edge_map: dict[str, list[EdgeDefinition]],
context: ExecutionContext,
max_retries: int = 3,
path: set[str] | None = None,
):
"""Execute workflow starting from a specific node"""
# Initialize path set for cycle detection (path-based, not global visited)
if path is None:
path = set()
# Check for circular dependency on the *current path* only
# This correctly allows diamond shapes (A→B, A→C, B→D, C→D)
if node.id in path:
logger.warning(f'Circular dependency detected at node: {node.id}')
context.node_states[node.id].status = NodeStatus.SKIPPED
context.node_states[node.id].error = 'Circular dependency detected'
context.node_states[node.id].end_time = datetime.now()
await self._persist_node_execution(node, context.node_states[node.id], context)
return
# Add node to current path
path.add(node.id)
# Check if node should be skipped
if await self._should_skip_node(node, context):
existing_state = context.node_states[node.id]
if existing_state.status == NodeStatus.SKIPPED:
existing_state.end_time = existing_state.end_time or datetime.now()
await self._persist_node_execution(node, existing_state, context)
path.discard(node.id)
return
# Execute current node
await self._execute_node(node, context, max_retries)
# If node failed and we should stop on error, return
if context.node_states[node.id].status == NodeStatus.FAILED:
path.discard(node.id)
return
node_state = context.node_states[node.id]
node_type_name = node.type.split('.')[-1] if '.' in node.type else node.type
# ── Control flow integration ────────────────────────────────
# For loop / iterator nodes: run the LoopExecutor over
# downstream body nodes for each item, then continue to the
# "completed" output edge.
if node_type_name in ('loop', 'iterator'):
items = node_state.outputs.get('_items') or []
if not items:
# iterator: items come from inputs
items = node_state.inputs.get('items', node_state.inputs.get('array', []))
if not isinstance(items, list):
items = [items] if items else []
max_iter = int(node.config.get('max_iterations', 100))
items = items[:max_iter]
# Collect downstream "body" nodes (connected via edges)
outgoing_edges = edge_map.get(node.id, [])
body_nodes = []
for edge in outgoing_edges:
target = node_map.get(edge.target_node)
if target:
body_nodes.append(target)
if body_nodes and items:
loop_exec = LoopExecutor(self)
results = await loop_exec.execute_loop(items, body_nodes, context, max_iter)
node_state.outputs['results'] = results
node_state.outputs['completed'] = True
else:
node_state.outputs['results'] = []
node_state.outputs['completed'] = True
path.discard(node.id)
return # body nodes already executed by LoopExecutor
# For parallel nodes: run downstream branches concurrently
if node_type_name == 'parallel':
outgoing_edges = edge_map.get(node.id, [])
branch_nodes = []
for edge in outgoing_edges:
target = node_map.get(edge.target_node)
if target:
branch_nodes.append([target])
if branch_nodes:
par_exec = ParallelExecutor(self)
results = await par_exec.execute_parallel(branch_nodes, context)
node_state.outputs['results'] = results
path.discard(node.id)
return # branch nodes already executed by ParallelExecutor
# ── Standard edge-based continuation ────────────────────────
# Get outgoing edges
outgoing_edges = edge_map.get(node.id, [])
# Execute next nodes based on edge conditions
for edge in outgoing_edges:
target_node = node_map.get(edge.target_node)
if not target_node:
continue
# Check edge condition
if edge.condition:
condition_met = await self._evaluate_condition(edge.condition, context)
if not condition_met:
continue
# Check if all inputs are ready
if await self._inputs_ready(target_node, edge_map, context):
await self._execute_from_node(target_node, node_map, edge_map, context, max_retries, path)
# Remove node from path when backtracking (allows diamond revisit)
path.discard(node.id)
async def _execute_node(self, node: NodeDefinition, context: ExecutionContext, max_retries: int = 3):
"""Execute a single node with retry logic"""
node_state = context.node_states[node.id]
node_state.status = NodeStatus.RUNNING
node_state.start_time = datetime.now()
# Get node instance (pass ap for access to services)
node_instance = self.registry.create_instance(node.type, node.id, node.config, ap=self.ap)
if not node_instance:
node_state.status = NodeStatus.FAILED
node_state.error = f'Unknown node type: {node.type}'
node_state.end_time = datetime.now()
self._record_execution_step(node, node_state, context)
await self._persist_node_execution(node, node_state, context)
return
# Resolve inputs
inputs = await self._resolve_inputs(node, context)
node_state.inputs = inputs
# Validate inputs
validation_errors = await node_instance.validate_inputs(inputs)
if validation_errors:
node_state.status = NodeStatus.FAILED
node_state.error = '; '.join(validation_errors)
node_state.end_time = datetime.now()
self._record_execution_step(node, node_state, context)
await self._persist_node_execution(node, node_state, context)
return
# Check if node supports streaming (has execute_stream method and stream config is enabled)
use_streaming = hasattr(node_instance, 'execute_stream') and node.config.get('stream', False)
# Execute with retries
for attempt in range(max_retries + 1):
try:
if use_streaming:
# Streaming execution with aggregation and timeout
aggregated_response = ''
try:
async with asyncio.timeout(300): # 5 minute timeout for streaming
async for chunk in node_instance.execute_stream(inputs, context):
if chunk:
aggregated_response += chunk
except asyncio.TimeoutError:
logger.warning(f'Node {node.id} ({node.type}) streaming timed out, falling back to non-streaming')
use_streaming = False
outputs = await node_instance.execute(inputs, context)
else:
# Get response from context if set by execute_stream, otherwise use aggregated
final_response = context.variables.pop('_last_llm_response', aggregated_response)
outputs = {'response': final_response, 'usage': {'prompt_tokens': 0, 'completion_tokens': 0, 'total_tokens': 0}}
logger.info(f'Node {node.id} ({node.type}) streaming completed, response length: {len(final_response)}')
else:
outputs = await node_instance.execute(inputs, context)
node_state.outputs = outputs
node_state.status = NodeStatus.COMPLETED
node_state.end_time = datetime.now()
break
except Exception as e:
node_state.retry_count = attempt + 1
logger.error(
f'Node {node.id} ({node.type}) execution failed (attempt {attempt + 1}/{max_retries + 1}): {e}',
exc_info=True,
extra={
'node_id': node.id,
'node_type': node.type,
'attempt': attempt + 1,
'max_retries': max_retries,
'execution_id': context.execution_id,
},
)
if attempt < max_retries:
await asyncio.sleep(1) # Brief delay before retry
else:
node_state.status = NodeStatus.FAILED
node_state.error = str(e)
node_state.end_time = datetime.now()
logger.error(
f'Node {node.id} ({node.type}) permanently failed after {max_retries + 1} attempts',
extra={
'node_id': node.id,
'node_type': node.type,
'error': str(e),
'execution_id': context.execution_id,
},
)
self._record_execution_step(node, node_state, context)
await self._persist_node_execution(node, node_state, context)
async def _resolve_inputs(self, node: NodeDefinition, context: ExecutionContext) -> dict[str, Any]:
"""Resolve input values for a node from connected nodes and context"""
inputs = {}
# Get inputs from context variables
if 'message' in context.variables:
inputs['message'] = context.variables['message']
# Get inputs from message context
if context.message_context:
inputs['message'] = context.message_context.message_content
inputs['message_content'] = context.message_context.message_content
inputs['sender_id'] = context.message_context.sender_id
inputs['platform'] = context.message_context.platform
else:
logger.warning(
f'[_resolve_inputs] node={node.id} ({node.type}): message_context is None!',
extra={
'node_id': node.id,
'node_type': node.type,
'execution_id': context.execution_id,
'variables_keys': list(context.variables.keys()) if context.variables else [],
},
)
# Log current inputs state after message_context processing
logger.debug(
f'[_resolve_inputs] node={node.id} after message_context: {list(inputs.keys())}',
)
# Get inputs from node config that reference other nodes
for key, value in node.config.items():
if isinstance(value, str) and value.startswith('{{') and value.endswith('}}'):
resolved = await self._resolve_expression(value[2:-2], context)
inputs[key] = resolved
else:
inputs[key] = value
# Get inputs from connected upstream nodes via edges
# Build a reverse map: for each incoming edge to this node, find the
# source node and the specific source/target port.
for edge in self._edges:
if edge.target_node != node.id:
continue
source_state = context.node_states.get(edge.source_node)
if not source_state or source_state.status != NodeStatus.COMPLETED:
continue
target_port = edge.target_port or 'input'
source_port = edge.source_port or 'output'
# Map the source node's output port value to this node's input port
if source_port in source_state.outputs:
inputs[target_port] = source_state.outputs[source_port]
elif 'output' in source_state.outputs:
# Fallback: if exact port not found, try generic 'output'
inputs[target_port] = source_state.outputs['output']
elif source_state.outputs:
# Last resort: use the first available output
inputs[target_port] = next(iter(source_state.outputs.values()))
# Smart input mapping: if a node needs 'message' but received a different
# port name (e.g., 'content' from llm_call), copy the value to 'message'.
# This handles edge connection mismatches where the sender uses a different
# port name than what the receiver expects.
if 'message' not in inputs or inputs.get('message') is None:
for fallback_key in ('content', 'response', 'input', 'output', 'result', 'text'):
if fallback_key in inputs and inputs[fallback_key] is not None:
inputs['message'] = inputs[fallback_key]
logger.debug(
f'[_resolve_inputs] node={node.id}: mapped {fallback_key} -> message',
)
break
logger.debug(
f'[_resolve_inputs] node={node.id} final inputs keys: {list(inputs.keys())}, message={repr(inputs.get("message", "<missing>")[:100] if isinstance(inputs.get("message"), str) else inputs.get("message"))}',
)
return inputs
async def _resolve_expression(self, expression: str, context: ExecutionContext) -> Any:
"""Resolve a variable expression like 'nodes.node1.outputs.text'"""
parts = expression.strip().split('.')
if not parts:
return None
if parts[0] == 'nodes' and len(parts) >= 4:
# nodes.node_id.outputs.output_name
node_id = parts[1]
if parts[2] == 'outputs' and node_id in context.node_states:
output_name = '.'.join(parts[3:])
return context.node_states[node_id].outputs.get(output_name)
elif parts[0] == 'variables':
# variables.var_name
var_name = '.'.join(parts[1:])
return context.variables.get(var_name)
elif parts[0] == 'conversation_variables':
# conversation_variables.var_name
var_name = '.'.join(parts[1:])
return context.conversation_variables.get(var_name)
elif parts[0] == 'message':
# message.content, message.sender_id, etc.
if context.message_context:
attr = parts[1] if len(parts) > 1 else None
if attr == 'content':
return context.message_context.message_content
elif attr == 'sender_id':
return context.message_context.sender_id
elif attr == 'platform':
return context.message_context.platform
elif attr == 'conversation_id':
return context.message_context.conversation_id
return None
async def _evaluate_condition(self, condition: str, context: ExecutionContext) -> bool:
"""Evaluate a condition expression safely using AST whitelist"""
try:
# Resolve variable references in condition
if '{{' in condition:
import re
pattern = r'\{\{([^}]+)\}\}'
# First pass: replace all variable references with placeholders
placeholders = {}
placeholder_idx = 0
def replace_with_placeholder(match):
nonlocal placeholder_idx
var_expr = match.group(1)
placeholder = f'__PH{placeholder_idx}__'
placeholders[placeholder] = var_expr
placeholder_idx += 1
return placeholder
condition_with_placeholders = re.sub(pattern, replace_with_placeholder, condition)
# Second pass: resolve each placeholder asynchronously
for placeholder, var_expr in placeholders.items():
value = await self._resolve_expression(var_expr, context)
if isinstance(value, str):
condition_with_placeholders = condition_with_placeholders.replace(placeholder, f'"{value}"')
elif value is None:
condition_with_placeholders = condition_with_placeholders.replace(placeholder, 'None')
else:
condition_with_placeholders = condition_with_placeholders.replace(placeholder, str(value))
condition = condition_with_placeholders
# Safe expression evaluation using AST whitelist
result = _safe_eval(condition)
return bool(result)
except Exception as e:
logger.warning(f'Condition evaluation failed: {condition} - {e}')
return False
async def _should_skip_node(self, node: NodeDefinition, context: ExecutionContext) -> bool:
"""Check if a node should be skipped"""
state = context.node_states.get(node.id)
if state and state.status in (NodeStatus.COMPLETED, NodeStatus.RUNNING, NodeStatus.SKIPPED):
return True
return False
async def _inputs_ready(
self, node: NodeDefinition, edge_map: dict[str, list[EdgeDefinition]], context: ExecutionContext
) -> bool:
"""Check if all inputs for a node are ready"""
# Find all edges that connect to this node
incoming_nodes = set()
for source_id, edges in edge_map.items():
for edge in edges:
if edge.target_node == node.id:
incoming_nodes.add(source_id)
# Check if all incoming nodes have completed
for source_id in incoming_nodes:
state = context.node_states.get(source_id)
if not state or state.status not in (NodeStatus.COMPLETED, NodeStatus.SKIPPED):
return False
return True
def _find_start_nodes(self, nodes: list[NodeDefinition], edges: list[EdgeDefinition]) -> list[NodeDefinition]:
"""Find nodes that have no incoming edges (start nodes)"""
target_nodes = {edge.target_node for edge in edges}
start_nodes = [node for node in nodes if node.id not in target_nodes]
# Also check for trigger nodes
trigger_types = {'message_trigger', 'cron_trigger', 'webhook_trigger', 'event_trigger'}
for node in nodes:
if node.type in trigger_types and node not in start_nodes:
start_nodes.insert(0, node)
return start_nodes
def _build_edge_map(self, edges: list[EdgeDefinition]) -> dict[str, list[EdgeDefinition]]:
"""Build a map of source node ID to outgoing edges"""
edge_map: dict[str, list[EdgeDefinition]] = {}
for edge in edges:
if edge.source_node not in edge_map:
edge_map[edge.source_node] = []
edge_map[edge.source_node].append(edge)
return edge_map
def _record_execution_step(self, node: NodeDefinition, node_state: NodeState, context: ExecutionContext):
"""Record an execution step in the history"""
duration_ms = 0
if node_state.start_time and node_state.end_time:
duration_ms = int((node_state.end_time - node_state.start_time).total_seconds() * 1000)
step = ExecutionStep(
step_id=f"step_{uuid.uuid4().hex[:8]}",
timestamp=datetime.now(),
node_id=node.id,
node_type=node.type,
status=node_state.status,
duration_ms=duration_ms,
error=node_state.error,
inputs=node_state.inputs,
outputs=node_state.outputs,
)
context.history.append(step)
async def _persist_node_execution(
self,
node: NodeDefinition,
node_state: NodeState,
context: ExecutionContext,
):
"""Persist node execution state for execution detail and logs."""
if not self.ap:
return
values = {
'execution_uuid': context.execution_id,
'node_id': node.id,
'node_type': node.type,
'status': node_state.status.value,
'inputs': node_state.inputs,
'outputs': node_state.outputs,
'start_time': node_state.start_time,
'end_time': node_state.end_time,
'error': node_state.error,
'retry_count': node_state.retry_count,
}
existing_query = sqlalchemy.select(persistence_workflow.WorkflowNodeExecution).where(
persistence_workflow.WorkflowNodeExecution.execution_uuid == context.execution_id,
persistence_workflow.WorkflowNodeExecution.node_id == node.id,
)
existing_result = await self.ap.persistence_mgr.execute_async(existing_query)
existing = existing_result.first()
if existing is None:
await self.ap.persistence_mgr.execute_async(
sqlalchemy.insert(persistence_workflow.WorkflowNodeExecution).values(**values)
)
else:
await self.ap.persistence_mgr.execute_async(
sqlalchemy.update(persistence_workflow.WorkflowNodeExecution)
.where(persistence_workflow.WorkflowNodeExecution.id == existing.id)
.values(**values)
)
class ParallelExecutor:
"""Execute multiple branches in parallel"""
def __init__(self, executor: WorkflowExecutor):
self.executor = executor
async def execute_parallel(
self, branches: list[list[NodeDefinition]], context: ExecutionContext
) -> list[dict[str, Any]]:
"""
Execute multiple branches in parallel.
Args:
branches: List of node sequences to execute in parallel
context: Execution context
Returns:
List of results from each branch
"""
tasks = []
for branch in branches:
task = self._execute_branch(branch, context)
tasks.append(task)
results = await asyncio.gather(*tasks, return_exceptions=True)
processed_results = []
for result in results:
if isinstance(result, Exception):
processed_results.append({'error': str(result)})
else:
processed_results.append(result)
return processed_results
async def _execute_branch(self, nodes: list[NodeDefinition], context: ExecutionContext) -> dict[str, Any]:
"""Execute a single branch"""
# Create a copy of context for this branch
branch_outputs = {}
for node in nodes:
await self.executor._execute_node(node, context, max_retries=3)
state = context.node_states.get(node.id)
if state and state.status == NodeStatus.COMPLETED:
branch_outputs[node.id] = state.outputs
elif state and state.status == NodeStatus.FAILED:
branch_outputs['error'] = state.error
break
return branch_outputs
class LoopExecutor:
"""Execute loop iterations"""
def __init__(self, executor: WorkflowExecutor):
self.executor = executor
async def execute_loop(
self, items: list[Any], loop_body: list[NodeDefinition], context: ExecutionContext, max_iterations: int = 100
) -> list[dict[str, Any]]:
"""
Execute a loop over items.
Args:
items: Items to iterate over
loop_body: Nodes to execute for each item
context: Execution context
max_iterations: Maximum number of iterations
Returns:
List of results from each iteration
"""
results = []
for i, item in enumerate(items[:max_iterations]):
# Set loop variables
context.variables['loop_item'] = item
context.variables['loop_index'] = i
context.variables['loop_is_first'] = i == 0
context.variables['loop_is_last'] = i == len(items) - 1
iteration_result = {}
for node in loop_body:
# Reset node state for this iteration
context.node_states[node.id] = NodeState(node_id=node.id, node_type=node.type, status=NodeStatus.PENDING)
await self.executor._execute_node(node, context, max_retries=3)
state = context.node_states.get(node.id)
if state:
iteration_result[node.id] = state.outputs
# Check for break condition
if state.outputs.get('break', False):
results.append(iteration_result)
return results
results.append(iteration_result)
# Clean up loop variables
context.variables.pop('loop_item', None)
context.variables.pop('loop_index', None)
context.variables.pop('loop_is_first', None)
context.variables.pop('loop_is_last', None)
return results

View File

@@ -1,284 +0,0 @@
"""Workflow node metadata loading and validation.
This module makes YAML files under ``templates/metadata/nodes`` the backend
source of truth for workflow node metadata. Python node classes still provide
execution logic, but UI-facing metadata is loaded from YAML.
"""
from __future__ import annotations
import copy
import logging
from importlib import resources
from pathlib import Path
from typing import Any, Iterable, Optional
import yaml
logger = logging.getLogger(__name__)
class MetadataLoadError(Exception):
"""Raised when a workflow node metadata file cannot be loaded."""
class MetadataValidationError(Exception):
"""Raised when workflow node metadata does not match the expected shape."""
class NodeMetadataValidator:
"""Validate workflow node metadata loaded from YAML files.
The validator is intentionally strict about the structural fields that the
editor needs, but tolerant of legacy YAML details such as missing top-level
``label`` or additional frontend field types.
"""
REQUIRED_FIELDS = ('name', 'category', 'inputs', 'outputs', 'config')
VALID_CATEGORIES = {'trigger', 'process', 'control', 'action', 'integration', 'misc'}
VALID_PORT_TYPES = {'any', 'string', 'number', 'integer', 'boolean', 'object', 'array', 'datetime', 'null'}
VALID_CONFIG_TYPES = {
'string',
'integer',
'number',
'float',
'boolean',
'select',
'json',
'textarea',
'text',
'secret',
'array[string]',
'file',
'array[file]',
'llm-model-selector',
'embedding-model-selector',
'rerank-model-selector',
'pipeline-selector',
'knowledge-base-selector',
'knowledge-base-multi-selector',
'bot-selector',
'tools-selector',
'model-fallback-selector',
'prompt-editor',
'plugin-selector',
'webhook-url',
'embed-code',
'workflow-selector',
}
def validate(self, metadata: dict[str, Any]) -> list[str]:
"""Return validation errors. An empty list means the metadata is valid."""
errors: list[str] = []
if not isinstance(metadata, dict):
return ['metadata root must be a mapping']
for field in self.REQUIRED_FIELDS:
if field not in metadata:
errors.append(f'missing required field: {field}')
if errors:
return errors
name = metadata.get('name')
if not isinstance(name, str) or not name.strip():
errors.append('field "name" must be a non-empty string')
category = metadata.get('category')
if category not in self.VALID_CATEGORIES:
errors.append(f'invalid category: {category}')
errors.extend(self._validate_ports(metadata.get('inputs'), 'inputs'))
errors.extend(self._validate_ports(metadata.get('outputs'), 'outputs'))
errors.extend(self._validate_config(metadata.get('config')))
return errors
def validate_or_raise(self, metadata: dict[str, Any]) -> dict[str, Any]:
"""Validate metadata and raise ``MetadataValidationError`` on failure."""
errors = self.validate(metadata)
if errors:
node_name = metadata.get('name', 'unknown') if isinstance(metadata, dict) else 'unknown'
raise MetadataValidationError(f'invalid metadata for {node_name}: {errors}')
return metadata
def _validate_ports(self, ports: Any, field_name: str) -> list[str]:
errors: list[str] = []
if not isinstance(ports, list):
return [f'{field_name} must be a list']
seen_names: set[str] = set()
for index, port in enumerate(ports):
path = f'{field_name}[{index}]'
if not isinstance(port, dict):
errors.append(f'{path} must be a mapping')
continue
name = port.get('name')
if not isinstance(name, str) or not name:
errors.append(f'{path}.name must be a non-empty string')
continue
if name in seen_names:
errors.append(f'{path}.name duplicates "{name}"')
seen_names.add(name)
port_type = port.get('type', 'any')
if port_type not in self.VALID_PORT_TYPES:
errors.append(f'{path}.type has unsupported value "{port_type}"')
return errors
def _validate_config(self, config: Any) -> list[str]:
errors: list[str] = []
if not isinstance(config, list):
return ['config must be a list']
seen_names: set[str] = set()
for index, item in enumerate(config):
path = f'config[{index}]'
if not isinstance(item, dict):
errors.append(f'{path} must be a mapping')
continue
name = item.get('name')
if not isinstance(name, str) or not name:
errors.append(f'{path}.name must be a non-empty string')
continue
if name in seen_names:
errors.append(f'{path}.name duplicates "{name}"')
seen_names.add(name)
item_type = item.get('type', 'string')
if item_type not in self.VALID_CONFIG_TYPES:
errors.append(f'{path}.type has unsupported value "{item_type}"')
min_value = item.get('min_value')
max_value = item.get('max_value')
if isinstance(min_value, (int, float)) and isinstance(max_value, (int, float)) and min_value > max_value:
errors.append(f'{path}.min_value must be <= max_value')
return errors
class NodeMetadataLoader:
"""Load and cache workflow node metadata from YAML files."""
def __init__(self, validator: Optional[NodeMetadataValidator] = None) -> None:
self._validator = validator or NodeMetadataValidator()
self._metadata: dict[str, dict[str, Any]] = {}
self._sources: dict[str, str] = {}
self._load_errors: list[dict[str, str]] = []
async def load_core_metadata(self, resource_dir: str = 'metadata/nodes') -> int:
"""Load all core node metadata from the ``langbot.templates`` package."""
return await self.load_package_directory('langbot.templates', resource_dir, source='core')
async def load_package_directory(self, package: str, resource_dir: str, source: str = 'core') -> int:
"""Load YAML files from a package resource directory."""
try:
root = resources.files(package).joinpath(resource_dir)
yaml_files = sorted(
(item for item in root.iterdir() if item.is_file() and item.name.endswith(('.yaml', '.yml'))),
key=lambda item: item.name,
)
except Exception as exc:
raise MetadataLoadError(f'failed to scan package directory {package}:{resource_dir}: {exc}') from exc
return self._load_files(yaml_files, source=source)
async def load_directory(self, directory: str | Path, source: str) -> int:
"""Load YAML files from an external filesystem directory, e.g. a plugin."""
directory_path = Path(directory)
if not directory_path.exists():
logger.warning('Workflow metadata directory does not exist: %s', directory_path)
return 0
if not directory_path.is_dir():
raise MetadataLoadError(f'workflow metadata path is not a directory: {directory_path}')
yaml_files = sorted(directory_path.glob('*.yml')) + sorted(directory_path.glob('*.yaml'))
return self._load_files(yaml_files, source=source)
def get_metadata(self, node_type: str) -> Optional[dict[str, Any]]:
"""Return metadata by full type or short node name."""
if node_type in self._metadata:
return copy.deepcopy(self._metadata[node_type])
short_name = node_type.split('.')[-1]
for registered_type, metadata in self._metadata.items():
if registered_type.split('.')[-1] == short_name or metadata.get('name') == short_name:
return copy.deepcopy(metadata)
return None
def get_all_metadata(self) -> dict[str, dict[str, Any]]:
"""Return a deep copy of all loaded metadata keyed by canonical node type."""
return copy.deepcopy(self._metadata)
def get_load_errors(self) -> list[dict[str, str]]:
"""Return metadata files that failed to load or validate."""
return copy.deepcopy(self._load_errors)
def clear(self) -> None:
"""Clear all cached metadata and errors."""
self._metadata.clear()
self._sources.clear()
self._load_errors.clear()
def _load_files(self, yaml_files: Iterable[Any], source: str) -> int:
count = 0
for yaml_file in yaml_files:
file_name = getattr(yaml_file, 'name', str(yaml_file))
try:
metadata = self._load_yaml(yaml_file)
self._validator.validate_or_raise(metadata)
node_type = build_node_type(metadata)
if node_type in self._metadata:
existing_source = self._sources.get(node_type, 'unknown')
if existing_source == 'core' and source != 'core':
raise MetadataLoadError(
f'plugin source "{source}" attempted to override core node "{node_type}"'
)
logger.warning(
'Workflow node metadata %s from %s overrides previous source %s',
node_type,
source,
existing_source,
)
cached_metadata = copy.deepcopy(metadata)
cached_metadata['_source'] = source
cached_metadata['_file'] = file_name
self._metadata[node_type] = cached_metadata
self._sources[node_type] = source
count += 1
except Exception as exc:
self._load_errors.append({'file': file_name, 'source': source, 'error': str(exc)})
logger.error('Failed to load workflow node metadata %s: %s', file_name, exc)
return count
def _load_yaml(self, yaml_file: Any) -> dict[str, Any]:
try:
if hasattr(yaml_file, 'open'):
with yaml_file.open('r', encoding='utf-8') as file:
data = yaml.load(file, Loader=yaml.FullLoader)
else:
with open(yaml_file, 'r', encoding='utf-8') as file:
data = yaml.load(file, Loader=yaml.FullLoader)
except Exception as exc:
raise MetadataLoadError(f'failed to parse YAML: {exc}') from exc
if not isinstance(data, dict):
raise MetadataLoadError('YAML root must be a mapping')
return data
def build_node_type(metadata: dict[str, Any]) -> str:
"""Build canonical ``category.name`` node type from metadata."""
category = metadata.get('category') or 'misc'
name = metadata.get('name') or ''
return f'{category}.{name}'

View File

@@ -1,61 +0,0 @@
"""
Monitoring helper for recording events during workflow execution.
This module provides convenient methods to record monitoring data
without cluttering the main workflow code.
NOTE: All frontend panel logging functionality has been removed.
A new solution will be implemented separately.
"""
from __future__ import annotations
import typing
import time
if typing.TYPE_CHECKING:
from ..core import app
from langbot_plugin.api.entities.builtin.workflow.query import WorkflowQuery
class WorkflowMonitoringHelper:
"""Helper class for workflow monitoring operations"""
# All frontend panel logging methods have been removed.
# A new solution will be implemented separately.
pass
class LLMCallMonitor:
"""Context manager for monitoring LLM calls in workflow"""
def __init__(
self,
ap: app.Application,
query: WorkflowQuery,
bot_id: str,
bot_name: str,
workflow_id: str,
workflow_name: str,
node_name: str,
model_name: str,
):
self.ap = ap
self.query = query
self.bot_id = bot_id
self.bot_name = bot_name
self.workflow_id = workflow_id
self.workflow_name = workflow_name
self.node_name = node_name
self.model_name = model_name
self.start_time = None
self.input_tokens = 0
self.output_tokens = 0
async def __aenter__(self):
self.start_time = time.time()
return self
async def __aexit__(self, exc_type, exc_val, exc_tb):
# LLM call monitoring has been removed.
# A new solution will be implemented separately.
return False

View File

@@ -1,350 +0,0 @@
"""
Monitoring helper for recording events during workflow execution.
This module provides convenient methods to record monitoring data
without cluttering the main workflow code.
Logging scheme (aligned with pipeline monitoring):
- Trigger log: stores original user message content directly
- LLM call log: uses record_llm_call only (no additional message record)
- LLM response log: stores response message content directly
- Reply log: stores reply content directly
Fields are extracted from WorkflowQuery object when available, with fallback to context_vars.
"""
from __future__ import annotations
import typing
import time
import json
if typing.TYPE_CHECKING:
from ..core import app
from langbot_plugin.api.entities.builtin.workflow.query import WorkflowQuery
class WorkflowMonitoringHelper:
"""Helper class for workflow monitoring operations"""
@staticmethod
def _get_session_id(query, context_vars: dict | None = None) -> str:
"""Build session_id from query or context_vars"""
# Try to get from query first
if not isinstance(query, str) and query.launcher_type:
launcher_type = query.launcher_type.value if hasattr(query.launcher_type, 'value') else str(query.launcher_type)
launcher_id = query.launcher_id or 'unknown'
return f'{launcher_type}_{launcher_id}'
# Fallback to context_vars
if context_vars and context_vars.get('_launcher_type') and context_vars.get('_launcher_id'):
return f"{context_vars['_launcher_type']}_{context_vars['_launcher_id']}"
return 'workflow_session'
@staticmethod
def _get_platform(query, context_vars: dict | None = None) -> str:
"""Get platform name from query or context_vars"""
if not isinstance(query, str) and query.launcher_type:
if hasattr(query.launcher_type, 'value'):
return query.launcher_type.value
return str(query.launcher_type)
return 'workflow'
@staticmethod
def _get_sender_name(query, context_vars: dict | None = None) -> str | None:
"""Get sender name from query or context_vars"""
# Try query first
if not isinstance(query, str):
if query.sender_name:
return query.sender_name
if query.message_event and hasattr(query.message_event, 'sender'):
sender = query.message_event.sender
if hasattr(sender, 'nickname'):
return sender.nickname
if hasattr(sender, 'member_name'):
return sender.member_name
# Fallback to context_vars
if context_vars:
return context_vars.get('_sender_name')
return None
@staticmethod
async def record_trigger_log(
ap: app.Application,
query,
workflow_id: str,
workflow_name: str,
bot_name: str = 'Workflow',
context_vars: dict | None = None,
) -> str:
"""Record trigger node log (stores original user message content directly)
Aligned with pipeline monitoring: record_query_start
"""
try:
session_id = WorkflowMonitoringHelper._get_session_id(query, context_vars)
platform = WorkflowMonitoringHelper._get_platform(query, context_vars)
sender_name = WorkflowMonitoringHelper._get_sender_name(query, context_vars)
# Get message content - store original content directly
message_content = ''
if isinstance(query, str):
message_content = query
elif not isinstance(query, str) and query.message_context:
message_content = query.message_context.message_content
elif not isinstance(query, str) and query.message_chain and hasattr(query.message_chain, 'model_dump'):
message_content = json.dumps(query.message_chain.model_dump(), ensure_ascii=False)
elif not isinstance(query, str) and query.user_message:
message_content = str(query.user_message)
# Get bot_id and user_id
bot_id = ''
user_id = None
if not isinstance(query, str):
bot_id = query.bot_uuid or ''
user_id = query.sender_id
elif context_vars:
bot_id = context_vars.get('_bot_id', '') or ''
user_id = context_vars.get('_user_id')
message_id = await ap.monitoring_service.record_message(
bot_id=bot_id,
bot_name=bot_name,
pipeline_id=workflow_id,
pipeline_name=workflow_name or 'Workflow',
message_content=message_content,
session_id=session_id,
status='success',
level='info',
platform=platform,
user_id=user_id,
user_name=sender_name,
role='user',
runner_name='local-workflow',
)
return message_id
except Exception as e:
ap.logger.error(f'Failed to record trigger log: {e}')
return ''
@staticmethod
async def record_llm_call_log(
ap: app.Application,
query,
workflow_id: str,
workflow_name: str,
node_name: str,
model_name: str,
input_tokens: int,
output_tokens: int,
duration_ms: int,
status: str = 'success',
error_message: str | None = None,
bot_name: str = 'Workflow',
context_vars: dict | None = None,
input_message: str | None = None,
message_id: str | None = None,
):
"""Record LLM call log with message_id association
Aligned with pipeline monitoring: record_llm_call with message_id
LLM calls are aggregated under the trigger log via message_id.
"""
try:
session_id = WorkflowMonitoringHelper._get_session_id(query, context_vars)
# Get bot_id
bot_id = ''
if not isinstance(query, str):
bot_id = query.bot_uuid or ''
elif context_vars:
bot_id = context_vars.get('_bot_id', '') or ''
# Record LLM call with message_id for association
await ap.monitoring_service.record_llm_call(
bot_id=bot_id,
bot_name=bot_name,
pipeline_id=workflow_id,
pipeline_name=workflow_name or 'Workflow',
session_id=session_id,
model_name=model_name,
input_tokens=input_tokens,
output_tokens=output_tokens,
duration=duration_ms,
status=status,
error_message=error_message,
message_id=message_id,
)
except Exception as e:
ap.logger.error(f'Failed to record LLM call log: {e}')
@staticmethod
async def record_llm_response_log(
ap: app.Application,
query,
workflow_id: str,
workflow_name: str,
node_name: str,
response_content: str,
bot_name: str = 'Workflow',
context_vars: dict | None = None,
):
"""Record LLM response log (stores response content directly)
Aligned with pipeline monitoring: record_query_response
"""
try:
session_id = WorkflowMonitoringHelper._get_session_id(query, context_vars)
platform = WorkflowMonitoringHelper._get_platform(query, context_vars)
sender_name = WorkflowMonitoringHelper._get_sender_name(query, context_vars)
# Get bot_id and user_id
bot_id = ''
user_id = None
if not isinstance(query, str):
bot_id = query.bot_uuid or ''
user_id = query.sender_id
elif context_vars:
bot_id = context_vars.get('_bot_id', '') or ''
user_id = context_vars.get('_user_id')
# Store response content directly, no prefix
await ap.monitoring_service.record_message(
bot_id=bot_id,
bot_name=bot_name,
pipeline_id=workflow_id,
pipeline_name=workflow_name or 'Workflow',
message_content=response_content[:2000], # Limit length
session_id=session_id,
status='success',
level='info',
platform=platform,
user_id=user_id,
user_name=sender_name,
role='assistant',
runner_name='local-workflow',
)
except Exception as e:
ap.logger.error(f'Failed to record LLM response log: {e}')
@staticmethod
async def record_reply_log(
ap: app.Application,
query,
workflow_id: str,
workflow_name: str,
node_name: str,
reply_content: str,
bot_name: str = 'Workflow',
context_vars: dict | None = None,
):
"""Record reply message log (stores reply content directly)
Aligned with pipeline monitoring: record_query_response
"""
try:
session_id = WorkflowMonitoringHelper._get_session_id(query, context_vars)
platform = WorkflowMonitoringHelper._get_platform(query, context_vars)
sender_name = WorkflowMonitoringHelper._get_sender_name(query, context_vars)
# Get bot_id and user_id
bot_id = ''
user_id = None
if not isinstance(query, str):
bot_id = query.bot_uuid or ''
user_id = query.sender_id
elif context_vars:
bot_id = context_vars.get('_bot_id', '') or ''
user_id = context_vars.get('_user_id')
# Store reply content directly, no prefix
await ap.monitoring_service.record_message(
bot_id=bot_id,
bot_name=bot_name,
pipeline_id=workflow_id,
pipeline_name=workflow_name or 'Workflow',
message_content=reply_content[:2000], # Limit length
session_id=session_id,
status='success',
level='info',
platform=platform,
user_id=user_id,
user_name=sender_name,
role='assistant',
runner_name='local-workflow',
)
except Exception as e:
ap.logger.error(f'Failed to record reply log: {e}')
class LLMCallMonitor:
"""Context manager for monitoring LLM calls in workflow"""
def __init__(
self,
ap: app.Application,
query,
bot_id: str,
bot_name: str,
workflow_id: str,
workflow_name: str,
node_name: str,
model_name: str,
context_vars: dict | None = None,
):
self.ap = ap
self.query = query
self.bot_id = bot_id
self.bot_name = bot_name
self.workflow_id = workflow_id
self.workflow_name = workflow_name
self.node_name = node_name
self.model_name = model_name
self.context_vars = context_vars
self.start_time = None
self.input_tokens = 0
self.output_tokens = 0
async def __aenter__(self):
self.start_time = time.time()
return self
async def __aexit__(self, exc_type, exc_val, exc_tb):
duration_ms = int((time.time() - self.start_time) * 1000) if self.start_time else 0
if exc_type is not None:
await WorkflowMonitoringHelper.record_llm_call_log(
ap=self.ap,
query=self.query,
workflow_id=self.workflow_id,
workflow_name=self.workflow_name,
node_name=self.node_name,
model_name=self.model_name,
input_tokens=self.input_tokens,
output_tokens=self.output_tokens,
duration_ms=duration_ms,
status='error',
error_message=str(exc_val) if exc_val else None,
bot_name=self.bot_name,
context_vars=self.context_vars,
)
else:
await WorkflowMonitoringHelper.record_llm_call_log(
ap=self.ap,
query=self.query,
workflow_id=self.workflow_id,
workflow_name=self.workflow_name,
node_name=self.node_name,
model_name=self.model_name,
input_tokens=self.input_tokens,
output_tokens=self.output_tokens,
duration_ms=duration_ms,
status='success',
bot_name=self.bot_name,
context_vars=self.context_vars,
)
return False

View File

@@ -1,164 +0,0 @@
"""Workflow node base class and decorators"""
from __future__ import annotations
import abc
from typing import Any, Callable, Optional, TYPE_CHECKING
if TYPE_CHECKING:
from .entities import ExecutionContext
from ..core import app
class WorkflowNode(abc.ABC):
"""Base class for all workflow nodes.
Node metadata (inputs, outputs, config schema, label, icon, etc.) is
defined exclusively in YAML files under templates/metadata/nodes/.
Python subclasses only provide execution logic and runtime behaviour.
"""
# Set by @workflow_node decorator
type_name: str = ''
# Category is kept as a fallback for registry when YAML is missing
category: str = 'misc'
# Pipeline config reuse (referenced by registry merge logic)
config_schema_source: Optional[str] = None
config_stages: list[str] = []
def __init__(self, node_id: str, config: dict[str, Any], ap: Optional['app.Application'] = None):
"""Initialize node with ID and configuration"""
self.node_id = node_id
self.config = config
self.ap = ap
@abc.abstractmethod
async def execute(self, inputs: dict[str, Any], context: ExecutionContext) -> dict[str, Any]:
"""Execute the node logic.
Args:
inputs: Input data from connected nodes
context: Execution context with workflow state
Returns:
Dictionary of output values
"""
pass
# ------------------------------------------------------------------
# Validation helpers — metadata is resolved from the registry at
# runtime so that YAML remains the single source of truth.
# ------------------------------------------------------------------
async def validate_inputs(self, inputs: dict[str, Any]) -> list[str]:
"""Validate input data against YAML port definitions.
Returns:
List of validation error messages (empty if valid)
"""
metadata = self._get_metadata()
if metadata is None:
return []
errors: list[str] = []
for port in metadata.get('inputs', []):
if port.get('required', True) and port.get('name') and port['name'] not in inputs:
errors.append(f"Missing required input: {port['name']}")
return errors
async def validate_config(self) -> list[str]:
"""Validate node configuration against YAML config schema.
Returns:
List of validation error messages (empty if valid)
"""
metadata = self._get_metadata()
if metadata is None:
return []
errors: list[str] = []
for cfg in metadata.get('config', []):
name = cfg.get('name', '')
if not name:
continue
required = cfg.get('required', False)
cfg_type = cfg.get('type', 'string')
if required and name not in self.config:
errors.append(f'Missing required config: {name}')
elif name in self.config:
value = self.config[name]
# Type validation
if cfg_type == 'integer' and not isinstance(value, int):
errors.append(f'Config {name} must be an integer')
elif cfg_type == 'number' and not isinstance(value, (int, float)):
errors.append(f'Config {name} must be a number')
elif cfg_type == 'boolean' and not isinstance(value, bool):
errors.append(f'Config {name} must be a boolean')
# Range validation
min_val = cfg.get('min_value')
max_val = cfg.get('max_value')
if min_val is not None and isinstance(value, (int, float)):
if value < min_val:
errors.append(f'Config {name} must be >= {min_val}')
if max_val is not None and isinstance(value, (int, float)):
if value > max_val:
errors.append(f'Config {name} must be <= {max_val}')
return errors
def get_config(self, key: str, default: Any = None) -> Any:
"""Get configuration value with default"""
return self.config.get(key, default)
def _get_metadata(self) -> Optional[dict[str, Any]]:
"""Retrieve YAML metadata for this node from the registry."""
from .registry import NodeTypeRegistry
registry = NodeTypeRegistry.instance()
return registry.get_metadata(self.type_name)
@classmethod
def to_schema(cls) -> dict[str, Any]:
"""Return a schema dict for this node type.
This is used by tests and tooling to inspect node capabilities.
"""
from .registry import NodeTypeRegistry
registry = NodeTypeRegistry.instance()
metadata = registry.get_metadata(cls.type_name)
if metadata:
return registry._metadata_to_schema(metadata)
# Fallback: build a minimal schema from class attributes
return {
'type': f'{cls.category}.{cls.type_name}' if cls.type_name else cls.type_name,
'category': cls.category,
'label': getattr(cls, 'name', cls.type_name),
'description': getattr(cls, 'description', ''),
'inputs': [],
'outputs': [],
'config_schema': [],
}
# ------------------------------------------------------------------
# Decorator for setting type_name attribute
# ------------------------------------------------------------------
def workflow_node(type_name: str) -> Callable[[type[WorkflowNode]], type[WorkflowNode]]:
"""Decorator to set the type_name attribute on a workflow node class.
Usage:
@workflow_node('llm_call')
class LLMCallNode(WorkflowNode):
...
The actual registration is now handled by the discovery engine.
"""
def decorator(cls: type[WorkflowNode]) -> type[WorkflowNode]:
cls.type_name = type_name
return cls
return decorator

View File

@@ -1 +0,0 @@

View File

@@ -1,263 +0,0 @@
"""Call Pipeline Node - invoke an existing pipeline
Node metadata is loaded from: ../../templates/metadata/nodes/call_pipeline.yaml
"""
from __future__ import annotations
from typing import Any, Optional
import pydantic
import langbot_plugin.api.definition.abstract.platform.adapter as abstract_platform_adapter
import langbot_plugin.api.definition.abstract.platform.event_logger as abstract_event_logger
import langbot_plugin.api.entities.builtin.pipeline.query as pipeline_query
import langbot_plugin.api.entities.builtin.platform.entities as platform_entities
import langbot_plugin.api.entities.builtin.platform.events as platform_events
import langbot_plugin.api.entities.builtin.platform.message as platform_message
import langbot_plugin.api.entities.builtin.provider.session as provider_session
from langbot_plugin.api.entities.builtin.workflow.entities import ExecutionContext
from ..node import WorkflowNode, workflow_node
class _NoOpEventLogger(abstract_event_logger.AbstractEventLogger):
"""No-op event logger for workflow pipeline adapter."""
async def info(
self,
text: str,
images: Optional[list[platform_message.Image]] = None,
message_session_id: Optional[str] = None,
no_throw: bool = True,
):
pass
async def debug(
self,
text: str,
images: Optional[list[platform_message.Image]] = None,
message_session_id: Optional[str] = None,
no_throw: bool = True,
):
pass
async def warning(
self,
text: str,
images: Optional[list[platform_message.Image]] = None,
message_session_id: Optional[str] = None,
no_throw: bool = True,
):
pass
async def error(
self,
text: str,
images: Optional[list[platform_message.Image]] = None,
message_session_id: Optional[str] = None,
no_throw: bool = True,
):
pass
@workflow_node('call_pipeline')
class CallPipelineNode(WorkflowNode):
"""Call pipeline node - invoke an existing pipeline"""
category = 'action'
async def execute(self, inputs: dict[str, Any], context: ExecutionContext) -> dict[str, Any]:
if not self.ap:
raise RuntimeError('Application instance not available — cannot call pipeline')
raw_query = inputs.get('query', '')
query_text = str(raw_query or inputs.get('input') or '')
pipeline_ref = str(self.get_config('pipeline_uuid', '') or '').strip()
if not pipeline_ref:
raise ValueError('No pipeline configured for call pipeline node')
pipeline_data = await self.ap.pipeline_service.get_pipeline(pipeline_ref)
if pipeline_data is None:
pipeline_data = await self.ap.pipeline_service.get_pipeline_by_name(pipeline_ref)
if pipeline_data is None:
raise ValueError(f'Pipeline not found: {pipeline_ref}')
pipeline_uuid = str(pipeline_data.get('uuid', '') or '')
if not pipeline_uuid:
raise ValueError(f'Pipeline UUID missing for: {pipeline_ref}')
runtime_pipeline = await self.ap.pipeline_mgr.get_pipeline_by_uuid(pipeline_uuid)
if runtime_pipeline is None:
raise ValueError(f'Runtime pipeline not loaded: {pipeline_uuid}')
adapter = _WorkflowPipelineCaptureAdapter(context=context)
adapter.bot_account_id = 'workflow-call-pipeline'
message_event = self._build_message_event(query_text, context)
message_chain = message_event.message_chain
launcher_type = (
provider_session.LauncherTypes.GROUP
if context.message_context and context.message_context.is_group
else provider_session.LauncherTypes.PERSON
)
launcher_id = context.session_id or context.execution_id
sender_id = (
context.message_context.sender_id
if context.message_context and context.message_context.sender_id
else context.user_id or f'workflow_{context.execution_id}'
)
query = pipeline_query.Query(
bot_uuid=context.bot_id,
query_id=-1,
launcher_type=launcher_type,
launcher_id=launcher_id,
sender_id=sender_id,
message_event=message_event,
message_chain=message_chain,
variables={
'_called_from_workflow': True,
'_workflow_execution_id': context.execution_id,
'_workflow_id': context.workflow_id,
**dict(context.variables or {}),
},
resp_messages=[],
resp_message_chain=[],
adapter=adapter,
pipeline_uuid=pipeline_uuid,
)
await runtime_pipeline.run(query)
response_text = adapter.get_last_text_response()
result = {
'pipeline_uuid': pipeline_uuid,
'pipeline_name': pipeline_data.get('name', ''),
'responses': adapter.responses,
'query_text': query_text,
}
return {'response': response_text, 'result': result}
def _build_message_event(
self,
query_text: str,
context: ExecutionContext,
) -> platform_events.MessageEvent:
message_chain_data = context.trigger_data.get('message_chain') or context.trigger_data.get('message', [])
if isinstance(message_chain_data, list) and message_chain_data:
message_chain = platform_message.MessageChain.model_validate(message_chain_data)
else:
message_chain = platform_message.MessageChain([platform_message.Plain(text=query_text)])
if context.message_context and context.message_context.is_group:
group = platform_entities.Group(
id=context.message_context.group_id or context.session_id or 'workflow_group',
name=context.message_context.raw_message.get('group_name', 'Workflow Group') if context.message_context.raw_message else 'Workflow Group',
permission=platform_entities.Permission.Member,
)
sender = platform_entities.GroupMember(
id=context.message_context.sender_id,
member_name=context.message_context.sender_name or 'Workflow User',
permission=platform_entities.Permission.Member,
group=group,
)
return platform_events.GroupMessage(
sender=sender,
message_chain=message_chain,
time=context.message_context.raw_message.get('time') if context.message_context.raw_message else None,
)
sender = platform_entities.Friend(
id=context.message_context.sender_id if context.message_context else context.user_id or 'workflow_user',
nickname=context.message_context.sender_name if context.message_context else 'Workflow User',
remark=context.message_context.sender_name if context.message_context else 'Workflow User',
)
return platform_events.FriendMessage(
sender=sender,
message_chain=message_chain,
time=context.message_context.raw_message.get('time')
if context.message_context and context.message_context.raw_message
else None,
)
class _WorkflowPipelineCaptureAdapter(abstract_platform_adapter.AbstractMessagePlatformAdapter):
"""Adapter to capture pipeline responses for workflow execution."""
class Config:
arbitrary_types_allowed = True
responses: list[dict[str, Any]] = []
context: Optional[ExecutionContext] = pydantic.Field(default=None, exclude=True)
def __init__(self, context: ExecutionContext):
super().__init__(config={}, logger=_NoOpEventLogger(), context=context)
self.responses = []
async def send_message(self, target_type: str, target_id: str, message: platform_message.MessageChain):
payload = {
'type': 'send',
'target_type': target_type,
'target_id': target_id,
'content': str(message),
'message_chain': message.model_dump(),
}
self.responses.append(payload)
return payload
async def reply_message(
self,
message_source: platform_events.MessageEvent,
message: platform_message.MessageChain,
quote_origin: bool = False,
):
payload = {
'type': 'reply',
'content': str(message),
'message_chain': message.model_dump(),
'quote_origin': quote_origin,
}
self.responses.append(payload)
return payload
async def reply_message_chunk(
self,
message_source: platform_events.MessageEvent,
bot_message: dict,
message: platform_message.MessageChain,
quote_origin: bool = False,
is_final: bool = False,
):
payload = {
'type': 'reply_chunk',
'content': str(message),
'message_chain': message.model_dump(),
'quote_origin': quote_origin,
'is_final': is_final,
}
self.responses.append(payload)
return payload
async def create_message_card(self, message_id, event: platform_events.MessageEvent) -> bool:
return False
def register_listener(self, event_type, callback):
return None
def unregister_listener(self, event_type, callback):
return None
async def run_async(self):
return None
async def is_stream_output_supported(self) -> bool:
return False
async def kill(self) -> bool:
return True
def get_last_text_response(self) -> str:
if not self.responses:
return ''
return str(self.responses[-1].get('content', '') or '')

View File

@@ -1,85 +0,0 @@
"""Call Workflow Node - invoke an existing workflow
Node metadata is loaded from: ../../templates/metadata/nodes/call_workflow.yaml
"""
from __future__ import annotations
from typing import Any
from langbot_plugin.api.entities.builtin.workflow.entities import ExecutionContext
from ..node import WorkflowNode, workflow_node
@workflow_node('call_workflow')
class CallWorkflowNode(WorkflowNode):
"""Call workflow node - invoke an existing workflow"""
category = 'action'
async def execute(self, inputs: dict[str, Any], context: ExecutionContext) -> dict[str, Any]:
if not self.ap:
raise RuntimeError('Application instance not available — cannot call workflow')
# Get workflow reference from config
workflow_ref = str(self.get_config('workflow_uuid', '') or '').strip()
if not workflow_ref:
raise ValueError('No workflow configured for call workflow node')
# Get workflow definition from service
workflow_data = await self.ap.workflow_service.get_workflow(workflow_ref)
if workflow_data is None:
raise ValueError(f'Workflow not found: {workflow_ref}')
workflow_uuid = str(workflow_data.get('uuid', '') or '')
if not workflow_uuid:
raise ValueError(f'Workflow UUID missing for: {workflow_ref}')
# Build variables to pass to the called workflow
variables = dict(inputs.get('variables', {}) or {})
# Inherit current workflow variables if configured
if self.get_config('inherit_variables', True):
for key, value in (context.variables or {}).items():
if key not in variables:
variables[key] = value
# Add context markers for debugging
variables['_called_from_workflow'] = True
variables['_parent_workflow_id'] = context.workflow_id
variables['_parent_execution_id'] = context.execution_id
# Execute the workflow
execution_id = await self.ap.workflow_service.execute_workflow(
workflow_uuid=workflow_uuid,
trigger_type='workflow_call',
trigger_data={
'variables': variables,
'parent_execution_id': context.execution_id,
},
session_id=context.session_id,
user_id=context.user_id,
bot_id=context.bot_id,
)
# Get execution result
execution = await self.ap.workflow_service.get_execution(execution_id)
if execution is None:
raise ValueError(f'Execution result not found: {execution_id}')
# Build result
result = {
'workflow_uuid': workflow_uuid,
'workflow_name': workflow_data.get('name', ''),
'execution_id': execution_id,
'status': execution.get('status', 'unknown'),
'variables': execution.get('variables', {}),
'error': execution.get('error'),
}
return {
'result': result,
'status': execution.get('status', 'unknown'),
'error': execution.get('error'),
}

View File

@@ -1,156 +0,0 @@
"""Code Executor Node - run Python or JavaScript code
Node metadata is loaded from: ../../templates/metadata/nodes/code_executor.yaml
"""
from __future__ import annotations
import ast
import io
import logging
import sys
import threading
from typing import Any
from langbot_plugin.api.entities.builtin.workflow.entities import ExecutionContext
from ..node import WorkflowNode, workflow_node
logger = logging.getLogger(__name__)
# 危险的内置函数和模块黑名单
_DANGEROUS_BUILTINS = {
'__import__', 'eval', 'exec', 'compile', 'open', 'file',
'input', 'exit', 'quit', 'globals', 'locals', 'vars',
'dir', 'help', 'breakpoint',
}
# 允许的安全内置函数
_SAFE_BUILTINS = {
'abs': abs, 'all': all, 'any': any, 'bin': bin, 'bool': bool,
'bytearray': bytearray, 'bytes': bytes, 'callable': callable,
'chr': chr, 'complex': complex, 'dict': dict, 'divmod': divmod,
'enumerate': enumerate, 'filter': filter, 'float': float,
'format': format, 'frozenset': frozenset, 'hash': hash,
'hex': hex, 'int': int, 'isinstance': isinstance, 'issubclass': issubclass,
'iter': iter, 'len': len, 'list': list, 'map': map, 'max': max,
'min': min, 'next': next, 'object': object, 'oct': oct, 'ord': ord,
'pow': pow, 'print': print, 'range': range, 'repr': repr,
'reversed': reversed, 'round': round, 'set': set, 'slice': slice,
'sorted': sorted, 'str': str, 'sum': sum, 'tuple': tuple,
'type': type, 'zip': zip,
}
def _check_code_safety(code: str) -> list[str]:
"""检查代码中是否包含危险操作"""
warnings = []
try:
tree = ast.parse(code)
for node in ast.walk(tree):
# 检查 import 语句
if isinstance(node, (ast.Import, ast.ImportFrom)):
warnings.append('Import statements are not allowed')
# 检查危险函数调用
if isinstance(node, ast.Call):
if isinstance(node.func, ast.Name) and node.func.id in _DANGEROUS_BUILTINS:
warnings.append(f'Dangerous function call: {node.func.id}')
# 检查 __import__ 通过 getattr 调用
if isinstance(node.func, ast.Attribute):
if node.func.attr in ('__import__', 'eval', 'exec', 'open', 'file'):
warnings.append(f'Dangerous attribute access: {node.func.attr}')
except SyntaxError as e:
warnings.append(f'Syntax error in code: {e}')
return warnings
class _ExecutionTimeoutError(Exception):
"""执行超时错误"""
pass
def _run_with_timeout(func, timeout: float = 10.0):
"""带超时限制的函数执行"""
result = [None]
error = [None]
def _target():
try:
result[0] = func()
except Exception as e:
error[0] = e
thread = threading.Thread(target=_target)
thread.daemon = True
thread.start()
thread.join(timeout)
if thread.is_alive():
raise _ExecutionTimeoutError(f'Code execution timed out after {timeout} seconds')
if error[0]:
raise error[0]
return result[0]
@workflow_node('code_executor')
class CodeExecutorNode(WorkflowNode):
"""Code executor node - run Python or JavaScript code"""
category = 'process'
async def execute(self, inputs: dict[str, Any], context: ExecutionContext) -> dict[str, Any]:
code = self.get_config('code', '')
language = self.get_config('language', 'python')
timeout = self.get_config('timeout', 10)
# 限制最大超时时间
timeout = min(max(timeout, 1), 30)
if not code:
return {'output': None, 'console': '', 'error': 'No code provided'}
if language == 'python':
return await self._execute_python(code, inputs, context, timeout)
else:
return await self._execute_javascript(code, inputs, context)
async def _execute_python(self, code: str, inputs: dict[str, Any], context: ExecutionContext, timeout: float) -> dict[str, Any]:
# 安全检查
warnings = _check_code_safety(code)
if warnings:
logger.warning('Code safety warnings: %s', warnings)
return {'output': None, 'console': '', 'error': '; '.join(warnings)}
stdout_capture = io.StringIO()
old_stdout = sys.stdout
def _exec_code():
nonlocal stdout_capture
sys.stdout = stdout_capture
try:
# 使用更安全的执行方式
compiled = compile(code, '<workflow>', 'exec')
safe_globals = {
'__builtins__': _SAFE_BUILTINS,
'__name__': '__workflow_sandbox__',
}
local_vars = {'inputs': inputs, 'output': None}
exec(compiled, safe_globals, local_vars)
return local_vars.get('output')
finally:
sys.stdout = old_stdout
try:
output = _run_with_timeout(_exec_code, timeout)
console_output = stdout_capture.getvalue()
return {'output': output, 'console': console_output, 'error': None}
except _ExecutionTimeoutError as e:
logger.error('Code execution timeout: %s', e)
return {'output': None, 'console': stdout_capture.getvalue(), 'error': str(e)}
except Exception as e:
logger.error('Code execution error: %s', e)
return {'output': None, 'console': stdout_capture.getvalue(), 'error': f'{type(e).__name__}: {e}'}
async def _execute_javascript(self, code: str, inputs: dict[str, Any], context: ExecutionContext) -> dict[str, Any]:
return {'output': None, 'console': '', 'error': 'JavaScript execution is not implemented'}

View File

@@ -1,125 +0,0 @@
"""Condition Node - branch based on condition
Node metadata is loaded from: ../../templates/metadata/nodes/condition.yaml
"""
from __future__ import annotations
import logging
import re
import signal
from typing import Any
from langbot_plugin.api.entities.builtin.workflow.entities import ExecutionContext
from ..node import WorkflowNode, workflow_node
from ..safe_eval import safe_eval_with_vars
logger = logging.getLogger(__name__)
# 正则表达式超时限制(秒)
_REGEX_TIMEOUT = 2
class _RegexTimeoutError(Exception):
"""正则表达式超时错误"""
pass
def _handle_timeout(signum, frame):
"""超时信号处理"""
raise _RegexTimeoutError('Regex match timed out')
def _safe_regex_match(pattern: str, text: str) -> tuple[bool, str]:
"""安全地执行正则表达式匹配,带有超时限制"""
# 设置超时信号
old_handler = signal.signal(signal.SIGALRM, _handle_timeout)
signal.setitimer(signal.ITIMER_REAL, _REGEX_TIMEOUT)
try:
result = bool(re.match(pattern, str(text)))
return result, ''
except _RegexTimeoutError:
logger.warning('Regex match timed out for pattern: %s', pattern[:50])
return False, 'Regex match timed out'
except re.error as e:
logger.warning('Invalid regex pattern: %s', e)
return False, f'Invalid regex: {e}'
finally:
signal.setitimer(signal.ITIMER_REAL, 0)
signal.signal(signal.SIGALRM, old_handler)
@workflow_node('condition')
class ConditionNode(WorkflowNode):
"""Condition node - branch based on condition"""
category = 'control'
async def execute(self, inputs: dict[str, Any], context: ExecutionContext) -> dict[str, Any]:
condition_type = self.get_config('condition_type', 'expression')
input_data = inputs.get('input')
result = False
if condition_type == 'expression':
expression = self.get_config('expression', 'false')
result = await self._evaluate_expression(expression, input_data, context)
elif condition_type == 'comparison':
result = await self._evaluate_comparison(input_data, context)
elif condition_type == 'contains':
left = self.get_config('left_value', '')
right = self.get_config('right_value', '')
result = right in left
elif condition_type == 'empty':
result = not bool(input_data)
elif condition_type == 'regex':
left = self.get_config('left_value', '')
pattern = self.get_config('right_value', '')
result, error = _safe_regex_match(pattern, left)
if error:
return {'true': None, 'false': input_data, 'error': error}
if result:
return {'true': input_data, 'false': None}
else:
return {'true': None, 'false': input_data}
async def _evaluate_expression(self, expression: str, data: Any, context: ExecutionContext) -> bool:
try:
local_vars = {'input': data, 'data': data, 'variables': context.variables}
return bool(safe_eval_with_vars(expression, local_vars))
except Exception as e:
logger.warning('Expression evaluation error: %s', e)
return False
async def _evaluate_comparison(self, data: Any, context: ExecutionContext) -> bool:
left = self.get_config('left_value', '')
right = self.get_config('right_value', '')
operator = self.get_config('operator', '==')
try:
left_num = float(left)
right_num = float(right)
if operator == '==':
return left_num == right_num
elif operator == '!=':
return left_num != right_num
elif operator == '>':
return left_num > right_num
elif operator == '<':
return left_num < right_num
elif operator == '>=':
return left_num >= right_num
elif operator == '<=':
return left_num <= right_num
except ValueError:
if operator == '==':
return left == right
elif operator == '!=':
return left != right
elif operator in ('>', '<', '>=', '<='):
return False
return False

View File

@@ -1,39 +0,0 @@
"""Coze Bot Node - call Coze API bot
Node metadata is loaded from: ../../templates/metadata/nodes/coze_bot.yaml
"""
from __future__ import annotations
from typing import Any
from langbot_plugin.api.entities.builtin.workflow.entities import ExecutionContext
from ..node import WorkflowNode, workflow_node
@workflow_node('coze_bot')
class CozeBotNode(WorkflowNode):
"""Coze bot node - call Coze API bot"""
category = 'integration'
async def execute(self, inputs: dict[str, Any], context: ExecutionContext) -> dict[str, Any]:
api_key = self.get_config('api_key', '')
bot_id = self.get_config('bot_id', '')
api_base = self.get_config('api_base', 'https://api.coze.cn')
query = inputs.get('query', '')
conversation_id = inputs.get('conversation_id')
# Safe API key truncation
masked_key = f'{api_key[:4]}...{api_key[-4:]}' if len(api_key) > 8 else '***' if api_key else ''
return {
'answer': '',
'conversation_id': conversation_id,
'success': False,
'_debug': {
'api_key': masked_key,
'bot_id': bot_id,
'api_base': api_base,
'query': query,
},
}

View File

@@ -1,26 +0,0 @@
"""Cron Trigger Node - triggers workflow on schedule
Node metadata is loaded from: ../../templates/metadata/nodes/cron_trigger.yaml
"""
from __future__ import annotations
from typing import Any
from langbot_plugin.api.entities.builtin.workflow.entities import ExecutionContext
from ..node import WorkflowNode, workflow_node
@workflow_node('cron_trigger')
class CronTriggerNode(WorkflowNode):
"""Cron trigger node - triggers workflow on schedule"""
category = 'trigger'
async def execute(self, inputs: dict[str, Any], context: ExecutionContext) -> dict[str, Any]:
from datetime import datetime
return {
'timestamp': datetime.now().isoformat(),
'schedule': self.get_config('cron', ''),
'context': context.trigger_data,
}

View File

@@ -1,68 +0,0 @@
"""Data Transform Node - transform data using templates or JSONPath
Node metadata is loaded from: ../../templates/metadata/nodes/data_transform.yaml
"""
from __future__ import annotations
from typing import Any
from langbot_plugin.api.entities.builtin.workflow.entities import ExecutionContext
from ..node import WorkflowNode, workflow_node
from ..safe_eval import safe_eval_with_vars
@workflow_node('data_transform')
class DataTransformNode(WorkflowNode):
"""Data transform node - transform data using templates or JSONPath"""
category = 'process'
async def execute(self, inputs: dict[str, Any], context: ExecutionContext) -> dict[str, Any]:
data = inputs.get('data')
transform_type = self.get_config('transform_type', 'template')
if transform_type == 'template':
template = self.get_config('template', '')
result = self._apply_template(template, data, context)
elif transform_type == 'jsonpath':
expression = self.get_config('expression', '$')
result = self._apply_jsonpath(expression, data)
elif transform_type == 'expression':
expression = self.get_config('expression', '')
result = self._evaluate_expression(expression, data, context)
else:
result = data
return {'result': result}
def _apply_template(self, template: str, data: Any, context: ExecutionContext) -> str:
result = template
if isinstance(data, dict):
for key, value in data.items():
result = result.replace(f'{{{{data.{key}}}}}', str(value))
for key, value in context.variables.items():
result = result.replace(f'{{{{variables.{key}}}}}', str(value))
return result
def _apply_jsonpath(self, expression: str, data: Any) -> Any:
if expression == '$':
return data
if expression.startswith('$.'):
parts = expression[2:].split('.')
result = data
for part in parts:
if isinstance(result, dict):
result = result.get(part)
elif isinstance(result, list) and part.isdigit():
result = result[int(part)]
else:
return None
return result
return data
def _evaluate_expression(self, expression: str, data: Any, context: ExecutionContext) -> Any:
local_vars = {'data': data, 'variables': context.variables}
try:
return safe_eval_with_vars(expression, local_vars)
except Exception:
return None

View File

@@ -1,38 +0,0 @@
"""Database Query Node - execute database queries
Node metadata is loaded from: ../../templates/metadata/nodes/database_query.yaml
"""
from __future__ import annotations
from typing import Any
from langbot_plugin.api.entities.builtin.workflow.entities import ExecutionContext
from ..node import WorkflowNode, workflow_node
@workflow_node('database_query')
class DatabaseQueryNode(WorkflowNode):
"""Database query node - execute database queries"""
category = 'integration'
async def execute(self, inputs: dict[str, Any], context: ExecutionContext) -> dict[str, Any]:
connection_type = self.get_config('connection_type', 'postgresql')
query = self.get_config('query', '')
query_type = self.get_config('query_type', 'select')
timeout = self.get_config('timeout', 30)
parameters = inputs.get('parameters', {})
return {
'results': [],
'row_count': 0,
'success': False,
'_debug': {
'connection_type': connection_type,
'query': query,
'query_type': query_type,
'timeout': timeout,
'parameters': parameters,
},
}

View File

@@ -1,37 +0,0 @@
"""Dify Knowledge Query Node - query Dify knowledge base
Node metadata is loaded from: ../../templates/metadata/nodes/dify_knowledge_query.yaml
"""
from __future__ import annotations
from typing import Any
from langbot_plugin.api.entities.builtin.workflow.entities import ExecutionContext
from ..node import WorkflowNode, workflow_node
@workflow_node('dify_knowledge_query')
class DifyKnowledgeQueryNode(WorkflowNode):
"""Dify knowledge base query node - query Dify knowledge base"""
category = 'integration'
async def execute(self, inputs: dict[str, Any], context: ExecutionContext) -> dict[str, Any]:
base_url = self.get_config('base_url', 'https://api.dify.ai/v1')
api_key = self.get_config('api_key', '')
dataset_id = self.get_config('dataset_id', '')
query = inputs.get('query', '')
# Safe API key truncation
masked_key = f'{api_key[:4]}...{api_key[-4:]}' if len(api_key) > 8 else '***' if api_key else ''
return {
'results': [],
'success': False,
'_debug': {
'base_url': base_url,
'api_key': masked_key,
'dataset_id': dataset_id,
'query': query,
},
}

View File

@@ -1,39 +0,0 @@
"""Dify Workflow Node - call Dify service API
Node metadata is loaded from: ../../templates/metadata/nodes/dify_workflow.yaml
"""
from __future__ import annotations
from typing import Any
from langbot_plugin.api.entities.builtin.workflow.entities import ExecutionContext
from ..node import WorkflowNode, workflow_node
@workflow_node('dify_workflow')
class DifyWorkflowNode(WorkflowNode):
"""Dify workflow node - call Dify service API"""
category = 'integration'
async def execute(self, inputs: dict[str, Any], context: ExecutionContext) -> dict[str, Any]:
base_url = self.get_config('base_url', 'https://api.dify.ai/v1')
api_key = self.get_config('api_key', '')
app_type = self.get_config('app_type', 'chat')
query = inputs.get('query', '')
conversation_id = inputs.get('conversation_id')
# Safe API key truncation
masked_key = f'{api_key[:4]}...{api_key[-4:]}' if len(api_key) > 8 else '***' if api_key else ''
return {
'answer': '',
'conversation_id': conversation_id,
'success': False,
'_debug': {
'base_url': base_url,
'api_key': masked_key,
'app_type': app_type,
'query': query,
},
}

View File

@@ -1,33 +0,0 @@
"""End Node - marks the end of workflow execution
Node metadata is loaded from: ../../templates/metadata/nodes/end.yaml
"""
from __future__ import annotations
from typing import Any
from langbot_plugin.api.entities.builtin.workflow.entities import ExecutionContext
from ..node import WorkflowNode, workflow_node
@workflow_node('end')
class EndNode(WorkflowNode):
"""End node - marks the end of workflow execution"""
category = 'control'
async def execute(self, inputs: dict[str, Any], context: ExecutionContext) -> dict[str, Any]:
result = inputs.get('result')
output_format = self.get_config('output_format', 'passthrough')
if output_format == 'text':
return {'output': str(result)}
elif output_format == 'json':
import json
try:
return {'output': json.dumps(result, ensure_ascii=False)}
except Exception:
return {'output': str(result)}
else:
return {'output': result}

View File

@@ -1,28 +0,0 @@
"""Event Trigger Node - triggers workflow on system events
Node metadata is loaded from: ../../templates/metadata/nodes/event_trigger.yaml
"""
from __future__ import annotations
from datetime import datetime
from typing import Any
from langbot_plugin.api.entities.builtin.workflow.entities import ExecutionContext
from ..node import WorkflowNode, workflow_node
@workflow_node('event_trigger')
class EventTriggerNode(WorkflowNode):
"""Event trigger node - triggers workflow on system events"""
category = 'trigger'
async def execute(self, inputs: dict[str, Any], context: ExecutionContext) -> dict[str, Any]:
# Safe access to trigger_data which may be None
trigger_data = context.trigger_data or {}
return {
'event_type': trigger_data.get('event_type', ''),
'event_data': trigger_data.get('event_data', {}),
'timestamp': trigger_data.get('timestamp', datetime.now().isoformat()),
}

View File

@@ -1,152 +0,0 @@
"""HTTP Request Node - make HTTP API calls
Node metadata is loaded from: ../../templates/metadata/nodes/http_request.yaml
"""
from __future__ import annotations
import ipaddress
import logging
from typing import Any
from urllib.parse import urlparse
from langbot_plugin.api.entities.builtin.workflow.entities import ExecutionContext
from ..node import WorkflowNode, workflow_node
logger = logging.getLogger(__name__)
# 内网地址黑名单
_PRIVATE_NETWORKS = [
ipaddress.ip_network('10.0.0.0/8'),
ipaddress.ip_network('172.16.0.0/12'),
ipaddress.ip_network('192.168.0.0/16'),
ipaddress.ip_network('127.0.0.0/8'),
ipaddress.ip_network('169.254.0.0/16'),
ipaddress.ip_network('0.0.0.0/8'),
ipaddress.ip_network('::1/128'),
ipaddress.ip_network('fc00::/7'),
ipaddress.ip_network('fe80::/10'),
]
# 危险协议
_DANGEROUS_SCHEMES = {'file', 'gopher', 'dict', 'ftp', 'telnet'}
def _is_safe_url(url: str) -> tuple[bool, str]:
"""检查 URL 是否安全(非内网地址)"""
try:
parsed = urlparse(url)
except Exception as e:
return False, f'Invalid URL: {e}'
# 检查协议
scheme = parsed.scheme.lower()
if scheme in _DANGEROUS_SCHEMES:
return False, f'Dangerous scheme: {scheme}'
if scheme not in ('http', 'https'):
return False, f'Unsupported scheme: {scheme}'
# 检查主机名
hostname = parsed.hostname
if not hostname:
return False, 'Missing hostname'
# 检查是否是危险主机名
dangerous_hosts = {'localhost', '0.0.0.0', '127.0.0.1', '::1'}
if hostname.lower() in dangerous_hosts:
return False, f'Dangerous hostname: {hostname}'
# 解析 IP 地址并检查是否在私有网络
try:
ip = ipaddress.ip_address(hostname)
for network in _PRIVATE_NETWORKS:
if ip in network:
return False, f'Private network address: {ip}'
except ValueError:
# 不是 IP 地址,尝试 DNS 解析检查
# 这里可以添加 DNS 解析检查,但为了避免复杂性,暂时跳过
pass
return True, ''
@workflow_node('http_request')
class HTTPRequestNode(WorkflowNode):
"""HTTP request node - make HTTP API calls"""
category = 'action'
async def execute(self, inputs: dict[str, Any], context: ExecutionContext) -> dict[str, Any]:
import aiohttp
url = self.get_config('url', '')
method = self.get_config('method', 'GET').upper()
timeout = self.get_config('timeout', 30)
content_type = self.get_config('content_type', 'application/json')
allow_redirects = self.get_config('allow_redirects', False) # 默认禁用重定向
# 限制超时时间
timeout = min(max(timeout, 1), 120)
if not url:
return {'response': None, 'status_code': 0, 'headers': {}, 'error': 'No URL provided'}
# 安全检查 URL
is_safe, error_msg = _is_safe_url(url)
if not is_safe:
logger.warning('Unsafe URL blocked: %s - %s', url, error_msg)
return {'response': None, 'status_code': 0, 'headers': {}, 'error': f'Unsafe URL: {error_msg}'}
# 验证 HTTP 方法
allowed_methods = {'GET', 'POST', 'PUT', 'DELETE', 'PATCH', 'HEAD', 'OPTIONS'}
if method not in allowed_methods:
return {'response': None, 'status_code': 0, 'headers': {}, 'error': f'Invalid method: {method}'}
# 创建 headers 副本,避免修改输入
headers = dict(inputs.get('headers', {}))
headers['Content-Type'] = content_type
auth_type = self.get_config('auth_type', 'none')
auth_config = self.get_config('auth_config', {})
if auth_type == 'bearer':
headers['Authorization'] = f'Bearer {auth_config.get("token", "")}'
elif auth_type == 'api_key':
header_name = auth_config.get('header', 'X-API-Key')
headers[header_name] = auth_config.get('key', '')
body = inputs.get('body')
logger.info('HTTP %s %s (timeout=%s)', method, url, timeout)
try:
async with aiohttp.ClientSession() as session:
async with session.request(
method=method,
url=url,
json=body if content_type == 'application/json' else None,
data=body if content_type != 'application/json' else None,
headers=headers,
timeout=aiohttp.ClientTimeout(total=timeout),
allow_redirects=allow_redirects,
) as response:
try:
response_data = await response.json()
except Exception:
response_data = await response.text()
logger.info('HTTP %s %s -> %d', method, url, response.status)
return {
'response': response_data,
'status_code': response.status,
'headers': dict(response.headers),
'error': None,
}
except aiohttp.ClientError as e:
logger.error('HTTP request failed: %s', e)
return {'response': None, 'status_code': 0, 'headers': {}, 'error': f'HTTP error: {e}'}
except Exception as e:
logger.error('HTTP request unexpected error: %s', e)
return {'response': None, 'status_code': 0, 'headers': {}, 'error': f'Unexpected error: {e}'}

View File

@@ -1,32 +0,0 @@
"""Iterator Node - Dify-style iterator for processing array items"""
from __future__ import annotations
from typing import Any
from langbot_plugin.api.entities.builtin.workflow.entities import ExecutionContext
from ..node import WorkflowNode, workflow_node
@workflow_node('iterator')
class IteratorNode(WorkflowNode):
"""Iterator node - iterate over array items one by one"""
category = 'control'
async def execute(self, inputs: dict[str, Any], context: ExecutionContext) -> dict[str, Any]:
items = inputs.get('items', [])
if not isinstance(items, list):
items = [items] if items else []
max_iterations = self.get_config('max_iterations', 1000)
items = items[:max_iterations]
return {
'item': items[0] if items else None,
'index': 0,
'is_first': True,
'is_last': len(items) <= 1,
'results': [],
'completed': len(items) == 0,
'_items': items,
}

View File

@@ -1,21 +0,0 @@
"""Knowledge Retrieval Node - search in knowledge base
Node metadata is loaded from: ../../templates/metadata/nodes/knowledge_retrieval.yaml
"""
from __future__ import annotations
from typing import Any
from langbot_plugin.api.entities.builtin.workflow.entities import ExecutionContext
from ..node import WorkflowNode, workflow_node
@workflow_node('knowledge_retrieval')
class KnowledgeRetrievalNode(WorkflowNode):
"""Knowledge retrieval node - search in knowledge base"""
category = 'process'
async def execute(self, inputs: dict[str, Any], context: ExecutionContext) -> dict[str, Any]:
query = inputs.get('query', '')
return {'documents': [], 'citations': [], 'context': f'[Knowledge base search for: {query}]'}

View File

@@ -1,37 +0,0 @@
"""Langflow Flow Node - call Langflow API
Node metadata is loaded from: ../../templates/metadata/nodes/langflow_flow.yaml
"""
from __future__ import annotations
from typing import Any
from langbot_plugin.api.entities.builtin.workflow.entities import ExecutionContext
from ..node import WorkflowNode, workflow_node
@workflow_node('langflow_flow')
class LangflowFlowNode(WorkflowNode):
"""Langflow flow node - call Langflow API"""
category = 'integration'
async def execute(self, inputs: dict[str, Any], context: ExecutionContext) -> dict[str, Any]:
base_url = self.get_config('base_url', 'http://localhost:7860')
api_key = self.get_config('api_key', '')
flow_id = self.get_config('flow_id', '')
input_value = inputs.get('input_value', '')
# Safe API key truncation
masked_key = f'{api_key[:4]}...{api_key[-4:]}' if len(api_key) > 8 else '***' if api_key else ''
return {
'result': None,
'success': False,
'_debug': {
'base_url': base_url,
'api_key': masked_key,
'flow_id': flow_id,
'input_value': input_value,
},
}

View File

@@ -1,829 +0,0 @@
"""LLM Call Node - invoke large language model with Agent capabilities.
Supports:
- Primary model with fallback models
- Knowledge base retrieval with reranking
- Max round context control
- Streaming output
"""
from __future__ import annotations
import json
import logging
import re
import time
from typing import Any, AsyncGenerator
import langbot_plugin.api.entities.builtin.provider.message as provider_message
import langbot_plugin.api.entities.builtin.rag.context as rag_context
from langbot_plugin.api.entities.builtin.workflow.entities import ExecutionContext
from ..node import WorkflowNode, workflow_node
from .. import monitoring_helper
logger = logging.getLogger(__name__)
# Pre-compiled regex patterns for CoT content removal (performance optimization)
_THINK_PATTERNS = [
re.compile(r'<think>.*?</think>', re.DOTALL | re.IGNORECASE),
re.compile(r'<thought>.*?</thought>', re.DOTALL | re.IGNORECASE),
re.compile(r'<reasoning>.*?</reasoning>', re.DOTALL | re.IGNORECASE),
re.compile(r'<\u601d\u8003>.*?</\u601d\u8003>', re.DOTALL | re.IGNORECASE),
re.compile(r'<\u63a8\u7406>.*?</\u63a8\u7406>', re.DOTALL | re.IGNORECASE),
]
# Template variable regex
_TEMPLATE_VAR_RE = re.compile(r'\{\{([^}]+)\}\}')
@workflow_node('llm_call')
class LLMCallNode(WorkflowNode):
"""LLM call node - invoke large language model"""
category = 'process'
def _resolve_template(self, template: str, inputs: dict[str, Any], context: ExecutionContext) -> str:
"""Resolve {{variable}} placeholders in a template string."""
if not template:
return ''
unresolved_vars = []
def replacer(match: re.Match) -> str:
expr = match.group(1).strip()
# Try inputs first
if expr in inputs:
return str(inputs[expr])
# Try context variables
if expr.startswith('variables.'):
var_name = expr[len('variables.'):]
return str(context.variables.get(var_name, ''))
# Try message context
if expr.startswith('message.') and context.message_context:
attr = expr[len('message.'):]
return str(getattr(context.message_context, attr, ''))
unresolved_vars.append(expr)
return match.group(0) # leave unresolved
result = _TEMPLATE_VAR_RE.sub(replacer, template)
# Log warning for unresolved variables
if unresolved_vars:
logger.warning(
f'LLM call node {self.node_id}: unresolved template variables: {unresolved_vars}'
)
return result
def _remove_think_content(self, text: str) -> str:
"""Remove CoT (Chain of Thought) thinking content from response."""
if not text:
return text
result = text
for pattern in _THINK_PATTERNS:
result = pattern.sub('', result)
return result.strip()
def _apply_content_filter(self, text: str) -> tuple[str, bool, str]:
"""Apply content safety filter to text.
Returns:
(filtered_text, is_blocked, user_notice)
"""
if not text or not self.ap:
return text, False, ''
# Check if content filter is enabled
safety_config = getattr(self.ap, 'pipeline_cfg', None)
if not safety_config:
return text, False, ''
# Check sensitive words
sensitive_words = []
try:
if hasattr(self.ap, 'sensitive_meta') and hasattr(self.ap.sensitive_meta, 'data'):
sensitive_words = self.ap.sensitive_meta.data.get('words', [])
except Exception as e:
logger.warning("Failed to load sensitive words from sensitive_meta: %s", e)
sensitive_words = []
if not sensitive_words:
return text, False, ''
found = False
filtered_text = text
for word in sensitive_words:
try:
matches = re.findall(word, filtered_text, re.IGNORECASE)
if matches:
found = True
mask_word = ''
mask = '*'
try:
if hasattr(self.ap, 'sensitive_meta') and hasattr(self.ap.sensitive_meta, 'data'):
mask_word = self.ap.sensitive_meta.data.get('mask_word', '')
mask = self.ap.sensitive_meta.data.get('mask', '*')
except Exception as e:
# Keep default mask settings when sensitive metadata is unavailable or malformed.
logger.debug(
f'LLM call node {self.node_id}: failed to read sensitive mask config, using defaults: {e}'
)
for m in matches:
if mask_word:
filtered_text = filtered_text.replace(m, mask_word)
else:
filtered_text = filtered_text.replace(m, mask * len(m))
except re.error:
# Invalid regex pattern, skip
continue
if found:
return filtered_text, False, '消息中存在不合适的内容, 请修改'
return text, False, ''
# RAG combined prompt template (same as localagent.py)
RAG_COMBINED_PROMPT_TEMPLATE = """
The following are relevant context entries retrieved from the knowledge base.
Please use them to answer the user's message.
Respond in the same language as the user's input.
<context>
{rag_context}
</context>
<user_message>
{user_message}
</user_message>
"""
def _build_system_prompt_with_format(self, base_prompt: str, output_format: str, json_schema: str) -> str:
"""Build system prompt with output format instructions."""
prompt = base_prompt
if output_format == 'json':
prompt += '\n\nPlease respond in valid JSON format.'
if json_schema:
prompt += f'\nFollow this JSON schema:\n{json_schema}'
elif output_format == 'markdown':
prompt += '\n\nPlease respond in Markdown format.'
return prompt
def _build_messages_from_prompt_array(
self,
prompt_array: list[dict],
inputs: dict[str, Any],
context: ExecutionContext,
output_format: str,
json_schema: str,
) -> list[provider_message.Message]:
"""Build messages list from prompt array (same format as pipeline).
Each item in prompt_array is {role: str, content: str}.
Resolves template variables in content.
"""
messages: list[provider_message.Message] = []
for item in prompt_array:
role = item.get('role', 'user')
content = item.get('content', '')
# Resolve template variables in content
resolved_content = self._resolve_template(content, inputs, context)
# Apply format instructions to system prompt
if role == 'system':
resolved_content = self._build_system_prompt_with_format(
resolved_content, output_format, json_schema
)
messages.append(provider_message.Message(role=role, content=resolved_content))
return messages
async def _get_model_candidates(self, model_uuid: str, fallback_models: list) -> list:
"""Build ordered list of models to try: primary model + fallback models."""
candidates = []
# Primary model
if model_uuid:
try:
primary = await self.ap.model_mgr.get_model_by_uuid(model_uuid)
candidates.append(primary)
except ValueError:
logger.warning(f'[LLM:{self.node_id}] Primary model {model_uuid} not found')
# Fallback models
for fb_uuid in fallback_models:
try:
fb_model = await self.ap.model_mgr.get_model_by_uuid(fb_uuid)
candidates.append(fb_model)
except ValueError:
logger.warning(f'[LLM:{self.node_id}] Fallback model {fb_uuid} not found, skipping')
return candidates
async def _invoke_with_fallback(
self,
candidates: list,
messages: list,
funcs: list | None,
extra_args: dict,
) -> tuple[Any, Any]:
"""Try non-streaming invocation with sequential fallback. Returns (message, model_used)."""
last_error = None
for model in candidates:
try:
msg = await model.provider.invoke_llm(
query=None,
model=model,
messages=messages,
funcs=funcs if model.model_entity.abilities.__contains__('func_call') else [],
extra_args=extra_args,
)
return msg, model
except Exception as e:
last_error = e
logger.warning(f'[LLM:{self.node_id}] Model {model.model_entity.name} failed: {e}, trying next...')
raise last_error or RuntimeError('No model candidates available')
async def _retrieve_knowledge(
self,
user_message_text: str,
knowledge_bases: list[str],
rerank_model_uuid: str,
rerank_top_k: int,
) -> str:
"""Retrieve from knowledge bases and optionally rerank results.
Returns the enhanced user message text with RAG context, or original text if no results.
"""
if not knowledge_bases or not user_message_text:
return user_message_text
all_results: list[rag_context.RetrievalResultEntry] = []
# Retrieve from each knowledge base
for kb_uuid in knowledge_bases:
try:
kb = await self.ap.rag_mgr.get_knowledge_base_by_uuid(kb_uuid)
if not kb:
logger.warning(f'[LLM:{self.node_id}] Knowledge base {kb_uuid} not found, skipping')
continue
result = await kb.retrieve(user_message_text, settings={})
if result:
all_results.extend(result)
except Exception as e:
logger.warning(f'[LLM:{self.node_id}] Failed to retrieve from KB {kb_uuid}: {e}')
# Rerank step: re-score results using a rerank model if configured
if all_results and rerank_model_uuid:
try:
rerank_model = await self.ap.model_mgr.get_rerank_model_by_uuid(rerank_model_uuid)
doc_texts = []
for entry in all_results:
text = ' '.join(c.text for c in entry.content if c.type == 'text' and c.text)
doc_texts.append(text)
doc_texts_capped = doc_texts[:64] # Cap for reranker input
scores = await rerank_model.provider.invoke_rerank(
model=rerank_model,
query=user_message_text,
documents=doc_texts_capped,
)
scored = sorted(scores, key=lambda x: x.get('relevance_score', 0), reverse=True)
top_indices = [s['index'] for s in scored[:rerank_top_k] if s['index'] < len(all_results)]
all_results = [all_results[i] for i in top_indices]
logger.info(
f'[LLM:{self.node_id}] Rerank complete: {len(doc_texts)} docs -> top {len(all_results)} kept (top_k={rerank_top_k})'
)
except ValueError:
logger.warning(f'[LLM:{self.node_id}] Rerank model {rerank_model_uuid} not found, skipping rerank')
except Exception as e:
logger.warning(f'[LLM:{self.node_id}] Rerank failed, using original order: {e}')
# Build RAG context text
if all_results:
texts = []
idx = 1
for entry in all_results:
for content in entry.content:
if content.type == 'text' and content.text is not None:
texts.append(f'[{idx}] {content.text}')
idx += 1
rag_context_text = '\n\n'.join(texts)
return self.RAG_COMBINED_PROMPT_TEMPLATE.format(
rag_context=rag_context_text,
user_message=user_message_text,
)
return user_message_text
def _build_messages_with_history(
self,
system_prompt: str,
user_message_text: str,
context: ExecutionContext,
max_round: int,
) -> list[provider_message.Message]:
"""Build messages list with conversation history up to max_round."""
messages: list[provider_message.Message] = []
# Add system prompt
if system_prompt:
messages.append(provider_message.Message(role='system', content=system_prompt))
# Get conversation history from context
conversation_history = context.variables.get('_conversation_history', [])
# Apply max_round limit (each round = 1 user + 1 assistant message)
if max_round > 0 and conversation_history:
# Keep only the last max_round * 2 messages (user + assistant pairs)
max_messages = max_round * 2
if len(conversation_history) > max_messages:
conversation_history = conversation_history[-max_messages:]
# Add conversation history
for msg in conversation_history:
if isinstance(msg, dict):
role = msg.get('role', 'user')
content = msg.get('content', '')
messages.append(provider_message.Message(role=role, content=content))
elif hasattr(msg, 'role') and hasattr(msg, 'content'):
messages.append(provider_message.Message(role=msg.role, content=msg.content))
# Add current user message
messages.append(provider_message.Message(role='user', content=user_message_text))
return messages
def _save_to_conversation_history(
self,
context: ExecutionContext,
user_message_text: str,
response_text: str,
max_round: int,
) -> None:
"""Save the exchange to conversation history."""
if max_round <= 0:
return
history = context.variables.get('_conversation_history', [])
history.append({'role': 'user', 'content': user_message_text})
history.append({'role': 'assistant', 'content': response_text})
# Enforce max_round limit
max_messages = max_round * 2
if len(history) > max_messages:
history = history[-max_messages:]
context.variables['_conversation_history'] = history
async def execute(self, inputs: dict[str, Any], context: ExecutionContext) -> dict[str, Any]:
# Support both new model_config format and legacy model + fallback_models format
model_config = self.get_config('model_config', None)
if model_config and isinstance(model_config, dict):
# New format: {primary: uuid, fallbacks: [uuid1, uuid2, ...]}
model_uuid = model_config.get('primary', '')
fallback_models = model_config.get('fallbacks', [])
else:
# Legacy format: separate model and fallback_models
model_uuid = self.get_config('model', '')
fallback_models = self.get_config('fallback_models', [])
if not model_uuid:
raise ValueError('No model configured for LLM call node')
if not self.ap:
raise RuntimeError('Application instance not available - cannot call LLM')
# Get error handling config
exception_handling = self.get_config('exception_handling', 'show-error')
failure_hint = self.get_config('failure_hint', 'Request failed.')
track_function_calls = self.get_config('track_function_calls', False)
# Get output format and json_schema config
output_format = self.get_config('output_format', 'text')
json_schema = self.get_config('json_schema', '')
# Agent config: knowledge bases, rerank, max_round
# (fallback_models already resolved above from model_config or fallback_models)
knowledge_bases = self.get_config('knowledge_bases', [])
rerank_model = self.get_config('rerank_model', '')
rerank_top_k = self.get_config('rerank_top_k', 5)
max_round = self.get_config('max_round', 10)
# Resolve prompts - support both new prompt array format and legacy format
prompt_array = self.get_config('prompt')
user_prompt = '' # Initialize for later use in _save_to_conversation_history
if prompt_array and isinstance(prompt_array, list):
# New format: prompt array like pipeline
messages = self._build_messages_from_prompt_array(
prompt_array, inputs, context, output_format, json_schema
)
# Get user input text for knowledge retrieval
user_input = inputs.get('input', '')
# Knowledge retrieval: enhance user input with RAG context
user_input = await self._retrieve_knowledge(
user_message_text=user_input,
knowledge_bases=knowledge_bases,
rerank_model_uuid=rerank_model,
rerank_top_k=rerank_top_k,
)
# Track user_prompt for conversation history
user_prompt = user_input
# Add user input as last message
if user_input:
messages.append(provider_message.Message(role='user', content=user_input))
# Apply max_round to conversation history
conversation_history = context.variables.get('_conversation_history', [])
if max_round > 0 and conversation_history:
max_messages = max_round * 2
if len(conversation_history) > max_messages:
conversation_history = conversation_history[-max_messages:]
# Insert conversation history before user input
history_messages = []
for msg in conversation_history:
if isinstance(msg, dict):
role = msg.get('role', 'user')
content = msg.get('content', '')
history_messages.append(provider_message.Message(role=role, content=content))
elif hasattr(msg, 'role') and hasattr(msg, 'content'):
history_messages.append(provider_message.Message(role=msg.role, content=msg.content))
# Insert history before user message
if history_messages and len(messages) > 0:
messages = messages[:-1] + history_messages + [messages[-1]]
else:
# Legacy format: separate system_prompt and user_prompt_template
system_prompt = self._resolve_template(self.get_config('system_prompt') or '', inputs, context)
user_prompt_template = self.get_config('user_prompt_template')
if user_prompt_template is None:
user_prompt_template = '{{input}}'
user_prompt = self._resolve_template(user_prompt_template, inputs, context)
# Build system prompt with format instructions
system_prompt = self._build_system_prompt_with_format(system_prompt, output_format, json_schema)
# Knowledge retrieval: enhance user prompt with RAG context
user_prompt = await self._retrieve_knowledge(
user_message_text=user_prompt,
knowledge_bases=knowledge_bases,
rerank_model_uuid=rerank_model,
rerank_top_k=rerank_top_k,
)
# Build messages with conversation history
messages = self._build_messages_with_history(
system_prompt=system_prompt,
user_message_text=user_prompt,
context=context,
max_round=max_round,
)
# Get model candidates (primary + fallbacks)
candidates = await self._get_model_candidates(model_uuid, fallback_models)
if not candidates:
raise ValueError('No valid model candidates available')
# Build extra args from config
extra_args: dict[str, Any] = {}
temperature = self.get_config('temperature')
if temperature is not None:
extra_args['temperature'] = float(temperature)
max_tokens = self.get_config('max_tokens', 0)
if max_tokens and int(max_tokens) > 0:
extra_args['max_tokens'] = int(max_tokens)
# Track start time for duration calculation
self._llm_start_time = time.time()
# Invoke LLM with fallback
try:
result_message, used_model = await self._invoke_with_fallback(
candidates=candidates,
messages=messages,
funcs=None,
extra_args=extra_args,
)
except Exception as e:
logger.warning(f'[LLM:{self.node_id}] LLM call failed: {e}')
# Handle based on exception handling strategy
if exception_handling == 'show-error':
raise
elif exception_handling == 'show-hint':
return {
'response': failure_hint,
'usage': {
'prompt_tokens': 0,
'completion_tokens': 0,
'total_tokens': 0,
},
'error': str(e),
'error_hint_shown': True,
}
else: # hide
return {
'response': '',
'usage': {
'prompt_tokens': 0,
'completion_tokens': 0,
'total_tokens': 0,
},
'error': str(e),
}
# Extract response text
response_text = ''
if isinstance(result_message.content, str):
response_text = result_message.content
elif isinstance(result_message.content, list):
for elem in result_message.content:
if hasattr(elem, 'text') and elem.text:
response_text += elem.text
elif isinstance(elem, str):
response_text += elem
# Remove CoT content (always remove to avoid leaking internal reasoning)
response_text = self._remove_think_content(response_text)
# Initialize usage default
usage = {
'prompt_tokens': 0,
'completion_tokens': 0,
'total_tokens': 0,
}
# Apply content safety filter
response_text, is_blocked, filter_notice = self._apply_content_filter(response_text)
if is_blocked:
logger.warning(f'[LLM:{self.node_id}] Response blocked by content filter: {filter_notice}')
return {
'response': filter_notice,
'usage': usage,
'blocked_by_filter': True,
}
# Extract usage info
if hasattr(result_message, 'usage') and result_message.usage:
u = result_message.usage
# Handle both object and dict usage
if isinstance(u, dict):
usage = {
'prompt_tokens': u.get('prompt_tokens', 0) or 0,
'completion_tokens': u.get('completion_tokens', 0) or 0,
'total_tokens': u.get('total_tokens', 0) or 0,
}
else:
usage = {
'prompt_tokens': getattr(u, 'prompt_tokens', 0) or 0,
'completion_tokens': getattr(u, 'completion_tokens', 0) or 0,
'total_tokens': getattr(u, 'total_tokens', 0) or 0,
}
elif hasattr(result_message, 'token_usage') and result_message.token_usage:
u = result_message.token_usage
# Handle both object and dict token_usage
if isinstance(u, dict):
usage = {
'prompt_tokens': u.get('prompt_tokens', 0) or 0,
'completion_tokens': u.get('completion_tokens', 0) or 0,
'total_tokens': u.get('total_tokens', 0) or 0,
}
else:
usage = {
'prompt_tokens': getattr(u, 'prompt_tokens', 0) or 0,
'completion_tokens': getattr(u, 'completion_tokens', 0) or 0,
'total_tokens': getattr(u, 'total_tokens', 0) or 0,
}
# Log successful response (matching Pipeline's cut_str behavior)
def _cut_str(s: str) -> str:
s0 = s.split('\n')[0]
if len(s0) > 20 or '\n' in s:
s0 = s0[:20] + '...'
return s0
logger.info(f'[LLM:{self.node_id}] Response: {_cut_str(response_text)}')
# Record LLM call log only (response log is redundant)
try:
if self.ap and context.query:
workflow_id = context.workflow_id or ''
workflow_name = context.variables.get('_workflow_name', 'Workflow')
bot_name = context.variables.get('_bot_name', 'Workflow')
node_name = self.get_config('name', self.node_id)
model_name = used_model.model_entity.name if used_model else 'unknown'
# Calculate duration
duration_ms = 0
if hasattr(self, '_llm_start_time'):
duration_ms = int((time.time() - self._llm_start_time) * 1000)
# Get message_id for LLM call association
message_id = context.variables.get('_monitoring_message_id')
# Record LLM call log with message_id association
await monitoring_helper.WorkflowMonitoringHelper.record_llm_call_log(
ap=self.ap,
query=context.query,
workflow_id=workflow_id,
workflow_name=workflow_name,
node_name=node_name,
model_name=model_name,
input_tokens=usage.get('prompt_tokens', 0),
output_tokens=usage.get('completion_tokens', 0),
duration_ms=duration_ms,
status='success',
bot_name=bot_name,
context_vars=context.variables,
message_id=message_id,
)
except Exception as e:
logger.warning(f'[LLM:{self.node_id}] Failed to record LLM logs: {e}')
# Save to conversation history
self._save_to_conversation_history(
context=context,
user_message_text=user_prompt,
response_text=response_text,
max_round=max_round,
)
# Build result
result: dict[str, Any] = {
'response': response_text,
'usage': usage,
'model_used': used_model.model_entity.name if used_model else None,
'model_uuid': used_model.model_entity.uuid if used_model else None,
}
# Parse JSON output if format is json
if output_format == 'json' and response_text:
try:
result['parsed'] = json.loads(response_text)
except json.JSONDecodeError as e:
logger.warning(f'[LLM:{self.node_id}] Failed to parse JSON: {e}')
result['parsed'] = None
result['parse_error'] = str(e)
# Add function call tracking info if configured
if track_function_calls:
result['function_calls'] = []
return result
async def execute_stream(
self, inputs: dict[str, Any], context: ExecutionContext
) -> AsyncGenerator[str, None]:
"""Execute the LLM call with streaming output.
Yields chunks of response text as they arrive.
Falls back to non-streaming if streaming is not available.
"""
# Support both new model_config format and legacy model + fallback_models format
model_config = self.get_config('model_config', None)
if model_config and isinstance(model_config, dict):
model_uuid = model_config.get('primary', '')
else:
model_uuid = self.get_config('model', '')
if not model_uuid:
raise ValueError('No model configured for LLM call node')
if not self.ap:
raise RuntimeError('Application instance not available - cannot call LLM')
exception_handling = self.get_config('exception_handling', 'show-error')
failure_hint = self.get_config('failure_hint', 'Request failed.')
# Resolve prompts - support both new prompt array format and legacy format
prompt_array = self.get_config('prompt')
if prompt_array and isinstance(prompt_array, list):
# New format: prompt array like pipeline
messages = self._build_messages_from_prompt_array(
prompt_array, inputs, context, 'text', '' # No format instructions for streaming
)
# Add user input
user_input = inputs.get('input', '')
if user_input:
messages.append(provider_message.Message(role='user', content=user_input))
else:
# Legacy format
system_prompt = self._resolve_template(self.get_config('system_prompt') or '', inputs, context)
user_prompt_template = self.get_config('user_prompt_template')
if user_prompt_template is None:
user_prompt_template = '{{input}}'
user_prompt = self._resolve_template(user_prompt_template, inputs, context)
# Build messages
messages = []
if system_prompt:
messages.append(provider_message.Message(role='system', content=system_prompt))
messages.append(provider_message.Message(role='user', content=user_prompt))
# Get model
runtime_model = await self.ap.model_mgr.get_model_by_uuid(model_uuid)
# Build extra args
extra_args: dict[str, Any] = {}
temperature = self.get_config('temperature')
if temperature is not None:
extra_args['temperature'] = float(temperature)
max_tokens = self.get_config('max_tokens', 0)
if max_tokens and int(max_tokens) > 0:
extra_args['max_tokens'] = int(max_tokens)
logger.info(f'[LLM:{self.node_id}] Streaming model {model_uuid}')
try:
# Try streaming first
stream = runtime_model.provider.invoke_llm_stream(
query=None,
model=runtime_model,
messages=messages,
funcs=None,
extra_args=extra_args,
)
full_response = ''
in_think_block = False
async for chunk in stream:
chunk_text = ''
if hasattr(chunk, 'content'):
if isinstance(chunk.content, str):
chunk_text = chunk.content
elif isinstance(chunk.content, list):
for elem in chunk.content:
if hasattr(elem, 'text') and elem.text:
chunk_text += elem.text
elif isinstance(elem, str):
chunk_text += elem
if chunk_text:
# Filter <think> blocks in streaming mode
if '<think>' in chunk_text or '<thought>' in chunk_text:
in_think_block = True
if in_think_block:
if '</think>' in chunk_text or '</thought>' in chunk_text:
in_think_block = False
chunk_text = chunk_text.split('</think>')[-1].split('</thought>')[-1]
else:
chunk_text = ''
if chunk_text:
full_response += chunk_text
yield chunk_text
# Store in context for downstream nodes
context.variables['_last_llm_response'] = full_response
except Exception as e:
logger.warning(f'[LLM:{self.node_id}] Streaming failed, falling back - {e}')
# Fallback to non-streaming
try:
result_message = await runtime_model.provider.invoke_llm(
query=None,
model=runtime_model,
messages=messages,
funcs=None,
extra_args=extra_args,
)
response_text = self._extract_response_text(result_message)
# Always remove <think> content in fallback
response_text = self._remove_think_content(response_text)
yield response_text
context.variables['_last_llm_response'] = response_text
except Exception as e2:
logger.error(f'[LLM:{self.node_id}] Fallback also failed - {e2}')
if exception_handling == 'show-hint':
yield failure_hint
elif exception_handling != 'hide':
raise
def _extract_response_text(self, result_message: provider_message.Message) -> str:
"""Extract response text from LLM result message."""
response_text = ''
if isinstance(result_message.content, str):
response_text = result_message.content
elif isinstance(result_message.content, list):
for elem in result_message.content:
if hasattr(elem, 'text') and elem.text:
response_text += elem.text
elif isinstance(elem, str):
response_text += elem
return response_text

View File

@@ -1,30 +0,0 @@
"""Loop Node - iterate over items"""
from __future__ import annotations
from typing import Any
from langbot_plugin.api.entities.builtin.workflow.entities import ExecutionContext
from ..node import WorkflowNode, workflow_node
@workflow_node('loop')
class LoopNode(WorkflowNode):
"""Loop node - iterate over items"""
category = 'control'
async def execute(self, inputs: dict[str, Any], context: ExecutionContext) -> dict[str, Any]:
items = inputs.get('items', [])
if not isinstance(items, list):
items = [items] if items else []
max_iterations = self.get_config('max_iterations', 100)
items = items[:max_iterations]
return {
'item': items[0] if items else None,
'index': 0,
'results': [],
'completed': len(items) == 0,
'_items': items,
}

View File

@@ -1,58 +0,0 @@
"""MCP Tool Node - Invoke MCP (Model Context Protocol) tools
This module contains the implementation for the MCP Tool workflow node.
Node metadata (label, description, inputs, outputs, config) is loaded from:
../../templates/metadata/nodes/mcp_tool.yaml
The i18n for label and description is handled on the frontend side.
"""
from __future__ import annotations
from typing import Any
from langbot_plugin.api.entities.builtin.workflow.entities import ExecutionContext
from ..node import WorkflowNode, workflow_node
@workflow_node('mcp_tool')
class MCPToolNode(WorkflowNode):
"""MCP tool node - invoke MCP (Model Context Protocol) tools"""
# Node type for registration
# Category and icon - these are not i18n
category = 'integration'
# Name and description - i18n handled on frontend side
# Frontend will use node type key to look up translation
# Inputs/outputs/config - loaded from YAML at runtime
async def execute(self, inputs: dict[str, Any], context: ExecutionContext) -> dict[str, Any]:
"""Execute the MCP tool node
Args:
inputs: Input data from connected nodes
context: Execution context with workflow state
Returns:
Dictionary of output values
"""
server_name = self.get_config('server_name', '')
tool_name = self.get_config('tool_name', '')
arguments_template = self.get_config('arguments_template', '')
timeout = self.get_config('timeout', 30)
arguments = inputs.get('arguments', arguments_template)
return {
'result': None,
'success': False,
'error': f"MCP tool '{server_name}/{tool_name}' not implemented yet",
'_debug': {
'server_name': server_name,
'tool_name': tool_name,
'arguments': arguments,
'timeout': timeout,
},
}

Some files were not shown because too many files have changed in this diff Show More